A Little Knowledge is a Dangerous Thing

An article entitled “A Comprehensive Care Management Program to Prevent Chronic Obstructive Pulmonary Disease Hospitalizations: A Randomized, Controlled Trial” from the VA cooperative studies program was recently published in the Annals of Internal Medicine (1).  This article describes the BREATH trial mentioned in a previous editorial (2). BREATH was a randomized, controlled, multi-center trial performed at 20 VA medical centers comparing an educational comprehensive care management program to guideline-based usual care for patients with chronic obstructive pulmonary disease (COPD). The intervention included COPD education during 4 individual and 1 group sessions, an action plan for identification and treatment of exacerbations, and scheduled proactive telephone calls for case management. After enrolling 426 (44%) of the planned total of 960 the trial was stopped because there were 28 deaths from all causes in the intervention group versus 10 in the usual care group (hazard ratio, 3.00; 95% CI, 1.46 to 6.17; p = 0.002). Deaths due to COPD accounted for the largest difference (10 deaths in the intervention group versus 3 in usual care; hazard ratio, 3.60; 95% CI, 0.99 to 13.08). This trial led us to perform a meta-analysis of educational interventions in COPD (3). In this meta-analysis of 2476 subjects we found no difference in mortality between intervention and usual care groups and that the recent Annals study was heterogenous compared to the other studies.

Should the recent VA study have been stopped early? Several reports demonstrate that studies stopped early usually overestimate treatment effects (4-7). Some have even suggested that stopping trials early is unethical (7). A number of articles suggest that trials should only be stopped if predetermined statistical parameters are exceeded, with the p value for stopping set at a very low level (4-7).  There was no planned interim analysis for any outcome in the recent VA trial. The rationale for stopping a study for an adverse effect when there is no a priori reasonable link between the intervention and the adverse effect is missing in this instance.  It seems unlikely that education would actually lead to increased deaths in COPD patients.  Any effect should logically have impacted the COPD related mortality, yet there was no significant increase for COPD related deaths in the intervention group. An accompanying editorial by Stuart Pocock makes most of these points and suggests that chance was the most likely cause of the excess deaths (8).

The VA Coop Trials coordinating center told the investigators that the reason for stopping the trial was that there were “significant adverse events” in the intervention group. Inquires regarding what adverse events went unanswered. This would seem to be a breakdown in VA research oversight. The information provided to both investigators and research subjects was incomplete and would seem to be a violation of the informed consent, which states the subject would be notified of any new information that significantly altered their risk.

Lastly, investigators were repeatedly warned by the VA coordinating center that “all communications with the media should occur through your facility Public Affairs office”. It seems very unlikely that personnel in any public affairs office have sufficient research training to answer any medical, statistical or ethical inquiries into the conduct of this study.

In our meta-analysis we have shown that self-management education is associated with a reduction in hospital admissions with no indication for detrimental effects in other outcome parameters. This would seem sufficient to justify a recommendation of self-management education in COPD. However, due to variability in interventions, study populations, follow-up time, and outcome measures, data are still insufficient to formulate clear recommendations regarding the form and content of self-management education programs in COPD.

Richard A. Robbins, M.D.*

Editor, Southwest Journal of Pulmonary

   and Critical Care

References

1. Fan VS, Gaziano JM, Lew R, et al. A comprehensive care management program to prevent chronic obstructive pulmonary disease hospitalizations: a randomized, controlled trial. Ann Intern Med 2012;156:673-683.
2. Robbins RA. COPD, COOP and BREATH at the VA. Southwest J Pulm Crit Care 2011;2:27-28.
3. Hurley J, Gerkin R, Fahy B, Robbins RA. Meta-analysis of self-management education for patients with chronic obstructive pulmonary disease. Southwest J Pulm Crit Care 2012;4:?-?.
4. Pocock SJ, Hughes MD. Practical problems in interim analyses, with particular regard to estimation.Control Clin Trials 1989;10:209S-221S.
5. Montori VM, Devereaux PJ, Adhikari NK, et al. Randomized trials stopped early for benefit: a systematic review. JAMA 2005;294:2203-9.
6. Bassler D, Briel M, Montori VM, et al. Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA 2010;303:1180-7.
7. Mueller PS, Montori VM, Bassler D, Koenig BA, Guyatt GH. Ethical issues in stopping randomized trials early because of apparent benefit. Ann Intern Med. 2007;146:878-81.
8. Pocock SJ. Ethical dilemmas and malfunctions in clinical trials research. Ann Intern Med 2012;156:746-747.

*Dr. Robbins was an investigator and one of the co-authors of the Annals of Internal Medicine manuscript (reference #1).

Reference as: Robbins RA. A little knowledge is a dangerous thing. Southwest J Pulm Crit Care 2012;4:203-4. (Click here for a PDF version of the editorial)