August 2013 Critical Care Journal Club: Less is More

Our August journal club reviewed failed efforts to impact the mortality of critical illness over the past 25 years.  We looked at six landmark randomized controlled trials with certain things in common. 

They each addressed treatment of a major aspect of critical illness.  Each was well-supported by previous literature, and biologically plausible.  Each resulted in a statistically-significant mortality benefit, and was published in a well-respected journal.  And each had an immediate, and in many cases, lasting effect on the bedside practice of critical care.

Yet the positive result of each of these six studies was subsequently convincingly refuted.

It is important to note, that these studies make up a good part of what we’ve learned in critical care over the past 25 years.  There have been some influential positive studies as well, but a great deal of effort has been spent implementing evidence-based practice, based on studies that were later shown to be ineffective.

Still, some good can come from reconsideration of these studies.  The purpose of this review is to encourage healthy skepticism based on a historical perspective of our past failures.  The main question we tried to answer for each study is: Why did the authors arrive at the wrong conclusions?

Suresh Uppalapu, Nick Sparacino, Elijah Poulos, Sandra Till, Heemesh Seth, and Josh Jewell reviewed the articles respectively in the order in which they appear in table 1, which summarizes some characteristics of each study.  The six studies represent efforts to develop protocols for goal-directed resuscitation using hemodynamic parameters (1); favorably alter the immunological (2,6) or microcirculatory (4) pathophysiology of sepsis; support the metabolic (glycemic) response to critical illness (5); and alter the immunological pathophysiology of ARDS (3).

Our analysis did not reveal any common methodological flaw to explain why these studies yielded misleading conclusions (summarized in Table 1).

Table 1. Summary of the 6 studies discussed in this journal club  

Items in red font possibly contributed to misleading study conclusions.  * post op mortality; ** mortality in a subgroup with gram negative rod bacteremia; *** ICU mortality. 

Two had small sample sizes and analyzed very small numbers of clinical events (1,3) - thus the p-value might dramatically increase if even a single event had turned out differently.  The same studies only reported short-term mortality (1,3) - it is clear now that ICU mortality should be followed out at least 28 days to avoid the mistake of interpreting a delayed death as a beneficial event.  Two of the studies were supported by pharmaceutical companies with significant potential financial interest in a positive result (2,4).  Two were stopped early at interim analysis (4,5).  Several had problems with external generalizability – having studied surgical patients at a single site (1,5).  But only one (2) was obviously flawed in terms of internal validity (although the flaw was not obvious to the editors of NEJM at the time of publication).  In general, these studies were widely accepted as valid when they were published. 

It is not an exaggeration to suggest that the practice of “evidence-based medicine” based on these studies, likely resulted in significant mortality and morbidity, and much wasted effort over the past 20 years.  Shoemaker’s study – the one our fellows are least likely to remember – did much to promote the general use of Swan Ganz catheters.  The concept of goal-directed therapy using supraphysiological Swan parameters was not convincingly refuted until nearly 20 years later.  Approximately a billion dollars was spent on Drotrecogin alpha over a decade in which it was recommended.  It is unclear how many patients may have suffered hemorrhagic complications of the drug.  The NICE-SUGAR trial provided a point estimate that suggests that one of every 38 patients treated with an intensive insulin protocol died likely related to severe hypoglycemia.  This is probably an overestimation, but extrapolation suggests the potential that great harm may have been done.

This brief history provides a background to support caution in our optimism for future research findings, and humility in the formulation of consensus recommendations and guidelines. In most all cases, our experience should have taught us that less is more.

Robert A. Raschke, MD

Associate Editor

References

1. Shoemaker WC, Appel PL, Kram HB, Waxman K, Lee TS. Prospective trial of supranromal values of survivors as therapeutic goals in high risk surgical patients. Chest. 1988;94:1176-86. [CrossRef] [PubMed] 
2. Ziegler EJ, Fisher CJ Jr, Sprung CL, Straube RC, Sadoff JC, Foulke GE, Wortel CH, Fink MP, Dellinger RP, Teng NN, et al. Treatment of gram-negative bacteremia and septic shock with HA-1A human monoclonal antibody against endotoxin. A randomized, double-blind, placebo-controlled trial. The HA-1A Sepsis Study Group. N Engl J Med. 1991;324(7):429-36. [CrossRef] [PubMed] 
3. Meduri GU, Headley AS, Golden E, Carson SJ, Umberger RA, Kelso T, Tolley EA. Effect of prolonged methylprednisolone therapy in unresolving acute respiratory distress syndrome: a randomized controlled trial. JAMA. 1998;280(2):159-65. [CrossRef] [PubMed] 
4. Bernard GR, Vincent JL, Laterre PF, LaRosa SP, Dhainaut JF, Lopez-Rodriguez A, Steingrub JS, Garber GE, Helterbrand JD, Ely EW, Fisher CJ Jr; Recombinant human protein C Worldwide Evaluation in Severe Sepsis (PROWESS) study group. Efficacy and safety of recombinant human activated protein C for severe sepsis. N Engl J Med. 2001;344(10):699-709. [CrossRef] [PubMed] 
5. van den Berghe G, Wouters P, Weekers F, Verwaest C, Bruyninckx F, Schetz M, Vlasselaers D, Ferdinande P, Lauwers P, Bouillon R. Intensive insulin therapy in critically ill patients. N Engl J Med. 2001;345(19):1359-67. [CrossRef] [PubMed] 
6. Annane D, Sébille V, Charpentier C, Bollaert PE, François B, Korach JM, Capellier G, Cohen Y, Azoulay E, Troché G, Chaumet-Riffaud P, Bellissant E. Effect of treatment with low doses of hydrocortisone and fludrocortisone on mortality in patients with septic shock. JAMA. 2002;288(7):862-71. [CrossRef] [PubMed] 

Reference as: Raschke RA. August 2013 critical care journal club: less is more. Southwest J Pulm Crit Care. 2013;7(3):162-4. doi: http://dx.doi.org/10.13175/swjpcc121-13 PDF