November 2012 Critical Care Journal Club

Mehta S, Burry L, Cook D, Fergusson D, et al. Daily sedation interruption in mechanically ventilated critically ill patients cared for with a sedation protocol.  JAMA 2012;308:1985-92. Abstract

This study was a multi-center, randomized controlled trial that compared protocolized sedation with protocolized sedation plus daily sedation interruption.  The protocol used to titrate benzodiazepine and opioid infusions incorporated a validated scale (Sedation-agitation Scale (SAS) or Richmond Agitation Sedation Scale (RASS) in order to maintain a comfortable but arousable state.  Four hundred and thirty mechanically ventilated, critically ill patients were recruited from medical and surgical ICUs in 16 institutions in North America.  The study showed no benefit in the group that underwent daily sedation interruption - length of intubation was 7 days, length of ICU stay was 10 days and length of hospital stay was 20 days in both groups.  There was no significant difference in the incidence of delirium (53 vs. 54%) or in unintentional extubation (5 vs. 6%).  However, nurses felt sedation interruption significantly increased their workload (visual analogue scale 4.22 vs. 3.80 p=0.001). Sedation interruption was also associated with higher mean daily doses of sedation drugs, and more daily boluses of benzodiazepines and opioids – for instance, the cumulative dose of midazolam in the interruption group was increased by 20 mg per day.

This study has strong internal validity.  It was a large multicenter RCT.  The outcomes were clinically important.  The main outcome comparisons between the two groups were so strikingly negative that they scarcely needed calculation of p-values.  There was no way to practically blind daily sedation interruptions, and the nursing estimate of workload could clearly have been biased by this, but the findings in this regard are consistent with common sense.  I think our nurses would agree that sedation interruption in critically-ill patients can be time-consuming and often require more medication boluses to re-establish sedation than they would have required had the infusion simply been continued.   Our delirium quality improvement working group is currently discussing this paper in our efforts to improve our ICU standard for sedation management.    

Yunos NM, Bellomo R, Hegarty C, Story D, Ho L, Bailey M.  Association between a chloride-liberal vs chloride-restrictive intravenous fluid administration strategy and kidney injury in critically ill adults.  JAMA  2012;308;1566-72. PDF

 This study was a historically-controlled cohort study, based on the hypothesis that administration of supraphysiological quantities of chloride in IV solutions such as normal saline may be detrimental to kidney function.  The authors compared a period of time in which chloride-liberal IV fluids were readily available in their ICU vs. an intervention period, a year later, in which a bundle of chloride-restricted intravenous fluids (lactated ringer’s solution (LR), Plasmalyte, and salt-poor 20% albumin) were available.  A total of 1533 critically-ill patients were included.  The authors found that chloride administration decreased dramatically in the intervention period and that patients treated with the chloride-restricted bundle had a smaller increase in creatinine (22.6 umol/L vs. 14.8 umol/L; p=0.03); a decrease in the incidence of acute kidney injury/failure (14% vs. 8%; p<0.001); and a decrease in the use of renal replacement therapy (10% vs. 6.4%; p=0.005).

This study had an interesting hypothesis, but was very poorly designed.   It provides a good example of what Sir Austin Bradford Hill meant when he bemoaned our over-reliance on P-values in studies that have extremely weak experimental designs.  The methodological shortcomings of the study include: 1) This was an observational study – a cohort study with historical controls.  One of the main threats to validity in this type of study is bias – many things besides the IV fluid bundle are likely to have changed in this ICU, and the patients they admitted, over the course of a year.  The authors make no attempt to report on these changes.  2) The intervention was bundled.  All such studies (including Dr. River’s study on goal-directed therapy for sepsis) are fundamentally flawed because they cannot determine the specific intervention that is associated with the outcome.  The bundle might include harmful interventions that cannot be teased-out.  In this specific case, it would have been much better to simply concentrate on replacing saline with LR. 3) The study was unblinded.  Although the authors claim it would have been difficult to blind, this just isn’t true – bags of isotonic IV solution could have been provided without the clinicians knowing whether they contained saline or LR.  Lack of blinding could have biased the decision whether or not to start dialysis in some patients. 4) The main outcome measures were either: laboratory surrogates (increase in creatinine), based on laboratory surrogates (incidence of acute kidney injury), or prone to bias (decision to start dialysis).  The difference in the increase in creatinine is statistically significant, but clinically meaningless.  The protective effect of the chloride-restrictive strategy was to mitigate the increase in creatinine by only 8 umol/L or 0.1 mg/dL.  In some patients this small change would have caused them to be reclassified into a worse RIFLE category of renal function, but it certainly doesn’t seem likely to qualitatively impact their well-being.  5)  The use of time-to-event analysis, Kaplan Meier curves, and Cox proportional hazards modeling is inappropriate for the outcome of acute renal injury.  These statistical techniques are highly powerful for showing the statistical significance of whether an intervention can delay an outcome, but do not necessarily show whether it can prevent the outcome.   Our clinical goal should be to prevent, not just delay by a few days, the onset of acute kidney injury.  The inappropriate use of these statistical techniques provides false statistical power in this situation.  This study therefore provides no useful information for clinicians besides the interesting hypothesis the authors began with.

Serpa Neto A, Cardoso SO, Manetta JA, et al.  Association between use of lung-protective ventilation with lower tidal volumes and clinical outcomes among patients without acute respiratory distress syndrome.  JAMA 2012;308:1651-9. PDF

The authors performed a meta-analysis based on twenty articles indentified in their literature search, in which patients without ARDS were treated by two ventilator strategies – one with lower and one with higher tidal volumes.  Primary outcome analysis showed that lower tidal volumes were associated with a decrease in lung injury development [RR 0.33 95%CI 0.23-0.47] and lower mortality [RR 0.64 95%CI 0.46-0.89].

In my opinion, the study was critically flawed by the study selection.  Randomized and non-randomized observational studies were included.  Five observational studies accounted for approximately 85% of the patients in the primary analysis of lung injury.  Many of the randomized studies had significant methodological flaws, such as failure to observe the principle of intention-to-treat.  Fourteen of the studies were carried out in the operating room – these were combined with six studies from MICUs and SICUs.  The median duration of intervention mechanical ventilation in the included studies was less than seven hours.  Seven of the studies did not use 6mL/Kg ideal body weight as their “low-tidal volume” intervention.  Five of the included studies didn’t report the duration of patient outcome follow-up, and four reported follow-up of less than eight hours!   The validity of meta-analyses is largely based on the selection of comparable studies with high individual internal validity – this meta-analysis fails both criteria.  

The pathophysiological principle that provides the rationale for low-tidal ventilation is the observation that the number of functional alveoli is significantly reduced in ARDS.  Therefore, even relatively “typical” tidal volumes delivered by a mechanical ventilator will overinflate the healthiest parts of the ARDS-beset lung.  It is more difficult to understand the biological plausibility of how a few hours of typical tidal volumes delivered to predominantly normal lungs in the operating room could cause increased lung injury and mortality.   It is especially difficult to look at outcomes such as mortality among studies with as little as 3 hours of follow-up.  It’s my opinion that this study doesn’t add to our understanding of how best to mechanically ventilate patients without ARDS in the O.R. or in our ICUs.  However the question of how to calculate the optimal tidal volume for patients without ARDS that require mechanical ventilation is very important.  It will require prospective randomized controlled trials in the ICU and (separately) in the operating room to answer. 

Pereira TV, Horwitz RI, loannidis JP.  Empirical evaluation of very large treatment effects of medical interventions.  JAMA  2012;308:1676-84. PDF

The authors reviewed the Cochrane Database of Systematic Reviews, and identified 85,000 meta-analysis forest plots in which a medical intervention was evaluated in relation to an outcome – any outcome from mortality to surrogate laboratory outcomes.  [Recall that a forest plot is a graphical display that illustrates the relative effect size and estimate precision, of a particular outcome in a group of studies included in a meta-analysis].  The authors found 13,397 forest plots in which at least a single study demonstrated a “very large” treatment effect (an odds ratio greater than 5.0 or less than 0.2).  Sixty percent of the time, very large treatment effects were observed in the first study of a particular intervention.  Studies with very large treatment effects were generally small trials, -the results were based on a median of less than twenty patient events.  Studies with very large treatment effects were also much less likely to address mortality, and more likely to address surrogate laboratory outcomes than other trials.  Over ninety percent of the time, very large treatment effects became significantly more modest as subsequent studies were added into meta-analyses.  The authors were only able to find a single study, among 228,000 clinical trials, in which an intervention had a large beneficial effect on mortality, with strong methodological and statistical aspects [a study on ECMO for newborns with severe respiratory failure].

My thanks to the authors on their exhaustive and well-designed study.  Most medical research studies that demonstrate very large treatment effects are spurious – the vast majority represents substantial overestimations of potential patient benefit.  It’s not surprising that purported large treatment benefits are more likely to be related to surrogate laboratory outcomes.  Studies based on surrogate outcomes are generally not helpful in making treatment decisions to begin with.  I guess it comes down to something my Dad once taught me – “If it sounds too good to be true, it probably is”.

Robert A. Raschke, MD

Associate Editor, Critical Care Medicine

Reference as: Raschke RA. November 2012 critical care journal club. Southwest J Pulm Crit Care 2012;5:292-5. PDF