What's Wrong with Expert Opinion?

In this month's Pulmonary Journal Club Dr. Mathew reviews an article by Feuerstein et al. (1) from Beth Israel Deaconess Medical Center and Harvard Medical School published in the Mayo Clinic Proceedings (2). The authors reviewed  the evidence basis for 153 interventional guidelines including 2 from the American College of Chest Physicians and the American Thoracic Society. Of the 3425 recommendations reviewed, 11% were supported by level A evidence, 42% by level B, and 48% by level C. These numbers are very close to the results published by Lee and Vielemeyer (3) for the Infectious Disease Society of America guidelines where only 14% of the guidelines were based on level A evidence and 55% by level C.

So what's wrong with the majority of guidelines based on expert opinion? After all, these are experts in the field and it can be argued that most of these opinions are probably right and that physicians want guidance from the experts. The problem is that they are opinion and sometimes wrong. When they are wrong the potential exists for causing large and devastating harm to patients. This has become an increasingly frequent. As examples:

1. Tight control of glucose in the intensive care unit which according to the largest and best done multi-center trial, causes a 14% increase in ICU mortality (4).
2. Xigris (activated protein C) for adults with septic shock which caused an increase in bleeding and a small but insignificant increase in mortality leading to withdrawal of the drug (5).
3. Perioperative beta blockers which Cole and Francis calculated caused an excess mortality of 800,000 deaths in Europe over the past 5 years (6).
4. Fluid boluses for in African children with severe infection which caused a 49% increase in mortality (7).

Guideline interventions leading to a decrease in mortality are rare and there are no carefully-done, randomized trials of guidelines that have shown a 14% decrease in mortality in the ICU, saved 800,000 lives or improved mortality by 49% in severe infection. So the question arises why were these guidelines put in place, and in some cases, why do they persist? In an editorial which was to be published on January 21 in the European Heart Journal, Cole and Francis raised the possibility that the responsibility for misconduct lies not just with misguided researchers but also the institutions and the institutional leaders that provide uncritical support to research factories. Further, they discussed the role of journal editors and, even, journal readers. However, the two editorials were withdrawn about an hour after the first was published.

It appears that some guidelines have become a cesspool of conflicts of interest (COI). As pointed out in the article Dr. Mathew reviewed, 62% of the guidelines failed to comment on COIs; when disclosed, 91% of guidelines reported COIs. In a egregious example of COI influencing guidelines, the research done by Don Poldermans on perioperative beta blockers has been discredited and he has been dismissed from his university (6). Poldermans also chaired the guideline writing committee for the European Society of Cardiology on perioperative beta blockers. The previously mentioned editorials by Cole and Francis discussing Poldermans' research and its implications were retracted by the European Heart Journal. Why the journal chose to retract the editorials is unclear but one wonders if threats of loss of advertising or lawsuits from pharmaceutical company lawyers may have had something to do with it.

The story of Xigris is a further example of COIs gone amuck (8,9). Eli Lilly, the manufacturer of Xigris, provided a $1.8 million grant to fund a task force on “Values, Ethics and Rationing in Critical Care” reportedly to further the concept that it was unethical to withhold Xigris from septic patients. Eli Lilly provided over 90% of the funding for The Surviving Sepsis Campaign, launched in October 2002 to create guidelines for the treatment of sepsis.  Many of the international experts who formulated the recommendations of this group had significant outside financial relationships with Eli Lilly. As subsequent prospective trials began to raise important concerns regarding the safety and efficacy of Xigris, these concerns were repeatedly and conspicuously absent from published recommendations of the Surviving Sepsis campaign. In 2004, Eli Lilly started a program of offering unrestricted grants to institutions for implementing Surviving Sepsis Campaign patient management bundles.

The leaders in healthcare from the Institute of Healthcare Improvement (IHI) to the local leaders often have substantial COIs combined with a weak backgrounds in medicine and research. For example, the evidence basis for IHI's 100,000 Lives Campaign was weak (10). However, the non-peer reviewed press releases allowed IHI to receive a landslide of “brand recognition” which undoubtedly led to substantial new revenues and philanthropic dollars (10). Locally, many CEOs and managers are operating under incentive systems that tie bonuses to guideline compliance. One chairman of medicine, asked me, "Why is my bonus tied to how many pneumococcal vaccines are administered?". Others may not be so willing to question the hand that feeds them.

It is unclear why professional societies and medical boards have been so silent about guidelines with a weak evidence base. Both were created to protect the public's health. Practice of medicine and nursing has been restricted to those with appropriate education and licensure who accept the responsibility for their actions. The guideline process can allow the unscrupulous to side step these regulations and responsibility, sometimes for their own financial gain. If the medical societies and medical boards are unwilling to intervene, perhaps a federal agency or regulator not vulnerable to such concerns might be better suited to regulate the implementation of guidelines.

Richard A. Robbins, MD*

Editor

References

1. Feuerstein JD, Akbari M, Gifford AE, Hurley CM, Leffler DA, Sheth SG, Cheifetz AS. Systematic analysis underlying the quality of the scientific evidence and conflicts of interest in interventional medicine subspecialty guidelines. Mayo Clin Proc. 2014;89(1):16-24. [CrossRef] [PubMed] 
2. Mathew M. January 2014 pulmonary journal club: interventional guidelines. Southwest J Pulm Crit Care. 2014;8(1):70. [CrossRef]
3. Lee DH, Vielemeyer O. Analysis of overall level of evidence behind infectious diseases society of America practice guidelines. Arch Intern Med. 2011;171:18-22. [CrossRef] [PubMed] 
4. NICE-SUGAR Study Investigators, Finfer S, Chittock DR, et al. Intensive versus conventional glucose control in critically ill patients. N Engl J Med. 2009;360(13):1283-97. [CrossRef] [PubMed] 
5. Ranieri VM, Thompson BT, Barie PS, et al. Drotrecogin alfa (activated) in adults with septic shock. N Engl J Med 2012; 366:2055-64. [CrossRef] [PubMed] 
6. Maitland K, Kiguli S, Opoka RO, et al. Mortality after fluid bolus in African children with severe infection. N Engl J Med. 2011;364(26):2483-95. [CrossRef] [PubMed] 
7. Eichacker PQ, Natanson C, Danner RL. Surviving Sepsis – Practice guidelines, marketing campaigns and Eli Lilly. N Engl J Med 2006;355:1640-2. [CrossRef] [PubMed]
8. Raschke RA. July 2012 critical care journal club. Southwest J Pulm Crit Care 2012;5:54-7.
9. Robbins RA. The unfulfilled promise of the quality movement. Southwest J Pulm Crit Care. 2014;8(1):50-63. [CrossRef]

*The views expressed in this editorial are those of the author and do not necessarily represent the views of the Arizona, New Mexico, Colorado or California Thoracic Societies or the Mayo Clinic.

Reference as: Robbins RA. What's wrong with expert opinion? Southwest J Pulm Crit Care. 2014;8(1):71-3. doi: http://dx.doi.org/10.13175/swjpcc008-14 PDF