Guidelines, Recommendations and Improvement in Healthcare

“You will never understand bureaucracies until you understand that for bureaucrats procedure is everything and outcomes are nothing.”-Thomas Sowell

Reference as: Robbins RA, Thomas AR, Raschke RA. Guidelines, recommendations and improvement in healthcare. Southwest J Pulm Crit Care 2011;2:34-37. (Click here for PDF version)

In the February, 2011 Critical Care Journal Club two articles were reviewed that dealt with Infectious Disease Society of America (IDSA) guidelines (click here for Critical Care Journal Club). The first by Lee and Vielemeyer (1) reviewed the evidence basis for the 4218 IDSA recommendations and found that only 14% were based on Level 1 evidence (data from >1 properly randomized controlled trial). The graph summarizing the data in Figure 1 of the manuscript is exemplary in its capacity to communicate the weak evidence basis for many of the IDSA recommendations.

A second study by Kett et al. (2) examined the outcomes when the American Thoracic Society (ATS)/IDSA therapeutic guidelines for management of possible multidrug-resistant pneumonia were followed. The authors found a 14% difference in survival when the guidelines were followed, but surprisingly, the survival was better if the guidelines were not followed. Dr. Kett and colleagues are to be congratulated for their candor in reporting their retrospective analysis of empirical antibiotic regimens for patients at risk for multidrug-resistant pathogens. The ATS/IDSA guidelines (3) state that “combination therapy should be used if patients are likely to be infected with MDR pathogens (Level II or moderate evidence that comes from well designed, controlled trials without randomization…”. However, the ATS/IDSA guidelines go on to state, “No data have documented the superiority of this approach compared with monotherapy, except to enhance the likelihood of initially appropriate empiric therapy (Level I evidence…from well conducted, randomized controlled trials)” (4).

The problem comes with the interpretation and implementation of these and other guidelines. Some, usually inexperienced clinicians or nonclinicians, seem to believe that following any set of guidelines will enhance the “quality” of patient care. Not all guidelines or studies are created equally. Some are evidence-based, important, correct and likely to make a real difference. These usually come from professional societies and are authored by well-respected, experts in the field whose goal is improve patient outcomes. As suggested by Kett’s article even these guidelines may not be infallible. Other guidelines are not evidence-based, unimportant, incorrect and can border on the trivial. These are often authored by nonprofessional, nonexperts to create a “political statistic” (5) rather than improve patient care.

If some guidelines are bad, how can those be separated from the good? We suggest 5 traits of quality guidelines: 

1. The guideline’s authors are identified and are well-respected, experts in the field appropriate to the guideline.
2. The authors identify potential conflicts of interest.
3. The evidence is graded and supported by references to relevant scientific literature.
4. The guidelines state how they selected and reviewed the references on which the guidelines are based.
5. After completion, the guidelines are reviewed by a group of reasonably knowledgeable individuals (for example the IDSA Board of Directors) that can be identified and are willing to risk the reputation of themselves and their organization on the guidelines.

Even with the above safeguards guidelines may be non-evidence-based, unimportant, incorrect or trivial, and if so, implementation may be at best a waste of resources, or at worst harmful to patient care. We ask that guideline writing committees show restraint in authoring documents which are little more than their opinions. Not every medical question, especially the trivial and the unimportant, needs a guideline. Furthermore, we would ask an endorsement from professional organizations that only guidelines based on randomized clinical trials be given a strong recommendation. As pointed out by Lee and Vielemeyer (1) only 23% of the IDSA guidelines were supported by randomized trials while 37% of strong recommendations were supported only by opinion or descriptive studies.

IDSA states on their guidelines website, “It is important to realize that guidelines cannot always account for individual variation among patients. They are not intended to supplant physician judgment with respect to particular patients or special clinical situations. IDSA considers adherence to the guidelines listed below to be voluntary, with the ultimate determination regarding their application to be made by the physician in the light of each patient’s individual circumstances” (6). Despite this and other disclaimers, guidelines often take on a life onto themselves, frequently carrying the weight of law, regardless of the supporting evidence. We call for professional societies to end the practice of strongly recommending those guidelines based on opinion. Such practices have led and will continue to lead to systematic patient harm. Only those guidelines based on strong evidence should be given a strong recommendation. If the professional societies believe an opinion on a particular issue is appropriate despite a lack of evidence, a different designation such as recommendation or suggestion should be used to clearly separate it from a guideline.  The term guideline should be reserved for those statements that are evidence-based, important, and almost certainly correct and can make a real difference to patients.

Richard A Robbins MD, Allen R Thomas MD, and Robert A Raschke MD

References

1. Lee DH, Vielemeyer O. Analysis of overall level of evidence behind infectious diseases society of America practice guidelines. Arch Intern Med. 2011;171:18-22.
2. Kett DH, Cano E, Quartin AA, Mangino JE, Zervos MJ, Peyrani P, Cely CM, For KD, Scerpella EG, Ramirez JA. Implementation of guidelines for management of possible multidrug-resistant pneumonia in intensive care: an observational, multicentre cohort study.  Lancet Infect Dis 2011 Jan 19. [Epub ahead of print].
3. American Thoracic Society, Infectious Diseases Society of America. Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Respir Crit Care Med 2005;171:388–416.
4. Paul M, Benuri-Silbiger I, Soares-Weiser K, Liebovici L. Beta-Lactam monotherapy versus beta-lactam–aminoglycoside combination therapy for sepsis in immunocompetent patients: systematic review and metaanalysis of randomised trials. BMJ, doi:10.1136/bmj.38028.520995.63 (published March 2, 2004). Available at URL http://bmj.bmjjournals.com/cgi/reprint/bmj.38028.520995.63v1.pdf?ck_nck (accessed February 11, 2011).
5. Churchill, Winston. London, UK. 1945. as cited in The Life of Politics, 1968,  Henry Fairlie, Methuen, pp. 203-204.
6. Infectious Disease Society of American. Standards, Practice Guidelines, and Statements Developed and/or Endorsed by IDSA. Available at URL http://www.idsociety.org/content.aspx?id=9088 (accessed February 12, 2011).