What's Wrong with Expert Opinion?

In this month's Pulmonary Journal Club Dr. Mathew reviews an article by Feuerstein et al. (1) from Beth Israel Deaconess Medical Center and Harvard Medical School published in the Mayo Clinic Proceedings (2). The authors reviewed  the evidence basis for 153 interventional guidelines including 2 from the American College of Chest Physicians and the American Thoracic Society. Of the 3425 recommendations reviewed, 11% were supported by level A evidence, 42% by level B, and 48% by level C. These numbers are very close to the results published by Lee and Vielemeyer (3) for the Infectious Disease Society of America guidelines where only 14% of the guidelines were based on level A evidence and 55% by level C.

So what's wrong with the majority of guidelines based on expert opinion? After all, these are experts in the field and it can be argued that most of these opinions are probably right and that physicians want guidance from the experts. The problem is that they are opinion and sometimes wrong. When they are wrong the potential exists for causing large and devastating harm to patients. This has become an increasingly frequent. As examples:

1. Tight control of glucose in the intensive care unit which according to the largest and best done multi-center trial, causes a 14% increase in ICU mortality (4).
2. Xigris (activated protein C) for adults with septic shock which caused an increase in bleeding and a small but insignificant increase in mortality leading to withdrawal of the drug (5).
3. Perioperative beta blockers which Cole and Francis calculated caused an excess mortality of 800,000 deaths in Europe over the past 5 years (6).
4. Fluid boluses for in African children with severe infection which caused a 49% increase in mortality (7).

Guideline interventions leading to a decrease in mortality are rare and there are no carefully-done, randomized trials of guidelines that have shown a 14% decrease in mortality in the ICU, saved 800,000 lives or improved mortality by 49% in severe infection. So the question arises why were these guidelines put in place, and in some cases, why do they persist? In an editorial which was to be published on January 21 in the European Heart Journal, Cole and Francis raised the possibility that the responsibility for misconduct lies not just with misguided researchers but also the institutions and the institutional leaders that provide uncritical support to research factories. Further, they discussed the role of journal editors and, even, journal readers. However, the two editorials were withdrawn about an hour after the first was published.

It appears that some guidelines have become a cesspool of conflicts of interest (COI). As pointed out in the article Dr. Mathew reviewed, 62% of the guidelines failed to comment on COIs; when disclosed, 91% of guidelines reported COIs. In a egregious example of COI influencing guidelines, the research done by Don Poldermans on perioperative beta blockers has been discredited and he has been dismissed from his university (6). Poldermans also chaired the guideline writing committee for the European Society of Cardiology on perioperative beta blockers. The previously mentioned editorials by Cole and Francis discussing Poldermans' research and its implications were retracted by the European Heart Journal. Why the journal chose to retract the editorials is unclear but one wonders if threats of loss of advertising or lawsuits from pharmaceutical company lawyers may have had something to do with it.

The story of Xigris is a further example of COIs gone amuck (8,9). Eli Lilly, the manufacturer of Xigris, provided a $1.8 million grant to fund a task force on “Values, Ethics and Rationing in Critical Care” reportedly to further the concept that it was unethical to withhold Xigris from septic patients. Eli Lilly provided over 90% of the funding for The Surviving Sepsis Campaign, launched in October 2002 to create guidelines for the treatment of sepsis.  Many of the international experts who formulated the recommendations of this group had significant outside financial relationships with Eli Lilly. As subsequent prospective trials began to raise important concerns regarding the safety and efficacy of Xigris, these concerns were repeatedly and conspicuously absent from published recommendations of the Surviving Sepsis campaign. In 2004, Eli Lilly started a program of offering unrestricted grants to institutions for implementing Surviving Sepsis Campaign patient management bundles.

The leaders in healthcare from the Institute of Healthcare Improvement (IHI) to the local leaders often have substantial COIs combined with a weak backgrounds in medicine and research. For example, the evidence basis for IHI's 100,000 Lives Campaign was weak (10). However, the non-peer reviewed press releases allowed IHI to receive a landslide of “brand recognition” which undoubtedly led to substantial new revenues and philanthropic dollars (10). Locally, many CEOs and managers are operating under incentive systems that tie bonuses to guideline compliance. One chairman of medicine, asked me, "Why is my bonus tied to how many pneumococcal vaccines are administered?". Others may not be so willing to question the hand that feeds them.

It is unclear why professional societies and medical boards have been so silent about guidelines with a weak evidence base. Both were created to protect the public's health. Practice of medicine and nursing has been restricted to those with appropriate education and licensure who accept the responsibility for their actions. The guideline process can allow the unscrupulous to side step these regulations and responsibility, sometimes for their own financial gain. If the medical societies and medical boards are unwilling to intervene, perhaps a federal agency or regulator not vulnerable to such concerns might be better suited to regulate the implementation of guidelines.

Richard A. Robbins, MD*

Editor

References

1. Feuerstein JD, Akbari M, Gifford AE, Hurley CM, Leffler DA, Sheth SG, Cheifetz AS. Systematic analysis underlying the quality of the scientific evidence and conflicts of interest in interventional medicine subspecialty guidelines. Mayo Clin Proc. 2014;89(1):16-24. [CrossRef] [PubMed] 
2. Mathew M. January 2014 pulmonary journal club: interventional guidelines. Southwest J Pulm Crit Care. 2014;8(1):70. [CrossRef]
3. Lee DH, Vielemeyer O. Analysis of overall level of evidence behind infectious diseases society of America practice guidelines. Arch Intern Med. 2011;171:18-22. [CrossRef] [PubMed] 
4. NICE-SUGAR Study Investigators, Finfer S, Chittock DR, et al. Intensive versus conventional glucose control in critically ill patients. N Engl J Med. 2009;360(13):1283-97. [CrossRef] [PubMed] 
5. Ranieri VM, Thompson BT, Barie PS, et al. Drotrecogin alfa (activated) in adults with septic shock. N Engl J Med 2012; 366:2055-64. [CrossRef] [PubMed] 
6. Maitland K, Kiguli S, Opoka RO, et al. Mortality after fluid bolus in African children with severe infection. N Engl J Med. 2011;364(26):2483-95. [CrossRef] [PubMed] 
7. Eichacker PQ, Natanson C, Danner RL. Surviving Sepsis – Practice guidelines, marketing campaigns and Eli Lilly. N Engl J Med 2006;355:1640-2. [CrossRef] [PubMed]
8. Raschke RA. July 2012 critical care journal club. Southwest J Pulm Crit Care 2012;5:54-7.
9. Robbins RA. The unfulfilled promise of the quality movement. Southwest J Pulm Crit Care. 2014;8(1):50-63. [CrossRef]

*The views expressed in this editorial are those of the author and do not necessarily represent the views of the Arizona, New Mexico, Colorado or California Thoracic Societies or the Mayo Clinic.

Reference as: Robbins RA. What's wrong with expert opinion? Southwest J Pulm Crit Care. 2014;8(1):71-3. doi: http://dx.doi.org/10.13175/swjpcc008-14 PDF

The Tremendous Threes! Annual Report from the Editor

With the end of 2013, the Southwest Journal of Pulmonary and Critical Care (SWJPCC) completed its third year of operation. Our first manuscript was posted on November 11, 2010. We posted 8 manuscripts our first year, 68 in 2011, 113 in 2012 and 164 in 2013 (Table 1).

Table 1. Yearly submissions, total postings and postings by category.

Accompanying our increase in manuscripts, our readership has steadily grown to over 12,000/month unique IP addresses and over 16,000/month page views (the number of files that are requested from a site, also known as “hits”) (Figure 1).

Figure 1. Growth of unique IP addresses and page views by month November 2010 to December 2013.

We had some big changes in 2013. Some of which are listed below:

• The Mayo Clinic Minnesota Critical Care partnered with the Arizona, New Mexico and Colorado Thoracic Societies in SWJPCC.
• Continuing medical education was offered for the Cases of the Month in Pulmonary, Critical Care and Imaging
• There was a marked increase in the number of imaging postings, particularly the “Medical Image of the Week”.
• We have begun a monthly series entitled “Ultrasound for Critical Care Physicians” taking advantage of an on-line’s journal capability to display movies.
• A Tucson Pulmonary Journal Club was added.
• We added digital object identifiers (doi) for each posting.
• We began using CrossRef to link references to their doi and to PubMed.
• CLOCKSS began preserving our content.

Many need to be thanked. First, thanks to our authors. You took a chance on a new journal and we appreciate the opportunity to publish your work. Second, thanks to our reviewers.  SWJPCC, like all journals, relies upon expert reviewers in order to publish the highest quality manuscripts. We thank the reviewers for their time and effort in the prompt submission of their reviews. A list of reviewers for 2013 is below:

• Owen Austrheim
• David Baratz
• Jay Blum
• Michel Boivin
• Rohit Budhiraja
• Janet Campion
• John Galgiani
• Michael Garrett
• Richard Gerkin
• Michael Gotway
• Richard Helmers
• Steven Klotz
• James Knepler
• KennethKnox
• Timothy Kuberski
• Calvin Kunin
• Manoj Mathew
• Vijaychandran Nair
• Sairam Pathsarathy
• Vinay Prasad
• Neal Rinee
• Clement Singarajah
• Linda Snyder
• Allen Thomas
• Lewis Wesselius

Our gratitude goes to the Arizona, New Mexico, and Colorado Thoracic Societies and the Mayo Clinic Rochester for their support. Thanks to our associate editors who have put in much more work than we had the right to ask. A special note of thanks to those who continue to do regular features in SWJPCC-Bob Raschke and Manoj Mathew for the critical care and pulmonary journal clubs; Mike Gotway, Lew Wesselius and Bob Raschke for the cases of the month; Rohit Budhiraja for the Sleep Question of the Month; and Ken Knox for the Medical Image of the Week; and Peter Wagner for his wine column, Slurping Around with PDW. SWJPCC acknowledges the Phoenix Pulmonary and Critical Care Research and Education Foundation which has provided the monetary support for SWJPCC and Squarespace our web host. Last, and most importantly, thanks to our readers. Please visit as often as you can and feel free to provide us with your input.

What’s ahead for 2014? We hope to improve the content, especially the scientific content, for 2014, but we will continue to emphasize clinical medicine and education. CME will continue to be offered for the previous 12 Pulmonary, Critical Care, and Imaging Cases of the Month for a total of 36 CME offerings at any one time. We would welcome suggestions for any improvements.

Richard A. Robbins, MD

Editor, SWJPCC

Reference as: Robbins RA. The tremendous threes! annual report from the editor. Southwest J Pulm Crit Care. 2014:8(1):1-3. doi: http://dx.doi.org/10.13175/swjpcc001-14 PDF

Obamacare and Computers-Who Is to Blame?

Count me among the unsympathetic to the recent Center for Medicare and Medicaid (CMS) problems with the rollout of Obamacare, aka the Affordable Care Act. Yesterday, Marilyn Tavenner, the Administrator of CMS, apologized for the troubled rollout of the federal health insurance web site and promised to fix the problems that have prevented many consumers from signing up for coverage (1). Today, Tavenner’s boss, Kathleen Sebelius, Health and Human Services Secretary acknowledged “frustrating” problems that would be fixed “as soon as possible”. She offered an apology for the site’s troubled launch, while also attributing the glitches to private-sector contractors (2). The later is particularly telling.

We have repeatedly heard how the “magic” of the computer can solve problems in health care (3). To this end, CMS created a Medicare Electronic Health Care (EHR) Incentive Program and touted that eligible professionals could receive up to $44,000 over 5 years for full implementation (4). However, CMS estimated the average cost of implementing an EHR over 5 years was $48,000 or a loss of $4,000 assuming the best reimbursement. It is not clear how close these dollar amounts match the actual numbers but a number of private practice physicians have complained that the cost was much more and the reimbursement much less (Robbins RA, unpublished observations). What was most disturbing is the implication that physicians are to blame when EHR implementation is slow or fails to achieve the promised improved care at lower costs (3).

The recent Obamacare rollout problems can be blamed on a variety of issues from too many contractors involved, inadequate testing, poor leadership, etc., but the main fault has been the perception that health information technology (IT) is easy. However, the available evidence suggests that health IT is not “magic”.  In most industries, IT has taken years, often decades to exert its effects (5).  Personally I believe health IT can have a huge beneficial effect on healthcare delivery-but it might take a decade or two. 

A meaningful partnership between clinicians, administrators and payers achieving and rewarding high-value care is needed. To do this physicians need considerable input, and perhaps more importantly, control of any EHR. Second, physicians need to be rewarded for good care which is centered on improved patient outcomes and not endless checklists that do little more than consume time. Failure to do so will result in inefficient and more costly care and not in the improvements Obamacare promised. To paraphrase Cassius from Julius Caesar, the fault is not in our contractors, but in ourselves. It is distressing that political ambition and arrogance may jeopardize the healthcare of millions of Americans.

Richard A. Robbins, MD*

Editor, SWJPCC

References

1. Somashekhar S. Administration official Marilyn Tavenner apologizes for HealthCare.gov problems. Washington Post. October 29, 2013. Available at: http://www.washingtonpost.com/national/health-science/administration-official-marilyn-tavenner-apologizes-for-healthcaregov-problems/2013/10/29/4d2a07ea-40c6-11e3-9c8b-e8deeb3c755b_story.html (accessed 10/30/13).
2. Branigin W, Somashekhar S. Kathleen Sebelius acknowledges “frustrating” problems with health-care web site. Washington Post. October 30, 2013. Available at: http://www.washingtonpost.com/politics/kathleen-sebelius-acknowledges-frustrating-problems-with-health-care-web-site/2013/10/30/8cf36c98-415e-11e3-a751-f032898f2dbc_story.html (accessed 10/30/13).
3. Robbins RA. Getting the best care at the lowest price. Southwest J Pulm Crit Care 2012;5:145-8.
4. http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/index.html?redirect=/ehrincentiveprograms/ (accessed 10/30/13).
5. Jha A. As the debate over Obamacare implementation rages, a success on the IT front. The Health Care Blog. July 12, 2013. Available at: http://thehealthcareblog.com/blog/2013/07/12/as-the-debate-over-obamacare-implementation-rages-a-success-on-the-it-front/ (accessed 10/30/13).

*The views expressed in this editorial are those of the author and do not necessarily represent the views of the Arizona, New Mexico or Colorado Thoracic Societies or the Mayo Clinic.

Reference as: Robbins RA. Obamacare and computers-who is to blame? Southwest J Pulm Crit Care. 2013;7(4):269-70. doi: http://dx.doi.org/10.13175/swjpcc145-13 PDF 

HIPAA-Protecting Patient Confidentiality or Covering Something Else?

A case of a physician fired from the Veterans Administration (VA) for violation of the Health Care Portability and Accountability Act of 1996 (HIPAA) illustrates a problem with both the law and the VA. Anil Parikh, a VA physician at the Jesse Brown VA in Chicago, was dismissed on a charge of making unauthorized disclosures of confidential patient information on October 19, 2007.  On January 3, 2011 the Merit Systems Protection Board (MSPB) reversed Dr. Parikh’s removal.

Dr. Parikh's initially made disclosures to the VA Office of Inspector General and to Senator Barack Obama and Congressman Luis Gutierrez, in whose district the Jesse Brown VA lies.  Dr. Parikh alleged that there were systematic problems within the Jesse Brown VA that resulted in untimely and inadequate patient care. The confidential patient information Parikh disclosed included examples of the misdiagnoses and misdirection of patients within the hospital. Specifically, Dr. Parikh alleged that a physician failed to diagnose a patient’s rectal abscess and sent him home rather than refer him for proper surgical treatment. Two patients who should have been accepted in the emergency room were improperly directed to the urgent care area. One of these patients who should have been admitted to the intensive care unit was improperly placed on the general medical floor, resulting in the eventual deterioration of his condition to the point where he required intubation. Parikh later testified that he made these disclosures out of concern for patient health and safety.

The IG referred the matter to Mr. James Jones, director of the Jesse Brown VA for investigation. Mr. Jones assigned Dr. Jeffrey Ryan, Associate Chief of Staff, to investigate the allegations. Dr. Ryan concluded that there was no evidence of mismanagement or misdiagnosis and the IG closed their case. Dr. Parikh then disclosed the information to Denise Mercherson, his own attorney; Dr. Fred Zar, the director of the internal medicine residency program at Loyola, the American College of Graduate and Medical Education (ACGME) and other members of Congress serving on Congressional VA oversight committees. After these disclosures, Parikh was fired by Mr. Jones.

After exhausting his appeals to be reinstated with the VA Office of Special Counsel, Parikh filed an individual right of action (IRA) with the MSPB contending that his disclosures were protected under the Whistleblower Protection Act (WPA), and that the VA removed him based on those protected disclosures. The administrative judge hearing the case found that Parikh failed to establish MSPB jurisdiction over his appeal because “he failed to make a nonfrivolous allegation that any of his disclosures were protected under the WPA”.  Parikh then filed a petition for review by the full board, and the MSPB reversed the initial decision.  The issue for MSPB was whether Parikh's disclosures were protected under the WPA. Although the administrative judge initially hearing the case found that Parikh failed to establish that he reasonably believed these disclosures were evidence of a substantial and specific danger to public health or safety, the full MSPB disagreed. They found that the nature of the harm that could result from patient care and management issues that Parikh disclosed was "severe” that could result in patient death.

The VA argued that Parikh's disclosures were prohibited under HIPPA. According to Lisa Yee and Timothy Morgan, lawyers for the Chicago VA General Counsel, Parikh's disclosures were not covered by the WPA because the WPA and the Privacy Act of 1974 excludes disclosures prohibited by law. The VA also argued that Dr. Parikh's disclosures were prohibited by HIPAA. The MSPB had little trouble rejecting both these arguments, finding that one of the exceptions is a disclosure to a Congressional committee. The VA lastly argued that Dr. Parikh's disclosures were prohibited by VA policy since the VA had not approved disclosure of the information. However, the MSPB found that the VA's policy in question was not a "substantive" rule, but merely a reference to the HIPPA and the Privacy Act. The MSPB found that the disclosures were a factor to his removal and ordered him reinstated with back pay.

Physicians considering a career with the VA should carefully examine this case. The MSPB concluded that the VA retaliated against Dr. Parikh, not for disclosing confidential patient information, but whistleblowing. After over 3 years, Dr. Parikh has his job back but his work situation is probably not “friendly”. And what has become of the VA administrators and their lawyers who violated WPA by retaliating against Dr. Parikh-to my knowledge, nothing.

The adversarial relationship between the VA administrators and physicians appears to be a one-way street. A physician can have their career destroyed by the VA, but if the accusations are unjustified, there are no consequences to the accusers. On the other hand, physicians that voice concerns for patient care and safety can have their professional reputation ruined by the VA. Particularly concerning is the misuse of HIPAA by VA attorneys as a weapon against physicians.

Dr. Parikh’s case would not appear to be an isolated event. A quick review of the news reveals a VA nurse in Albuquerque was charged with sedition for criticism of the Bush administration’s handling of hurricane Katrina and Iraq (2).  In Phoenix a VA physician was fired after forwarding an e-mail from a Senator John McCain staffer suggesting physicians go to a McCain political rally and lobby for a new VA research building (3). The Phoenix VA chief of hematology/oncology resigned after his name was placed in the National Practioner Databank; an action he felt was unjustified (4). Most recently the Phoenix VA public relations director was demoted after giving unfavorable testimony about VA administrators (5). If the VA is having trouble recruiting as their recent TV advertising suggests, they might consider a different approach. A good start would be the use of HIPAA to protect patient confidentiality rather than cover something else.

Richard A. Robbins, MD

Editor

References

1. US Merit System Protection Board. 2011 MSPB 1. Docket No. CH-1221-08-0352-B-2. Available at: http://www.mspb.gov/. Accessed 9/10/13. 
2. Dees DE. VA nurse in New Mexico accused of sedition. Mother Jones. 2006. Available at: http://www.motherjones.com/mojo/2006/02/va-nurse-new-mexico-accused-sedition. Accessed 9/10/13. 
3. Franklin RE. VA doc fired for political email. Arizona Star. 2011. Available at: http://azstarnet.com/news/local/va-doc-fired-for-political-email/article_3e353bbf-b04a-52ff-8a9c-6cb49e78a47a.html. Accessed  9/10/13.
4. Robbins RA. Profiles in medical courage: Thomas Kummet and the courage to fight burearcracy. Southwest J Pulm Crit Care. 2013;6(1):29-35.
5. Wagner D. VA official in Arizona demoted after her testimony. Arizona Republic. Available at http://www.azcentral.com/news/arizona/articles/20130314va-official-arizona-pedene-demoted-after-testimony.html  accessed 9/10/13.

Reference as: Robbins RA. HIPAA-protecting patient confidentiality or covering something else? Southwest J Pulm Crit Care. 2013;7(4):236-8. doi: http://dx.doi.org/10.13175/swjpcc128-13 PDF

Are Medical Guidelines Better Than Flipping a Coin?

A recent article by Prasad et al. (1) in the Mayo Clinic Proceedings reviewed all original articles published over 10 years (2001-2010) in the New England Journal of Medicine (NEJM). Articles were classified on the basis of whether they addressed a medical practice, whether they tested a new or existing therapy, and whether results were positive or negative. Most striking was that of the 363 articles examining standards of care, 146 (40.2%) reversed that practice, whereas 138 (38.0%) reaffirmed it. The remaining percentage remained inconclusive.

As pointed out in an accompanying editorial, the NEJM is widely read, has high visibility and has a large influence on the mass media and medical practitioners (2). However, the effect of articles published in the NEJM, Lancet and JAMA, the top 3 general medical journals in terms of impact factor, are markedly inflated (3,4). Presumably, a randomized trial published in these journals must be true because these are the “best” medical journals. 

Prasad’s conclusions that the NEJM reversed accept medical practice about half the time would be consistent with the Cochrane Review of Clinical Trials.  El Dib et al. (5) concluded in 2004 that there is insufficient evidence to endorse the examined interventions 47.8% of the time. A repeat evaluation in 2011 showed that the percentage of insufficient evidence remained about the same (6).

Now before anyone gets too upset, I happen to agree that NEJM, Lancet and JAMA are probably the best and most influential medical journals. Authors send their best work to these journals because they are widely read. The editors choose articles based on their importance and whether the work is new, innovative, or contradicts accepted medical practice. All of this makes these journals the most influential.

Not surprisingly, authors of guidelines give more credibility to these higher impact journals. In other words, a randomized trial done in the NEJM is more likely to influence a guideline writing committee that a trial from the Southwest Journal of Pulmonary and Critical Care. Looking at Bob Raschke’s recent journal club reviewing 6 landmark randomized controlled trials that were eventually reversed, 5 were from the NEJM or JAMA (7). Several of the outcomes from these studies were the basis for guidelines.

Guideline writing committees really cannot do better than the medical literature.  However, if half the established standards of care are wrong as Prasad suggests, half the guidelines based on these standards of care are also wrong. Should we require higher levels of evidence before practice guidelines are recommended-perhaps at least two, or in cases of marginal effects, even more trials. To me the overwhelming answer has to be yes.

Lee and Vielemeyer (8) found that only 14% of the Infectious Disease Society of America (IDSA) guidelines are based on level I evidence (data from >1 properly randomized controlled trial). Much of this 14% and the 86% that are below level I evidence will eventually be proven wrong. I doubt that other medical societies are performing much better. Serving on a guideline writing committee is a compliment paid by professional colleagues. However, as Lee and Vielemeyer point out, the guidelines tend to be more opinion than science. This is especially true when the data supporting standards of care is weak, nonexistent or conflicting. Experts often rationalize that an answer is needed, even when the correct response might be “I don’t know”.

All this points out that reading and interpreting medical literature is difficult. It takes knowledge, experience and a healthy dose of skepticism.  Experts relying on the best evidence frequently get it wrong. Improvement lies in the intellectual honesty of the guidelines committees and research. Well designed clinical trials are usually expensive and time-consuming, not what health care administrators want to hear in a time of restricted budgets. However, can we afford not to invest in getting it right?

Richard A. Robbins, MD*

References

1. Prasad V, Vandross A, Toomey C, et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc. 2013;88(8):790-8. [CrossRef] [PubMed] 
2. Ioannidis JP. How many contemporary medical practices are worse than doing nothing or doing less? Mayo Clin Proc. 2013;88(8):779-81. [CrossRef] [PubMed] 
3. ISI Web of Science. Journal Citation Reports. Available at: http://thomsonreuters.com/journal-citation-reports.  Accessed August 7, 2013.
4. Siontis KC, Evangelou E, Ioannidis JP. Magnitude of effects in clinical trials published in high-impact general medical journals. Int J Epidemiol. 2011;40(5):1280-91. [CrossRef] [PubMed] 
5. El Dib RP, Atallah AN, Andriolo RB. Mapping the Cochrane evidence for decision making in health care. J Eval Clin Pract. 2007;13(4):689-692. [CrossRef] [PubMed] 
6. Villas Boas PJ, Spagnuolo RS, Kamegasawa A, et al. Systematic reviews showed insufficient evidence for clinical practice in 2004: what about in 2011? The next appeal for the evidence-based medicine age. J Eval Clin Pract. 2013;19(4):633-7. [CrossRef] [PubMed] 
7. Raschke RA. August 2013 critical care journal club: less is more. Southwest J Pulm Crit Care. 2013;7(3):162-4. [CrossRef]
8. Lee DH, Vielemeyer O. Analysis of overall level of evidence behind infectious diseases society of America practice guidelines. Arch Intern Med. 2011;171:18-22. [CrossRef] [PubMed] 

*The opinions expressed are those of the author and do not necessarily reflect the opinion or policies of the Arizona, New Mexico, or Colorado Thoracic Societies, the Mayo Clinic, or most guideline writing committees.

Reference as: Robbins RA. Are medical guidelines better than flipping a coin? Southwest J Pulm Crit Care. 2013;7(3):181-3. doi: http://dx.doi.org/10.13175/swjpcc124-13 PDF