Changes in Medicine: Fellowship

Reference as: Robbins RA. Changes in medicine: fellowship. Southwest J Pulm Crit Care 2011:3:34-36. (Click here for a PDF version)

Pulmonary fellowship in the late 70’s and early 80’s was largely unstructured.  I had the advantage of doing two fellowships. One was at the University of Nebraska Medical Center and was predominantly clinical. There was one other fellow and we spent our time going to clinic, reading pulmonary function tests, supervising exercise testing,  doing consults, and providing inpatient care both on the floors and the intensive care unit (ICU). We became involved with most of the patients in the ICU who were there for more than a day or two. The work was long and hard. We were mostly autonomous and only loosely supervised.

The attending physicians relied on us to call when we needed help or there was something we thought they should know. Call was at home but it was unusual to leave before 8 PM. The fellows alternated call every other weekend making it tolerable. There were plenty of procedures.  I did over 150 bronchoscopies my first year and performed sufficient numbers of intubations, thoracentesis, chest tubes, pulmonary artery cathers, etc. to be comfortable. There was little time or emphasis on research or other scholarly activity.

The other fellowship at the National Institutes of Health was the opposite. Research was clearly emphasized and most of our time was spent in the laboratory. Patient care was confined to patients on research protocols or consults to other services who had patients with incidental pulmonary problems. Procedures other than our research based protocols were rare.

At the time there were few critical care fellowships.  A fellow interested in the ICU usually entered a pulmonary fellowship or more rarely a cardiology fellowship. Anesthesia also practiced in the ICU at some institutions. Pediatric ICUs were left to the pediatricians. The American Board of Internal Medicine did require 36 months of fellowship but only 12 months needed to be clinical which was largely undefined.

A number of regulatory agencies entered fellowship regulation during the past 30 years. Most importantly has been the Accreditation Council on Graduated Medical Education (ACGME). As with residencies, the ACGME accredits the fellowship, and therefore, makes the rules. ACGME now recommends 24 months of clinical activity with a host of training requirements pertaining to patient care and medical knowledge (1). In addition, requirements now exist for competencies in practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice.  Procedural training such as bronchoscopy (minimum now at 100) and the newer procedures such as sleep studies and ultrasound are also recommended or required. Fellowship directors are familiar with the ACGME’s program information form (PIF) which now extends to at least 75 pages describing the program. In addition, much of the PIF is devoted to answering questions such as “Describe at least one learning activity, other than lecture, by which residents develop a commitment to carrying out professional responsibilities and an adherence to ethical principles” or “Describe the learning activity(ies) through which residents achieve competence in the elements of systems-based practice: work effectively in various health care delivery settings and systems, coordinate patient care within the health care system; incorporate considerations of cost-containment and risk-benefit analysis in patient care; and, advocate for quality patient care and optimal patient care systems and work in interprofessional teams to enhance patient safety and care quality”. So the educational requirements to meet patient care and medical knowledge requirements as well as the newer requirements have been greatly extended leaving little time for scholarly activity or research. Many, if not most, fellows now leave their fellowship having never conducted a research study nor authoring a peer-reviewed manuscript.

Other organizations such as the Joint Commission of Healthcare Organizations, American College of Chest Physicians (2) and a variety of insurance carriers have waded in on credentialing requiring certain numbers of procedures for fellows to be certified as competent. Although these requirements are not unreasonable, they are arbitrary and the evidence basis on which they were formed is unclear.

The amount of paperwork regarding fellowships has undoubtedly increased for both the fellowship programs as well as the fellows themselves tracking procedures, etc. The number of personnel necessary to administer these regulatory activities has also undoubtedly increased. Supervision of fellows has also increased with attending physicians having more input into patient care. However, whether these lead to better trained physicians or better patient care is unknown. My suspicion is that it has not, at least there appears to be no evidence that anyone benefits. On the other hand, the amount of resources spent on supervision and documentation may actually lead to a decrease in the resources available for important educational and patient care activities actually result in harm to the fellows and possibly the patients. Regulatory agencies should investigate before mandating or even recommending educational requirements. More commonly the agency convenes a group of “experts” for advice. Often there is no reliable data, and therefore, the “expert’ panel makes recommendations based on their opinions. Not only the regulatory agencies but the panels of experts need to show restraint in making recommendations when there is no data. We often tell our fellows that it is alright to say “I don’t know”. Regulatory agencies and expert panels should also be willing to admit their limitations.

Richard A. Robbins, MD

Editor, Southwest Journal of Pulmonary and Critical Care

References

1. http://www.acgme.org/ (accessed 8-8-11).
2. Ernst A, Silvestri GA, Johnstone D. Interventional pulmonary procedures: guidelines from the American college of chest physicians. Chest 2003;123;1693-1717.

Changes in Medicine: Residency

Reference as: Robbins RA. Changes in medicine: residency. Southwest J Pulm Crit Care 2011:3:8-10. (Click here for a PDF version)

The most important time in a physician’s educational development is residency, especially the first year. However, residency work and responsibility have come under the scrutiny of a host of agencies and bureaucracies, and therefore, is rapidly changing. Most important in the alphabet soup of regulatory agencies is the Accreditation Council for Graduate Medical Education (ACGME) which accredits residencies and ultimately makes the governing rules.

Resident work hours have received much attention and are clearly decreasing. However, the decline in work hours began in the 1970’s before the present political push to decrease work hours. The residency I entered in 1976 had every third night call during the first year resident’s 6-9 months on general medicine or wards. It had changed from every other night the year before. On wards, we normally were in the hospital for our 24 hours of call and followed this with a 10-12 hour day before going home and getting some well needed sleep. The third day was again a 10-12 hour day before repeating the cycle. This averages over 100 hours per week. There was one week of paid vacation and days off were rare. Both days off and vacations were expected to be done on electives.

First year residents were often poorly supervised despite a senior resident being on call with every 2 interns. Attending physicians were never in the hospital at night. I remember being told by a senior resident, that he was going to bed but I could call him if there was an emergency I could not handle-but he expected me to handle any emergency. I got the message not to call him.

The reduction in work hours was driven by residency directors trying to recruit sufficient residents to fill their slots. Residencies that required every other night call or had indigent level salaries were quickly becoming noncompetitive. By the time I left residency after 3 years, call had decreased to every fourth or fifth night and salaries had risen from about $10,000/year to $14,000/year for first year residents.

The reduction in work hours was brought to public attention and accelerated by the Libby Zion case of 1984 (1). The 18 year old Zion died from a complication of the monoamine oxidase inhibitor she had taken prior to hospitalization exacerbated by administration of meperidine and possibly by cocaine. When her father, Sidney Zion, a journalist/lawyer, learned that her doctors had been medical residents covering dozens of patients and receiving supervision only by phone, he became convinced his daughter's death was due to inadequate staffing at the New York teaching hospital where she died. Determined to ensure that others not fall victim to the same gaps that he blamed for his daughter's death, he crusaded to change resident work hours and supervision with frequent editorials and public appearances.

Over several years a sequence of events occurred to keep Zion’s death in the public eye: a grand jury was called to investigate Zion’s death; the New York State health commissioner appointed the Bell Committee to make recommendations regarding work hours; and a civil lawsuit against the doctors and hospitals was filed by Sidney Zion. All deemed the hospital negligent for leaving a first year resident alone in charge of 40 patients that night. The Bell Commission recommended that residents could work no more than 80 hours a week or more than 24 consecutive hours and senior physicians needed to be physically present in the hospital at all times and these recommendations were adopted by New York State.

The ACGME under political pressure to deal with resident work hours, appointed the Work Group on Resident Duty Hours and the Learning Environment in September 2001. The work group recommended new ACGME standards that were remarkably similar to those of the Bell Commission and these were adopted by the ACGME in 2003 (2).

The rationale behind the work hour reduction is that by working fewer hours and under greater supervision the care delivered by more rested and supervised residents will be better. A tragedy, in addition to Ms. Zion’s death, is that 27 years later we still do not know if this basic premise is true. Although the reduction in resident work hours and the in house presence of attending physicians has undoubtedly increased costs, the impact on length of stay and mortality remain largely unknown (3). The observational, retrospective research that has been done on the impact of resident hour reduction has been sufficiently flawed to make conclusions difficult (4-6). This is unfortunately part of a common trend in administrative medicine, i.e., to initiate changes based on political pressure and later attempt studies to justify the changes.

Concern has been voiced that reduction in work hours and autonomy due to increased supervision may compromise resident education (7). Although there would appear to be little evidence to date supporting this one way or another, I add my voice to those who raise this concern. Making independent decisions is vital to the maturation of residents to independent physicians. The present trend of reducing work hours and increasing supervision, may delay that learning experience to the first year or two of independent practice where correction and constructive criticism are unlikely to occur.

As work hours of residents decline, as medical knowledge expands, and as medical care becomes more complex our residencies will be hard pressed to train competent physicians. One approach is to lengthen the residencies to compensate for the reduced work hours (8). Adding another year or two of residency and/or fellowship is nothing more than extending the indentured servitude of residents to teaching hospitals. 3-6 years of post-graduate training is enough and extending the resident’s time may be more to provide adequate in house coverage than to improve the residents’ education.

I would recommend some carefully designed studies to investigate the impact of shorter work hours. The impact on mortality and length of stay should be examined along with the resident’s fund of knowledge. Perhaps armed with some sound data policy makers can make sound decisions regarding resident education, something we might call evidence-based medicine.

Richard A. Robbins, M.D.

Editor, Southwest Journal of Pulmonary and Critical Care

References

1. Lerner BH. A Case That Shook Medicine. The Washington Post, November 28, 2006. Available at http://www.washingtonpost.com/wp-dyn/content/article/2006/11/24/AR2006112400985.html

2. Friedmann P, Williams WT Jr, Altschuler SM, et al. Report of the ACGME Work Group on Resident Duty Hours. 2002. Available at: http://www.acgme.org/DutyHours/wkgroupreport611.pdf

3. Nuckols TK, Bhattacharya J, Wolman DM, Ulmer C, Escarce JJ. Cost implications of reduced work hours and workloads for resident physicians. N Engl J Med. 2009;360:2202-15.

4. Landrigan CP, Rothschild JM, Cronin JW, Kaushal R, Burdick E, Katz JT, Lilly CM, Stone PH, Lockley SW, Bates DW, Czeisler CA. Effect of reducing interns' work hours on serious medical errors in intensive care units. N Engl J Med 2004;351:1838-48.

5. Volpp KG, Rosen AK, Rosenbaum PR, Romano PS, Even-Shoshan O, Wang Y, Bellini L, Behringer T, Silber JH.. Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. JAMA. 2007;298:975-83.

6. Volpp KG, Rosen AK, Rosenbaum PR, Romano PS, Even-Shoshan O, Canamucio A, Bellini L, Behringer T, Silber JH. Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform. JAMA. 2007;298:984-92.

7. McCoy CP, Halvorsen AJ, Loftus CG, McDonald FS, Oxentenko AS. Effect of 16-hour duty periods on patient care and resident education. Mayo Clin Proc. 2011;86:192-6.

8. Larson EB, Fihn SD, Kirk LM, Levinson W, Loge RV, Reynolds E, Sandy L, Schroeder S, Wenger N, Williams M; Task Force on the Domain of General Internal Medicine. Society of General Internal Medicine (SGIM). The future of general internal medicine. Report and recommendations from the Society of General Internal Medicine (SGIM) Task Force on the Domain of General Internal Medicine. J Gen Intern Med. 2004;19:69-77.

The opinions expressed in this editorial are the opinions of the author and not necessarily the opinions of the Southwest Journal of Pulmonary and Critical Care or the Arizona Thoracic Society.

Changes in Medicine: Medical School

Reference as: Robbins RA. Changes in medicine: medical school. Southwest J Pulm Crit Care 2011:3:5-7. (Click here for a PDF version)

I recently retired and have been encouraged to write about what has changed in medicine. However, the changes have been sufficiently extensive that one editorial would be too long. Therefore, this will be the first of several editorials examining medical school, residency, fellowship and practice.

The beginning of my own medical career was 1972 when I entered medical school, graduating in 1976. My reasons for choosing the specific school I entered were several: 1. A scholarship was provided that paid tuition; 2. It was a state school and otherwise relatively cheap; 3. The school would accept me after 3 years of college and without a college degree; 4. It was the medical school of my undergraduate school and I knew many of the entering students; and 5. I was told that it mattered less where you did your medical school training than where you did your residency. I saw no reason to delay admission to obtain a college degree and wanted to proceed with my medical education.

Most medical students in 1972 were like me, white and male. The most obvious change in the past 40 years has been the increasing number of women. My class of about 150 had only a few, maybe 5, women. The percentage of women graduates has gradually risen until in 2009-2010, women received 8,133 (48.3%) of the 16,838 MD’s awarded (1). However, the numbers of underrepresented minorities has not kept pace with the increasing percentage of women. The number of blacks graduating from medical school has modestly risen from about 700 in 1980 to a little over 1109 in 2008 with a rise in Hispanics from a few hundred in 1980 to 1183 in 2008. Yet those numbers still only represent 6.9% and 7.3% of medical school graduates, respectively, far below the 12% for blacks and over 15% for Hispanics of the general population (2,3).

Over 30 years of academic medicine I have not observed much change in the medical students’ abilities by the time I see them on a pulmonary or critical care rotation their senior year. The high numbers of applicants suggest that medical school acceptance is still difficult and the mean grade point average from college of an entering student is still well above 3.5. There has been little significant change in medical school education since the Flexner report in 1910 (4). Most medical schools still consist of 2 years of pre-clinical and 2 years of clinical education just like it did when I matriculated way back in 1972-6.  There have been the occasional novel educational programs in medical schools such as 3 year programs, a combined 6 year undergraduate and MD, or earlier clinical introduction, but most of these have fallen by the wayside. I’ve witnessed graduates from several of these programs and these medical education experiments do not seem to have adversely affected the medical students’ performance by the time I see them their senior year. I still find them bright, enthusiastic and articulate and ready to continue their journey to becoming doctors as house officers.

However, a major change which may be influencing medical training and career choice is the debt incurred by medical students. Although poverty was common in my class of 1976, large debt was rare. Now approximately 86 percent of U.S. medical students graduate with some debt, and of those, the average debt is almost $160,000 (6). Students at Doctor of Osteopathy (DO) schools appear to be particularly hard hit. In the US there are only 26 osteopathic schools compared to 133 allopathic medical schools that offer the MD degree. Yet, 6 of the top 10 medical schools that lead to the most medical student debt are osteopathic schools. Medical students graduating from those 6 schools averaged over $198,000 of indebtedness in 2009 (7). It has been claimed that this debt is a major influence on residency choice with fewer students going into residencies as primary care physicians because of their debt (7). However, medical student debt seems less likely to influence residency choice since most residencies pay about the same. Rather it seems that income potential after completing training may be having some influence. Primary care physicians often receive incomes half of some specialists (6).  Medical students realize this income differential and for some may be a major influence on choosing a specialty.

The concern that medical student indebtedness can influence the rest of their careers has been voiced by many and I echo this concern. This is especially true given that medical students face at least 3 years as a house officer, where salaries of about $50-60,000/year is insufficient to allow quickly paying off student loans. Although it seems unlikely that the high cost of some medical schools can be justified, I would not suggest Government cost regulation of medical school fees. My own experience with over 30 years of Government bureaucracy is that inevitably they will dictate medical curriculum based on politics, rather than science. Instead, I would propose a system of relieving medical student debt by allowing some students to obtain debt forgiveness by Government service. More on this in the later editorials in this series.

Richard A. Robbins, M.D.

Editor, Southwest Journal of Pulmonary and Critical Care

References

1. https://www.aamc.org/download/153708/data/charts1982to2011.pdf (accessed 7-10-11).
2. http://www.scribd.com/doc/53281274/Minorities-and-Medical-Education-AAMC-Facts-and-Figures (accessed 7-10-11).
3. Cammarata J. Minorities in Medicine: Still an Unmet Need. Medscape 2010 http://www.medscape.com/viewarticle/720541 (accessed 7-10-11).
4. Flexner A. Medical Education in the United States and Canada, 1910. Available online at http://www.carnegiefoundation.org/sites/default/files/elibrary/Carnegie_Flexner_Report.pdf (accessed 7-10-11).
5. http://www.washington.edu/uaa/advising/downloads/gpamcat.pdf (accessed 7-10-11).
6. Prep V. Weigh Medical Student Debt, Specialty Choice. US News and World Report.  2011. Available on line at http://www.usnews.com/education/blogs/medical-school-admissions-doctor/2011/06/20/weigh-medical-student-debt-speciality-choice (accessed 7-10-11).
7. Hopkins K. 10 Medical Schools That Lead to Most Debt: Some students are graduating with more than $200,000 in debt. US News and World Report. 2011. Available online at http://www.usnews.com/education/best-graduate-schools/articles/2011/04/14/10-medical-schools-that-lead-to-most-debt (accessed 7-10-11).
8. Back PB, Kocher R. Why Medical School Should Be Free. New York Times. 2011. Available online at http://www.nytimes.com/2011/05/29/opinion/29bach.html?_r=2&ref=contributorshttp://www.nytimes.com/2011/05/29/opinion/29bach.html (accessed 7-10-11).

The opinions expressed in this editorial are the opinions of the author and not necessarily the opinions of the Southwest Journal of Pulmonary and Critical Care or the Arizona Thoracic Society.

The Pain of the Timeout

Reference as : Robbins RA. The pain of the timeout. Southwest J Pulm Crit Care 2011:2:102-5. (Click here for a PDF version)

An article in the Washington Post entitled “The Pain of Wrong Site Surgery” (1) caught my eye earlier this month. In 2004 the Joint Commission of Healthcare Organizations (Joint Commission or JCAHO), prompted by media reports of wrong site surgery, mandated the “universal protocol” or surgical timeout. These rules require preoperative verification of correct patient, correct site, marking of the surgical site and a timeout to confirm everything just before the procedure starts. In announcing the rules, Dr. Dennis O’Leary, then president of the Joint Commission, stated “This is not quite ‘Dick and Jane,’ but it’s pretty close,” and that the rules were “very simple stuff” to prevent events such as wrong site or patient surgery which are so egregious and avoidable that they should be “never events” because they should never happen. During the following years different components have been added to the timeout and the timeout has been extended to cover most procedures in the hospital.

However, the article goes on to state that “some researchers and patient safety experts say the problem of wrong-site surgery has not improved and may be getting worse, although spotty reporting makes conclusions difficult”. Last year 93 cases were reported to the Joint Commission in 2009 compared to 49 in 2004. Furthermore the article states that reporting data from Minnesota and Pennsylvania, two states that require reporting have not shown a decrease over the past few years.

The reason for the increasing incidence of wrong site or wrong patient operations is not totally clear. Dr. Mark Chassin, who replaced O’Leary as president of the Joint Commission in 2008, said he thinks such errors are growing in part because of increased time pressures. Preventing wrong-site surgery also “turns out to be more complicated to eradicate than anybody thought,” he said, because it involves changing the culture of hospitals and getting doctors — who typically prize their autonomy, resist checklists and underestimate their propensity for error — to follow standardized procedures and work in teams. Dr. Peter Pronovost, medical director of the Johns Hopkins Center for Innovation in Quality Patient Care, echoed those sentiments by suggesting that doctors only pay lip service to the rules. Studies of wrong-site errors have consistently revealed a failure by physicians to participate in a timeout. Dr. Ken Kizer, former Undersecretary at the Department of Veterans Affairs and President of the National Quality Forum, advocates reporting doctors to a federal agency so wrong site surgery or patient cases can be investigated and the results publicly reported.

Several points made in the article need to be clarified

1. The reason that it is unclear whether the present Joint Commission mandates actually prevents wrong site or patient surgery is that no data was systematically collected prior to implementation of the timeout to ensure that it works and no data has been collected since implementation. As with most bureaucracies, the Joint Commission emphasis has been more on ensuring compliance rather than studying the effectiveness of an intervention.
2. Although no one condones wrong site or patient surgery, it is fortunately relatively rare. Stahel et al. (2) reported 132 wrong-site and wrong-patient cases during a 6 and a half year period by over 5000 physicians. They found only one death which was attributed to a wrong-sided chest tube placement for respiratory failure (2). This is questionable because a wrong sided chest tube does  not necessarily result in a patient’s death (3). Another 43 patients had significant harm from their wrong site or patient procedure and are listed below (Table 1). 
3. Based on the above, occurrence of these wrong site or patient operations would appear to be mostly in the operating room. The surgeon often enters the operating room after the patient is under general anesthesia, prepped and draped. Unless the surgeon saw the patient in the operating room prior to anesthesia and marked the operative site, it would not be possible for the surgeon to know that the correct site and patient are present.  It is not stated in the article how many of the operations reported had a timeout or the surgeon labeled the operative site but it is implied in the article that it was few. The first author of the manuscript, Philip Stahel, an orthopedic surgeon from the University of Colorado, explained the results stating that “many doctors resent the rules, even though orthopedists have a 25 percent chance of making a wrong-site error during their career….” Dr. John R. Clarke, a professor of surgery at Drexel University College of Medicine and clinical director of the Pennsylvania Patient Safety Authority, agreed stating, “There’s a big difference between hospitals that take care of patients and those that take care of doctors…The staff needs to believe the hospital will back them against even the biggest surgeon.”
4. Dr. Peter Pronost extends this sentiment by stating “Health care has far too little accountability for results. . . . All the pressures are on the side of production; that’s how you get paid.” He adds that increased pressure to turn over operating rooms quickly has trumped patient safety, increasing the chance of error.

I would offer some suggestions:

1. Focus should be on the operating room since this is where most of the wrong site or wrong patient procedures occur. I’m frustrated by the unnecessary timeouts that occur during bronchoscopy. For example, where the patient is known to me, enters the bronchoscopy suite awake and alert, and the biopsies are done under direct vision, fluoroscopic or CT guidance there is no real chance of wrong site or patient surgery. Similar procedures do not need a timeout. The Joint Commission needs to recognize this and stop its “one size fits all” approach.
2. What is needed is data. Right now it is unclear whether a timeout makes any difference. A scientific valid study of the timeout procedure is needed but not observational studies, designed only to create political statistics that a timeout works. The Joint Commission and other regulatory health care organizations need to break the habit of mandating interventions based on no or little evidence.
3. The Joint Commission mandates have apparently had little impact on reducing wrong site or patient operations. Making further mandates would seem to offer little hope. If, as Dr. Chassin believes, that time is the issue, adding more items to a checklist will not likely improve the problem and probably make it worse.
4. If time is the culprit in the operating room then simplifying the process as much as possible might be useful. I have been told of one operating room in Phoenix where a timeout is so extensive that it can take up to 30 minutes. Marking the site by the surgeon should be mandatory and a simplified, standardized checklist read and confirmed by the nurse, anesthesiologist and/or surgeon will hopefully simplify the timeout and enhance data collection.
5. I would agree with both Provost and Kizer that accountability needs to be present. However, Kizer’s idea of a Federal repository may be ineffectual at improving outcomes. Witness the National Practioner Databank which has done nothing to improve health care and blames only physicians for lapses in healthccare. It would seem that many of the physicians quoted above do the same, i.e., blame only the doctors. Dr. Chassin suggests a team approach to medicine, i.e., an operating room team. I agree but it seems inconsistent to refer to a team approach and only hold the physicians accountable. Instead, I would suggest a mandatory reporting system with a free, transparent and searchable data base available to everyone. This data bank should report not only the surgeon(s) but everyone else in the operating room. Hospitals also need to be identified so that they cannot deflect their accountability by blaming surgeons while emphasizing operating room turn around over patient safety. This means not only the hospital but the CEO or administrator needs to accept some responsibility. The CEO or administrator controls the finances and often touts their “accountability”. It is time to put some teeth to that claim. Such a transparent data base will not only allow patients to check on surgeons but also hospitals, nurses, and anesthesiologists. Furthermore, it will allow the healthcare providers to check on each other as well as substandard hospitals and their administrators.

Richard A. Robbins, M.D.

Editor, Southwest Journal of Pulmonary and Critical Care

References

1. Boodman SG. The pain of wrong site surgery. Washington Post. Published June 20, 2011. Available at URL http://www.washingtonpost.com/national/the-pain-of-wrong-site-surgery/2011/06/07/AGK3uLdH_story.html (accessed 6-21-11).
2. Stahel PF, Sabel AL, Victoroff MS, Varnell J, Lembitz A, Boyle DJ, Clarke TJ, Smith WR, Mehler PS. Wrong-site and wrong-patient procedures in the universal protocol era: analysis of a prospective database of physician self-reported occurrences. Arch Surg.2010;145:978-84
3. Singarajah C, Park K. A case of mislabeled identity. Southwest J Pulm Crit Care 2010;1:22-27.

The opinions expressed in this editorial are the opinions of the author and not necessarily the opinions of the Southwest Journal of Pulmonary and Critical Care or the Arizona Thoracic Society.

Guidelines, Recommendations and Improvement in Healthcare

“You will never understand bureaucracies until you understand that for bureaucrats procedure is everything and outcomes are nothing.”-Thomas Sowell

Reference as: Robbins RA, Thomas AR, Raschke RA. Guidelines, recommendations and improvement in healthcare. Southwest J Pulm Crit Care 2011;2:34-37. (Click here for PDF version)

In the February, 2011 Critical Care Journal Club two articles were reviewed that dealt with Infectious Disease Society of America (IDSA) guidelines (click here for Critical Care Journal Club). The first by Lee and Vielemeyer (1) reviewed the evidence basis for the 4218 IDSA recommendations and found that only 14% were based on Level 1 evidence (data from >1 properly randomized controlled trial). The graph summarizing the data in Figure 1 of the manuscript is exemplary in its capacity to communicate the weak evidence basis for many of the IDSA recommendations.

A second study by Kett et al. (2) examined the outcomes when the American Thoracic Society (ATS)/IDSA therapeutic guidelines for management of possible multidrug-resistant pneumonia were followed. The authors found a 14% difference in survival when the guidelines were followed, but surprisingly, the survival was better if the guidelines were not followed. Dr. Kett and colleagues are to be congratulated for their candor in reporting their retrospective analysis of empirical antibiotic regimens for patients at risk for multidrug-resistant pathogens. The ATS/IDSA guidelines (3) state that “combination therapy should be used if patients are likely to be infected with MDR pathogens (Level II or moderate evidence that comes from well designed, controlled trials without randomization…”. However, the ATS/IDSA guidelines go on to state, “No data have documented the superiority of this approach compared with monotherapy, except to enhance the likelihood of initially appropriate empiric therapy (Level I evidence…from well conducted, randomized controlled trials)” (4).

The problem comes with the interpretation and implementation of these and other guidelines. Some, usually inexperienced clinicians or nonclinicians, seem to believe that following any set of guidelines will enhance the “quality” of patient care. Not all guidelines or studies are created equally. Some are evidence-based, important, correct and likely to make a real difference. These usually come from professional societies and are authored by well-respected, experts in the field whose goal is improve patient outcomes. As suggested by Kett’s article even these guidelines may not be infallible. Other guidelines are not evidence-based, unimportant, incorrect and can border on the trivial. These are often authored by nonprofessional, nonexperts to create a “political statistic” (5) rather than improve patient care.

If some guidelines are bad, how can those be separated from the good? We suggest 5 traits of quality guidelines: 

1. The guideline’s authors are identified and are well-respected, experts in the field appropriate to the guideline.
2. The authors identify potential conflicts of interest.
3. The evidence is graded and supported by references to relevant scientific literature.
4. The guidelines state how they selected and reviewed the references on which the guidelines are based.
5. After completion, the guidelines are reviewed by a group of reasonably knowledgeable individuals (for example the IDSA Board of Directors) that can be identified and are willing to risk the reputation of themselves and their organization on the guidelines.

Even with the above safeguards guidelines may be non-evidence-based, unimportant, incorrect or trivial, and if so, implementation may be at best a waste of resources, or at worst harmful to patient care. We ask that guideline writing committees show restraint in authoring documents which are little more than their opinions. Not every medical question, especially the trivial and the unimportant, needs a guideline. Furthermore, we would ask an endorsement from professional organizations that only guidelines based on randomized clinical trials be given a strong recommendation. As pointed out by Lee and Vielemeyer (1) only 23% of the IDSA guidelines were supported by randomized trials while 37% of strong recommendations were supported only by opinion or descriptive studies.

IDSA states on their guidelines website, “It is important to realize that guidelines cannot always account for individual variation among patients. They are not intended to supplant physician judgment with respect to particular patients or special clinical situations. IDSA considers adherence to the guidelines listed below to be voluntary, with the ultimate determination regarding their application to be made by the physician in the light of each patient’s individual circumstances” (6). Despite this and other disclaimers, guidelines often take on a life onto themselves, frequently carrying the weight of law, regardless of the supporting evidence. We call for professional societies to end the practice of strongly recommending those guidelines based on opinion. Such practices have led and will continue to lead to systematic patient harm. Only those guidelines based on strong evidence should be given a strong recommendation. If the professional societies believe an opinion on a particular issue is appropriate despite a lack of evidence, a different designation such as recommendation or suggestion should be used to clearly separate it from a guideline.  The term guideline should be reserved for those statements that are evidence-based, important, and almost certainly correct and can make a real difference to patients.

Richard A Robbins MD, Allen R Thomas MD, and Robert A Raschke MD

References

1. Lee DH, Vielemeyer O. Analysis of overall level of evidence behind infectious diseases society of America practice guidelines. Arch Intern Med. 2011;171:18-22.
2. Kett DH, Cano E, Quartin AA, Mangino JE, Zervos MJ, Peyrani P, Cely CM, For KD, Scerpella EG, Ramirez JA. Implementation of guidelines for management of possible multidrug-resistant pneumonia in intensive care: an observational, multicentre cohort study.  Lancet Infect Dis 2011 Jan 19. [Epub ahead of print].
3. American Thoracic Society, Infectious Diseases Society of America. Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Respir Crit Care Med 2005;171:388–416.
4. Paul M, Benuri-Silbiger I, Soares-Weiser K, Liebovici L. Beta-Lactam monotherapy versus beta-lactam–aminoglycoside combination therapy for sepsis in immunocompetent patients: systematic review and metaanalysis of randomised trials. BMJ, doi:10.1136/bmj.38028.520995.63 (published March 2, 2004). Available at URL http://bmj.bmjjournals.com/cgi/reprint/bmj.38028.520995.63v1.pdf?ck_nck (accessed February 11, 2011).
5. Churchill, Winston. London, UK. 1945. as cited in The Life of Politics, 1968,  Henry Fairlie, Methuen, pp. 203-204.
6. Infectious Disease Society of American. Standards, Practice Guidelines, and Statements Developed and/or Endorsed by IDSA. Available at URL http://www.idsociety.org/content.aspx?id=9088 (accessed February 12, 2011).