Improving Quality in Healthcare

Figure 1. Dr. Katz is a little jaded about quality metrics (1).

Everyone is in favor of quality healthcare and improving it. However, to date, initially highly touted quality measures prove to be meaningless metrics in about 5-10 years. That is, when the measures are scientifically studied, they are found to be of little worth. The cycle is then repeated, i.e., new and highly touted measures are again selected and found to be useless in 5-10 years. The latest in this cycle may be the Centers for Medicare and Medicaid’s (CMS) Merit-based Incentive Payment System (MIPS). The theory underlying MIPS has been that paying for quality rather than quantity will incentivize healthcare providers to improve quality. As part of the deal creating the Affordable Care Act (Obamacare) MIPS was established as a pay for performance system which promised to improve healthcare while reducing costs. However, healthcare costs have continued to rise (2). Data on improvement in quality has been lacking.

Now, Bond et al. (3) have reported a study suggesting that MIPS incentivization of quality improvement in healthcare quality has questionable benefits. Among US primary care physicians in 2019, MIPS scores were inconsistently associated with performance on process and outcome measures. Bond’s study included 3.4 million patients attributed to 80,246 primary care physicians. Physicians were divided into thirds based on their MIPS score. Compared with physicians with high MIPS scores, physicians with the lowest MIPS scores had significantly worse mean performance on 3 of 5 process measures: diabetic eye examinations, diabetic HbA1c screening and mammography screening, but significantly better mean performance on rates of influenza vaccination and tobacco screening. MIPS scores were inconsistently associated with risk-adjusted patient outcomes: compared with physicians with the highest MIPS scores, physicians with the lowest MIPS scores had significantly better mean performance on emergency department visits per 1000 patients but worse performance on all-cause hospitalizations, and did not have significantly different performance on 4 ambulatory care-sensitive admission outcomes. Nineteen percent of physicians with the lowest MIPS scores had composite outcomes performance in the top quintile, while 21% of physicians with the highest MIPS scores had outcomes in the bottom quintile. These findings suggest that the MIPS program may be ineffective at measuring and incentivizing quality improvement among US physicians.

It is unclear why improvement  in intermediate surrogate markers is used rather than improvement in outcomes. Bond’s study measured MIPS scores against ER visits and hospitalizations. Patients, providers, insurers, bureaucrats, politicians, taxpayers- in other words, nearly everyone- would agree that reductions in ER visits and hospitalizations is desirable if it can be accomplished without patient harm. Similarly, reduction in unexpected deaths and improvement in patients’ feeling of well being are goals that all can support. However, the goals of healthcare are different depending on which population is asked. Patients might support their well-being, insurance cost, and provider access as being most important, whereas payors might support costs as most important. Providers might support efficiency of care and reimbursement as important. So ultimately what surrogate markers like MIPS do is choose one point of view which often does not affect outcomes (4).

There are many ways to achieve a goal depending on expertise, resources and patient characteristics. Flexibility in care allows the person most likely to understand the efficiencies of their particular system- the providers- to use their local knowledge to benefit the patients. Outside influences emphasizing surrogate markers, cost, or politics have historically failed. Unless one is willing to accept healthcare shown not to benefit patients as acceptable, MIPS should be eliminated. Replacing MIPS with an equally flawed system set of surrogate markers will likely not help.

It seems that outcome measures offer several advantages over process measures. Outcome measures include unexpected mortality, hospital readmissions, safety of care, effectiveness of care, timeliness of care, efficiency of care, and patient well-being (5). These are all thought to be important by patients, insurers, providers and even politicians. In my view, the process leading to these ultimate outcome goals is less important and the process producing the same or similar results will likely vary between providers and hospitals.

CMS should refocus their quality efforts on outcomes rather than processes which have failed as quality indicators. Physicians must decide whether they wish to continue participation in systems such as MIPS and the accompanying increase in paperwork. Unless something changes the trends of increasing paperwork over meaningless metrics will continue.

Richard A. Robbins MD

Editor, SWJPCCS

References

1. Lehmann C. Comics for Docs: Medical Cartoons Poke Fun at Today's Practices. Medscape. July 15, 2022. Available at: https://www.medscape.com/slideshow/medical-cartoons-6015473#2 (accessed (1/12/23).
2. Kurani N, Ortaliza J, Wager E, Fox L, Amin K. How Has U.S. Spending on Healthcare Changed Over Time? Peterson-KFF Health System Trasecker. February 25, 2022. Available at: https://www.healthsystemtracker.org/chart-collection/u-s-spending-healthcare-changed- time/#Total%20national%20health%20expenditures,%20US%20$%20Billions,%201970-2020 (Accessed 1/4/23).
3. Bond AM, Schpero WL, Casalino LP, Zhang M, Khullar D. Association Between Individual Primary Care Physician Merit-based Incentive Payment System Score and Measures of Process and Patient Outcomes. JAMA. 2022 Dec 6;328(21):2136-2146. [CrossRef] [PubMed]
4. Robbins RA, Thomas AR, Raschke RA. Guidelines, recommendations and improvement in healthcare. Southwest J Pulm Crit Care. 2011;2:34-37.
5. Tinker A. The Top Seven Healthcare Outcome Measures and Three Measurement Essentials. Health Catalyst. June 29, 2022. Available at: https://www.healthcatalyst.com/insights/top-7-healthcare-outcome-measures (accessed 1/5/23).

Cite as: Robbins RA. Improving Quality in Healthcare. Southwest J Pulm Crit Care Sleep. 2023;26(1):8-10. doi: https://doi.org/10.13175/swjpccs002-23 PDF

IOM Releases Report on Graduate Medical Education

On July 29 the Institute of Medicine (IOM) released a report on graduate medical education (GME) (1). This is the residency training that doctors complete after finishing medical school. This training is funded by about $15 billion annually from the Federal government with most of the monies coming from the Center for Medicare and Medicaid Services (CMS). The report calls for an end to providing the money directly to the teaching hospitals and to dramatically alter the way the funds are paid. Instead payments would be made to community clinics phased in over about 10 years. To administer the program, the report recommends the formation of two committees: 1. A GME Policy Council in the Office of the Secretary of the U.S. Department of Health; and 2. A GME Center within the Centers for Medicare & Medicaid Services to manage the operational aspects of GME CMS funding. The later committee would administer two funds: 1. A GME Operational Fund to distribute ongoing support for residency training positions that are currently approved and funded; and 2. A GME Transformation Fund to finance initiatives to develop and evaluate innovative GME programs, to determine and validate appropriate GME performance measures, to pilot alternative GME payment methods, and to award new Medicare-funded GME training positions in priority disciplines and geographic areas.

If adopted, the plan would end decades of attempts by CMS to coerce medical school graduates into primary care, especially in rural, underserved areas. By controlling funding for GME training, CMS would be able to dictate how physician training. Negative reaction was expected and swift from the American Hospital Association, the American Medical Association and the American Council on Graduate Medical Education, whose members would lose CMS money (2-4). Also expected, the proposal was supported by the American Academy of Family Physicians whose members who would gain under the proposal (5).

The IOM committee has a point. Despite a growing public investment in GME, there are persistent problems with uneven geographic distribution of physicians, too many specialists, not enough primary care providers, and a lack of cultural diversity in the physician workforce. Furthermore, according to the report "a variety of surveys indicate that recently trained physicians in some specialties cannot perform simple procedures often required in office-based practice.”

However, can a committee formed by CMS be expected to improve the health of America? Based on the composition of the committee and their past performance we think not. First, the committee was co-chaired by Don Berwick who was head of the Institute for Healthcare Improvement (IHI), CMS Administrator and presently a candidate for Massachusetts governor (6). During Berwick's tenure, the IHI proposed a number of non- or weakly evidence-based metrics. Many of these have been found to make no impact on patient-centered outcomes such as mortality, length of stay, readmission rates, morbidity, etc. (7). An example was the 18 month 100,000 Lives Campaign which according to Berwick prevented 122,300 avoidable deaths. However, the methodology, incomplete data and sloppy estimation of the number of deaths makes Berwick's claim dubious. Furthermore, when the campaign was expanded to the 5,000,000 Lives Campaign the "results" could not be reproduced. Also during Berwick's tenure, IHI prematurely championed tight control of blood sugar in the ICU, an intervention which resulted in a 14% increase in ICU mortality when properly studied (8). Undaunted, Berwick put many of these same meaningless metrics in place when he became administrator of CMS. One of these metrics, readmission rates, has been associated with a higher mortality (9). Now Berwick is running for Massachusetts governor. One wonders how politics might have affected the report.

Other members of the committee include the committee co-chair, Gail Wilensky, who was administrator of HCFA (the precursor of CMS), nurses, physician assistants, economists, a representative from industry and a number of academics. Missing were members of the large community of practicing physicians. It seems the IOM committee was assembled to produce a political rather than an evidence-based answer of how to solve patient care disparities. To paraphrase a well-known quote, the first casualty of politics is usually the truth. It seems likely that the proposed GME Center within CMS would have a similar composition to Berwick's present IOM committee and would likely offer political rhetoric rather than meaningful reform to GME. Similarly to those championed by Berwick at IHI and later CMS, we suspect that a series of meaningless metrics would be required that would do nothing other than add a paper burden to a medical system already drowning in paperwork. By removing local control, CMS will likely ignore local strengths. For example, the University of Colorado has an extremely strong pulmonary and critical care division. Although America needs this physician expertise, especially critical care, it seems likely that CMS might move these residency slots to family practice or general medicine. We believe that local control with appropriate incentives, is more likely to solve these problems than a centralized bureaucracy in Washington.

Lastly, a word about the report's claim graduates lack the skills to perform basic procedures. Our observations are similar and we are inclined to accept the claim. However, we point out that it was decisions of committees such as those proposed that required attending physicians to perform procedures in order to be reimbursed and that residents have fewer opportunities to perform procedures due to work hour restrictions. The committee's implication that somehow physician trainers are to blame seems quite disingenuous. Not identified in the report but crucial to physician development is developing skills to critically evaluate medical literature, rather than blindly follow the guidelines proposed by CMS, IHI or others of a similar ilk. 

The proposals in the IOM report are a bad idea from a committee whose head has been rife with bad ideas. The committee's report is not the "New Flexner Report" but will be the coffin nail in the death of quality, caring physicians if adopted.

Richard A. Robbins, MD

Clement U. Singarajah, MD

Phoenix Pulmonary and Critical Care Research and Education Foundation

Gilbert, AZ

References

1. Institute of Medicine. Graduate medical education that meets the nation's health needs. July 29, 2014. Available at: http://www.iom.edu/Reports/2014/Graduate-Medical-Education-That-Meets-the-Nations-Health-Needs.aspx (accessed 8/5/14).
2. American Hospital Association. IOM panel recommends new financing system for physician training. July 29, 2014. Available at: http://www.ahanews.com/ahanews/jsp/display.jsp?dcrpath=AHANEWS/AHANewsNowArticle/data/ann_072914_IOM&domain=AHANEWS (accessed 8/5/14).
3. Hoven AD. AMA urges continued support for adequate graduate medical education funding to meet future physician workforce needs. July 29, 2014. Available at: http://www.ama-assn.org/ama/pub/news/news/2014/2014-07-29-support-graduate-medical-education-funding.page (accessed 8/5/14).
4. Kirch DG. IOM’s vision of GME will not meet real-world patient needs. July 29, 2014. Available at: https://www.aamc.org/newsroom/newsreleases/381882/07292014.html (accessed 8/5/14).
5. Blackwelder R. Recommended GME overhaul will support a physician workforce to meet nation’s evolving health needs. July 29, 2014. Available at: http://www.aafp.org/media-center/releases-statements/all/2014/gme-physician-workforce.html (accessed 8/5/14).
6. About Don. Available at: http://www.berwickforgovernor.com/about-don (accessed 8/5/14).
7. Robbins RA. The unfulfilled promise of the quality movement. Southwest J Pulm Crit Care. 2014;8(1):50-63. [CrossRef]
8. NICE-SUGAR Study Investigators. Intensive versus conventional insulin therapy in critically ill patients. N Engl J Med 2009;360:1283-97. [CrossRef] [PubMed]

Reference as: Robbins RA, Singarajah CU. IOM releases report on graduate medical education. Southwest J Pulm Crit Care. 2014;9(2):123-5. doi: http://dx.doi.org/10.13175/swjpcc107-14 PDF