CMS’ Star Ratings Miscalculated

Modern Healthcare is reporting that the Centers for Medicare and Medicaid Services (CMS) has miscalculated hospitals star ratings since they were first released in 2016 (1). Officials at Rush University Medical Center in Chicago exclusively disclosed their analysis and correspondence to Modern Healthcare. The investigators found that instead of evenly weighting the eight measures in the safety of care group, the CMS' star ratings formula relied heavily on one measure— The Patient Safety and Adverse Events Composite, known as PSI 90 —for the first four releases of the ratings and then complication rates from hip and knee replacements for the latest release. The single measure accounted for about 98% of a hospital's performance in the safety group, according to Rush's analysis. The safety group can also greatly influence a hospital's overall star rating, the analysis concluded. Rush's findings likely prompted the CMS to announce this week that it would postpone the July release of its star ratings (1).

The statistical model the CMS uses likely caused the miscalculation. The model, called latent variable modeling, uses scores for seven groups of measures to calculate the star ratings:

1. Mortality
2. Safety of Care
3. Readmission
4. Patient Experience
5. Effectiveness of Care
6. Timeliness of Care
7. Efficient Use of Medical Imaging

The three outcome groups—mortality, safety and readmissions—are each weighted the most at 22% each. Measures within each group are supposed to be evenly weighted to calculate the hospital's performance in that area. Rush's analysis found that the weight given to the PSI-90 measure was much greater than the seven other measures in the safety group. Specifically, PSI-90 was weighted 1,010 times stronger than the catheter-associated urinary tract infections measure, 81 times stronger than the C. difficile infection rates measure, 51 times stronger than the central line-associated bloodstream infection rates measure and 20 times stronger than either the surgical site infection rate measure.

Latent variable modeling changes the weighting and is inappropriate for measuring clinical outcomes, said David Levine, senior vice president of advanced analytics and informatics at Vizient (1). "Given the disproportionate weighting of the safety scores over time, they did not represent a composite measure," said Dr. Omar Lateef, an author of the analysis and Rush's senior vice president and chief medical officer (1). Lateef said he and his colleagues at Rush were alarmed by a rating drop from 5 to 3 stars because they have improved performance on five of the eight safety measures since the December release. " Lateef added that although CMS was initially dismissive of Rush’s concerns that CMS has come around since presented with Rush’s analysis.

CMS announced earlier this week that it was delaying release of the star ratings "to address stakeholders concerns." No date has been set for when the new ratings will be released.

Richard A. Robbins, MD

Editor, SWJPCC

Reference

1. Maria Castellucci M. CMS star rating system has been wrong for two years, health system finds. Modern Healthcare. June 15, 2018. Available at: http://www.modernhealthcare.com/article/20180615/TRANSFORMATION01/180619933?utm_source=modernhealthcare&utm_medium=email&utm_content=20180615-TRANSFORMATION01-180619933&utm_campaign=am (accessed 6/15/18).

Cite as: Robbins RA. CMS' star ratings miscalculated. Southwest J Pulm Crit Care. 2018;16(6):338-9. doi: https://doi.org/10.13175/swjpcc078-18 PDF