Yes, Quality Metrics Get Gamed

I focus a lot on the importance of getting people simple, relevant quality information to help them compare providers and insurers. But I realize this information will not be perfect. Let me give you an example as it relates to provider quality rankings.

Say there is a hospital that is doing pretty well on the Vizient rankings. They are near the top, but their mortality rate kept them from being the top spot. When the providers at this hospital have their meetings about what should be done to improve their mortality rating, they have two options for how to do that:

1. Do extensive research into all the patients who died in the hospital in the last year and identify the variety of factors that may have contributed to those deaths, then launch initiatives and processes to eliminate as many of those contributors as possible. There will need to be a lot of research and implementation time spent on this, plus they will need to get buy-in (not easy) to get people to change how care is delivered, or they will possibly need to make changes to their EMR (even harder) to help protect against issues that were identified. The end result will likely be a modest reduction in some of those factors, which may or may not result in actual measurable differences in mortality the next year.
2. Look at the billing codes providers are putting into the EMR and talk to them about making sure to get all those key codes in place (don’t forget acute metabolic encephalopathy, protein-calorie malnutrition, type II diabetes with complications, or hypokalemia!) so that their overall patient acuity is higher (i.e., more accurately reflects how sick their patients are), so then when their risk adjustment is done the next year to calculate their mortality rating, they will have a big improvement and snag that top spot.

You can see by how I described each option which one is the low-hanging fruit and is also much more likely to result in an improved ranking.

This is not just a theoretical issue–I see it first hand all the time. It happens everywhere. The sad thing to me is not that people who work in hospitals are trying to make their numbers look as good as possible, but that those conversations take up so much meeting time that it crowds out any discussion on how to actually improve the safety and quality of care for patients.

So, yes, quality metrics are not perfect. They have their issues. But I do feel that they generally are successful at revealing large differences in quality between different providers. And that is probably enough to motivate patients to avoid lower-quality options and shift market share to reward higher-value providers.