Is this Really How We Should Measure Quality?

By Evan Falchuk

The OSHA-ization of health care quality continues.

A research group and a consulting firm have been hired by the state of Massachusetts to head up a new initiative to publish cost and quality information on Massachusetts doctors.  But the quality measures they will use are the same old ones we have seen for a long time.  They mean very little to most patients, and even less to doctors as a measure of how good their work may be.

To understand what I mean, look at what is being measured.

For the category “Adult Diagnostic and Preventative Care,” there are only four quality measures.  They are:

  • rates of colorectal screening tests
  • the number of patients in an insured population who lowered their blood pressure in a given year
  • correct imaging test use for lower back pain
  • rates of use of a spirometry test for COPD

The good news is Massachusetts doctors do better than the national average on these measures.  The bad news is it’s hard to say what that means as far as how good any doctor is who is measured this way.

Maybe it’s better in women’s health.  There, the four quality measures are:

  • rates of breast cancer screening for women 40-69
  • rates of cervical cancer screening for women 21-64
  • rates of chlamydia screening for women 16-20
  • rates of chlamydia screening for women 21-25

Hmm.  So if I am a 30 year-old woman trying to figure out how good my doctor is, the only thing that is being measured is whether he does a cervical cancer screening on me or not.  How about pediatrics?

  • rates of well visits
  • correct antibiotic use for upper respiratory infections
  • follow-up with children starting medications for ADHD

I could go on, but there’s a pattern.  All of these “quality” measures are crunching medical billing data and styling it as a quality metric.  And so every metric is going to be focused on things that are easily measurable by a review of those bills.

But there’s a more disturbing pattern.  The information is simply not valuable to consumers.  Worse, I think it is deeply misleading.  A medical group that does chlamydia screenings on 100% of its patients may be good or bad – or it just may be smart enough to know that if they do the state of Massachusetts will rate them with five gold stars.  But consumers won’t be able to tell the difference. All they will know is that practice A is “high quality,” while practice B isn’t.  Some doctors are starting to sound the alarm about this.

And this is the larger point.  Our health care is organized in a way that systematically undervalues the thinking, processing and deciding aspects of medicine- the things that really matter to you when you’re a patient who is sick trying to get help.  Our system treats medicine as an assembly-line process amenable to assembly-line metrics.  But it’s not.

Doctors, like others in professions requiring judgment and reflection, need time to think, and ought to be judged by how well they do that. Since the leading cause of misdiagnosis is a failure of synthesis – a failure by the doctor to put together available information in a way that leads them to the right conclusion – our system ought to be built around helping make sure this happens each and every time.

So, instead of a web site where you could see how often a medical practice does chlamydia screenings, imagine you could find out how often doctors at a hospital got their patients the right diagnosis and treatment?  Now that would be a useful way to measure quality.

  • Pingback: Better Health » The 5 Best And Worst Recent Health Policy Quotes

  • Pingback: How Atul Gawande is Being Misunderstood « See First Blog

  • Pingback: Quality – I find a kindred spirit | DB’s Medical Rants

  • http://haldallmd.blogspot.com Hal Dall, MD

    Spot on! This type of nonsense is coming to a national health care system near you!

  • Pingback: Better Health » How Atul Gawande is Being Misunderstood

  • docanon

    If I read this correctly, you want a measure of quality that reflects diagnostic accuracy. So does every single health services researcher, government regulator, health system manager, and consumer advocate.

    The key thing you must understand is that nobody suggests that the currently-available publicly reported quality measures represent a complete capture of health care quality. They are quite limited in scope. This is not news.

    Quality measurement is already evolving in the direction you suggest. This will require considerable resources, but I believe these would be well-spent. I would love for patients to have a sense of all aspects of quality, including acute care, chronic care, diagnosis, patient experience, and costs.

    The real debate is over whether imperfect and incomplete quality measure reporting is better than no reporting at all. In other words, is there some kind of detrimental effect–on patient health–of public reporting on the currently available measures? For this to be the case, you’d have to make the case that reporting on available measures distracts from unmeasured aspects of quality, or that higher performance on the available measures is inherently negatively correlated with other aspects of quality. For example, you might argue that aggressive treatment of blood pressure will cause physicians to ignore other health problems…or to treat too aggressively. Or you might argue that quality is some kind of zero-sum game: if you’re good at one thing, you’re bad at something else.

    These are empirical questions, not conceptual ones. Therefore I would encourage you to review the existing literature on them. Though this literature is young (you’re really only looking at a few years of peer-reviewed articles), what you find might surprise you. In particular, check out Werner and Chang in JGIM 2008.

    Finally, I would encourage you to think more broadly about the appropriate locus of measuring quality. It is not always the doctor (those who stay fixed on this unit of analysis systematically undervalue other participants in care). As health care is delivered by a system, quality measures are often intended to system-level performance. So again, nobody (at least nobody intelligent, which excepts the Massachusetts GIC) is arguing that the existing quality measures should be reported at the individual-doctor level…or that performance on these measures will help a patient select the “best doctor” for him or her.

  • http://twitter.com/efalchuk Evan Falchuk

    docanon – thanks for the comment and for pointing me to Werner and Chang’s interesting study.

    I disagree that quality is a zero-sum game. Werner and Chang do a good job demonstrating this.

    They also show that time spent with the doctor was one of the important factors in quality care. And they suggest that the traditional quality metrics might be encouraging that they spend this time by requiring multiple follow up visits.

    In which case, I say, good.

    But I also say, if the time doctors spend with their patients is so empirically valuable, why don’t we just say so and set up our system that way?

    What I find most troubling is that these measures are being presented as consumer tools for finding good quality. They are not. To the extent they suggest to a consumer that practice A is better than practice B on an objective scale, they are deeply misleading.

    But more broadly, I am less interested in how people find the “best doctor,” as I am in the question of getting the right diagnosis and treatment.

    Measures of quality ought to be focused on these issues – how well doctors puzzle out these problems – than on whether they do the specific tasks these groups measure.

  • Pingback: Better Health » Healthcare Costs: Under-Thinking Leads To Over-Testing

  • Pingback: Healthcare Costs: Under-Thinking Leads To Over-Testing

  • Pingback: My Reaction to “Putting Patients First” « See First Blog

  • http://www.pillzmart.com Vigrx plus

    nice post

  • Pingback: Why Would You Pay More for Quality? « See First Blog

  • http://profiles.yahoo.com/u/LUNNKMP6WW3K2MJ4ZYJUHTQU2Y athikities supabiola

    If you have sex, it can be STD. STI symptoms are not always obvious. If you suspect that you have symptoms of STD, see a doctor. More details go to http://www.treatmentforgonorrhea.com/

  • "Medicine is learned by the bedside and not in the class room. Let not your conception of manifestations of disease come from work heard in the lecture room or read from the book: see and then research, compare and control. But see first."
    - Sir William Osler, MD
    The Father of Modern Medicine
  • Connect



    BestDoctors.com


    On Twitter


    On Facebook


    Via RSS


    On YouTube


    Subscribe via Email

  • Benefits Package

    BenefitsPackageButton

    Join the best in employer health-benefits blogging!

  • Follow Us on YouTube:

  • Recent Posts

  • Recent Comments

  • Categories

  • Archives