Wednesday, May 26, 2010

Physician Ratings vs. Healthcare Narratives

An interesting study came out last week which aligns with some of my thinking and thoughts on individual physician ratings:
• The overall activity of ratings is pretty low (e.g. there are not many ratings being posted by patients). I think a major part of that is because consumers are not really confident in how to rate doctors since (1) there is a fair amount of subjectivity which makes it hard to put into a "standardized rating" and (2) they don't do it often. In other words, consumers might feel they can be experts on restaurants and movies since they have a lot of experience at those venues AND there are often obvious metrics. Meanwhile, most people just see a doctor 2-3 times in a year (and then it's just for 15 minutes), they usually just see 1-2 doctors (so not a lot of comparisons), AND the experience each time might be different depending on a variety of factors (e.g. how sick they are, the type of syndrome they have, how they respond to the therapy, etc.). In other words, most patient's actual experience with the doctors is often short and incomplete (at least with respect to fully rating them).
• Most of the ratings are positive… which makes sense since that is often human nature. But at least this dispels the myth that only "haters" would post! However, it should be pointed out that it often only takes 1 bad review to sour most people, unless there are over 5-10 good ones also present. A small amount of reviews can be easily biased by one bad one.

The LA times even did an article about this study: Physician rating websites mainly sing doctors' praises, study finds. It is a good story, although I think they used this study to too quickly dismiss these sites. Rather, I would use this study to explain that:
1. Physician rating sites are still in their early phases and the jury is still out on how important they will be.
2. For a true objective rating of doctors, we ideally need much greater numbers of consumers doing these ratings, and ideally in a more consistent manner (e.g. randomly poll patients, otherwise you will mostly get the most extreme examples on either end to post a review).
3. There needs to be a way for physicians to respond, particularly to negative posts (although without violating patient confidentiality). For example, I am impressed that now does allow for that function.

With that said, I think there is also a very important role for more "narrative" reviews of healthcare as well. However, that is likely to be best done at the systemic level (e.g. total healthcare experience across an enterprise), rather than geared towards a single physician. For example, I recently heard about a website called Patient Opinion . This is a non-profit organization in the United Kingdom founded by Paul Hodgkin, a GP who wanted to make the wisdom and insights of patients, available to the NHS (National Health Service). He states "the old ways of doing this – inviting a patient to sit on a working party or carrying out a survey – did not work very well", so he devised Patient Opinion as a way for thousands of patients to both share their own experience and gain support from others. The results are story after story about the good and bad in the system. And hospitals and other entities actually read and respond to these stories. They have found that these stories have more power and information than any objective rating scale could ever provide - and that they are often able to fix systemic problems based upon these narratives. Wow - that's a very powerful thought...


  1. Hi Lyle
    I'm one of the team running Patient Opinion in the UK - so thanks for the mention!
    Our experience is much as you say. Ratings are fine - but they tell you little about what to fix or how to fix it.
    Our focus is on changing, not choosing. We think that patients give feedback to healthcare providers because they want things to be better for the next guy. People will often say: "everone was lovely, but here was just this one thing..."
    So, in that one story, they tell you what to fix - and often, how to fix it too. And some stories are so compelling that they also provide a powerful motivation for healthcare staff to act differently.
    We have over 100 health care organisations now responding to feedback online, and often saying what they are doing to improve their service. We recently added a way for the organisation to show a "we changed" post-it note to their response when they make an improvement.
    So soon, we'll be able to show you who's listening, who's improving - and who isn't.

  2. Lyle,
    As a fellow blogger, I have struggled with the physician rating issue. My most charitable treatment of the topic involved needing three, second opinions regarding my daughter's surgery.

    The story is here:

    If you believe in evidence-based medicine which I suspect we both do, half of those four opinions were badly under-informed.

    Were they simply ignorant, motivated by economics, medicolegal concerns, conservative tendencies to do as little as possible, liberal tendencies to be adequately aggressive, or simply having a bad day? As a consumer, with n of one data, it was difficult to say.

    Perhaps only agencies {provider, regulatory, payer, etc} are the only appropriate parties to cast physician ratings, by virtue of the larger view. What do you think?