UsableMarkets

My wife recently completed a transaction on eBay where she was none to happy with the seller. While no fraud occurred the seller was, shall we say, not polite. I won’t recount the sordid details, but suffice to say that the transaction was an extremely unpleasant experience for her.

But then, as part of concluding the transaction, my wife felt compelled to indicate to the seller that she would leave her positive feedback, which astonished me. Why rate someone positively who does not deserve it?

The answer indicates one of the serious problems rating systems have when applied to other people. In short: People don’t like it when you rate them badly, regardless if they deserve it or not.

Rating movies is one thing. A movie, after all, can’t accuse you of slander. But people are a bit more sensitive, and so on eBay a culture has developed where sellers expect positive feedback despite the quality of their service. And if you don’t give it to them, they’ll rate you badly. So some people (including my wife) feel compelled to always rate everyone highly, despite the service they’ve received, merely to protect their own rating.

This “defensive rating” of other people completely defeats the purpose of rating on eBay. Now, if someone has a positive rating, it’s possible that it doesn’t really mean anything.

Doctors hate ratings, too
If you think this problem is limited to eBay, you would be wrong. A crucial piece of “consumer driven” healthcare is that doctors will be rated and ranked so that consumers can better select a doctor. Needless to say this is not something doctors are taking lying down.

One recent post from the WSJ’s Health Blog should make the point: Doctors to Patients: Don’t Slam Us Online Without Permission. I quote:

Next time you go to the doctor, look for a new form buried in the stack of insurance and health-history paperwork you’re asked to complete. You might find a contract that would require you ask your doctor for permission to grade him or her online.

Pandora’s (Rating) Box
But it’s unlikely that rating systems, despite resistenance and abuse, will go away. And the reason for this is that rating systems – about humans or things – provide a (sometimes) meaningful data point to consumers. And now that we consumers are used to it, it is not something we even want to go away. So rating systems are now as much a part of the internet as blogging (another dubious innovation).

While there is no easy solution for rating people fairly, application designers can do two things to help. One: try and imagine the different ways that the system can be abused, and create a system that filters out the “noise.” And two: design them in such a way that consumers can easily understand the strengths and weaknesses of that particular system.

The world’s oldest rating system

As always, thanks for listening.
~alex