Good Data or Bad Data: The Idiosyncratic Rater Effect.

Read Marcus’ article in HBR to learn how to solve for IRE

Performance management redesign. It may sound boring, but it’s actually everything. Every employee is affected by your performance management system. It determines how they get paid, promoted, evaluated. It touches everybody.

We all know this, and it’s why in recent years many companies have taken radical steps to improve their performance management. One bold step many companies have tried is getting rid of ratings altogether. Recently a CEB report indicated that companies who did so are doing worse than companies who have kept them. But the whole debate about ratings vs. no ratings is a complete red herring. The question isn’t ratings or no ratings. It’s do you want good data or bad data?

Ratings generate bad data. The reason why has been known for decades. It’s called the “idiosyncratic rater effect.” Big term, but it simply means that we human beings are unreliable raters of other human beings. All the data show that when I rate you, over 60% of your rating is about me, and not you. Which is a big problem, because we are paying, promoting, training, and deploying you based on that rating — which doesn’t reflect you. So ratings are a terrible performance measurement tool, because they claim to measure one thing but actually measure another.

So, by all means, throw out your current rating system. But don’t do it because people hate ratings. Do it because ratings are bad data. It comes down to a measurement question, and nobody seems to be trying to answer it: how do we reliably reveal the true range in performance in an organization so that we can invest differentially in our people? That’s the question to ask.