Data Fluency Series: Case Study

This is the last episode in a new series on Data Fluency. You can also watch the first episode, the second episode, the third episode, or the fourth episode.  

Let’s show you what good data looks like, and what we went through to get it. We wanted to see what happens on the world’s most productive teams. Going into it, we knew that it is possible to measure performance on teams – you can reliably measure it and you do get variation, because some teams perform much better than other teams, and some teams perform much lower. We wanted to know why, and if you could use questions to predict performance. You will get paid, promoted, and fired based on bad data. Click To Tweet

In essence, we wanted to learn what conditions exist on the most productive teams. It would tell us a lot about employee engagement and leader effectiveness – but the question remained, how can you measure the feelings that the high-performing teams had, but that the low-performing teams didn’t?

Here’s how we started: We asked question, after question, after question to the high-performers and the low-performers. These had to be questions about the individual’s own intentions and experiences (to avoid the Idiosyncratic Rater Effect), so none of them asked the individual to rate someone else on anything.

Secondly, all of the questions had to use extreme language to create variation. Words like “every day” or “great confidence” or “surrounded” or “always.” Using these extreme words meant that we got variation in our answers, with the high-performers choosing “Agree” or “Strongly Agree” across the board, and the low-performers choosing “Disagree” or “Strongly Disagree.”

Lastly, we needed these questions to predict positive outcomes in the future. If someone answered “Agree” or “Strongly Agree” to a question, they needed to have higher performance or retention down the road.

When we did that, we ended up with these 8 questions.

Every question involves the individual rating his or her own experience and intentions. Every question produces range. And every one of them exists in the survey because it predicts future performance. You can predict performance with 8 questions. Click To Tweet

That’s what good data looks like, and that’s what you need to go through to get it. It will take about ten years to have confidence in the data, but if your company is making talent decisions – how to pay you, who to fire, when to promote – having data that actually validly predicts future performance is more than helpful – it’s necessary.

These eight questions are used in the Engagement Pulse, a part of StandOut – the technology solution to Talent Activation.  To learn more, go to www.TMBC.com or click here

One Comment

  1. Helga February 12, 2020 at 11:53 PM - Reply

    Hi Marcus and Team, I’m Helga from Hungary, huge fan and follower. I love the videos, each and every time I feel I learn something new and revolutionary, so keep them coming.

    I have a question related the criterion-related validity part of this research. If I got that part right, you said that when selecting these 8 questions, you measured if the “Strongly Agree” respondents demonstrated the following consequent bahaviour: stayed longer, had higher productivity, were better at customer service. My question is: have you selected these three because they are the ones that have the strongest correlation revenue and/or profitability (or something else?)? And how were these three measured? My biggest frustration is that I don’t know how to measure employee productivity in the first place (their overall contribution to revenue/profit), or customer experience for that matter, EFFECTIVELY (reliably, validly, with variation), in a complex environment where you can’t count what people produce. Any tips for that?

    Or, you’re saying that based on this research, if we measure these 8 questions, and they get higher, the revenue or profit is going to get higher, and no need to measure productivity, retention, customer experience (because those are just the “middlemen” for higher profitability or revenue)?

    I’m really curious about the deeper details of this research – Is there a detailed report somewhere about the methodology, and the results of this research that I can dig into? (Eg. all the other questions that didn’t make it to the final 8).

    Not sure I phrased my question right (not a native speaker) so let me know if you need further clarification.

Leave A Comment