Reports and quality
When the metric rates people
The moment a number rates someone, it stops measuring what it measured. Not through bad faith: by design.
A dashboard showing how many files close on time is useful while it serves to see where work gets stuck. The day that same number enters the appraisal of whoever runs those files, it starts to move — and not because the work improved.
What happens to each metric once it rates people
| Metric | How it is «improved» without improving anything |
|---|---|
| Files closed on time | They are closed before they are complete, then reopened |
| Incidents logged | Fewer get logged: recording one penalises whoever records it |
| Average response time | A quick «we are looking into it» that answers nothing |
| Documents verified | The easy ones get verified and the doubtful ones parked |
Important
The second row is the dangerous one and the least anticipated: **when logging a problem hurts whoever logs it, problems stop being logged**. The dashboard improves, reality does not change, and you lose the very information that fixed things. It is the one case where a metric can leave you worse off than measuring nothing at all, because it also breeds confidence.
How to measure without breaking the measure
Watch out
There is a tell that exposes the problem before results show it: **a metric that improves without anyone having changed anything**. If the percentage climbs one month and nobody can explain what was done differently, what usually changed is how things get recorded, not the work. It is worth ten minutes in the detail: those ten minutes decide whether the dashboard is still good for anything.
What is worth looking at per person
- 1
Load, not performance
How many files each carries; useful for sharing work out.
- 2
Blockers outside their control
Anything waiting on an outside answer is nobody's fault inside.
- 3
And what repeats in one specific place
If a step always jams with the same person, it is usually the step.
Worth knowing
If the metric will feed an appraisal or pay, there are employment and data protection implications that depend on your country and your collective agreement. **Check that with your adviser before switching it on**, not after.
›So people are not measured?
They are accompanied. An isolated figure without conversation is always read wrong.
›How do I know this is already happening?
Look for a metric that improves without anyone knowing why.
›What if management asks for a ranking?
Give the aggregate and explain the effect. Showing it once usually suffices.
A real case
The situation
A company adds «incidents logged» to the quality team's monthly appraisal.
What you do
- Takes it out of the appraisal and keeps the metric on the process dashboard only
What you get
Small incidents get logged again, and those are the ones that warn before the big one.
The situation
Files closed per person starts being measured and the team starts closing things half done.
What you do
- Also measures the quality of what is closed, not just the count
- Checks how many get reopened afterwards
- Shares the criteria before starting to measure
What you get
The indicator stops rewarding speed and starts describing the real work.
The situation
The indicator compares whoever handles easy suppliers with whoever handles hard ones.
What you do
- Separates by file type before comparing
What you get
The comparison is honest.
The situation
The number is used in appraisals without ever being explained.
What you do
- Explains what it measures and what it does not before using it
What you get
Nobody first learns how they are measured at their appraisal.
The situation
One person scores badly and they were the only one covering absences.
What you do
- Checks the detail before drawing conclusions
What you get
Context is taken into account.
The situation
The indicator improves because incidents stopped being recorded.
What you do
- Checks the record is still complete
What you get
The improvement is real rather than one of recording.
The situation
People are measured when the problem is the process.
What you do
- Checks whether the bottleneck is one person's or everyone's
What you get
The process is fixed rather than people pressed.
The situation
The team stops reporting problems to avoid scoring badly.
What you do
- Separates the management indicator from the appraisal one
What you get
Problems keep surfacing while they can still be fixed.
This article answers
- metrics to rate the team
- the dashboard does not reflect reality
- per-person quality targets
- how to stop numbers being massaged