Colors blind the eye. Sounds deafen the ear. Flavors numb the taste. Pursuit of result can weaken the mind. Goals can distort one’s behavior. The great engineer observes the situation but trusts the inner vision. The engineer rejects external and cultivates self. They allow goals to come and go.
It’s time to talk about metrics.
Managers want metrics, C-level folks want metrics, and there are tons of ways to generate metrics from your software development efforts. Velocity. Hour reporting. Created vs Resolved. CFD. Burn down. Burn up. Epic reports. Release reports. Deployment times. Mean times between failures. How many cars in the parking lot on Saturday.
It’s not a problem of being able to generate metrics. That can be done. It’s a problem of being able to figure out what metics to generate – which ones are noise, which ones are harmful, and which ones are actually useful.
Colors blind the eye.
Sounds deafen the ear.
Flavors numb the taste.
People like metrics. They want all the metrics. Problem is, it’s surprisingly easy to reach information overload.
Another interpretation of this passage calls out a number specifically – 5. “Five colors blind the eyes, 5 tones deafen the ear” etc. That’s probably a good rule of thumb, if you’re actively reporting on more than 5 metrics in one specific effort, you’re overloading your own processing power, and likely at the point where adding more data doesn’t add more value.
Your development process has a bottleneck somewhere (they all do). If you are working on improving anything other than the bottleneck, you’re wasting effort… and if the bottleneck requires more than five metrics in order to correctly describe it, you are using the wrong metrics.
Pursuit of result can weaken the mind.
Goals can distort one’s behavior.
So this is where Lao Tzu warns us about metrics not being just useless, but actually being harmful.
One of the things about metrics that often gets overlooked is the fact that they tend to both measure and influence how an organization behaves, directly or indirectly. Pursuit of meeting a goal weakens decision making, because it adds another factor into the decision. This can easily lead to local optimizations that make a particular metric look good, at the expense of the process as a whole.
A tier 1 help desk may be performing wonderfully, because they resolve their issues within a day of them coming in – by simply escalating everything that comes in to tier 2. Locally, the metric is great because every call is handled quickly. Over the larger picture, the organization is failing, because the customer’s problem isn’t handled in a timely fashion- but that issue isn’t noticed, because we set up our metrics localized, and the organization naturally optimized for that local measurement.
It’s important to set goals and metrics at a high level, encompassing the end to end process first. Then, lower metrics (and acceptance criteria) can be discovered that support the overall.
The great engineer observes the situation
but trusts the inner vision.
So we’ve learned that metrics can be useless, and worse, they can be actively harmful. That said, the world can still be observed. Metrics can’t always be trusted, but they can still provide insight. After observing the situation, though, the engineer needs to trust their inner feelings on how things are actually working. People say that metrics provide understanding of the situation – but they don’t. Thinking about what’s been learned and listening to self provides understanding.
The engineer rejects external and cultivates self.
Great engineers (and the best managers) don’t need metrics to know if things are going well or not. They can see how their team is doing simply by observing the situation, and trusting their inner vision. As such, there’s more value in exercising and improving the ability to understand the situation using the small indicators of daily activity than improving reported metrics.
I know that statement makes me unpopular with management, but it’s true.
I feel this self-cultivation, and “observation and inner vision” theme is one of the primary benefits of standup meetings. Great scrum masters can tell when there’s an issue to act on simply from the tone of voice of a team member, or other nuances. No metric will ever capture these, and even if it did, it would be a lagging metric. By cultivating self and trusting inner vision, the scrum master can take action immediately, instead of waiting to see effects of the issue downstream. This is far more powerful than any metric, even if it’s harder to identify.
They allow goals to come and go.
Metrics have a shelf life.
Like all of the named things, metics are (meta) manifestations created via the tao; and like all of the named things they will someday be reclaimed. At some point, the metrics you are using now will no longer be useful (or relevant) to the project – the current challenges will have been overcome, and new metrics will be needed to make progress on the next bottleneck. Let them go when it’s time for them to go, and accept the new ones to help you observe the situation when you need them.