Goodhart’s Law on Software Development
As humans, we need to measure things to make them meaningful. The only way we can set our goals based on these metrics. We tend to measure everything we can, length of the roads, heaviness of goods, temperature of the weather, pressure, even time. On the base of our measuring systems we got a good, understandable & imaginable pivot metric and we compare other things with them, we know how long a meter is so we can say that 10cm is smaller and 100 km is lots of meters. When you get a taxi cab and ask the driver to get you to the place B, you can say that how much will it cost, how much time is needed nearly.
On the software projects, you get in the cab, ask the driver if she can get you to place B, if yes how much will it cost, the answer is like “Fuck you mate, how can anyone know that? It may cost 50$ or 250$, I guess… On what time? Geez, it depends a lot!”
It’s because software development is still a craft instead of a one-way process, we carry both quality & security concerns, readability of the code, (re-)usability, managing the technical debt… And when all of these are combined, it’s hard to tell a measurable pivot.
Software development companies tried SLOC, man-hours on projects but these are also too weak to light, now many organizations experiment the Agile processes and use story points (velocity) to tell a bit about the future. Story points are based on comparing tasks with others and give them a roughly size. The sizing is done by the team itself.
Also many organizations confuse the velocity as a target of the productivity. If you set a goal on the SLOC you’ll get more lines of code, which will end up unreadable, unusable code. If you set a goal on man-hour(s) you may end with high technical debt. And if you set a goal on velocity, you can be sure on one thing. Your teams will get higher velocity. I mean they can easily set their productivity to x2 by doing nothing more.
The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.
Campbell’s Law
Another applicable strategy on setting goals is to set multiple goals instead of one. With this kind of strategy, managers try to make it hard to corrupt on the metrics. The business value’s delivery time can be one of it, number of the bugs customers’ faced can be another, code coverage, escaped defects and so on.
But what I believe is, if you set a system against people, they will eventually come over on it, soon or later. You’ll need it to make more and more complex and there will be always backdoors and tricks.
The third way is the conventional, human discretion methodology. The managers asks questions to understand what is actually going on and using the metrics as a lighter knowing they might not be flash all of the road but helping tools.
And the fourth way of managing software processes is the OKRs (Objectives and Key Results). When an objective is defined it comes with it’s key results for each department of organization and each Key Result can be identified/measured with different metrics. With this kind of methodology companies try to take over on Goodhart’s Law by changing the metrics regularly (on every quarter) based on the objectives.
The OKR methodology and it’s details may be another blog post, till then see you later :) Thanks for reading.
