When Measurement Becomes the Job — Escaping Goodhart's Law

A metric is useful when it helps you make a better decision. It becomes harmful when people optimize the number instead of the work it was meant to represent.

That is the practical problem behind Goodhart's Law. In his 1975 paper, later published by Springer as "Problems of Monetary Management: The U.K. Experience", Charles Goodhart wrote about statistical regularities breaking down when they are used for control. The familiar line, "when a measure becomes a target, it ceases to be a good measure," is a useful paraphrase, not a direct quote. For a lighter introduction to the law itself, see Wikipedia's Goodhart's law page.

You do not need a data team or a new dashboard to avoid this problem. Start with a shared spreadsheet or the tracker you already use. Choose three things: one decision you need to make, one outcome signal that tells you whether it helped, and one guardrail that tells you when to stop.

Separate outcomes, proxies, and targets

An outcome is the result you care about: customers get correct answers, releases are dependable, or a support team resolves requests without unnecessary delay.

A proxy is an observable signal that may help you judge that outcome. First-response time, reopened tickets, deployment frequency, or time spent reviewing a draft can all be useful proxies.

A target is what happens when you attach a threshold, reward, deadline, or management pressure to the proxy.

The distinction matters because proxies are incomplete. If you set a target of closing more tickets, people can close tickets quickly and leave customers to reopen them. If you reward story points completed, a team can split work into smaller estimates or choose easy work. If you require more AI-drafted replies, people can send weak drafts just to meet the count. The incentive changes the work.

InfoWorld's discussion of story points and "tokenmaxxing" is a useful prompt to examine your own incentives. It is not evidence that every engineering metric is broken. Ask: what behavior would this target reward, even if it doesn't improve the result you want?

Make the delivery example concrete

Suppose your delivery lead wants work to move through the backlog faster.

• Incentive: recognize teams for completing more story points each sprint.
• Outcome: customers receive useful, reliable changes on the agreed priority.
• Guardrail: track production defects and customer-reported regressions for released work.

Story points can still help a team plan a sprint. They should not become a cross-team performance score. Once compensation, comparison, or pressure attaches to them, teams have a reason to make estimates look good rather than make delivery better.

The same applies to ticket counts, pull-request counts, lines of code, and individual "flow" scores. Do not build a universal productivity score, use metrics to surveil people, or rank individuals by work-item throughput. Software work varies too much in ambiguity, risk, maintenance burden, and collaboration to make those comparisons fair or useful.

Use team-level measures to improve a system. Use manager judgment, customer context, and direct review for people decisions.

Run a reversible AI support-reply pilot

AI support drafting is a good place to practice disciplined measurement because the work has visible inputs, a human review step, and a clear fallback.

Run this pilot through your existing help desk and a shared spreadsheet:

Start with a baseline. For five business days, record the normal review-and-send time for eligible tickets, plus corrections and escalations. Do not claim improvement until you have something comparable.

Define eligible cases narrowly: new email tickets asking only for order status, where the order number is present and the order system shows a current status. Exclude refunds, cancellations, delivery disputes, account changes, threats of legal action, complaints, VIP handling, and any ticket without a clear record.

For two weeks or a set volume such as 50 eligible tickets, let the tool draft a reply using approved order data. A support agent must check the facts and send every reply. The tool does not send messages, change orders, issue refunds, or make promises.

The support lead owns the pilot, reviews the weekly sample, records corrections, and decides whether to continue. If the stop condition is met, pause immediately. Keep the manual process available throughout. That makes the test reversible.

The goal is not "maximize AI usage." The decision is whether drafting saves reviewer time without increasing customer harm. One outcome signal and one guardrail are enough for that decision.

Audit a metric before you target it

Before a proxy becomes a target, spend 10 minutes answering these questions in writing:

1. What decision will this number change in the next month?
2. What outcome are we trying to improve?
3. Why is this proxy connected to that outcome?
4. What important quality, risk, or cost does it fail to show?
5. How could a reasonable person improve this number without improving the outcome?
6. What incentive will the target create?
7. What guardrail would reveal harm early?
8. Who owns interpreting the result and acting on it?
9. What result would make us pause, change, or retire the measure?
10. When will we review whether this metric is still useful?

If you cannot write clear answers, keep the number as context, not a target.

Keep a decision log, not a reporting ritual

A decision log is for decisions with tradeoffs: whether to continue a pilot, change a workflow, alter a service level, or retire a metric. It is not a replacement for routine operational reporting.

Use a tab in the same spreadsheet:

The cadence should match risk. Review fast pilots weekly or at a volume milestone because failures can compound quickly. Review stable measures monthly or quarterly, when enough work has accumulated to support a useful decision.

Use frameworks as checks, not scorecards

A few resources can improve your questions without creating another measurement program.

• The SPACE framework helps you remember that developer productivity includes satisfaction, performance, activity, communication, and efficiency. It does not support compressing people into one score.
• DORA's capabilities research is useful for examining delivery practices and outcomes together. It is not a recipe for judging individual engineers from deployment data.
• Google SRE's guidance on postmortem culture is a reminder to study failures without turning measurement into blame.
• The UK government's Magenta Book provides evaluation guidance: define the question, compare against a baseline where possible, and state limits.
• The NIST AI Risk Management Framework is useful for assigning accountability and managing AI risk. It does not certify an AI workflow as safe.

Start with one real decision

Pick one decision your team is already struggling to make. Write the outcome, choose one proxy, add one guardrail, and name an owner.

Then run the smallest reversible test you can.

If the measure improves while the work gets worse, believe the work.