Measuring Decision Quality: A Framework Beyond Outcomes

Outcome bias has colonised how we evaluate decisions, and it's making us systematically worse at learning from experience.

We measure decisions by their results. A campaign succeeds because sales rose. A hiring choice was right because the employee performed. A strategic pivot was wise because the market moved our way. This feels intuitive—outcomes are concrete, measurable, unambiguous. But this framework obscures something critical: the quality of a decision is not determined by what happened afterward. It's determined by what was knowable at the time it was made.

Consider a pharmaceutical company that approves a drug based on rigorous Phase III trials showing efficacy. Three years later, a rare side effect emerges in one million users. The decision was sound. The outcome was catastrophic. Conflating these two creates a false equivalence that poisons organisational learning. Teams begin to fear good decisions that happen to fail, and they celebrate lucky ones that succeeded despite poor reasoning.

The problem deepens when we recognise that most business decisions operate under genuine uncertainty. A product launch, a market entry, a partnership—these unfold in conditions where no amount of analysis eliminates risk. If we only judge decisions by outcomes, we're essentially running a lottery and calling the winners "strategic." Over time, this teaches organisations to either become paralysed by analysis or to develop a culture of reckless optimism, where bold moves are praised regardless of their decision-making process.

What would a framework beyond outcomes look like?

Start with information quality at decision time. What data was available? How reliable was it? Were there known unknowns that should have triggered caution? A decision made with incomplete information but sound reasoning deserves a different evaluation than one made with available information that was ignored. This isn't about hindsight—it's about whether the decision-maker operated within the epistemic constraints they actually faced.

Second, reasoning transparency. How was the decision reached? Were assumptions explicit or buried? Were alternatives genuinely considered or merely performed? A decision that emerges from structured deliberation—where trade-offs are named, where dissenting views are actively sought—has a different character than one made through intuition or political pressure, even if both produce identical outcomes. The reasoning process is itself a quality signal.

Third, calibration over time. This requires tracking decisions across a portfolio and examining whether the confidence expressed matched the accuracy achieved. A leader who says "I'm 70% confident in this outcome" and is right 70% of the time is well-calibrated. One who expresses 70% confidence but is right only 40% of the time is overconfident—a systematic bias that will compound across many decisions. Calibration is measurable. It requires discipline, but it reveals whether someone's judgment is actually reliable.

Fourth, decision reversibility. Some choices are nearly irreversible; others can be adjusted as new information arrives. A framework that accounts for this distinction encourages different decision-making approaches. A reversible decision can afford to move faster with less certainty. An irreversible one demands more caution. Conflating these creates either paralysis or recklessness.

Finally, process consistency. Did the organisation apply the same standards to similar decisions? Inconsistency—where one risky bet is celebrated and another identical one is condemned—suggests that outcomes are driving evaluation, not process. Consistency in how decisions are made, reviewed, and learned from is itself a measure of decision quality.

Implementing this framework requires discomfort. It means acknowledging that some failed decisions were actually well-made. It means recognising that some successful ones were lucky. It means building measurement systems that track process, not just results. But organisations that do this develop something rare: the ability to learn from decisions rather than merely react to outcomes. They build judgment. They compound wisdom.

The alternative is to remain trapped in outcome bias, forever mistaking luck for strategy, and never quite understanding why.