The Outcome Problem: Why Results Don't Measure Decision Quality

A company makes a terrible decision with incomplete information, executes it flawlessly, and generates a windfall profit. A strategist makes a defensible choice based on rigorous analysis, implements it competently, and watches the market shift unexpectedly, destroying value. Which decision was better?

The answer reveals why most organizations have no reliable way to assess whether their leaders actually make good decisions. We conflate outcomes with decision quality, then wonder why our best performers sometimes fail catastrophically, and why mediocre thinkers occasionally succeed.

This confusion runs deeper than simple luck acknowledgment. It's structural. Outcome-based evaluation creates a feedback system that rewards hindsight bias and punishes intellectual honesty. A leader who took a 70% probability bet that failed learns nothing useful—the outcome tells them their reasoning was wrong, even if it was sound. A leader who took a 40% probability bet that succeeded believes they were right, even if they got lucky. Over time, organizations promote the lucky and eliminate the thoughtful.

The problem intensifies because outcomes are visible and decision processes are not. A board can see quarterly results. They cannot see the reasoning that led to a strategic pivot, the scenarios considered, the uncertainties acknowledged, or the assumptions stress-tested. So they anchor to what they can measure. This creates a perverse incentive: obscure your reasoning, claim credit for wins, and blame external factors for losses. The executives who survive are often those best at narrative construction, not decision-making.

Consider a pharmaceutical company evaluating a drug candidate. The decision to advance to Phase III trials involves probabilistic thinking about efficacy, safety, market timing, and competitive landscape. A rigorous decision process might conclude: "We should proceed because the expected value is positive, even though we estimate a 60% chance of ultimate failure." If the drug fails, outcome-focused evaluation concludes the decision was wrong. But the decision was sound—it just lost the coin flip. If the drug succeeds, everyone celebrates the decision-maker's vision, though they may have simply gotten lucky.

What changes when you measure decision quality separately from outcomes?

First, you can actually learn. A decision that was well-reasoned but unlucky teaches you something about the world. A decision that was poorly-reasoned but lucky teaches you nothing except that you got away with something. By decoupling quality from results, you create space for genuine feedback on your reasoning process.

Second, you can identify your actual decision-makers. In outcome-based systems, luck and skill are inseparable. In quality-based systems, you can track whose reasoning consistently holds up under scrutiny, whose assumptions prove durable, whose probabilistic estimates calibrate well over time. This is measurable. It requires discipline—documenting the decision, the reasoning, the key uncertainties, the probability estimates—but it's entirely doable.

Third, you shift incentives toward intellectual rigor. If your evaluation system rewards decision quality rather than outcomes, people stop hiding their reasoning. They start stress-testing assumptions because that's what gets recognized. They become comfortable acknowledging uncertainty because uncertainty is part of good reasoning, not a sign of weakness.

The practical barrier isn't conceptual—it's organizational will. Measuring decision quality requires patience. You cannot evaluate a decision immediately. You must wait for outcomes to emerge, then work backward to assess whether the reasoning was sound given what was known then, not what is known now. This takes discipline that quarterly earnings calls do not encourage.

Yet organizations that implement this—that build decision review processes, that separate quality assessment from outcome evaluation, that promote people based on reasoning rather than luck—report a measurable shift. Better strategic choices. Fewer catastrophic failures. More sustainable performance.

The uncomfortable truth: your organization probably cannot tell the difference between a good decision-maker and a lucky one. Until it can, it will keep promoting the latter.