Counterfactual Analysis: What Could Have Been

The best decisions are often invisible because they prevent disasters that never happen.

This is the central problem with measuring decision quality. We live in a single timeline. We see the outcome of the choice we made, not the outcome of the choice we didn't make. A marketing director launches a campaign that generates 15% uplift. Success. But what if a different creative approach would have generated 22%? The counterfactual—the path not taken—remains forever unknown. We celebrate the visible win and never know we left value on the table.

This blindness shapes how organisations actually evaluate decisions. They measure outcomes, not quality. These are not the same thing. A poor decision can produce a good outcome through luck. A brilliant decision can produce a poor outcome through circumstance. Yet we reward the former and punish the latter, creating perverse incentives that compound over time.

The real cost emerges at scale. When decision-makers learn that outcomes matter more than reasoning, they optimise for outcomes. They cherry-pick metrics. They avoid decisions with uncertain payoffs. They become conservative in ways that look prudent but are actually risk-averse. The organisation stops making the decisions that would have worked, because it never learns that they would have worked.

Counterfactual analysis offers a way out, though not the way most people imagine.

The mistake is thinking you can reconstruct what would have happened. You cannot. The counterfactual is not a prediction you can verify. It is a structured assumption about how the world works. When a product launch underperforms, you cannot know whether a different positioning would have succeeded. You can only model what you believe the relationship between positioning and demand to be, then reason through the implications. This is not fortune-telling. It is disciplined thinking about causation.

The discipline matters because it forces explicitness. When you articulate the causal model—the mechanism by which your decision should produce an outcome—you expose your assumptions to scrutiny. You discover what you actually believe versus what you assume you believe. A campaign underperforms. Your implicit model might be: "We chose the wrong audience segment." But when you write out the causal chain, you might realise your actual belief is: "The audience segment was right, but the message resonated less than we predicted because we underestimated competitor noise." These are different problems requiring different counterfactuals.

This is where measurement becomes useful. Not to prove you were right, but to calibrate your causal model. Did the outcome match what your model predicted? If yes, your model is working. If no, something in your reasoning broke. The counterfactual analysis then becomes an investigation: which assumption was wrong? This creates a feedback loop. Over time, your causal models improve. Your decisions improve. Not because you get lucky more often, but because you understand causation more accurately.

The organisations that do this well rarely talk about it. They don't frame it as "counterfactual analysis." They simply have a culture where decisions are treated as experiments with explicit hypotheses. Before launch, they write down what they expect to happen and why. After launch, they compare prediction to reality. When they diverge, they investigate. This creates institutional memory about what actually works, not just what happened to work once.

The alternative is the status quo: measuring outcomes and calling it decision quality. This works until it doesn't. It produces a portfolio of decisions that look good in hindsight but are actually mediocre. The organisation becomes skilled at explaining why things happened, not at predicting whether they will happen. When the environment shifts—when the old patterns stop working—the organisation has no framework for adaptation. It has only a history of outcomes.

Counterfactual analysis is not about perfect foresight. It is about honest reasoning. It is about building organisations that learn from what they do, not just from what they see. The best decisions remain invisible. But the reasoning behind them does not have to be.