Why a comparison class is chosen before a claim

The fastest-growing firm in its sector, measured across a three-year window beginning the year after a poor one, among companies of a particular size band. Every qualifier was chosen with the answer already visible. Remove any one of them and a different company holds the title.

What everyone sees

Superlative claims are checked for accuracy, and they pass, so scrutiny stops. Accuracy is the wrong test. The claim is a conjunction of a measurement and a reference class, and only the measurement is being audited while the reference class is where the work was done.

What is actually happening

Kahneman and Tversky’s treatment of intuitive prediction argued that sound forecasting requires identifying an appropriate reference class of comparable cases and using its distribution, precisely because judgements built from the features of a single case ignore base rates. Flyvbjerg applied this operationally to large projects, showing that reference class forecasting produced systematically better estimates than expert judgement — and, unavoidably, that the choice of class determines the answer, since a project belongs to many classes at once. The same property that makes reference classes powerful makes them exploitable. Choosing the class is not a preliminary step before the analysis; it is most of the analysis, and it can be performed backwards from a desired conclusion.

Why it stays hidden

The manoeuvre hides because the qualifiers read as precision. A claim specifying sector, period and size band appears more rigorous than a bare one, so the very details that were reverse-engineered are received as evidence of care. Nobody asks how many other combinations were tried, and that number is never reported.