Statistical Illusions Entry #0442 Classified Declassified

Why a cluster is expected in random data

Randomness produces clumps. A pattern spread out evenly is the unlikely arrangement, so finding a cluster is not yet finding a cause.

No visual record attached The written record below is complete.
Plate 492 — The four towns that produced no report

Intuition test — answer before you read on

Why is a visible cluster in data not yet evidence of a cause?

Six cases in one small town within eleven months, none in the four towns around it. The map looks deliberate. Scatter the same number of points at random across the same population a thousand times and clusters of that size appear in most of the runs.

What everyone sees

A cluster is read as evidence of a mechanism, because randomness is imagined as evenness — points politely spaced, no crowding. Real random processes do not distribute evenly. Independent events arrive in bursts and gaps, and the intuition that expects regularity will find something to explain in almost every dataset.

What is actually happening

Falk and Konold examined how people judge randomness and found systematic misjudgement in the same direction: sequences with too few repetitions and too much alternation were rated as most random, while genuinely random sequences containing runs were rated as patterned. Miller and Sanjurjo add a sharp lesson about the opposite error — they identified a selection bias in the classic method used to test for streaks in shooting data, showing that the standard estimator understates streakiness, so evidence once treated as settling the hot-hand question does not settle it. Both directions of mistake matter here. Clusters appear without causes, and the tools used to dismiss them can themselves be wrong.

Why it stays hidden

The base rate hides because nobody sees the unremarkable maps. The four towns with no cases generate no report, the eleven months with nothing generate no article, and only the compelling arrangement is ever examined — which is the definition of selecting the sample on the outcome.

Independent events arrive in bursts, so a cluster is the expected texture rather than a signal. Evenness is what would need explaining.

Independent events arrive in bursts, so a cluster is the expected texture rather than a signal. Evenness is what would need explaining.

The hidden part — entry #0442

Collect this card

Independent events arrive in bursts, so a cluster is the expected texture rather than a signal. Evenness is what would need explaining.

0 / 10,000 collected

Sources & further reading 2
  1. Falk & Konold — making sense of randomness: implicit encoding as a basis for judgment
  2. Miller & Sanjurjo — surprised by the hot hand fallacy? a truth in the law of small numbers

Circulate this file

Annotations are reserved for archive members.

Sign in to annotate