Why a cluster is expected in random data
Randomness produces clumps. A pattern spread out evenly is the unlikely arrangement, so finding a cluster is not yet finding a cause.
Filed by The Archivist 2 min read
Intuition test — answer before you read on
Why is a visible cluster in data not yet evidence of a cause?
Correct answer: B
Option A is a general caution that misses the specific mechanism. Option C is one possibility among many. People rate over-alternating sequences as most random and genuinely random runs as patterned, and even the standard method for testing streaks carries a selection bias — so both the perception and the correction can be wrong.
Six cases in one small town within eleven months, none in the four towns around it. The map looks deliberate. Scatter the same number of points at random across the same population a thousand times and clusters of that size appear in most of the runs.
What everyone sees
A cluster is read as evidence of a mechanism, because randomness is imagined as evenness — points politely spaced, no crowding. Real random processes do not distribute evenly. Independent events arrive in bursts and gaps, and the intuition that expects regularity will find something to explain in almost every dataset.
What is actually happening
Falk and Konold examined how people judge randomness and found systematic misjudgement in the same direction: sequences with too few repetitions and too much alternation were rated as most random, while genuinely random sequences containing runs were rated as patterned. Miller and Sanjurjo add a sharp lesson about the opposite error — they identified a selection bias in the classic method used to test for streaks in shooting data, showing that the standard estimator understates streakiness, so evidence once treated as settling the hot-hand question does not settle it. Both directions of mistake matter here. Clusters appear without causes, and the tools used to dismiss them can themselves be wrong.
Why it stays hidden
The base rate hides because nobody sees the unremarkable maps. The four towns with no cases generate no report, the eleven months with nothing generate no article, and only the compelling arrangement is ever examined — which is the definition of selecting the sample on the outcome.
Independent events arrive in bursts, so a cluster is the expected texture rather than a signal. Evenness is what would need explaining.
Independent events arrive in bursts, so a cluster is the expected texture rather than a signal. Evenness is what would need explaining.
Collect this card
Independent events arrive in bursts, so a cluster is the expected texture rather than a signal. Evenness is what would need explaining.
0 / 10,000 collected
Sources & further reading 2
- Falk & Konold — making sense of randomness: implicit encoding as a basis for judgment
- Miller & Sanjurjo — surprised by the hot hand fallacy? a truth in the law of small numbers
Cross-references
Related files
Filed near this one in the index.
-
No visual on fileStatistical Illusions Entry #0428
Why a ninety-nine per cent accurate test is usually wrong
A test that is right ninety-nine times in a hundred can still be wrong about most of the cases it flags, and simple arithmetic shows why.
NoviceThe hidden part #0428Accuracy is a property of the test. Whether an alert is right also depends on how rare the thing is.
Statistics Open file -
No visual on fileStatistical Illusions Entry #0437
The reason a study that replicates is worth two that do not
A single study is a claim. A replication is a test of that claim. Two unreplicated studies are two untested claims, not twice the evidence.
AdeptThe hidden part #0437Novelty fills journals. Replication fills knowledge. The incentive structure rewards the wrong one.
Statistics Open file -
No visual on fileStatistical Illusions Entry #0434
Why a percentage change needs its starting point
A fifty per cent increase from two is one. A fifty per cent increase from two million is one million. The percentage is identical; the meaning is not.
AdeptThe hidden part #0434A percentage without a base is not a statistic. It is a frame looking for a denominator the reader will supply.
Statistics Open file -
No visual on fileScarcity & Queues Entry #0462
The reason an invitation raises acceptance more than an offer
An offer says the item is available. An invitation says the recipient was selected. The difference is not in the item but in the identity it assigns.
AdeptThe hidden part #0462An offer presents a product. An invitation presents an identity. People accept identities faster than products.
Scarcity Open file
Annotations are reserved for archive members.
Sign in to annotate