Rhetoric & Omission Entry #0585 Classified Declassified

The reason a statistic needs a comparison to mean anything

A number alone is a fact without a frame. The comparison supplied beside it is what gives it size, direction and significance.

Filed by The Archivist 2 min read

No visual record attached The written record below is complete.
Plate 226 — one number, two comparisons, two opposite conclusions.

Intuition test — answer before you read on

A report says the error rate is three per cent. When does this number mean something?

A report says the error rate is three per cent. Without a comparison, the number floats: is three per cent good or bad? Compared to last year’s five per cent it is progress. Compared to a competitor’s one per cent it is a problem. The number is the same; the comparison determines the conclusion.

What everyone sees

A statistic looks like a complete piece of information: a number plus a unit should be enough. The assumption is that the reader can evaluate a number on its own, the way they can evaluate a photograph without a caption.

What is actually happening

Kahneman and Miller’s norm theory holds that every stimulus is evaluated against a norm, and the norm is constructed from whatever comparison is available. Best described the use and misuse of statistics in public discourse and showed that the same number can support opposite conclusions depending on which comparison is offered. Tufte argued that every data display is a comparison and that a number presented without one leaves the reader to supply their own, which may be the wrong one.

Why it stays hidden

The incompleteness hides because the number carries a sense of precision that feels like completeness. Three per cent sounds like a fact, and facts are supposed to be self-sufficient. The comparison that gives the fact its meaning is chosen by the presenter and is not marked as a choice — it is presented as context, which sounds neutral, and context controls the conclusion.

A number alone is not an argument. The comparison beside it is the argument, and the person who chose the comparison chose the conclusion.

A number alone is not an argument. The comparison beside it is the argument, and the person who chose the comparison chose the conclusion.

The hidden part — entry #0585

Collect this card

A number alone is not an argument. The comparison beside it is the argument, and the person who chose the comparison chose the conclusion.

0 / 10,000 collected

Sources & further reading 3
  1. Kahneman & Miller, "Norm Theory: Comparing Reality to Its Alternatives", Psychological Review, 1986
  2. Best, "Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists", 2001
  3. Tufte, "The Visual Display of Quantitative Information", 2001
Index terms

Cross-references

Related files

Filed near this one in the index.

  • No visual on file
    Narrative Structure Entry #0530

    Why an unresolved ending is remembered longer

    An interrupted task stays in mind longer than a completed one, which is why unresolved endings outlive the stories that finish tidily.

    Master
    The hidden part #0530

    Completion releases attention. Anything left open keeps a claim on it.

    Narrative Open file
  • Why “clue” used to be a ball of yarn
    Hidden Words Entry #0001

    Why “clue” used to be a ball of yarn

    Before it meant evidence, a clue was a physical thread — the one Theseus unwound through the labyrinth so he could find the way back out.

    Master
    The hidden part #0001

    Following a clue is not a metaphor for detection. It is a description of walking backwards along string.

    Words Open file
  • No visual on file
    Narrative Structure Entry #0535

    The reason testimony beats aggregate evidence

    One person’s account defeats a table of ten thousand. The account has a face, a sequence and a cost. The table has none of these.

    Master
    The hidden part #0535

    A story activates a path the table cannot reach. The table has magnitude; the story has a face. The face wins, and this is a finding about the audience.

    Narrative Open file
  • No visual on file
    Naming & Sound Entry #0560

    The reason a new category needs a new word

    Without a name the new thing is described by analogy to old things. The analogy supplies expectations the new thing cannot meet, and it fails before it is understood.

    Master
    The hidden part #0560

    A name is not a label. It is a category boundary. Without it the new thing is judged by standards it was never meant to meet.

    Naming Open file

Circulate this file

Annotations are reserved for archive members.

Sign in to annotate