Filtered dossier

Statistical Illusions

Entirely true figures pointing at an entirely false conclusion.

  • 12 / 12
  • No visual on file
    Statistical Illusions Entry #0437

    The reason a study that replicates is worth two that do not

    A single study is a claim. A replication is a test of that claim. Two unreplicated studies are two untested claims, not twice the evidence.

    Adept
    The hidden part #0437

    Novelty fills journals. Replication fills knowledge. The incentive structure rewards the wrong one.

    Open file
  • No visual on file
    Statistical Illusions Entry #0436

    Why a p-value is not a probability that you are right

    The p-value measures how surprising the data would be if the null hypothesis were true. It says nothing about the probability that the hypothesis itself is true.

    Novice
    The hidden part #0436

    A p-value tells you how surprising the data is. It does not tell you how true the hypothesis is.

    Open file
  • No visual on file
    Statistical Illusions Entry #0435

    The reason a graph without zero can double an effect

    Truncating the y-axis does not change the data. It changes the slope the eye sees, and the eye reads slope as magnitude before the mind reads the numbers.

    Master
    The hidden part #0435

    A graph with correct data and a truncated axis is not lying about the numbers. It is lying about the shape.

    Open file
  • No visual on file
    Statistical Illusions Entry #0434

    Why a percentage change needs its starting point

    A fifty per cent increase from two is one. A fifty per cent increase from two million is one million. The percentage is identical; the meaning is not.

    Adept
    The hidden part #0434

    A percentage without a base is not a statistic. It is a frame looking for a denominator the reader will supply.

    Open file
  • No visual on file
    Statistical Illusions Entry #0433

    The reason a small sample produces the biggest effects

    Rank any set of groups by outcome and the smallest ones fill both extremes. Low counts produce wide swings, and the swing is mistaken for the finding.

    Adept
    The hidden part #0433

    Small samples swing hardest in both directions. Publishing only one tail turns the swing into a recommendation.

    Open file
  • No visual on file
    Statistical Illusions Entry #0432

    Why a correlation of one variable with time explains nothing

    Two quantities that both rise over the years will correlate strongly with each other. The shared trend does the work and neither variable has touched the other.

    Novice
    The hidden part #0432

    Two trends will always agree. The coefficient reports the shared direction of time, not a relationship between the variables.

    Open file
  • No visual on file
    Statistical Illusions Entry #0427

    The reason a rising rate can mean a shrinking problem

    The rate went up and the headline called it a crisis. The count went down and nobody mentioned it. Both statements described the same data.

    Adept
    The hidden part #0427

    Denominators move independently. A rising rate can coexist with a falling count whenever the base grows faster than the outcome.

    Open file
  • No visual on file
    Statistical Illusions Entry #0426

    Why an average can describe nobody in the room

    The average income in a room of ten people can be a number that none of them earn, and yet it is reported as if it describes them all.

    Novice
    The hidden part #0426

    Central tendency erases the spread. The average describes the arithmetic, not the room.

    Open file
  • No visual on file
    Statistical Illusions Entry #0428

    Why a ninety-nine per cent accurate test is usually wrong

    A test that is right ninety-nine times in a hundred can still be wrong about most of the cases it flags, and simple arithmetic shows why.

    Novice
    The hidden part #0428

    Accuracy is a property of the test. Whether an alert is right also depends on how rare the thing is.

    Open file
  • No visual on file
    Statistical Illusions Entry #0431

    The reason a survey of survivors misses the finding

    Study only the cases that remain and the causes of disappearance become invisible. The missing rows are usually the ones carrying the answer.

    Adept
    The hidden part #0431

    The sampling frame is the finding. If the population was selected by outcome, shared features prove nothing about cause.

    Open file
  • No visual on file
    Statistical Illusions Entry #0430

    Why the best performers regress next season

    Extreme results are part skill and part luck. The luck does not repeat, so the best performers drift back without anything having gone wrong.

    Master
    The hidden part #0430

    When a measurement is partly noise, the top of the table is where the noise was largest.

    Open file
  • No visual on file
    Statistical Illusions Entry #0429

    The reason two true statistics can contradict each other

    One batter can have a higher average than another in every single season and a lower average across both. Neither figure is wrong; the weighting differs.

    Adept
    The hidden part #0429

    A combined average is weighted by attempt counts. Change the weights and the comparison can reverse without any figure being wrong.

    Open file