Statistical Illusions Entry #0445 Classified Declassified

The reason a self-selected panel overstates strong opinions

Answering costs time, and the people who pay it are those with something to say. Indifference is the hardest state to get into a dataset.

No visual record attached The written record below is complete.
Plate 495 — The seventy-nine per cent with no view

Intuition test — answer before you read on

Why does a low-response survey overstate polarisation?

Eleven per cent respond. Of those, a third are furious, a third are enthusiastic, and the report describes a divided workforce. The seventy-nine per cent who did not reply were mostly people for whom the topic did not rise to the level of an opinion.

What everyone sees

A low response rate is understood as reduced precision — a smaller sample with wider intervals, correctable by weighting to known demographics. Weighting addresses composition, not intensity. Whether someone bothers to answer is correlated with how strongly they feel, and no demographic variable in the file carries that information.

What is actually happening

Rogelberg and Stanton set out the structure of organisational survey nonresponse, distinguishing passive non-response from active refusal and showing that respondents and non-respondents differ on attitudinal variables in ways that demographic weighting cannot repair, because the propensity to answer is itself an attitude. Keeter and colleagues compared a standard survey with a much more intensive effort achieving far higher response and found that many estimates were similar, while a subset — particularly measures of civic engagement and issue intensity — differed systematically. The nuance matters for honesty: low response does not corrupt everything, but it reliably overstates whichever variables correlate with the willingness to answer, and intensity of opinion is the clearest of those.

Why it stays hidden

The distortion hides because the report reads as a distribution. Percentages sum to a hundred, subgroups are compared, and the indifferent majority appears nowhere — not as a category, not as a caveat. Findings are then acted on as if the workforce were divided, which produces communications aimed at a conflict most people were not in.

Willingness to answer is itself an attitude, and weighting cannot repair it. Indifference is the state a voluntary survey cannot record.

Willingness to answer is itself an attitude, and weighting cannot repair it. Indifference is the state a voluntary survey cannot record.

The hidden part — entry #0445

Collect this card

Willingness to answer is itself an attitude, and weighting cannot repair it. Indifference is the state a voluntary survey cannot record.

0 / 10,000 collected

Sources & further reading 2
  1. Rogelberg & Stanton — understanding and dealing with organizational survey nonresponse
  2. Keeter, Kennedy, Dimock, Best & Craighill — gauging the impact of growing nonresponse on estimates from a national survey

Circulate this file

Annotations are reserved for archive members.

Sign in to annotate