Why the order of questions changes the answers
Ask about happiness first and then about politics, or reverse the order: the same person gives different answers to the same questions.
Intuition test — answer before you read on
Two versions of a survey swap the order of a political-trust question and a life-satisfaction question. What changes?
A polling firm runs two versions of the same survey. Both contain identical questions about life satisfaction and trust in government. In version A, satisfaction comes first. In version B, the order is reversed. The correlation between the two answers is markedly higher in version B, where the political question has already coloured what satisfaction means.
What everyone sees
Respondents treat each question as independent. They believe they are reporting a stable opinion that existed before the survey arrived, not constructing one in real time. The surveyor likewise assumes that moving a question up or down the page is a formatting choice, not an intervention.
What is actually happening
Schuman and Presser’s foundational work on question-order effects showed that earlier items create a context that later items are interpreted within. Strack, Martin and Schwarz demonstrated the mechanism with a dating-satisfaction and life-satisfaction pair: when the dating question came first, its answer bled into the general evaluation, inflating the correlation. The effect is not random noise; it is directional, because earlier answers supply the frame through which later answers are constructed.
Why it stays hidden
The effect hides because questionnaires feel neutral. A printed page with numbered questions resembles a test — something that records what is already known. The idea that the instrument alters the measurement is familiar in physics but feels absurd when applied to a human being answering questions about their own life, which is precisely why it is so rarely suspected.
Earlier answers become context for later ones. A survey does not measure opinion; it measures opinion in the order the surveyor chose.
Earlier answers become context for later ones. A survey does not measure opinion; it measures opinion in the order the surveyor chose.
Collect this card
Earlier answers become context for later ones. A survey does not measure opinion; it measures opinion in the order the surveyor chose.
0 / 10,000 collected
Sources & further reading 3
- Schuman & Presser, "Questions and Answers in Attitude Surveys", 1981
- Strack, Martin & Schwarz, "Priming and Communication", Social Cognition, 1988
- Tourangeau, Rips & Rasinski, "The Psychology of Survey Response", 2000
Cross-references
Related files
-
No visual on fileDecision Architecture Entry #0134
Why the second question on a survey is the real one
The opening question sets a frame. Every answer that follows is given inside it, and the respondent cannot see the frame because they are standing in it.
AdeptThe hidden part #0134The first question is not a question. It is the frame in which the second question will be understood.
Decisions Open file -
Hidden Logic Entry #0012Why the second-cheapest wine always wins
Restaurants do not expect you to buy the cheapest bottle. They expect you to refuse it — and they price the list so that refusal lands exactly where the margin is fattest.
NoviceThe hidden part #0012The cheapest bottle is not for sale. It is a decoy that makes the next one up feel like restraint.
Logic Open file -
No visual on fileParadox & Proof Entry #0161
Why unanimous agreement lowers confidence in a verdict
Perfect consensus in a large group implies a shared error. The very agreement that looks like certainty is evidence against it.
NoviceThe hidden part #0161Perfect agreement among many is not strong evidence. It is evidence of a shared input, which may be the truth or may be the error.
Paradox Open file
Annotations are reserved for archive members.
Sign in to annotate