Choice Design Entry #0238 Classified Declassified

Why the first result is clicked regardless of relevance

Rank is cheap to read and relevance is expensive to assess. The searcher substitutes the cheap signal and experiences the substitution as judgement.

No visual record attached The written record below is complete.
Plate 253 — The cue that stood in for the judgement

Intuition test — answer before you read on

Swapping the top two search results moves the clicks with position, not content. Why?

Researchers swap the top two results in a search engine without telling users. Click-through follows the position, not the content: whatever now sits first receives most of the clicks. Users report choosing the result that looked most relevant, and the result they chose is the one that moved.

What everyone sees

Position bias is often described as laziness, which is unfair and unhelpful. Assessing relevance requires reading several snippets and modelling which page answers the question — genuine cognitive work. Following rank requires none. The searcher is not avoiding effort out of indifference; they are using a signal that is usually correlated with what they want.

What is actually happening

Joachims and colleagues used eye-tracking to show that users examine the top results far more and that swapping ranks moves clicks with position. Pan and colleagues found that participants trusted rank over relevance even when the top result was manipulated to be a poor match. Gigerenzer and Goldstein’s work on fast-and-frugal heuristics explains why: a single cue that correlates with the criterion can outperform integration of many cues under time pressure, so rank-following is a rational default that becomes exploitable the moment ranking is controlled by an interested party.

Why it stays hidden

The substitution hides because the searcher generates a justification after the click. Having chosen the first result, they can describe why it looked apt, and the description is sincere. What they cannot observe is the counterfactual: the same account would have been produced for whichever result occupied the position, which is exactly what the swap experiments demonstrate.

The searcher is not reading relevance and finding rank. They are reading rank and calling it relevance.

The searcher is not reading relevance and finding rank. They are reading rank and calling it relevance.

The hidden part — entry #0238

Collect this card

The searcher is not reading relevance and finding rank. They are reading rank and calling it relevance.

0 / 10,000 collected

Sources & further reading 2
  1. Joachims et al. — accurately interpreting clickthrough data as implicit feedback
  2. Pan et al. — in Google we trust: users’ decisions on rank, position, and relevance

Circulate this file

Annotations are reserved for archive members.

Sign in to annotate