Why the first result is clicked regardless of relevance
Rank is cheap to read and relevance is expensive to assess. The searcher substitutes the cheap signal and experiences the substitution as judgement.
Filed by The Archivist 2 min read
Intuition test — answer before you read on
Swapping the top two search results moves the clicks with position, not content. Why?
Correct answer: B
Option A moralises what is actually an efficient strategy. A single well-correlated cue often beats integrating many under time pressure. The vulnerability is not laziness but the fact that the cue is controlled by whoever sets the ranking.
Researchers swap the top two results in a search engine without telling users. Click-through follows the position, not the content: whatever now sits first receives most of the clicks. Users report choosing the result that looked most relevant, and the result they chose is the one that moved.
What everyone sees
Position bias is often described as laziness, which is unfair and unhelpful. Assessing relevance requires reading several snippets and modelling which page answers the question — genuine cognitive work. Following rank requires none. The searcher is not avoiding effort out of indifference; they are using a signal that is usually correlated with what they want.
What is actually happening
Joachims and colleagues used eye-tracking to show that users examine the top results far more and that swapping ranks moves clicks with position. Pan and colleagues found that participants trusted rank over relevance even when the top result was manipulated to be a poor match. Gigerenzer and Goldstein’s work on fast-and-frugal heuristics explains why: a single cue that correlates with the criterion can outperform integration of many cues under time pressure, so rank-following is a rational default that becomes exploitable the moment ranking is controlled by an interested party.
Why it stays hidden
The substitution hides because the searcher generates a justification after the click. Having chosen the first result, they can describe why it looked apt, and the description is sincere. What they cannot observe is the counterfactual: the same account would have been produced for whichever result occupied the position, which is exactly what the swap experiments demonstrate.
The searcher is not reading relevance and finding rank. They are reading rank and calling it relevance.
The searcher is not reading relevance and finding rank. They are reading rank and calling it relevance.
Collect this card
The searcher is not reading relevance and finding rank. They are reading rank and calling it relevance.
0 / 10,000 collected
Sources & further reading 2
- Joachims et al. — accurately interpreting clickthrough data as implicit feedback
- Pan et al. — in Google we trust: users’ decisions on rank, position, and relevance
Cross-references
Related files
Filed near this one in the index.
-
Hidden Mechanics Entry #0009
Why elevator close buttons do nothing
In most lifts installed since the early nineties, the door-close button is disconnected for everyone except firefighters. It still lights up. That is the point.
NoviceThe hidden part #0009The button is not wired to the door. It is wired to your patience.
Mechanics Open file -
No visual on filePlatform Mechanics Entry #0286
Why moderation is a pricing problem, not a values problem
At scale, every piece of content must be reviewed at a cost that approaches zero. The cost determines what gets caught, and the values follow the budget.
NoviceThe hidden part #0286Values are free to declare. Moderation costs money per decision. At scale, the budget determines what the values actually mean.
Platforms Open file -
No visual on fileHabit Loops Entry #0303
Why changing a route changes a routine
A habit feels like a decision stored in the person, but it is actually stored in the environment. Move the person and the habit loses its cue.
NoviceThe hidden part #0303Habits are stored with their surroundings. Change the surroundings and the habit has to be rebuilt from scratch — or it simply does not fire.
Habit Open file -
No visual on filePlatform Mechanics Entry #0282
Why recommendation systems narrow taste while widening catalogue
The library grows every month and the same titles keep arriving at the top. Availability and discoverability are separate quantities that can move apart.
NoviceThe hidden part #0282Catalogue size and reachable set are different numbers. Recommendation confidence is built from attention already spent.
Platforms Open file
Annotations are reserved for archive members.
Sign in to annotate