The reason a rating system inflates over time
Nobody in the system gains from a low score. When every participant benefits from inflation, the scale compresses upward until it stops distinguishing anything.
Filed by The Archivist 2 min read
Intuition test — answer before you read on
A five-star scale drifts above 4.7 and functions as two points. What drives the compression?
Correct answer: B
Option A is one contributing channel among several. The structural driver is that sellers, buyers and the platform all gain from higher numbers, while dissatisfied users disproportionately abstain — so the sample skews up before generosity enters.
A marketplace launches a five-star scale. Within three years the average listing sits above 4.7, and buyers have learned to treat anything below 4.5 as a warning. The five-point scale now operates as a two-point scale, and the compression happened without any rule change.
What everyone sees
This is often described as reviewers being too generous, which locates the cause in individual softness. But the pattern appears across marketplaces with different populations and different review mechanics, which suggests something structural rather than a coincidence of temperament.
What is actually happening
Every actor faces an incentive pointing upward. Sellers lose sales below the threshold, so they solicit high ratings and dispute low ones. Buyers who leave a low rating face possible retaliation in two-sided systems, which Bolton, Greiner and Ockenfels showed suppresses negative feedback. Platforms benefit from high averages because they signal a healthy market. Dellarocas and Wood documented that dissatisfied users disproportionately decline to rate at all, biasing the sample upward before any generosity is involved. With no participant advantaged by accuracy at the bottom of the scale, the distribution drifts up until the informative range is exhausted.
Why it stays hidden
The inflation hides because each rating is individually defensible. A seller who provided acceptable service has not earned three stars in any ordinary sense, and a buyer who withholds a complaint has done nothing wrong. The scale degrades through the accumulation of reasonable acts, which is why no participant can identify the moment it stopped working or a party responsible for it.
A rating scale does not collapse because reviewers are kind. It collapses because nobody in it profits from a low number.
A rating scale does not collapse because reviewers are kind. It collapses because nobody in it profits from a low number.
Collect this card
A rating scale does not collapse because reviewers are kind. It collapses because nobody in it profits from a low number.
0 / 10,000 collected
Sources & further reading 2
- Bolton, Greiner & Ockenfels — engineering trust: reciprocity in the production of reputation information
- Dellarocas & Wood — the sound of silence in online feedback
Cross-references
Related files
Filed near this one in the index.
-
No visual on fileStatistical Illusions Entry #0437
The reason a study that replicates is worth two that do not
A single study is a claim. A replication is a test of that claim. Two unreplicated studies are two untested claims, not twice the evidence.
AdeptThe hidden part #0437Novelty fills journals. Replication fills knowledge. The incentive structure rewards the wrong one.
Statistics Open file -
No visual on fileStatistical Illusions Entry #0434
Why a percentage change needs its starting point
A fifty per cent increase from two is one. A fifty per cent increase from two million is one million. The percentage is identical; the meaning is not.
AdeptThe hidden part #0434A percentage without a base is not a statistic. It is a frame looking for a denominator the reader will supply.
Statistics Open file -
No visual on fileScarcity & Queues Entry #0462
The reason an invitation raises acceptance more than an offer
An offer says the item is available. An invitation says the recipient was selected. The difference is not in the item but in the identity it assigns.
AdeptThe hidden part #0462An offer presents a product. An invitation presents an identity. People accept identities faster than products.
Scarcity Open file -
No visual on fileScarcity & Queues Entry #0458
The reason an expiring discount beats a larger permanent one
A permanent reduction can be acted on at any time, which means it can be postponed indefinitely. A deadline converts an intention into a dated task.
AdeptThe hidden part #0458An open offer can be postponed forever. The deadline is not pressure; it is the thing that gives the intention a date.
Scarcity Open file
Annotations are reserved for archive members.
Sign in to annotate