Why recommendation systems narrow taste while widening catalogue
The library grows every month and the same titles keep arriving at the top. Availability and discoverability are separate quantities that can move apart.
Filed by The Archivist 2 min read
Intuition test — answer before you read on
A service adds 40,000 titles while users report seeing the same 200. Why?
Correct answer: B
Collaborative filtering can only recommend confidently where interaction data exists, and new items start with none. Fleder and Hosanagar showed recommenders can reduce aggregate diversity even while helping individuals — catalogue size and reachable set are separate quantities.
A service adds forty thousand titles in a year. Individual users report seeing the same two hundred. Both facts are true, and they are produced by the same system: one number describes what exists and the other describes what is reachable from the home screen.
What everyone sees
A larger catalogue is presented as more choice, and the presentation is literally accurate. Users describe the service as having everything and simultaneously complain that there is nothing to watch. The contradiction is usually attributed to indecision rather than to the ranking layer sitting between the catalogue and the user.
What is actually happening
Collaborative filtering recommends what similar users engaged with, so items with more interaction data are easier to recommend confidently and new items begin with none. Fleder and Hosanagar showed that recommenders can reduce aggregate diversity even while helping individuals, and Celma and Cano found popularity bias in music recommendation graphs. The catalogue grows in the database while the reachable set is shaped by the confidence the model has, which is a function of existing attention.
Why it stays hidden
The narrowing hides because the interface reports abundance. The catalogue count is advertised, the reachable set is not measured, and no user can see the titles they were never shown. Search offers an apparent escape and mostly does not provide one, since it requires knowing the name of something the recommender did not mention.
Catalogue size and reachable set are different numbers. Recommendation confidence is built from attention already spent.
Catalogue size and reachable set are different numbers. Recommendation confidence is built from attention already spent.
Collect this card
Catalogue size and reachable set are different numbers. Recommendation confidence is built from attention already spent.
0 / 10,000 collected
Sources & further reading 3
- Fleder & Hosanagar, "Blockbuster Culture’s Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity", Management Science, 2009
- Celma & Cano, "From Hits to Niches? Or How Popular Artists Can Bias Music Recommendation", RecSys, 2008
- Anderson, "The Long Tail", 2006
Cross-references
Related files
Filed near this one in the index.
-
No visual on filePlatform Mechanics Entry #0276
Why the feed shows what keeps you, not what you wanted
A ranked feed is optimised for the next minute of attention rather than for the thing you opened the application to do.
NoviceThe hidden part #0276A feed answers what will keep you here, which is a different question from what you came for.
Platforms Open file -
Hidden Mechanics Entry #0009
Why elevator close buttons do nothing
In most lifts installed since the early nineties, the door-close button is disconnected for everyone except firefighters. It still lights up. That is the point.
NoviceThe hidden part #0009The button is not wired to the door. It is wired to your patience.
Mechanics Open file -
No visual on filePlatform Mechanics Entry #0286
Why moderation is a pricing problem, not a values problem
At scale, every piece of content must be reviewed at a cost that approaches zero. The cost determines what gets caught, and the values follow the budget.
NoviceThe hidden part #0286Values are free to declare. Moderation costs money per decision. At scale, the budget determines what the values actually mean.
Platforms Open file -
No visual on fileHabit Loops Entry #0303
Why changing a route changes a routine
A habit feels like a decision stored in the person, but it is actually stored in the environment. Move the person and the habit loses its cue.
NoviceThe hidden part #0303Habits are stored with their surroundings. Change the surroundings and the habit has to be rebuilt from scratch — or it simply does not fire.
Habit Open file
Annotations are reserved for archive members.
Sign in to annotate