The reason recommendations feel personal but arrive collectively
A suggestion tuned to you is usually computed from people who resemble you. The system does not know your taste; it knows your neighbours.
Filed by The Archivist 2 min read
Intuition test — answer before you read on
How can a recommendation fit you closely without the system modelling you as an individual?
Correct answer: B
Option A describes attribute-based prediction, which collaborative filtering explicitly avoids: the original systems used no information about content or demographics. Predictions come from agreement patterns among users, or from item co-consumption, both of which are neighbourhood averages.
A recommendation appears that fits precisely, and the impression is of being understood. The computation underneath contains no model of you as an individual. It located accounts whose past behaviour overlaps with yours and offered what they consumed and you have not, which is a statement about a group rather than about a person.
What everyone sees
Personalisation is read as a profile: the system has assembled a description of your preferences and consults it. Some systems do hold attribute models. The dominant mechanism is comparative rather than descriptive, and it can produce accurate suggestions while holding no representation of why you might like anything.
What is actually happening
Resnick and colleagues introduced collaborative filtering by predicting a reader’s rating from the ratings of users who had agreed with them previously, requiring no information about the content itself. Sarwar and colleagues reformulated the same logic around items, computing similarity between items from the overlap in who consumed them, which scaled the approach without adding any model of the individual. In both designs the prediction is an average over a neighbourhood. Accuracy therefore rises with the number of people who resemble you, and personalisation is really the resolution at which the crowd has been sliced.
Why it stays hidden
The collective basis hides because the output is addressed to one person and arrives in a private context. There is no visible neighbourhood, no list of the accounts your suggestion was derived from, and no indication that the same item is being shown to thousands of others in the same slice. The experience of being known is produced by delivery, not by the computation.
The system holds no theory of your taste. It holds a list of people whose behaviour has overlapped with yours.
The system holds no theory of your taste. It holds a list of people whose behaviour has overlapped with yours.
Collect this card
The system holds no theory of your taste. It holds a list of people whose behaviour has overlapped with yours.
0 / 10,000 collected
Sources & further reading 2
- Resnick et al. — GroupLens: an open architecture for collaborative filtering
- Sarwar et al. — item-based collaborative filtering recommendation algorithms
Cross-references
Related files
Filed near this one in the index.
-
Hidden Mechanics Entry #0008
The hole in your pen cap is a lung
That small hole in the top of a ballpoint cap is not for airflow to the ink, or for pressure, or for manufacturing. It is there because people chew pens and occasionally swallow the cap.
Adept -
No visual on filePlatform Mechanics Entry #0287
The reason a platform’s rules change after you build on them
The rules were generous when they needed builders. Once the builders are invested, the rules can be revised, because the cost of leaving exceeds the cost of complying.
AdeptThe hidden part #0287Generous terms attract investment. Investment creates dependency. Dependency permits revision. The sequence is not accidental.
Platforms Open file -
No visual on filePlatform Mechanics Entry #0284
Why network effects make second place worthless
The value of the first network rises with each user. The value of the second rises too, but the gap between them widens rather than closes.
AdeptThe hidden part #0284The first network large enough owns the people. The second can copy every feature except the one that matters.
Platforms Open file -
No visual on fileHabit Loops Entry #0302
The reason a habit survives the loss of its reward
The reward that started the habit can vanish entirely, and the habit continues. What was once a means to an end has become the end itself.
AdeptThe hidden part #0302Repetition outlives reinforcement. The habit that was built by a reward no longer needs it, and no longer checks.
Habit Open file
Annotations are reserved for archive members.
Sign in to annotate