Kevin Zielnicki, Guy Aridor, Aurélien Bibaut, Allen Tran, Winston Chou, and Nathan Kallus
A recommendation is both a suggestion and a small act of persuasion. It can help you discover something you already had a latent taste for, but it can also make an option more attractive simply by putting it in front of you. Untangling those two effects is surprisingly difficult.
This is the central measurement problem in our paper. If someone watches a recommended title, did the system make a good match, or did a prominent recommendation mechanically produce a click? The distinction matters. A recommender that understands taste creates a different kind of value from one that merely controls the shelf space.
We build a discrete-choice model of Netflix viewing that allows people to differ in their tastes, accounts for what they have watched before, and gives recommendations their own possible effect on utility. The recommendation algorithm also creates idiosyncratic variation in what is shown. That variation helps us separately identify the value of targeting, exposure, and the underlying titles.
The resulting model lets us run counterfactuals that would be costly to stage at full scale. Replacing the current recommender with a conventional matrix-factorization system would reduce engagement by about 4 percent. Replacing it with popularity-based recommendations would reduce engagement by about 12 percent. Both alternatives would also make consumption less diverse.
Most of the gain does not come from the brute fact of exposure. It comes from matching different people to different things they are likely to value. And the largest gains appear among titles of middling popularity—not universal hits, which are easy to find, and not the deepest niches, whose audiences may simply be small.
The practical lesson is that a personalized system should not be valued only by counting interactions with recommended rows. Its real contribution is the set of choices it makes possible: helping each person find worthwhile options in a catalog too large for anyone to search alone.