The New York Times on collaborative filtering

The New York Times has a long article on the Netflix prize and collaborative filtering. Very interesting to anyone who’s obsessed with collaborative filtering, as I am. What I found particularly interesting is that customers who pay Netflix but don’t regularly send back movies are bad for the company:

For Netflix, this is doubly important. Customers pay a flat monthly rate, generally $16.99 (although cheaper plans are available), to check out as many movies as they want. The problem with this business model is that new members often have a couple of dozen movies in mind that they want to see, but after that they’re not sure what to check out next, and their requests slow. And a customer paying $17 a month for only one movie every month or two is at risk of canceling his subscription; the plan makes financial sense, from a user’s point of view, only if you rent a lot of movies. (My wife and I once quit Netflix for precisely this reason.) Every time Hastings increases the quality of Cinematch even slightly, it keeps his customers active.

Even though Netflix spends less money shipping movies to them, those customers are at much greater risk of canceling their accounts. Making sure customer queues are full of things they are eager to see is important.

I also thought this was really interesting:

As the teams have grown better at predicting human preferences, the more incomprehensible their computer programs have become, even to their creators. Each team has lined up a gantlet of scores of algorithms, each one analyzing a slightly different correlation between movies and users. The upshot is that while the teams are producing ever-more-accurate recommendations, they cannot precisely explain how they’re doing this. Chris Volinsky admits that his team’s program has become a black box, its internal logic unknowable.

It reminds me of some work I’ve done in the past.