October 5, 2010

"Dear Phil, your tutor has information about you and no particular reason to tell the truth". In his recent "Dear Economist" column (*), Tim Harford tackles the most crucial dilemma of our Information Age. How to create a recommendation system which works.

The future of Netflix shows how crucial such a task can be. As consumers shift from DVD playing to online video streaming, its competitiveness is reduced in Verge G. Kopytoff's analysis (**) by the relative lack of depth and currentness of "[its] catalog of 20,000 streaming movies", about a fifth of its DVD list. But he forgets to point out the more encompassing the catalog, the more valuable a good recommendation engine. Having invested on its own early on, Netflix will not compete on financial resources alone, the factor which powers streaming catalog size.

Back to Tim Harford, the reader of these fillips will immediately savor the double irony of his introductory sentence.

As I said, our lack of time combines with the glut and utter unreliability of information sources to make recommendations indispensable and yet recommenders themselves may prove less than truthful. Second the best system ever found to insure recommendations are untainted, i.e. the free personal judgments passed by teachers over their students, is presented here as suspect for lacking a sustainable economic incentive.

The problem with giving a monetary incentive to recommenders is that it so easily undermines the value of their recommendations. Tim Harford's correspondent, no relationship of mine despite his signing his name "Phil C", recognizes that much as he suggests having the prospective employer pay his tutor only to shoot down the idea as creating unacceptable biases. Were Phil C rich enough to pay his tutor, it still would not work.

The latter case has been amply illustrated by the role played by credit rating agencies in the current financial crisis. As reported by Louise Story (***), a former Moody's employee, Eric Kolchinsky, "in 2007 [...] questioned how his firm rated C.D.O.'s" and "told [its] credit policy team in 2008 that the company's C.D.O. rating method was "irresponsible"". Gretchen Morgenson now reports "D Keith Johnson, a former president of Clayton Holdings, a company that analyses mortgage pools", made similar representations to its executive team (****). Its own CEO later conceded built-in conflicts of interest to the US Congress. His employee must have expected a reward for his timely warnings.

Instead Eric Kolchinsky got "suspended at Moody's in 2009" and "blacklisted by the private sector financial industry". Contrary to Phil C's tutor, Moody's did not seem to need any monetary incentive to propagate a strong negative rating on its whistleblower to anyone willing to listen.

For positive recommendations, Tim Harford's solution is to have the recommender take a small percentage of the compensation stream of the person recommended by the employer who accepted the recommendation. This approach neatly avoids introducing a bias for either party as the base of the recommender's reward, i.e. each year of employment, can be assumed to reflect the admission of a mutual benefit by both parties.

Such a scheme is quite different from the practice of real estate agents paid on commission, known to bias them towards getting a transaction signed even to the detriment of either party. First the rate Tim Harford suggested is very small, 1 per thousand instead of a few percents, and its base much smaller, making undue pressure to close not worth the effort. Second it is proportional to the duration of an agreement rather than a one time fee, mimicking the reason why rental managers paid as a percentage of the rent try to pick the best tenants to avoid problems down the line.

Nitpickers beware. If the labor market is itself biased, making it overly difficult for a party to break an agreement found hurtful in hindsight, a built in bias towards transactions reappears but the failure falls first on the market, not on the recommendation system.

Yet Tim Harford's model is flawed, which incidently shows his approach to economics to be falsifiable and hence scientific (1).

Academic recommenders do not need more incentives to give impartial recommendations. They are already paid a salary predicated on the tacit agreement they are to recommend their best students for free. Besides, each successful recommendation brings an increase to the recommender's reputation, a value it would be foolish to exchange for a revenue stream either too small or too corrupt. I may disagree with Yochai Benkler, it does not mean his argument according to which the monetizating of human interactions destroys their value is entirely without merit.

The issue behind monetizing recommendations is that it tends to lower trust and trust is the foundation of authority as a source of truth. Interestingly the latter now appears on the Internet to be trendier than mere popularity. This is the theme of Jenna Wortham's article on social search (*****). "Of course, your friends are not generating the amount of data that a company like Amazon may use to make its automatic recommendations". But, as computer science professor John Riedl says, "the tradeoff is that you will be more comfortable with the recommendation".

What Phil C's case actually reveals is but the sad side-effect tenure can have on an "old university tutor". Secure in his employment, a market failure from the perspective of compelling him to fulfill his duties, satiated by a surfeit of reputation, he is led to "complain when he has to write references" for free. Phil C should remind him getting students to listen to his laments for the sake of a recommendation should be enough of an incentive.

Unfortunately Tim Harford's introduction indirectly reveals a more serious issue. As Jenna Wortham underlines, Facebook Questions and Places "will help Facebook amass even more data on its users' tastes". Unless one goes to school in Newark, one should not expect Mark Zuckerberg to be benevolent. He is not shy about tapping his users' free work as David Gelles remarks apropos its "Like button", a popularity based "map of the internet" competing with Google's recommendation engine (******). Neither will he shy away from exploiting his users' valuable information.

Dear Phil, Facebook has information about you and no particular reason to resist sharing it with advertisers for its exclusive benefit.

Philippe Coueignoux

• (*) ........... Dear Economist, by Tim Harford (Financial Times) - September 25, 2010
• (**) ......... Shifting Online, Netflix Faces New Competition, by Verge G. Kopytoff (New York Times) - September 27, 2010
• (***) ....... A Former Moody's Executive, in Suing Firm, Claims He Was Blacklisted, by Louise Story (New York Times) - September 14, 2010
• (****) ..... Raters Ignored Proof of Unsafe Loans in Securities, Panel is Told, by Gretchen Morgenson (New York Times) - September 27, 2010
• (*****) ... Search Takes a Social Turn, by Jenna Wortham (New York Times) - September 13, 2010
• (******) . E-commerce takes instant liking to Facebook's button, by David Gelles (Financial Times) - September 21, 2010
• (1) for more details, see Falsifiability in the wikipedia