Shimon Whiteson's Publications

Sorted by DateClassified by Publication Type

A Probabilistic Method for Inferring Preferences from Clicks

Katja Hofmann, Shimon Whiteson, and Maarten de Rijke. A Probabilistic Method for Inferring Preferences from Clicks. In CIKM 2011: Proceedings of the Twentieth Conference on Information and Knowledge Management, pp. 249–258, October 2011.

Download

[PDF]634.4kB  

Abstract

Implicit feedback, such as clicks on documents in a result list, is an increasingly popular alternative to explicit relevance judgments. Previous work has shown that so-called interleaved comparison methods can utilize click data to detect small differences between rankers and can be applied to learn ranking functions online. In this paper, we analyze three existing interleaved comparison methods and find that they are all either biased or insensitive to some differences between rankers. To address these problems, we present a new method based on a probabilistic interleaving process. We derive an unbiased estimator of comparison outcomes and show how marginalizing over possible comparison outcomes given the observed click data can make this estimator even more effective. We validate our approach using a recently developed simulation framework based on a learning to rank dataset and a model of click behavior. Our experiments confirm the results of our analysis and show that our method can reliably identify the preferred ranker using significantly less click data than existing methods.

BibTeX Entry

@InProceedings{hofmann:cikm11,
  author       = "Katja Hofmann and Shimon Whiteson and Maarten de Rijke",
  title	       = "A Probabilistic Method for Inferring Preferences from Clicks",
  booktitle    = "CIKM 2011: Proceedings of the Twentieth Conference on Information and Knowledge Management",
  month	       = "October",
   pages	       = "249-258",    
  year	       = 2011,
}

Generated by bib2html.pl (written by Patrick Riley ) on Mon Aug 03, 2015 15:24:58