Our CLEF 2015 paper on evaluating click models for web search is online now:

  • Artem Grotov, Aleksandr Chuklin, Ilya Markov, Luka Stout, Finde Xumara, and Maarten de Rijke. A comparative study of click models for web search. In CLEF 2015. Springer, September 2015. Bibtex, PDF
    @inproceedings{grotov-comparative-2015,
    Author = {Grotov, Artem and Chuklin, Aleksandr and Markov, Ilya and Stout, Luka and Xumara, Finde and de Rijke, Maarten},
    Booktitle = {CLEF 2015},
    Date-Added = {2015-06-15 15:19:16 +0000},
    Date-Modified = {2015-06-20 11:05:55 +0000},
    Month = {September},
    Publisher = {Springer},
    Title = {A comparative study of click models for web search},
    Year = {2015}}

Click models have become an essential tool for understanding user behavior on a search engine result page, running simulated experiments and predicting relevance. Dozens of click models have been proposed, all aiming to tackle problems stemming from the complexity of user behavior or of contemporary result pages. Many models have been evaluated using proprietary data, hence the results are hard to reproduce. The choice of baseline models is not always motivated and the fairness of such comparisons may be questioned. In this study, we perform a detailed analysis of all major click models for web search ranging from very simplistic to very complex. We employ a publicly available dataset, open-source software and a range of evaluation techniques, which makes our results both representative and reproducible. We also analyze the query space to show what type of queries each model can handle best.