Cascading non-stationary bandits: Online learning to rank in the non-stationary cascade model by Chang Li and Maarten de Rijke is online now at this location.

In the paper, we argue that non-stationarity appears in many online applications such as web search and advertising. We study the online learning to rank problem in a non-stationary environment where user preferences change abruptly at an unknown moment in time. We consider the problem of identifying the K most attractive items and propose cascading non-stationary bandits, an online learning variant of the cascading model, where a user browses a ranked list from top to bottom and clicks on the first attractive item. We propose two algorithms for solving this non-stationary problem: CascadeDUCB andCascadeSWUCB. We analyze their performance and derive gap-dependent upper bounds on the $n$-step regret of these algorithms. We also establish a lower bound on the regret for cascading non-stationary bandits and show that both algorithms match the lower bound up to a logarithmic factor. Finally, we evaluate their performance on a real-world web search click dataset.

  • Chang Li and Maarten de Rijke. Cascading non-stationary bandits: Online learning to rank in the non-stationary cascade model. In IJCAI 2019: Twenty-Eighth International Joint Conference on Artificial Intelligence, page 2859–2865, August 2019. Bibtex, PDF
    @inproceedings{li-2019-cascading,
    Author = {Li, Chang and de Rijke, Maarten},
    Booktitle = {IJCAI 2019: Twenty-Eighth International Joint Conference on Artificial Intelligence},
    Date-Added = {2019-05-30 22:36:52 +0200},
    Date-Modified = {2019-08-04 15:53:45 +0200},
    Month = {August},
    Pages = {2859--2865},
    Title = {Cascading non-stationary bandits: Online learning to rank in the non-stationary cascade model},
    Year = {2019}}

Other papers and presentations at IJCAI are part of the SCAI workshop:

  • Jiahuan Pei, Arent Stienstra, Julia Kiseleva, and Maarten de Rijke. SEntNet: Source-aware Recurrent Entity Networks for Dialogue Response Selection. In 4th International Workshop on Search-Oriented Conversational AI (SCAI), August 2019. Bibtex, PDF
    @inproceedings{pei-2019-sentnet,
    Author = {Pei, Jiahuan and Stienstra, Arent and Kiseleva, Julia and de Rijke, Maarten},
    Booktitle = {4th International Workshop on Search-Oriented Conversational AI (SCAI)},
    Date-Added = {2019-06-06 11:55:06 +0200},
    Date-Modified = {2019-06-06 11:56:16 +0200},
    Month = {August},
    Title = {SEntNet: Source-aware Recurrent Entity Networks for Dialogue Response Selection},
    Year = {2019}}
  • Yangjun Zhang, Pengjie Ren, and Maarten de Rijke. Improving Background Based Conversation with Context-aware Knowledge Pre-selection. In 4th International Workshop on Search-Oriented Conversational AI (SCAI), August 2019. Bibtex, PDF
    @inproceedings{zhang-2019-improving,
    Author = {Zhang, Yangjun and Ren, Pengjie and de Rijke, Maarten},
    Booktitle = {4th International Workshop on Search-Oriented Conversational AI (SCAI)},
    Date-Added = {2019-06-06 11:53:36 +0200},
    Date-Modified = {2019-06-06 11:55:01 +0200},
    Month = {August},
    Title = {Improving Background Based Conversation with Context-aware Knowledge Pre-selection},
    Year = {2019}}

  • Maarten de Rijke and Pengjie Ren. SERP-based Conversations. In 4th International Workshop on Search-Oriented Conversational AI (SCAI), August 2019.