ConverSE: Conversational Search Engines

Updating the probability of relevance on the basis of a conversation, while deciding whether a further clarification is necessary to better determine what resource is relevant.

Searching for information in digital repositories plays a central role in today’s life. With the amount of available data growing rapidly daily, we rely on intelligent agents, search engines, to retrieve relevant information. Information Retrieval has come a long way during the past decades, demonstrated eminently by the success of web search engines. Web search technology, exploiting the abundance of signals from users on the Web, excels at locating a handful of highly relevant results. They achieve that by learning a general, static model of relevance from past user tasks which is used autonomously to predict relevance in similar future tasks.

There are information seeking tasks however that require developing distinct models of relevance for each user’s task. In domains such as evidence-based medicine, law or intelligence analysis users’ information needs are unique, tasks are multi-faceted, relevance changes during the user’s search process, and users agonize over enunciating their needs. A general and static model of relevance cannot tackle these challenges. The new grand challenge in information retrieval is modelling relevance under such constraints.

I propose a research program aiming to reform the way relevance is being modelled. Rather than learning a static model, I aim to design algorithms that learn for each individual task with the help of human interaction, by coupling human input and machines during the learning process. I propose models conditioned on context accrued both in active and passive mode. All research results will be evaluated in international evaluation campaigns, some of which I coordinate, and demonstrated with a search engine tailored to challenging use cases in law, patents, empirical medicine and investigative journalism. The scientific outcome of this research will be a set of algorithms that model relevance for each distinct task and a methodology to evaluate their quality.

Scientific Output

  • Jie Zou, Jimmy Xiangji Huang, Zhaochun Ren, Evangelos Kanoulas, “Learning to Ask: Conversational Product Search via Representation Learning”. 2022. ACM Transactions on Information Systems (TOIS) 
  • Jie Zou, Mohammad Aliennejadi, Evangelos Kanoulas. Maria Soledad Pera, Yiqun Liu. Users Meet Clarifying Questions: Toward a Better Understanding of User Interactions for Search Clarification. 2022. ACM Transactions on Information Systems (TOIS) 
  • Svitlana Vakulenko, Evangelos Kanoulas, Maarten de Rijke. A Large Scale Analysis of Mixed Initiative in Information-Seeking Dialogues for Conversational Search.  2021. ACM Transactions on Information Systems (TOIS), (publisher’s link)
  • Pengjie Ren, Zhumin Chen, Zhaochun Ren, Evangelos Kanoulas, Christof Monz, and Maarten de Rijke. Conversations with Search Engines. 2021. ACM Transactions on Information Systems (TOIS), (publisher’s link)
  • Li, Dan, and Evangelos Kanoulas. “When to Stop Reviewing in Technology-Assisted Reviews: Sampling from an Adaptive Distribution to Estimate Residual Relevant Documents.” ACM Transactions on Information Systems (TOIS) 38.4 (2020): 1-36. (publisher’s link)
  • Jie Zou, Evangelos Kanoulas, “Towards Question-Based High-Recall Information Retrieval: Locating the Last Few Relevant Documents for Technology Assisted Reviews”, ACM Transactions on Information Systems (TOIS), 2020 (publisher’s link).
  • Antonis Minas Krasakis, Andrew Yates, Evangelos Kanoulas, “Zero-shot Query Contextualization for Conversational Search”. In Proceedings of the 45th international ACM SIGIR conference on Research and development in information retrieval (SIGIR ’22). ACM, New York, NY, USA.ArXiV; Publisher’s link
  • Jie Zou, Evangelos Kanoulas, Pengjie Ren, Zhaochun Ren, Aixin Sun, Cheng Long, “Improving Conversational Recommender Systems via Transformer-based Sequential Modelling”. In Proceedings of the 45th international ACM SIGIR conference on Research and development in information retrieval (SIGIR ’22). ACM, New York, NY, USA. Publisher’s link
  • Leif Azzopardi, Mohammad Aliannejadi and Evangelos Kanoulas, “Towards Building Economic Models of Conversational Search.” In Proceedings of the Advances in Information Retrieval – 44th European Conference on IR (ECIR’22). ArXiV
  • Julien Rossi, Svitlana Vakulenko and Evangelos Kanoulas, “VerbCL: A Dataset of Verbatim Quotes for Highlight Extraction in Case Law”, In Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM’21). ArXiv; Publisher’s link;
  • Mohammad Aliannejadi, Leif Azzopardi, Hamed Zamani, Evangelos Kanoulas, Paul Thomas and Nick Craswell, “Analyzing Mixed Initiatives and Search Strategies during Conversational Search”, In Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM’21). ArXiv;
  • Jie Zou, Evangelos Kanoulas, Yiqun Liu, “An Empirical Study on Clarifying Question-Based Systems”, In Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM ’20). ACM, New York, NY, USA. ArXiV; Publisher’s link; 
  • Antonios Minas Krasakis, Mohammad Aliannejadi, Nikos Voskarides and Evangelos Kanoulas, “Analysing the Effect of Clarifying Questions on Document Ranking in Conversational Search” In Proceedings of the 6th ACM SIGIR  International Conference on the Theory of Information Retrieval (ICTIR’20) (publisher’s link)
  • Georgios Sidiropoulos, Nikos Voskarides, Evangelos Kanoulas, “Knowledge Graph Simple Question Answering for Unseen Domains”, In Proceedings of the International Conference of Automated Knowledge Base Construction (AKBC’20) (publisher’s link)
  • Jie Zou, Yifan Chen and Evangelos Kanoulas, “Towards Question-based Recommender Systems”. In Proceedings of the 43rd international ACM SIGIR conference on Research and development in information retrieval (SIGIR ’20). ACM, New York, NY, USA. (publisher’s link)
  • Dan Li, Panagiotis Zafeiriadis and Evangelos Kanoulas, “APS: An Active PubMed Search System for Technology Assisted Review”. In Proceedings of the 43rd international ACM SIGIR conference on Research and development in information retrieval (SIGIR ’20). ACM, New York, NY, USA. (publisher’s link)
  • Svitlana Vakulenko, Evangelos Kanoulas and Maarten de Rijke, “An Analysis of Mixed Initiative and Collaboration in Information-Seeking Dialogues”. In Proceedings of the 43rd international ACM SIGIR conference on Research and development in information retrieval (SIGIR ’20). ACM, New York, NY, USA. (publisher’s link)C54. Nikos Voskarides, Dan Li,
  • Pengjie Ren, Evangelos Kanoulas and Maarten de Rijke, “Query Resolution for Conversational Search with Limited Supervision”. In Proceedings of the 43rd international ACM SIGIR conference on Research and development in information retrieval (SIGIR ’20). ACM, New York, NY, USA. (publisher’s link)
  • Julien Rossi and Evangelos Kanoulas. Legal information retrieval with generalized language models. Proceedings of the 6th Competition on Legal Information Extraction 2020 (COLLIE ‘20). (pdf)
  • Julien Rossi and Evangelos Kanoulas: Legal Search in Case Law and Statute Law.  In Proceedings of the 32nd International Conference on Legal Knowledge and Information Systems (JURIX ‘19).
  • Jie Zou and Evangelos Kanoulas: Learning to Ask: Question-based Sequential Bayesian Product Search. In Proceedings of the 28th ACM International Conference on Conference on Information and Knowledge Management (CIKM ’19). ACM, New York, NY, USA. (publisher’s link).
  • David van Dijk, Marco Ferrante, Nicola Ferro and Evangelos Kanoulas. “A Markovian Approach to Evaluate Session-based IR Systems”. In Proceedings of the Advances in Information Retrieval – 41st European Conference on IR (ECIR’19). (publisher’s link)