We’re looking for a strong candidate for a fully funded four-year PhD position on a collaborative project with Microsoft Research Cambridge. The research will focus on the development of new algorithms for leveraging data reuse in order to efficiently evaluate and optimize the behavior of information retrieval systems. See this page for the advertisement, further requirements, and conditions. The deadline for applications is March 22, 2015.
José van Dijck, Johan Oomen and I obtained an NWO Creative Industries grant to work on next generation search engine technologies for for exploring large multimedia archives. The target users are media-professionals. The proposed innovations at the interface of computer science and media studies come in three kinds. First, we will develop, test and release self-learning search algorithms that adapt and improve their behavior while being used. Second, we will create robust methods for semantically analyzing content in media archives. Third, we will develop new search engine result page presentations that provide automatically generated storylines as narratives for professionals in the creative industries. The algorithmic solutions will be implemented in the research environment of the Netherlands Institute for Sound and Vision and released as open source search solutions.
Now playing: Bright Eyes — Messenger Bird’s Song
Our ACM Transactions on Information Systems paper called “A comparative analysis of interleaving methods for aggregated search” by Aleksandr Chuklin, Anne Schuth, Ke Zhou and Maarten de Rijke is available online now.
A result page of a modern search engine often goes beyond a simple list of “ten blue links.” Many specific user needs (e.g., News, Image, Video) are addressed by so-called aggregated or vertical search solutions: specially presented documents, often retrieved from specific sources, that stand out from the regular organic web search results. When it comes to evaluating ranking systems, such complex result layouts raise their own challenges. This is especially true for so-called interleaving methods that have arisen as an important type of online evaluation: by mixing results from two different result pages interleaving can easily break the desired web layout in which vertical documents are grouped together, and hence hurt the user experience.
We conduct an analysis of different interleaving methods as applied to aggregated search engine result pages. Apart from conventional interleaving methods, we propose two vertical-aware methods: one derived from the widely used Team-Draft Interleaving method by adjusting it in such a way that it respects vertical document groupings, and another based on the recently introduced Optimized Interleaving framework. We show that our proposed methods are better at preserving the user experience than existing interleaving methods while still performing well as a tool for comparing ranking systems. For evaluating our proposed vertical-aware interleaving methods we use real world click data as well as simulated clicks and simulated ranking systems.