Research

Visual data is becoming an integral part of society. For the general public visual data and its sharing through social media has become a way of living and is leading to a new digital culture. Scientists use cameras to record observations as source for scientific discoveries. Criminals have embraced the Internet to distribute illegal and disturbing content. Visual collections and their contextual data potentially contain a wealth of information which can range from scientific discoveries, image demographics, trends, or preferences, to forensic evidence and intelligence. Automatic algorithms are inferior to the human capabilities of identifying visual semantics or subtle patterns. Yet image collections are too voluminous to be processed by humans.

In MultiX we research multimedia analytics by developing AI techniques for getting the richest information possible from the data (image/video/text/graphs) interactions surpassing human and machine intelligence, and visualizations blending it all in effective interfaces for applications in health, forensics and law enforcement, cultural heritage, urban livability, and social media analysis.

Group Members

Full professors

Associate Professors

Assistant Professors

PostDocs

  • Merel de Leeuw den Bouter, Deep Fake Detection

    • Current Phd students

    • Meike Kombrink Detection of Steganography (at NFI)
    • Shuai Wang Business Theory Driven Multimodal (Hyper)graph learning, and its applications in the Cultural Industry and Science (at ABS)
    • Amir Soleimani : Natural Language Processing for Fraud Analytics
    • Floris Gisolf: Interactive analysis of large scale visual incident data
    • Sarah Ibrahimi : Finding relations in Law Enforcement multimedia data
    • Jia-Hong Huang : Video Summarization and its applications in journalism
    • Maarten Sukel : AI for the city
    • Eleni Konstantina Sergidou speaker verification (at NFI)
    • Jiayi Shen, Variational Multi-Task learning
    • Ivona Najdenkoska, Vision and Language Modelling for Report Generation and Image Captioning
    • Tom van Sonsbeek, Joint Learning from Electronic Health Records and Medical Images
    • Thanos Efthimiou, Determining the value of art using graph convolutional networks (at ABS)
    • Ujjwal Sharma Harnessing multi-modal urban data for intelligent location analytics for retail establishments (at ABS)
    • Robert Bwana, Crowdsourcing App for Responsible Production in Africa (at ABS)

    PhD students that received their degree under my full or partial guidance:

    • Andrea Macarulla , Face Comparison in Forensics: A Deep Dive into Deep Learning and Likelihood Ratios (with NFI), February 2024
    • Devanshu Arya : Multimodal Deep Learning on Hypergraphs, June 2022
    • Gjorgji Strezoski : Information Sharing Methods for Multi-Task Learning, May 2022
    • Gosia Migut: Integration of Machine Learning and Interactive Visualizations for Cognition Friendly Decision Making, Nov 2019 (now at TU-Delft)
    • Ork de Rooij: Interactive Content-Based Visualizations for Multimedia Search, October 2017 (now at Qualcomm)
    • Jan Zahalka The Machine in Multimedia Analytics, July 2017 (now at Czech Technical University)
    • Fangbin Liu: High Performance Adaptive Image Processing on Multi-Scale Hybrid Architectures, Nov 2015, (now at Park Now Group)
    • Dang Trung Kien: Semi-interactive construction of 3D event logs for scene investigation. May 2013
    • Xirong Li: Content-based visual search learned from social media, March 2012, (now at Renmin University, China)
    • Giang Nguyen "Interactive Image Search using Similarity-Based Visualization, December 2006, (now at Track Unit, Denmark)"
    • Laura Hollink "Semantic annotation for retrieval of visual resources, November 2006, (now at CWI)"
    • Cees Snoek "The Authoring Metaphor to Machine Understanding of Multimedia, October 2005, (now at University of Amsterdam)"
    • Andy Bagdanov: "Style Characterization of Machine Printed Text", May 2004, (now at University of Firenze).
    • Jeroen Vendrig: "Interactive Exploration of Visual Content", October 2002, (Canon, now at Prooftec).
    • Tat Hieu Nguyen: "Segmentation of Video Into Spatio-Temporal Objects", march 2001, (now at Boeing)