Pascal Mettes

My research goal is to find a bridge between prior knowledge and deep networks. Deep learning in computer vision thrives under examples but commonly ignores additional explicit knowledge about the research problem. Think about hierarchical relations between categories, relational knowledge between tasks, or spatio-temporal knowledge. The research of my team focuses on discovering the shared geometry between pixels and knowledge. Specifically, we research hyperbolic and hyperspherical geometry for deep learning and learning with prototypes. Our advances and new research proposals are investigated for computer vision problems such as hierarchical recognition, segmentation, localization, and zero-shot recognition in images and videos.

Recent news highlights

[08-2023] "Poincaré ResNet" accepted to ICCV 2023.
[08-2023] "HypLL: The Hyperbolic Learning Library" is accepted to ACM Multimedia 2023.
[06-2023] HypLL: The Hyperbolic Learning Library is now available! [github] [paper]
[06-2023] We are organizing a new tutorial on hyperbolic learning at CVPR 2023.
[06-2023] I am giving a keynote at the CVPR 2023 anti-UAV workshop.
[05-2023] Our survey on Hyperbolic Deep Learning in Computer Vision is available online.
[02-2023] The recordings of the ECCV 2022 hyperbolic tutorial is now available online.
[11-2022] "Maximum Class Separation as Inductive Bias in One Matrix" accepted to NeurIPS 2022.
[10-2022] We are organizing a tutorial on Hyperbolic Representation Learning at ECCV 2022.
[08-2022] "Less than Few: Self-Shot Video Instance Segmentation" accepted to ECCV 2022.
[09-2022] We are organizing the second Video Symposium in Amsterdam.
[06-2022] "Hyperbolic Image Segmentation" accepted to CVPR 2022.
[04-2022] We are organizing the Netherlands Conference on Computer Vision 2022.

Organization and network

• 2023 - CVPR workshop on Hyperbolic Deep Learning for Computer Vision.
• 2023 - Area chair for NeurIPS 2023
• 2023 - Area chair BMVC (since 2020)
• 2022 - ECCV workshop on Hyperbolic Representation Learning for Computer Vision
• 2022 - Area chair for CVPR 2023
• 2022 - Video Understanding Symposium in Amsterdam
• 2022 - Panelist at the International Conference on Multimedia Retrieval
• 2022 - Netherlands Conference on Computer Vision
• 2021 - Member of ELLIS - European Laboratory for Learning and Intelligent Systems
• 2021 - ICCV workshop on Structured Representations for Video Understanding
• 2016 - ACM Multimedia web chair

Recent talks

• 2023 - Keynote at CVPR 2023 anti-UAV workshop, Canada
• 2023 - Talk at eScience Center, the Netherlands
• 2023 - Talk at Science Park Imaging Symposium, the Netherlands
• 2023 - Talk at FruitPunch AI, the Netherlands
• 2023 - Talk at Rising Star Symposium at KAUST, Saudi Arabia
• 2022 - Lecturer at School in AI summer school, Italy
• 2022 - Talk at University of Bristol, UK. Host: prof. Dima Damen
• 2022 - Talk at Columbia University, USA. Host: dr. Carl Vondrick
• 2022 - Talk at FAIR, USA. Host: dr. Karen Ullrich
• 2022 - Talk at Utrecht University, the Netherlands. Host: dr. Ronald Poppe
• 2022 - Talk at Delft University of Technology, the Netherlands. Host: dr. Jan van Gemert
• 2021 - Lecturer at Efficient Deep Learning winter school on Hyperbolic Deep Learning
• 2021 - Talk at TU Dresden, Germany. Host: prof. Martin Keller-Ressel
• 2021 - Talk at University of Tübingen, Germany. Host: prof. Zeynep Akata
• 2021 - Talk at Diagnostic Image Analysis Group, Radboud UMC, the Netherlands
• 2021 - Speaker at Bertelsmann Video AI Event, virtual

Grants

• PI - ELLIS PhD award with prof. Rita Cucchiara
• PI - NWO ClickNL
• PI - Google Perception Academic Funding with dr. Thomas Mensink
• PI - Data Science Center grant with dr. Erwin Berkhout
• PI - Informatics Institute collaboration grant with prof. Paul Groth
• co-PI - HAVA-Lab with prof. Cees Snoek
• co-PI - NWO ClickNL with dr. Nanne van Noord
• co-PI - ACTA AiO competition with dr. Erwin Berkhout

Awards

• Doctoral Consortium Award - ACM Multimedia 2016
• Benchmark winner - TRECVID Multimedia Event Detection 2015
• Reviewer award - CVPR21, ICLR21, ECCV20, ICML20, NeurIPS19, ICMR18, ICCV17, ICMR15

Teaching

• Applied Machine Learning 2018-now
• Education Committee MSc Information Studies 2019-now
• Thesis AI coordinator 2019-2022
• Entry committee MSc Artificial Intelligence 2019-2021
• Guest lecturer in Computer Vision I and Computer Vision II 2018-2021

Hyperspherical Prototype Networks

Code and pre-computed prototypes are available [here].

Shuffled ImageNet-Banks for Video Event Detection and Search

Pre-trained models can be downloaded by following the instructions [here].

Spatial-Aware Object Embeddings for Zero-Shot Localization and Classification of Actions

Code and pre-computed prototypes are available [here].

Water Detection through Spatio-Temporal Invariant Descriptors

The complete Video Water Dataset is available [here] (warning 14GB).