Thomas Mensink

News

jan'19 | In March I'll join Google Research in Amsterdam, I'll work with Cristian Sminchisescu in Zurich and Nal Kalchbrenner of the Amsterdam Brain team! I'll remain affiliated with the UvA.
jan'19 | Research on detecting fake videos (from the SAVI project with SRI and IDIAP) featured in MercuryNews
dec'18 | Two MSc AI projects (for UvA students): extending DeepNCM and encoding context in visual representations. See below for more information
dec'18 | New paper about generating optical flow ground-truth is available on arXiv, work with An, Tushar, Anil, Sezer and Theo
nov'18 | I helped answering the question do we need to be afraid of fake-news for Know-Shit (3FM Tussenuur)
nov'18 | I've been installed as one of the first members of the Amsterdam Young Academy
sep'18 | Added information about Applied Machine Learning and Project AI for 2018, see here
sep'18 | I'll lead the "Captioning News Footage" workshop in the ICT with Industy (jan 2019), together with Daan Odijk, see NWO website for more information
Want to join our team? Contact me!
aug'18 | Our paper about Emoji prediction (with Spencer and Cees) is now available under early access on IEEEXplore
jul'18 | Our paper Three for one and one for three: Flow, Segmentation, and Surface Normals is accepted for oral presentation at BMVC, toghether with An, Anil, and Theo
may'18 | The project Spotting Audio-Visual Inconsistencies with SRI and IDIAP aired in the news this week(s): including AG Connect (in Dutch), Tech Crunch, CBS news (with the program leader), and (again) AG Connect (in dutch, more in detail, (local pdf copy)).
mar'18 | Two papers accepted for ICLR Workshops:
- DeepNCM: Deep Nearest Class Mean Classifiers, with Samantha Guerriero and Barbara Caputo, see openreview and the code on GitHub
- IterGANs for Object Rotation, with Ysbrand Galama, see openreview, the extended version on arXiv, and source code on GitHub.
jan'18 | Fisher Vector code on Github
dec'17 | Rijksmuseum Challenge: code on github and data on figshare
aug'17 | Video recordings of ZSL 2017 are online
july'17 | Slides of the Zeros Shot Learning tutorial are online
july'17 | I've been awarded an outstanding reviewer award for CVPR 2017
july'17 | 5 MSc Information Studies students graduated this month, congrats!
july'17 | Samantha Guerriero (student from Barbara Caputo @ Sapienza in Rome) is visiting to work on metric learning
may'17 | Video with results of our ICMR paper is online
apr'17 | I will present at Spui25 on the topic of "quantifying love" (in dutch, April 19th).
apr'17 | ICMR paper accepted: "Music-Guided Video Summarization" (see pdf below), together with Thomas Jongstra, Pascal Mettes and Cees Snoek.
apr'17 | I have a MSc AI thesis project opening on historical inpainting by incorporating prior knowledge, see pdf. Drop me a line when interested!
apr'17 | Our CVPR'17 proposal for the Zero-shot Learning tutorial has been accepted
apr'17 | Our ICCV'17 proposal for the 5th Workshop on Web-scale Vision and Social Media (VSM) has been accepted
mar'17 | The automark software, to automatically check progress of python programming exercises, developped by Spencer is now available on GitHub!
feb'17 | Open position: 3D Computer Vision from a single image, (deadline closed), for future reference see (pdf).
feb'17 | I started as Assistant Professor in 3D Deep Learning in the Computer Vision group of Theo Gevers at the UvA.

Recent Publications

See full list of publications

le18arxiv

Unsupervised Generation of Optical Flow Datasets from Videos in the Wild

HoangAn Le, Tushar Nimbhorkar, Thomas Mensink, Sezer Karaoglu, Anil Baslamisli, Theo Gevers
| 2018 | ArXiV preprint (1812.01946).
[bibtex] [arxiv]

galama18arxiv

IterGANs: Iterative GANs to Learn and Control 3D Object Transformation

Ysbrand Galama, Thomas Mensink
| 2018 | ArXiV preprint (1804.05651), submitted to CVIU.
[bibtex] [code] [arxiv]

le18bmvc

Three for one and one for three: Flow, Segmentation, and Surface Normals

HoangAn Le, Anil Baslamisli, Thomas Mensink, Theo Gevers
In British Machine Vision Conference (BMVC) | 2018 | oral, acceptance rate 4.3%.
[bibtex] [pdf] [arxiv]

galama18iclrw

IterGANs: Iterative GANs for Rotating Visual Objects

Ysbrand Galama, Thomas Mensink
In International Conference on Learning Representations - Workshop (ICLRw) | 2018.
[bibtex] [pdf] [poster] [code] [arxiv]

guerriero18iclrw

DeepNCM: Deep Nearest Class Mean Classifiers

Samantha Guerriero, Barbara Caputo, Thomas Mensink
In International Conference on Learning Representations - Workshop (ICLRw) | 2018.
[bibtex] [pdf] [poster] [code]

cappallo18tmm

The New Modality: Emoji Challenges in Prediction, Anticipation, and Retrieval

Spencer Cappallo, Stacey Svetlichnaya, Pierre Garrigues, Thomas Mensink, Cees G. M. Snoek
In Transactions on Multi Media (TMM) | 2018 | In press.
[bibtex] [pdf] [arxiv] [doi]

Applied Machine Learning (AML)
Information for the year 2018-2019
- Currently 150 students are enrolled. And we're fully booked. So if you do not intend to take this course, please de-register so that others could
- There is no waitinglist and (late) enrollment is not possible anymore
- There is space for "sit-in" students, i.e., students who are interested in following the lectures, without credits.
  Sit-in students:
  1. Will be allowed to follow the lectures: Mon 17:00-19:00 and Wed 11:00-13:00, Science Park
  2. Will get access to all materials, discussion platform, assignments/exercises
  3. Will not have access to tutorials, lab sessions, due to space constraints
  4. Will not be able to do the exam
  5. Will not be entitled ECTS credits.
    In the exceptional case regular students drop out from class, some sit-in students might switch to credits, more info in week 2-3 of the course
  6. Should send an email to Jan Jetze Beitler with their name, student ID, and e-mail adress
- More infomration, including preliminary schedule and syllabus will be announced in october
Project AI, Project AI 2 and Project AI 3
Information for the year 2018-2019
- Project AI is a 4 week course in blok 6 (june). It is a project course, working in groups of 4 students. More info to follow (april/may).
- Project AI 2 and Project AI 3 are individual projects.
  Each project has to be approved by the Examinations Board. Ususally the procedure is as follows:
  1. You have a (rough) project idea
  2. You find a suitable supervisor
  3. Fill in the request to the Examinations Board, see student.uva.nl/ai/ ⇒ A-Z ⇒ Examinations Board ⇒ Bottom of page "Form individual project MSc"
    1. Togheter with your supervisor, ask an examinator
    2. Usually projects are 6ECTS, for large projects it could be 12 ECTS
    3. Requests take around 8 weeks for approval
  4. After approval, start the project
  5. After completion, the supervisor and examinator will grade your project
- Note: In total you're entitled to have a maximum of 12 ECTS of project courses (PAI, PAI2, and PAI3) on your study programme!
Teaching (Outdated)
- 2017 Computer Vision II, MSc AI
- 2016 Applied Machine Learning, MSc IS (HCM and DS track)
- 2013-2015 Visual Search Engines, MSc IS (HCM track)
Current Supervision (Outdated)
- MSc IS 2017: Daniël Bartolomé-Rojas, Diederik Beker, Michal Kozal, Yu-Ri Tan, Madli Uutma, Bastiaan Waanders, and Aeron Yung
- Msc AI 2017: Ysbrand Galema, Thomas Jongstra (finished March'17) and Sébastien Negrijn
- BSc AI 2017: Caitlin Lagrand

Teaching & Supervision

Open Projects

MSc AI thesis

Encoding Context in Visual Representations Abstract: The goal of this project is to study how context is encoded in visual representations. There is evidence that activation patterns in brain regions related to the visual observations are influenced by an external context, eg by music. So, depending on the style of music, the same visual stimulus yield different activation patterns. In this project, we aim to reproduce this behavior by training ConvNets with an additional context input. The goal is to observe to what extend we obtain similar results and how those insights can be used for further understanding of the brain.

This is a joint project with Jorrit Montijn (Netherlands Institute for Neuroscience, Google Scholar).

Prior knowledge:
- Followed Computer Vision 1 / Deep Learning
- Interest in Machine Learning
- Interest in Neuroscience
- Willing to work at both NIN and Science Park
- Experience with TensorFlow/PyTorch
Large Scale Visual Classification Using Class Means
Abstract: The final layer of (almost) any classification network is a soft-max layer. An alternative is to learn using class prototypes, eg the mean representation of each class. In this project we explore Deep Nearest Mean Classification for large scale classification. The starting point is the DeepNCM paper, but the goal is to extend DeepNCM and to explore training on large(r) datasets in the order of 3,000 classes and 10M training images.

Prior knowledge:
- Followed Computer Vision 1 / Deep Learning
- Interest in Machine Learning
- Experience with TensorFlow/PyTorch