This week I will talk about my research at the 16th International Conference on Informatics and Information Technologies. We will cover computer vision and multi-task learning fundamentals as well as state of the art approaches in the these fields. You can find more details about the talk and complete conference programme on the CIIT website .
Our work on exploting secondary latent features for task grouping got accepted for oral presentation in ICMR 2019 in Ottawa, Canada. This paper introduces Selective Sharing, a method using the factorized gradients per task as a signal that helps in grouping tasks that benefit eachother’s learning process. The grouping is conditioned on a predefined metric so different strategies can be explored. We are preparing the repo for the code release and the site will be updated with a link to the official proceedings.
Over the past two years my research interests have revolved arround Multi-Task Learning (MTL) as a learning paradigm. It is a vast field of diverse research in all domains of computer science from NLP and Signal Processing to Computer Vision and Multimedia. In what follows I will motivate, describe and discuss an approach to MTL we developed called Task Routing. Multi-Task and Many-Task Learning By definition (Carruana 1997), multi-task learning is a learning paradigm that seeks to improve the generalization performance of machine learning models by optimizing for more than one task simultaneously.
Our favorite artistic dataset is published in ACM TOMM V.14 Issue 4, November 2018. Baselines are the starting point of any quantitative multimedia research, and benchmarks are essential for pushing those baselines further. In this article, we present baselines for the artistic domain with a new benchmark dataset featuring over 2 million images with rich structured metadata dubbed OmniArt. OmniArt contains annotations for dozens of attribute types and features semantic context information through concepts, IconClass labels, color information, and (limited) object-level bounding boxes.
The VISTORY Project on the cover of I/O Magazine in a featured artcile The Science of Art. Behind the façade of the majestic Ateliergebouw in Amsterdam you can find a research institute that is unique in the world. At this Netherlands Institute for Conservation+Art+Science+ (NICAS), art historians, conservators, physicists, chemists, mathematicians and ICT researchers work together to better understand, access and preserve cultural heritage. Check out the full article on I/O Magazine’s website or order a printed copy.
Using the millions of images contained in the OmniArt dataset we created humanity’s average artwork. After averaging every painting, sculpture, installation, costume, figurine and photograph in the dataset we came up with a centrally symmetrical brown blur. This image can be interpreted in many different way. Having the central and peripheral lighter regions can signify the common light directions in painting and photography. Light can come to the center as a focus point in portraits or from the corners or top in landscapes.
The ICT.OPEN event is organised annually by the Netherlands Organisation for Scientific Research (NWO) under the auspices of ICT research Platform Netherlands (IPN). It is made to showcase Dutch ICT research and present active projects amongst which is the VISTORY project with the OmniArt dataset. Our latest work on the OmniArt benchmark is going to be presented there. Join us at our poster on post number 13 on March 19th and 20th in the Flint Theater in Amersfort, The Netherlands.
Object play a key role in understanding what is happening in an image. Using our object level annotation tool we have annotated 5000 data samples so far for objects that are not so common or are not easy to translate from real world images. For common objects that do not change their appearance significantly we can use the knowledge obtained from the real world. For example, bicycles have had the same primitive parts for a while now - two wheels, a frame, a seat and a power transfer mechanism from the feet to the wheels.
Today OmniArt hit the 2.5M mark in the number of data samples it contains. Our system is listening for changes as you are reading this post and expanding the dataset even further. In the current form, OmniArt features more than 2 million different faces in paintings, sketches and drawing. Our model’s gender estimation is that 70% of the faces are male and 30% are female. Female portraits mostly contain more than one person in their content while male subjects are mostly alone.
Deep models are at the heart of computer vision research recently. With a significant performance boost over conventional approaches, it is relatively easy to treat them like black boxes and enjoy the benefits they offer. However, if we are to improve and develop them further, understanding their reasoning process is key. Motivated by making the understanding process effortless both for the scientists who develop these models and the professionals using them, in this paper, we present an interactive plug&play web based deep learning visualization system.