Herke van Hoof is currently assistant professor at the University of Amsterdam in the Netherlands. He is part of the Amlab headed by Professor Max Welling as well as the UvA-Bosch Delta lab.  Herke works on machine learning for autonomous robots in perceptually challenging environments. For robots to master a wide array of tasks, machine learning is indispensable, but it is equally important that such tasks can be learned in non-standardized and unstructured environments such as homes or hospitals. Learning tasks in such complicated environment puts additional demands on algorithms for machine learning, perception, and control.

One example of such a task is exploring the objects present in a novel environment. Segmenting objects using passive sensing is inherently limited. By interacting with the environment, the robot can improve its understanding of the different objects that are present. However, interaction is costly. By expressing the uncertainty in the robot’s understanding of the world, it becomes possible to select actions based on the information they are expected to yield about the environment, and thus speed up the learning progress.

In another project, we consider reinforcement learning with high-dimensional inputs. Current approaches have usually tried to learn features in a separate step. However, such features cannot be informed by what is relevant for the task at hand. We have taken a complementary approach, where we have developed a non-parametric reinforcement learning method that only depends on the similarity between data-points, independent of the embedding dimensionality.

Currently, I’m working on new ways to exploit known robot models and/or simulators to make reinforcement learning more efficient. I am looking to use a generative model of the robot to characterise its belief over unknown parameters, and pre-training a policy that learns to trade-off exploration and exploitation based on this characterisation.

Before joining the University of Amsterdam, Herke van Hoof was a postdoc at McGill University in Montreal, Canada, where he worked with Professors Joelle Pineau, Dave Meger, and Gregory Dudek. He obtained his PhD at TU Darmstadt, Germany, under the supervision of Professor Jan Peters, where he graduated in November 2016. Herke got his bachelor and master degrees in Artificial Intelligence at the University of Groningen in the Netherlands.

Recent News

  • Open PhD position (3/10/2020)

    We have an open PhD position on the topic of using symbolic knowledge representation as prior knowledge in reinforcement learning systems. This position is part of the Hybrid Intelligence centre, where we are currently recruiting 13 PhDs and postdocs across the Netherlands with another 14 positions to be announced soon. To directly apply for the position on hybrid approaches for learning sequential behavior, click here.

  • Wouter’s ICML paper published on-line (3/10/2020)

    Wouter Kool’s paper on ancestral Gumbel-top-K sampling for sampling without replacement from large structured domains is now published in JMLR (Kool, van Hoof & Welling, pdf). Congratulations, Wouter!

  • Two AAMAS papers accepted (1/15/2020)

    Jan Wöhlke and Elise van der Pol got their submissions accepted to AAMAS! Congrats Jan & Elise! Publication details will appear shortly on the Delta lab website.

An archive of news items can be found on the News page.

Key References

Kool, Wouter ; van Hoof, Herke ; Welling, Max

Estimating Gradients for Discrete Random Variables by Sampling without Replacement Forthcoming

International Conference on Learning Representations, Forthcoming.

Links | BibTeX

Kool, W; van Hoof, H; Welling, M

Ancestral Gumbel-Top-k Sampling for Sampling without Replacement

Journal of Machine Learning Research, 2020.

Links | BibTeX

Smith, M; van Hoof, H; Pineau, J

An Inference-Based Policy Gradient Method for Learning Options

International Conference on Machine Learning, pp. 4703-4712, 2018.

Links | BibTeX

Van Hoof, H; Neumann, G; Peters, J

Non-parametric Policy Search with Limited Information Loss

Journal of Machine Learning Research, 18 (73), pp. 1-46, 2017.

Links | BibTeX

van Hoof, Herke; Kroemer, Oliver; Peters, Jan

Probabilistic Segmentation and Targeted Exploration of Objects in Cluttered Environments

IEEE Transactions on Robotics (TRo), 5 , pp. 1198-1209, 2014.

Links | BibTeX

A full list of publications can be found at the Publications page.