Experiential, Distributional and Dependency-based Word Embeddings have Complementary Roles in Decoding Brain Activity

Samira Abnar, Rasyan Ahmed, Max Mijnheer, Willem Zuidema


We evaluate 8 different word embedding models on their usefulness for predicting the neural activation patterns associated with concrete nouns. The models we consider include an experiential model, based on crowd-sourced association data, several popular neural and distributional models, and a model that reflects the syntactic context of words (based on dependency parses). Our goal is to assess the cognitive plausibility of these various embedding models, and understand how we can further improve our methods for interpreting brain imaging data.

We show that neural word embedding models exhibit superior performance on the tasks we consider, beating experiential word representation model. The syntactically informed model gives the overall best performance when predicting brain activation patterns from word embeddings; the GloVe distributional method gives the overall best performance when predicting in the reverse direction (words vectors from brain images). Interestingly, however, the error patterns of these different models are markedly different. This may support the idea that the brain uses different systems for processing different kinds of words. Moreover, we suggest that taking the relative strengths of different embedding models into account will lead to better models of the brain activity associated with words.

Language, art and music are extremely revealing about workings of the human mind

I was interviewed by Gisela Govaart about my research. The interview is published online here.


Language, art and music are extremely revealing about workings of the human mind” – An interview with Jelle Zuidema
by Gisela Govaart, January 2016

Jelle Zuidema is assistant professor in cognitive science and computational linguistics at the Institute for Logic, Language and Computation. He does research on these topics, coordinates the Cognition, Language and Computation lab and supervises five PhD and several MSc students there. He teaches in the interdisciplinary master’s programs Brain & Cognitive Sciences (MBCS), Artificial Intelligence, and Logic, and coordinates the Cognitive Science track in the MBCS. Jelle was the organizer of the SMART CS events from 2011 until 2015.

Jelle Zuidema

“I started my studies with two programs in parallel at the University of Utrecht: Liberal Arts – where I focused on Literature – and Artificial Intelligence. In my final two years I dropped Liberal Arts, because I decided I needed to specialize; I got my degree in AI, with a specialization in Theoretical Biology. My thesis was on Evolution of Language, so it was a rather weird mix. I was first interested in evolution, and then my supervisor suggested: since you have this background in computational linguistics and logic, why don’t you look at the evolution of language. So it was a bit accidental, but immediately things started to fall into place, and I got really excited about the topic, and decided that I wanted to do my PhD on that as well. For my PhD I moved first briefly to Paris, and then I was in Brussels for two years, in the group of Luc Steels. After two years Brussels I moved to Edinburgh, and I actually got my PhD degree from the University of Edinburgh in the group of Simon Kirby.”

Continue reading