Ev Fedorenko keynote speaker at EMNLP 2021: "Language models will never think, but they help us to understand how the brain works"

Two days to go for EMNLP 2021, the international Conference on Empirical Methods in Natural Language Processing held from November 7 – 11. The conference will take place this year in hybrid form, with attendees participating both virtually and on-site in the Dominican Republic. EMNLP 2021 will deliver the most up-to-date insights on the state-of-the-art in the field. Just like every field in human life, also computational linguistics has to face the quick development of Deep Learning. 2015 was “like the year when the full force of the (Deepl Learning, ed.) tsunami hit the major Natural Language Processing (NLP) conferences” (Manning 2015).

CL will help to better understand the brain or is it the other way around? (Foto: Unsplash)

The relationship between CL and Deep Learning in the future will be dealth with from many different perspectives at the EMNLP 2021. This will be a record edition with 3,717 full papers submitted by 11,425 authors. 841 submissions were accepted to the main conference. Among them, 315 were accepted as oral papers, and 526 were accepted as posters. An answer will try to be given also by the three keynote speakers: Professors Ido Dagan (Main Conference on Sunday, November 7), Evelina Fedorenko (Monday, November 8) and Steven Bird (Tuesday, November 9). Thursday and Friday November 10-11 will be workshops on the stage. We at Innovation Nation have (virtually) met one of the main speakers. Of course we chose the female one, even though Evelina (Ev) Fedorenko claim is “what genitalia I have or how I identify is orthogonal to the science I do”. Ev Fedorenko is a 41, Russian-born American cognitive neuroscientist, head of Language Lab at MIT. She kindly accepted to enlighten us on how neurociensce and NLP can “augment” each other to develop a better comprehension of how language works in the brain and how to use this knowledge to find better NLP approaches. The goal of her research program is “to understand the representations and computations that enable us to share complex thoughts with one another via language, and their neural implementation. A decade ago, I developed a robust new approach to the study of language in the brain based on identifying language-responsive cortex functionally in individual participants” Fedorenko writes in the introduction of her keynotespeech “The Language System in the Human Brain”.

Evelina Fedorenko (Foto: McGovern Institute for Brain Research)

Professor Fedorenko, EMNLP is a conference on NLP and – marginally – on ML applied to language. Cognitive plausibility in this field has long been subordinated to computational performance. Should your presence as keynote speaker at EMNLP 2021 let us hope that finally the time has come for a more neurobiological/cognitive approach to NLP and ML?

“It’s hard for me to speak to this. It’s only in the last few years that I have started to engage seriously with the NLP/ML communities given the increasing integration between AI/ML and neuroscience, across domains. There have now been several cases where computational models that were not at all built to mimic anything about human cognition/behavior/brain have been shown to provide a good fit to human neural/behavioral data. This suggests that some of these models can be used as computationally explicit hypotheses about how human brains achieve different cognitive tasks. This to me is extremely exciting. How far the ‘reverse’ direction (i.e., effects from neuroscience to NLP/ML) will go is hard for me to say. But I am certainly excited to see the AI community and the animal/human neuroscience community interact increasingly more”.

In the framework built by the previous question and answer, what can Cognitive Neuroscience help in questioning Universal dependencies, inferences and noncategorical linguistic phenomena in general?

“Inferences is an interesting case. The human language system does not support inferential processing: the human language system learns linguistic regularities from the input and stores a set of (ever-changing) linguistic knowledge representations. These representations can be used to interpret linguistic signals and to generate linguistic utterances, but the actual thought takes place downstream (or upstream in production). So, one insight from cognitive neuroscience is that the language system is separate from the thought system(s), so one should not expect ANN language models to think/reason”.

One of your claims is: “The primary goal of language is efficient information transfer, rather than enabling complex thought”. Could we say that the very same push (information transfer, also in the animal, vegetal, unicellular realm) drives all life forms’ development?

“That’s probably too broad a claim. Physical and perceptual/motor/cognitive properties of living organisms are shaped by the environment and the kinds of problems the organism faces and needs to solve for survival. Different species vary widely in how important interaction/communication with conspecifics is. Of course, primates including humans are extremely social species; besides, because of our big brains, much of the brain development happens post birth, so we are totally helpless as babies, which means paying attention to conspecifics is critical for survival, which presumably means that our brains are built so as to be attuned to conspecifics and be able to track all sorts of social signals”.

What does it mean, for you, to be a #womaninscience?

“On the one hand, what genitalia I have or how I identify is orthogonal to the science I do. I am a scientist because I am passionate about language, and understanding the human brain. On the other hand, in the society we live in, being a woman in science carries with it a great responsibility: neuroscience, like many other fields of science, is still dominated by white males, so it’s important for the women who do make it to not only be great role models for aspiring young scientists, but also to work on lowering the barriers for women and other URMs to join the field, which can be done through public outreach, by training and supporting early career female and URM scientists, talking openly about the struggles women and other URMs face. I try my best to help in these ways”.