Thursday 27 May 2021 | 13:00 BST
Jean-Rémi King (CNRS Researcher, Department of Cognitive Science of Ecole Normale Supérieure, Paris, France)
Deep learning has recently allowed major progress in complex tasks such as summarization and text completion. Do these algorithms process language similarly to humans, and is this similarity driven by systematic principles? Here, we investigate deep networks trained on (1) image/sound, (2) word and (3) sentence processing to test whether their activations linearly map onto human brain responses to text and speech, as recorded with magneto-encephalography (MEG, n=204), functional Magnetic Resonance Imaging (fMRI, n=589), and intracranial electrodes (n=176 patients, 20K electrodes). Our results show a systematic mapping between deep nets and the human brain: not only the two share similar dynamics and computational hierarchies, but the better the deep net is, the more it resembles brain responses, and the more it predicts subjects’ comprehension. These results suggest that heavily trained deep learning algorithms may, at least partially, converge to brain-like solutions to cognition.