Our official English website, www.x-mol.net, welcomes your
feedback! (Note: you will need to create a separate account there.)
A shared model-based linguistic space for transmitting our thoughts from brain to brain in natural conversations
Neuron ( IF 14.7 ) Pub Date : 2024-08-02 , DOI: 10.1016/j.neuron.2024.06.025 Zaid Zada 1 , Ariel Goldstein 2 , Sebastian Michelmann 1 , Erez Simony 3 , Amy Price 1 , Liat Hasenfratz 1 , Emily Barham 1 , Asieh Zadbood 4 , Werner Doyle 5 , Daniel Friedman 5 , Patricia Dugan 5 , Lucia Melloni 5 , Sasha Devore 5 , Adeen Flinker 6 , Orrin Devinsky 5 , Samuel A Nastase 1 , Uri Hasson 1
Neuron ( IF 14.7 ) Pub Date : 2024-08-02 , DOI: 10.1016/j.neuron.2024.06.025 Zaid Zada 1 , Ariel Goldstein 2 , Sebastian Michelmann 1 , Erez Simony 3 , Amy Price 1 , Liat Hasenfratz 1 , Emily Barham 1 , Asieh Zadbood 4 , Werner Doyle 5 , Daniel Friedman 5 , Patricia Dugan 5 , Lucia Melloni 5 , Sasha Devore 5 , Adeen Flinker 6 , Orrin Devinsky 5 , Samuel A Nastase 1 , Uri Hasson 1
Affiliation
Effective communication hinges on a mutual understanding of word meaning in different contexts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We developed a model-based coupling framework that aligns brain activity in both speaker and listener to a shared embedding space from a large language model (LLM). The context-sensitive LLM embeddings allow us to track the exchange of linguistic information, word by word, from one brain to another in natural conversations. Linguistic content emerges in the speaker’s brain before word articulation and rapidly re-emerges in the listener’s brain after word articulation. The contextual embeddings better capture word-by-word neural alignment between speaker and listener than syntactic and articulatory models. Our findings indicate that the contextual embeddings learned by LLMs can serve as an explicit numerical model of the shared, context-rich meaning space humans use to communicate their thoughts to one another.
中文翻译:
一个基于模型的共享语言空间,用于在自然对话中将我们的思想从一个大脑传递到另一个大脑
有效的沟通取决于对不同语境中词义的相互理解。我们使用皮层电图记录了五对癫痫患者自发的面对面对话期间的大脑活动。我们开发了一个基于模型的耦合框架,将说话者和听者的大脑活动与大型语言模型 ( LLM ) 的共享嵌入空间对齐。上下文相关的LLM嵌入使我们能够逐字跟踪自然对话中从一个大脑到另一个大脑的语言信息交换。语言内容在单词发音之前出现在说话者的大脑中,并在单词发音之后迅速重新出现在听者的大脑中。上下文嵌入比句法和发音模型更好地捕捉说话者和听者之间逐字的神经对齐。我们的研究结果表明, LLMs学习的上下文嵌入可以作为人类用来相互交流思想的共享的、上下文丰富的意义空间的明确数值模型。
更新日期:2024-08-02
中文翻译:
一个基于模型的共享语言空间,用于在自然对话中将我们的思想从一个大脑传递到另一个大脑
有效的沟通取决于对不同语境中词义的相互理解。我们使用皮层电图记录了五对癫痫患者自发的面对面对话期间的大脑活动。我们开发了一个基于模型的耦合框架,将说话者和听者的大脑活动与大型语言模型 ( LLM ) 的共享嵌入空间对齐。上下文相关的LLM嵌入使我们能够逐字跟踪自然对话中从一个大脑到另一个大脑的语言信息交换。语言内容在单词发音之前出现在说话者的大脑中,并在单词发音之后迅速重新出现在听者的大脑中。上下文嵌入比句法和发音模型更好地捕捉说话者和听者之间逐字的神经对齐。我们的研究结果表明, LLMs学习的上下文嵌入可以作为人类用来相互交流思想的共享的、上下文丰富的意义空间的明确数值模型。