The ecology of human communication is face to face. In these contexts, speakers dynamically modify their communication across vocal (e.g., speaking rate) and gestural (e.g., cospeech gestures related in meaning to the content of speech) channels while speaking. What is the function of these adjustments? Here we ask whether speakers dynamically make these adjustments to increase communicative success, and decrease cognitive effort while speaking. We assess whether speakers modulate word durations and produce iconic (i.e., imagistically evoking properties of referents) gestures depending on the predictability of each word they utter. Predictability is operationalized as surprisal and computed from computational language models trained on corpora of child-directed, or adult-directed language. Using data from a novel corpus (Ecological Language Corpus) of naturalistic interactions between adult-child (aged 3-4), and adult-adult, we show that surprisal predicts speakers' multimodal adjustments and that some of these effects are modulated by whether the comprehender is a child or an adult. Thus, communicative efficiency applies generally across vocal and gestural communicative channels not being limited to structural properties of language or vocal modality. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

中文翻译：

---

针对儿童和成人的多模式语言的沟通效率。


人类交流的生态是面对面的。在这些背景下，说话者在说话时动态地修改声音（例如，语速）和手势（例如，与语音内容的含义相关的同声手势）通道的通信。这些调整的作用是什么？在这里，我们询问说话者是否动态地进行这些调整以提高沟通的成功率，并减少说话时的认知努力。我们评估说话者是否会根据他们说出的每个单词的可预测性来调节单词持续时间并产生标志性（即，形象地唤起所指对象的属性）手势。可预测性被操作为令人惊讶的，并通过在儿童导向或成人导向语言的语料库上训练的计算语言模型来计算。使用来自成人与儿童（3-4 岁）和成人与成人之间自然互动的新语料库（生态语言语料库）的数据，我们表明，惊讶可以预测说话者的多模态调整，并且其中一些影响是通过是否理解者是儿童或成人。因此，沟通效率通常适用于声音和手势沟通渠道，而不限于语言或声音模态的结构特性。 （PsycInfo 数据库记录 (c) 2024 APA，保留所有权利）。