OpenAI’s ChatGPT language model has gained popularity as a powerful tool for problem-solving and information retrieval. However, concerns arise about the reproduction of biases present in the language-specific training data. In this study, we address this issue in the context of the Israeli–Palestinian and Turkish–Kurdish conflicts. Using GPT-3.5, we employed an automated query procedure to inquire about casualties in specific airstrikes, in both Hebrew and Arabic for the former conflict and Turkish and Kurdish for the latter. Our analysis reveals that GPT-3.5 provides 34 ± 11% lower fatality estimates when queried in the language of the attacker than in the language of the targeted group. Evasive answers denying the existence of such attacks further increase the discrepancy. A simplified analysis on the current GPT-4 model shows the same trends. To explain the origin of the bias, we conducted a systematic media content analysis of Arabic news sources. The media analysis suggests that the large-language model fails to link specific attacks to the corresponding fatality numbers reported in the Arabic news. Due to its reliance on co-occurring words, the large-language model may provide death tolls from different attacks with greater news impact or cumulative death counts that are prevalent in the training data. Given that large-language models may shape information dissemination in the future, the language bias identified in our study has the potential to amplify existing biases along linguistic dyads and contribute to information bubbles.

中文翻译：

---

用户语言如何影响 ChatGPT 中的冲突死亡人数估计


OpenAI 的 ChatGPT 语言模型作为解决问题和信息检索的强大工具而广受欢迎。然而，人们担心特定语言的训练数据中存在的偏差会重现。在这项研究中，我们在以色列-巴勒斯坦和土耳其-库尔德冲突的背景下探讨了这个问题。使用 GPT-3.5，我们采用了自动查询程序来查询特定空袭中的伤亡情况，前者冲突使用希伯来语和阿拉伯语，后者使用土耳其语和库尔德语。我们的分析表明，当以攻击者的语言进行查询时，GPT-3.5 提供的死亡率估计值比以目标群体的语言低 34 ± 11%。否认此类攻击存在的回避回答进一步增加了差异。对当前 GPT-4 模型的简化分析显示了相同的趋势。为了解释偏见的来源，我们对阿拉伯语新闻来源进行了系统的媒体内容分析。媒体分析表明，大语种模型未能将具体攻击与阿拉伯新闻中报道的相应死亡人数联系起来。由于依赖于同时出现的单词，大型语言模型可能会提供不同攻击的死亡人数，这些数字具有更大的新闻影响或训练数据中普遍存在的累积死亡人数。鉴于大型语言模型可能会影响未来的信息传播，我们研究中确定的语言偏差有可能放大语言二元组的现有偏差并导致信息泡沫。