Sarcasm, sentiment, and emotion are three typical kinds of spontaneous affective responses of humans to external events and they are tightly intertwined with each other. Such events may be expressed in multiple modalities (e.g., linguistic, visual and acoustic), e.g., multi-modal conversations. Joint analysis of humans’ multi-modal sarcasm, sentiment, and emotion is an important yet challenging topic, as it is a complex cognitive process involving both cross-modality interaction and cross-affection correlation. From the probability theory perspective, cross-affection correlation also means that the judgments on sarcasm, sentiment, and emotion are incompatible. However, this exposed phenomenon cannot be sufficiently modelled by classical probability theory due to its assumption of compatibility. Neither do the existing approaches take it into consideration. In view of the recent success of quantum probability (QP) in modeling human cognition, particularly contextual incompatible decision making, we take the first step towards introducing QP into joint multi-modal sarcasm, sentiment, and emotion analysis. Specifically, we propose a QU antum probab I lity driven multi-modal sarcasm, s E ntiment and emo T ion analysis framework, termed QUIET. Extensive experiments on two datasets and the results show that the effectiveness and advantages of QUIET in comparison with a wide range of the state-of-the-art baselines. We also show the great potential of QP in multi-affect analysis.

中文翻译：

---

用于联合多模态讽刺、情感和情感分析的量子概率驱动框架

讽刺、感伤和情绪是人类对外部事件的三种典型的自发情感反应，它们相互紧密地交织在一起。此类事件可以以多种模态（例如，语言、视觉和听觉）来表达，例如多模态对话。人类多模态讽刺、情绪和情感的联合分析是一个重要但具有挑战性的课题，因为它是一个复杂的认知过程，涉及跨模态交互和跨情感关联。从概率论的角度来看，交叉情感相关性也意味着对讽刺、情绪和情绪的判断是不相容的。然而，由于经典概率论的兼容性假设，这种暴露的现象无法充分建模。现有的方法也没有考虑到这一点。鉴于量子概率（QP）最近在人类认知建模方面取得的成功，特别是上下文不兼容的决策，我们迈出了将 QP 引入联合多模态讽刺、情感和情感分析的第一步。具体来说，我们提出一个量子概率 I liity驱动的多模式讽刺，s 情感和情绪 离子分析框架，称为QUIET。对两个数据集进行的大量实验和结果表明，与各种最先进的基线相比，QUIET 的有效性和优势。我们还展示了 QP 在多重影响分析中的巨大潜力。