Stress has been identified as one of major causes of health issues. To detect the stress levels with higher accuracy, fusion of multimodal physiological signals is a promising technique. However, there is an asynchrony between physiological signals observed from different perspectives. Exploring the temporal alignment relationship between modalities is helpful to improve the quality of multimodal fusion. This paper proposes an end-to-end multimodal stress detection model based on Bidirectional Cross- and Self-modal Attention (BCSA) mechanism. Specifically, we first construct different feature extractors based on the characteristics of Blood Volume Pulse (BVP) and Electrodermal Activity (EDA) to complete automated temporal feature extraction. Second, cross-modal attention is used to seek the alignment relationship between the two modalities and fully fuse cross-modal information. The self-modal attention is used to attenuate noise and redundant information, highlight important information and obtain salient stress representations. Finally, the stress representations of the two modalities are processed separately, and the mean square error (MSE) is used to narrow the gap between them. Experimental results on the UBFC-Phys dataset and WESAD dataset show that the proposed model can effectively improve the accuracy of stress recognition, and outperforms several state-of-the-art methods.

中文翻译：

---

用于压力识别的多模态生理模式的动态对准和融合


压力已被确定为健康问题的主要原因之一。为了更准确地检测压力水平，多模态生理信号的融合是一种有前途的技术。然而，从不同角度观察到的生理信号之间存在异步性。探索模态之间的时间对齐关系有助于提高多模态融合的质量。本文提出了一种基于双向交叉和自模态注意（BCSA）机制的端到端多模态压力检测模型。具体来说，我们首先根据血量脉冲（BVP）和皮肤电活动（EDA）的特征构建不同的特征提取器来完成自动化时间特征提取。其次，跨模态注意力用于寻求两种模态之间的对齐关系，充分融合跨模态信息。自模态注意力用于衰减噪声和冗余信息，突出重要信息并获得显着的压力表示。最后，分别处理两种模态的应力表示，并使用均方误差（MSE）来缩小它们之间的差距。在UBFC-Phys数据集和WESAD数据集上的实验结果表明，所提出的模型可以有效提高应力识别的准确性，并且优于几种最先进的方法。