Complex & Intelligent Systems ( IF 5.0 ) Pub Date : 2024-11-13 , DOI: 10.1007/s40747-024-01610-0 Meng Wang, Guanci Yang, Kexin Luo, Yang Li, Ling He
Stroke is the prevalent cerebrovascular disease characterized by significant incidence and disability rates. To enhance the early perceive and detection of potential stroke patients, the early stroke behavior detection based on improved Video Masked Autoencoders (VideoMAE) for potential patients (EPBR-PS) is proposed. The proposed method begins with novel time interval-based sampling strategy, capturing video frame sequences enriched with sparse motion features. On the basis of establishing the masking mechanism for adjacent frames and pixel blocks within these sequences, The EPBR-PS employes pipeline mask strategy to extract spatiotemporal features effectively. Then, the local convolution attention mechanism is designed to capture local dynamic feature information, and central to the EPBR-PS is the integration of local convolutional attention mechanism with VideoMAE's multi-head attention mechanism. This integration facilitates the simultaneous leveraging of global high-level semantics and local dynamic feature information. Dual attention mechanism-based method for the fusion of these global and local features is proposed. After that, the optimal parameters of EPBR-PS were determined through the experiment of learning rate and fusion weights of different features. On the NTU-ST dataset, comparative analysis with eight models demonstrated the superiority of EPBR-PS, evidenced by the average recognition accuracy of 89.61%, surpassing that 1.67% over the benchmark VideoMAE. On the HMDB51 dataset, EPBR-PS has Top1 of 71.31%, which is 0.73% higher than that of the VideoMAE, providing the viable behavior detection for perception early signs of potential stroke in the home environment. This code is available at https://github.com/wang-325/EPBR-PS/.
中文翻译:
基于改进视频屏蔽自动编码器的早期卒中行为检测
中风是一种普遍的脑血管疾病,其特点是发病率和残疾率高。为了增强对潜在脑卒中患者的早期感知和检测,该文提出一种基于改进视频掩码自动编码器 (VideoMAE) 的潜在患者早期脑卒中行为检测 (EPBR-PS)。所提出的方法从新颖的基于时间间隔的采样策略开始,捕获富含稀疏运动特征的视频帧序列。EPBR-PS 在建立这些序列中相邻帧和像素块的掩码机制的基础上,采用管道掩码策略来有效提取时空特征。然后,设计局部卷积注意力机制来捕获局部动态特征信息,EPBR-PS 的核心是将局部卷积注意力机制与 VideoMAE 的多头注意力机制集成。这种集成有助于同时利用全局高级语义和本地动态特征信息。该文提出一种基于双注意力机制的全局和局部特征融合的方法。之后,通过不同特征的学习率和融合权重实验确定了 EPBR-PS 的最优参数。在 NTU-ST 数据集上,与 8 个模型的比较分析证明了 EPBR-PS 的优越性,平均识别准确率为 89.61%,超过了基准 VideoMAE 的 1.67%。在 HMDB51 数据集上,EPBR-PS 的 Top1 为 71.31%,比 VideoMAE 高 0.73%,为家庭环境中潜在卒中的早期感知迹象提供了可行的行为检测。此代码可在 https://github.com/wang-325/EPBR-PS/ 获取。