Early-life exposure to stress results in significantly increased risk of neurodevelopmental impairments with potential long-term effects into childhood and even adulthood. As a crucial step towards monitoring neonatal stress in neonatal intensive care units (NICUs), our study aims to quantify the duration, frequency, and physiological responses of care manipulation activities, based on bedside videos and physiological signals. Leveraging 289 h of video recordings and physiological data within 330 sessions collected from 27 neonates in 2 NICUs, we develop and evaluate a deep learning method to detect manipulation activities from the video, to estimate their duration and frequency, and to further integrate physiological signals for assessing their responses. With a 13.8% relative error tolerance for activity duration and frequency, our results were statistically equivalent to human annotations. Further, our method proved effective for estimating short-term physiological responses, for detecting activities with marked physiological deviations, and for quantifying the neonatal infant stressor scale scores.

生命早期暴露于压力会导致神经发育障碍的风险显着增加，并可能对儿童甚至成年产生长期影响。作为监测新生儿重症监护病房 (NICU) 新生儿应激的关键一步，我们的研究旨在根据床边视频和生理信号量化护理操作活动的持续时间、频率和生理反应。利用从 2 个 NICU 的 27 名新生儿收集的 289 小时的视频记录和 330 个会话中的生理数据，我们开发和评估了一种深度学习方法，以检测视频中的操作活动，估计其持续时间和频率，并进一步整合生理信号评估他们的反应。活动持续时间和频率的相对误差容限为 13.8%，我们的结果在统计上与人工注释相当。此外，我们的方法被证明对于估计短期生理反应、检测具有明显生理偏差的活动以及量化新生儿应激源量表分数是有效的。