Aerial images are increasingly adopted and widely used in various research areas. In marine mammal studies, these imagery surveys serve multiple purposes: determining population size, mapping migration routes, and gaining behavioral insights. A single aerial scan using a drone yields a wealth of data, but processing it requires significant human effort. Our research demonstrates that deep learning models can significantly reduce human effort. They are not only able to detect marine mammals but also track their behavior using continuous aerial (video) footage. By distinguishing between different age classes, these algorithms can inform studies on population biology, ontogeny, and adult-calf relationships. To detect beluga whales from imagery footage, we trained the YOLOv7 model on a proprietary dataset of aerial footage of beluga whales. The deep learning model achieved impressive results with the following precision and recall scores: beluga adult = 92%—92%, beluga calf = 94%—89%. To track the detected beluga whales, we implemented the deep Simple Online and Realtime Tracking (SORT) algorithm. Unfortunately, the performance of the deep SORT algorithm was disappointing, with Multiple Object Tracking Accuracy (MOTA) scores ranging from 27% to 48%. An analysis revealed that the low tracking accuracy resulted from identity switching; that is, an identical beluga whale was given two IDs in two different frames. To overcome the problem of identity switching, a new post-processing algorithm was implemented, significantly improving MOTA to approximately 70%. The main contribution of this research is providing a system that accurately detects and tracks features of beluga whales, both adults and calves, from aerial footage. Additionally, this system can be customized to identify and analyze other marine mammal species by fine-tuning the model with annotated data.

中文翻译：

---

使用深度学习在航拍视频中定位和跟踪白鲸


航空影像越来越多地被各个研究领域采用和广泛使用。在海洋哺乳动物研究中，这些影像调查有多种用途：确定种群规模、绘制迁徙路线地图和获得行为洞察。使用无人机进行一次航拍扫描会产生大量数据，但处理这些数据需要大量的人力。我们的研究表明，深度学习模型可以显著减少人力。它们不仅能够检测海洋哺乳动物，还可以使用连续的航拍（视频）镜头跟踪它们的行为。通过区分不同的年龄组，这些算法可以为种群生物学、个体发育和成年犊牛关系的研究提供信息。为了从图像片段中检测白鲸，我们在专有的白鲸航拍数据集上训练了 YOLOv7 模型。深度学习模型取得了令人印象深刻的结果，准确率和召回率得分如下：成年白鲸 = 92%—92%，幼崽白鲸 = 94%—89%。为了追踪检测到的白鲸，我们实施了深度简单在线和实时跟踪 （SORT） 算法。不幸的是，深度 SORT 算法的性能令人失望，多目标跟踪精度 （MOTA） 分数从 27% 到 48% 不等。一项分析显示，低跟踪准确性是由于身份切换造成的;也就是说，一头相同的白鲸在两个不同的帧中被赋予了两个 ID。为了克服身份切换问题，实施了一种新的后处理算法，将 MOTA 显著提高到约 70%。这项研究的主要贡献是提供了一个系统，可以从航拍镜头中准确检测和跟踪白鲸的特征，包括成年白鲸和幼鲸的特征。 此外，该系统可以定制，通过使用注释数据微调模型来识别和分析其他海洋哺乳动物物种。