Passive acoustic monitoring (PAM) is commonly used to obtain year‐round continuous data on marine soundscapes harboring valuable information on species distributions or ecosystem dynamics. This continuously increasing amount of data requires highly efficient automated analysis techniques in order to exploit the full potential of the available data. Here, we propose a benchmark, which consists of a public dataset, a well‐defined task and evaluation procedure to develop and test automated analysis techniques. This benchmark focuses on the special case of detecting animal vocalizations in a real‐world dataset from the marine realm. We believe that such a benchmark is necessary to monitor the progress in the development of new detection algorithms in the field of marine bioacoustics. We ultimately use the proposed benchmark to test three detection approaches, namely ANIMAL‐SPOT, Koogu and a simple custom sequential convolutional neural network (CNN), and report performances. We report the performance of the three detection approaches in a blocked cross‐validation fashion with 11 site‐year blocks for a multi‐species detection scenario in a large marine passive acoustic dataset. Performance was measured with three simple metrics (i.e., true classification rate, noise misclassification rate and call misclassification rate) and one combined fitness metric, which allocates more weight to the minimization of false positives created by noise. Overall, ANIMAL‐SPOT performed the best with an average fitness metric of 0.6, followed by the custom CNN with an average fitness metric of 0.57 and finally Koogu with an average fitness metric of 0.42. The presented benchmark is an important step to advance in the automatic processing of the continuously growing amount of PAM data that are collected throughout the world's oceans. To ultimately achieve usability of developed algorithms, the focus of future work should be laid on the reduction of the false positives created by noise.

中文翻译：

---

海洋生物声学深度学习：须鲸检测的基准

被动声学监测（PAM）通常用于获取全年连续的海洋声景数据，其中包含有关物种分布或生态系统动态的宝贵信息。不断增加的数据量需要高效的自动化分析技术，以便充分发挥可用数据的潜力。在这里，我们提出了一个基准，它由公共数据集、明确定义的任务和评估程序组成，用于开发和测试自动化分析技术。该基准重点关注在海洋领域的真实数据集中检测动物发声的特殊情况。我们认为，这样的基准对于监测海洋生物声学领域新检测算法的开发进展是必要的。我们最终使用所提出的基准来测试三种检测方法，即 ANIMAL-SPOT、Koogu 和简单的定制顺序卷积神经网络 (CNN)，并报告性能。我们以分块交叉验证的方式报告了三种检测方法的性能，其中包含 11 个站点年分块，用于大型海洋被动声学数据集中的多物种检测场景。性能通过三个简单指标（即真实分类率、噪声误分类率和呼叫误分类率）和一个组合适应度指标来衡量，该指标为噪声产生的误报的最小化分配了更多权重。总体而言，ANIMAL‐SPOT 表现最好，平均适应度指标为 0.6，其次是自定义 CNN，平均适应度指标为 0.57，最后是 Koogu，平均适应度指标为 0.42。所提出的基准是推进自动处理从世界各地海洋收集的不断增长的 PAM 数据的重要一步。为了最终实现所开发算法的可用性，未来工作的重点应该放在减少噪声造成的误报上。