The objective of this study was to develop a fully automated multiple-cow real-time lameness detection system using a deep learning approach for cattle detection and pose estimation that could be deployed across dairy farms. Utilising computer vision and deep learning, the system can analyse simultaneously both the posture and gait of each cow within a camera field of view to a very high degree of accuracy (94–100%). Twenty-five video sequences containing 250 cows in varying degrees of lameness were recorded and independently scored by three accredited Agriculture and Horticulture Development Board (AHDB) mobility scorers using the AHDB dairy mobility scoring system to provide ground truth lameness data. These observers showed significant inter-observer reliability. Video sequences were broken down into their constituent frames and with a further 500 images downloaded from google, annotated with 15 anatomical points for each animal. A modified Mask-RCNN estimated the pose of each cow to output 5 key-points to determine back arching and 2 key-points to determine head position. Using the SORT (simple, online, and real-time tracking) algorithm, cows were tracked as they move through frames of the video sequence (i.e., in moving animals). All the features were combined using the CatBoost gradient boosting algorithm with accuracy being determined using threefold cross-validation including recursive feature elimination. Precision was assessed using Cohen’s kappa coefficient and assessments of precision and recall. This methodology was applied to cows with varying degrees of lameness (according to accredited scoring, n = 3) and demonstrated that some characteristics directly associated with lameness could be monitored simultaneously. By combining the algorithm results over time, more robust evaluation of individual cow lameness was obtained. The model showed high performance for predicting and matching the ground truth lameness data with the outputs of the algorithm. Overall, threefold lameness detection accuracy of 100% and a lameness severity classification accuracy of 94% respectively was achieved with a high degree of precision (Cohen’s kappa = 0.8782, precision = 0.8650 and recall = 0.9209).

本研究的目的是开发一种全自动多奶牛实时跛行检测系统，使用深​​度学习方法进行牛检测和姿势估计，该系统可以在奶牛场部署。利用计算机视觉和深度学习，该系统可以同时分析摄像机视野内每头牛的姿势和步态，准确度非常高（94-100%）。记录了包含 250 头不同跛行程度的奶牛的 25 个视频序列，并由三名经认可的农业和园艺发展委员会 (AHDB) 流动性评分员使用 AHDB 乳品流动性评分系统进行独立评分，以提供真实的跛行数据。这些观察者表现出显着的观察者间可靠性。视频序列被分解为其组成帧，并从谷歌下载了另外 500 张图像，并为每只动物标注了 15 个解剖点。修改后的 Mask-RCNN 估计每头牛的姿势，输出 5 个关键点来确定背部拱起，并输出 2 个关键点来确定头部位置。使用 SORT（简单、在线和实时跟踪）算法，当奶牛在视频序列的帧中移动（即移动的动物）时，它们就会被跟踪。所有特征均使用 CatBoost 梯度增强算法进行组合，并使用三重交叉验证（包括递归特征消除）确定准确性。使用 Cohen kappa 系数以及精确度和召回率评估来评估精确度。该方法适用于具有不同跛行程度的奶牛（根据认可评分，n = 3），并证明可以同时监测与跛行直接相关的一些特征。 通过结合一段时间内的算法结果，可以获得对个体奶牛跛行的更稳健的评估。该模型在预测真实跛行数据并将其与算法输出进行匹配方面表现出高性能。总体而言，三倍跛行检测准确度为 100%，跛行严重程度分类准确度为 94%，并且具有高精度（Cohen 的 kappa = 0.8782、精确度 = 0.8650 和召回率 = 0.9209）。