This paper tackles the limitation of a closed-world object detection model that was trained on one species. The expectation for this model is that it will not generalize well to recognize the instances of new species if they were present in the incoming data stream. We propose a novel object detection framework for this open-world setting that is suitable for applications that monitor wildlife, ocean life, livestock, plant phenotype and crops that typically feature one species in the image. Our method leverages labeled samples from one species in combination with a novelty detection method and Segment Anything Model, a vision foundation model, to (1) identify the presence of new species in unlabeled images, (2) localize their instances, and (3) retrain the initial model with the localized novel class instances. The resulting integrated system assimilates and learns from unlabeled samples of the new classes while not “forgetting” the original species the model was trained on. We demonstrate our findings on two different domains, (1) wildlife detection and (2) plant detection. Our method achieves an AP of 56.2 (for 4 novel species) to 61.6 (for 1 novel species) for wildlife domain, without relying on any ground truth data in the background.

本文解决了在一个物种上训练的封闭世界物体检测模型的局限性。此模型的预期是，如果新物种的实例存在于传入的数据流中，则它不会很好地泛化来识别它们。我们为这种开放世界设置提出了一种新的对象检测框架，该框架适用于监控野生动物、海洋生物、牲畜、植物表型和通常在图像中以一个物种为特征的农作物的应用程序。我们的方法利用来自一个物种的标记样本与新颖性检测方法和视觉基础模型 Segment Anything Model 相结合，以 （1） 识别未标记图像中新物种的存在，（2） 定位它们的实例，以及 （3） 使用本地化的新类实例重新训练初始模型。由此产生的集成系统吸收了新类别的未标记样本并从中学习，同时不会 “忘记” 模型训练的原始物种。我们在两个不同的领域展示了我们的发现，（1） 野生动物检测和 （2） 植物检测。我们的方法在野生动物领域实现了 56.2（对于 4 个新物种）到 61.6（对于 1 个新物种）的 AP，而无需依赖背景中的任何地面实况数据。