Existing Out-of-Distribution (OoD) detection methods address to detect OoD samples from In-Distribution (InD) data mainly by exploring differences in features, logits and gradients in Deep Neural Networks (DNNs). We in this work propose a new perspective upon loss landscape and mode ensemble to investigate OoD detection. In the optimization of DNNs, there exist many local optima in the parameter space, or namely modes. Interestingly, we observe that these independent modes, which all reach low-loss regions with InD data (training and test data), yet yield significantly different loss landscapes with OoD data. Such an observation provides a novel view to investigate the OoD detection from the loss landscape, and further suggests significantly fluctuating OoD detection performance across these modes. For instance, FPR values of the RankFeat (Song et al. in Advances in Neural Information Processing Systems 35:17885–17898, 2022) method can range from 46.58% to 84.70% among 5 modes, showing uncertain detection performance evaluations across independent modes. Motivated by such diversities on OoD loss landscape across modes, we revisit the deep ensemble method for OoD detection through mode ensemble, leading to improved performance and benefiting the OoD detector with reduced variances. Extensive experiments covering varied OoD detectors and network structures illustrate high variances across modes and validate the superiority of mode ensemble in boosting OoD detection. We hope this work could attract attention in the view of independent modes in the loss landscape of OoD data and more reliable evaluations on OoD detectors.

现有的分布外（OoD）检测方法主要通过探索深度神经网络（DNN）中特征、逻辑和梯度的差异来从分布内（InD）数据中检测OoD样本。我们在这项工作中提出了一种关于损失景观和模式集成的新视角来研究 OoD 检测。在 DNN 的优化中，参数空间中存在许多局部最优，即模式。有趣的是，我们观察到这些独立模式都通过 InD 数据（训练和测试数据）达到低损耗区域，但通过 OoD 数据产生显着不同的损耗情况。这样的观察提供了一种从损失情况研究 OoD 检测的新颖视角，并进一步表明这些模式下 OoD 检测性能的显着波动。例如，RankFeat（Song et al. in Advances in Neural Information Processing Systems 35:17885–17898, 2022）方法的 FPR 值在 5 种模式中的范围可以从 46.58% 到 84.70%，显示出跨独立模式的不确定检测性能评估。受跨模式 OoD 损失景观的这种多样性的启发，我们重新审视通过模式集成进行 OoD 检测的深度集成方法，从而提高性能并有利于 OoD 检测器减少方差。涵盖各种 OoD 检测器和网络结构的大量实验说明了不同模式之间的巨大差异，并验证了模式集成在增强 OoD 检测方面的优越性。我们希望这项工作能够从 OoD 数据丢失情况中的独立模式以及对 OoD 检测器更可靠的评估的角度引起关注。