当前位置: X-MOL 学术Complex Intell. Syst. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Enhancing learning on uncertain pixels in self-distillation for object segmentation
Complex & Intelligent Systems ( IF 5.0 ) Pub Date : 2024-06-15 , DOI: 10.1007/s40747-024-01519-8
Lei Chen , Tieyong Cao , Yunfei Zheng , Yang Wang , Bo Zhang , Jibin Yang

Self-distillation method guides the model learning via transferring knowledge of the model itself, which has shown the advantages in object segmentation. However, it has been proved that uncertain pixels with predicted probability close to 0.5 will restrict the model performance. The existing self-distillation methods cannot guide the model to enhance its learning ability for uncertain pixels, so the improvement is limited. To boost the student model’s learning ability for uncertain pixels, a novel self-distillation method is proposed. Firstly, the predicted probability in the current training sample and the ground truth label are fused to construct the teacher knowledge, as the current predicted information can express the performance of student models and represent the uncertainty of pixels more accurately. Secondly, a quadratic mapping function between the predicted probabilities of the teacher and student model is proposed. Theoretical analysis shows that the proposed method using the mapping function can guide the model to enhance the learning ability for uncertain pixels. Finally, the essential difference of utilizing the predicted probability of the student model in self-distillation is discussed in detail. Extensive experiments were conducted on models with convolutional neural networks and Transformer architectures as the backbone networks. The results on four public datasets demonstrate that the proposed method can effectively improve the student model performance.



中文翻译:


增强自蒸馏中不确定像素的学习以进行对象分割



自蒸馏方法通过传递模型本身的知识来指导模型学习,在对象分割方面表现出了优势。然而,事实证明,预测概率接近0.5的不确定像素会限制模型性能。现有的自蒸馏方法无法指导模型增强对不确定像素的学习能力,因此改进有限。为了提高学生模型对不确定像素的学习能力,提出了一种新颖的自蒸馏方法。首先,融合当前训练样本中的预测概率和地面真值标签来构造教师知识,因为当前预测信息可以表达学生模型的性能并更准确地表示像素的不确定性。其次,提出了教师和学生模型的预测概率之间的二次映射函数。理论分析表明,所提出的利用映射函数的方法可以指导模型增强对不确定像素的学习能力。最后详细讨论了利用学生模型的预测概率进行自蒸馏的本质区别。以卷积神经网络和 Transformer 架构作为骨干网络的模型进行了大量的实验。四个公共数据集上的结果表明,所提出的方法可以有效提高学生模型的性能。

更新日期:2024-06-15
down
wechat
bug