Zero-cost proxies are nowadays frequently studied and used to search for neural architectures. They show an impressive ability to predict the performance of architectures by making use of their untrained weights. These techniques allow for immense search speed-ups. So far the joint search for well performing and robust architectures has received much less attention in the field of NAS. Therefore, the main focus of zero-cost proxies is the clean accuracy of architectures, whereas the model robustness should play an evenly important part. In this paper, we analyze the ability of common zero-cost proxies to serve as performance predictors for robustness in the popular NAS-Bench-201 search space. We are interested in the single prediction task for robustness and the joint multi-objective of clean and robust accuracy. We further analyze the feature importance of the proxies and show that predicting the robustness makes the prediction task from existing zero-cost proxies more challenging. As a result, the joint consideration of several proxies becomes necessary to predict a model’s robustness while the clean accuracy can be regressed from a single such feature. Our code is available at https://github.com/jovitalukasik/zcp_eval.

如今，零成本代理经常被研究并用于搜索神经架构。它们表现出令人印象深刻的能力，即利用未经训练的权重来预测架构的性能。这些技术可以极大地加快搜索速度。到目前为止，在 NAS 领域，对性能良好且健壮的架构的联合搜索受到的关注要少得多。因此，零成本代理的主要关注点是架构的干净准确性，而模型鲁棒性应该起同样重要的作用。在本文中，我们分析了常见的零成本代理在流行的 NAS-Bench-201 搜索空间中作为稳健性性能预测器的能力。我们对稳健性的单一预测任务以及干净和稳健的准确性的联合多目标感兴趣。我们进一步分析了代理的特征重要性，并表明预测鲁棒性使现有零成本代理的预测任务更具挑战性。因此，为了预测模型的稳健性，必须共同考虑多个代理，而干净的准确率可以从单个这样的特征中回归。我们的代码可在 https://github.com/jovitalukasik/zcp_eval 获取。