Recently, bilevel optimization (BLO) has taken center stage in some very exciting developments in the area of signal processing (SP) and machine learning (ML). Roughly speaking, BLO is a classical optimization problem that involves two levels of hierarchy (i.e., upper and lower levels), wherein obtaining the solution to the upper-level problem requires solving the lower-level one. BLO has become popular largely because it is powerful in modeling problems in SP and ML, among others, that involve optimizing nested objective functions. Prominent applications of BLO range from resource allocation for wireless systems to adversarial ML. In this work, we focus on a class of tractable BLO problems that often appear in SP and ML applications. We provide an overview of some basic concepts of this class of BLO problems, such as their optimality conditions, standard algorithms (including their optimization principles and practical implementations) as well as how they can be leveraged to obtain state-of-the-art results for several key SP and ML applications. Further, we discuss some recent advances in BLO theory and its implications for applications, and we point out some limitations of the state of the art that require significant future research efforts. We hope that this article, together with the associated open source BLO toolbox we developed for algorithm benchmarking, can serve to accelerate the adoption of BLO as a generic tool to model, analyze, and innovate on a wide array of emerging SP and ML applications.

中文翻译：

---

双层优化简介：信号处理和机器学习的基础和应用

最近，双层优化 (BLO) 已成为信号处理 (SP) 和机器学习 (ML) 领域一些非常令人兴奋的发展的中心舞台。粗略地说，BLO是一种经典的优化问题，涉及两级层次（即上层和下层），其中要获得上层问题的解需要求解下层问题。 BLO 之所以流行，很大程度上是因为它在 SP 和 ML 等涉及优化嵌套目标函数的建模问题方面功能强大。 BLO 的突出应用包括从无线系统的资源分配到对抗性机器学习。在这项工作中，我们重点关注 SP 和 ML 应用程序中经常出现的一类易于处理的 BLO 问题。我们概述了此类 BLO 问题的一些基本概念，例如它们的最优条件、标准算法（包括它们的优化原理和实际实现）以及如何利用它们来获得最先进的结果适用于多个关键 SP 和 ML 应用。此外，我们讨论了 BLO 理论的一些最新进展及其对应用的影响，并指出了现有技术的一些局限性，需要未来进行大量的研究工作。我们希望本文与我们为算法基准测试开发的相关开源 BLO 工具箱能够加速采用 BLO 作为通用工具，对各种新兴 SP 和 ML 应用进行建模、分析和创新。