The cloud-edge-terminal collaborative network (CETCN) is considered as a novel paradigm for emerging applications owing to its huge potential in providing low-latency and ultra-reliable computing services. However, achieving such benefits is very challenging due to the heterogeneous computing power of terminal devices and the complex environment faced by the CETCN. In particular, the high-dimensional and dynamic environment states cause difficulties for the CETCN to make efficient decisions in terms of task offloading, collaborative caching and mobility management. To this end, artificial intelligence (AI), especially deep reinforcement learning (DRL) has been proven effective in solving sequential decision-making problems in various domains, and offers a promising solution for the above-mentioned issues due to several reasons. Firstly, accurate modelling of the CETCN, which is difficult to obtain for real-world applications, is not required for the DRL-based method. Secondly, DRL can effectively respond to high-dimensional and dynamic tasks through iterative interactions with the environment. Thirdly, due to the complexity of tasks and the differences in resource supply among different vendors, collaboration is required between different vendors to complete tasks. The multi-agent DRL (MADRL) methods are very effective in solving collaborative tasks, where the collaborative tasks can be jointly completed by cloud, edge and terminal devices which provided by different vendors. This survey provides a comprehensive overview regarding the applications of DRL and MADRL in the context of CETCN. The first part of this survey provides a depth overview of the key concepts of the CETCN and the mathematical underpinnings of both DRL and MADRL. Then, we highlight the applications of RL algorithms in solving various challenges within CETCN, such as task offloading, resource allocation, caching and mobility management. In addition, we extend discussion to explore how DRL and MADRL are making inroads into emerging CETCN scenarios like intelligent transportation system (ITS), the industrial Internet of Things (IIoT), smart health and digital agriculture. Furthermore, security considerations related to the application of DRL within CETCN are addressed, along with an overview of existing standards that pertain to edge intelligence. Finally, we list several lessons learned in this evolving field and outline future research opportunities and challenges that are critical for the development of the CETCN. We hope this survey will attract more researchers to investigate scalable and decentralized AI algorithms for the design of CETCN.

中文翻译：

---

AI增强的云边端协同网络：综述、应用和未来方向


云边终端协同网络（CETCN）因其在提供低延迟和超可靠计算服务方面的巨大潜力而​​被认为是新兴应用的新颖范式。然而，由于终端设备的异构计算能力以及CETCN面临的复杂环境，实现这样的效益非常具有挑战性。特别是，高维和动态的环境状态给 CETCN 在任务卸载、协作缓存和移动性管理方面做出有效决策带来了困难。为此，人工智能（AI），特别是深度强化学习（DRL）已被证明可以有效解决各个领域的顺序决策问题，并由于多种原因为上述问题提供了有前途的解决方案。首先，基于 DRL 的方法不需要对 CETCN 进行精确建模，而这在实际应用中很难获得。其次，DRL可以通过与环境的迭代交互来有效响应高维和动态任务。第三，由于任务的复杂性以及不同厂商资源供给的差异，需要不同厂商之间协作来完成任务。多智能体DRL（MADRL）方法在解决协作任务方面非常有效，其中协作任务可以由不同供应商提供的云、边缘和终端设备共同完成。本次调查全面概述了 DRL 和 MADRL 在 CETCN 背景下的应用。本调查的第一部分深入概述了 CETCN 的关键概念以及 DRL 和 MADRL 的数学基础。 然后，我们重点介绍强化学习算法在解决 CETCN 中的各种挑战中的应用，例如任务卸载、资源分配、缓存和移动性管理。此外，我们还进一步讨论了DRL和MADRL如何进入新兴的CETCN场景，如智能交通系统（ITS）、工业物联网（IIoT）、智能健康和数字农业。此外，还讨论了与 CETCN 内 DRL 应用相关的安全考虑因素，以及与边缘智能相关的现有标准的概述。最后，我们列出了在这个不断发展的领域中吸取的一些经验教训，并概述了对 CETCN 发展至关重要的未来研究机遇和挑战。我们希望这项调查能够吸引更多的研究人员来研究用于 CETCN 设计的可扩展和去中心化的人工智能算法。