Due to the high mobility, high chance of line-of-sight (LoS) transmission, and flexible deployment, unmanned aerial vehicles (UAVs) have been used as mobile edge computing (MEC) servers to provide ubiquitous computation services to mobile users (MUs). However, the limited energy storage, caching capacity, and computation resources of UAVs bring new challenges for UAV-aided MEC, e.g., how to recharge UAVs and how to jointly optimize service-caching, computation-offloading, and UAVs flight trajectories. To overcome the above-mentioned difficulties, in this paper we study the joint optimization for service-caching, computation-offloading, and UAVs flight trajectories for UAV-aided MEC, where multiple rechargeable UAVs cooperatively provide MEC services to a number of MUs. First, we formulate an energy minimization problem to minimize all MUs' energy consumptions by taking into account the mobility of MUs and the energy replenishment of UAVs. Then, using the hierarchical multi-agent deep reinforcement learning (HMDRL), we develop a two-timescale based joint service-caching, computation-offloading, and UAVs flight trajectories scheme, called HMDRL-Based SCOFT. Using HMDRL-Based SCOFT, we derive UAVs' service-caching policies in each time frame, and then derive UAVs flight trajectories and MUs' computation-offloading in each time slot. Finally, we validate and evaluate the performances of our proposed HMDRL-Based SCOFT scheme through extensive simulations, which show that our developed scheme outperforms the other baseline schemes to converge faster and greatly reduce MUs' energy consumptions.

中文翻译：

---

使用分层多智能体深度强化学习对可充电无人机辅助 MEC 上的服务缓存、计算卸载和无人机飞行轨迹进行联合优化


由于无人机（UAV）具有高移动性、高视距（LoS）传输机会和灵活部署等特点，已被用作移动边缘计算（MEC）服务器，为移动用户（MU）提供无处不在的计算服务）。然而，无人机有限的能量存储、缓存容量和计算资源给无人机辅助MEC带来了新的挑战，例如如何为无人机充电以及如何联合优化服务缓存、计算卸载和无人机飞行轨迹。为了克服上述困难，本文研究了无人机辅助 MEC 的服务缓存、计算卸载和无人机飞行轨迹的联合优化，其中多个可充电无人机协同为多个 MU 提供 MEC 服务。首先，我们考虑动车组的移动性和无人机的能量补充，制定能量最小化问题，以最小化所有动车组的能源消耗。然后，使用分层多智能体深度强化学习（HMDRL），我们开发了一种基于两个时间尺度的联合服务缓存、计算卸载和无人机飞行轨迹方案，称为基于 HMDRL 的 SCOFT。使用基于 HMDRL 的 SCOFT，我们推导了无人机在每个时间范围内的服务缓存策略，然后推导了无人机在每个时间段的飞行轨迹和 MU 的计算卸载。最后，我们通过广泛的模拟验证和评估了我们提出的基于 HMDRL 的 SCOFT 方案的性能，这表明我们开发的方案优于其他基线方案，收敛速度更快，并大大降低了 MU 的能耗。