Optimizing edge caching is crucial for the advancement of next-generation (nextG) wireless networks, ensuring high-speed and low-latency services for mobile users. Existing data-driven optimization approaches often lack awareness of the distribution of random data variables and focus solely on optimizing cache hit rates, neglecting potential reliability concerns, such as base station overload and unbalanced cache issues. This oversight can result in system crashes and degraded user experience. To bridge this gap, we introduce a novel digital twin-assisted optimization framework, called D-REC, which integrates reinforcement learning (RL) with diverse intervention modules to ensure reliable caching in nextG wireless networks. We first develop a joint vertical and horizontal twinning approach to efficiently create network digital twins, which are then employed by D-REC as RL optimizers and safeguards, providing ample datasets for training and predictive evaluation of our cache replacement policy. By incorporating reliability modules into a constrained Markov decision process, D-REC can adaptively adjust actions, rewards, and states to comply with advantageous constraints, minimizing the risk of network failures. Theoretical analysis demonstrates comparable convergence rates between D-REC and vanilla data-driven methods without compromising caching performance. Extensive experiments validate that D-REC outperforms conventional approaches in cache hit rate and load balancing while effectively enforcing predetermined reliability intervention modules.

中文翻译：

---

数字孪生辅助数据驱动优化，在无线网络中实现可靠的边缘缓存


优化边缘缓存对于下一代 （nextG） 无线网络的发展至关重要，可确保为移动用户提供高速和低延迟的服务。现有的数据驱动优化方法通常缺乏对随机数据变量分布的认识，只专注于优化缓存命中率，而忽略了潜在的可靠性问题，例如基站过载和缓存不平衡问题。这种疏忽可能会导致系统崩溃和用户体验下降。为了弥合这一差距，我们引入了一种名为 D-REC 的新型数字孪生辅助优化框架，它将强化学习 （RL） 与各种干预模块集成在一起，以确保 nextG 无线网络中的可靠缓存。我们首先开发了一种联合垂直和水平孪生方法来有效地创建网络数字孪生，然后将其用于 D-REC 作为 RL 优化器和保护措施，为我们的缓存替换策略的训练和预测评估提供充足的数据集。通过将可靠性模块整合到约束马尔可夫决策过程中，D-REC 可以自适应地调整操作、奖励和状态以符合有利的约束，从而最大限度地降低网络故障的风险。理论分析表明，D-REC 和普通数据驱动方法之间的收敛率相当，而不会影响缓存性能。广泛的实验验证了 D-REC 在缓存命中率和负载平衡方面优于传统方法，同时有效地实施了预定的可靠性干预模块。