This paper investigates the joint codebook selection and modulation-coding-scheme (MCS) adaptation issue for the enhanced mobile broadband (eMBB) service in millimeter-wave (mmWave) cellular systems. The proposed scheme guarantees efficient mmWave eMBB service through an intelligent joint codebook selection and MCS adaptation scheme that exploits deep reinforcement learning (DRL), referred to as DeepCM. DeepCM’s objective maximizes the transmission data rate while satisfying a target block error rate (BLER) constraint. A first step formulates this joint problem into a two-time-scale system that performs MCS adaptation on a small-time scale, whereas a second step optimizes the codebook on a large-time scale. DeepCM introduces a new DRL algorithm, termed Dual-DQN, by incorporating the operations on two time scales into the original deep Q-network (DQN). Dual-DQN essentially enables the operations on different time scales to benefit from each other, through closed-loop decision guidance and reward evaluation. Thereafter, to fulfill the preset BLER constraint, DeepCM uses a constrained -greedy strategy for decision-making and further modifies the conventional DRL training mechanism. Basically, DeepCM continuously adjusts the agent’s feasible-action space towards the system objective. With the constrained Dual-DQN, DeepCM can attain its goal even without any prior network information. Simulation results show that DeepCM, compared with TS-DRL, DRL-OLLA, and TS2 schemes, guarantees the target BLER requirement while yielding a much higher data rate. Various simulations demonstrate the powerful robustness of DeepCM under miscellaneous scenarios. Furthermore, DeepCM can handle well dynamic target-BLER change.

中文翻译：

---

基于深度强化学习的毫米波eMBB服务联合码本选择和MCS自适应


本文研究了毫米波 (mmWave) 蜂窝系统中增强型移动宽带 (eMBB) 服务的联合码本选择和调制编码方案 (MCS) 自适应问题。所提出的方案通过利用深度强化学习（DRL）（称为 DeepCM）的智能联合码本选择和 MCS 自适应方案来保证高效的毫米波 eMBB 服务。 DeepCM 的目标是最大化传输数据速率，同时满足目标块错误率 (BLER) 约束。第一步将这个联合问题表述为双时间尺度系统，在小时间尺度上执行 MCS 自适应，而第二步则在大时间尺度上优化码本。 DeepCM 引入了一种新的 DRL 算法，称为 Dual-DQN，通过将两个时间尺度上的操作合并到原始的深度 Q 网络（DQN）中。 Dual-DQN本质上是通过闭环决策引导和奖励评估，使不同时间尺度上的操作能够相互受益。此后，为了满足预设的BLER约束，DeepCM采用约束贪婪策略进行决策，并进一步修改了传统的DRL训练机制。基本上，DeepCM 不断调整智能体的可行动作空间以实现系统目标。借助受限 Dual-DQN，即使没有任何先验网络信息，DeepCM 也可以实现其目标。仿真结果表明，与 TS-DRL、DRL-OLLA 和 TS2 方案相比，DeepCM 保证了目标 BLER 要求，同时产生了更高的数据速率。各种模拟证明了 DeepCM 在各种场景下的强大鲁棒性。此外，DeepCM 可以很好地处理动态目标 BLER 变化。