With the development of wireless communication systems, the large-scale deployment of Internet-of-Things (IoT) devices becomes popular. Due to limited energy, the multiple access approaches without carrier sensing requirement are widely deployed in IoT devices, including Aloha and Time Division Multiple Access (TDMA). However, these approaches encounter the transmission inefficiency issue, especially in dynamic heterogeneous networks comprising nodes with diverse protocols and varying numbers and transmission configurations over time. In this paper, combining Curriculum Learning (CL) and Multi-task Reinforcement Learning (MTRL), we propose the Curriculum Multi-task Heterogeneous-Agent Proximal Policy Optimisation (CMHA) algorithm to improve the throughput performance while guaranteeing fairness in dynamic heterogeneous networks. We introduce the Elastic Weight Consolidation (EWC) in the CMHA to further enhance generalization capacity, which can better address the challenging MTRL problem in dynamic heterogeneous networks. Combining the monotonic improvement feature of Heterogeneous-Agent Proximal Policy Optimization (HAPPO) and the generalization capacity of EWC, the proposed CMHA can achieve a nearly monotonic improvement in all possible scenarios. The simulations show that the CMHA algorithm 1) has sufficient generalization capacity for massive scenarios in dynamic heterogeneous networks; 2) can significantly enhance the network throughput; 3) can guarantee the fairness of both agents and heterogeneous nodes.

中文翻译：

---

动态异构无线网络中基于课程多任务HAPPO的多接入


随着无线通信系统的发展，物联网（IoT）设备的大规模部署变得流行。由于能量有限，无需载波侦听的多址方法广泛应用于物联网设备，包括 Aloha 和时分多址 (TDMA)。然而，这些方法遇到了传输效率低下的问题，特别是在包含具有不同协议、数量和传输配置随时间变化的节点的动态异构网络中。本文结合课程学习（CL）和多任务强化学习（MTRL），提出了课程多任务异构代理近端策略优化（CMHA）算法，以提高吞吐量性能，同时保证动态异构网络的公平性。我们在 CMHA 中引入弹性权重合并（EWC）以进一步增强泛化能力，可以更好地解决动态异构网络中具有挑战性的 MTRL 问题。结合异构代理近端策略优化（HAPPO）的单调改进特征和EWC的泛化能力，所提出的CMHA可以在所有可能的场景中实现近单调改进。仿真结果表明，CMHA算法1）对于动态异构网络中的海量场景具有足够的泛化能力； 2）可以显着提升网络吞吐量； 3）能够保证代理和异构节点双方的公平性。