Hybrid Software Defined Networks (Hybrid SDNs), which combines the robustness of distributed network and the flexibility of centralized network, is now a prevailing network architecture. Previous hybrid SDN Traffic Engineering (TE) solutions search an optimal link weight setting or compute the splitting ratios of traffic leveraging heuristic algorithms. However, these methods cannot react timely to the fluctuating traffic demands in dynamic environments and suffer a hefty performance degradation when traffic demands change or network failures happen, especially when network scale is large. To cope with this, we propose a Multi-Agent reinforcement learning based TE method MATE that timely determines the route selection for network flows in dynamic hybrid SDNs. Through dividing the large-scale routing optimization problem into small-scale problem, MATE can better learn the mapping between the traffic demands and routing policy, and efficiently make online routing inference with dynamic traffic demands. To collaborate multiple agents and speed up the convergence in the training process, we innovatively design the actor network and introduce previous actions of all agents in the training of each agent. Extensive experiments conducted on different network topologies demonstrate our proposed method MATE has superior TE performance with dynamic traffic demands and is robust to network failures.

中文翻译：

---

MATE：混合软件定义网络中流量工程的多代理强化学习方法


混合软件定义网络（Hybrid SDN）结合了分布式网络的稳健性和集中式网络的灵活性，是目前流行的网络架构。以前的混合 SDN 流量工程 (TE) 解决方案会搜索最佳链路权重设置或利用启发式算法计算流量的分流比。然而，这些方法无法及时响应动态环境中波动的流量需求，并且当流量需求变化或发生网络故障时，特别是当网络规模较大时，性能会严重下降。为了解决这个问题，我们提出了一种基于多代理强化学习的 TE 方法 MATE，该方法可以及时确定动态混合 SDN 中网络流的路由选择。通过将大规模路由优化问题分解为小规模问题，MATE可以更好地学习流量需求与路由策略之间的映射，并高效地针对动态流量需求进行在线路由推理。为了协作多个智能体并加速训练过程中的收敛，我们创新地设计了参与者网络，并在每个智能体的训练中引入所有智能体之前的动作。在不同网络拓扑上进行的大量实验表明，我们提出的方法 MATE 在动态流量需求下具有卓越的 TE 性能，并且对网络故障具有鲁棒性。