Network failures, especially link failures, happen frequently in Internet Service Provider (ISP) networks. When link failures occur, the routing policies need to be re-computed and failure recovery usually takes a few minutes, which degrades the network performance to a great extent. Therefore, a proper failure recovery scheme that can realize a fast and timely routing policy computation needs to be designed. In this paper, we propose FRRL, a Reinforcement Learning (RL) approach to intelligently perceive network failures and timely compute the routing policy for improving the network performance when link failure happens. Specifically, to perceive the link failures, we design a Topology Difference Vector (TDV) encoder module in FRRL for encoding the topology structure with link failures. To efficiently compute the routing policy when link failures happen, we integrate the TDV in the agent training for learning the map between the encoded failure topology structure and routing policies. To evaluate the performance of our proposed method, we conduct experiments on three network topologies and the experimental results demonstrate that our proposed method has superior performance when link failures happen compared to other methods.

中文翻译：

---

FRRL：一种用于混合 SDN 中链路故障恢复的强化学习方法


网络故障（尤其是链路故障）在 Internet 服务提供商 （ISP） 网络中经常发生。当链路发生故障时，需要重新计算路由策略，故障恢复通常需要几分钟时间，这在很大程度上降低了网络性能。因此，需要设计一个合适的故障恢复方案，能够实现快速及时的路由策略计算。在本文中，我们提出了 FRRL，这是一种强化学习 （RL） 方法，可以智能感知网络故障并及时计算路由策略，以便在发生链路故障时提高网络性能。具体来说，为了感知链路故障，我们在 FRRL 中设计了一个拓扑差分向量 （TDV） 编码器模块，用于对链路故障的拓扑结构进行编码。为了在发生链路故障时有效地计算路由策略，我们将 TDV 集成到代理训练中，以学习编码的故障拓扑结构和路由策略之间的映射。为了评估我们提出的方法的性能，我们在三种网络拓扑上进行了实验，实验结果表明，与其他方法相比，我们提出的方法在发生链路故障时具有更好的性能。