Nowadays, unmanned aerial vehicles (UAVs) organized in a flying ad hoc network (FANET) can successfully carry out complex missions. Due to the limitations of these networks, including the lack of infrastructure, wireless communication channels, dynamic topology, and unreliable communication between UAVs, cyberattacks, especially wormholes, weaken the performance of routing schemes. Therefore, maintaining communication security and guaranteeing the quality of service (QoS) are very challenging. In this paper, a novel Q-learning-based secure routing scheme (QSR) is presented for FANETs. QSR seeks to provide a robust defensive system against wormhole attacks, especially wormhole through encapsulation and wormhole through packet relay. QSR includes a secure neighbor discovery process and a Q-learning-based secure routing process. Firstly, each UAV gets information about its neighboring UAVs securely. To secure communication in this process, a local monitoring system is designed to counteract the wormhole attack through packet relay. This system checks data packets exchanged between neighboring UAVs and defines three rules according to the behavior of wormholes. In the second process, UAVs perform a distributed Q-learning-based routing process to counteract the wormhole attack through encapsulation. To reward the safest paths, a reward function is introduced based on five factors, the average one-hop delay, hop count, data loss ratio, packet transmission frequency (PTF), and packet reception frequency (PRF). Finally, the NS2 simulator is applied for implementing QSR and executing different scenarios. The evaluation results show that QSR works better than TOPCM, MNRiRIP, and MNDA in terms of accuracy, malicious node detection rate, data delivery ratio, and data loss ratio. However, it has more delay than TOPCM.

中文翻译：

---

一种新颖的基于 Q 学习的安全路由方案，具有针对飞行自组织网络中虫洞攻击的强大防御系统


如今，在飞行特设网络（FANET）中组织的无人机（UAV）可以成功执行复杂的任务。由于这些网络的局限性，包括缺乏基础设施、无线通信通道、动态拓扑以及无人机之间不可靠的通信，网络攻击，尤其是虫洞，削弱了路由方案的性能。因此，维护通信安全并保证服务质量（QoS）非常具有挑战性。在本文中，针对 FANET 提出了一种新颖的基于 Q 学习的安全路由方案（QSR）。 QSR 旨在提供一个强大的防御系统来抵御虫洞攻击，特别是通过封装的虫洞和通过数据包中继的虫洞。 QSR包括安全邻居发现过程和基于Q学习的安全路由过程。首先，每架无人机安全地获取其邻近无人机的信息。为了确保此过程中的通信安全，设计了本地监控系统，通过数据包中继来抵消虫洞攻击。该系统检查相邻无人机之间交换的数据包，并根据虫洞的行为定义了三个规则。在第二个过程中，无人机执行基于Q学习的分布式路由过程，通过封装来抵消虫洞攻击。为了奖励最安全的路径，引入了基于平均一跳延迟、跳数、数据丢失率、数据包传输频率（PTF）和数据包接收频率（PRF）五个因素的奖励函数。最后，应用NS2模拟器来实现QSR并执行不同的场景。评估结果表明，QSR在准确率、恶意节点检出率、数据送达率、数据丢失率等方面均优于TOPCM、MNRiRIP、MNDA。 然而，它比 TOPCM 有更多的延迟。