Penetration testing, a crucial industrial practice for securing networked systems and infrastructures, has traditionally depended on the extensive expertise of human professionals. Addressing the scarcity of human experts, the development of automated penetration testing tools emerges as a promising avenue. Against the backdrop of rapid advancements in artificial intelligence technologies, reinforcement learning has demonstrated considerable potential for realizing automated penetration testing. However, existing research predominantly concentrates on reinforcement learning-based automated penetration testing tools within static scenarios, with limited exploration in dynamic network environments. This paper addresses a noteworthy challenge in developing autonomous agents for real-world applications, particularly focusing on scenarios marked by environmental changes. Such alterations necessitate autonomous agents to continuously monitor environmental characteristics, and adapt, and adjust learned actions to ensure the system’s effective operation. Consequently, the paper proposes an automated reinforcement learning-based penetration testing scheme tailored for dynamic network scenarios, named DynPen. DynPen captures observed changes in the scenario, aiding the penetration testing agent in decision-making based on historical experiences. Simulation results demonstrate the proposed scheme’s efficacy in significantly expediting the convergence speed of the penetration testing agent using reinforcement learning algorithms. Furthermore, the scheme successfully maintains the learning agility and adaptability of the agent in dynamic network scenarios.

中文翻译：

---

DynPen：使用深度强化学习在动态网络场景中进行自动渗透测试


渗透测试是保护网络系统和基础设施的重要工业实践，传统上依赖于人类专业人员的广泛专业知识。为了解决人类专家的稀缺问题，开发自动化渗透测试工具成为一条很有前途的途径。在人工智能技术快速发展的背景下，强化学习在实现自动化渗透测试方面显示出巨大的潜力。然而，现有的研究主要集中在静态场景中基于强化学习的自动渗透测试工具，对动态网络环境的探索有限。本文解决了为实际应用开发自主代理的一个值得注意的挑战，特别是关注以环境变化为标志的场景。这种改变需要自主代理持续监控环境特征，并适应和调整学习到的行动，以确保系统的有效运行。因此，本文提出了一种为动态网络场景量身定制的基于强化学习的自动化渗透测试方案，名为 DynPen。DynPen 捕获场景中观察到的变化，帮助渗透测试代理根据历史经验做出决策。仿真结果表明，所提方案在使用强化学习算法显著加快渗透测试代理的收敛速度方面的有效性。此外，该方案成功地保持了智能体在动态网络场景中的学习敏捷性和适应性。