Access requests are used in cooperative tasks among Connected and Automated Vehicles (CAVs). Authorization decisions are determined by dynamic trust values in Trust Value-based Access Control (TBAC). Manual policy management in TBAC faces challenges when dealing with massive amounts of access data and dynamic environments. Traditional machine learning methods fail to adequately represent the complex spatio-temporal, decentralized and independent access requests. VPolicy-GRL (Policy Learning for Connected Autonomous Vehicles by Graph Reinforcement Learning) is proposed, enhancing the conventional Deep Q Network (DQN) with a graph neural network (GNN) framework. Firstly, a novel three-layer framework of TBAC for CAVs is proposed. Secondly, trust relations among CAVs are represented as several spatio-temporal dynamic graphs, enabling context-aware authorization decisions. Malicious access requests are categorized into unauthorized access requests caused by Sybil attacks and interfering access requests caused by DoS attacks. Thirdly, the DQN model for VPolicy-GRL is designed. Results demonstrate that VPolicy-GRL achieves superior feature abstraction with STGCN compared to GCN, which also obtains higher true rejection rates than Double DQN and DQN for both unauthorized and interfering access requests. Additionally, VPolicy-GRL exhibits improved convergence speed and stability.

中文翻译：

---

联网自动驾驶车辆的自适应策略学习通过图强化学习防御恶意访问请求


访问请求用于联网和自动车辆 (CAV) 之间的协作任务。授权决策由基于信任值的访问控制 (TBAC) 中的动态信任值决定。 TBAC 中的手动策略管理在处理大量访问数据和动态环境时面临挑战。传统的机器学习方法无法充分表示复杂的时空、分散和独立的访问请求。提出了 VPolicy-GRL（通过图强化学习进行互联自动驾驶车辆的策略学习），用图神经网络（GNN）框架增强了传统的深度 Q 网络（DQN）。首先，提出了一种新颖的 CAV TBAC 三层框架。其次，CAV 之间的信任关系表示为多个时空动态图，从而实现上下文感知的授权决策。恶意访问请求分为Sybil攻击引起的未授权访问请求和DoS攻击引起的干扰访问请求。第三，设计了VPolicy-GRL的DQN模型。结果表明，与 GCN 相比，VPolicy-GRL 通过 STGCN 实现了卓越的特征抽象，对于未经授权和干扰的访问请求，其真实拒绝率也比 Double DQN 和 DQN 更高。此外，VPolicy-GRL 还表现出更高的收敛速度和稳定性。