Recently, Mobility as a Service (MaaS) has garnered increasing attention by integrating various modes of transportation to provide users with a unified travel solution. However, In multimodal transportation planning, we primarily face three challenges: Firstly, a multimodal travel network is constructed that covers multiple travel modes and is highly scalable. Secondly, the routing algorithm fully considers the dynamic and real-time nature of the multimodal travel process. Finally, a generalized travel cost objective function is constructed that considers the psychological burden of transfers on passengers in multimodal travel scenarios. In this study, we firstly constructed an integrated multimodal transport network based on graph theory, which covers four transport modes, namely, the metro, the bus, the car-sharing and the walking. Subsequently, by introducing a double-Q learning mechanism and an optimized dynamic exploration strategy, we propose a new algorithm, Q_EDQ, the algorithm aims to learn the globally optimal path as efficiently as possible, with faster convergence speed and improved stability. Experiments utilizing real bus and metro data from Xi’an, Shaanxi Province, were conducted to compare the Q_EDQ algorithm with traditional genetic algorithms. In the conducted four experiments, compared to the optimal paths planned by traditional genetic algorithms, the improved Q-algorithm achieved a minimum efficiency increase of 12.52% and a maximum of 35%. These results demonstrate the enhanced capability of the improved Q-algorithm to learn globally optimal paths in complex multimodal transportation networks. Compared to the classical Q algorithm, the algorithmic model in this study shows an average performance improvement of 10% to 30% in global optimal path search, as well as convergence performance including loss and reward values.

中文翻译：

---

Q_EDQ：基于强化学习的多模式出行场景下高效路径规划


最近，移动即服务 （MaaS） 通过整合各种交通方式为用户提供统一的出行解决方案，引起了越来越多的关注。然而，在多式联运规划中，我们主要面临三个挑战：首先，构建一个涵盖多种出行方式且具有高度可扩展性的多式联运出行网络。其次，路线算法充分考虑了多式联运出行过程的动态性和实时性。最后，构建了一个广义旅行成本目标函数，该函数考虑了多模式出行场景中换乘对乘客的心理负担。本研究首先基于图论构建了一个综合多式联运网络，该网络涵盖了地铁、公交、共享汽车和步行四种交通方式。随后，通过引入双Q学习机制和优化的动态探索策略，我们提出了一种新的算法，Q_EDQ该算法旨在尽可能高效地学习全局最优路径，具有更快的收敛速度和更高的稳定性。利用来自陕西省习安市的真实公交和地铁数据进行了实验，将 Q_EDQ 算法与传统遗传算法进行了比较。在进行的 4 个实验中，与传统遗传算法规划的最优路径相比，改进的 Q 算法实现了最低 12.52% 的效率提升和最高 35% 的效率提升。这些结果表明，改进的 Q 算法在复杂的多式联运网络中学习全球最优路径的能力得到了增强。 与传统的 Q 算法相比，本研究中的算法模型在全局最优路径搜索中平均性能提高了 10% 到 30%，收敛性能（包括损失和奖励值）也提高了 10% 到 30%。