In high-speed railway systems, unexpected disruptions can result in delays of trains, significantly affecting the quality of service for passengers. Train Timetable Rescheduling (TTR) is a crucial task in the daily operation of high-speed railways to maintain punctuality and efficiency in the face of such unforeseen disruptions. Most existing studies on TTR are based on integer programming (IP) techniques and are required to solve IP models repetitively in case of disruptions, which however may be very time-consuming and greatly limit their usefulness in practice. Our study first proposes a multi-task deep reinforcement learning (MDRL) approach for TTR. Our MDRL is constructed and trained offline with a large number of historical disruptive events, enabling to generate TTR decisions in real-time for different disruption cases. Specifically, we transform the TTR problem into a Markov decision process considering the retiming and rerouting of trains. Then, we construct the MDRL framework with the definition of state, action, transition, reward, and value function approximations with neural networks for each agent (i.e., rail train), by considering the information of different disruption events as tasks. To overcome the low training efficiency and huge memory usage in the training of MDRL, given a large number of disruptive events in the historical data, we develop a new and high-efficient training method based on a Quadratic assignment programming (QAP) model and a Frank-Wolfe-based algorithm. Our QAP model optimizes only a small number but most “representative” tasks from the historical data, while the Frank-Wolfe-based algorithm approximates the nonlinear terms in the value function of MDRL and updates the model parameters among different training tasks concurrently. Finally, based on the real-world data from the Beijing–Zhangjiakou high-speed railway systems, we evaluate the performance of our MDRL approach by benchmarking it against state-of-the-art approaches in the literature. Our computational results demonstrate that an offline-trained MDRL is able to generate near-optimal TTR solutions in real-time against different disruption scenarios, and it evidently outperforms state-of-art models regarding solution quality and computational time.

中文翻译：

---

一种用于实时铁路列车重新调度的多任务深度强化学习方法


在高速铁路系统中，意外中断可能导致列车延误，从而严重影响乘客的服务质量。列车时刻表重新调度 （TTR） 是高速铁路日常运营中的一项关键任务，需要在面对此类不可预见的中断时保持准点率和效率。大多数关于 TTR 的现有研究都是基于整数规划 （IP） 技术，并且需要在中断的情况下重复求解 IP 模型，但这可能非常耗时，并极大地限制了它们在实践中的实用性。我们的研究首先提出了一种用于 TTR 的多任务深度强化学习 （MDRL） 方法。我们的 MDRL 是离线构建和训练的，具有大量历史中断事件，能够针对不同的中断案例实时生成 TTR 决策。具体来说，我们将 TTR 问题转化为考虑列车重定时和重新路由的马尔可夫决策过程。然后，我们将不同干扰事件的信息视为任务，构建了 MDRL 框架，其中包含每个代理（即铁路列车）的状态、动作、转换、奖励和价值函数近似的定义。为了克服 MDRL 训练中训练效率低、内存占用大的问题，鉴于历史数据中存在大量的破坏性事件，我们开发了一种基于二次赋值规划 （QAP） 模型和基于 Frank-Wolfe 的算法的新型高效训练方法。我们的 QAP 模型仅从历史数据中优化了一小部分但最具 “代表性” 的任务，而基于 Frank-Wolfe 的算法近似了 MDRL 值函数中的非线性项，并同时更新不同训练任务之间的模型参数。 最后，基于京张高速铁路系统的真实数据，我们通过与文献中最先进的方法进行基准测试来评估 MDRL 方法的性能。我们的计算结果表明，离线训练的 MDRL 能够针对不同的中断场景实时生成近乎最优的 TTR 解决方案，并且在解决方案质量和计算时间方面明显优于最先进的模型。