The advent of same-day delivery services has achieved immense popularity, driven by escalating customer expectations on fast shipping and the need for market competitiveness. To optimize such services, the use of heterogeneous fleets with vehicles and drones has proven effective in reducing the resource requirements needed for delivery. This paper focuses on investigating the same-day delivery dispatching and routing problem with a fleet of multiple vehicles and drones. In this problem, stochastic and dynamic requests, coupled with their stringent time constraints, require dispatchers to make real-time decisions about optimally assigning vehicles and drones, ensuring both efficiency and effectiveness in delivery operations while taking into account the routing. To tackle this complex problem, we model it with a route-based Markov decision process and develop a novel hierarchical decision approach based on deep reinforcement learning (HDDRL). The first level of the hierarchy is tasked with determining the departure times of vehicles, balancing the trade-offs between the delivery frequency and efficiency. The second level of the hierarchy is dedicated to determining the most suitable delivery mode for each request, whether by vehicles or drones. The third level is responsible for planning routes for vehicles and drones, thereby enhancing route efficiency. These three levels in the hierarchical framework collaborate to solve the problem in a synchronized manner, with the objective of maximizing the service requests within a day. Empirical results from computational experiments highlight the superiority of the HDDRL over benchmark methods, demonstrating not only its enhanced efficacy but also its robust generalization across diverse data distributions and fleet sizes. This underscores the HDDRL’s potential as a powerful tool for enhancing operational efficiency in same-day delivery services.

中文翻译：

---

优化车辆和无人机的当日交付：一种分层深度强化学习方法


在客户对快速配送的期望不断提高和对市场竞争力的需求的推动下，当日送达服务的出现已经广受欢迎。为了优化此类服务，使用带有车辆和无人机的异构车队已被证明可以有效减少交付所需的资源需求。本文重点介绍调查由多辆车和无人机组成的车队的当日送达调度和路线选择问题。在这个问题中，随机和动态请求，加上它们严格的时间限制，要求调度员做出关于优化分配车辆和无人机的实时决策，确保交付操作的效率和有效性，同时考虑到路线。为了解决这个复杂的问题，我们使用基于路线的马尔可夫决策过程对其进行建模，并开发了一种基于深度强化学习 （HDDRL） 的新型分层决策方法。层次结构的第一级任务是确定车辆的出发时间，平衡交付频率和效率之间的权衡。层次结构的第二级专门用于确定每个请求的最合适的交付模式，无论是通过车辆还是无人机。第三级负责规划车辆和无人机的路线，从而提高路线效率。分层框架中的这三个级别协作以同步方式解决问题，目标是在一天内最大化服务请求。计算实验的实证结果突出了 HDDRL 优于基准方法，不仅证明了其增强的有效性，还证明了它在不同数据分布和队列规模中的稳健泛化。 这凸显了 HDDRL 作为提高当日送达服务运营效率的强大工具的潜力。