Coordinated truck and drone delivery is gaining popularity in logistics as it can greatly reduce operation costs. However, existing studies on related operations management problems typically ignore the following important features: (i) the random appearance of requests, which require operators to dynamically respond to the requests; and (ii) the decisions of optimal launch and retrieval locations for trucks and drones instead of fixed to customer locations, which can significantly impact the overall time costs. To tackle these challenges, this study investigates the dynamic collaborative truck-drone routing problem with randomly arriving requests and synchronization on routes. We model the problem as a Markov Decision Process (MDP) and solve the MDP via a reinforcement learning (RL) approach. The proposed RL approach determines: (i) whether each request should be serviced upon arrival, (ii) which truck or drone should be assigned for the request, and (iii) the optimal en-route take-off and landing positions for paired trucks and drones. We further employ a framework of decentralized learning and centralized dispatching in RL to increase performance. Numerical experiments are conducted to assess the proposed solution approach on instances generated based on both the Solomon dataset and real-world operational data of a logistics operator in Singapore over several benchmark algorithms under various battery endurance levels of drones and distinct transportation scenarios including node-based dynamic collaborative truck-drone routing problem, dynamic non-collaborative truck and drone routing problem, and dynamic vehicle routing problem. The results show that our RL solution outperforms the benchmark algorithm in total profit by an average of 28.03 %, and our en-route takeoff and landing scenario outperforms the benchmark scenarios in total profit by an average of 8.43 % in multi-day instances. Additionally, compared to the traditional node-based landing scenario, employing our en-route takeoff and landing strategy can save 0.9 h/(drone*day) of waiting time on average.

中文翻译：

---

具有途中同步和随机请求的动态协作卡车-无人机交付


卡车和无人机的协调交付在物流中越来越受欢迎，因为它可以大大降低运营成本。然而，关于相关运营管理问题的现有研究通常忽略了以下重要特征：（i） 请求的随机出现，这需要操作员动态响应请求;（ii） 决定卡车和无人机的最佳发射和检索位置，而不是固定在客户位置，这可能会显着影响总体时间成本。为了应对这些挑战，本研究调查了随机到达请求和路线同步的动态协作卡车-无人机路线问题。我们将问题建模为马尔可夫决策过程 （MDP），并通过强化学习 （RL） 方法解决 MDP。拟议的 RL 方法确定：（i） 是否应在抵达时处理每个请求，（ii） 应为请求分配哪辆卡车或无人机，以及 （iii） 成对卡车和无人机的最佳途中起飞和降落位置。我们进一步在 RL 中采用去中心化学习和集中调度框架来提高性能。在无人机的各种电池续航能力水平和不同的运输场景下，基于所罗门数据集和新加坡物流运营商的真实世界运营数据生成的实例上进行了数值实验，以评估所提出的求解方法，这些实例基于多种基准算法，包括基于节点的动态协同卡车-无人机路线问题、动态非协作卡车和无人机路线问题， 以及动态车辆路径问题。结果表明，我们的 RL 解决方案在总利润方面平均比基准算法高出 28 倍。03 %，我们的途中起飞和降落场景在多日实例的总利润方面平均比基准场景高出 8.43%。此外，与传统的基于节点的着陆场景相比，采用我们的航路起降策略平均可以节省 0.9 小时/（无人机*天）的等待时间。