Deep reinforcement learning has led to dramatic breakthroughs in the field of artificial intelligence for the past few years. As the amount of rollout experience data and the size of neural networks for deep reinforcement learning have grown continuously, handling the training process and reducing the time consumption using parallel and distributed computing is becoming an urgent and essential desire. In this paper, we perform a broad and thorough investigation on training acceleration methodologies for deep reinforcement learning based on parallel and distributed computing, providing a comprehensive survey in this field with state-of-the-art methods and pointers to core references. In particular, a taxonomy of literature is provided, along with a discussion of emerging topics and open issues. This incorporates learning system architectures, simulation parallelism, computing parallelism, distributed synchronization mechanisms, and deep evolutionary reinforcement learning. Further, we compare 16 current open-source libraries and platforms with criteria of facilitating rapid development. Finally, we extrapolate future directions that deserve further research.

中文翻译：

---

使用并行和分布式计算加速深度强化学习：一项调查


在过去的几年里，深度强化学习导致了人工智能领域的巨大突破。随着用于深度强化学习的推出体验数据量和神经网络规模的不断增长，使用并行和分布式计算处理训练过程并减少时间消耗正成为一种紧迫而重要的愿望。在本文中，我们对基于并行和分布式计算的深度强化学习的训练加速方法进行了广泛而彻底的研究，提供了该领域的全面调查，提供了最先进的方法和核心参考资料的指针。特别是，提供了文学分类法，以及对新兴话题和开放问题的讨论。这包括学习系统架构、仿真并行、计算并行、分布式同步机制和深度进化强化学习。此外，我们将当前的 16 个开源库和平台与促进快速开发的标准进行了比较。最后，我们推断出值得进一步研究的未来方向。