Swarm robots offer fascinating opportunities to perform complex tasks beyond the capabilities of individual machines. Just as a swarm of ants collectively moves large objects, similar functions can emerge within a group of robots through individual strategies based on local sensing. However, realizing collective functions with individually controlled microrobots is particularly challenging because of their micrometer size, large number of degrees of freedom, strong thermal noise relative to the propulsion speed, and complex physical coupling between neighboring microrobots. Here, we implemented multiagent reinforcement learning (MARL) to generate a control strategy for up to 200 microrobots whose motions are individually controlled by laser spots. During the learning process, we used so-called counterfactual rewards that automatically assign credit to the individual microrobots, which allows fast and unbiased training. With the help of this efficient reward scheme, swarm microrobots learn to collectively transport a large cargo object to an arbitrary position and orientation, similar to ant swarms. We show that this flexible and versatile swarm robotic system is robust to variations in group size, the presence of malfunctioning units, and environmental noise. In addition, we let the robot swarms manipulate multiple objects simultaneously in a demonstration experiment, highlighting the benefits of distributed control and independent microrobot motion. Control strategies such as ours can potentially enable complex and automated assembly of mobile micromachines, programmable drug delivery capsules, and other advanced lab-on-a-chip applications.

中文翻译：

---

反事实奖励使用独立控制的群体微型机器人促进集体运输


集群机器人提供了绝佳的机会来执行超出单个机器能力的复杂任务。就像一群蚂蚁集体移动大型物体一样，通过基于局部传感的个体策略，一组机器人中也可以出现类似的功能。然而，由于微型机器人的微米尺寸、较大的自由度、相对于推进速度的强热噪声以及相邻微型机器人之间复杂的物理耦合，使用单独控制的微型机器人实现集体功能尤其具有挑战性。在这里，我们实现了多智能体强化学习 （MARL），为多达 200 个微型机器人生成控制策略，这些机器人的运动由激光光斑单独控制。在学习过程中，我们使用了所谓的反事实奖励，这些奖励会自动为单个微型机器人分配学分，从而实现快速和公正的训练。在这种高效的奖励计划的帮助下，群体微型机器人学会了将大型货物物体集体运送到任意位置和方向，类似于蚂蚁群。我们表明，这种灵活且多功能的集群机器人系统对群体规模的变化、故障单元的存在和环境噪声具有强大的能力。此外，在演示实验中，我们让机器人集群同时操纵多个物体，突出了分布式控制和独立微机器人运动的好处。像我们这样的控制策略有可能实现移动微型机器、可编程给药胶囊和其他高级芯片实验室应用的复杂和自动化组装。