Robotic systems are becoming pervasive and adopted in increasingly many domains, such as manufacturing, healthcare, and space exploration. To this end, engineering software has emerged as a crucial discipline for building maintainable and reusable robotic systems. The field of robotics software engineering research has received increasing attention, fostering autonomy as a fundamental goal. However, robotics developers are still challenged trying to achieve this goal given that simulation is not able to deliver solutions to realistically emulate real-world phenomena. Robots also need to operate in unpredictable and uncontrollable environments, which require safe and trustworthy self-adaptation capabilities implemented in software. Typical techniques to address the challenges are runtime verification, field-based testing, and mitigation techniques that enable fail-safe solutions. However, there is no clear guidance to architect ROS-based systems to enable and facilitate runtime verification and field-based testing. This paper aims to fill in this gap by providing guidelines that can help developers and quality assurance (QA) teams when developing, verifying or testing their robots in the field. These guidelines are carefully tailored to address the challenges and requirements of testing robotics systems in real-world scenarios. We conducted (i) a literature review on studies addressing runtime verification and field-based testing for robotic systems, (ii) mined ROS-based applications repositories, and (iii) validated the applicability, clarity, and usefulness via two questionnaires with 55 answers overall. We contribute 20 guidelines: 8 for developers and 12 for QA teams formulated for researchers and practitioners in robotic software engineering. Finally, we map our guidelines to open challenges thus far in runtime verification and field-based testing for ROS-based systems and, we outline promising research directions in the field. Guidelines website and replication package: https://ros-rvft.github.io .

中文翻译：

---

基于 ROS 的机器人系统的运行时验证和现场测试


机器人系统正变得越来越普遍，并被越来越多的领域采用，例如制造、医疗保健和太空探索。为此，工程软件已成为构建可维护和可重用机器人系统的关键学科。机器人软件工程研究领域受到越来越多的关注，将自主性作为基本目标。然而，鉴于仿真无法提供逼真仿真真实世界现象的解决方案，机器人开发人员在尝试实现这一目标时仍然面临挑战。机器人还需要在不可预测的和不可控的环境中运行，这需要在软件中实现安全可信的自适应能力。应对这些挑战的典型技术是运行时验证、基于现场的测试和缓解技术，这些技术可实现故障安全解决方案。但是，对于构建基于 ROS 的系统以支持和促进运行时验证和基于现场的测试，没有明确的指导。本白皮书旨在通过提供指南来填补这一空白，这些指南可以帮助开发人员和质量保证 （QA） 团队在现场开发、验证或测试他们的机器人。这些指南经过精心定制，旨在解决在实际场景中测试机器人系统的挑战和要求。我们进行了 （i） 关于机器人系统运行时验证和现场测试研究的文献综述，（ii） 挖掘基于 ROS 的应用程序存储库，以及 （iii） 通过两份问卷验证了适用性、清晰度和有用性，总共有 55 个答案。我们贡献了 20 条指南：8 条针对开发人员，12 条针对 QA 团队，专为机器人软件工程领域的研究人员和从业者制定。 最后，我们将我们的指南映射到迄今为止基于 ROS 的系统的运行时验证和现场测试中面临的开放挑战，并概述了该领域有前景的研究方向。准则网站和复制包： https://ros-rvft.github.io .