Statistical model checking (SMC) is a simulation-based formal verification technique to deal with the scalability problem faced by traditional model checking. The main workflow of SMC is to perform iterative simulations. The number of simulations depends on users’ requirement for the verification results, which can be very large if users require a high level of confidence and precision. Therefore, how to perform as fewer simulations as possible while achieving the same level of confidence and precision is one of the core problems of SMC. In this paper, we consider the estimation problem of SMC. Most existing statistical model checkers use the Okamoto bound to decide the simulation number. Although the Okamoto bound is sound, it is well known to be overly conservative. The simulation number decided by the Okamoto bound is usually much higher than it actually needs, which leads to a waste of time and computation resources. To tackle this problem, we propose an efficient, sound and lightweight estimation algorithm using the Clopper-Pearson confidence interval. We perform comprehensive numerical experiments and case studies to evaluate the performance of our algorithm, and the results show that our algorithm uses 40%-60% fewer simulations than the Okamoto bound. Our algorithm can be directly integrated into existing model checkers to reduce the verification time of SMC estimation problems.

中文翻译：

---

用于高效统计模型检查估计的 Clopper-Pearson 算法


统计模型检查（SMC）是一种基于模拟的形式验证技术，用于解决传统模型检查面临的可扩展性问题。 SMC的主要工作流程是进行迭代模拟。模拟的次数取决于用户对验证结果的要求，如果用户要求高的置信度和精度，模拟的次数可能会非常大。因此，如何在达到相同水平的置信度和精度的情况下进行尽可能少的模拟是SMC的核心问题之一。在本文中，我们考虑SMC的估计问题。大多数现有的统计模型检查器使用冈本界限来决定模拟数。尽管冈本界限很合理，但众所周知它过于保守。冈本界决定的模拟数通常远高于实际需要，这导致时间和计算资源的浪费。为了解决这个问题，我们提出了一种使用 Clopper-Pearson 置信区间的高效、可靠且轻量级的估计算法。我们进行了全面的数值实验和案例研究来评估我们算法的性能，结果表明我们的算法使用的模拟量比冈本界限少 40%-60%。我们的算法可以直接集成到现有的模型检查器中，以减少 SMC 估计问题的验证时间。