Compute resources that enable Continuous Integration (CI, i.e., the automatic build and test cycle applied to the change sets that development teams produce) are a shared commodity that organizations need to manage. To prevent (erroneous) builds from consuming a large amount of resources, CI service providers often impose a time limit. CI builds that exceed the time limit are automatically terminated. While imposing a time limit helps to prevent abuse of the service, builds that timeout (a) consume the maximum amount of resources that a CI service is willing to provide and (b) leave CI users without an indication of whether the change set will pass or fail the CI process. Therefore, understanding timeout builds and the factors that contribute to them is important for improving the stability and quality of a CI service. In this paper, we investigate the prevalence of timeout builds and the characteristics associated with them. By analyzing a curated dataset of 936 projects that adopt the CircleCI service and report at least one timeout build, we find that the median duration of a timeout build (19.7 minutes) is more than five times that of a build that produces a pass or fail result (3.4 minutes). To better understand the factors contributing to timeout builds, we model timeout builds using characteristics of project build history, build queued time, timeout tendency, size, and author experience based on data collected from 105,663 CI builds. Our model demonstrates a discriminatory power that vastly surpasses that of a random predictor (Area Under the Receiver Operating characteristic Curve, i.e., $AUROC$ = 0.939) and is highly stable in its performance ( $AUROC$ optimism = 0.0001). Moreover, our model reveals that the build history and timeout tendency features are strong indicators of timeout builds, with the timeout status of the most recent build accounting for the largest proportion of the explanatory power. A longitudinal analysis of the incidences of timeout builds (i.e., a study conducted over a period of time) indicates that 64.03% of timeout builds occur consecutively. In such cases, it takes a median of 24 hours before a build that passes or fails occurs. Our results imply that CI providers should exploit build history to anticipate timeout builds.

中文翻译：

---

描述持续集成中的超时构建


支持持续集成（CI，即应用于开发团队生成的变更集的自动构建和测试周期）的计算资源是组织需要管理的共享商品。为了防止（错误的）构建消耗大量资源，CI 服务提供商通常会施加时间限制。超过时间限制的 CI 构建将自动终止。虽然施加时间限制有助于防止滥用服务，但会导致超时 (a) 消耗 CI 服务愿意提供的最大资源量，以及 (b) 让 CI 用户不知道更改集是否会通过或者 CI 过程失败。因此，了解超时构建及其影响因素对于提高 CI 服务的稳定性和质量非常重要。在本文中，我们研究了超时构建的普遍性以及与之相关的特征。通过分析采用 CircleCI 服务并报告至少一次超时构建的 936 个项目的精选数据集，我们发现超时构建的中位持续时间（19.7 分钟）是产生通过或失败的构建的五倍多结果（3.4 分钟）。为了更好地了解导致超时构建的因素，我们根据从 105,663 个 CI 构建收集的数据，使用项目构建历史、构建排队时间、超时趋势、大小和作者经验的特征来对超时构建进行建模。我们的模型展示了远远超过随机预测器的辨别力（接收器操作特征曲线下的面积，即$AUROC$ = 0。939）并且性能高度稳定（ $AUROC$乐观= 0.0001）。此外，我们的模型表明，构建历史和超时趋势特征是超时构建的有力指标，其中最近构建的超时状态占解释力的最大比例。对超时构建发生率的纵向分析（即在一段时间内进行的研究）表明，64.03% 的超时构建是连续发生的。在这种情况下，构建通过或失败的时间平均需要 24 小时。我们的结果表明 CI 提供商应该利用构建历史来预测超时构建。