BACKGROUND Up to half of all surgical adverse events are due to non-technical errors, making non-technical skill assessment and improvement a priority. No specific tools are available to retrospectively identify non-technical errors that have occurred in surgical patient care. This original study aimed to develop and provide evidence of validity and inter-rater reliability for the System for Identification and Categorization of Non-technical Error in Surgical Settings (SICNESS). METHODS A literature review, modified Delphi process, and two pilot phases were used to develop and test the SICNESS tool. For each pilot, 12 months of surgical mortality data from the Australian and New Zealand Audit of Surgical Mortality were assessed by two independent reviewers using the SICNESS tool. Main outcomes included tool validation through modified Delphi consensus, and inter-rater reliability for: non-technical error identification and non-technical error categorization using Cohen's κ coefficient, and overall agreement using Fleiss' κ coefficient. RESULTS Version 1 of the SICNESS was used for pilot 1, including 412 mortality cases, and identified and categorized non-technical errors with strong-moderate inter-rater reliability. Non-technical error exemplars were created and validated through Delphi consensus, and a novel mental model was developed. Pilot 2 included an additional 432 mortality cases. Inter-rater reliability was near perfect for leadership (κ 0.92, 95% c.i. 0.82 to 1.00); strong for non-technical error identification (κ 0.89, 0.84 to 0.93), communication and teamwork (κ 0.89, 0.79 to 0.99), and decision-making (κ 0.85, 0.79 to 0.92); and moderate for situational awareness (κ 0.79, 0.71 to 0.87) and overall agreement (κ 0.69, 0.66 to 0.73). CONCLUSION The SICNESS is a reliable and valid tool, enabling retrospective identification and categorization of non-technical errors associated with death, occurring in real surgical patient interactions.

中文翻译：

---

开发和验证一种用于识别和分类与手术死亡率相关的非技术错误的新工具。


背景 多达一半的手术不良事件是由于非技术错误造成的，这使得非技术技能评估和改进成为优先事项。没有特定的工具可用于回顾性识别外科患者护理中发生的非技术错误。这项原始研究旨在为手术环境中非技术错误识别和分类系统 （SICNESS） 开发和提供有效性和评分者间可靠性的证据。方法 采用文献综述、改良的 Delphi 过程和两个试点阶段来开发和测试 SICNESS 工具。对于每个试点，两名独立评价员使用 SICNESS 工具评估了来自澳大利亚和新西兰手术死亡率审计的 12 个月的手术死亡率数据。主要结果包括通过修改后的 Delphi 共识进行工具验证，以及以下方面的评分者间可靠性：使用 Cohen 的 κ 系数进行非技术错误识别和非技术性错误分类，以及使用 Fleiss 的 κ 系数进行总体一致性。结果 SICNESS 的第 1 版用于试点 1，包括 412 例死亡病例，并识别和分类具有强-中等评分者间可靠性的非技术错误。通过 Delphi 共识创建和验证了非技术性错误示例，并开发了一种新的心智模型。试点 2 包括额外的 432 例死亡病例。评分者间的可靠性对于领导来说几乎是完美的（κ 0.92,95% ci 0.82 到 1.00）;在非技术错误识别（κ 0.89、0.84 至 0.93）、沟通和团队合作（κ 0.89、0.79 至 0.99）和决策（κ 0.85、0.79 至 0.92）方面表现出色;态势感知 （κ 0.79， 0.71 至 0.87） 和总体一致性 （κ 0.69， 0.66 至 0.73） 为中等。 结论 SICNESS 是一种可靠且有效的工具，能够对真实手术患者互动中发生的与死亡相关的非技术错误进行回顾性识别和分类。