STUDY OBJECTIVE To use artificial intelligence (AI) to predict billing code levels for emergency department (ED) encounters. METHODS We accessed ED encounters from our health system from January to September 2023. We developed an ensemble model using natural language processing and machine learning techniques to predict billing codes from clinical notes combined with clinical characteristics and orders. Explainable AI techniques were used to help determine the important model features. The main endpoint was to predict evaluation and management professional billing codes (levels 2 to 5 [Current Procedural Terminology codes 99282 to 99285] and critical care). Secondary endpoints included predicting professional billing codes at different decision boundary thresholds and generalizability of the model at other EDs. RESULTS There were 321,893 adult ED encounters coded at levels 2 (<1%), 3 (5%), 4 (38%), 5 (51%), and critical care (5%). Model performance for professional billing code levels of 4 and 5 yielded area under the receiver operating characteristic curve values of 0.94 and 0.95, accuracy values of 0.80 and 0.92, and F1-scores of 0.79 and 0.91, respectively. At a 95% decision boundary threshold, level 5 predicted charts had a precision/positive predictive value of 0.99 and recall/sensitivity of 0.57. The most important features using Shapley Additive Explanations values were critical care note, number of orders, discharge disposition, cardiology, and psychiatry. CONCLUSION Currently available AI models accurately predict billing code levels for ED encounters based on clinical notes, clinical characteristics, and orders. This has the potential to automate coding of ED encounters and save administrative costs and time.

中文翻译：

---

人工智能预测急诊室遭遇的计费代码级别。


研究目标 使用人工智能 (AI) 预测急诊科 (ED) 遇到的账单代码级别。方法 我们从 2023 年 1 月至 9 月从我们的卫生系统中获取了急诊室就诊情况。我们使用自然语言处理和机器学习技术开发了一个集成模型，根据临床记录并结合临床特征和订单来预测账单代码。可解释的人工智能技术被用来帮助确定重要的模型特征。主要终点是预测评估和管理专业计费代码（2 级至 5 级[当前程序术语代码 99282 至 99285] 和重症监护）。次要终点包括在不同决策边界阈值下预测专业计费代码以及模型在其他 ED 的通用性。结果 321,893 例成人 ED 就诊次数编码为 2 级 (<1%)、3 级 (5%)、4 级 (38%)、5 级 (51%) 和重症监护 (5%)。专业计费代码级别 4 和 5 的模型性能分别产生接收器操作特征曲线下的面积值 0.94 和 0.95、准确度值 0.80 和 0.92、F1 分数分别为 0.79 和 0.91。在 95% 决策边界阈值下，5 级预测图表的精确度/阳性预测值为 0.99，召回率/敏感性为 0.57。使用 Shapley 附加解释值的最重要特征是重症监护病历、医嘱数量、出院处置、心脏病学和精神病学。结论 目前可用的人工智能模型根据临床记录、临床特征和订单准确预测 ED 就诊的账单代码级别。这有可能实现 ED 就诊的自动化编码，并节省管理成本和时间。