Background Copy Number Variations (CNVs) are critical genetic markers in diversity and disease, yet their accurate extraction from medical literature remains challenging due to the complexity of genetic data. While specialized NLP models like CNV-ETLAI have been developed for this task, the advent of Large Language Models (LLMs) such as GPT-4 presents a potential alternative with broader applicability. This study evaluates the efficacy of GPT-4 against CNV-ETLAI in extracting CNVs from medical journal articles, aiming to enhance genetic research and clinical decision-making. Methods We configured GPT-4 to process and interpret medical journal PDFs, designing custom prompts for CNV information extraction. The performance of GPT-4 was benchmarked against CNV-ETLAI using a dataset of 146 true positive CNVs extracted from 23 journal articles. Performance metrics focused on accuracy in extracting CNVs from both text and tables, recognizing the importance of structured data interpretation in genomic analysis. Results CNV-ETLAI demonstrated superior accuracy, achieving a 98% success rate in CNV extraction, compared to GPT-4’s 49%. Specifically, CNV-ETLAI outperformed GPT-4 in table extraction accuracy (99% vs. 41.2%) and context extraction accuracy (96% vs. 63.2%). Despite GPT-4's lower performance, its capacity for improvement and adaptability was noted, indicating potential future applicability in medical data extraction. Conclusions The study highlights CNV-ETLAI's current superiority in extracting CNVs from medical texts, particularly in interpreting structured data like tables. However, the adaptability and potential for growth in LLMs like GPT-4 suggest they could soon become valuable tools for medical data extraction, offering a more versatile and powerful solution across a broader range of applications. The promise of LLMs, despite their current limitations, underscores the need for continued research and development in AI technologies for genomic data interpretation.

中文翻译：

---

B-119 GPT-4 和 CNV-ETLAI 在从医学期刊中提取拷贝数变异方面的性能比较：缩小基因组数据解释中大型语言模型和专用 NLP 工具之间的差距


背景拷贝数变异（CNV）是多样性和疾病的关键遗传标记，但由于遗传数据的复杂性，从医学文献中准确提取它们仍然具有挑战性。虽然专门的 NLP 模型（如 CNV-ETLAI）已针对此任务开发出来，但大型语言模型 ( LLMs )（如 GPT-4）的出现提供了一种具有更广泛适用性的潜在替代方案。本研究评估了 GPT-4 相对于 CNV-ETLAI 从医学期刊文章中提取 CNV 的效果，旨在加强遗传研究和临床决策。方法 我们配置 GPT-4 来处理和解释医学期刊 PDF，设计用于 CNV 信息提取的自定义提示。 GPT-4 的性能使用从 23 篇期刊文章中提取的 146 个真阳性 CNV 数据集与 CNV-ETLAI 进行基准测试。性能指标侧重于从文本和表格中提取 CNV 的准确性，认识到结构化数据解释在基因组分析中的重要性。结果 CNV-ETLAI 表现出卓越的准确性，CNV 提取成功率达到 98%，而 GPT-4 的成功率为 49%。具体来说，CNV-ETLAI 在表提取准确度（99% vs. 41.2%）和上下文提取准确度（96% vs. 63.2%）方面优于 GPT-4。尽管 GPT-4 的性能较低，但其改进和适应性的能力受到关注，表明未来在医疗数据提取方面的潜在适用性。结论 该研究强调了 CNV-ETLAI 目前在从医学文本中提取 CNV 方面的优势，特别是在解释表格等结构化数据方面。 然而，像 GPT-4 这样的LLMs的适应性和增长潜力表明它们很快就会成为医疗数据提取的有价值的工具，为更广泛的应用提供更通用、更强大的解决方案。尽管LLMs目前存在局限性，但它的前景强调了继续研究和开发用于基因组数据解释的人工智能技术的必要性。