In many clinical and research settings, the scarcity of high-quality medical imaging datasets has hampered the potential of artificial intelligence (AI) clinical applications. This issue is particularly pronounced in less common conditions, underrepresented populations and emerging imaging modalities, where the availability of diverse and comprehensive datasets is often inadequate. To address this challenge, we introduce a unified medical image–text generative model called MINIM that is capable of synthesizing medical images of various organs across various imaging modalities based on textual instructions. Clinician evaluations and rigorous objective measurements validate the high quality of MINIM’s synthetic images. MINIM exhibits an enhanced generative capability when presented with previously unseen data domains, demonstrating its potential as a generalist medical AI (GMAI). Our findings show that MINIM’s synthetic images effectively augment existing datasets, boosting performance across multiple medical applications such as diagnostics, report generation and self-supervised learning. On average, MINIM enhances performance by 12% for ophthalmic, 15% for chest, 13% for brain and 17% for breast-related tasks. Furthermore, we demonstrate MINIM’s potential clinical utility in the accurate prediction of HER2-positive breast cancer from MRI images. Using a large retrospective simulation analysis, we demonstrate MINIM’s clinical potential by accurately identifying targeted therapy-sensitive EGFR mutations using lung cancer computed tomography images, which could potentially lead to improved 5-year survival rates. Although these results are promising, further validation and refinement in more diverse and prospective settings would greatly enhance the model’s generalizability and robustness.

在许多临床和研究环境中，高质量医学成像数据集的稀缺阻碍了人工智能 （AI） 临床应用的潜力。这个问题在不太常见的疾病、代表性不足的人群和新兴的成像方式中尤为明显，在这些领域中，多样化和全面的数据集的可用性往往不足。为了应对这一挑战，我们引入了一种名为 MINIM 的统一医学图像-文本生成模型，该模型能够根据文本指令在各种成像模式中合成各种器官的医学图像。临床医生的评估和严格的客观测量验证了 MINIM 合成图像的高质量。MINIM 在面对以前从未见过的数据域时表现出增强的生成能力，展示了其作为通才医疗 AI （GMAI） 的潜力。我们的研究结果表明，MINIM 的合成图像有效地增强了现有数据集，提高了多种医疗应用的性能，例如诊断、报告生成和自我监督学习。平均而言，MINIM 将眼科、胸部、大脑和 17% 的乳房相关任务提高 12%、15% 和 13%。此外，我们证明了 MININ 在从 MRI 图像准确预测 HER2 阳性乳腺癌方面的潜在临床效用。使用大型回顾性模拟分析，我们通过使用肺癌计算机断层扫描图像准确识别靶向治疗敏感的 EGFR 突变来证明 MINIM 的临床潜力，这有可能提高 5 年生存率。 尽管这些结果是有希望的，但在更多样化和前瞻性的环境中进一步验证和改进将大大提高模型的泛化性和稳健性。