Background

Program directors use US Medical Licensing Exam (USMLE) scores as criteria for ranking applicants. First-time pass rates of the American Board of Surgery (ABS) Qualifying (QE) and Certifying (CE) Examinations have become important measures of residency program quality. USMLE Step 1 will become pass/fail in 2022.

Study design

American Board of Surgery QE and CE success rates were assessed considering multiple characteristics of highly ranked (top 20) applicants to 22 general surgery programs in 2011. Chi-square, t-test, Wilcoxon Rank sum, linear and logistic regression were used, as appropriate.

Results

The QE and CE first attempt pass rates were 96% (235/244) and 86% (190/221), respectively. QE/CE success was not significantly associated with sex, race, research experience, or publications. Alpha Omega Alpha (AΩA) status was associated with success on the index CE (98% vs 83%; p = 0.008). Step 1 and Step 2 Clinical Knowledge (CK) scores of surgeons who passed QE on their first attempt were higher than scores of those who failed (Step 1: 233 vs 218; p = 0.016); (Step 2CK: 244 vs 228, p = 0.009). For every 10-point increase in Step 1 and 2CK scores, the odds of passing CE on the first attempt increased 1.5 times (95% CI 1.12, 2.0; p = 0.006) and 1.5 times (95% CI 1.11, 2.02, p = 0.008), respectively. For every 10-point increase in Steps 1 and 2CK scores, the odds of passing the QE on the first attempt increased 1.85 times (95% CI 1.11, 3.09; p = 0.018) and 1.86 times (95% CI 1.14, 3.06, p = 0.013), respectively.

Conclusions

USMLE Step 1 and Step 2 CK examination scores correlate with American Board of Surgery QE and CE performance and success. The USMLE decision to transition Step 1 to a pass/fail examination will require program directors to identify other factors that predict ABS performance for ranking applicants.