External validation of ultrasound-based models for differentiating between benign and malignant adnexal masses: a nationwide prospective multicenter study (IOTA phase 6)

Background: The diagnostic performance of the IOTA methods, the O-RADS lexicon, and the RMI has been validated in prospective and retrospective studies, but most validation studies tested the performance in the hands of experienced ultrasound examiners. Objective: To prospectively validate the performance of the Risk of Malignancy Index, the International Ovarian Tumor Analysis Simple Rules Risk Model, the International Ovarian Tumor Analysis Assessment of Different NEoplasias in the adneXa model, and the International Ovarian Tumor Analysis 2-step strategy across different types of ultrasound centers in Italy. A retrospective post hoc analysis estimates malignancy prevalence in Ovarian-Adnexal Reporting and Data System risk groups when using the 2-step strategy or the Ovarian-Adnexal Reporting and Data System lexicon. Study design: This is a multicenter prospective observational study including regional referral centers and district hospitals in Italy. Methods: Consecutive patients with an adnexal mass examined with ultrasound by an International Ovarian Tumor Analysis–certified gynecologist with different levels of expertise were included, provided they underwent surgery <180 days after the inclusion scan. Ultrasound examination was performed transvaginally or transrectally and was supplemented with an abdominal scan when necessary. Reference standard was the histology of the adnexal mass following surgical removal. Discrimination (area under the receiver operating characteristic curve), calibration, and clinical utility were assessed to illustrate the diagnostic performance of the methods. For the retrospective post hoc analysis, we report the prevalence of malignancy in the Ovarian-Adnexal Reporting and Data System risk groups (Ovarian-Adnexal Reporting and Data System 2: risk of malignancy <1%; Ovarian-Adnexal Reporting and Data System 3: risk of malignancy 1% to <10%; Ovarian-Adnexal Reporting and Data System 4: risk of malignancy 10% to <50%; Ovarian-Adnexal Reporting and Data System 5: risk of malignancy ≥50%), with the Ovarian-Adnexal Reporting and Data System risk group assigned using either the 2-step strategy or the Ovarian-Adnexal Reporting and Data System lexicon. Results: Between May 2017 and March 2020, 1431 patients were enrolled from 21 Italian centers (10 oncological and 11 nononcological). Based on histology, 995 (69.5%) tumors were benign and 436 (30.5%) were malignant (115, 8.0% borderline; 263, 18.4% primary invasive; 58, 4.1% metastatic tumors). For Risk of Malignancy Index, the area under the receiver operating characteristic curve was 0.85 (95% confidence interval, 0.81 to 0.87), whereas for all International Ovarian Tumor Analysis models (Simple Rules Risk Model, Assessment of Different NEoplasias in the adneXa with and without CA125, and 2-step strategy with and without CA125), the area under the receiver operating characteristic curves ranged from 0.91 (95% confidence interval, 0.88–0.93) to 0.92 (95% confidence interval, 0.89–0.94). All International Ovarian Tumor Analysis models demonstrated a higher net benefit than Risk of Malignancy Index across risk thresholds (exchange rates) from 1% to 50%. All International Ovarian Tumor Analysis models slightly underestimated the risk of malignancy, but Simple Rules Risk Model showed the least degree of underestimation. The prevalence of malignancy and the corresponding 95% confidence interval in the 4 Ovarian-Adnexal Reporting and Data System risk categories, as calculated using the 2-step strategy and Ovarian-Adnexal Reporting and Data System lexicon, were 0.97% (95% confidence interval, 0.4–2.6) and 1.2% (95% confidence interval, 0.5–2.9) for Ovarian-Adnexal Reporting and Data System 2, 7.2% (95% confidence interval, 5.0–10.3) and 6.0% (95% confidence interval, 3.6–9.6) for Ovarian-Adnexal Reporting and Data System 3, 37.9% (95% confidence interval, 32.4–43.8) and 27.8% (95% confidence interval, 23.6–32.5) for Ovarian-Adnexal Reporting and Data System 4, and 84% (95% confidence interval, 79.8–87.4) and 83.1% (95% confidence interval, 79.0–86.6) for Ovarian-Adnexal Reporting and Data System 5. Conclusion: Risk of Malignancy Index had lower ability than the International Ovarian Tumor Analysis models to distinguish between benign and malignant adnexal tumors in patients examined by either expert or nonexpert ultrasound operators in Italy. All the International Ovarian Tumor Analysis models—including Simple Rules Risk Model, Assessment of Different NEoplasias in the adneXa, and the 2-step strategy with or without CA125—had similar ability. The prevalence of malignancy in each of the 4 Ovarian-Adnexal Reporting and Data System risk categories closely matched the assigned malignancy risk regardless of whether the 2-step strategy or Ovarian-Adnexal Reporting and Data System lexicon was used.