Confidence of expert ultrasound operators in making a diagnosis of adnexal tumor: effect on diagnostic accuracy and interobserver agreement.

OBJECTIVE: To assess the degree of confidence with which expert ultrasound operators make a diagnosis of benign, borderline and invasive malignant ovarian tumors and its effect on diagnostic accuracy and interobserver agreement. METHOD: Digitally stored static two-dimensional B-mode images of representative cases of benign, borderline and invasive malignant ovarian tumors were independently assessed by three expert ultrasound operators who had not performed the original real-time ultrasound examination. The experts classified the tumors as benign, borderline or invasive malignant and they also indicated the degree of confidence with which they made the diagnosis (certain, probable and uncertain). The diagnostic accuracy and interobserver agreement, in differentiating benign, borderline and invasive malignant ovarian tumors, were calculated depending on the level of confidence with which the diagnosis was made. RESULTS: One hundred and sixty-six cases were included in the final data analysis. The diagnostic accuracy of all three experts decreased with decreasing level of confidence. Interobserver agreement between any two experts was very high when they were certain of the diagnosis (rates of agreement 98%, 99% and 100%), but it was significantly lower with a moderate level of confidence (rates of agreement 78%, 71% and 76%) (P < 0.01 for any two experts). The agreement in both diagnosis and confidence was lowest in cases of borderline ovarian tumors compared to benign and primary invasive lesions. CONCLUSIONS: The accuracy of expert ultrasound operators using pattern recognition depends on the degree of certainty with which the diagnosis is made. Interobserver variability is also influenced by the operators' confidence in making the diagnosis. Our findings suggest that the level of confidence with which the diagnosis is made should be included in the ultrasound report.