Learning from Multiple Annotators: An Ensemble Model-Based Classification Approach

Supervised learning with multiple sets of noisy labels presents a complex challenge, arising when several annotators are required to manually label the same training samples, potentially resulting in inconsistencies in class assignments compared to the ground truth. To efficiently learn a classifier in this context, an ensemble approach is developed by leveraging model-based discriminant analysis trained individually on distinct sets of noisy labels. Several strategies are proposed to combine the base learners, extending solutions proposed in the literature for the binary classification setting to the multi-class framework. An application involving the identification of gastrointestinal lesions from colonoscopic videos, revised by seven clinicians, demonstrates the applicability of our proposal.