Objectives: Artificial intelligence (AI)-based health technologies (AIHTs) have already been applied in clinical practice. However, there is currently no standardized framework for evalu- ating them based on the principles of health technology assessment (HTA). Methods: A two-round Delphi survey was distributed to a panel of experts to determine the significance of incorporating topics outlined in the EUnetHTA Core Model and twenty additional ones identified through literature reviews. Each panelist assigned scores to each topic. Topics were categorized as critical to include (scores 7–9), important but not critical (scores 4–6), and not important (scores 1–3). A 70 percent cutoff was used to determine high agreement. Results: Our panel of 46 experts indicated that 48 out of the 65 proposed topics are critical and should be included in an HTA framework for AIHTs. Among the ten most crucial topics, the following emerged: accuracy of the AI model (97.78 percent), patient safety (95.65 percent), benefit–harm balance evaluated from an ethical standpoint (95.56 percent), and bias in data (91.30 percent). Importantly, our findings highlight that the Core Model is insufficient in capturing all relevant topics for AI-based technologies, as 14 out of the additional 20 topics were identified as crucial. Conclusion: It is imperative to determine the level of agreement on AI-relevant HTA topics to establish a robust assessment framework. This framework will play a foundational role in evaluating AI tools for the early diagnosis of dementia, which is the focus of the European project AI-Mind currently being developed.

Di Bidino, R., Daugbjerg, S. B., Papavero, S. C., Haraldsen, I. H., Cicchetti, A., Sacchini, D., Health technology assessment framework for artificial intelligence-based technologies., <<International Journal of Technology Assessment in Health Care>>, 2024; International Journal of Technology Assessment in Health Care (40(1)): 1-9. [doi:https://doi.org/10.1017/S0266462324000308] [https://hdl.handle.net/10807/299017]

Health technology assessment framework for artificial intelligence-based technologies.

Di Bidino, Rossella
Membro del Collaboration Group
;
Daugbjerg, Signe B.
Membro del Collaboration Group
;
Papavero, Sara Consilia
Membro del Collaboration Group
;
Cicchetti, Americo
Membro del Collaboration Group
;
Sacchini, Dario
Membro del Collaboration Group
2024

Abstract

Objectives: Artificial intelligence (AI)-based health technologies (AIHTs) have already been applied in clinical practice. However, there is currently no standardized framework for evalu- ating them based on the principles of health technology assessment (HTA). Methods: A two-round Delphi survey was distributed to a panel of experts to determine the significance of incorporating topics outlined in the EUnetHTA Core Model and twenty additional ones identified through literature reviews. Each panelist assigned scores to each topic. Topics were categorized as critical to include (scores 7–9), important but not critical (scores 4–6), and not important (scores 1–3). A 70 percent cutoff was used to determine high agreement. Results: Our panel of 46 experts indicated that 48 out of the 65 proposed topics are critical and should be included in an HTA framework for AIHTs. Among the ten most crucial topics, the following emerged: accuracy of the AI model (97.78 percent), patient safety (95.65 percent), benefit–harm balance evaluated from an ethical standpoint (95.56 percent), and bias in data (91.30 percent). Importantly, our findings highlight that the Core Model is insufficient in capturing all relevant topics for AI-based technologies, as 14 out of the additional 20 topics were identified as crucial. Conclusion: It is imperative to determine the level of agreement on AI-relevant HTA topics to establish a robust assessment framework. This framework will play a foundational role in evaluating AI tools for the early diagnosis of dementia, which is the focus of the European project AI-Mind currently being developed.
2024
Inglese
Di Bidino, R., Daugbjerg, S. B., Papavero, S. C., Haraldsen, I. H., Cicchetti, A., Sacchini, D., Health technology assessment framework for artificial intelligence-based technologies., <<International Journal of Technology Assessment in Health Care>>, 2024; International Journal of Technology Assessment in Health Care (40(1)): 1-9. [doi:https://doi.org/10.1017/S0266462324000308] [https://hdl.handle.net/10807/299017]
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10807/299017
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact