ASGE at DDW AI in GI Workshop | May 2022-The Need for AI in GI practice: Why and How to Evaluate AI Devices for Clinical Significance?

Pdf Summary

In this document, Dr. Cesare Hassan discusses the need for artificial intelligence (AI) in gastroenterology practice and how to evaluate AI devices for their clinical significance. The author emphasizes that AI should reduce variability in the quality of healthcare by being consistent, ubiquitous, and comprehensive. However, there are potential sources of variability in AI outcomes that cannot be excluded, such as deep learning architecture, training datasets, and inappropriate use.<br /><br />The author highlights the importance of knowing the patient population to which AI is being applied, as well as the distribution of centers and endoscopists. Evaluating the clinical significance of AI involves pre-clinical studies, including standalone performance assessments and technical interactions, as well as clinical studies that assess efficacy and clinically-oriented outcomes.<br /><br />Standalone performance assessments aim to determine if AI is as good as human ground truth, without the need for blinding or randomization. These assessments can include multi-case multi-reader studies. The document presents examples of standalone performance in lower GI, upper GI, and characterization.<br /><br />Bias in standalone performance can arise from selection bias, operator bias, and spectrum bias. The document explores whether advanced imaging, such as blue-light, can improve AI performance and whether it depends on the level of expertise.<br /><br />AI clinical studies examine the interaction between human and artificial intelligences and assess if AI improves clinically relevant outcomes. The document presents meta-analyses on AI's impact on miss rate, adenoma detection rate (ADR), advanced ADR, sessile serrated lesion detection, and miss rate vs. ADR.<br /><br />Post-hoc analysis of AI clinical studies looks at false positives and how endoscopists are alerted to images that AI incorrectly identifies.<br /><br />In conclusion, the document emphasizes the importance of understanding the training dataset and conducting rigorous evaluations of AI devices to minimize variability in outcomes. Standalone performance assessments and clinical studies are vital in assessing the clinical significance and benefits of AI in gastroenterology practice.

Asset Subtitle

Cesare Hassan, MD, PhD

Keywords

artificial intelligence

gastroenterology practice

clinical significance

variability

healthcare

deep learning architecture

training datasets

patient population

endoscopists

clinical studies