Gastroenterology & Artificial Intelligence: 3rd Annual Global Artificial Intelligence Summit | September 2021-7- How to Evaluate AI in the GI Literature and Clinical Trials

7- How to Evaluate AI in the GI Literature and Clinical Trials

Pdf Summary

This document discusses the evaluation of artificial intelligence (AI) in gastrointestinal (GI) literature and clinical trials. The author focuses on AI's standalone performance, training bias, testing dataset, and its effectiveness in detecting neoplasia and characterizing lesions. <br /><br />In terms of AI's standalone performance, the author highlights the need to compare AI to human ground truth and determine if AI can detect and predict histology like a pathologist. The author also discusses the advantages of AI in terms of eliminating psychological bias and the disadvantages such as the lack of transparency in the algorithms being tested. <br /><br />Regarding training bias, the author emphasizes the importance of considering factors such as patient population, center expertise, and technical settings in the training dataset. The author also provides examples of different AI systems and their regulatory approval and training case numbers. <br /><br />When discussing the testing dataset, the author mentions the need to consider whether the cases used are different from the training dataset and whether they are generalizable to community-based endoscopy. The author also highlights the importance of testing on videos or still images. <br /><br />The document then delves into the effectiveness of AI in detecting neoplasia and characterizing lesions, including the interaction between AI and human endoscopists. The author mentions the need for randomized controlled trials to evaluate AI's impact and explores the consequences of false positives. The author also discusses the value of AI and its potential cost-effectiveness in colonoscopy. <br /><br />The document concludes with practice pearls, including the importance of having full details on the training and testing databases before using an AI system and the necessity of conducting RCTs to understand the interaction between human and artificial intelligence.

Keywords

artificial intelligence

gastrointestinal literature

clinical trials

standalone performance

training bias

testing dataset

neoplasia detection

lesion characterization

psychological bias

transparency in algorithms