We'll now proceed with a panel discussion in this arena on AI for research. This will include Dr. Ruben and Dr. Omar Ahmad and Dr. Shyam Thakkar. Dr. Ahmad is a gastroenterologist and senior clinical research scientist at the University College of London with a special interest in interventional endoscopy and advanced imaging. His PhD research focused on clinical translation of AI and endoscopy. This interdisciplinary work has led to the development of AI software for real-time use during colonoscopy, currently being used via cloud deployment clinically. His specific interests include identifying barriers to implementation of machine learning in healthcare. He's published numerous international initiatives related to effective validation and implementation of AI and serves as an expert on AI working groups for international endoscopy societies. He currently serves as editor for the AI in clinical medicine textbook. Dr. Thakkar is director of advanced therapeutic endoscopy and professor of medicine at West Virginia University. In this role, he's focused on vision and strategy of the endoscopy service line that supports the needs of the state of West Virginia and the surrounding region. His clinical practice focuses on advanced therapeutic endoscopy, while his research focuses on utilizing AI to improve the quality of colonoscopy, for which he's received numerous grants and oversees a lab focused on AI artificial intelligence. Welcome to the three of you. John, we'll start off with a question. Great. Well, thank you for that. Those great talks. David, perhaps we could start you in one of your slides, you talked about the idea of integrating multiple data sources and including imaging as well as clinical data. My question to you and the researchers on the panel is, how do we weigh different sorts of data in our models and whether or not in our trials, can we just reanalyze them with some sensitivity? What is your thought on weighing different data sources as we make these models? It's a really important question and I'm grateful for the thought. It's not that different initially, at least, to our multivariable regression analyses and how we build models. It's just more complex because there are more variables. The ones that will be weighed more heavily have two particular parameters. One is that they are more directly linked to outcomes of interest with a stronger odds ratio, but the second one ends up being how reliable we can measure them. That's what I hinted at in my brief presentation, which is that some of our approaches using computers as assistants are going to be more detailed and provide more information than we ever have been able to with the human eye and existing prior technologies. How do you move the field forward? It's this constant going back and forth in terms of our analyses to figure it out. I'd be interested in what my colleagues think about that as well. Dr. Thacker or Omar, Dr. Ahmad, any follow-up there? Yeah, I would agree exactly with what Dr. Rubin said there. I think it's really going to be a bench-to-outcomes-to-back-to-bench kind of model where it's circular in terms of where we see the improvement and how we further improve to exceed the current standard of care. I have a question. I'll direct to Dr. Ahmad, but it would be pertinent to all three of you, and I'd be interested in each of your perspectives. With natural language processing, our ability to digest huge volumes of data or experience has largely or will potentially replace traditional data extraction, perhaps even the data interpretation of traditionally retrospective studies, which currently rely on all variety of help from overworked students, residents, fellows, and work-restricted nurses, and are often necessary to generate questions for optimal prospective studies. With these data sets, might these same natural learning technologies answer the question sufficiently to forego prospective evaluation in the absence of a new device or a new chemical needing FDA approval or some other regulatory sign-off? Might we forego prospective studies altogether or even randomized studies with big data sets? I think the short answer is no, we're not there yet, absolutely not. But certainly, I think the benefits of leveraging things like NLP and big data is we can get novel insights. So you alluded to the fact that, for example, you may find novel compounds that are worth studying, and we saw that during the COVID-19 pandemic, actually, where a big data company was able to identify a novel compound used that was subsequently evaluated in a large-scale randomized controlled trial in the United Kingdom, and it proved to be successful. So I think what we'll largely see with NLP, AI, machine learning, data-driven technology is the gift of time to physicians. You mentioned we don't want to be spending hours and days looking back at literature. This will offer new insights that we can then deploy and evaluate much more rapidly. But I think the gold standard will always remain a prospective randomized controlled trial. I don't know what the rest of my panel think about that. I actually agree with you completely, except to add that I think the trials could be done much more efficiently. And we could certainly get signals earlier in the trial, have shorter induction time points or end points of interest, and be able to use some biomarkers to know whether we're having an impact or not, so that we can move directly to another option for the same patient who's already been recruited into an experimental protocol, or just abandon it completely. And for the industry side, the investment for them is they save millions of dollars if they know upfront that an early phase molecule or experimental intervention doesn't work. Yeah. I would add that big data, while it can be very insightful, can also, what Tyler spoke about earlier in his talk, potentially have inherited systemic biases within it. And so, as Omar alluded to, is prospective trials still remain the gold standard. But as Dr. Rubin said, using the knowledge that we can from AI to make those more efficient could be extremely beneficial. Thank you. So I wanted to ask the panel a little bit about the quality of the data input and how that affects the models that we have. We saw the dramatic image of Dimitri and trying to use his data points. And obviously, someone's got to clean that up initially to start teaching the model then to understand it. And the other question that relates to that is, if you have 10 cases of a malignant stricture and you take 100 images per each case, or do you want to have 100 cases of malignant strictures with 10 images, does that affect the quality of the model you have, or does it not matter as long as it's predictive? So maybe we can talk about how the researcher deals with the inputs aspects of what we're developing. Jonathan, I think you're absolutely right. The inputs are key. People often say garbage in, garbage out. What we're worried about here, we're seeing a lot of models being developed in research, a lot of publications, but actually taking a model to production into the clinical environment is a completely different ballgame. The models need to generalize to new unseen populations. And if your training data set doesn't reflect the general population that we're going to deploy the model, you can run into a lot of trouble, you can get poor performance. So we need to have diversity in the data set that reflects your intended use for the device. You referred to malignant strictures, is it better to have thousands of images of 10 strictures or lots of strictures? I think the answer we'd all say is as many images as possible from as many patients as possible because you want that diversity. And there's a risk we've seen very early on with a lot of the endoscopic imaging work in AI, where we've overfitted models, where the models have been trained excessively on small cohorts and actually then when they have new unseen data, they don't generalize very well. So I think it's absolutely key. Does that requirement for larger amounts of data and perhaps an increasing reliance on data scientists even, does that squeeze smaller institutions and individual investigators who might not be part of a major collaborative out of much of the research that's going on in this area? I think that's a really good question. It lends itself to the need for open source data sets. So thoughtful and talented individuals can access things to help us move the field forward. I'll add that it really depends on what we're talking about. So for example, in the example given of looking at a stricture, for example, the appearance of a stricture and the diagnosis of it may be more dichotomous, it's there or it's not, than perhaps a continuous variable of a biomarker that is across a spectrum of heterogeneous disease types. So I don't think that there's a simple answer, because if it ends up being something that we could distill down to a binary analysis, that's going to be something that might be very easy to do and reliable and validated quite simply with a small number. But it's some of the more complex things, and one of the biggest challenges in IBD is that we've lumped all these patient types, a true heterogeneous disorder, not just Crohn's or UC, but probably 50 different inflammatory conditions with different presentations into the regulatory driven model of it's either Crohn's or ulcerative colitis. And that's partially why our results look so bad. So we have to divorce ourselves from that artificial construct in order to redefine this. And we may find that some of the discoveries will be quite straightforward and easier than others. And I want to encourage people to think outside the box in that regard in the IBD space. Let's turn for a second to the issue of looking at the future of clinical research in this area. And maybe from a 10,000 foot vantage point, Asimah presented a number of goals of research that are beyond diagnosis, early diagnosis, early detection of neoplasia into improving efficiency, physician time, access to care, and health care disparities. So where do you each see the research priority development? Is it what AI can do, or what are the unmet needs? So how do you each see the next 10 years looking in terms of where we're going to be focusing our research effort? Well, I love that part of her talk. And I hadn't even thought about AI in that regard. But it really did open my mind to whether AI or other models of electronic assessment and monitoring could divorce us from our implicit biases that influence our care in negative or positive ways, and therefore, help us to not make errors that are inherent to our own training or experiences. So I think that there's incredible potential from that point of view, and that really was something I enjoyed hearing about. Yeah, Jonathan, just to say that I think for me, being a bit of a data geek, I think it's all about the data ecosystem. We're going to have to invest in that in the next few years upfront before we can really see the benefits of AI and machine learning. So she talked a lot about interoperability, and we've seen things move quite fast during the COVID pandemic. Hopefully, we can keep that momentum as perhaps one of the only decent things that came out of the pandemic. But we still have a huge problem with data silos. Brett was referring to, if you're a smaller organization, how can you get involved in AI work? Well, we have big problems with software vendors. So EHR, for example, unfortunately, they don't have much of a vested interest or incentive for software to talk to each other. There are international interoperability standards, but unfortunately, they're not being followed. And also, standardization of data entry and capture, it's incredibly laborious to enter standardized data. And a lot of EHR is designed for billing, not for AI or medical research. It's a problem. So as David was alluding to, having more biosensors or automated data capture could really help that. So I think we're going to see big advances in the data ecosystem in the next few years before we truly harness the power of AI. Yeah, I would also add that I think going forward, we need to see a little bit more of the infrastructure stabilized in terms of the AI research. We've seen some papers come out and do a really nice job of delineating how proper AI research should be conducted and put forth. But nonetheless, it's not consistent across all the research that comes out right now. I think we're also going to see over the next several years more value-based research. There's such a big movement towards value-based care in medicine that I think it's a prime opportunity for AI to help us deliver high outcomes at low cost to ultimately create great health care value. And outside of that, I think one opportunity that we can really see be very beneficial is physician wellness through all this. The more that AI gets employed, how it impacts the wellness of the physicians and how it can be used to improve the wellness of physicians to ultimately heighten care. Thank you all for your perspectives. It's clear that the future is bright, but there are huge hurdles. And so far, we've been plucking off the low-hanging fruit of image interpretation and image detection of lesions. So there's a long way to go, but it's exciting.

AI for research

panel discussion

colonoscopy

data quality

physician wellness