I will turn this over now to Shravanthi, and Shravanthi's task is to discuss how is machine learning different from statistics and what are the basic concepts surrounding that. So Shravanthi, if you're ready, you can go ahead and start sharing your screen and start your presentation. Thank you, Prateek and Tyler. I think we kind of laid some basics as to what machine learning is, artificial intelligence is, and how we are actually using some of those concepts in the realm of gastroenterology and endoscopy. Now, the reason it can kind of feel like counterintuitive, why are we talking about statistics in this machine learning topic, right? So as clinicians, we are all attuned to evaluating articles, journal articles, and trying to see if it makes clinical sense using statistical technologies. But now as we move on to more machine learning applications, it's very important to understand how a machine comes to a certain inference and how it actually, how do we extrapolate those information into clinical practice and so forth. So these are my disclosures. So the overview of my talk will be basically comparing statistics and machine learning. Again, these are not two comparable groups. This will be at a very high level discussion as to what statistics does in general and what kind of inferences you can get from machine learning technologies and how as clinicians we do understand statistical inferences. And also I will just put in a plug about how machine learning applications and thought process goes into the whole cycle when we have to develop an algorithm. So again, I bring this slide in the beginning as well as the end. The goal is to kind of understand at the end of the presentation what are the implications from a machine learning standpoint. And some of the things that we have to look for when we read a journal article is, can you interpret the model? What is the inference of the model? And what is the role of explainability of the model? Can we trust this model? And how transparent is this machine learning model? Now this is a cartoon I pulled from the internet, which I thought was funny. I will leave you to read the cartoon yourself. So you see this person, this broken statistics and somebody puts a frame around it and then say it's machine learning. And then when you say it's artificial intelligence, you have much more audience up there. And this quote is quite funny as well. So when you're fundraising, it's AI, when you're hiring, it's machine learning, but when you're implementing, it's logistic regression. So the last sentence actually struck me hard because the reason they say when you're implementing it's logistic regression is because of the interpretability of the model. You are able to relate as to why this model will work and why that specific variable that the machine learning algorithm said was an important predictor makes clinical sense. Now, again, it's not my goal for us to understand all of these different Venn diagrams, but I just wanted to also bring it up again that statistics and machine learning are almost mutually exclusive and other things like your pattern recognition has an overlap between statistics and machine learning and AI and compose different other aspects and machine learning is a path to get algorithms for performance. So this is just an overview of different technologies and different words that we normally use when we are talking about development of AI algorithms. Now what is the major difference between machine learning and statistics? It's basically the purpose. Machine learning models are designed to just make accurate prediction as accurate as possible. They do not deal with the relationships between different variables that are used in prediction of that model or get any inference from the model. All they are designed for is to get your 99.9 accuracy. Now statistical models on the other end, which we are very attuned to as clinicians, are designed for inference and relationship between different models. That's why you talk about associations, correlations, and also talk about how significant these relationships are and so forth. Again, machine learning emphasizes an optimization and performance over inference and that's what I've already talked about. I apologize for this busy slide, but it kind of gives you an overview of how machine learning algorithms basically work. What typically you do, I just put it in a very crude way, but what typically happens is that you have a big data dump, which is very messy and very noisy in real life and a lot of data scientists spend about 80 to 90% of their time trying to clean this data. It's not running the algorithm, it's basically just cleaning the data. Once this data is well assorted, the algorithm processes the data and what it's trying to see is what are the patterns in this particular data set? Is this one correlating with this point or which point is closer to which point and so forth? It does that over a million of iterations and learns over and over and improves and fine tunes the final algorithm, which can be externally validated. Based on the patterns, you are basically making predictions on an entirely new data set that probably never saw the data. If you have a data set that is split into training and then internal testing, there is a high risk that it could be overfitted and I'll just go over the overfitting concept a little bit later. Now, machine learning also treats algorithm as a black box and a lot of current investments and companies are looking at debunking the black box to provide some layers of explainability because it is very important in terms of how for us to understand how these algorithms work so that we can adopt them into clinical practice and also from an ethical and transparency standpoint as well. Now, machine learning algorithms are typically applied to high dimensional data sets, meaning you have thousands and ten thousands of variables and you might have only ten patients, but their entire genome sequence might be on those variables or their demographics medications. There are continuous vital monitoring and these are very rich data sets that statistics cannot handle. So that's what we call as high dimensional data sets. Now, statistics, on the other hand, is very planned and meticulously done. So we know exactly why a data is collected. For example, in a randomized control trial or a prospective study, you know this is the hypothesis that you want to prove and these are the a priori variables that you want to be included in the model. You know how the data is distributed, meaning those nice bell curves that we want to make sure and make sure there are no outliers in the data. And that's the underlying distribution of the population. And all the selection of what kind of model you want to use when you're running a logistic regression or what underlying mathematical principles you want to use are carefully chosen by your statistician. And they make sure that this repetition over and over multiple experiments stays stable. So you exactly know which variable is behaving in what way. And you can also predict with relative tight confidence interval that this algorithm or sorry this model will actually perform at, you know, 95% accuracy. That's where we look at the p values, right? The less than 0.05 is what the error rate is. So typically, as I said, statistical models are applied to low dimensional data sets. Now, when a machine learning professional is giving you an inference, let's say typically the way the inference comes out is the model is 85% accurate in predicting variable Y given A, B, and C. But we are used to data like coming from a statistician where we say the model is 85% accurate in predicting Y given A, B, and C. And I'm 90% certain that you will obtain the same result when you use it on a similar data set. And that's exactly what your confidence intervals are and your p value is. Now that we know how machine learning algorithms work, it becomes very important that we also know if we are able to interpret what the machine learning algorithm is saying. A classic example would be, let's say you have a picture of a polyp. You know that it is a polyp, but let's say you're running 1,000 polyps through the model and different types of stool in the model. And let's say a new image did not see a previous stool in the colon. It might pick up things on, let's say you're washing the colon every time you're seeing a stool. It might pick up the washing of the colon, the water particle, as something relevant to your Boston bowel prep score and can actually develop an algorithm completely with the washing and the water itself. Now from a prediction standpoint, the accuracy can be 99%. But when you go back and see, you cannot interpret that model. It's picking up the wrong signals, but with high accuracy. So it's important to understand how these models work. That's where the explainability, trustworthiness, transparency, and inferences come into place. Now this is an article that was published in 2019, basically telling us where these explainable models and explanation interfaces will be seen in the near future. So currently our models are these. You have input data, you have a machine learning algorithm that's running, it's accuracy and performance. And it's just spitting out some information telling you that the algorithm is 99% effective. Now as clinicians, we are wondering, why do I need to trust this black box, right? So the next generation of algorithms will be input data, your black box, an explainable model, and an user-friendly explainable interface where a clinician or the end user is clearly able to see what is it within the model that this machine learning algorithm is picking up to tell that this is a polyp versus not, or this is a dysplastic lesion versus not. Again, these are some of the terminologies that kind of differ between statistics and machine learning. I don't want to go into too much detail, but you can refer back to this at a later time. Now I also wanted to touch on how machine learning algorithms, the process works. This is not a machine learning algorithm. This is just computer-aided detection. It's a rule-based approach. So the way it works is first, let's say you have a problem. These are how EKGs were built in the 1970s, the automatic detection that you still see on the EKG is actually a rule-based algorithm. So you have a problem. You write a rule saying that, let's say in our case, let's say if it's a polyp, you say if it has X, Y, Z features based on nice classification, then it becomes an adenoma versus a non-adenoma. The rules are already written. The algorithm is not picking up any new patterns. And then you get the solution, you get the solution, you analyze the error, and then again, go back to the problem and say, why didn't it identify this particular polyp accurately? And then say, oh, OK, maybe it did not take the fifth point of the rule accurately and or it got confused. So I need to rewrite the rule saying that on the fifth point, if it shows a dot on the vasculature, then it means it is an adenoma versus not and so forth. So that's basically a non-algorithmic approach or a rule-based approach. Now machine learning process is basically we are not writing the rules. I'm not telling the machine that, OK, these are the nice classification characteristics of an adenoma versus non-adenoma. I'm not doing that. I'm just giving the machine, identifying the problem, get the right data, and train the machine learning model and evaluate the solution. And of course, run it through multiple iteration to analyze the errors, and it goes in circles. So that's how current machine learning processes work. Now in automated versions, which we will see in the future once they're FDA approved, as Tyler mentioned, is this process of learning about the error. This part is completely automated, and it goes through multiple iteration while it's running live. And these kind of algorithms are the beauty of machine learning because they can learn from the data as we progress, and they're not stagnant. Now having told about the machine learning, how it works itself, I just want to briefly touch upon a few challenges that we commonly face when we are trying to develop a machine learning algorithm. In this cartoon, you'll see that, OK, this guy is saying, use the CRS database to size the market. This guy says, that data is wrong. Then use the SIPS database. This guy says, that data is also wrong. Can we average them? Sure, I can multiply them too. So that's where we are in terms of getting good, clean data. It's a real big problem, and there are different approaches to it, which we'll discuss as we move forward in the course. But basically, we can have insufficient quantity of training data. The training data might not be representative of real world situation. It could be extremely clear, or it could be very noisy and bad data. The machine learning algorithm can actually pick up irrelevant features, like the water pump that is pumping water if the colon is not clean. It can pick that up and say, every time there is water drop, then your Boston ball prep score is probably 2. There could be an overfitting or underfitting of data. I know both Tyler and Prateek have touched on this. I would put a cartoon version of it. This is how exactly an overfitted model looks, that there is no room for improvement. It's a perfect 100%. And then there are other concepts which are more in the realm of data scientists called hyperparameter tuning and model selection, meaning how do you tune the training data and what kind of model you choose, and things like that, which are beyond my comprehension. So thank you for your attention, and I'll see you soon later.

machine learning

statistics

clinical practice

accuracy

inference