And now we'll switch gears to our state-of-the-art talk. So it's my pleasure to introduce Scott Penberthy. Scott works for Google Cloud CTO office. And he's worked in engineering mainframe systems and healthcare claims processing. And what I was really intrigued by is for searching for exoplanets and minerals on the moon through NASA. So Scott is going to give us the state-of-the-art talk about healthcare and artificial intelligence. And Scott's been a good friend of the ASGE for the past year or so, helping us out in various AI educational endeavors. So Scott, first of all, thank you again for being here. It's a pleasure to have you. Welcome. There we go. So just to give you a sense of, this is kind of the map of Google. We're now called Alphabet. It's a holding company. I'm this, see that cloud with all the wonderful colors? That's where I sit. I'm in this part of Google, Google Cloud. I have friends here from Verily, Verily, you guys are in the back room somewhere. They're a company we have that does a lot of really fascinating things and pushing the state-of-the-art of science. We have Google Health, and we have a lot of investments in healthcare, several thousand people. And what I've worked on in cloud is how do I use AI to really make a difference? And we talk a lot about pilot detection and a lot of CNNs and that sort of technique. And I'm looking for the boring billions. In other words, if there's a $4 trillion stack in the United States, how can we knock that down a bit? Like, what's the real pain? How do you use AI and these techniques to kind of take the drudgery away from work? And so things I work on are never going to be on the front page of nature, if you will. But they do is they make computers suck less, which is kind of nice. And so the idea here is this is an issue. The pictures you're showing and everything else, no one wants those to get out. I mean, data itself is very sensitive. And I saw in just a couple of presentations, like, how do I get better data? And what this is a chart showing, it's just all I see are big numbers. What this means is that if data is leaked, I know one payer who's paying a billion-dollar fine because someone left some data open. We've had, you run into issues where someone will look at data and they don't know where it comes from. They say, oh, look at my nice analysis. It's full of PHI. And there's just a 22-year-old grad student trying to make a difference and show off her work. And she didn't know. And so how do you protect this data? And when I talked to a top payer, he goes, ah, I know what I want to do. I want to do is homomorphic encryption. I said, how's that going for you? He goes, I can't get the regulators to spell it. And I said, well, what is it like? I don't know if you've seen the mathematics behind this, but the idea is homomorphic. It's like, you know when you see a globe, that's called a homomorph of the earth. It looks like the earth. The earth is actually completely round, but it's good enough, right? And so it's a homomorph. In other words, if you can do things on the globe, it kind of looks like the real earth. It's a homomorph. It's a mathematical term. And encryption means, well, I want a globe that represents the earth, but I want it to be encrypted. So I look at this funny landmass. I don't know what it is. It's the United States, but it represents that enough. And I can play with it without actually knowing it's the United States. How do I do that? And so this is sort of a math diagram saying you have the word you, and you want to say, I love you. You encrypt it. You get this weird thing, X dollar sign 5A, but that's where you reason with your math. And then you want to be able to change a letter like B, and then it comes back. And what you're doing is in the encrypted world, the letter B is an exclamation. I love you comes out. So could we somehow project from the real data into a homomorph and play there was the idea. Everyone wants to do it, no one's doing it. And so I saw this conversation and I said, wow, we're doing that in AI with how you recognize postal zip codes. It's, you know, it's a very famous done in the nineties. He won one of the Turing prize, which is sort of the Nobel prize in computing. And the idea is they had these handwritten digits and he said, how do I, could I do it for those things? And these are like 25 by 25 pixels, which on a modern computer, you can't even see unless you have reading glasses, it's a dot, right? But that's how big the technology was. And what they did, they built this thing called an auto encoder. I love how this thing works. So these little blocks, what they're doing is two, that's the, that's the bits. That's the picture of the bits. And what they do is they squeeze it mathematically. So these are like just matrices of numbers and they get smaller and you're, you're doing multiplications and additions as you get smaller and smaller and smaller. And you just like squeezing it at the very, very center of this thing. You've got a flat, a flat array. It's like a column in Excel. Imagine that. Right? So you squeeze it down. You've got to sell a column of numbers. You can imagine that. Right? Well, that, if you, if you study Shannon's, you know, information theory, it's the information in the data, not the data itself. So it's the two-ness. What's a two? I don't know. It's the two-ness of two. And if you have enough of the two-ness, can you then turn it around and go backwards and regenerate the two? And it turns out you can, you can set up the math to say, oh, you really suck initially. And it goes back and forth. And finally it figures out the multiplications and additions to make that work. Kind of cool. So it's auto. So it's automatic. It just keeps going back and forth and it's encoding. It's taking something from, from data and encoding it in a smaller representation of, it's called a tensor in the middle. And it works. And so this is, I love this story. There's a person we call the gone father. He's and he's a till the generative adversarial network and true story. He was having beers in grad school. We've all had a beer or so in grad school. And there's talking about this thing back and forth and he goes, if I cut it in half, what do you mean you cut it in half? You know, I cut it right there in the middle. What if I twiddle those bits? What happens? Like the, there's the two. So the computer figures out what two or three means he goes, what if I take that and start playing with those bits? Well, you might change what the picture is, right? Cause yeah, but I've never played with that before. So if I cut it in half, that's the top half, right? Memory. I just cut it in half. Then he said, Oh, what if I then had a generated image and I tell if it's a really a number or not. So could I somehow twiddle the bits and do another network that says it's trying to fake it out. And I'm playing this game, a game theory playing. Is this, is this digit or not? Is it digit or not? Could I then figure out how to twiddle bits to give me a one through nine? Could I figure that out? And they're like, sure. So with two beers after a discussion goes home at 10 o'clock at night and gets us running. What you're seeing is the computer initially not knowing he's, he doesn't know how to use the bits, but the computer's playing with the bits. And eventually it goes from noise to actually generating one through nine, gets it running by one o'clock in the morning, true story. And he said, Oh my God, I can actually have a computer that knows how to generate bits. And if I teach it, that's a one, that's a two. I can now say, give me a one. And it generates one after one, after one, they all look different because it's using random numbers in that space. So when his friend goes, can you do a face? No, no, no, no. Faces have statistical properties. No, they don't. Yeah, they do. It's just a distribution of points in space. We have eyes and nose and mouth. Most people, you got some hair, some skin color. He said, well, let's try black and white and little small images. And so they took a 2D image and he started saying, and they did the same thing as squeezed a face through the, through that cone and had to come back out and they had the computer play with the bits, published a paper that was called the generative adversarial nets in 2014. And this is, they had this thing running, generate basic faces. It's, ah, that's kind of cute, but is there any value in this? This is NVIDIA four years later, NVIDIA said, what if I just threw a lot of compute at this and got a really big image? And that didn't take the trivial two beer version and had a team of scientists work on this for three years. And this is called this person does not exist.com. You can see it online and people saw this and they were taken aback because that's not a real person. Well, it is, but it's not. And you hit it and it gives you a random picture. That's as good as a regular picture of a human being. And occasionally you'll see some artifacts in the back, like what's behind her. I don't know. The face is pretty good. That's 2018. So then some of those scientists go, that's kind of neat, but it's like, wow, you can't control this thing because you type it in. It's cool. You can take pictures, the Nike and use them in your, in your PowerPoints. And you have photos that you do, you know, they're free and they're not pictures of actors. You had a license. That's kind of neat. So someone said, well, how do I control this? What if I take text and Texas just bits? Cause it's just like, you know, it's bits in a computer. I can squeeze bits through an auto encoder, right? If I can do that, why don't I just tack it on? That's the red thing on the bottom on the left. And I just, I slap it on the side and I squeeze that through the GAN. So the idea is that I have a picture and I'm telling you in text, this is a cat. This is a dog. This is a, this is a woman with blonde hair. This is a woman from Asia with glasses. And then with a, you know, a mole, this is us. This is someone from India who has a wearing a lovely, you know, a natural thing for getting when you get married, right? Imagine you could describe the text with the image, jam that through the pipe. This is a month old. On the left, you type in digital art of a man looking upwards, comma eyes wide and wonder awestruck in a style of Pixar up character, white background. That's generated in less than 15 seconds from scratch. The next one over a closeup of a woman's face captured in low light with a soft focus. There's a gentle pink hue to the image and the woman's features are lightly blurred. It's a closeup that's generated. The third one, photograph of a stylish black man taking animatedly on a phone, talking on a phone, mid shot, outdoors in LA, harsh overhead sunlight, midday on summer. Generates that picture. We're now able to have with these models that are not the trivial two beer version he had, those from 1990, has 175 billion neurons. And we're able to generate these things. And that text that I'm talking about, it's called a prompt. So you can now take a prompt and a prompt is a turning out there's, there's different, it's like a tweet. It's a structure. You can program a computer by being very careful with how you describe things. And there's, it's the art of the prompt is a new way to program computers. This is three weeks old. So now we're getting to the point where you can now describe the kind of thing you want and how you describe as a programming language and it generates the images that you want. And they're starting to do this for video shorts and see where this is going. So we have something like that at Google. I'll give these URLs and we have a, you know, it's now sort of tit for tat, different teams opening. I has one, Microsoft has one. It's kind of fun to watch these teams compete and they just keep getting bigger and bigger and better and better. And it's first images. Now it's going into movies. You can, and you can imagine this going forward. And this is a photo of a Corgi dog riding a bike in time square period. It is wearing sunglasses and a beach hat. And that comes out, doesn't exist. I love this one. This was a two grad students out of Texas that saw that and go, that's so cool. So they re there's the, they didn't publish the code. They published the algorithm. So they spent a few weeks writing the algorithm of baby version and they released it called mini Dolly said, you can't call it mini Dolly. So they called it crayon. And now they're making mint off the ads while you wait two minutes for the image to get generated. They're millionaires in the last 60 days, it's called crayon.com. It's fun. This, I just typed in Albert Einstein playing baseball and that generated in two minutes. And I watched a few ads, not bad, right? So then, then I remembered, I saw this, I remember my conversation with the CISO and we started thinking at Google, Hey, what if we did this for healthcare data? Like could I imagine a polyp could I could the chart that all the pictures you're showing us, couldn't I generate one that has the same properties of a real patient, but she doesn't exist, but has all the data you need to show and, and have, and not worry about a release form to include your image in your, in your publication, nor to, or to fill in the different areas you need for your research. Could we do this with this data? And we're running experiment at Google saying, well, if I start with, I call it the phenome, the EHR, the imaging data, the genomics, the social, the entire, because you saw a chart earlier saying, I want more version of the patient, take all that data. We ingested into a sort of common format. And the idea is, could I sort of build an API like a prompt that gets you an image that you want for the data? It's great for population health. And so this is sort of the way it works is you get a source database. You may de-identify it. That's optional. Then you train just the same thing I showed you before there's research public published on this. We're doing one version of it. And then can you synthesize data, have a third party audit that's going, yep, yep. That person doesn't exist. Sure looks like a real person has all the characteristics of the person with these flaws, but it gives you lots of data to do this. And now it's good for doing synthetic analysis to give you your models that you can then test on sparse data with the real models is the idea. And it's working. It's cool. We take this random data. It's sort of a dumpster fire of data. If you look at healthcare data, it's typically a dumpster fire and you pour it in along with images and recognize that on the right, it's exactly, it's the thing from two beers from 2014, a little more sophisticated, a lot more neurons, a lot more compute, but that, and the idea is, can you actually do this now and make it differentially private so that you take a person in and out of the data set, the data set doesn't change. So it's great for population health, not for individual diagnosis, but a lot of population health lets you do things about, is this a polyp? Is it not, you know, that that's worth it. And then the idea is, can we then generate all kinds of things on image? That's kind of cool. I could do a polyp image. That'd be interesting for a colon or, or, or pancreas, but what about medical record data? Sure. Why not? What about doctor's notes? I can do that too. Right? So we're starting to think about, can you train these things and could this be an interesting tool for science? And that's what I'm saying is a, is an advanced sort of state of the art in AI. This is about a less than a month old. And we're finding is that what we're doing now is trying to figure out where AI can actually make a difference and fill in those things that are difficult for us. And just an example of that. So anyway, so thanks everyone for your time this morning to give you a sense of synthetic data and AI sort of the one sample of this huge frontier.

Google Cloud CTO office

healthcare data

artificial intelligence

homomorphic encryption

deep learning techniques