Why AI Literacy Matters for the Future of Healthcare with José Acosta | Ep. 43

In this episode of The Med Device Cyber Podcast, hosts Trevor Slattery and Christian Espinosa are joined by Dr. Jose Acosta, a retired naval trauma surgeon with a 40-year career in medicine, including serving as the Command Surgeon for the US Pacific Fleet. Dr. Acosta, an early adopter of new technologies, shares his expert perspective on the integration of Artificial Intelligence, particularly Large Language Models (LLMs), into the healthcare industry. The discussion is centered around the concept of "AI literacy," which Dr. Acosta argues is a critical competency for the next generation of healthcare professionals. He defines this literacy not merely as the ability to use AI prompts, but as a comprehensive understanding of the technology's inner workings, its inherent limitations, and the ethical and governance frameworks required for its safe deployment. Dr. Acosta emphasizes that while AI tools offer tremendous potential, especially in areas like diagnostics and administrative tasks, their application must be approached with caution. A key argument is the distinction between being "pretty good" and being precise. In many industries, an 85-95% accuracy rate is acceptable, but in medicine, where patient lives are at stake, the standard must be near-perfect precision. This leads to a discussion of the "productivity paradox," where AI, instead of saving time, can create more work for clinicians who must meticulously verify its output and manage increased patient loads expected by administrators. The conversation also explores specific applications like ambient AI scribes, which can reduce the burden of data entry but may also strip away the crucial "human touch" and rapport-building conversations in patient interactions if not governed properly. The podcast delves into the balance between leveraging AI's benefits and mitigating its risks. The hosts and Dr. Acosta discuss the security vulnerabilities of AI, such as the potential for training data to be poisoned or outputs to be tampered with, reinforcing the need for robust cybersecurity measures. Ultimately, the consensus is that AI will not replace clinicians but will act as a powerful assistive tool. The professional who masters the use of AI, understanding both its power and its pitfalls, will be the one who excels in the future. The episode serves as a nuanced exploration of AI's burgeoning role in medicine, highlighting the urgent need for education, responsible governance, and a clear-eyed view of both the opportunities and the challenges that lie ahead.

Hello and welcome to another episode of the Med Device Cyber Podcast. We're joined here today by a very special guest, Jose Acosta, who I recently met on a trip to Boston where we had some really interesting conversations about new application for AI and healthcare and seeing what some of the wonderful things that can happen as a result of this new technology while still understanding there's a little bit of a safety balance we need to have in place there as well as of course, security. I'm also joined by our co-host Christian Espinosa. And I'll start with you, Jose. How are you doing today? Guest: Yeah, Trevor, I'm doing great. It's great to see you and it's great to meet Christian. Host: Yeah, and you're back in uh, New Mexico now, in Albuquerque, is that right? Guest: Correct. I I live in New Mexico, I work in St. Louis and parts in between. Host: Awesome. And then Christian, you're coming in from Phoenix today, right? I know it's kind of hard to pin down where you are in the world at any moment. Christian: I'm in Tempe uh, area, uh Phoenix area. Yep, Tempe, Arizona. I used to live in St. Louis area, Jose, back in the day. Used to teach at Washington University a little bit and lived in Illinois. Guest: So, pretty interesting, Christian. I, um, the first time I, uh, I went to St. Louis was back in 1979 and I was, uh, a transferred from a college in Puerto Rico to WashU. So I was there from 71, 79 through 81, um, and all the summers in between. Christian: Cool. So, you're um, an advocate for a, uh, AI literacy, Jose, it sounds like. So, when we talk about AI literacy, from what lens are you referring to? Uh, I know we talked a little bit before we started recording, but maybe you can elaborate a little bit. Guest: Sure, sure. Christian, let me give you a little bit of my background and that way, it'll kind of give context to, uh, to, um, where I'm coming from in terms of AI literacy. Graduated 40 years ago from medical school, trained as a surgeon, trained as a trauma surgeon, and um, served, uh, in the Navy for the next 30 years. And the vast majority of that was at the bedside, but at towards the end of my naval career, I moved from the bedside to the C-suite, um, finishing as the, uh, Command Surgeon for the US Pacific Fleet. And so, um, retired several years ago. Always have been an early adopter. And so that kind of gives you a frame of where I'm coming from. When I talk about AI literacy, it really focuses mainly on large language models, but you can extend it to all the other tools in AI. And what I'm really interested, as a physician, is that the students that we are training for the future understand the technology way beyond just a simple prompt, understand, um, some of the math that that's used for the predict, for the prediction, that individuals understand where, uh, these models are not accurate. Um, get an understanding of, um, privacy in terms of leakage of information that's put into one of these large language models. And so it's the entire spectrum from understanding the technology, some of the basic tools, understanding that all these models are somewhat different, um, understanding how to interface with them, with the correct prompts, understanding governance, which is very important, and ethics. So, you see it's a lot of things, not just I open my LLM and put in a prompt and um, there we go. Christian: Awesome. And from, curious your take, I think, uh, AI is fairly decent for diagnostics, like, uh, image enhancement, with an MRI or, uh, ultrasound. I think we're a little bit premature for therapeutic use of AI. Just curious what your thoughts are on that. Guest: So, so, so, Christian, I, very interesting. I, uh, you know, the more I use the tools, I realize that just like you said, they, they're pretty good. But the problem in medicine is that you have to be precise. And so, um, I was talking to Trevor when I was in Boston, I had just learned about, you know, alignment in these models. And I had, you know, looked at some of the, uh, the tools and, um, some of the, uh, information related to alignment. And one of the big, big, big LLMs had a graph that I found very interesting that that, um, had the alignment, a specific topic, and what percent of the times that the chats meet that requirement. And it was pretty good. It was 85, 90, 95%. But the problem is in medicine, you have to be as close to 100% as possible. And so that is what I think providers end up against, which is, for example, some of the AI imaging tools, they're great. And I know there's some approved by the FDA and they pre-screen, um, studies, and, and they're good, but when you get down to precision, I don't think it's there yet. And that's on the imaging side, which is, which is certainly much more advanced, right, with the number of FDA approvals. But the, um, the diagnostic even more so. I think Christian, it's breaking that that barrier. Precision for a physician means a lot more than precision to other users. Host: I think it's another interesting area where, you know, obviously people can't be 100% precise 100% of the time either. And so I think that we're gonna start seeing a shift where there's never, I don't believe there'll ever be a perfect solution. I don't think that people will ever be able to get it right 100% of the time, nor do I think AI will be able to. But I think that we're gonna start seeing ourselves getting closer to a point where AI and humans are making similar decisions at similar accuracy, the closer we get to just seeing how mature these models can become. I also think that it's worth thinking about what is the context that we're building these AIs around. Thinking about a huge LLM, Claude, ChatGPT, Gemini, something like that. There's so much context going into what it's trying to do. And, yeah, seeing something 85, 90, 95% accuracy. That is pretty good. But it's pulling context from other sources to try to get to these answers that it's providing. And trying to refine that, of course getting into the conversation of how you're properly training AI models, then that can go down its own rabbit hole. But ensuring that we have the right background and the right context can help bridge that gap a little bit more and more, I think as we're getting more mature with AI models. Guest: And you know, the thing is, Trevor, I, I just was listening to a podcast, the Hard Fork, on from the New York Times, and they had the, one of the senior people from Waymo that was discussing their tax, their service, right? And what, what caught my interest was that Waymo is more, is safer than if you are driving by yourself. And they have the data to prove that. And so in medicine, we are very big on show me. And I think we're, we're early, we're still early. There's not enough literature, peer reviewed literature, which takes forever. But there isn't enough to say, listen, these models are good, and they are, in many cases, as good or better than the clinician operating them. We don't, I don't think we have that data just yet. And it goes back to it's such a complex, those models are so complex that when you publish, you're going to have to say, I publish with this model, here's the the AI model card, this is the day I did it, and these are the results I obtained. Or, you're gonna have to go to an open access model, right, that's not going to change. But it's going to be very important as we do the literature, as we create literature on the topic of safety and accuracy, that the people writing the papers understand the difference in these tools. That's key. Host: I think one other thing with AI, of course, with our focus, we're more understanding how is AI secure. And as part of that, it's an element within an overall FDA submission. I think that Christian and I can lose sight a little bit sometimes on how much goes into this submission since we have such a small focus on just this cybersecurity portion. We want to make sure we're getting that right 100% as opposed to trying to be more generalist in that regard. But point being, there's such a big focus on security in our lens with AI due to the reason that you can't try to hack into a clinician. You can't try to hack into a surgeon. You can hack into a computer. You could try to compromise information systems like that. Or with the use of AI, it's a little bit of this Wild West scenario where it's still so new and the security considerations around it are evolving so fast that I'm a little bit concerned about seeing some of the different applications that it's entering without understanding exactly how we can kind of tighten the reins a little bit on how it's safely and securely used. And so I do think that obviously there are tons of benefits to having AI in these medical environments, but we have to be aware of some of the risks that can go into it. I'll go back to my earlier example of the training data. How do we know that this training data hasn't been poisoned when it's being fed into the AI? How do we know that the output hasn't been tampered with? How do we know that the model is actually purpose built for this task in a secure and safe, effective manner? So, looking at it from that lens, obviously is a little bit different from our perspective. But I think it's a very big piece of the puzzle with AI that's been a little bit difficult to solve recently. Christian: Well I think with the what Jose said earlier, if we're getting 85% accurate with AI, that's not accurate enough. And the, the case that always comes up in my mind is where the AI wellness slash therapy app where it was providing therapy to a suicidal patient, and after a couple months, the app told the patient to go ahead and kill themselves. And the patient killed themselves, uh, and, you know, the the parents and the family are suing that company. But again, that's the 15% it got it wrong and that even if it's 1% And that's use is not the risk is too great, right? Because it's a human would probably not say that, I'm assuming. But the AI did uh for various reasons. So it's not even really a cyber security issue we can certainly inject things into the model and make it say that, but this is just a model challenge. Guest: Right and and you know the thing is Christian that the the alignment piece so having read the it's called um open spec which is open AI's and I think it's from September their most recent alignment you realize that there is the model's there, then you have alignment and then you have the user. And so what goes into that alignment is very important, right? And then also the other thing is drift. Drift. So, and I've, and I, and I've noticed it, you know, I, I use it all the time, I use it a lot for, um, research and I use it a lot for coding, um, one of the statistical programs, and even in a session, you see it drift, then you have to come back and say, no, please, don't give me a lecture. Provide me this. And so, all those things make it, I believe it's, we're very early. And again, I'm a early adopter, so I, I'm a firm believer and I think it has its place. But I think in terms of security, um, providers have to, you know, understand that the data is not going to be lost, that people know how to use it. So I think it's all these things put together. And that's why I'm pushing the AI literacy because, you know, providers don't even know the question to ask. Don't even know the questions to ask. So if a patient were to come to my office when I was in practice and say, listen, um, doc, I just got off using a large language model and I have gallstones and I'm symptomatic, but I went in there and it said, um, just bear down, you'll be fine, right? I have to be able to tell the person why that is not correct as opposed to that I don't believe it. You have to say no, listen, you know, which model and it's not a you're not going to ask him 20 questions, but you have to explain to them how these these tools are not precise. Host: So with how we're seeing some of these trends shift in AI, how we're seeing it become more and more commonplace, I'd be curious to hear your thoughts what you'd expect to be some of the biggest shifts and breakthroughs in the next maybe year, five years, 10 years, where we really see AI taking a healthcare industry from here. Guest: Okay, so I'm going to start one area. And that is ambient AI scribes. So, ambient AI scribes, and just, just to give a little bit of context, I go back to when we used to write notes on paper. And so, you would sit down with your patient, you'd speak to the patient, as they're speaking, you're taking notes. But you had some interaction. Then we move to data entry physician. So, the, the physician still maybe there's a percentage of physicians that do data entry. So, you sit there in the clinic for an appointment, and the, uh, the doctor's asking you questions and entering data. Because if you don't enter the data now, you're gonna be doing it tonight. Very famous, called pajama time, right? Work after hours, okay. The ambient scribe AI allows you to speak with the patient, and then in the background, it is, the, um, the tool is capturing that information, creating a, uh, a document that makes sense. And not only that, but populating all the different fields from, um, that that are in the, uh, the electronic health record. So, I believe that's a big move. I'd say, certainly being adopted a lot more in the academic medical centers, but it's billions of dollar industry and I may be overdoing it, but it's, it's a big deal. So, I think that is an area that's gonna really help, um, in many ways, right? It's gonna help with, uh, with being more accurate in your medical record, getting information promptly, maybe decreasing the amount of time, pajama time. Although, Trevor, there's always risk that an administrator may say, you know, you saved half an hour, we can probably sneak in there like two or three more patients. So there's that you have to be careful with that. Um, but I would say that's a big one. Um, certainly uh, imaging, but you know the imaging thing is very interesting, Trevor. I uh, I read a lot of this uh, AI stuff and there was a projection that by this, by maybe in a couple years we would do away with radiologists. And the reality is, we need more radiologists now than ever, right? Because radiology is not only reading the film, it starts by deciding, are you getting the right study? All the way to providing consultation to the referring physician. So, AI will be in that field. AI will assist the radiologist. At this point, AI will not replace the profession of radiology. So, those are a couple areas I think, uh, um, important research. I think we're going to be seeing, um, you know, the, the establishment give in, right? Right now, many of them say, don't, you know, if you're touching AI, don't send us your manuscript. I think we're gonna see a movement using, um, um, being allowed to use more AI support in when we're doing research. Host: I think that AI scribe example is especially really interesting. I know I use a tool called Otter.ai, which essentially, yeah, just captures your meeting notes, and I love it. Um, I ran out of credits for it the other day, and I felt like I was thrown into the stone age. I didn't, I had to go back to taking notes manually. But one thing I always think about is it has, there have to be guardrails around how I'm using a tool like that. When I pull it up, I look at what happened during this meeting. And it might say, oh, you told that, you told Tom you would give him 10 apples. I go, I don't remember saying that. Where did you get this information from? And it's misquoting or trying to summarize something that I didn't unde, or that I said differently. And so when I look at the actual source of where it's coming from, I get how it gets to this conclusion, but it's inaccurate. And I would be concerned about if I'm going through a long conversation with my physician, talking about what problems I have, and there's an AI scribe to, to try to get through a little bit more of, you know, what actually happened during this encounter, there should definitely be some guardrails in place on how to safely and effectively use a tool like that. And it goes right back to your point on AI literacy. The physician needs to understand what are the limitations of a tool like this? Where can there be a potential risk that's introduced? Uh, of course, if we're going through this conversation, the AI summarizes everything and says, yeah, based on this conversation, it sounds like the patient should, um, start on, extreme example, but should start on radiation treatment. With that, without that being necessary, and the physician doesn't go through the process of verifying it, there can be a lot of risk introduced. So I think that the the user facing and the user education element is vital with a tool like that and understanding that it's used in a safe and secure context. Guest: And, and you have to be careful because if you show up with a list of 10 differential diagnosis, now you put the physician in a, in a, and a physician assistant, nurses, PAs, everybody. Everybody, right? You put them in, um, you, you might send them down a rabbit hole. Right? So, can you imagine, you, you're on the ward, right? And you're, you're sick and you know, you spend most of the time interfacing with your nurse. Right? That's where you spend your time interfacing. The doc comes in every so often. Okay. And so, can you imagine you have a list of 10 things and you say, hey, can you, can you look at this list of 10 things and can you tell me what, what, what you guys are missing? And so, it's, it's that being able to come back and say, no, this list, you know, these are the concerns with it so. That's the other thing, right? Host: I wonder how often someone comes ready to the doctor's office with I asked ChatGPT what's wrong with me, and it thinks this. You have to do X Y and Z. I'd imagine that is a constant headache for physicians. Guest: It's interesting, Trevor, that you bring that up because everybody's different. Every person is a little bit different. There are patients, and that one, I, I always tell my, my last operative case was in 2018 so it's been a little bit but, um, but there would be patients that, that would say, doc, don't, don't, don't give me all this baloney. Help me, um, like I was a family member of yours. And then you have others that come in, um, and, and, you know, are very specific. So you get, you get it's, it's a mixture, it's a mixture of things. But you have to be ready to, to, to deal with all those, right? All those the empathy part all the way to the person that uh, that knows more about the specific topic than you do because they read it on the internet. Host: Everyone who has access to the internet is an, is an expert on everything it seems like. Christian: Yeah, I think it's interesting that we're talking about in school, learning how to prompt AI, like as a foundation when we still haven't added things like how to set a goal, how to have emotional intelligence, you know, how to balance a checkbook. Some of the fundamental things are still not in school, but when we're talking, you know, it probably will happen, like how to program AI, and we've skipped all these other things that actually matter in my opinion, uh, from a education perspective. Host: You know, I did learn how to balance a checkbook in school, but I can think, I can count on one hand the amount of times I've actually had to write a check. Christian: Well, it's not so much the checkbook. It's it's manage money I would say. Host: Yeah. Guest: You know, you know, it's funny because I heard something about this recently, the person, I forget I, but the person was saying we're great at doing complex things. Right? So, we were able to get a human being, a person to the moon before we put wheels on our suitcases. So we like the complex. We go for the complex, but I'm telling you, I'm from the era and I'm sure both of you are too that suitcases without wheels and that was terrible. was terrible. And so we did we went to the moon before we were able to get those little spinners on and so um, that that may be what we're talking about that uh we we we go for the complex or um, you know we don't want to address emotional intelligence and we have difficulty addressing goals. Host: Yeah, and I think it could be more commonplace things. A suitcase or you know, dealing with money, dealing with finance. These common things everyone has to deal with just the mundane parts of life. It's not as exciting to try to innovate as saying, we're trying to make computer think like people. We're trying to make them talk like people. Do things nobody could have ever imagined. That's exciting. That's cutting edge. Everyone wants to be a part of it. But going back to, well, if someone has wheels to the suitcase, it's gonna make a lot of lives easier. Yeah, it's just a suitcase though. It's not as exciting and so people I think you're right, people try to get caught up in exciting, complicated, difficult things and leave some more very important but simpler problems to the wayside. Christian: Cool. Well, we're coming up on time, so let's, uh, I usually go around the room and ask for some last minute words of wisdom about AI in this context. Uh, and maybe some predictions for the future. So I'll start with you, Trevor, and then we'll go over to Jose. Host: I think a really important thing to keep in mind, listening to this as a medtech innovator or as as an early adopter, someone trying to integrate AI into your workflow, into your health care delivery organization, your product, whatever it is, is understand what goes into your AI is what you're getting out of it. Garbage in, garbage out. High quality training is going to lead to high quality results. And it's not something that you can rush. You can't just throw questions into an AI and hope for the best or just plug it into the use case and expect it to work. Focus on what you're putting into it, focus on what you're getting out of it, and take your time with trying to iterate and produce meaningful results with AI. Christian: All right, Jose, what what's your last minute words of wisdom here to Guest: Just a couple things. First to thank Christian, you and Trevor for inviting me. Me and this is a big deal for me my first first podcast. So it's a it's a big, big deal. So that's that's number one. Thank you. Um, number two, um, I like I mentioned earlier, I'm an early adopter, I believe that there is going to be uh, um, tremendous growth and these tools are gonna help us and they're gonna improve our life, um, in many different domains. I think understanding them, AI literacy is key, and having guardrails. Having guardrails. Um, things, privacy, ethics, governance, etc. But again, I, uh, I see in terms of in medicine, the growth will be in the abient scribe. It's, it's, it's going at an incredible pace. I think in two or three years from now, it'll be a thing of the past talking about it. Well we will expect for that to work. In terms of diagnosis and in terms of treatment, I think it will complement the physician, the nurse, um, and others. Uh, but, uh, but in closing, I, I really again, uh, Christian, thank you, Trevor, thank you. I really appreciate it. And I appreciate what you guys do. Because you're on the safety side of this. you really, I know you, I mean, I don't think about it, but you're on the safety side of this and very appreciative. Christian: Well thank you. Uh, yeah, we're on the, the patient safety side. And you mentioned Waymo earlier. So I will I guess back up with both you said. I think we need to take our time with AI. Uh here in Phoenix where I live I take waymo all the time, and there's a scenario where I was in a waymo and was behind a construction truck that had a signal going left and one going right, and the waymo literally just went back and forth like this. It couldn't figure out what to do, which side to go around. And I think a human had to take over and make tell the way what to do to go around the left side. So I think we're still at that state and we'll be at that state for a long time with AI where we still need that human interaction because it's going to be quite some time before we mature with AI. Host: We get the Waymo takeover notifications from manual override all the time if you're ever taking one by Oracle Park during a game. The whole city just, yeah, yeah, the traffic is so bad in downtown. there'll be a little pop-ups like, hey, just letting you know I'm getting you through this traffic and then we'll get you back on the road. And then the Waymo takes back over once you get out of downtown. Christian: Oh, I haven't had that many takeaways here, just a few. I guess uh you got more traffic maybe in San Francisco area. Host: Yeah and the roads here just make no sense. Christian: Well thanks again Jose for uh joining us and thanks everyone for tuning in. We hope to see you on the next episode of the Med Device Cyber podcast.