Episode 5 · June 3, 2025 · 41m listen · 6,881 words · ~34 min read

AI in Medical Devices: Opportunities & Regulation with Matt Lemay | Ep. 22 - Full Transcript | The Med Device Cyber Podcast

Read the complete, searchable transcript of Episode 5 of The Med Device Cyber Podcast - expert conversations on medical device cybersecurity, FDA premarket and postmarket guidance, SBOM management, threat modeling, and penetration testing.

Prefer the listening experience? Open the episode page for the synopsis, key takeaways, topics, and Apple / YouTube listen links.

Episode summary

This episode of The Med Device Cyber Podcast features hosts Trevor Slattery and Christian Espinosa in conversation with Matt Lemay, the CEO of Lemay AI, to discuss the integration of artificial intelligence into medical devices and other regulated industries. The discussion provides a comprehensive overview of the opportunities and challenges associated with AI in medtech, emphasizing the critical need for a structured, engineering-driven approach to ensure safety, compliance, and cybersecurity. Matt Lemay introduces his company, which specializes in guiding organizations through their AI adoption journey, from initial strategy to implementation and regulatory approval. He draws heavily on his background in a medical device startup, where he was responsible for implementing the ISO 13485 quality management standard. This experience, he explains, has deeply influenced his company's methodology, which prioritizes the rigorous, risk-aware principles of engineering to build AI systems that are robust and verifiable, a necessity in high-stakes environments like healthcare. The core of the conversation revolves around the practical challenges of developing and deploying AI in a regulated context. Matt argues that the time for medtech companies to engage with AI was "somewhere between not yet and five years ago," highlighting the rapid evolution of both the technology and its regulatory landscape. A significant point of discussion is the emerging ISO 42001 standard for AI management systems, which Lemay presents as a vital, certifiable framework for creating traceable and compliant AI. The speakers differentiate between AI applications based on their intended use, noting that systems for exploratory data analysis face different regulatory hurdles than those making autonomous diagnostic decisions. This distinction leads to a broader discussion on the critical cybersecurity risks associated with AI. Several key risks are identified, with the most prominent being the "garbage in, garbage out" problem, where the quality of training data directly impacts the model's accuracy and safety, potentially leading to dangerous misdiagnoses. Matt also raises concerns about "data drift"—the degradation of a model's performance over time as real-world data evolves away from the training data. The conversation addresses the implications of deployment architecture (on-device vs. cloud), which affects everything from performance and latency to data privacy and sovereignty. A major unresolved topic is liability; using the example of ticketing autonomous vehicles, the speakers explore the complex question of who is at fault when an AI system fails. The episode concludes by touching on the need for AI to better communicate its own uncertainty, moving away from the tendency to "hallucinate" answers and toward a more collaborative interaction with human users, which will be essential for its responsible adoption in medicine.

Key takeaways from this episode

AI development in regulated industries like medtech must be approached with the same rigor and risk-aware principles as traditional engineering to ensure safety and compliance.
The quality of training data for AI models is a critical security and safety concern, as poor or biased data can lead to inaccurate outcomes and harmful misdiagnoses.
Emerging certifiable standards, such as ISO 42001 for AI management systems, are essential for creating the verifiable and traceable frameworks needed for regulatory approval.
The intended use of an AI system, whether for simple data analysis or critical diagnostic calls, largely determines the level of regulatory scrutiny and oversight required.
The deployment strategy for an AI model (on-device, on-premise, or cloud) has significant implications for cost, performance, data privacy, and cybersecurity.
Determining legal liability when an autonomous AI system makes a mistake is a complex and largely unresolved issue that impacts manufacturers, developers, and users alike.
To be viable on resource-constrained medical devices and to facilitate verification, complex AI models can sometimes be converted into simpler, more efficient mathematical forms.
AIs often present information with high confidence even when incorrect, highlighting the need to develop systems that can recognize and communicate their own uncertainty in critical decision-making.

Full episode transcript

Hello and welcome back to another episode of the Med Device Cyber Podcast. Today we're going to talk about artificial intelligence in medical devices as well as AI in regulated industries. We have a special guest on today, Matt LeMay, who is the CEO of LeMay AI. How are you doing today, Matt? Matt: I'm good. Thanks for having me. It's going to be a good conversation. Trevor: Looking forward to it. Christian: Yeah, it's a little bit of context uh, I met Melissa and I met Matt on the cultural tour, I believe it was at MedTech World Dubai. And then Matt gave me this awesome book here, the 50 inventions that shaped the modern economy. I haven't started reading it yet. I just got it a couple days ago, but I've kind of thumbed through it and it looks pretty awesome. Matt: Absolutely. So Christian and I were in the lobby of the Intercontinental at Festival City and you just see a group of people that are all CEOs and co-founders and engineers and medical devices and everyone's in shorts and flip flops and polos going on a tour. It was it was great conversations all around. We're I'm definitely looking forward to the next MedTech world event. Christian: Yeah, for sure. So you want to give us a little context about what you do Matt, what your organization does and and kind of and spin spin it a little bit on MedTech since we're focused on MedTech. Matt: Absolutely. Well, I think MedTech definitely shaped the entire growth of our organization. So, my team, Lemay AI, we're a team of 30 people, about 85% engineers that specialize in helping clients on their AI adoption journey specifically in regulated industries, including MedTech. We do this by delivering tailored AI solutions at whatever stage you're at. So, if you need strategic guidance on which projects you should implement, if you need some core implementation support, if you need some scale up or even now regulatory approval as AI is becoming more and more relevant in each one of these regulated industries, we have a framework for helping clients along this journey. Specifically about myself, I actually came from a medical device startup where we implemented ISO 13485 from scratch where I met my now co-founder, Daniel. And we did a lot of work back in the day not just in systematizing our design and development processes to be able to achieve a CE mark, FDA approval processes, Health Canada approval. Each one of those compliance mechanisms requiring a certain amount of of oversight and that's what help shape how we actually do AI. So, part of what we do now and how we do it is heavily influenced with engineering principles that are required to comply with a lot of these emerging standards. Christian: So when would an organization uh, like say a MedTech manufacturer. When would they want to engage with you if they're going to do, let's say image enhancement using AI on software as a medical device? Matt: That's a fantastic question. The there's a lot of ways of approaching it, and the simple answer is somewhere between not yet and five years ago. So, at the not yet level, what we're seeing in the regulatory changes and landscape is a lot of people have been pushing for various governance frameworks on how to do safe AI. The thing with a lot of these governance frameworks is that they're hard to audit and they're hard to verify and therefore hard to include in a lot of these medical device oversight frameworks and audit processes. What we're seeing right now is that there's actually a lot of work happening under one particular standard, ISO 42001, which specifically prescribes how to manage your AI systems. So, 42001 being called AI management systems. What we find very, very interesting with that standard is that it is certifiable. So for the first time, you can have an AI included in your medical devices that is able to be verified by an external third party. When it comes to image recognition, what you also have to look in mind is what is the purpose of this image recognition which will immediately impact the strategy that you want to pursue and whether or not a team like ours actually makes sense. So, if you say, I want to do exploratory data science, and I want to look at the pictures of dermatology, x-ray, I want to look at cells at the microscopic level to understand how they're moving, how they're understanding. And your intent at that level is much more investigative, it's much more open-ended, then you really don't need a lot of certification. You can actually engage in that direction as long as you're in respect with GDPR and a lot of these uh, data protection mechanisms, you're going to be fine.

1 / 9