From Concept to Compliance: A Guide to Med Device Approval | Ep. 24

Hello and welcome back to the Med Device Cyber podcast. Today we're going to be talking about some of the key regulations that are applicable to medical device cybersecurity, and some conversation about quality systems and making sure that you have a secure quality system, something that is well-designed so that you're compliant through any regulatory approval processes that you need. We're joined here today by Mark Swanson and Steve Gumpertz from QRX Partners. How are you guys doing today? We're good, doing well. Awesome. You guys are up in Denver, right? With a little bit of rain, a little bit of fog. Yeah, we're at the ASQ WCCQI conference, the ASQ World Conference on Quality and Improvement. So, that's where we're at. Very nice. Yeah, we haven't been to that one. I know we went to RAPS in Long Beach last year, that was a pretty good event. But yeah, typically we're at the LSI device talks events like that. But we just got off of a stint. I think we were at three conventions at once, and so everyone was flying all over the place and nobody was really sure where anyone was. But I think I'm in my fourth in four weeks. Oh wow. It's good that I even could figure out what city I'm in. I know. Yeah. Sometimes I wake up and I'm just like, where am I? I'm in a hotel somewhere, that's as far as I know. Yeah, I'm talking at all these conferences and it's just funny, people come up and go, 'Oh, I just saw your talk,' and I have to go, 'Which one?' Yep, exactly. That was my March, too. I was at a couple different ISO meetings with TC210 in Japan, and then over to Paris for TC 176 on 9001. So, very nice. Well, I'd love to hear a little bit about what you guys do at QRX Partners and then some background on yourselves as well. Sure. So, QRX, it focuses obviously, as the letters would imply, on quality and regulatory. Although it typically goes in the other order, where we work on regulatory first and then figure out the quality constraints or requirements according to the regulatory plan. We've been in business for five years, based in the Twin Cities in Minnesota, but we have a global presence. Our primary focus is on smaller companies, particularly early stage. We find that those companies are often underserved in getting the guidance they need. You know, it's usually a couple of really smart engineers or doctors and they have a great idea for a new device, and they have zero understanding of the regulatory pathway ahead of them. They're in this mode of, they know they need money, but they don't even know how much to ask for and how it's going to be staggered. And that's where we help them figure out, look, we understand. You know, sometimes in their exuberance, they're like, 'Yeah, we got like two credit cards between us, we're going to max them out.' And you know, 'What, six months and we'll be on the market, right?' And then we come in and say, 'No, you're really looking at, you're going to need like three, four million dollars and this is going to take you two to three years.' But we'll help you understand that pathway, where the pauses might be, when you'll have to go out and get more funding, and then how do we find you the best pathway through the regulatory bodies and then get them set up with a quality management system. Because Steve and I have both been at the large companies and so we understand all of these different pieces and bringing that knowledge to the smaller companies. I mean, there's nothing worse than, you know, because it takes longer time, you run out of money and you can't bring your product to market. And so, we want to avoid that for those small companies, help them get there quickly, using our expertise. A really interesting statistic that I heard, and this was just from one investor, but we were talking with him about his portfolio. He's focused on MedTech startups and mostly mostly in that early stage, seed round, Series A. And he was saying that 93% of his portfolio fails. Not surprised by that. Yeah, I know. And it seems crazy to think of, but I guess that 7% is successful enough to offset those 93%. Some of that is going to be it just doesn't pan out, right? The technologies or the benefits don't pan out relative to what's on the market today, what we call the generally recognized standard. Just my humble opinion. But it's those companies that can't get there fast enough, right? So they're working on something but somebody else is working on it too, and that other company gets more funding or whatever. You know, that type of thing happens, they don't have the right expertise, all those types of things. Yeah, that was where I was going to go next with that same kind of thing was, and I think a lot of times they don't, whether it's pride or they don't know how to find the resource they need to guide them, and then they stumble, right? They think they read the guidance documents, you read the instructions on the FDA website and it sounds like, 'Oh, I understand how to make a submission.' And then they send a 510(k), and then they get a deficiency letter back or a non-refusable, and then they lose time, right? And they're burning money that whole time because they don't know where the landmines are and they just keep stepping on them. You can tell Steve and I have worked together for a while and we're kind of finishing each other's sentences and that kind of stuff. We've been at this a long time. So, I don't know, you want to talk about your background a little bit, Steve? Yeah. So, I've been in the MedTech space primarily on devices for 34 years. That was after seven years in the computer products industry where I did software development. So I've been, Mark said, we've been in some of the larger companies, worked at smaller companies, companies that are peripheral to the industry, and you know, now consulting through QRX. Mark and I also teach at one of the local universities here in Minneapolis, St. Cloud State University, where they have three master's degree programs specific to MedTech: one in quality, one in regulatory, and one in clinical. Mark at one point was the program director for the quality program. I helped create that program and I teach in four of the courses, and Mark, you teach three. Yep. So we get a chance to actually educate people to become true quality professionals or regulatory professionals, rather than, as I always like to joke, 'Don't be a hobbyist in this.' Yep. Yeah. That's not, I think that regulatory is not really an industry that you can dabble in. It's not something where you can just know a thing or two here or there. There's just too much. It's pretty all the time, right? So we post every Wednesday, we post a column on LinkedIn or a series called 'Inadequate Response,' where, and it's me looking at some of the weekly warning letters and then trying to pick out what went wrong, right, to get to that point. And I often make that point: it sounds like a company that guessed at it, right? They didn't really take it seriously or didn't have the right competent personnel in place to keep them out of trouble, that they just tried to figure out, 'Oh, all right, I think we understand this. Let's do it this way,' and then found out that was absolutely the wrong interpretation. And sometimes worse yet, when they get it wrong, they want to argue that their way is right. He's just not going to win that argument with the FDA. Yep. Yeah. The FDA quite literally wrote the book on what you're supposed to do and what you're not supposed to do. And so, yeah, you're not going to win too many of those debates with the inspector. Oh, no. No. Yeah. I've seen that situation come up a fair amount when we're handling deficiency response. There'll be a design flaw or some, you know, issue with the risk assessment, and the client comes to us and they say, 'Hey, we think we're right and this is,' you know, 'This is how we want to explain to the FDA that we're right.' I always say, 'Look, you can think you're right. I can even think you're right. The FDA is the one who is right.' And so that's what you have. One of my favorite warning letters that I analyzed for the column was one where a company their response to the 483 findings was to come back and tell FDA, 'No, you're wrong, we're not a medical device, so none of this applies to us.' And FDA had clearly said, 'You meet all the criteria,' and they were going to argue the interpretation of the act as to whether or not they qualified as a med device. It was like, you're not going to win this one. One thing that I see pretty often, not necessarily on, you know, is it a medical device or not, but an argument on, 'Is our product a cyber device?' And we see a lot of problems with that, mainly just because of how the FDA defines a cyber device on their website. It's a whole bunch of 'ands.' They say, you know, 'You have to be a software-enabled device and have wireless connectivity and run,' all these different criteria when in effect they're, they should be 'ors.' And that is the way that the FDA treats it. So they say, 'Do you have software or are you connected to other products or do you have an operating system?' Usually, as soon as you have software, you fall under that bucket. But a lot of companies have tried to make the argument that we're not a cyber device because we don't meet the published definition even though the unpublished definition is what should be adhered to. And I can tell you, you know, I had a submission that I just did and the only, we gave them pictures, of course, as part of the submission, and there's a, you know, a screen, right? An LCD screen. And it's all it is, is a PID temperature controller. And they're like, 'You have software.' I like, 'Wait a minute, it's not really software.' But they're like, 'Nope.' And so I had to fill out all of the stuff for cyber and that kind of stuff, just because there is firmware inside of those enabled, you know, pieces. So even if the software is inaccessible, which is something that I feel like really trips up a lot of manufacturers, that software can be completely boxed in. You have, like, you know, no wireless, no nothing. All it's doing is showing a result on a screen with some little, you know, microcontroller just running a single function. And you still need to do all of your software documentation, all of your cybersecurity documentation. You still need to do cybersecurity testing on it. And the trick is, now more and more devices have some sort of software built into them that traditionally didn't, right? Knee implants used to be really straightforward. It's all mechanical, right? Not anymore. Now there are sensors. Sensors apply firmware that has to talk to something that has to process that information. Congratulations, you're a cyber device. Yeah. And and of course, I mean, the hospitals are asking for a lot of that stuff, too, because they want the data to get transferred from whatever device into their hospital, you know, information system. And so they're looking for that interconnectivity, right? And those types of things. So, I mean, it's natural for us. So anyway, if you don't mind, I'll just go back into a little bit of my background. I spent a little more than a decade, about 12 years at Mother Medtronic in the cardiovascular division, working on heart valves and blood pumps and oxygenators, all those kinds of things. When I was there, I got a lot of experience with some combination products. So Medtronic gave me a good, good wide base for my experience. And then about 13 years ago, I went out on my own, and then came back together, like Steve said, five years ago into QRX. The two of us coming together, and been doing, so been doing the kind of regulatory quality consulting. You know, my idea when I when I started my own company was to make sure that we were taking care of the small companies. And so, you know, Steve with a common vision that we come together and make sure that the small companies get the expertise that they need. So if any small manufacturers are listening and they need to know what are those key areas that they need to cross, because obviously there, you know, hundreds of different boxes to tick to try to make sure that you have all your regulations in place, you're compliant with everything you need to be compliant with, you have a good quality system, what is the starting point? It can seem really intimidating and daunting. So I want to hear your opinions on where do you go from? Yeah, it's the classification process, right, which consists of going through FDA's database of product codes, which is huge and can be very confusing. I was just talking the other day at a webinar or with a client, sorry. And if you type in the word 'balloon,' right, looking for a balloon catheter, you get 34 hits of possible product codes. And some of them are class three, some are class two. I think there was even one that was a class one. That's the challenge, is, now which is mine, right? Which one of those do I fall under? And so, you have to understand how to read because some of them are coming out of the neurological side of FDA, some are coming out of the cardiac piece, right? You got to look at all the details. And then you got to go see, well, who else is using that code and does their device look like mine, right? And then at some point if you exhaust all that and go, 'Nothing here matches,' then you go, 'Uh oh, I might be class three,' right, because there's no predicate for it. But that's the challenge is just figuring out, but question one is always, 'Am I a medical device?' That's again, the definition is broad, so it's almost always going to be a yes, but there is some nuances in there. And then it's, 'Okay, what medical device am I and what pathway am I going to have to follow?' And then followed right behind that is, 'Is there anybody else out there doing the similar thing that I can use them as a predicate, right?' Because that's going to be the most common pathway to market is going to be, 'Do what we call me too,' right? You have a product on the market, I can have my product on the market that does the same thing. And so you're looking for other devices that are in a similar space. Yeah. And then there's kind of grey areas, right? It's not always just clear one classes one versus two versus three. I was working with a client and found two possible product codes to use that a lot of devices like theirs were using. And the owner was like, 'Well, no problem because they're both Class One.' I said, 'No, not really.' One of them was what's called Class One Reserved, which means even though it's Class One, you still have to do a 510(k), right? And I said, 'Unfortunately, that's the one that matches your device better. So, you're on the 510(k) pathway.' And then you get into things like, 'All right, there are no predicates,' right, which by default means you're now going to be Class Three PMA pathway. But if you're not high risk, then you can make the case for, 'Wait, we want to go what's called the De Novo pathway,' which is somewhere between a two and a three. And this is, you know, again, right, why it's not for the hobbyists. And a big reason, I you come back to that, is because lots of times it comes back to what they've written as their indication for use, right? And so when we talk about that classification, those little nuances, it depends on what you write there. And so we, this is why we have to tell them, look, 'Don't wait to come and talk to us,' because we need to get up there front. Because when you start creating your documentation, creating your product specs, that all comes off of that indications for use, and it's going to decide down the road what your pathway is going to be and potentially how hard that pathway is going to be. And it's amazing how often they haven't even, they don't even know what an intended use statement is. They have in their head what they're trying to make. Maybe they've written a couple of sentences. And then I said, 'Well, in order to dive in and do the research, I need to see all of the details of your intended use.' And then they kind of tilt their head and go, 'What's that?' And I go, 'Well, what exactly does it do? What are you treating? Who are you treating? Who's going to use it?' Right? Under what condition? There's a lot to it. Yeah. And suddenly they go, 'Oh, we've never thought about all that.' I said, 'Step one, you have to think about all about all that. Otherwise, you can't proceed,' you know, 'from the first step.' Well, and I go into the complications sometimes that get gets put out there, too, because if you tell me your pediatric use, there's five categories of pediatrics. Which one, which one of those are you talking about? Are you talking about infants? Are you talking about, you know, young adults? Which ones are you talking about? So these again are some of the reasons we need to do some navigation for them. I feel like that's kind of startup 101 is solve a problem. And so it's surprising to hear that a lot of manufacturers are coming in with, 'We have an idea for the product but we don't really know why we have the product.' I think they do. I think again, they have it in their head. It's just they've never had to formalize it. Nobody's asked them those specific questions, but they can usually answer them. And then, you know, so I usually just give them a, 'Just answer me these seven questions,' or something like that, and they can usually do that pretty quickly, right, once somebody's asked them. 'Oh yeah, we had thought about that.' Good. Let's get this written down, put it into one statement. Now I can use that as the basis for figuring things out. It's more about taking it from the big picture, the concept that they have and the idea, into the details, because FDA is looking for the details, right? And so that's what we, we help our clients do is, 'Let's get those details written down so everybody else is on the same page.' And then it's also refining that statement, be careful of the words they choose, right? They can't use words like 'cure' if that's not what they're actually going to do, right? Or it's going to be 'best,' you know, things that you can't actually quantify. They all, you know, back to our cybersecurity, everybody loves now to say that their software is AI-based, right? We go, 'Well, that sounds sexy, but if you're actually going to do that, if you really are, just understand there's more hoops now.' So, is it a marketing ploy or are you actually leveraging machine learning and AI, true AI software? It's funny how often we find out that no, they're just developing software. It might be really good software, really complex software. Yeah. Doesn't necessarily make it AI under the current definitions. It's fast. It's, it, it displays good information. All of those types of things. And, you know, but and and they're learning from it, right? So often, and we complete this cycle, we we can talk about the stuff what we're talking about at the beginning, but at some point, when you get your product on the market, you start getting feedback, you start getting use, you know, even as you're going through the initial stages, you start getting some use and you're going to make some changes. Okay. But that's not machine learning. That's you learning what the machine is doing and making changes, right? That's just regular learning. Yeah. Yeah. It's kind of interesting, the whole AI craze. It seems similar to, it was around like 2010, they started classifying medical mobile apps because at the time, you know, smartphones were really rolling in and then every single manufacturer as a marketing ploy just wanted to attach a companion app onto their product. And, you know, 'Why do you have a companion app? Well, I don't know, marketing said we needed one.' And then the FDA kind of figured out what was going on there and they started regulating those and said, 'If you're attaching a companion app to this, now we're going to consider it,' you know, 'a component of a medical device there. Here are the hoops you have to jump through.' And so, accessory. Yeah, it's an accessory. I know with AI, it's still in its infancy. We're still seeing the regulations develop and they're changing constantly. And I think that it's going to be a very difficult beast to wrangle just because of how fast AI itself changes. But I'm sure, you know, as as we become a little bit more comfortable as global industry and community, we'll start to see some regulations get more solidified. And then I honestly think we'll see some manufacturers start to pull back on their whole AI stance just due to the fact that there are going to be so many more hurdles to go through. Well, yeah, it'll be about clarity, right? They'll finally know exactly what the rules are and then they'll know what the risks are and whether they want to claim, make some of those claims as a business. Yep. Yep. But you're right. Right now it's a lot of guidance, right? Which everybody hears and thinks, 'Well, that means it's optional,' right? Well, FDA actually doesn't understand the term guidance. So, you need to understand when they say guidance, they mean this is what we want you to do. I also love reading through any of the guidance documents. They say, 'We recommend that you do X, Y, and Z,' which means 'Do X, Y, Z or else.' Well, obviously, technology is always going to move forward and some, you know, AI seems to be moving really quickly. Cybersecurity is obviously a hot topic, you know, not just recently. I mean, a lot of companies go, 'Oh, yeah. What's with all these new rules around cybersecurity?' Go, 'Actually, they're not new. They're always supposed to be doing this. It's just now they're clear,' right, 'as a requirement.' So, well, and as you guys know, I I'm I'm working in standards development, you know, 9001, 13485 for med device and quality management systems. And we get asked all the time, 'Well, what about for AI?' I'm like, 'Well, AI is a medical,' you know, 'if it's a medical device, it's a medical device under quality system. There's no need for a special quality system for AI or machine learning.' You know, understanding how all of these things fit together. And and talk about, you know, product design, design and development, those types of things. That's already covered in ISO 13485. It's and and we very intentionally didn't make it to hardware, software, in vitro diagnostics. I mean, there's any number of things that you can talk about in the med device space that have these little nuances, but 13485 is generic enough that it covers all of those things. And then you need to look to maybe some other standards out there. And so one that comes up often for me in in the questions is ISO 42001, which is information technology management systems, artificial intelligent management system. Okay, that's not a quality management system. It's a system for managing AI, for managing that technology, and that doesn't make, doesn't make any changes to the quality management system requirements. It just adds to it. Exactly. And AI, it's not as new as I think a lot of people believe it is. It's been around a long time. The term was actually coined in the, like, the late 1930s. Yeah, it's, it's an old concept. You guys remember back in like Windows XP when you had Clippy on your computer? It, that's, you know, if you look at the definition, that is technically AI. And even moving past, you know, silly little keyboard assistant, there are a lot of technologies that have utilized machine learning over the past 20, 30 years. This isn't any new, this isn't a new concept, and so it's applied to these regulations before, it applies to these regulations now. Obviously, we need some new guardrails because of how quickly AI is evolving and the fact that it's becoming really, really smart really, really fast, but it's not a new technology. Yeah. I always say the piece that makes it different is the machine learning is where you step over a line, right? The AI before, you know, the old definition AI was just, yeah, we would take a set of data and now it was the software could analyze that and come up with some answers, right, or recommendations on things. Now it's sort of live, you're feeding it populations of data that's continually changing, so it's continually supposed to be learning, at least during development, and that, so I think that machine learning adds a different aspect to it, right? Because now you didn't get to really curate the, the data like you would have in the old days where it was a finite set of data. Um, you know, if it's a large language model, which you wouldn't want to really base a medical device on. But it brings up the questions of, yeah, how are you going to control what the software has to learn from? Right? I said it yesterday in in a in a one of the conf, the sessions I was sitting in. I said, 'You know, what if you ask an idiot a question, you get stupid answers.' Right. That's the nature of the problem with machine learning. Feed it bad information and it learns the wrong things. Exactly. It's garbage in, garbage out. And you alluded to, right, it's just, it's learning is learning, right, whether it's a machine or a human. And there's all kinds of great science fiction books that you could learn all kinds of, you know, warnings from that talked about this stuff decades ago going, 'This is difficult. How hard is it to teach a three-year-old, right? What's right and what's wrong and what's dangerous and what's not? Now you're going to try to emulate that in a piece of software.' 'No, this can't be that hard to do,' right? Well, and and again, I I often see AI as a tool, right? It's a tool that can help you in development. The question is, are you actually implementing AI in your device? Is it learning and changing as it goes or are you using the AI machine learning in order to enable a change? Right? So that's that's how I say, I mean, it you can talk about AI and AI being used in design and development. That's no problem. Because it just is, it aids development. It's a tool. It's when you put the actual AI and it would make changes on its own. That's where FDA gets really, really concerned, because I mean, you got to face it, we talked about from a regulatory standpoint, we are very change averse. We don't want things to make changes on its own. We want to have control over that. And so if you're using AI machine learning to learn that and then implement a change, you're going to learn. I get that. It's it's the same thing. I was attending one of the talks on risk analysis, a tool called FMEA. They're using AI in creating these failure mode effects analysis. I'm like, 'Well, yes, it's AI, but you've already done all of the programming up front. I'm just now using a tool that's really smart and can go look at it.' So, it's not really AI FMEA. It's AI in the development of the tool so that the tool is better, right? Have a product, right? Yeah. And I and I think you see that in the existing frameworks and guidance that are out there, right? Is that no regulatory body is going to approve a device that learns in the field, right? Because then that brings up Skynet and Terminator thing. The idea is, you train it, right, until you're sure that it's doing the right things and then you release that version of of its knowledge and then you bring in more data in a controlled environment and let it, let the next version continue to learn. The challenge is the subtlety of that, right? How do you know what it's actually learning? Some of it may be obvious. Some of it is going to be hard to figure out until it's actually in the field and it makes a certain decision. You go, 'Oh, wow. Well, how did it, how did it follow that logic path, right?' I mean, software is full of that to begin with, right? We know software is always flawed. It's the only medical device that goes out knowing it's not working correctly, right? Now, add in the AI element and there's all this subtlety to learning. So it's, it's not a wonder, right, that the regulatory bodies are being really cautious and conservative about this because nobody can say yet exactly how are you going to know exactly what changed in its learning, right? How are you going to map that and say, 'Oh, no, no, don't, don't learn that. That's the wrong information.' Right? There's, there's no way. It'd be like trying to go to a person's brain and go, 'Oh, no, no. You learned that fact wrong. Let's pull that up.' Right? Well, and in and yesterday in that talk, one of the things he said is that we know that it's wrong 60% of the time because he, he started the whole talk with, 'Does everybody trust AI completely?' And of course, nobody has their hand up, right? Because it's wrong an awful lot. You have to be able to to to review and check. We can't just take what it's giving us and click okay and then go. We, we know that, right? Even if it's only wrong 10% of the time, you know, in the medical space, that's too high, or like a just failure rate. The one thing that, you know, is always a little bit of a hot topic around AI is the difference in regulations in America as opposed to Europe. And so I'd be curious to hear your thoughts. Obviously Europe is taking a bit more of a cautious, regulated approach. And it kind of seems like in the US, of course, there are regulations in place. There are guard rails implemented depending on your AI application, but for the most part, it kind of seems like the Wild West. Even, you know, a lot of other countries are far more regulated than the US right now. So I'd be curious to hear kind of what your opinions are and the pros and cons of each approach. Yeah, with the, the European, the AI directive, in that part of the regulation. I mean, there's lots of concerns there. I think you're right that, I mean, they are taking a more cautious approach, which means they're being more restrictive. And then you have to navigate both the AI regulation and med device regulation and somehow put these two things together and making sure you're not violating any of them. And so that that to me squashes a little bit of the innovation that can come with AI. I think the FDA has taken a more direct approach because they brought a bunch of experts into the FDA that are experts in AI machine learning so that they could learn and keep seeing what's out there. Because what, you know, a little bit of the dirty little secret is that when when companies make submissions to the FDA, if you're at the FDA, you get to see all of that. If you're not at the FDA, if I'm, you know, just a, you know, Joe Schmo out in in in in town here, I don't get to see all of the other stuff that people are submitting to the FDA, but inside they do. And so bringing that expertise in, learning from that, and continuing to learn from that, I think will continue to foster that innovation. I think there that at least the device side is doing it correctly. From my opinion, but it's interesting. I was at a talk a couple of months ago, a legal firm that was talking about the legal aspects of this, and they did note, there are a couple of states that have AI laws now in place. I think I think it was, I think it was Colorado and Utah, I think. I think Utah was the farthest along and they essentially copy the EU in in their laws. And I think California's looking at it. Yeah. And then he said there might be like three or four other states. So the states aren't ignoring it, you know, even if nothing at the federal level or they're, you know, they're being, you know, more, which of course is, it contributes to the Wild West, you know, thing that you just said, you know, 'How, whose rules am I following?' Right? Yeah. And I think it becomes a little bit of an interesting area putting that regulation down to the state level. And so I wonder, obviously, California out in Silicon Valley, that's kind of the home of all of this development, all of this AI, all of this innovation, technology just typically comes from that area. And so if all of the regulations are going to get more strict in California, I wonder how much you're going to see some of these companies moving out to Nevada, Arizona, some of these other places with a little bit more of a blossoming tech scene that don't have, I was going to say, it's the question of whose rules do I follow. Yeah, exactly. There is some of where you're located, but it's al also where you're marketing your product. I was going to say, yeah, it may not matter where you're located. If you're going to, like, because today, if you're going to try to sell a medical device in California, then you subject yourself to California inspections. And and as much as we can can talk about the FDA, don't forget that the states have their own enforcement agencies as well, Department of Health and all of those types of things that they can do some pieces here. So, you know, that's that's what we're watching out for. And again, it's the reason that we're out here is because we are watching out for all of these things. We're trying to stay up on the latest trends. It's why I'm part of the ISO working groups is because then we can look around the corner for our clients to help them navigate that and make sure that they're not going over the, you know, the lines in the road. Definitely. Yeah. I think that it ultimately comes down to whatever you're doing, whatever technology are you're using. Just get prepared for it. What regulations are you going to need to know how to follow? How are you setting up your quality system before you design a product? The amount of companies that come to us and they say, 'We just finished development. We're so excited.' We start asking them about their software documentation, about what they have in their QM, their risk management procedures, and we just get blank looks and zero information. And they say, 'Well, we wanted to submit in a month. Can we still do that?' No. The answer is no. We can't still do that. Yeah. I mean, on the broader sense, you we'll ask the question, 'So tell us what applicable standards you chose,' right? And they go, 'Well, 820 and 13485.' Yeah. Exactly. Yeah. That's not the full list, you know. Well, in fact, you know, I recently ran across a client that, you know, they talked, they knew 62304, which is the medical device software. And I'm like, 'Well, but do you have the amendment one?' And they're like, 'Wait, there was a change?' Yes, there was a significant change because they made a mistake in the original version talking about the probability of occurrence of one or 100% for a lot of the software. And that's simply not true. It doesn't happen all the time. You have to have those things. And there's better ways to do risk analysis and threat assessment and that kind of stuff from the AI machine learning side and software than there is, you know, for the traditional design risk analysis or DFMEA. So there's things to learn and understand, you know, what the expectations are. I've always thought that's interesting in 62304 how it's talking about probability of occurrence when you're doing all this analysis. And if you put the word 'probability' in your risk analysis for an FDA submission, they will kick it back without second thought. They are not tolerating that. You need to say exploitability. You need objective factors. And so I feel like that's another issue that a lot of people have a hard time navigating is some of these guidances are really conflicting even just directly in the FDA guidance. What the FDA guidance says compared to what EAR says are not the same things. And so that that's really a, you know, well, but it's because FDA, like I said, FDA has engaged these experts to come in and help them understand what they need to know, and so they're putting the latest and greatest into some of the FDA templates and all that kind of stuff, and the standards take a while to catch up, right? So don't necessarily just rely on the standards, you need to figure out what's really going on and asking these questions. It's one of the reasons that especially with software, we will always do a presubmission. We will always go to the FDA before we're doing our final submission and ask these questions. You know, 'Do we, is what we're doing sufficient to meet your requirements?' Because they'll come back and say, 'Oh, no. We've updated that. Here's some additional information.' I think that's a, it's a little bit of a double-edged sword that EAR doesn't need that review process. They don't need to publish a draft EAR. Because we're on version five now, right? But it, it changes really fast, which is good. It's actually 5.3 or something like that. So yeah, it's like you got to remember to not just like download it and keep using that old version. You always have to go get the newest one, right? It changes fast, which, you know, it's good if some, if something in the industry is changing, regulations are slow to adapt, like he said. And so EAR can be faster to adapt. But, you know, actually what I said is standards are slow. If you want to talk about how regulations take, you know, keep keep in mind that the QSRag is 1998 that we're just now updating in 2026. So that that's how fast regulation can often works. So now we're redefining slow even. It's real slow. There was some controversy last year because FDA had floated the idea that they might just start publishing final drafts of guidance without, you know, public comment. And I suspect it's, you know, it sounded like them just trying to take over, you know, be power trip, but I suspect it was more like because we need to get the information out really fast, right? Because the standards are going to take a while, the regs are going to take even longer. And I think they were just looking for how can we get you our thinking faster, right? So, you know, obviously they had to take a step back from that because there was a lot of push back from industry. 'Wait, no, no, no, you can't just throw guidance at us without a chance for us to look at it.' But, you know, things like updates to EAR and updates to guidance that's already final, I get it where they may want to just put it out there and not go, 'We're doing an update to this guidance document that we put out three years ago. We don't want to wait 69 months to get feedback because industry needs this information today.' Well, we're coming up on time here. This has obviously been fantastic conversation on all the crazy stuff going on in the regulatory world. At the end of our episodes, we always just like to kind of cap it off. If you could just give a word of advice to manufacturers, to someone who's starting up their company, someone trying to navigate all these regulations, how would you sum up just a quick little summary of what they need to know? For me, it's always get expert help. I know it sounds like it's expensive. It will be far less expensive than the errors you're going to make downstream. Expert help like QRX. Yeah. Well, and and because one of the things we say, there's one thing that we know, preventative action is always less expensive than corrective action. So, getting around the problem, not hitting the wall is always better than, you know, having the crash. And and I'll finish that up with you, you need to engage somebody early. We often engage with, you know, potential clients and they never become our client, but we give them some information about the pathway that they need to follow, you know, things that they need to find out, things that they need to put in their pitch deck, these types of things. Because we know that they need that information to get along, because we don't like the 7% success rate. We we'd like it to be much higher, and and our clients are much better than that. So definitely. Well, it's been fantastic to have you both on the Med Device Cyber Podcast and yeah, thanks for jumping in.