Vulnerability, Penetration & Other Cybersecurity Testing Types Explained | Ep. 33

Welcome back to The Med Device Cyber Podcast. This is going to be an exciting episode. We're talking about the main fun part of cybersecurity that everyone wants to go over, which is cybersecurity testing. I'm your co-host, Trevor Slatterie, joined by the other co-host, Christian Espinosa. How are you doing right now, Christian? I'm doing well. I think it's interesting you think this is the exciting part. I guess it is. People think penetration testing and testing is sexy, and documentation is boring, typically, in cybersecurity. So maybe that's what you're referring to. I think documentation is sexy because without the documentation, the testing doesn't really matter. Yeah, no, that's what I get every time I say, "Yeah, I'm in cybersecurity." They go, "Wow, are you a penetration tester? That's so amazing." No, people usually go, "Oh, cybersecurity. Okay. Yeah. Awesome." Well, I'm doing pretty good. I'm a little bit tired. I've recently traveled about 28 hours, I think, from door to door, maybe like 30. Kind of like a six-hour layover in San Francisco. And slept a little bit, and slept in little hour chunks, and yeah, it is what it is. I was 15 hours time difference from where I am today, 15 hours ahead in the future. But that's part of, you know, growing a business and going to conferences and events is like dealing with jet lag. Yep. And I guess you're down in Singapore, so I guess you're getting used to the dry heat instead of the humid heat now. Yeah, Phoenix is a much drier heat, which is much more tolerable than the Singapore heat, which reminded me of growing up in Arkansas, just the humidity. It was like, it was miserable. I don't miss the humidity at all. I feel the opposite way. When I was living in Malaysia, I love the heat. It would be, you know, 95 degrees, 80% humidity all the time. When I go step outside in Phoenix, I mean, we'll start my car. It says it's 125 degrees inside the car. Feels like your soul's leaving your body when you step outside. I feel like my soul's leaving my body in the humidity when it's just like sweating out my soul. As soon as I take a shower, you step outside, it's like you lose a pound of sweat in like three minutes in Singapore. And you never feel clean. You feel sticky all the time. You do. Yes. And I can't sleep in humid either. I remember in Arkansas, actually in St. Louis, I got an ear infection. I'd went camping and I was sweating so much, my ear was like in water the whole time, basically, and I got an ear infection from it. So yeah, I'm not a big fan of humidity. Well, there you go. All right, so let's jump into our topic here. We're not talking about humidity. We're talking about our topic today is on cybersecurity testing for medical devices. And kind of what falls under that overall umbrella of cybersecurity testing, which are a lot of things that the FDA asks for and other regulatory authorities ask for. So where do, what do you want to start? Like what is like one of the main types of testing? Maybe we'll start with vulnerability testing. Let's start with that. That's one of the things the FDA asks for. And there's some common misconceptions between vulnerability testing and penetration testing. How would you explain like vulnerability testing first? So vulnerability testing, we're really looking for any risks through just various methods of information collection, through threat collection. This can be through automated tooling. This can be through manual review. But we're more looking at problems from, I guess, a static and automated perspective where penetration testing is going to be a little bit more of a dynamic and manual perspective. I think that the line is often drawn a little bit too explicitly between the two, since they often blend together very nicely, and vulnerability testing should be used in many cases as an input into penetration testing. But there is still a distinction in the tooling and in the process there. Yeah, and that's a good point. Vulnerability testing, you typically identify the vulnerabilities, and then that is often the first step in penetration testing because you have to identify a vulnerability to exploit it, and then penetration testing takes it a step further to see, once you exploit one vulnerability, what else can you therefore exploit, like once you're in a system or devices. For example, Exactly. And a lot of vulnerability assessments, so if we're looking at what the FDA wants to see, we'll pick one of the examples that they want to see, or static and dynamic testing of the source code. And so if we're looking at static testing of the source code, and we identify that we get a bunch of findings back saying they aren't handling any input sanitization in this codebase, and so we're making sure that they are making sure that bad input can't go into an input field, like a username or a password field, then that can be a good clue for the penetration tester to go, "Oh, well, if we're identifying this during the vulnerability testing phase, during our penetration testing phase, we're likely going to want to drill in a little bit deeper to these input fields, and see what we can do with that lack of sanitization." Maybe we can find an injection attack or try to extract some sensitive information through those fields. And so that's where they definitely have that blending in together. But there is, I feel like, so the main distinctions are penetration testing is usually done from a very manual perspective. And so a good penetration tester is hands-on keyboard, or hands-on the device, interacting with it directly, trying to have a little bit more of an in-depth approach to really see how far you can take a problem, where vulnerability assessments, they're quick, they're fast, they cover a lot of ground, but they aren't going down to those next layers. They aren't going all the way down to try to cover every bit of ground that's possible. It's a bit more of quick information collection. Cover as much ground as possible and look for the low-hanging fruit. And you mentioned like static code analysis under vulnerability testing as a sort of subset. There's also software composition analysis, which is effectively generating the software bill materials. Can you elaborate a little bit on that on the SBOM and how that is used for vulnerability testing? Definitely. So when we're looking at the software bill of materials, we'll start with looking at that source code. The first step for looking at all these different risks, we're going to look at that static testing. So that'll go back to an example like I said, with the lack of input sanitization. We can find hard-coded credentials. We might find memory problems with that static testing. So that's looking at poor coding practices introducing risk. Now what we want to do after is generate that software bill of materials. That's that software composition analysis. So what is the software composed of? What builds what goes into building this software? What third-party components are used? We have to cover questions like that. So, we're generating this bill of materials. That's a list of everything in the device. And then we say, are there any risks in that bill of materials? So, it's a little bit separate from looking at what are the problems directly in the device. It's more looking at what are the problems with components you're including in the device. And this can go down deep as many layers of recursion as you want for any one of those components in the device. What components are they using? What risk is involved there? As you can imagine, the further down this chain you get, the less likely those vulnerabilities are going to make their way into the final system, but it's still something that you want to look at. And this is also with the software composition analysis where, is a sort of a subset of the SBOM, if it's not an identified third party, if it's software that came from somewhere that we don't know, that would be the SOUP, or the software of unknown provenance. And there's often risk associated with that. As we've discovered, a lot of developers will go to some form and just copy software with hard-coded passwords and put it directly into the source code. Stack Overflow has been the downfall of many an engineer. Stack Overflow is the most common form. Yep. Yeah, I've seen. We've seen on tests before vulnerabilities from breached credentials or hard-coded credentials somewhere. And then we're able to actively exploit it and decrypt sensitive information, passwords, PHI, lots of really, really nasty stuff. And then we look down to the source code to see what's actually going on there. And then you search for that source code, and sure enough, you find it verbatim on a Stack Overflow paste or post from 15 years ago, and then someone leaves the hard-coded credential in there. It then gets decrypted, and so it's more common than you'd think really. Engineers don't want to have to reinvent the wheel if there's code out there for it. That's why software libraries exist. That's why these alternative solutions exist. You shouldn't have to write everything from scratch, but you make sure that what you're using to fill this need is safe and secure and you aren't introducing insecure code into your product. Yeah, that's kind of interesting because technically, I don't know if that would be SOUP, because we know, we know where it came from. It's from, came from Stack Overflow. Yeah, I guess that's fair. Now that that opens up an interesting debate where how can you determine, well, first off, if you have no idea where the code came from, how did the engineers find it? Maybe that's, you know, or ChatGPT. Probably it could. Now, I guess now today, a lot of code could come from ChatGPT. Just try to write your code for you. Yeah. And ChatGPT is pulling it from who knows where. From Stack Overflow. Probably. From Stack Overflow. Yeah. All right. So, we talked about software composition analysis, which is the SBOM, the SOUP, and we talked about static code analysis as part of vulnerability testing. We touched upon penetration testing, which is exploiting the vulnerabilities. And there's quite a few subcategories of pen testing. Let's just zoom out a little bit and talk about black, gray, and white box penetration testing because typically with a medical device, we do white box penetration testing, which means we have access to the source code, documentation, the developers if necessary. You want to explain the three, the differences between those three real briefly. So these are talking about different levels of access. With black box testing, you're acting as an attacker would. No insider information, going in completely blind. There's definitely merit to each approach. Moving into gray box testing is somewhere in between. Oftentimes, this will be a set of credentials to test an authenticated perspective, but not really that much more information, or maybe just a high-level overview of how the application works or the product works. And then white box testing, you're fully looking under the hood. You have full access to source code, credentials, architecture, documentation, software requirements, like you mentioned. You can talk to engineers, say, "Tell me exactly how this function works. Show me what the data flow is in the code." So, that's really scrubbing in and trying to find everything that you can, which is our preferred approach when we're looking at this FDA process, since it leaves very little room for anything to be missed. Right. The white box is the most comprehensive, but the FDA asks for closed box as well, which is equivalent to black box. Yep. And there's certainly merit to the black box approach as well. And that's usually going to be a more authentic test, I guess you could put it as, since that's taking the approach that most actual hackers are going to be taking. They aren't going to have this insider information that we have in an ideal situation. They're looking at it from the outside in. So if you're getting a black box penetration test done, I think this is especially valuable for network testing. You have to go in completely blind. Try to get any bit of information. You can do your own reconnaissance. Try to find your own credentials and work your way into the system. For a medical device, it's great to have this approach, but you're usually going to see a lot more value going all the way in with that white box testing, which is why we like to have the full start to finish coverage. And I think it's important to understand, a lot of people with penetration testing and just testing in general, cybersecurity testing, they just assume we don't have to test certain things. Like a lot of our clients or prospects will say, "Well, our USB port is only used for our maintenance crew, our maintenance updates, right, by our maintenance staff." So, they assume that that's out of scope, but if it's exposed on the device, we have to test that, right? Or the another one is saying, "Oh, it's only for updates." You say, "Well, is there no way we can interact with data? Guess we've never really, or not for updates, it's only for charging." Well, there's, is there any way we can interact with data? Guess we don't really know. We just use it for charging and then you try and sure enough, you can, you know, download something onto it or interact with the OS underneath. And so using this will go back into what the FDA wants to look at for the vulnerability testing, which again, that line gets really blurred really quickly. They want to see abuse cases. And so the use case, what you should do under normal operation. The abuse case is, you know, well, you say it's only for testing, but what if I plug a hotkey into it and see what happens? What if I, you have an HDMI port and using it to connect to a third-party monitor. What if I see if the CEC protocol is enabled and I can shut the device off while it's operating through the HDMI port. And so they're the abuse case is essentially what shouldn't you do to the device, which is probably the first thing an attacker is going to try to do is whatever you're not supposed to do, whatever you can do to make something go wrong. So that's when it gets really important to get super creative with being a penetration tester and think what is everything that we could do here. Yeah. So, a scope of a device. I know in a previous episode you talked about the three-piece suit, which is a lot of devices have medical devices have a device physical, which has interfaces, some sort of app on a mobile device, and then a cloud component. So, all of those would need to be tested. They're all in scope for vulnerabilities and pin tested and everything else, including APIs that the mobile app may need to or uses to connect to the cloud app. And I think a lot of people, a lot of manufacturers don't understand like every single thing that makes up your system if there's an entry point into it has to be tested from a vulnerability and penetration testing perspective. Right. Yeah. And that'll go to trying to say something's out of scope. There are ways to consider an interface or port out of scope. If you bury it deep inside the device, you have to essentially shred it apart to get to it. Yeah. The evidence that someone shredded into the device is going to be enough to say this has been compromised. Don't use it. As soon as you have something on the outside, it's very hard to just write it off. There are a lot of things you can get around. You can do a physical port lock. You can do tamper evidence seals on it, but anything on the outside needs to be covered. And so it's kind of interesting. Sometimes we'll have trying to understand the scope of a device. We say we need to go through every single connection, every single data transfer, anything that can do anything on your device. Can do you have HDMI? Do you have Wi-Fi? Do you have Bluetooth? Stuff you might not even think of. Do you have an NFC tag? Anything like that on your device? Manufacturers go, "No, no, no. We only have USB." And then we get the device and we go, "This has an NFC tag on it. What's that doing?" "Oh, we didn't think that was relevant for cybersecurity concerns." Go, "Well, it is. There are things you can do to the device with this tag. We can rewrite the tag. We can say that single-use disposable hasn't been used and then introduce sterility and sanitary concerns." There's a lot of risk to that. And so it's, I think, from a cybersecurity perspective, when all you do is live and breathe this stuff, you think, "Oh, everything can go wrong with anything all the time." And that's not, that's not a normal mindset that a lot of people have. And so they don't really think,