Episode 38 · December 16, 2025 · 19m listen · 3,632 words · ~18 min read
The Differences Between Black, Gray, and White Penetration Testing | Ep. 50 - Full Transcript | The Med Device Cyber Podcast
Read the complete, searchable transcript of Episode 38 of The Med Device Cyber Podcast - expert conversations on medical device cybersecurity, FDA premarket and postmarket guidance, SBOM management, threat modeling, and penetration testing.
Prefer the listening experience? Open the episode page for the synopsis, key takeaways, topics, and Apple / YouTube listen links.
Episode summary
In this episode of the Med Device Cyber Podcast, hosts Christian Espinosa and Trevor Slattery from Blue Goat Cyber delve into the critical topic of penetration testing for medical devices. The discussion centers on clarifying the distinctions between the three primary methodologies: black box, gray box, and white box testing. Also known as ethical hacking, penetration testing is a vital component of medical device cybersecurity, and the hosts explain why understanding the differences is crucial for manufacturers seeking regulatory approval from bodies like the U.S. Food and Drug Administration (FDA).
The episode breaks down each testing type based on the level of information provided to the security tester. Black box testing is presented as a scenario where the tester has no prior knowledge of the device's internal workings, simulating an external attacker who might stumble upon the device. This approach is realistic for opportunistic threats but is the least comprehensive. Gray box testing represents a middle ground, where the tester is given partial information, such as user-level credentials or high-level architecture diagrams, mimicking an attacker with some insider knowledge. Finally, white box testing is described as the most thorough and in-depth approach. In this scenario, the testers are granted full access to all relevant materials, including source code, detailed documentation, and direct communication with software developers, giving them complete visibility into the system.
The core argument of the episode is geared towards medical device manufacturers navigating the regulatory landscape. While the FDA and other global bodies may not explicitly mandate a specific type of penetration test, they require a justification for the chosen methodology and often reject submissions due to 'insufficient' testing. Espinosa and Slattery strongly advocate for a white box approach, presenting it as the most reliable way to ensure due diligence and satisfy regulatory expectations. They caution that opting for a cheaper, less comprehensive black box test often proves to be a false economy. Such tests risk missing critical vulnerabilities, leading to regulatory rejections, costly delays in getting to market, and the eventual need to conduct a more thorough test anyway. They use the adage 'buy once, cry once' to emphasize that investing in a comprehensive white box test from the outset is the most efficient and effective strategy for ensuring both regulatory compliance and patient safety.
Key takeaways from this episode
Penetration testing for medical devices is categorized into three types: black, gray, and white box, which differ based on the level of information provided to the tester.
Black box testing simulates an external attacker with zero prior knowledge, offering a realistic but less comprehensive security assessment.
Gray box testing is a hybrid approach where the tester has some limited knowledge, such as user credentials, to simulate an attack from a privileged user or insider.
White box testing is the most thorough method, giving the tester full access to source code, documentation, and developers to find vulnerabilities at the deepest level.
While the FDA doesn't mandate a specific type, it often rejects submissions for 'insufficient' testing, which can happen with less comprehensive black or gray box approaches.
For regulatory submissions, white box testing is highly recommended as it provides the most complete and defensible evidence of due diligence and security robustness.
Choosing a less comprehensive test to save costs upfront can lead to expensive delays, resubmissions, and the need for more testing later, making the 'buy once, cry once' principle applicable.
The goal of penetration testing in the medical device context is not just to check a box, but to ensure the device is secure and patient safety is protected, which a white box approach best supports.
Full episode transcript
Page 1 of 5· Paragraphs 1 - 14
Christian: Hi, welcome back to another episode of the Med device Cyber podcast. Today we're today we're talking about penetration testing specifically what are the differences in black, gray and white penetration testing. Penetration testing is also known as ethical hacking.
Christian: And we're talking about this in the context of medical device cyber security and what the FDA and other regulatory bodies are really looking for because sometimes black is not enough, gray is not enough. White might be the preferred, but we'll dig into that topic here in a second.
Christian: I've got these cool uh glasses. They don't look so cool on screen, but they Trevor claims I will sleep like a baby tonight because I'm wearing these glasses even though I haven't slept in three days because I just got back from Singapore and leaving for Europe tomorrow. So I feel like I have permanent jet lag.
Trevor: Well, they won't help with that, but they will make it easier to fall asleep when you're staring at a screen for 14 hours a day as we typically do.
Christian: So we'll start with black. Black the device is like a black box. We don't know anything about it. We don't have any really not much documentation other than maybe a user manual. We don't have a lot of visibility into it. We can't talk to the software developers. We have user level access. Is that a good explanation for Black Box penetration testing?
Trevor: Exactly. It would be thought of as an attacker walks into a room, they want to cause some damage to something, but they don't have any prior insight into whatever it is. They see a device sitting on a table, they just grab it and try to hack into it.
Trevor: So, that's the perspective that we're coming into a black box penetration test from. It's a little bit difficult at times as, you know, penetration testers, of course, we're doing this differently from actual bad guys. We are contracted to do it. People are willingly paying and asking us to hack into their products. So, they know we're doing it, but they still have to try to keep as many secrets from us as possible from when we're trying to understand what are we testing, what are we allowed to do, what aren't we? They have to set some guy some guidelines on that without giving up too much information.
Trevor: So it can be a little bit interesting and sometimes it's funny too, working with clients on these engagements for a Black Box test. We'll say, oh well, could you explain what this process looks like and they go, no, you have to figure it out yourself. So it's a little bit of a more exploratory type of testing.
Trevor: It is also going to be the most realistic from like a grab and go attacker. So if you're looking at what is typical for malicious hackers, they're trying to look for low-hanging fruit, they're trying to grab on to the first thing that they can see that they think they can hack into. That is most indicative of a black box testing approach. The first thing that someone can see that they can try to attack without any prior knowledge.
Trevor: As opposed to this, sometimes a bit of a misconception where attackers are doing a ton of background research, really trying to find a way in, try to really focus on a single target. Usually they're more looking at it from just grab and go. So a little bit less depth of the testing, but a bit more of a realistic scenario.
Christian: Grab and go. I haven't heard that term before, but basically you're saying if I'm a hacker in my home, I've got my whatever that TV is in the room. Samsung I think it is. If I try to hack in that TV, that's a black box type of penetration test, right?
Trevor: Exactly. Now, if you had let's say the password to the admin settings on the TV. Or if you knew about all of the parts inside the TV that builds it out or going further past that, you had access to the actual source code that's running on the processors within the TV, then you're no longer coming in from that black box outside perspective. So that's where you'll see a little bit of that difference.
Christian: So let's bring a better a better example. I have this uh iHealth um measures your pulse and your uh heart rate and all that stuff. your um blood pressure. Because I therefore I my blood pressure is a little high because Trevor was stressing me out about stuff. It's it's better now.
Christian: So, but yeah, if I this has Bluetooth connection to my cell phone though, if I were to try to hack into this, uh, I only have the user manual that's black hat, right?