Samir Boussarhane on New MITRE Caldera for OT Attack Simulators

Samir Boussarhane, senior cybersecurity engineer at MITRE, joins the Nexus Podcast to discuss some new simulator plug-ins added to Caldera for OT. Caldera for OT is an open-source adversary emulation platform that automates security assessments for operational technology (OT) systems. 

Samir provides context on a new simulator called the Aloha Water Treatment plant, which emulates a water utility and serves as a training platform for students, engineers, and IT security teams alike. Caldera for OT now also supports protocols such as BACnet, Modbus, and includes an HVAC simulator.

Subscribe and listen to the Nexus Podcast here. 

Access the Aloha Water Treatment simulator.  

Medium article on the Aloha Water Treatment simulator.

Transcript

SPEAKER_01 0:14

Alright, welcome back to the next podcast. Tamir Boostarin of Mitre is my guest, and we're going to talk about Caldera for OT. Caldera for OT is an open source adversary emulation platform designed to test security defenses in OT environments. There have been some recent updates for the platform, and that's we're going to spend some time talking about. Before we jump into the episode, though, I hope you've been enjoying the great guests we've had on since the start of the year. The best way to keep up with the show is really just to subscribe naturally. And we are on every major platform, whether it's Apple, Spotify, Audible, Amazon, you name it. Pretty easy to find out there. So if you have not subscribed yet, please do. It really helps out the show. So let's get started. Bring in Samir. How are you doing, man? Good to see you. Doing good and happy to be here. Thanks for taking the time. I appreciate it. Before we jump in, kind of maybe introduce yourself a little bit to the listeners and maybe talk about your role with uh within MITRE and Caldera for OT specifically.

SPEAKER_00 1:17

Sure. So Center Booster Han. Um been with MITRE for about three or four years, supporting the Caldera team and a lot of the OT research that we're developing. Some part of my role is for the simulators we're talking about today, led development for that. We have the Aloha Water Treatment Simulator that's a water treatment plant supporting Modbus and BACnet. And then more recently, we had a HVAC simulator for BACnet for building automation controls that came out and involved with leading research at the University of Hawaii to develop those.

SPEAKER_01 1:49

It's pretty cool stuff. I mean, these there's been a lot of talk. I know I was at the S4 conference, and I know that there were a lot of discussions about these uh attack emulation platforms and how they're really gaining some prominence among uh some of the OT folks. So pretty uh pretty timely research and and projects on your end, I'm sure you're hearing the same kind of feedback.

SPEAKER_00 2:11

Yeah, and especially with the recent uh SISA directory targeting OT devices, feels very timely.

SPEAKER_01 2:18

Yeah, for sure. Um before we get into kind of the new simulator and and some of those updates, maybe you could cover just exactly for some listeners that might not be familiar, explain exactly what Caldera for OT is, who uses it, how they use it. Just take me in from a high level what what it's all about.

SPEAKER_00 2:36

Sure. So Caldera for OT is a plugin and extension for Caldera. Caldera is like an open source adversary emulation platform that MITRE helps develop. Uh with that, anyone can download this off to GitHub and you set up your server, you have agents, and you can deploy attacks, the payloads, the adversary abilities to TTPs, map to attack are all there, available open source, and you can customize, you can add your own abilities. Open source, and we'd love to see if you can get anybody to contribute and commit back to MITRE. The OT plugin has for like when you're looking at Caldera for Caldera in general, it's more IT-based. OT we're extending that to have your protocol-specific payloads or adversary inlation capabilities. So there's plugins right now for BACNET, Modbus, DMP3, Profinet, and all of those are very, very specific as opposed to like IT, where we have a lot more coverage. So we're trying to make it as easy as possible. If you want to run adversary capabilities for defenders, red teamers, purple teamers, you could just download these payloads, execute them against your OT infrastructure. If you don't have OT infrastructure, we're trying to make these simulators so you can still test your detections and learn adversary like tradecraft OT space.

SPEAKER_01 3:46

And so what are the inputs, so to speak, are or the threat Intel sources? Are you guys specifically sticking to miter attack for ICS, for example, or um are what are some of the other input sources?

SPEAKER_00 3:59

Right. So for the OT space, uh the Intel doesn't always necessarily get to us simply like, hey, adversary X ran this TTP or this protocol function. So we try to have as much coverage as we can within the protocol and then map to attack for ICS as far as the impact. So for like an OT process, we want to be able to read and write to affect the process. We can map that to attack and make sure for our abilities that you're able to modify memory on the OT device, and then from there you can say, alright, if I was an adversary, I'd want to change this to turn on this valve. That's more of the like threat informed, uh, based on how you can impact, not necessarily like a one-for-one command line that here's what the adversary ran. Just make sure we have the ability to perform manipulations or read device configurations within the protocol.

SPEAKER_01 4:48

What's a typical or who is a typical user for this platform?

SPEAKER_00 4:53

Right, really anyone. So one great application of Caldera that I've been doing in RCY is education and training. Uh, I didn't personally learn the OT when I was in undergrad or graduate school, and it's not necessarily covered in all curriculums. So being able to put those in payloads, put them in a pretty package, put them in Python scripts that you can just download, have students able to click a button, see their adversary emulation happen, see the outputs. It's great for training and exercise. Like if you're trying to do a large-scale exercise and automate this, Kedar is a great tool for that.

SPEAKER_01 5:27

So it's not necessarily for a production environment. This is kind of an educational tool or just a just kind of uh a training mechanism, right?

SPEAKER_00 5:36

The payloads and abilities are like uh modular, so you can take them out in your production environment. It's not necessarily best suited for that, but uh it's like a server and agent architecture, so you could deploy it wherever you want. But very, very useful tool for training and new users of OT tools.

SPEAKER_01 5:53

Have you heard of any unique use cases that maybe you guys hadn't considered or I have not yet, but simulator's new, and if anyone has any unique use cases, happy to learn about them. And is this the only platform, uh one of these attack emulation platforms that's specific for OT that you're aware of, or are there others? As far as free and open source, this is the only one I'm currently aware of. What are the kind of the drivers, the need for this for such a platform? Is it, you know, like I know there's always a concern about getting your hands on actual PLCs to do testing, you don't want to touch your production environment. Um, does that alleviate that concern? What are some of the drivers for this?

SPEAKER_00 6:33

Right. That was the main driver with the simulators that we developed. We were going out to exercise and show our new OT capabilities to where we had plugins like, all right, here's some updates to the Modbus plugins so you can run this against your Modbus PLCs, updates to the backnet plugin so you can up the and run these against your backnet PLCs. As we start doing more workshops, we realized that not necessarily everyone had PLCs, and those can be very expensive. If you ever go on eBay, you can see them go from hundreds of dollars to thousands of dollars very quickly. So we wanted to make those simulators to where now we can lower the barrier of entry into OT security and have free open source Python scripts that will behave as a realistic simulator and like PLC. It's not gonna get into those vendor-specific functions that PLCs would have uh depending on the vendor. But general reading and writing and memory operations will get to be able to cover that. And we're able to see that go across networks so we can build better detections.

SPEAKER_01 7:31

So as the user is running this, what are what are they seeing? I guess what are the outputs that they would expect to see? Are they seeing an at like a simulated environment and values changing, or what what's the what's the output that they see?

SPEAKER_00 7:44

Yeah, so for Aloha water treatment plant and HVAC sim, there's Python scripts you run and they're just servers. So with that servers, there is like a graphical interface to you can change the values. So for Aloha, you can change the inflow rate, the outflow rate, and then in the visualization, you see how the process is affected. Now you can use those as an endpoint for any adversary like tool you have that's OT specific. So I could take my cutter for OT, I could take a custom-made like Modbus client and perform rights against that, and then I will see on the process how that changes. Really serving as an endpoint to where now I can see just using OT-specific functions, protocol communication, how my processes changed. And then if you have like Wireshark in the background or whatever you want to use for network traffic analysis, we can see how these different uh protocol commands are sent across network, how responses are done, how discovery functions are working. So instead of just having a like adversary emulation platform send out these commands and their responses, now we can see like more realistic behaviors.

SPEAKER_01 8:46

And can users feed like their own samples of their own traffic to the platform, or is it kind of generic?

SPEAKER_00 8:54

Yeah, so when we say traffic, uh we're not actually like um recording traffic, it's more of a the server sends like endpoint and it'll take any responses. So you could just throw random responses at it and see how that breaks it. But it's just like giving realistic responses back to it based on like protocol specification. So any user could take any like client they want and just attack it and test. But we're not like uh encounter itself inputting traffic, it's more of a the simulator's uh responding to anything it gets. Got it. Okay.

SPEAKER_01 9:23

All right, so let's talk about the Aloha water treatment plant. Um, basically, what was kind of the the driver for this one? It mimics a a water storage tank, correct?

SPEAKER_00 9:33

Yep, it's a water treatment plant where very simple, it has your intake for your water, it goes through a filter in the center, and then outtake. Then you can manipulate those two. Um then there's an auto mode that tries to keep it around two-thirds full, the manual mode where you can, wherever you set the inflow and outtake, it overhauls that for the physics. Uh, that was really just designed as a effort with the University of Hawaii where we're giving OT lectures, and uh it's very easy to explain, like, hey, here's how this works, or the PLC right in front of you. But if you want to go home and learn more about OT security and you don't have one of those expensive PLCs, that was a big barrier that we were getting responses saying, like, oh, this is really cool, but I don't have the hundreds of dollars to spend on a PLC randomly. So that's what we developed that for, and then students were able to learn at home and then see the nice like GUI visualization of how their different commands reflect and process.

SPEAKER_01 10:28

As it runs, as it works, what can it tell a defender? I mean, what does the platform tell users kind of like about different protocol activity, vulnerabilities, dependencies, etc.?

SPEAKER_00 10:39

Right. So on the platform itself, the visualization will warn you if you're getting to what the process defines as a like critical or uh warning systems like if emergency stop activates the HMI representation, like we'll flash like a warning, or if you're about to overflow, it'll warn for that or underflow. As far as like network traffic, um it's not flagging suspicious traffic. But one area that we'd like to look to like recommend users look into is have Wireshark open while you're doing this so you can recognize these like packets. So if we go to like the attack for ICS, we know that there are functions and protocols that we want to be more aware of for our defenders. Like we want to know anything that can impact the process, reads and writes. We want to make sure that we're able to see that across network. So go through the cutter for OT, uh, look at how the abilities are mapped to attack for ICS, and you have your high priority abilities that you want to be able to defend against. Look at that traffic, and then from there you can learn how you detect that going across the network.

SPEAKER_01 11:46

Have you heard from folks in you know water utilities, wherever they may be? Are they using it or is it strictly from students right now?

SPEAKER_00 11:55

I have not. It's been mostly students and people trying to get into OT space right now.

SPEAKER_01 11:59

And what do you think are the the biggest benefits of this kind of platform for for a student, for example, or from anyone trying to get into OT? Like, what does it help visualize or what does it help make clearer, I guess?

SPEAKER_00 12:11

Yeah, like uh at least my experience when I was starting to learn about OT, it felt like very complicated and hard to wrap your head around, especially if you're not familiar with ladder logic or these specific protocols that are all can vary vastly. Having all that open source, free, and well-documented resources that MITRE is producing, I think is a great learning point. Uh like every ability that we have, we document like the inputs for the payload, the outputs for the payload, how this affects the protocol, what the um resources and like actual specification in the protocol would be. It's a great resource and well-documented way to get into OT. And then the visualization makes it very easy to just run some Python scripts and actually see, okay, so I'm doing these reads and writes against memory. Here's how that would actually affect the realistic process.

SPEAKER_01 13:01

So you have different simulators uh with or I'm sorry, you have the same simulator but with a modbus and a backnet plugin. Is that correct?

SPEAKER_00 13:10

Right. So for Aloha, um it's the same backend. We just wrote it in two different libraries. Uh PyModbus and open source modbus library, back zero and open source uh backnet library. Same process, but you can run it in these different versions to have it respond against those different protocols so you can do your testing. Then there's uh BACNET SIM, which is more recent. It's like a fan system that's in BACnet. And same thing where you have different process now, like going from water treatment plans to building automation control systems, to where you can just run it on Python scripts, run your adversary behaviors against it, and get realistic responses back.

SPEAKER_01 13:47

And are there different attack scenarios for each of these uh different plugins? I mean, what kind of attack scenarios is there are is it mimicking or simulating?

SPEAKER_00 13:56

Right. So for uh we have medium blogs that we released with both of the simulators that go through different attacks uh scenarios and has graphics on what you should expect as far as you like walk through it. All these hack scenarios really are how can you impact the process? Can you stop the water flow? Can you stop the fans and what you would need to do to do that? So encourage anyone to go through the medium articles and we walk through the attack like vectors and what you need to do and what you'd expect to get out of it.

SPEAKER_01 14:27

Yeah, we can link to those in the in the show notes for sure. Um do you see this as useful uh perhaps as a red teaming exercise or as part of an overall risk assessment, for example, and and why or why not?

SPEAKER_00 14:42

Yeah, absolutely. I think uh as you like you mentioned earlier, people get very wary when it comes to testing any sort of red teaming in the production environment. I think it serves as a very good starting point as a stand-in to where we know it's gonna respond to the protocols for our system. We know that we need to be able to detect these before they happen. Let's take that over here, build our little sandbox environment, and do our red teaming, and then ask whether are we be able to protect this in our actual infrastructure? I think it's a good stand-in to try to avoid all those frictions that come with trying to red team production.

SPEAKER_01 15:18

Um, and just a question about the inclusion of BACnet. Obviously, it's a building management system protocol, pretty well known and well pretty popular. What what drove you towards including that as a plug-in? Um, I know we're hearing about a lot of BMS being connected online. Is that kind of the main driver for that or is it something else?

SPEAKER_00 15:38

That and um a few years ago, the large target hack that happened through the building automation system. So we felt that was like a relevant protocol to make sure we discussed. Uh BACNET also has a lot of interesting discovery capabilities that I think are good for new learners to learn. So for one, it's the Whois command where if you backnet device, you run the Whois message. Any BACnet device on the network responds. So it's very easy to begin in the network, recon, figure out where your backend devices are. Then you can use the Caldera for OT ability epics report pointed at those devices, and they'll give you back all their information, all their objects in memory. So it's very easy to go as far as like a adversary emulation plan and storyline in BACnet. And it's one of the more beginner-friendly OT protocols, so want to make sure we include coverage for that.

SPEAKER_01 16:28

And how would you characterize the overall security of BACnet as a protocol?

SPEAKER_00 16:33

Um, difficult question since it really depends on the implementation. Some people have BACnet that uses encryption and authentication. I would say that's not as common as it should be. But as far as protocol, it's very easy to learn more information and do your discovery against endpoints. Modbus, uh, the other protocol that Aloha is written in, you kind of have to know more about the design of the system that you're reading against, just because it doesn't give you so much information. Backnet's great for an adversary standpoint, but difficult for the defenders since you're able to give up so much information.

SPEAKER_01 17:08

And I mean, Modbus isn't necessarily secure by default either. I mean, there are add-ons for security for that protocol as well, right? Right. And again, in terms of did this the inclusion of the BACnet uh plugin here, did that set the stage for the HVAC sim? Tell me about a little bit about that simulator.

SPEAKER_00 17:27

Right. So for with that, uh the modbus and backnet plugins were pretty much our very much our most old documents plugins. So we wanted to make sure to include for that. And in our workshops, GDOH, that's the two protocols we were really briefing on. And then water treatment's not necessarily a backnet civic process, so we wanted to make something more realistic as far as like building automations. And that simulator was actually developed by a student group at the University of Hawaii at Manoa. So for their semester research project, they developed that simulator, tested it, and we were able to release it. So the students did a great job with that, and we can show that everyone in that group actually had no background in OT at all. Semester we were able to develop something new, test it, and have a road product come out.

SPEAKER_01 18:13

Just for Caldera for OT in in general, is there a best practice in terms of you know how often do I run this? Daily, quarterly, continuously? I mean, what is there a recommendation or what do you find most useful?

SPEAKER_00 18:26

Yeah, so as it stands right now, it's not necessarily a like run this every day to elicit like a detection. It's more of a here's how we can exercise, here's how we can train, here's how we can test easily. Um I wouldn't recommend running like reads and writes the instruction equipment every day or anything like that, but if you're wanting to sandbox detections and depending on how often you want to do that, I think it's a great tool for that.

SPEAKER_01 18:52

And do you hear anything about it something like this being used for um IT people that are kind of new to OT or now have to manage and secure OT. Um, have you heard anything anecdotally like that that people on the IT side are kind of you know moving over to check out these kinds of platforms and see what they can do with them?

SPEAKER_00 19:15

Yeah, we we have plenty of stories like that where we have different organizations that exercise their IT red teaming, and now with more recent uh news and CTI coming out about OT systems being a critical thing that we need to protect, defenders and attackers looking for those OT capabilities and happy to point them towards Kedder for OT. But definitely over the years, there's been more of a shift towards critical infrastructure and OTICS SCADA and wanting to protect that.

SPEAKER_01 19:45

So I I did want to, before we wrap up, talk a little bit about the the attacks that have been um connected and linked to Iran against the Rockwell PLCs. Um, how much do you see something like that elevating uh or prioritizing the security of these platforms? Um it's not often you see real-world attacks targeting PLCs, for example, um, and these vulnerabilities are pretty old. Like just uh curious as to your reaction when you heard that news.

SPEAKER_00 20:17

Yeah, definitely interesting. Um I'm not up to date on the exact specifics, but I'm tracking that it was a certain model of Alan Bradley PLCs that had a vulnerability, and they were searching for that model. Am I correct on that?

SPEAKER_01 20:29

Yes, correct. Yeah. I think it's pretty old vulnerability. I think it was 2021. Um and it was pretty easy to access um the PLCs and and manipulate from there.

SPEAKER_00 20:45

Yeah, and then uh kind of alluded to it earlier. So with the OT simulators and Calder for OT plugins, we're using the protocols that are common within like the protocol specification, so like Modbus to be a compliant modbus device, you can support modbus reads, modbus writes. A lot of those adversary behaviors that like we're talking about with Iran and the micro A50s, that's even outside of those functions. So it's very important to when we're trying to protect OT to think of what's already built in and native, but also like what adversary could do through the vendor space. Because that's not nearly as well documented. And we really want to be able to collect traffic that's going towards OT systems at all. And I know in IT that we say, all right, if you want me to close the firewall, I need to know everything because so much traffic goes through. In OT space, it really should be the opposite. So we have to be better as defenders to flag any specific traffic in like uh OT space to defend against it. But as far as like as far as like the Iran vulnerabilities, I think it's great to bring attention to the critical infrastructure. May not be as well protected as we think, and want to make sure that we're flagging these issues before anything bad could happen.

SPEAKER_01 21:57

I mean the whole vulnerability in a Exposure management question within OT is just it it it doesn't seem like it's ever gonna go away in terms of patching, in terms of I mean, I don't think this particular vulnerability can be fixed without a major overhaul from the vendor, regardless. But um there's there's long windows of exposures with some of these vulnerabilities, unfortunately. Um all right, so we wrap up um just what is next for Caldera for OT? And are you expecting more of these plugins or more protocol support? Tell me what's kind of on the drawing board.

SPEAKER_00 22:31

Yeah, so with that uh University of Hawaii Capstone group that produced HVAXM last semester, we're also sponsoring another team this year to do DMP3 for electric substation. That development's underway and hoping to release that later this year. More updates, continued updates, and like improving our documentation and our protocols. Be able to announce more of that later in the year, but still trying to improve security posture and lower that barrier of entry.

SPEAKER_01 22:58

It's very cool stuff. It must be fun to be on kind of the leading edge of this kind of development.

SPEAKER_00 23:03

Yeah, definitely fun research to be part of.

SPEAKER_01 23:05

All right, Samir, thank you so much for joining the podcast. I really appreciate it. Thanks for having me. All right, take care.