There is an old saying about not forcing a square peg into a round hole. The square peg is IT and Operational Technology (OT) network security. The round hole is the insecure Industrial Control System (ICS) field device. Square peg issues such as Common Vulnerabilities and Exposures (CVEs) and zero trust apply to any Internet Protocol (IP) network, whether it’s IT or OT, but they don’t apply to ICS field devices. Round hole issues are engineering and physics issues like common cause failures and the Aurora vulnerability.
September 11, 2022 (9/11), Top Cyber News issued their September issue -https://www.linkedin.com/feed/update/urn:li:activity:6974854180238729216/. I have several articles in the magazine all focused on the round hole of ICS cyber security.
September 8, 2022 RSA held the RSAC 365 Virtual Seminar & Innovation Showcase: OT & ICS Security. The session was focused on the square peg of IP network cyber security. The speakers were experts in their fields, but their expertise did not include industrial systems or cyber security of ICS field devices. For example, Jake Steele from MITRE was speaking on the MITRE Attack tool when he was asked a question as to the order of OT and ICS. His response, which is what every speaker on the session essentially echoed, was that OT was the top level and ICS was a subset. This stems from the lack of an adequate definition of OT. Operations would encompass both OT networks and ICS field devices. However, the term OT generally just addresses the OT networks (as discussed throughout the RSA session). ICS devices are not subservient to OT as ICS devices are needed to monitor and control the actual physical processes. Without the ICS devices working properly, facilities cannot operate reliably and safely whereas facilities can operate without the IP networks as demonstrated by the recovery from the 2015 Ukrainian power grid cyber attack.
Definitions
There are a number of definitions that are not shared or understood across the greater engineering and cyber security communities which makes it difficult, if not impossible, to have the divergent teams working together.
Cybernetics is the science of communications and automatic control systems in both machines and living things. Today, cybernetics has been transformed into the term “cyber” and in many people’s eyes it no longer addresses the physical nature of devices that control physics.
The U.S. Government Accountability Office (GAO) in GAO-21-477 defines a cyber incident as “an event that jeopardizes the cybersecurity of an information system or the information the system processes, stores, or transmits; or an event that violates security policies, procedures, or acceptable use policies, whether resulting from malicious activity or not. Cyber incidents, including cyberattacks, can damage information technology assets, create losses related to business disruption and theft, release sensitive information, and expose entities to liability from customers, suppliers, employees, and shareholders.” The incidents being discussed affect integrity and availability.
A network is anything that allows communication between people and systems. Networks can be verbal, paper, serial, or IP. In operational environments, all of these networks are used. The most popular process sensor networks are serial networks. However, to the IT and OT security communities, networks are only the IP networks such as Ethernet.
OT is generally referred to as anything that is not IT. However, in most usage including the September 8th RSA session, OT are the IP networks.
The gap between OT and engineering
This is the “ForeWord” to my articles in Top Cyber News:
“IT and OT cyber security focuses on the Internet Protocol networks and are under the purview of the CISO. Control system field devices such as process sensors are used for reliability, safety, predictive maintenance, and cyber security. Control system cyber security focuses on the field devices such process sensors and their associated lower-level networks which are often serial.
These field devices have no cyber security and are under the purview of engineering. Protecting these field devices is different from protecting IT or OT networks and requires different technologies and training. When control systems are impacted, the results are obvious – trains or planes crash, pipelines rupture, power is lost. Because of the lack of control system cyber forensics and training, these incidents are generally not identified as being cyber-related. Yet, to date, there have been more than 11 million control system cyber incidents with more than 34,000 deaths. There is a need to address this cyber security gap in technology, training, and culture.”
The September 8th RSA session demonstrated the gap that continues to exist between OT network security practitioners and the engineering community – the square peg in the round hole.
Connecting the dots
As Top Cyber News was issued on 9/11, it reminded me that one of the issues with 9/11 was a failure to connect the dots. Unfortunately, dots are still not being connected in control system cyber security: incidents continue to occur in all sectors that are not shared within the sector or between sectors. That is obvious from my database where the same types of incidents occur within multiple sectors. The focus on IT and OT also limits recognizing cyber incidents that didn’t involve IP networks as being identified as being cyber-related. This was obvious from the RSA session.
Education
Cyber security is taught as a subdiscipline of computer science. There are very few universities that require an introduction to engineering for cyber security. Conversely, there are very few universities that require the engineering disciplines of electrical, mechanical, chemical, nuclear, or systems engineering to include an introduction to cyber security. This past year, I was a senior research associate at the University of Missouri Science and Technology. The course I supported required a capstone project to take an engineering/utility company and determine how well the student felt it met the NIST Cyber Security Framework. The students were neither engineers nor from the utility industry, and they could only use publicly available data for their projects. They found issues that weren’t identified by the utilities’ cyber security organizations – appropriate training can work. I also talked to a utility senior manager who was teaching a cyber security course at a university and felt it was too complex to even mention control systems. Unfortunately, this is the norm and it’s why there is such an education gap. The process sensor issues are not being addressed even with the work being done by CISA and ISA on OT cyber security training.
Lack of understanding of systems and components
The fundamental approach for offensive cyber operators when they attack an industrial or manufacturing system is to identify the impact they want to achieve and then study the systems to find out the best way to accomplish the goal. That is, they want access to specific pumps, motors, valves, relays, etc. to accomplish their goal. Accomplishing that goal may involve a combination of physical, IT, OT, and control system cyber approaches. Additionally, offensive cyber operators may use the IP networks as part of their attack technique using approaches such as man-in-the middle attacks to provide the operators with misleading information. Often, the cyber approaches may be very basic as the control systems often are not designed to keep cyber attackers out. Consequently, state-of-the-art zero days are not needed. Process sensors are 100% trusted and are the input to OT monitoring systems that cannot detect or correct sensor data. Consequently, Isiah Jones mentioned in his September 9, 2022 blog: “using configuration compliance checker, calibration, maintenance and programming utilities and components for offensive purposes path of least resistance. most of that stuff doesn’t have any ISA/IEC 62443 part 4-2...” In simple English, this means the offensive path of least resistance is where there is no cyber security - the process sensors and their ecosystems. This is what the ICS cyber “kill chain” defenders continue to ignore.
The fundamental approach of OT cyber security defenders is to assume that what needs to be defended are the OT networks and therefore that it is not important to understand how the systems they are trying to protect work. Unlike the offensive attacker’s attempt to cause a specific impact, compromising an OT network does not directly lead to an affect on specific pieces of equipment. For example, if the OT network is in a power plant, there is no understanding by many OT security defenders how a power plant and the equipment in the plant work and the associated system interactions. Process sensors are also the input to OT networks, and OT security experts commonly assume these to be uncompromised, authenticated, and correct which makes the attacks possible. It is not a “fair fight” when the defenders won’t address what the attackers are targeting especially when many of the networks and devices being targeted have no cyber logging or forensics.
Understanding how the systems and components work is not just a cyber exercise as the process sensors are the input for predictive maintenance, digital transformation, Industry4.0, smart manufacturing, smart grid, etc. In a recent plant test, the Windows-based HMI was not effective and, in fact, provided misleading information on the state of the process sensors and plant equipment. Monitoring tools for process sensors and plant equipment need to be purpose-built, not general-purpose systems such as Windows. More details will be included in the November issue of IEEE Computer magazine: “Using Machine Learning to Work Around the Operational and Cyber Security Limitations of Legacy Process Sensors" https://www.controlglobal.com/blogs/unfettered/windows-based-hmis-are-too-slow-for-monitoring-process-sensors-or-plant-equipment-anomalies
When sensors are wrong, equipment can be damaged and people can die - https://www.controlglobal.com/blogs/unfettered/another-process-sensor-incident-that-has-killed-people.
Unfortunately, you don’t need to be a cyber expert to impact sensors. Two fast food workers told police they wanted their shift at the fast-food restaurant to slow down. During the interview, they told the police their intentions were that if the railroad crossing gates could malfunction and they could somehow block traffic, that would prevent people from getting to the restaurant, and they could have a slow night at work. Police said one of the fast food workers placed a makeshift device on the tracks that affected the crossing gate sensors. The railroad’s dispatch center could not have determined that the crossing gate signals were being intentionally disrupted. This was a control system cyber incident, one of many that have affected rail transportation.
Inadequate government approaches
TSA’s pipeline cyber security requirements are inadequate as can be seen from the January IEEE Computer magazine article, “Control System Cyber Incidents Are Real—and Current Prevention and Mitigation Strategies Are Not Working”. EPA’s OT cyber security requirements also are inadequate for control systems. There have already been more than 125 control system cyber incidents in water/wastewater that include complete loss of water, water hammer, chemical contamination (not Oldsmar), pumping water from a superfund site (contaminated water) into the drinking water system and recent examples such a sewage treatment facility recently that was overbilled because the flow sensors indicated higher than actual sewage flow. This is trivial compared to the collapse of the Taum Salk earthen dam and the loss of billions of gallons of water because of inaccurate sensors. A recent article on water system cyber security did not address the sensors or actuators - https://aws.amazon.com/blogs/industries/smart-metering-for-water-utilities/.
The Cyber Incident Reporting for Critical Infrastructure Act of 2022 (CIRCIA) assumes cyber forensics exist which is not the case for legacy ICS field devices. The zero trust initiative also does not apply to legacy control system devices that do not have minimal cyber security capabilities and are 100% trusted.
Summary
Control system cyber security is more than just protecting IP networks. In order to defend and optimize the plants, buildings, facilities, and transportation, one needs to understand how the systems and components work. It doesn’t make sense that the approaches attackers have used to successfully compromise physical infrastructures continue to be ignored by the cyber defenders.
Joe Weiss
Leaders relevant to this article: