Hacking IT and OT networks can be expected to continue regardless of the cyber security technologies employed. These attacks will include, but won’t be limited to, ransomware attacks. Ransomware typically affects manufacturing and industrial facilities either through loss of IT data or suspension of operations due to “an abundance of caution”. It would be a mistake, however, to think that all threats to industrial or manufacturing processes can be reduced to threats to IT systems, and this suggests one family of industrial subsystems that can be too readily overlooked: process sensors.
Process sensors from a security point of view
OT monitoring systems monitor the networks and perform asset identification to identify the physical characteristics of all devices on the OT network including process sensors. (These, it’s important to note, are not IoT devices). The OT network monitoring characteristics include hardware, software, and patch versions, vulnerability disclosures, etc. However, OT monitoring systems cannot assess the integrity or authentication of the process sensor reading. This was demonstrated in the February 2022 process sensor monitoring project where more than half of the process sensors on a manufacturing line were inoperable or out-of-calibration. However, the Windows-based HMI (which would have the same data as OT monitoring systems) did not indicate those conditions. Consequently, it is vital that a “non-hackable”, ground-truth view of the physical process be available regardless of the state of IT and OT networks. Moreover, process sensor monitoring is agnostic with respect to the cause of an anomaly. Whether that anomaly comes from sensor miscalibration, sensor drift, process or equipment anomalies, or cyber threats, the monitoring will flag it as anomalous.
Process sensors are the responsibility of the process engineers and are rarely considered in cyber security assessments until the signals become Ethernet packets. Network security personnel address cyber security of the sensors once their outputs have been converted to Ethernet packets. However, network security personnel ignore the cyber security aspects of raw process sensor readings as they assume process sensors are uncompromised, authenticated, and correct. Once process sensor signals have been converted to Ethernet packets, they become as cyber vulnerable as other Ethernet packets. Consider as an example, the 2015 Ukrainian cyberattack, which involved hacking the serial-to-Ethernet converters. There are many ways for the “raw” sensor signal to be compromised before the signal becomes an Ethernet packet. These compromises cannot be found once the signals become Ethernet packets. This becomes even more problematic when the cloud providers don’t address the integrity or authentication of the incoming process sensor data.
Reaching ground truth
When the process sensor data is collected and analyzed off-line from the IT and OT Internet Protocol (IP) networks, process sensor monitoring systems become independent, “ground truth” verification of the OT network systems. Even though it is not addressing ransomware, process sensor monitoring also has relevance for the electric industry. The US Department of Energy (DOE) Electric Emergency Incident and Disturbance Report (Form DOE OE-417) collects information from the utilities on electric incidents and emergencies. One of the categories is “Complete loss of monitoring or control capability at its staffed Bulk Electric System control center for 30 continuous minutes or more.” There have been more than 150 of these incidents to date. Knowing what is happening at the substation independent of the IP networks would be critical when control center monitoring is lost. Monitoring the process sensors also supports Moody’s assessment of Presidential Executive Order (EO) 13920, “US electric utilities will benefit from cybersecurity measures in executive order” which Moody’s considers credit positive. (It should be noted that even though the EO was suspended, the need for monitoring still exists as the concerns expressed in the EO are real and continuing.)
Return on Investment (ROI)
Two recent cases demonstrate the economic benefit of monitoring process sensors at the physics level.
The first case was identified in the November IEEE article - https://www.controlglobal.com/blogs/unfettered/blog/21437429/ieee-paper-on-process-sensor-monitoring-what-you-need-to-know-about-process-sensor-cyber-security. The project was performed in February 2022 for product quality and productivity, not cyber security. The process sensor and main feedpump issues had an approximately 3% hit on net productivity in a billion-dollar facility. The cyber security benefit “came along for the ride” as it should.
In the second case, a December 27, 2022 ransomware attack on the Canadian Copper Mountain Mining (CMMC) IT systems resulted in the copper mill being shut down as a preventive measure. (Shutting down the process is often the hacker’s ultimate goal. Convincing the facility operators to shutdown is a successful attack.) The mill was shut down because CMMC did not know the status of its control systems (abundance of caution). Monitoring the physics of the process sensors would have provided the status of the control systems and the process providing a justification to continue operation even with the IT and OT networks unavailable. A rough estimate of the financial impact on CMMC of the decision to shut down the mill was approximately $5 million including lost margin and other expenses (no facility equipment damage).
As I am not an expert in credit ratings or insurance, I had experts in those areas provide their perspectives.
Insurance risks
According to Marc Schein, Marsh McLennan Agency’s National Co-Chair Cyber Center of Excellence, “Companies that have experienced plant/facility shutdowns could find coverage under the business interruption clause of a cyber-insurance policy. However, companies that can’t illustrate good cyber hygiene may be uninsurable or find insurance carriers limiting cyber insurance coverage for ransomware by including co-insurance and/or sub-limits for ransomware.” In the CMMC case, ransomware was the initiating event, but the cause of the business interruption was the decision to shut down the mill even though there were no obvious impacts on operation. In the CMMC and other ransomware cases such as the case with a large meat processing facility, process sensor monitoring at the physics level could have provided the justification to continue operation, precluding the resulting business interruption done “for an abundance of caution”.
Credit ratings risk
Demonstrating credit rating agencies’ concerns about ransomware attacks on critical infrastructures, Moody’s issued a report “Colonial's suspension of pipeline operations shows sector's vulnerability to cyber risk.” These issues go beyond credit ratings and speak to long-term enterprise value. Boards need to protect shareholder value/enterprise value and credit ratings directly relate to that because credit ratings can influence the cost of capital. Having a production facility go offline could have material impact for some companies – for example, companies where the production facility represents a large share of revenues or profits. But for others, for example highly diversified where a single facility won’t really move the profits, then the risk is lower obviously. Also, what other mitigations are in place? As an example, Freeport LNG relied on business insurance during an outage. Regulatory trends are pointing to increased scrutiny, so an organization’s ability to monitor and measure performance is rising. Specifically, impacts on plant productivity can have significant impacts on physical and economic risk. As identified in the first case, a 3-5% hit on net productivity could have significant economic impacts on an organization’s financial conditions. Additionally, the inability to know if the process sensors are working properly can be a significant facility and safety risk.
Recommendation
Company executives, risk managers, Board of Directors, insurance companies, and credit rating agencies need to recognize the economic and safety benefits of process sensor monitoring. The flip side is not only unnecessary shutdowns, but also unsafe and inefficient operation.
Joe Weiss
Leaders relevant to this article: