Securing control systems consists of securing Operational Technology (OT) networks and control system field devices (e.g., process sensors, actuator, drives, etc.). However, there is still a gap in cyber security education and technology at the field device level. May 28, 2019, two independent announcements affecting control system supply chain cyber security were made that taken together spell a lack of cyber security, safety, and resilience of all infrastructures including the electric grid. Supply chain issues can be subcomponents within the purview of the control system supplier or they can be outside the scope of the supplier.
The first announcement was the NERC Cyber Security Supply Chain Risks: Staff Report and Recommended Actions Docket No. RM17-13-000. According to NERC, the supply chains for information and communications technology and industrial control systems may provide various opportunities for adversaries to initiate cyber attacks, thereby presenting risks to the Bulk Electric System.
The second announcement was a notification of counterfeit Yokogawa process transmitters. Process transmitters measure pressure, level, flow, temperature, etc. Pressure and differential pressure transmitters are used in process control applications across commercial, industrial, manufacturing, and defense applications including nuclear power plant safety applications and non-nuclear Safety Integrated Systems (SIS). Process transmitters are the true "edge devices" in a control system but generally lack cyber security or authentication. Specifically, Yokowaga transmitters are extensively used globally including in North America. The first Yokogawa notification on counterfeit transmitters was in 2014 and was based on counterfeit devices found outside North America (https://www.yokogawa.com/pr/topics/2014/pr-topics-2014-12-en.htm ). Per Yokogawa, the recent announcement based on a new report of counterfeit products in North America. The known affected counterfeits were limited to the EJA-110E series. The counterfeit products were procured through an unauthorized, counterfeit supply chain with which Yokogawa has no business relationship. Most likely the counterfeits were sold for profit like selling fake Prada purses for a discounted price. Yet, counterfeit transmitters can act as Trojan horses to deliver malware behind firewalls. Counterfeit transmitters can also be misconfigured, inaccurate, or incapable of meeting design requirements (see attached figure).
Counterfeit transmitters are not a unique problem to Yokogawa. There have been numerous cases where counterfeit or “gray market” transmitters from other vendors have been used but there hasn’t been a formal notification from another transmitter vendor as with Yokogawa. Sinclair Koelemij from Honeywell responded to the Linked-in discussions on the Yokogawa announcement with the following: “There are numerous other examples of counterfeit field devices and sensors, even in combination with counterfeit ATEX (ATmospheric EXplosible) certifications (ATEX certification is a requirement for all companies who manufacture electrical equipment that is used in hazardous environments and is intended to be marketed in the European Union). The supply chain is critical for OT security, this includes all elements of an automation system. Not only from a cyber risk perspective, mounting counterfeit equipment in the field can lead to very serious accidents. In the case of a false ATEX certification even massive explosions.” Other control system suppliers have had customer calls concerning transmitter performance where the supplier cannot reconcile the installed transmitter serial number with the supplier's records.
Counterfeit transmitters become a nuclear safety concern because of what is called Commercial Grade Dedication which effectively allows for the use of non-nuclear qualified safety devices in nuclear safety applications. The term “counterfeit” is not used for Commercial Grade Dedication but the term “reasonable assurance” is used as a catch-all term. Counterfeit transmitters certainly cannot provide reasonable assurance of expected performance. Counterfeit transmitters are also a major concern for SIS applications as many safety systems use the same transmitters as in basic process control applications.
It is not clear how wide spread these counterfeit transmitters have made their way. Counterfeit transmitters can be a common-cause failure mechanism which is VERY dangerous. Moreover, they can be pre-programmed defeating any cyber security program. Consequently, there is a need to have a program to identify counterfeit devices before they are installed as well as after in case they get through the screening process.
Going back to the NERC supply chain submittal, the only sensors identified in the NERC submittal were motion sensors for physical security. Process instrumentation and safety systems that utilize counterfeit transmitters can cause kinetic damage across multiple facilities – potentially a significant grid reliability problem. Because counterfeit transmitters can be pre-programmed independent of Ethernet (routable) OT networks and yet feed the OT networks, counterfeit transmitters can impact NERC High, Medium, or Low impact systems. However, there is no cyber security program to address counterfeit transmitters. Moreover, transmitters are installed by instrument technicians who are generally not part of any cyber security team and therefore have minimal to no cyber security training.
It appears that government and standards efforts have not adequately addressed the cyber security of these critical devices. However, the governments of Russia and Iran do seem to care about this topic. In the 2014 time frame a Russian security researcher gave a presentation at the ICS Cyber Security Conference on hacking the wired Highway Addressable Remote Transducer (HART) protocol which are the 4-20 milli-Amp process sensor networks. At the same conference, a presentation was given by a researcher from the Air Force Institute of Technology on a proof-of-concept study on fingerprinting process sensors. This was followed up in 2016 with RF DNA fingerprinting results from from Wired HART transmitters from Emerson, Honeywell, Siemens, and Yokogawa (Lopez, J. Leifer, N.C., Busho, C.R., and Temple, M.A., "Enhancing Critical Infrastructure and Key Resources (CIKR) Level 0 Physical Process Security Using Field Device Distinct Native Attribute Features" IEEE Transactions on Information Forensics and Security- 2017). Additionally, I gave a presentation at the August 2017 Defcon Conference on the lack of cyber security of field devices including process sensors. In October 2017, I received a “Like” on my Linked-In account from Iran on my Defcon presentation.
For the past several years, I have written extensively about the need to monitor process sensors at the raw signal level in real time. Fingerprinting process sensors, which included the specific Yokogawa series of sensors, should be able to detect the difference between original (OEM) and counterfeit Yokogawa sensors particularly as the website states there is a difference in the circuit structure and principles of measurement (there were no counterfeit sensors in the fingerprinting work). Moreover, the OT network monitoring and threat detection vendors start by assuming sensors are uncompromised, authenticated, and correct which which may not be correct assumptions. ISA99 is addressing cyber security of process sensors at the policy level in ISA/IEC62443-4-2.
I felt out-of-band monitoring of sensors could help with supply chain before I read the Yokogawa announcement. Given the Yokogawa announcement and the Stuxnet and Triton attacks which needed to compromise operator displays, real time out-of-band sensing is needed ASAP.
It has been evident that control system cyber security has suffered from cultural gaps/governance issues which often led to the lack of cyber security in process sensors/transmitters and the lack of instrument engineers/technicians participating in cyber security teams. This also brings up the question as what is OT. If the transmitters are not considered part of OT, this is NOT an IT/OT convergence problem. If the transmitters are considered OT, it becomes critical that instrument engineers and technicians become part of the cyber security team.
As mentioned, according to NERC, the supply chains for industrial control systems may provide various opportunities for adversaries to initiate cyber attacks affecting the Bulk Electric System. Yet, NERC has avoided addressing control system field devices and networks (sensors being inside the Electronic Security Perimeter makes them out-of-scope). The irony is process sensors are critical for reliability (the “R” in NERC) yet NERC continues to ignore them. This has to change.
Neither Stuxnet nor Triton were believed to be threats until they actually occurred. The same appears to be the case with cyber security of process sensors. Control system cyber security needs to address both networks and control system field devices. This includes people (having instrumentation experts involved), process (monitoring for counterfeit sensors and certifications), and technologies (on-line sensor monitoring). The bottom line is if you have control of the transmitters, you have control of the process which should be the point of performing control system cyber security.
Joe Weiss
Leaders relevant to this article: