Two of the most consequential control system cyber events (attacks) in 2020 were supply chain attacks. The first event was the Chinese installing hardware backdoors in large electric transformers, an incident that prompted Presidential Executive Order (EO) 13920. The hardware backdoors are obvious control system threats. The second event was the Russian SolarWinds cyberattack. Even though SolarWinds is a significant threat to IT networks and the Cloud, it is also a control system threat, although less obvious than the hardware backdoors.
In the transformer case, the scope of the compromise remains unknown. There are more than 200 large Chinese-made electric transformers in the US bulk electric system and it is unknown how many of these transformers have hardware backdoors installed. It is also unknown what and how much other Chinese-made equipment throughout the US (international) commercial and industrial infrastructure have Chinese hardware implants.
The industry response to the Executive Order missed the attack vector – the control systems. The Chinese attackers installed hardware implants most likely prior to field installation of the transformers to provide remote command and control capabilities. In this manner, “spoofed” sensor signals (there is no cyber security or authentication in existing process sensors) are completely trusted by the transformer equipment. This means the Chinese can effectively gain control of the transformers without any network forensics being the wiser. Consequently, the Executive Order stated: ”The term bulk-power system electric equipment means items used in bulk-power system substations, control rooms, or power generating stations, including reactors, capacitors, substation transformers, current coupling capacitors, large generators, backup generators, substation voltage regulators (LTC), shunt capacitor equipment, automatic circuit reclosers, instrument transformers, coupling capacity voltage transformers, protective relaying, metering equipment, high voltage circuit breakers, generation turbines, industrial control systems, distributed control systems, and safety instrumented systems. Items not included in the preceding list and that have broader application of use beyond the bulk-power system are outside the scope of this order.”
However, the industry response has been exactly opposite of what the Executive Order directed: a focus on network equipment and ignoring hardware and control system devices. Edison Electric Institutes’ (EEI) procurement guidelines, for example, do not address any control system-unique issues yet it was the control systems that were attacked. The same can be said of other industry responses including from the Department of Energy (DOE). Yet it was DOE’s concern with what they found in the Chinese transformer installed at the WAPA Ault substation that resulted in the interception of the next Chinese-made transformer delivered to the port of Houston in early 2020 and its delivery to the Sandia National Laboratory (SNL) to be torn apart to understand the details of the hardware implants. This was not a trivial decision. It meant that a utility would be missing a large electric transformer. Unfortunately, there has been silence about what, if anything, DOE found from the examination of the Chinese transformer at SNL. For whatever reason, Idaho National Laboratory’s (INL) Sarah Freeman recent on-demand RSA2021 webcast on “Supply Chain Security: A New Kind of Halting Problem” https://go.rsaconference.com/rsac365-webcast/webcast-68-supply-chain-ty?vs=OTMwM2NjOGYtYjRjNi00MzEwLThlZTktZWRhMDZiNGMzNzNlOzsyOTkxMjgwOjkxODE2MDY0MDkzS0 did not mention this supply chain attack on the Chinese-made large-electric transformers even though she discussed implants in Chinese-made Lenovo PCs. It would seem that a supply chain attack against large electric transformers is arguably more consequential than a compromised supply chain affecting PCs.
The recent Russian SolarWinds hack has been identified as one the most significant cyberattacks against the US. There has been a continuing stream of guidance from DHS CISA, SolarWinds, and multiple industry organizations. Yet there has been almost no mention of the control system aspects of the attack including in Sarah Freeman’s above-mentioned presentation. Cyber vulnerable building systems and devices including HVAC, managed switches, power distribution units, perimeter and process sensors, and Uninterruptible Power Supplies (UPSs) can cause significant damage to equipment and harm to people. Because of the lack of control system cyber forensics with these devices, it is difficult to know what has actually occurred. Building control cyber incidents have caused physical damage and network impacts (recall the Target hack and the Dallas hospital cyberattack). Moreover, the Russians have already used the SNMP vulnerability in SolarWinds in the 2015 Ukrainian power grid cyberattacks (https://www.controlglobal.com/blogs/unfettered/solarwinds-orion-the-weaponization-of-a-network-management-system). Whether the Russians intend to exploit this vulnerability or “sell it”, it should still be a major concern.
The common threads between the Chinese transformer hardware implants and the SolarWinds cyberattacks were they were both supply chain attacks of trusted suppliers and were not detected by IT network monitoring or threat intelligence in a timely fashion. Ransomware attacks can also be added to this list as they are often not detectable until it is too late. Consequently, these attack vectors reinforce the need for any operational process to be monitoring the process sensors at the “atomic level” where they cannot be hacked. This can provide a real-time assessment of the integrity of the physical process independent of the too-often demonstrated cyber-vulnerability of Internet Protocol (IP) networks.
Because of the importance of SolarWinds/control system issues and the lack of understanding, I have been asked by a number of organizations to give presentations on this subject.
Joe Weiss
