July 15, 2020 I gave a 1 hour presentation on control system cyber security for the Purdue University Summer Seminar Series. There were 183 pre-registrations of which 119 attended. The registrations were from 16 countries – Australia, Austria, Brazil, China, Germany, India, Israel, Kuwait, Lithuania, Mexico, Netherlands, New Zealand, Saudi Arabia, Singapore, UK, US. Actual attendees were from India, Israel, Kuwait, Lithuania, Mexico, Netherlands, Saudi Arabia, Singapore, UK, US. For those unable to attend, the recording is on the Purdue Cerias website - https://www.youtube.com/watch?v=B04JtQVhufs).
After 20 years, control system cyber security has made significant strides in monitoring and securing OT (control system) networks. However, after 20 years, control system cyber security has made minimal strides in monitoring or securing the actual control system devices (e.g., process sensors, actuators, drives, analyzers, etc.) and lower level device networks which is where you go “boom in the night”. Much of this is because of the culture clash between the engineering community who understand the devices but generally have been removed form control system cyber security efforts and the network security personnel who do not understand control system devices or control system processes. The impact of the culture gap is at least partially due to network security’s erroneous assumptions:
- Process sensor input to all OT networks are uncompromised, authenticated, and correct so that securing the network packers is sufficient to protect the control systems and physical processes,
- control system devices can only be accessed from Ethernet (IP) network,
- all control system anomalies can be found from Ethernet (IP) network,
- network vulnerabilities directly correspond to physical system impacts,
- cyber security frameworks can be directly applied to control system cyber security, and
- cyber security is about zero trust.
There were 10 questions raised that I did not have a chance to answer on the webinar. I thought the questions and answers would be of general interest.
1). Q: Joe, this is great. You said "Our information sharing doesn't work." What do you think needs to be improved, and how would you improve it?
Answer: Information sharing on cyber network vulnerabilities are addressed in DHS and vendor disclosures as well as industry ISACs. The information sharing that is missing is about actual cyber-related incidents. NERC Lessons Learned don’t address incidents as being cyber-related. The various industry ISACs have not addressed cyber incidents occurring within their industry. The sharing on control system incidents to date most often has been by engineers who have seen incidents that were out of the ordinary. Informally, my old conference (ICS Cyber Security Conference which no longer exists) served as an informal information sharing vehicle for the engineers to discuss real control system cyber-related incidents. Unfortunately, I don’t believe the government can help because of the private organizations concern about making these incidents public. I wrote a chapter in my book, Protecting Industrial Control Systems from Electronic Threats Chapter 9 “Information Sharing and Disclosure”. I will have more to say about this subject in a separate blog at www.controlglobal.com/unfettered.
2). Q: What is your view on Executive order for going back to analog system? We are all driving through zero carbon and digitalization- How to achieve the balance between them?
Answer: Hardwired analog systems with no “intelligence” such as electromechanical float switches as part of a hard-wired relay ladder logic system would be independent of a cyber attack from external threat agents, including hardware backdoors. However, adding any intelligence and modern communication capabilities would make the analog systems as vulnerable as digital systems to a backdoor sensor attack. Both smart and dumb systems would be potentially vulnerable with respect to a physical, hands on insider attack. That is the reason for defense-in-depth. The only way to get the balance between zero carbon and digitalization (or manufacturing and digitalization) is to have engineering and network security work the issues together.
3). Q: what approach do we take to secure level 0 and level 1 equipment?
Answer: I mentioned in my presentation that a major misconception is that all process sensor communications have traverse the Ethernet IP network and that network monitoring can identify any sensor anomalies. Consequently, there is a need to develop control system policies, procedures, and use existing equipment (as well as network) monitoring technologies. However, existing equipment or network monitoring technologies likely will not be sufficient to identify counterfeit devices or address hardware backdoors. This would most likely require new sensor monitoring technology that address the “physics” of the sensors which would be the input to both equipment and network monitoring. This new sensor monitoring technology has been proven in laboratory testing against multiple sensor vendors. In addition, there needs to be an industry recognition that the requirements within standards like ISA 62443 apply to the entire control system, level 0 through to the DMZ. Part of this understanding is that the control system and its network is owned by engineering personnel (operations, engineering support, maintenance) rather than the IT personnel, who should be used in a support role as a service provider.
4). Q: So to keep validating the low-level sensor data real time, we will need to know the algorithm that computes the result (e.g., temperature, pressure, etc.) but manufacturers may not wish to share their proprietary algorithms. Then, what can be done?
Answer: The sensors need to be identified by physics “fingerprinting” (see above). This would identify counterfeits as well as identify any sensor deviations regardless of cause. That is, it will identify deviations whether from sensor drift, loose screws, calibration issues, hardware problems, or cyber compromises. Once the deviation is identified, there are available tools that should be capable of determining the cause. It is a shame to say in 2020 we still don’t know when to use our diagnostic toolboxes because of the lack of awareness.
5). Q: Could you also have an engineer's SOC rather than an IT/OT SOC.? They would focus on the engineering aspects.
Answer: Without being flippant, that is the control room or control center.
6). Q: How to mitigate supply chain risks?
Answer: This is a very complex question because supply chain risks can range from an entire device to software, to firmware, to microprocessors, as well as integration of multiple instances of these. It requires the cooperation of procurement, physical security, cyber security, test shops/labs, engineering, and maintenance. Sensor monitoring to detect counterfeit or hardware backdoors would be a critical piece of the solution. Asset owners should also require their product and service providers to comply with standards like ISA 62443-2-4 and then to vet them against those requirements. I would be happy to further discuss my thoughts offline.
7). Q: Two questions : Is there any validated OT architecture that may hinder the possibility of backdoor attacks where the device would look for a master to trigger?
Answer: I don’t think so as the backdoor could bypass the OT architecture – the reason for the Presidential Executive Order.
8). Q: I had a question about the levels. Do you think there is an advantage in separating level-0 devices to continuously-monitored devices (PLCs, HMIs) and smart IO Devices (IO Link based devices, Ethernet IP devices/Profinet devices)
Answer: Two years ago, we created a special ISA99 Working Group on Level 0,1 devices – ISA99.04-07 to address Level 0,1 devices. The working group concurred that “legacy“ Level 0,1 devices cannot be secured to current cyber security requirements. Additionally, the Purdue Reference Model was identified as being out-of-date for addressing modern control system device technology for cyber and communication capabilities as there no longer are clear distinctions between levels even for the same device. There is an advantage to segregating sensors based on the zone they are located. Each zone should have its security requirements based on risk and countermeasures that are unique to that zone. For instance, a safety-instrumented system (SIS) involves sensors, logic solver, final elements as well as an engineering workstation. Having a SIS zone makes it easier to accomplish least privilege from both a physical and logical access perspective.
9). Q: Are Controls devices companies taking any action to certify programmable hardware electronics to validate no malicious logic is included on logic or printed circuit hardware?
Answer: I think that is the ultimate intent of ISASecure and commercial test/certification companies. The devices certified to date are controllers and master stations. None of the Level 0,1 devices has completed cyber security certifications.
10). Q: Another questions I had was about a recent change in the industry direction, to put all devices on the IP network now. I bring new machines to our plant, and 100% of our machines have an Ethernet network and a NAT gateway to expose device.
Answer: Unfortunately, that is becoming a common practice especially with Industry4.0 and other digital upgrade initiatives. However, I believe there is a real need to question whether safety devices should be part of the overall integrated plant Ethernet network. Moreover, I think there needs to be a reassessment of the need to connect control system devices directly to the Internet without some form of proxy to shield them from public access.
Joe Weiss
Leaders relevant to this article: