Anatomy of a cyber-attack and successful recovery

"Isolate means disconnecting cables and communications, and basically 'putting police tape around a compromised area,' so it can be examined later." Gary Williams, cybersecurity services offer leader, Schneider Electric, detailed his company's response to the recent TRISIS/Triton malware attack on its Middle East refinery/petrochemical end-user customer.

Recovering from a serious cybersecurity attack requires all kinds of resilience. This includes many logical technical fixes, but it also demands a large dose of less-obvious professional and personal resolve.

Despite these multiple challenges, this is exactly the journey that Schneider Electric and its client undertook during the past year after its end user—a large oil and gas refinery and petrochemical facility in the Middle East—was subjected to a premeditated, focused and sustained cyber intrusion and attack. An unknown attacker injected the TRISIS/Triton malware into a Tricon safety instrumented system (SIS) engineering workstation running EcoStruxure Triconex software, which had been left in "program" mode. When the malware attempted a reprogram, the controller recognized an anomaly and took the plant to a safe state via a shutdown in August 2017.

A subsequent misstep by the malware was detected by the SIS, which triggered a safe shutdown of the related application. The incident was reported in December 2017, and Schneider Electric's recovery, remediation and investigation efforts have been ongoing since that time. "The truth is this cyber attack could have been at any site or safety system, but in retrospect it's probably fortunate that it happened to a Tricon system that was able detect the problem, and bring the application to a safe state," said Gary Williams, cybersecurity services offer leader, Schneider Electric.

An update on the TRISIS/Triton cyber attack and Schneider Electric's response, entitled "From Fact to Fiction: What Happens After a Cyber Incident," was presented by Williams and Steve Elliott, senior director of marketing, Schneider Electric Process Automation, this week at the Triconex User Group conference in Galveston, Texas.  

Intrusion evolution

Elliott reported that the affected equipment had been installed at the Middle Eastern plant in 2007. Since then, the refinery had expanded its applications, and invested about $18 billion to implement petrochemical and finished products applications. It also conduced a major inspection and turnaround, which included more than 25,000 contractors and other personnel onsite.

Williams added that, though the attack began in August 2017, the intrusion may have started about two years earlier. This is because the malware had to penetrate several layers of the plant's network to reach its distributed control system (DCS) and the SIS that was ultimately affected. "We still don't know the intent of this attack," said Williams.

Whatever the motivation, Williams explained that Schneider Electric has a three-part procedure for effective cybersecurity—isolate, identify and eliminate.

"It's important to understand that isolate doesn't mean turning off machines or other equipment because this could eliminate all the evidence for forensic investigation of an attack," said Williams. "In this case, isolate means disconnecting cables and communications, and basically 'putting police tape around a compromised area,' so it can be examined later.

"For us, this means comparing the software on a workstation or other device to a virgin copy of its software. This lets us identify what code should be there and what should not. Once we find out that some unauthorized or unknown code has been added, we can hit the emergency button and escalate the process. Only three parties know what really happened: the attacker, the forensics team, and the end user."

Williams added that, once the malware was identified and eliminated, the plant's owner advised all its sites to follow Schneider Electric's recommendations on intrusions, such as enhanced password management. For its part, Schneider Electric also alerted software patching entities, and gave them a solution for preventing similar attacks elsewhere.

"We also developed a tool that other users could employ to check their systems," he added. "And, we were invited to Washington, D.C., where we participated in several autopsy efforts to help understand the technicalities of attacks, which is the beginning of developing policies and procedures in the hope that future standards can incorporate some of the lessons learned."

Collateral damage

Though not directly related to process control, Elliott reported that Schneider Electric has also had to cope with a tide of factually lacking and/or erroneous media reports about the incident, and even observed some equipment suppliers seeking to capitalize and make online sales in its wake.

"There were a lot of blog posts that just seemed to want to create an audience, but had few facts. There are now 390,000 mentions of ‘Triton’ on Google," added Elliott. "It also seemed like in about 48 hours, Triton was also suddenly a topic of many upcoming conference sessions. The first we attended was the S4 conference, where we again tried to be as transparent as possible, even though we had to remain quiet about some details until we had the true facts."

Beyond following Williams' technical advice about following IEC 62443 security recommendations for implementing "zones and conduits" in industrial networks—and having suppliers' contact information handy—Elliott added that process control and automation users and suppliers must jointly address cybersecurity threats.

"This is not just our problem or one user's problem. This is an industry-wide problem," concluded Elliott. "We all have to work together, including getting governments and legislators involved. And, cybersecurity isn't just about safety systems, but also about every kind of process automation system. We have to work together to beat these attacks."