The North American Electric Corporation (NERC) Critical Infrastructure Protection (CIP) cyber security standards were developed to increase the cyber security and reliability of the electric grid. Unfortunately, they are not doing either.
The NERC CIP’s were developed by the electric industry with industry-developed exclusions. Because the scope was the BULK electric grid and the associated exclusions, the scope of cyber security of the electric grid is quite limited. Electric distribution is excluded (majority of Smart Grid falls under this exclusion), serial (non-routable protocols often using serial-network converters) communications are excluded, telecommunications is excluded, the “brightline” criteria exclude smaller facilities (the brightline criteria establishes minimum levels for facilities to be considered critical), etc. Additionally, the exclusions in the NERC CIPs provide a road map to attackers as they identify what is in-scope, and just as important, what is out-of-scope and consequently not addressed.
The NERC CIP audit methodology is a very onerous and expensive process (independent of the potential fines for failing audits). This has resulted in many utilities manipulating the NERC CIP process to minimize the number of devices and facilities to be addressed. Marlene Ladendorff is a cyber security professional who developed the cyber security program at a nuclear utility and is now with the Idaho National Laboratory. Her doctoral thesis was “The Effect of North American Electric Reliability Corporation Critical Infrastructure Protection Standards on Bulk Electric System Reliability” (http://pqdtopen.proquest.com/doc/1619581598.html?FMT=AI). The strongest theme from her thesis was “Entities Removing Equipment to Avoid CIPs”. That is, removing programmable digital devices and replacing them with the older serial devices in order to avoid being classified as Critical Infrastructure or Critical Cyber Assets (CCA) and necessitate inclusion in a NERC CIP audit. Yet the same systems being removed from transmission systems to avoid the NERC CIPs are being installed for Smart Grid applications which are outside the NERC CIP scope. What does this really mean to Smart Grid security?
From Marlene’s thesis, the following examples were provided:
- Participant 2 in her study found that a company had the most sophisticated network protection he had seen. However, NERC staff reviewed their architecture and wanted them to tear it out. It took the company 6 months to convince NERC that this was the best protection they could do for the control systems the company was operating.
- Participant 3 outlined a situation where an exercise was cancelled by their compliance group, citing potential non-compliance issues with one of the CIP standards as the reason. The logic behind the compliance groups’ action was that if a potential weakness was found, it may need to be reported and the entity risked receiving a fine from NERC. Participant 3 questioned the compliance group about their decision, stating that it was impossible to discover and fix weaknesses if exercising and testing was not allowed to find those weaknesses. The compliance group continued to refuse the testing, resulting in a catch-22 situation.
- Participants 5, 6, 10, and 11 experienced situations where “some of the transmission owners….are gaming the system in order to prevent the application of the CIP standards.” To accomplish this, some companies modified their networks to avoid compliance issues with CIP-003 through CIP-009.
- With the expense involved in compliance with the CIP standards, Participant 10 pointed out that “organizations worked very hard to not have or have very little…assets that they had to protect”, assets that would fall into scope of the CIP standards. Some entities were trying so hard to keep equipment out of scope that they spent money to “rip out fiber and CAT-5 [networking cable] and replaced it with serial [cable] to get away from routable protocols” that would have brought networks into the compliance scope. Entities calculated that it would be cheaper to replace fiber and CAT-5 network cable with serial cable in order to remove equipment from the CIPs scope. Doing so eliminated the requirement to comply with CIP standards for those networks and equipment.
- Participant 11 witnessed situations in more than a few utilities where remote access implementations were converted back to serial communications in order to reduce the amount of equipment requiring CIP compliance. Participant 7 echoed the comment from Participant 6, stating that entities took some networking hardware out and replaced it with “serial communications, only trying to skirt CIP compliance. Every entity I know plays the game that way.”
- Some utilities are making a cost-benefit decision on providing security versus paying fines. Depending on the cost of the fine compared to the cost to install NERC CIP compliance, some utilities have made the decision to pay the fine rather than make the security improvement. What does this mean for cyber security of the electric grid?
The following are technical issues that are being propagated by the NERC CIP process:
- Some substations employ nearly-automatic protective relay systems. These systems can sense when breakers re-close due to commands from the EMS/SCADA system, without ever receiving potentially compromised commands from the SCADA system directly. Many utilities would like to keep the relay systems inaccessible from remote access, as they do not need to be connected and any such connection increases cyber risk. However, NERC CIP demands that passwords on protective relays change periodically. This means that utilities with hundreds to thousands of substations will most likely connect their protective systems to external networks (usually over the Internet) to support a compliance requirement that can actually compromise security. Which is the greater risk - that someone will physically break into a substation and try to guess an old password on the relay or that someone will try to hack the substation remotely?
- Since the NERC CIP guidance requires anti-malware and anti-virus protection, some utilities are mandating protective relays to have malware protection even though adding this function will reduce the effectiveness and function of the relay. In some utilities, the security organizations are overruling the technical organizations to meet NERC CIP requirements. Are you really more secure or reliable if the protective relays don’t work?
- Another example of the inconsistency of the NERC CIP guidance is that when it comes to grid reliability is the use of “black start” facilities. Black Start facilities are those necessary to restart the grid after a complete grid outage. This function is considered critical by grid planning and operations organizations as well as organizations within NERC. During the review of the NERC CIP Revision 5 process, ISO New England raised a concern that adopting a new requirement for specific controls for Low Impact assets could have unintended consequences, such as the withdrawal of black start resources. This would make the grid less reliable.
Some of the security hardware can affect control system performance. A NERC report identified that a device locking tool used to meet NERC CIP requirements caused a disturbance that resulted in the loss of SCADA services. This is obviously making the grid less reliable and secure.
Perhaps the most important point is there have already been four major cyber-related electric outages in the US (more than 90,000 customers). If the NERC CIPs were fully implemented, they would not have prevented any of these outages. What does that say about the efficacy of the CIP’s when the NERC CIPs do not address previous cyber-related outages, attacks such as Stuxnet, or vulnerabilities such as Aurora?
These examples clearly demonstrate the NERC CIP approach is not adequately securing the grid or even maintaining existing grid reliability. In the end, utilities need to have the freedom to implement the proper infrastructure and cyber security appropriate to maintaining a reliable system without the fear of legal exposure or examination by those who are not familiar with the utilities system operations. Who in the utility industry is willing to stand up and state “the emperor wears no clothes”?
Joe Weiss
