A simulated terrorist attack on the North American power grid last November exposed weaknesses in handling cyberthreat data and marshaling emergency support to restore electric power service to blacked-out cities, according to a report released yesterday.
The North American Electric Reliability Corp., the high-voltage grid’s security monitor, released its evaluation of the GridEx III "war games," a closed-door training event involving 4,400 industry and government participants that confronted grid operators with hypothetical cyberattacks and armed assaults on substations across the United States and parts of Canada and Mexico.
NERC said the exercise gave a large segment of the grid industry a vivid, hands-on experience with the unprecedented fallout from the attack scenarios, which ended with extended power losses to certain cities.
Although the game plan included sophisticated cyber warfare tactics, it was not an actual penetration test on utilities’ cyber defenses, said Bill Lawrence, associate director of NERC’s cyberthreat information sharing portal. "There was nothing that came out of the exercise that would point at a widespread [cyber] vulnerability" around the industry, Lawrence said.
The exercise revealed problems with NERC’s cyberthreat-sharing portal, the Electricity Information Sharing and Analysis Center (E-ISAC), NERC said. The center’s job is to collect malware indicators as they are discovered by utilities or government agencies and rapidly share these with the electric power sector.
For GridEx III, NERC created a "mirrored" version of the E-ISAC portal with the same functionality as the real portal. It could not keep up with the flood of distress calls coming in from power companies in the exercise, NERC’s report said.
Responding after the event, power companies said the portal "needed enhancements to handle real-time, urgent communication with portal members. During the exercise, information was quickly buried within the portal and it became difficult to highlight important information."
GridEx III participants reported that some of the industry’s information-sharing and reporting practices "are redundant, time-consuming to use, and provide no feedback mechanism to those who most need the information."
A second major conclusion from the exercise was the need for a master plan to coordinate federal, state and local government support for the electric power industry if it ever did face a widespread, lasting outage, NERC said.
After an attack, the recovery resources of federal agencies, governors and national guard units, communications and other crucial infrastructure providers would have to come together, as seamlessly as possible, to help utilities restore power, said Paul Stockton, former assistant secretary of the Defense Department for homeland defense.
"It’s important to reiterate that the electric power industry is in charge of electric power restoration and will always be so. The question is, how can federal, state, local and tribal governments forge a unity of effort" to help the power sector, Stockton said in an interview. "It’s a great challenge," he said (EnergyWire, March 9).
On the second day of the Nov. 18-19 exercise, 32 power company senior executives and government officials wrestled with question of a coordinated restoration of the grid, including unresolved issues of who gets power first if there isn’t enough to go around and how scarce replacement equipment would be distributed.
"I know there’s been work on the classified level with Department of Defense and the Department of Energy, as well as some select industry members, in terms of prioritizing [recovery actions], at least from federal government side," Lawrence said.
"So it is definitely fertile ground to talk about the prioritization," he said, "because even though the grid goes across borders and goes across state boundaries, it would definitely be a challenge to, say, rob Peter to pay Paul, if certain assets need to be moved across those boundaries."
Following the exercise, the Electricity Subsector Coordinating Council, the industry’s security leadership group, is revising its attack response playbook of what the chief executives would do in a crisis to help each other’s companies, and whether new policies or legislation is needed, said NERC chief security officer Marcus Sachs. "We all agreed the playbook should be updated within a year," he said.
Thinking the ‘unthinkable’
Recent cyber ransomware attacks on hospitals illustrate the challenge of defining industry and government responsibilities in restoring vital services, Sachs said. The same issues would confront the power sector after a major attack: "Whose lane is this in? Does Congress need to help out? What authority would government have to mandate actions?" These are emerging questions, Sachs said. "We really haven’t thought it through at a policy level."
A third focus of GridEx III was on the role of the 16 regional reliability coordinators on the North American grid that would have to manage grid crises in their areas. Under NERC’s procedures, the coordinators have the highest authority for running day-to-day grid operations.
"What we wanted to do is challenge the reliability coordinators to be on that ragged edge of making decisions on [where] to roll things back or keep the lights on, in order to maximize the reliability of the system," Lawrence said.
There was plenty of value for power companies that took part, said Brian Harrell, a former NERC executive who developed the GridEx program and is now a security consultant at Navigant. "Cyber and physical threats are constantly evolving and require quick action and flexibility that comes from constant vigilance and collaboration. Exercising security response as an industry will strengthen relationships, improve security policy and increase the flow of critical information to the sector," he said.
However, only one-quarter of the utility participants sent in "lessons learned" reports requested by NERC, a "relatively low number" that NERC said it will seek to improve in the next exercise in 2017.
E&ETV’s The Cutting Edge: EnergyWire‘s Behr discusses new details on NERC cyberattack exercise
This week, the North American Electric Reliability Corp. released its unclassified summary of its latest GridEx exercise, which tested how utility participants would handle a blackout following a cyber and physical attack on the U.S. grid. How vulnerable is the grid to an attack like the one simulated at GridEx, and how should state regulators take the lessons learned from the exercise to formulate a plan of action? On today’s The Cutting Edge, EnergyWire reporter Peter Behr discusses the unclassified summary’s details and next steps. Today’s The Cutting Edge will air on E&ETV at 12:30 p.m. EDT.
