Colonial touts cyber ‘best practices.’ It was still hacked

By Christian Vasquez, Peter Behr | 05/19/2021 07:02 AM EDT

The pipeline company at the center of the most disruptive cyberattack in U.S. energy history defended itself in a detailed statement to E&E News yesterday. But its actions and U.S. policies are raising concerns about the safety of millions of miles of energy infrastructure.

Fuel holding tanks are seen at Colonial Pipeline Co.'s Dorsey Junction Station in Woodbine, Md. The company has shed light on some of its cybersecurity practices leading up to a hack of its systems this month that disrupted eastern U.S. gasoline supplies.

Fuel holding tanks are seen at Colonial Pipeline Co.'s Dorsey Junction Station in Woodbine, Md. The company has shed light on some of its cybersecurity practices leading up to a hack of its systems this month that disrupted eastern U.S. gasoline supplies. Drew Angerer/Getty Images

The pipeline company at the center of the most disruptive cyberattack in U.S. energy history says it followed defensive guidelines set by the government and private sector.

But a ransomware cyberattack this month still forced Colonial Pipeline Co. to shut down its 5,500-mile system, which supplies nearly half the fuel used along the U.S. East Coast. The weeklong disruption led to panic buying at gas stations and an outcry for more stringent and enforceable cybersecurity measures for the oil and gas industry.

Colonial defended many of its cybersecurity actions in a statement to E&E News yesterday, pointing out that it "had many best practices in place" before the May 7 attack by the DarkSide ransomware gang.

Advertisement

But the fact that the suspected Russia-based hackers managed to break into the Georgia company’s networks raises questions about whether the voluntary security standards for U.S. pipelines are up to the task of protecting critical infrastructure from cyberthreats — even if followed to the letter.

Colonial told E&E News that the company participates in "annual interview assessments" led by the Transportation Security Administration, the agency charged with overseeing pipeline security. The company added that the reviews "have historically included assessments of our cybersecurity controls."

Colonial spokesperson Meredith Griffanti said the company also participated in a Pipeline and Hazardous Materials Safety Administration virtual assessment last year, which included a review of its "cybersecurity capabilities."

The ransomware attack, which locked up Colonial’s computer files and demanded payment for the key, has renewed calls from some lawmakers and cybersecurity experts to reassess TSA’s ability to manage digital defenses of the roughly 2.7 million miles of U.S. pipeline networks. And a separate IT outage at Colonial yesterday — which briefly blocked some critical communications with shippers — highlights the challenges of recovering from severe cyberattacks (Greenwire, May 18). Colonial said those "network issues" did not disrupt fuel deliveries and were due to the "hardening efforts that are ongoing and part of our restoration process" rather than a malware reinfection.

The Government Accountability Office issued a blog post yesterday reviving long-held concerns over pipeline cybersecurity oversight by both the federal and private sector. The watchdog noted that three of its recommendations relating to pipeline cybersecurity workforce and risk management have yet to be implemented.

The explanations Colonial provided yesterday still leave basic questions about why an attack on the company’s IT systems, presumably including business and commercial networks, posed such a threat to operational technology (OT) systems that it had to shut its pipeline network. That shutdown decision was made "in the name of safety and in effort to protect the integrity of the OT systems," Griffanti said, even though the company did not believe there was a "high risk" that such networks would be harmed directly.

"But given the seriousness of the situation, however, we needed to take our OT systems offline and take the time to ensure that environment was secure before resuming operations," she said.

Griffanti said Colonial maintains "stringent segmentation between our IT and OT environments, as well as our policies that have for years blocked direct connection between OT and the internet."

Colonial also said it aligns its security strategy with "industry best practices" provided by the National Institute of Standards and Technology, other best practices from the American Petroleum Institute, and the MITRE ATT&CK framework, which maps out cyber-criminal behavior.

Griffanti said the company intends to share "appropriate information with regulatory bodies and trade organizations" following an investigation of the root cause and scope of the cyberattack.

She also touted Colonial’s involvement in several information-sharing programs, such as the Department of Energy’s Cybersecurity Risk Information Sharing Program, the Oil and Natural Gas Information Sharing and Analysis Center, and the National Cybersecurity and Communications Integration Center run by the Department of Homeland Security’s top cyber office.

"We receive alerts and valuable threat intelligence from many of these entities and incorporate them into our methodologies for incident response," Griffanti said.

Natural gas worries

While Colonial made significant investments and training in cybersecurity, it apparently did not anticipate the specific threats it could face from a wide-ranging ransomware attack, leaving a hole unplugged in its defenses.

Jonathon Monken, a principal with Converge Strategies LLC, a consultancy that advises clients on defense and recovery strategies, said gas pipelines could be forced to curtail or shut down operations if their commercial systems were tied up by ransomware hackers, simply because they might not be sure where their gas deliveries were going in every case. That could put payments in jeopardy, creating a potential major financial loss.

Before joining Convergence Strategies, Monken was a security official at the PJM Interconnection LLC, the grid operator for 13 Eastern and Midwestern states and the District of Columbia.

In 2018, PJM made an investigation of the security of gas deliveries for its gas-fired generators, a vital energy source for its system. The investigation was unusual because PJM was joined by a large pipeline that agreed to join in the threat analysis — a level of grid-gas cooperation that isn’t common, Monken said in an interview.

PJM spokesperson Jeffrey Shields said the goal was to "test the resilience of the natural gas pipeline system to respond to an extended outage due to a physical or cyberattack and then to analyze the impact of such an extended outage on reliability of the PJM system." The analysis results are confidential, but were shared with key federal agencies, he added.

"The analysis deliberately stressed the system to find the tipping point at which issues would arise and to identify key drivers of reliability risk," Shields said. "Overall, the analysis found no immediate threat to the PJM system. It is reliable and will remain reliable into the future."

Monken said natural gas industry leaders are correct in noting that pipeline operating systems are not an easy target for hackers.

Pipeline networks rely on a series of compression stations along the route to maintain pressure and gas flow.

Operators sent to run a compression station manually in an emergency rely on computer information and local human-machine interface terminals that are maintained on-site, not shared among many stations. So to take down a pipeline network’s operating system requires imagining a team of terrorists able to plug infected jump drives into each of the compression stations along the route — much like the infamous Stuxnet cyberattack bypassed defenses at an Iranian nuclear facility over a decade ago.

The PJM study "rightly assessed the risk to be low," Monken said. "The opportunity to ‘brick’ a group of stations at the same time is virtually nil, as long as they aren’t networked together."

"But that’s not the end of it," he added.

The commercial risk remains if a pipeline with a compromised IT system tried to keep operating without the huge amount of data required to manage and fulfill orders and track and record payments. Without that record, a pipeline could not be certain of getting paid, Monken said.

"Am I literally pushing gas to customers without being sure I’d be compensated? The losses might not be calculable," he said.

Colonial said financial risk was not part of its decision to shut down. Its statement indicated that it apparently has not fully restored its invoicing system, but it resumed operation anyway.

Griffanti added that Colonial "has the ability to run parts of its system manually" in the event of a cyber or other emergency. But she also pointed out that "we have restarted our entire pipeline system, and product delivery has commenced to all markets we serve."

Infrastructure security officials said protecting against such a complex threat calls for development of a "design basis threat," a carefully worked-out scenario for assessing a potential attacker’s capabilities and tactics, in light of a network’s most critical vulnerabilities.

There is not an established threat definition process for gas pipelines, Monken said. Each pipeline makes its own assessments and response. "It’s up to them," he said.

One lesson from the Colonial attack is that pipelines and the generators that depend on them for fuel "need a much more realistic assessment of the cyber risks they face," Monken added.