Water utility hack poses power grid warning

By Christian Vasquez | 02/23/2021 07:20 AM EST

A first-of-its-kind cyberattack on a water treatment facility in Florida is now ringing alarm bells for U.S. electric utilities — many of which face the same uphill battles to defend themselves against hackers, experts say.

A recent cyberattack on a water treatment facility in Florida holds lessons for electric power utilities, cybersecurity experts say. An electric substation is pictured outside Bellingham, Wash.

A recent cyberattack on a water treatment facility in Florida holds lessons for electric power utilities, cybersecurity experts say. An electric substation is pictured outside Bellingham, Wash. Robert Ashworth/Flickr

A first-of-its-kind cyberattack on a water treatment facility in Florida is now ringing alarm bells for small U.S. electric utilities — many of which face the same uphill battles to defend themselves against hackers, experts say.

Early this month, an attacker tried to poison the water supply of Oldsmar, Fla., by hacking into a utility control network and raising levels of sodium hydroxide — a dangerous chemical used in water treatment — to over 100 times their normal concentrations. An operator who noticed the hack in real time stopped the chemicals from reaching the water supply.

The attack has sparked federal investigations and calls for tighter security at water utilities and other critical infrastructure providers (Energywire, Feb. 10).

Advertisement

Now, officials are pressing the federal government for more details and raising concerns about the parallels between the power sector.

Last Wednesday, Sen. Mark Warner (D-Va.), chairman of the Senate Select Committee on Intelligence, sent a letter to the FBI and EPA requesting an update on the status of the criminal probe and the agencies’ plans to share information about the threat with other industries.

Warner said the unsuccessful hacking attempt may illustrate "broader security weaknesses … within critical infrastructure sectors reliant on similar industrial control systems," such as health care and energy.

"This incident has implications beyond the 15,000-person town of Oldsmar," Warner wrote.

Cybersecurity experts note that hundreds of rural or municipal electric utilities face many of the same problems that besiege the water sector, including tight budgets and lax security standards.

Smaller power providers often lack the staffing to focus on cybersecurity, said Leo Simonovich, vice president and global head of industrial cyber and digital security at Siemens Energy AG. Municipal utilities or rural electric cooperatives might have only one person dedicated to security across the entire organization, and experts familiar with the operational technology that manages the power grid and other critical infrastructure are rare compared to information technology specialists.

"So many operators are dual hats" — protecting both IT and OT infrastructure — "and don’t have the expertise or the right set of tools to detect these kinds of attacks," Simonovich said.

This spells trouble as the grid undergoes a rapid shift to renewable energy sources that are more digitally interconnected, Simonovich said, noting that a compromised smaller utility could provide a pathway into a larger electricity system.

"Utilities need to do a better job of keeping up with this energy transition and are, frankly, having a hard time," Simonovich said.

A recent report by Moody’s Investors Service found that not-for-profit utilities such as municipal power providers and co-ops have weaker cybersecurity defenses compared with their bigger, investor-owned counterparts, largely because they lack resources to shore up defenses against hackers (Energywire, Nov. 5, 2020).

Moody’s said smaller utilities are also slow to invest in basic cybersecurity practices like multifactor authentication, which adds an extra security step — like a text message — to email and computer logins. Such safeguards can make it much harder for hackers who steal a username and password to break into a system.

Some government initiatives are aimed at helping smaller utilities improve their digital defenses, and the Biden administration has pledged to make security U.S. critical infrastructure a top priority following a slew of high-profile breaches.

Yesterday, Department of Homeland Security Secretary Alejandro Mayorkas announced that the agency’s cyber arm plans to "urgently" accelerate state and local cybersecurity defenses, including through potential grant programs.

"Cybersecurity is more important than ever, and we will build on the Department’s excellent work as we transform our whole-of-government approach to tackle the challenge we face as a nation," Mayorkas said. "This week is just the beginning of a series of actions DHS will pursue nationally and internationally to improve cybersecurity at all levels."

The Department of Energy’s cyber office last year announced a $12 million investment to support developing cybersecurity tools for distribution and municipal utilities by 2023 (Energywire, Sept. 28, 2020). Both the American Public Power Association, which represents more than 2,000 community-owned electric utilities, and the National Rural Electric Cooperative Association, the trade association for 900 local rural electric co-ops, received $6 million.

Alex Hofmann, vice president of technical and operations services at APPA, said that the industry group is "in a unique position to help our members ‘up their game’ on cybersecurity."

DOE "recognizes this and late last year joined us in a cooperative agreement to fund initiatives that will help our members to develop specifications for and deploy hardware and software innovations to protect their critical systems," Hofmann said. "Additionally, we continue to provide educational offerings to our members as well as opportunities and forums to share best practices."

Stephen Bell, a spokesperson for NRECA, said that "one of the strengths of electric cooperatives is their culture of working together to manage growing threats, promote continuous improvement, and develop solutions that keep the grid secure."

Bell said DOE-backed programs boost co-ops’ cyberdefenses by supporting security assessments, tabletop exercises and training.

"As threats and threat actors evolve, cooperatives improve their capability to defend against them," Bell said. "Maintaining the security and resilience of the grid requires a flexible and risk-based approach that draws on a variety of tools, resources and options."

Internet-facing dangers

Smaller utilities don’t have to follow grid cybersecurity rules set by the North American Electric Reliability Corp., also known as critical infrastructure protection (CIP) standards.

"The standards really only apply to some of the utilities, not all of them," said Patrick Miller, U.S. coordinator for the Industrial Cybersecurity Center. Most distribution utilities aren’t considered part of the "bulk electric system" and are exempt, though Miller noted that there are some that follow CIP standards voluntarily.

But even for utilities that must adhere to CIP standards, some requirements, like multifactor authentication — which Miller said could have stopped the Florida water attack — don’t apply to "low-impact" facilities.

"The things that we’re talking about that would have stopped [the Oldsmar intrusion] from happening actually only apply to a very small number of utilities," Miller said.

Local authorities said the hacker in Oldsmar evidently got in through TeamViewer, a remote access program that gave the attacker a way into the water treatment facility’s controls. Miller noted that the same software is ubiquitous in the energy sector: "It’s everywhere, and it’s rarely secure," he said.

"Just do a Shodan scan for TeamViewer, and you’ll be just mortified," Miller said, referring to an online search tool that can be used to find internet-facing critical infrastructure devices.

But Miller also pointed out that utilities are protected by more than just cyberdefenses. Many specialized controllers that manage essential operations can only accept a limited number of commands and will "toss out" other inputs that don’t meet those set guidelines, he said. A hacker would need to know those boundaries and the types of equipment used at a given facility, knowledge that’s often limited to insider threats or nation-backed hackers, he said.

Michael Arceneaux, chief operating officer of the Association of Metropolitan Water Agencies and managing director of WaterISAC, said the lessons from the Oldsmar attack could reverberate in other industries.

"With electricity, it’s not uncommon that there’s a water and power department, and the definition between IT and OT is similar for any sector, really," he said. "So if the hackers use something as simple as TeamViewer, it could be used in any sector."

The attack on the water facility through such popular software points to a larger problem with critical infrastructure and internet-facing devices, said Joe Slowik, a senior security researcher at DomainTools.

"One of the first lessons or observations to take is that if something’s online and accessible, people are going to find it," Slowik said.

Utilities must balance security and the convenience that comes with increased connectivity, he added.

"While remote connectivity is necessary, having direct remote connectivity to control system equipment isn’t necessary," Slowik. "What’s that best balance between something that’s actually useful for the operators versus something that is just immediately accessible and reachable?"

Even smaller utilities that are facing budget constraints can take proactive defenses against cyberattacks, says Dave Forbes, an analytics leader focused on infrastructure at Booz Allen Hamilton.

Emphasizing the need for a basic understanding of smaller utilities’ computer networks can help workers prioritize the most vulnerable parts of the system.

"Having a culture of cybersecurity at an organization is sometimes step one," Forbes said.

Reporter Hannah Northey contributed.