Breaking critical infrastructure in a quest for solutions

By Blake Sobczak | 08/12/2015 08:10 AM EDT

LAS VEGAS — The 55-gallon steel drum popped like a firecracker as it collapsed, drawing applause from the few dozen onlookers who came to see its destruction.

LAS VEGAS — The 55-gallon steel drum popped like a firecracker as it collapsed, drawing applause from the few dozen onlookers who came to see its destruction.

Jason Larsen, principal security consultant at IOActive Inc., cracked a smile as he glanced back to the graph displayed on his laptop. His shirt read, "Keep calm and don’t f**k it up." He had kept calm and managed to break the model bootleg whiskey still from his computer.

Larsen studies how to hack into and remotely damage industrial control systems, the complex and often delicate networks that run everything from distilleries to the power grid. His latest research uses algorithms to pick out physical weak spots from reams of sensor data, brushing away the "fog of war" shrouding faraway industrial networks.

Hackers in the industrial control system “village” at DEF CON were challenged to spin the centrifuges by breaching this model nuclear facility’s computer defenses. | Photo by Blake Sobczak.

"Just because you’ve hacked into a control system doesn’t mean that bad things necessarily immediately follow — not every button explodes the plant," Larsen told EnergyWire. "It takes a focused effort, but the trend is we’re seeing more and more people apply that focused effort."

Still, Larsen sought to distance himself from a message of "everything is broken, please run screaming."

"At this point, we know what the problem is," he said, suggesting that electric utilities and other industrial control systems (ICS) operators’ attitudes had matured toward the risks in their systems.

Jennifer Steffens, CEO of IOActive, a cybersecurity research and consulting firm that has uncovered several recent high-profile vulnerabilities in both industrial systems and "Internet of things" devices, agreed that "it’s really time we start focusing on some of the positive."

Last month, an IOActive researcher made headlines when, working alongside another security researcher from Twitter, he remotely hacked and controlled a Jeep Cherokee with a Wired reporter at the wheel. Days later, Chrysler issued a formal recall for 1.4 million cars affected by the bug, out of what the company called "an abundance of caution."

"There’s a lot of change that has happened and can happen," Steffens said. "Not everybody can necessarily agree on what the best solution is — but if everybody keeps focusing on discussing solutions, we’ll go a lot further than just saying, ‘It’s broken, high five, good luck.’"

Gaming out a watershed moment

The DEF CON and Black Hat cybersecurity conferences in Las Vegas last week featured thousands of hackers who pick apart flaws in computer systems or find ways to fix them, for a living or just for fun. A small but growing group within that community has begun exploring the industrial processes and controllers that people rely on for everyday services like water and electricity.

At DEF CON, many of these specialists hung around the "ICS Village," where they could play Tetris using industrial protocols or try their hand at "The Grid: A Multiplayer Game of Destruction."

The village also featured a "supersecret" nuclear facility that visitors could hack into. The tone of the game was tongue-in-cheek, but it alluded to one of the only real cyberattacks known to have damaged industrial control environments — the Stuxnet worm that infected Iranian nuclear centrifuges five years ago.

Jason Larsen
Jason Larsen, principal security consultant for IOActive Inc., prepares to demonstrate what remote cyber-physical damage could look like. | Photo by Blake Sobczak.

Several researchers at IOActive called Stuxnet a watershed moment for critical infrastructure security. Electric utilities grew more aware of the potential to exploit their operational systems — even those thought to be isolated from the Internet and thus less susceptible to remote attack. Larsen, who started hacking into smart meters as far back as 2008, said manufacturers slowly began improving the designs. Where once it might have taken a single researcher a week to come up with a laundry list of vulnerabilities, in recent years a pair of researchers probing a smart grid device for two weeks or more might come up with only one or two problems.

While consumer-level products are starting to incorporate more security, much of the core equipment in ICS networks has little defense. Cybersecurity researchers Colin Cassidy, Eireann Leverett and Robert M. Lee teamed up to find nearly a dozen critical vulnerabilities in common Ethernet "switches" that manage network traffic on manufacturing floors or power plants. They unveiled their findings at both Black Hat and DEF CON, having already disclosed the bugs to the impacted companies so they could roll out fixes.

‘We want the vendors’ help’

Historically, many hackers have had trouble calling attention to flaws in computer products. When they raise issues privately, they are often ignored. Some have even faced threats of legal action.

On Monday, a lengthy blog post from the chief security officer at software giant Oracle Corp. warned would-be hackers that reverse-engineering Oracle’s code to suss out vulnerabilities violates customers’ license agreements. "Please do not waste our time on reporting little green men in our code," Oracle’s Mary Ann Davidson said. Her post was later taken down after drawing backlash from the online security community.

But Lee said he was impressed by the responses from the companies caught up in the team’s latest discoveries, particularly Opengear, which offers ruggedized Ethernet gateways for monitoring remote sites like substations or wind farms.

"We want the vendors’ help — it’s great when Opengear comes back and says, not only are we not going to sue you, but here have another [switch]: Find these vulnerabilities."

Robert Waldie
, distinguished engineer at Opengear, said he tries to stay on good terms with the technical community, noting that "security isn’t something that you do once."

"As a vendor it’s about making security a priority and realizing it is an ongoing process, and that educating our customers is an ongoing process," he said in an interview.

When Lee’s team first approached Waldie in February, he welcomed its input and worked out a fix for the product within a few weeks. (The flaw allowed attackers to shake out user credentials from the switch in question, potentially granting access to all connected devices.)

Crumpled barrel
The barrel in Jason Larsen’s hacking demonstration crumpled with a bang during a combined steam and vacuum collapse. | Photo by Blake Sobczak.

"I know that some of the hackers like to make fun of vendors and some of the vendors might have a bit of mistrust for the security researchers, but it doesn’t have to be that way," Waldie said, adding, "I think we’re all on the same side."

If manufacturers get combative, Lee pointed out that it’s still sometimes possible to defend unpatched products. For instance, operators can monitor the information flowing through ICS networks and keep logs of network traffic, training their engineers to look out for anomalies.

"It can really be that simple," Lee said. "We don’t need the vendors — we don’t need anybody else to defend our environments."

The other manufacturers with weak switches included Siemens, General Electric Co. and GarrettCom. But just because equipment is "broken" from a security standpoint does not mean the systems underlying it are doomed, a point Lee and other cybersecurity experts sought to drive home at the conferences.

Despite threats like Stuxnet and, more recently, a cyberattack that wrecked a furnace in a German steel mill last year, there have been no publicly disclosed cases of hackers causing physical damage to a critical system in the United States.

"It’s easy to forget that every time you turn on the faucet, clean water comes out," noted Adam Firestone, president of cybersecurity firm Kaspersky Government Security Solutions Inc. "Things aren’t built thoughtlessly. While we all want more secure control systems, we also don’t want less reliable ones."

Firestone recommended critical infrastructure companies gain a strong enough grounding in cybersecurity to take the risk-based approach favored by federal pacesetters such as the National Institute of Standards and Technology.

"You don’t have to understand every 1 and 0; you just have to understand how it interferes with your world and how you can mitigate that interference," he said of cyberthreats to control systems.

Steffens of IOActive encouraged critical infrastructure firms to continue to allow a more open dialogue, urging them not to turn anyone away "just because they have the word ‘hacker’ in front of their title."

"When the security researcher is knocking at your front door trying to give you information about something that’s vulnerable, or a potential fix, I’m pretty sure they’re the good guys," she said. "The bad guys don’t want you to know what they know."