Sluggish movement on power grid cyber security

One year after the worst blackout in US history drew attention to the fragility of the North American power grid, progress on protecting the grid from computer intrusions has been slow in coming.

This week the North American Electric Reliability Council (NERC) - the not-for-profit industry group responsible for keeping electricity flowing throughout the United States and Canada - released a list of measures taken to shore up electric grid reliability in the year since the 14 August, 2003 northeast blackout, when a sagging high voltage line in Ohio cascaded into a failure that left 50 million people in eight states and a Canadian province without power.

Topping the cyber security portion of NERC's list, the council recently voted to renew for one year a set of rules, called the Urgent Action Cyber Security Standard 1200, that sets minimum cyber security requirements for utility companies in the US and Canada. But that standard - by coincidence enacted the day before the blackout - is relatively small in scope: it applies only to utility control centers, and specifically exempts substations, power plants, and the remotely-operated control systems and relays sprinkled throughout the grid. "It doesn't go far enough," acknowledges Tom Kropp, manager of enterprise information security at the Electric Power Research Institute, an industry think tank. "It is very, very limited in what it applies to."

The reason the standards don't reach further, says NERC cyber security chief Lou Leffler, is a pragmatic one: the industry didn't want to impose requirements on itself that it couldn't meet. "There are some area where the technology doesn't exist at this point in time to provide all the protection that we'd like," says Leffler.

Concern in Washington

SCADA (Supervisory Control and Data Acquisition) systems, in particular, allow utilities to remotely control and monitor generation equipment and substations over phone lines, radio links and, increasingly, IP networks. That makes them an obvious target for cyber attackers. But some existing SCADA systems can't economically be retrofitted with encryption or authentication technology without introducing unacceptable latency into the link, i.e., slowing down communications, Leffler says, voicing a sentiment heard often in the industry. "The devices to provide that kind of encryption, certification or what-not just do not exist," says Leffler.

In the wake of the northeast blackout, the narrow focus of the industry's cyber security standard even drew the attention of presidential candidate John Kerry, who, in his capacity as US Senator, asked the chairman of the Federal Energy Regulatory Commission to explain the omission of power plants and control systems from the NERC standard, and from a proposed federal standard that was never ratified.

"As you know, the increased integration of generation, transmission and distribution, and control and communications functions, makes the security of the power grid increasingly dependant on the security of its components," Kerry wrote, in a letter dated 8 September, 2003. "I strongly support your efforts to increase the protection of our electric power infrastructure, but I am concerned that the very systems used to control the safe and reliable operation of power generation have been excluded from the rule."

Responding to Kerry, FERC chairman Patrick Wood wrote that the failure of individual power plants is not a threat to the grid as a whole, and echoed NERC's position that control systems, while "clearly vulnerable points," could not be secured with cost-effective off-the-shelf solutions, and were therefore properly omitted from security standards.

Scattered Incidents

If the current rules are limited, observers expect more from the sequel: NERC is working on a new, permanent cyber security standard expected to be in place by the time Urgent Action 1200 expires, one year from now. "What NERC wanted to do with the current one is to set a threshold, give it a try, get the industry comfortable with it and then move on to a more stringent standard," says Kropp. "I think the intent is for [the next standard] to go farther ."

"It is my understanding that it will cover the SCADA connectivity, to the extent that there is existing technology to do that," says NERC's Leffler. "I hope that the industry, that the vendors, can develop cost effective security solutions for all of our control systems. I think that is one of the intents."

To that end, there are myriad efforts underway to develop SCADA security solutions. Working with NERC, the Department of Energy has produced written guidelines to help utilities voluntarily tighten their control systems, and the department funds a well-regarded National SCADA Testbed at the Idaho National Engineering and Environmental Laboratory. This year also saw congressional hearings and a GAO report on the issue of control system cyber security, and an announcement from at least one sizable computer security vendor jumping into the SCADA security market. "There's also a funded, focused effort within the Department of Homeland Security to address this," says Joe Weiss, a control system cyber security consultant at KEMA. "That is a big deal."

Reported cases of power grid cyber security incidents are rare, but not unheard of. In the most dramatic incident, early last year the Slammer worm penetrated a private computer network at Ohio's Davis-Besse nuclear power plant and disabled a safety monitoring system for nearly five hours. According to an industry report, the same worm downed a utility's critical SCADA network after penetrating a control center network through a VPN connection, and, separately, disrupted a power company's SCADA traffic by consuming bandwidth on a shared facility.

The northeastern blackout was not causes by cyber attack, but a software bug contributed to its scope. A silent failure of the alarm function in an Ohio utilities computerized Energy Management System (EMS) is listed in the joint US-Canada report on the blackout as one of the direct causes of the outage. In April the makers of the software, GE Energy, told SecurityFocus the failure was caused by a race condition in the EMS software that has since been patched.

In all, utilities have had enough work to do on basic reliability, that cyber security has taken a back seat over the last 12 months, says EPRI's Kropp. "What I think people have done is they've taken the reliability aspects and the maintenance aspects more seriously," Kropp says. "I think companies are looking at the tools they have to monitor the grid. They're taking much more seriously the preventive maintenance aspects, like cutting tree branches, and making sure the transmission lines are intact and in good shape... They've been taking a second look at their software to make sure there aren't any problems with it. Those all had to be done before they could start worrying about security."