06:40 PM
Connect Directly

How Incident Response Fails In Industrial Control System Networks

Experts say a solid incident response plan is the best way to minimize the damage of a cyberattack--but IR isn't so simple for the ICS/SCADA world.

Worries of eventual cyberattacks on utilities as well as chemical and other industrial sites have intensified in the wake of the recent attacks that led to a power blackout in western Ukraine. But the key element in mitigating the damage from a cyberattack -- an incident response plan -- is something that many industrial sites just don't have in place.

Conventional incident response procedures don't neatly map to the ICS/SCADA environment, either. Responding to an attack on an industrial control system (ICS) comes with challenges the pure IT environment typically does not face, and some of the standard IR steps just don't translate to a power plant or manufacturing plant, according to Chris Sistrunk, senior ICS security consultant for Mandiant/FireEye.

Industrial sites under NERC/CIP (North American Electric Reliability Corporation's Critical Infrastructure Protection) and Chemical Facility Anti-Terrorism Standards (CFATS) regulations have IR plans, he notes, but there are still many other ICS/SCADA organizations that do not fall under those regs and lack IR plans, including some in the water, manufacturing, and oil & gas industries.

Uptime and availability -- think electricity and other disruption-averse services -- are king in the ICS/SCADA space, as is physical (life and limb) security. That's why operators of those networks rarely apply vendor patch updates for security bugs and other issues: if a patch could potentially disturb an existing system configuration, or require any downtime or disruption, it's not likely to be installed.

"If a critical system has a virus and it hasn't actually affected the system, they may not do anything about it," for example, Sistrunk says. But if it's an engineer's laptop, it most likely will get the standard cleanup in IR, he says.

A programmable logic controller (PLC) cannot be re-imaged the same way a laptop can be, he says. An industrial system typically doesn't collect logs of events like a conventional computer does, either, and if it does, the logs may not feed to a SIEM, he points out. "Some devices don't have syslog or other types of logs," Sistrunk says.

He advocates network monitoring here, using Netflow packet capture, for example. If industrial networks aren't monitoring for the latest threats, they may not know for sure if they've been hit by a Black Energy or Havex malware attack, for example.

Network security monitoring could be set to detect known indicators of compromise for a specific attack. Sistrunk and colleague Rob Caldwell last year at the S4x15 ICS/SCADA industry conference demonstrated a set of open source network security monitoring tools from the open source Security Onion Linux suite, including Wireshark, NetworkMiner, Bro, and Snorby.

Ralph Langner, founder of Langner Communications and a renowned Stuxnet expert, says many industrial firms don't have a firm grasp on their network and system environment. "They don't have specific insight into digital dependencies and network connections," he says. And a solid IR plan requires knowing your environment and how best to contain a threat or infection there, he says.

The stakes are even higher when there's a physical consequence in an attack, he says. "You now have to coordinate with process engineers. You cannot simply reboot a control system. It's going to affect the process and might make things worse with forensics efforts.

"The plant manager will not let you mess around with all of those digital systems and reboot," Langner says. "Incident response is very difficult and very challenging in a cyber-physical environment. Nobody has procedures" for it, he says.

Langner says the key is training plant and industrial site operators on how to thwart cyberattacks. 

The good news is that there are some IR resources for the industrial space. The National Institute of Standards and Technology's SP800 Series document drills down on control systems, for example. Sistrunk says those guidelines are a good jumping-off point for unregulated organizations.

Eric Knapp, chief technical advisor for Honeywell's Industrial Cybersecurity Center, says the forensics information that investigators need from an attack on an ICS/SCADA system typically isn't available to them. "Control system software logging data is not what [they] are interested in," he says. "If you don't understand the control process system, the information has no meaning. Can you make logs that security people care about?"

In most IR engagements in industrial networks today, the victims aren't able to determine where the attacks came from nor how to protect themselves again in the future. "They don't have a well-documented way to keep track of incidents that happen," such as where, when, and how an infection came into the network, says FireEye/Mandiant's Sistrunk, who says he's heard this time and again from forensics investigators.

"We know the vulns are there and the exploits are there. Now the malware is eventually going to come," says Sistrunk, who in June blogged about IR challenges to ICS/SCADA.

If an industrial network operator suspects a breach, know who to call, he says. "The specialized IR team will need to include management, legal, engineers, technicians, safety specialists, vendors, and ICS security experts during IR activity," he wrote.

According to Sistrunk, the team's responsibilities include:

·         "Determining what to do about the threat and when

·         Restoring the ICS as safely and quickly as possible, without destroying forensic evidence

·         Reviewing any available logs and other forensics data that will be critical in piecing together what happened

·         Determining the root cause of the incident and providing recommendations for improving the ICS security to help prevent future incidents"


Related Content:


Kelly Jackson Higgins is Executive Editor at She is an award-winning veteran technology and business journalist with more than two decades of experience in reporting and editing for various publications, including Network Computing, Secure Enterprise ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Strategist
2/1/2016 | 10:55:28 AM
Useful Article
Hi, Kelly -- a very useful article for the C-suite.  It is succinct, to the point and offers some excellent ideas.  Also, it is a useful "training guide" for the CIO and the IT side of the business to understand the difficulties with ICS and incident response.  Well done!  Ernie Hayden
User Rank: Ninja
1/31/2016 | 11:08:21 PM
Re: Incident response
That's the lessons learned phase of incident response and I think thats more of an after thought than part of the process. Incident response is more about identifying, mitigating, and damage control. Once the situation is under control you can then postulate how to better prevent it in the future. But that shouldn't occur until the SIRP has been closed.
User Rank: Ninja
1/31/2016 | 10:46:39 AM
Re: Citing the proper NIST guide
Thanks for the link to the Revision 2 update.  I noted R2 has some great new information, including:

- New tailoring guidance for NIST SP 800-53, Revision 4 security controls including the introduction of overlays

- An ICS overlay for NIST SP 800-53, Revision 4 security controls that provides tailored security control baselines for Low, Moderate, and High impact ICS

There's a whole article right there on the overlays and security control baselines!
User Rank: Ninja
1/30/2016 | 11:01:49 AM
Re: Citing the proper NIST guide
Great. Thanks for the link, I was wondering about this.
User Rank: Ninja
1/30/2016 | 11:00:55 AM
Incident response

For me incidence response it about prevention,  you would analyze the situation, understand how it happened and define counter measure then predict the future based on the trend you experienced.
User Rank: Apprentice
1/29/2016 | 8:33:24 PM
Re: Citing the proper NIST guide
An Analysis of a Military's Information Security Program null via amazon
Kelly Jackson Higgins
Kelly Jackson Higgins,
User Rank: Strategist
1/29/2016 | 3:53:07 PM
Re: Citing the proper NIST guide
Thanks for the link!
User Rank: Apprentice
1/29/2016 | 12:59:20 PM
Re: Citing the proper NIST guide
Yes, but NIST SP800-82 Revision 2 was released last year. Use it instead.
Dr. Orebaugh
Dr. Orebaugh,
User Rank: Apprentice
1/29/2016 | 9:31:16 AM
Citing the proper NIST guide
Hi, I wanted to make sure you cite the proper NIST 800-series guide for ICS.  It is NIST 800-82 Guide to Industrial Control Systems Security 

Locate it here:
Government Shutdown Brings Certificate Lapse Woes
Curtis Franklin Jr., Senior Editor at Dark Reading,  1/11/2019
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Current Issue
The Year in Security 2018
This Dark Reading Tech Digest explores the biggest news stories of 2018 that shaped the cybersecurity landscape.
Flash Poll
How Enterprises Are Attacking the Cybersecurity Problem
How Enterprises Are Attacking the Cybersecurity Problem
Data breach fears and the need to comply with regulations such as GDPR are two major drivers increased spending on security products and technologies. But other factors are contributing to the trend as well. Find out more about how enterprises are attacking the cybersecurity problem by reading our report today.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2019-01-16
An issue was discovered in NTPsec before 1.1.3. Because of a bug in ctl_getitem, there is a stack-based buffer over-read in read_sysvars in ntp_control.c in ntpd.
PUBLISHED: 2019-01-16
An issue was discovered in NTPsec before 1.1.3. process_control() in ntp_control.c has a stack-based buffer over-read because attacker-controlled data is dereferenced by ntohl() in ntpd.
PUBLISHED: 2019-01-16
An issue was discovered in NTPsec before 1.1.3. An authenticated attacker can cause a NULL pointer dereference and ntpd crash in ntp_control.c, related to ctl_getitem.
PUBLISHED: 2019-01-16
An issue was discovered in NumPy 1.16.0 and earlier. It uses the pickle Python module unsafely, which allows remote attackers to execute arbitrary code via a crafted serialized object, as demonstrated by a numpy.load call.
PUBLISHED: 2019-01-16
An issue was discovered in NTPsec before 1.1.3. An authenticated attacker can write one byte out of bounds in ntpd via a malformed config request, related to config_remotely in ntp_config.c, yyparse in, and yyerror in ntp_parser.y.