03:24 PM
John Foley
John Foley
Connect Directly

7 Lessons: Surviving A Zero-Day Attack

Pacific Northwest National Laboratory CIO Jerry Johnson takes you inside the cyber attack that he faced down--and shares his security lessons learned.

When Pacific Northwest National Laboratory detected a cyber attack--actually two of them--against its tech infrastructure in July, the lab acted quickly to root out the exploits and secure its network. PNNL then did something few other cyber attack victims have been willing to do. It decided to talk openly about what happened.

The lab's CIO, Jerry Johnson, last week provided a detailed accounting of the cyber attacks. Speaking at the IW500 Conference in Dana Point, Calif., Johnson described how intruders took advantage of a vulnerability in one of the lab's public-facing web servers to plant a "drive-by" exploit on the PCs of site visitors, lab employees among them. For weeks, the hackers then surreptitiously scouted PNNL's network from the compromised workstations.

Simultaneously, a spear-phishing attack hit one of the lab's major business partners, with which it shared network resources. This second group of hackers was able to obtain a privileged account and compromise a root domain controller that was shared by the lab and its partner. When the intruders tried to recreate and elevate account privileges, this action triggered an alarm, alerting the lab's cybersecurity team.

Within hours, the lab made the decision to disconnect its network in order to sever the hackers' communications paths and contain any further damage. Over the July 4 weekend, while the rest of us were grilling burgers, PNNL's security team conducted cyber forensics, reconstructed the domain controller, re-imaged systems, and restored network services that had been taken off line.

[ Want more lessons learned from InformationWeek 500 winners? See 20 Innovative IT Ideas To Steal.]

Who was behind the attacks? That's one question CIO Johnson won't discuss. But it's worth noting that Dept. of Energy facilities were reportedly targets in the series of cyber attacks known as Operation Shady RAT that were carried out against more than 70 companies, defense contractors, and government agencies over the past few years. Based on the available evidence, some experts have speculated that those attacks originated in China.

At the IW500 conference, in a session titled "Anatomy of a Zero-Day Attack," Johnson was candid about how the lab responded to the intrusions. He also shared the following list of lessons learned from the experience:

1. There's danger in multi-level security environments. The lab had a well-protected IT security perimeter, but the attacks made it through anyway. An advocate of "defense in depth," Johnson is putting increased emphasis on protecting the data itself.

2. Purge legacy, minority technologies. The Web server in the first attack was based on a little-used technology at the lab, Adobe ColdFusion. Such out-of-sight, out-of-mind technologies are inherently vulnerable because they don't get the same degree of attention as an organization's primary platforms.

3. Monitor cybersecurity events 24 x 7. Advanced persistent threats like those that hit PNNL are just that--persistent--and require constant vigilance. Across federal government, agencies are investing in "continuous monitoring," with a goal of obtaining a near real-time view into the status of computer system security.

4. Maintain a core forensics capability. If your network does get hacked, security teams must be able to reconstruct events and assess the damages. What you learn can help prevent a relapse.

5. Include a senior project manager on your response team. Responding to a breach requires not only attention to detail and carefully coordination, but an ability to engage top management at a moment's notice and, if necessary, escalate decision making.

6. Be prepared to call for help, and don't wait. You may need to bring in security experts, business partners, law enforcement, or other outsiders. At PNNL, Johnson alerted the public affairs office, in order to prepare for the inevitable media inquiries.

7. Have an emergency communications continuity plan. When PNNL pulled the plug on its network, the hackers lost their ability to inflict further damage. Unfortunately, the decision also meant that lab employees lost network services, including email and voice mail. Be prepared for that eventuality by sharing cell phone numbers and alternative email address in advance.

As Operation Shady RAT and a similar cyber attack on Google and other companies demonstrate, the risks are complex and growing. Johnson agreed to talk about it as a way of helping other organizations bolster their defenses. For that, he deserves a tremendous amount of credit. Secrecy is the norm in the wake of a cyber attack, but openness will lead to better preparedness.

Join us for GovCloud 2011, a day-long event where IT professionals in federal, state, and local government will develop a deeper understanding of cloud options. Register now.

John Foley is editor of InformationWeek Government.

To find out more about John Foley, please visit his page.

Comment  | 
Print  | 
More Insights
Register for Dark Reading Newsletters
White Papers
Current Issue
E-Commerce Security: What Every Enterprise Needs to Know
The mainstream use of EMV smartcards in the US has experts predicting an increase in online fraud. Organizations will need to look at new tools and processes for building better breach detection and response capabilities.
Flash Poll
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
Published: 2015-10-15
The Direct Rendering Manager (DRM) subsystem in the Linux kernel through 4.x mishandles requests for Graphics Execution Manager (GEM) objects, which allows context-dependent attackers to cause a denial of service (memory consumption) via an application that processes graphics data, as demonstrated b...

Published: 2015-10-15
netstat in IBM AIX 5.3, 6.1, and 7.1 and VIOS 2.2.x, when a fibre channel adapter is used, allows local users to gain privileges via unspecified vectors.

Published: 2015-10-15
Cross-site request forgery (CSRF) vulnerability in eXtplorer before 2.1.8 allows remote attackers to hijack the authentication of arbitrary users for requests that execute PHP code.

Published: 2015-10-15
Directory traversal vulnerability in QNAP QTS before 4.1.4 build 0910 and 4.2.x before 4.2.0 RC2 build 0910, when AFP is enabled, allows remote attackers to read or write to arbitrary files by leveraging access to an OS X (1) user or (2) guest account.

Published: 2015-10-15
Cisco Application Policy Infrastructure Controller (APIC) 1.1j allows local users to gain privileges via vectors involving addition of an SSH key, aka Bug ID CSCuw46076.

Dark Reading Radio