Attacks/Breaches
3/14/2014
11:58 AM
50%
50%

Target Ignored Data Breach Alarms

Target's security team reviewed -- and ignored -- urgent warnings from threat-detection tool about unknown malware spotted on the network.

Target confirmed Friday that the hack attack against the retailer's point-of-sale (POS) systems that began in late November triggered alarms, which its information security team evaluated and chose to ignore.

"Like any large company, each week at Target there are a vast number of technical events that take place and are logged. Through our investigation, we learned that after these criminals entered our network, a small amount of their activity was logged and surfaced to our team," said Target spokeswoman Molly Snyder via email. "That activity was evaluated and acted upon."

Unfortunately, however, the security team appears to have made the wrong call. "Based on their interpretation and evaluation of that activity, the team determined that it did not warrant immediate follow up," she said. "With the benefit of hindsight, we are investigating whether, if different judgments had been made, the outcome may have been different."

[Collaboration with competitors may be the key to slowing security threats. See Retail Industry May Pool Intel To Stop Breaches.]

Target arguably wasn't breached because it failed to invest in proper information security defenses. In fact, Snyder said the company had "invested hundreds of millions of dollars in data security, had a robust system in place, and had recently been certified as PCI-compliant." Likewise, the retailer apparently heeded multiple warnings from US-CERT -- part of the Department of Homeland Security -- about the increasing threat of POS-malware attacks against retailers.

Unusually for a retailer, Target was even running its own security operations center in Minneapolis, according to a report published Thursday by Bloomberg Businessweek. Among its security defenses, following a months-long testing period and May 2013 implementation, was software from attack-detection firm FireEye, which caught the initial November 30 infection of Target's payment system by malware. All told, up to five "malware.binary" alarms reportedly sounded, each graded at the top of FireEye's criticality scale, and which were seen by Target's information security teams first in Bangalore, and then Minneapolis.

Image credit: Jay Reed on Flickr.
Image credit: Jay Reed on Flickr.

When reviewing Target's log files, digital forensic investigators also found the November 30 alerts, as well as multiple alerts from December 2, all of which tied to attackers installing multiple versions of their malware -- with the alerts including details for the external servers to which data was being sent -- Bloomberg Businessweek reported. Later on December 2, attackers began siphoning 40 million credit and debit card numbers from POS terminals, as well as personal information on 70 million customers. Ultimately, they exfiltrated at least 11 GB of data, according to Aviv Raff, CTO of Israel-based cybersecurity technology company Seculert, which found one of three FTP servers to which the data was sent. From there, the data was transferred to a server hosted by Russian-based hosting service vpsville.ru.

Obviously, had Target's security team reacted differently, they might have contained what turned into a massive data breach. But the security team didn't even have to be in the loop. The FireEye software could have been set

Next Page

Mathew Schwartz served as the InformationWeek information security reporter from 2010 until mid-2014. View Full Bio

Previous
1 of 2
Next
Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
DarrenM555
50%
50%
DarrenM555,
User Rank: Apprentice
3/14/2014 | 12:20:45 PM
They ignored it?
Between this and the "thigh gap" fiasco, it's a wonder they keep any customers. I don't shop there very often but I'll certainly think twice about giving them any of my hard-earned money in the future.
JoeS149
100%
0%
JoeS149,
User Rank: Apprentice
3/14/2014 | 12:35:59 PM
Target Security team is inexperienced and or incompetent.
A competent IT and security individual  would have been in code red attempting to stop the attack. The fact the target "security team" did not recognize the threat shows a lack of technical understanding and/or experieence.

It has been a number of years  since I have done system security however a simple  thing to do is filter out all IP addresses outside of the needed range. Certain  countries(i.e. China, Russia) have been threats for years and years. The Target "security team" didn't understand this?


On the positive side, maybe now the non-tech world which is using technology to make money will spend more money on better security.
Somedude8
50%
50%
Somedude8,
User Rank: Apprentice
3/14/2014 | 1:01:57 PM
Re: Target Security team is inexperienced and or incompetent.
I am not so sure that IP filtering would have helped at with the infiltration, since the penetration vector was through a contractor, unless that HVAC contractor was in the blacklist, in which case they wouldn't have been able to do their jobs. IP filtering would of course not help with exfilatration.

This development really highlights the growing difficulty of filtering the signal from the noise in an age of exponentially expanding volume of data. Its like many of us are falling in to the same trap that amateur website owners often do: If everything is in all caps, people will read everything because all caps means its important right? I would not be at all surprised if the same people that evaluated the alarm mentioned in the article were also monitoring alarms from countless workstations and who knows what else. Doesn't surprise me at all that this got lost in the shuflle. But it still terrifies me!

This also underscores the near uselessness of the PCI spec. It is not a something to use to avoid a breach, its something to use to reduce the chance of a lawsuit. "Hey! We were PCI compliant! Its not our fault!"
Laurianne
50%
50%
Laurianne,
User Rank: Apprentice
3/14/2014 | 3:15:40 PM
Re: Target Security team is inexperienced and or incompetent.
"This also underscores the near uselessness of the PCI spec. It is not a something to use to avoid a breach, its something to use to reduce the chance of a lawsuit." True PCI is about covering your business. The retail data breaches are causing pain, but healthcare data breaches may someday make these look tame by comparison.
Thomas Claburn
100%
0%
Thomas Claburn,
User Rank: Moderator
3/14/2014 | 3:42:04 PM
Re: Target Security team is inexperienced and or incompetent.
I wonder whether this incident will help retailers understand that retaining credit card data is more trouble than its worth. "No Data" should become the next "Big Data."
hhendrickson274
50%
50%
hhendrickson274,
User Rank: Apprentice
3/14/2014 | 1:43:59 PM
These stories all present misleading or incomplete data with sensational titles
I don't know any more than what is in the various articles written about this, but everyone is some quick to jump on the Target team for reviewing and ignoring the alarms.  And articles like this with sensational titles don't help. That's really disingenuos without understanding the entire circumstances around the situation.  No meniton is made to the volume of alerts that may have been coming out of the FireEye system (or other systems they had deployed) to know if this was seen as normal noise or not.  Was that team used to seeing alerts similar to this that turned out to be false positives or of little significance? 


What I can fault them for would be not taking at least basic precautions like blocking outbound access to the IP that the malware was communicating with, and sending a sample off to their A/V vendor for analysis and inclusion in signature updates.  I can't say that either of those would have really made much of an impact, but I'm not sure how much business Target does with users in Russia to understand why they would feel outbound connections from their POS to a Russian based IP wouldn't be suspicious.  Maybe they did some of these things, I have no idea. 

I guess what my point is, let's not rush to judgement before we have all the facts.  They are only coming out in dribs and drabs at this point.  Hindsight is 20/20 and it's easy to be critic.  I'd rather we tried to be constructive and learned from this event.
BGREENE292
100%
0%
BGREENE292,
User Rank: Apprentice
3/15/2014 | 5:53:42 PM
Re: These stories all present misleading or incomplete data with sensational titles
.
hhendrickson274 said, "... these stories all present misleading or incomplete data with sensational titles..."
 
 
NOT HARDLY
 
Enough information is already present for an informed judgment about the Target IT team response. To plead unlikely extenuating circumstances such as (1) the team was overwhelmed by the volume of alerts, and was unable to distinguish signal from noise, or (2) the team might have seen similar alarts, which were investigated (despite the overwhelming volume of alerts) and dismissed as probable false positives, or (3) any Russian IP address is not necessarily cause for suspicion, since "(we do not know why) they would feel outbound connections from their POS to a Russian based IP wouldn't be suspicious" is worthy of a press release from Target public relations.

The FireEye system did its job well enough, elevating the alerts to the attention of Target IT, which eliminates the "sheer overload" excuse. Likewise, if the administrator had turned off automated response, it was critical to forge a field-tested policy for dealing with such detections manually, and then follow it to the letter. As for a number of Russian IPs, that in itself carries enough negative freight to merit special consideration-- aside from the principle that any "strange" address merits investigation.

Target did none of these things. What Target did is typical of the "90-day Wonder" policy of generating new managers ex nihilo, an IT person placed in the job for reasons that have little to do with experience or competence. As the survival tactic of one lacking experience, that manager essentially bought a well-respected brand, and then tried to hide behind it-- blaming FireEye for what was a Target responsibility.

Any Target promotion of a favored, specific person over those with more skill and'or experience is also excruciating commentary on the politics of Target management, since it focuses on factors which have little or nothing to do with professionalism. Such "fast track" promotions insidiously kill incentive among staff to demonstrate responsibility and competence. Fast track staffing is also disingenuous to the extreme, a breach of trust between executive management and staff-- especially those who were told promotion is based on demonstrated effort, competence and experience.

With the extremely questionable managerial culture at Target, the only possible defense against a charge of deliberately risky behavior with customer accounts is "mistakes were made"-- an abject confession of incompetence. While every manager is entitled to on-the-job training, that training should ensure millions of customer credit cards and bank accounts are not also at risk.
ke4roh
50%
50%
ke4roh,
User Rank: Apprentice
3/14/2014 | 2:17:47 PM
Image credit?
Wikimedia Commons did not create this image.  The image was taken by Flickr user Jay Reed who requires attribution to HIM for its distribution.  Wikimedia says that here. Please credit the photographer and copyright owner rather than the venue on which you found the picture!
VWalker
50%
50%
VWalker,
User Rank: Apprentice
3/14/2014 | 2:50:00 PM
Re: Image credit?
Thank you - I've fixed the attribution here and on a previous story where we used this image. Vicki Walker, News Editor.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Moderator
3/14/2014 | 2:25:44 PM
Does automated security watch for the right things?
I'd like to know the context: how many total alerts did FireEye provide during the hour it signaled the intrusion? How did it distinguish those that applied to the intrusion. I woujld think a notice that malware was being fanned out to multiple Target servers should be made to stand out. If you know the malware won't automatically be eliminated, what's the action plan to get it out of there? Wsa there any alert on 11GBs of internal data flowing out to Russia? Even in context, I'm afraid Target's response is going to be judged and judged harshly. Continuous sensitive credit card data should have triggered alarms that normal transaction data wouldn't. If it can happen to anyone with a large number of alerts pouring at them, then we're in more trouble than I realized.
Duane T
100%
0%
Duane T,
User Rank: Apprentice
3/14/2014 | 6:58:36 PM
You need more security that tech that tells you you've been infected
PCI and Security are like insurance, unfortunately Target spent $M on detection and left the response process to manual labor. But your insurance shouldn't just tell you that you're sick. This is like having insurance that just tells you that you indeed have an illness. They should have also spent at least 10% of that budget on process and technology to automatically investigate, prioritize, and lock down/contain their detected threats. You would think that they could have asked FireEye who they recommend for automated incident response. The tech is out there and available, and all this craziness and costs could be avoided.

Think of it this way, Target probably saw 1000s if not 10s of thousands of alerts each day, and they know it. They probably detect more than they can process effectively, and the result is that malware gets through. They probably could have spent a fraction more to get automated incident response technology in house.
rradina
50%
50%
rradina,
User Rank: Apprentice
3/16/2014 | 9:19:13 PM
Deactivation of FireEye's Automatic Response
There's a reason this was done.  Over the years protection software has triggered false alarms and quarantined needed programs and libraries rendering either software or subsystems (like printing) inoperable.  The last thing you want is to have thousands of POS lanes die because an automated response, triggered by a false positive, removed an important program or library module.
SaneIT
50%
50%
SaneIT,
User Rank: Apprentice
3/17/2014 | 9:02:21 AM
Re: Deactivation of FireEye's Automatic Response
Do you know what Target's procedure is when they see an alarm in the software, false or otherwise?  I would think that they have a policy in place to investigate the alarm to determine its validity. I know that things can move very slowly in the corporate world but this is the type of issue that most companies prepare for.
rradina
50%
50%
rradina,
User Rank: Apprentice
3/17/2014 | 5:33:00 PM
Re: Deactivation of FireEye's Automatic Response
They should respond manually.  If the product constantly cries wolf, either the alert config needs review or the product needs to be replaced.  If that's not an option then they should push the alerts to Splunk and mine the noise for credible events that correlate with other intrusion events (assumes firewalls and other stuff are pushed to Splunk).  My point was automated responses might be tolerated for devices that aren't customer facing but you do not want call center devices, bank ATMs or POS systems downed by a false alarm that automatically removes a vital component.

As a side note, I still don't understand why a POS system could have ANYTHING new installed on it outside of planned events.  They shoud use white list protection or an OS that won't run unsigned apps (like IOS, Android or Windows RT).
SaneIT
50%
50%
SaneIT,
User Rank: Apprentice
3/18/2014 | 8:43:30 AM
Re: Deactivation of FireEye's Automatic Response
You would think that the POS terminals would be locked down as tightly as possible.  It's not like your cashiers should be installing anything on them but not knowing all the details it is possible that the application used the name of a Windows service or application.  
rradina
50%
50%
rradina,
User Rank: Apprentice
3/18/2014 | 11:54:26 AM
Re: Deactivation of FireEye's Automatic Response
Locking them down assumes an OS security exploit was not used to install the malware.  I think it's been established Target's POS uses Windows.  I'll even go further and make an assumption that it's probably XP.

I'm not aware of any XP built-in solution to prevent a security hole being exploited to install malware.  If it's a remote attack vector, it'll typically involve a network service of some kind.  Most services generally have escalated privileges and if compromised, the hacker can almost always use them to gain root access.  

What Windows needs is a helper that monitors via read/write hooks and compares all file-system changes on system/software components with a dictionary made on the original system's image.  If anything is found out of spec, an alert is issued and the processes that use the corrupt image are terminated.  Further, such a helper also needs to scan DLLs and applications IN MEMORY to make sure they too are appropriate.  If not, the processes are terminated.  If an new process begins that's tied to an executable that's not part of the original image, it's terminated before it even finishes loading into memory.

Such products exist for XP and had they been using them, it would have been really tough to infect their POS systems even if a USB thumb drive was inserted.  Hackers would first have to figure out how to disable that software before exploiting the system.  Unfortunately this would require hacking the system so the protection mechanism can be hacked.  It's a chicken and egg scenario.  Certainly not foolproof but arguably difficult enough to perhaps convince them a company using such protection is not low hanging fruit.
Duke_Bauer
50%
50%
Duke_Bauer,
User Rank: Apprentice
3/24/2014 | 12:03:15 PM
Re: Deactivation of FireEye's Automatic Response
I believe this solution exists (McAfee Solidcore)
rradina
50%
50%
rradina,
User Rank: Apprentice
3/24/2014 | 3:19:56 PM
Re: Deactivation of FireEye's Automatic Response
It certainly does.  My last employer has been using it since ~2004/5 -- before McAfee bought Solidcore.  Back then the employer was flagged for not having virus protection on their POS systems.  We had to constantly ask for a compensating control.  That left me with a poor impression of the PCI rules and those who conducted the audits.  It's similar when calling a support line that isn't staffed by trained and experienced resources.  They cannot truly understand problems.  They can only read a script and follow a yes/no logic tree.
hho927
50%
50%
hho927,
User Rank: Apprentice
3/17/2014 | 2:25:10 PM
Block botnets
Target IT dept fail many ways. 1) If Target blocked all connections to botnet centers, the malware could not send data out. 2) The HVAC vendor said they didn't monitor Target remotely, why did Target give them a corp/network account? 3) Target should not give that account full access to the POS. 4) Security,access auditing was ignored. 5) Ignored alarms. 6) POS should have a seperate network. Target tried to save money here.
PaulS681
50%
50%
PaulS681,
User Rank: Apprentice
3/17/2014 | 7:16:26 PM
unacceptable
It just keeps getting worse for Target. To now know they had the systems in place that could have stopped this breach if they just used the system correctly is unacceptable. This just goes to show you that the best systems are rendered useless id people don't use them correctly.
pfretty
50%
50%
pfretty,
User Rank: Apprentice
3/19/2014 | 4:04:28 PM
Happens far too often
Unfortunate, but the fact that they ignored the warning signs isn't a surprise. There is a dramatic need for a shift in culture. One would think the cost alone would be enough. On average attacks cost companies $11.6 million according to the 2013 HP Ponemon Cost of Cyber Crime report (http://www.hpenterprisesecurity.com/ponemon-study-2013).

Peter Fretty (j.mp/pfrettyhp)
Register for Dark Reading Newsletters
White Papers
Cartoon
Current Issue
Dark Reading Tech Digest, Dec. 19, 2014
Software-defined networking can be a net plus for security. The key: Work with the network team to implement gradually, test as you go, and take the opportunity to overhaul your security strategy.
Flash Poll
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2013-4440
Published: 2014-12-19
Password Generator (aka Pwgen) before 2.07 generates weak non-tty passwords, which makes it easier for context-dependent attackers to guess the password via a brute-force attack.

CVE-2013-4442
Published: 2014-12-19
Password Generator (aka Pwgen) before 2.07 uses weak pseudo generated numbers when /dev/urandom is unavailable, which makes it easier for context-dependent attackers to guess the numbers.

CVE-2014-2026
Published: 2014-12-19
Cross-site scripting (XSS) vulnerability in the search functionality in United Planet Intrexx Professional before 5.2 Online Update 0905 and 6.x before 6.0 Online Update 10 allows remote attackers to inject arbitrary web script or HTML via the request parameter.

CVE-2014-2716
Published: 2014-12-19
Ekahau B4 staff badge tag 5.7 with firmware 1.4.52, Real-Time Location System (RTLS) Controller 6.0.5-FINAL, and Activator 3 reuses the RC4 cipher stream, which makes it easier for remote attackers to obtain plaintext messages via an XOR operation on two ciphertexts.

CVE-2014-5212
Published: 2014-12-19
Cross-site scripting (XSS) vulnerability in nds/search/data in iMonitor in Novell eDirectory before 8.8 SP8 Patch 4 allows remote attackers to inject arbitrary web script or HTML via the rdn parameter.

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
Join us Wednesday, Dec. 17 at 1 p.m. Eastern Time to hear what employers are really looking for in a chief information security officer -- it may not be what you think.