Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


09:42 AM

Fast Scanning To Fuel 'Golden Age' Of Global Flaw Finding

A network scanner that can survey the Internet in less than an hour will make it easier for research groups to expose vulnerabilities on the Internet

A network scanner designed from scratch by three University of Michigan researchers can scan the entire IPv4 Internet in about 45 minutes, drastically reducing the speed at which such scans can be accomplished.

Announced at last month's USENIX Security conference, the scanner, dubbed ZMap, uses a modular approach to scanning to speed the process, the pseudo-random selection of IP addresses to avoid overwhelming small networks and validation of the responses by a separate system to verify the results. The researchers -- Zakir Durumeric, Eric Wustrow, and J. Alex Halderman -- used the scanner to track protocol use on the Internet, find systems vulnerable to HTTPS weak key flaw, and discover unadvertised services. Without fast scans of the Internet, many types of research would not be feasible, says Durumeric, a Ph.D. candidate in computer science at the University of Michigan.

"You can imagine that if you did your scans over three months and then did all the follow-up processing, the Internet could have grown, in terms of the use of certain protocols, by 10 percent," Durumeric says. "So you have a whole new degree of specificity."

The techniques combined to create ZMap are not all new but have not before been brought together in a single program. In the same way that the Shodan service made the results of Internet scans more accessible, the ability to quickly perform customized scans of the Internet will likely result in a "golden age" of vulnerability scanning, says HD Moore, chief research officer for Rapid7, a vulnerability management firm. Rather than waiting for days or weeks for scans to complete, researchers can do a lot more with fewer resources.

"It really shrinks the size of the Internet in a way that we couldn't do before," Moore says. "It's not big data anymore."

Network scanners were originally designed to scan small networks, keeping track of the current state of the scan as it progressed. However, when scanning a network the size of the Internet, the state data can grow too large for most systems, Durumeric says. To solve that problem, some scanning projects have broken their scans into batches, scanning a complete subnetwork before moving onto the next subnet. Yet, if the scan is done quickly, it can overwhelm the provider with requests.

[Researchers and attackers catalog vulnerable systems connected to the Internet, from videoconferencing systems set to auto-answer, to open point-of-sale servers, to poorly configured database systems. See Global Scans Reveal Internet's Insecurities In 2012.]

ZMap solves both problems by generating pseudo-random IP addresses using a particular method, known as multiplicative group of integers, so that each address appears only once. The process is broken into a fast engine for generating and sending the packets, and an asynchronous collector that receives the packets and logs the data. The state of each connection is not kept, Durumeric says. Instead, packets are matched by putting identifying data in the unused fields of the network packet.

"Really, each of these packets is the same except for where it goes, so we don't need to go through all these same validation steps," he says. "We just need to update a small amount of information and send it onto the next host."

While the three researchers from the University of Michigan have shortened the time it takes to do a scan, the bandwidth required to scan the entire Internet quickly will likely limit such projects to academic research groups and large corporations.

"When we say we are scanning at a gigabit speed, we are using an entire gigabit connection," he says. "Most home users have 1- or 2 megabytes."

Rapid7's HD Moore had already begun working with the University of Michigan on a large-scale study using scans to find websites, Internet-connected servers, and cloud services that link back to a business' domain to help companies find unknown or rogue assets. Certificates used with secure services, for example, include the domain of the certificate holder, providing a link that the researchers can connect back to the firm.

"They all point to identifying assets out there that you may not know belongs to your company," Moore says.

Looking for vulnerabilities is another fertile field. There is no shortage of vulnerable systems out there, Moore says. By using fast scanning to highlight the weaknesses before they can be exploited by attackers, the Internet benefits as a whole, he says. Many ISPs distribute poorly configured routers, and Rapid7 and Moore have already highlighted problems in universal plug-and-play devices, such as routers, as well as insecure video conferencing systems.

The net benefit of fast scanning should be that more vulnerabilities will be detected and eliminated because attackers have already been using botnets and other techniques to scan for vulnerable services in networks. The Carna botnet and the report on its scanning results showed the possibilities of illegal scanning projects. They could adopt ZMap's techniques for scanning, but will not benefit from the technology as much as defenders, says Moore.

However, eventually the golden age of fast scanning will end. As IPv6 becomes increasingly deployed, scanning the entire network will become impossible. Even limiting scans to known assigned IPv6 addresses and using other information to attempt to identify hosts will not narrow the field much for researchers.

"With IPv6, if it's not a published entry somewhere, you are not going to find it," Moore says. "You have to send almost as much traffic to identify a single subnet as the entire IPv4 Internet. So the numbers are not going to work out."

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message. Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Exploits Released for As-Yet Unpatched Critical Citrix Flaw
Jai Vijayan, Contributing Writer,  1/13/2020
Microsoft to Officially End Support for Windows 7, Server 2008
Kelly Sheridan, Staff Editor, Dark Reading,  1/13/2020
Active Directory Needs an Update: Here's Why
Raz Rafaeli, CEO and Co-Founder at Secret Double Octopus,  1/16/2020
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: This comment is waiting for review by our moderators.
Current Issue
The Year in Security: 2019
This Tech Digest provides a wrap up and overview of the year's top cybersecurity news stories. It was a year of new twists on old threats, with fears of another WannaCry-type worm and of a possible botnet army of Wi-Fi routers. But 2019 also underscored the risk of firmware and trusted security tools harboring dangerous holes that cybercriminals and nation-state hackers could readily abuse. Read more.
Flash Poll
[Just Released] How Enterprises are Attacking the Cybersecurity Problem
[Just Released] How Enterprises are Attacking the Cybersecurity Problem
Organizations have invested in a sweeping array of security technologies to address challenges associated with the growing number of cybersecurity attacks. However, the complexity involved in managing these technologies is emerging as a major problem. Read this report to find out what your peers biggest security challenges are and the technologies they are using to address them.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2020-01-17
Spring Framework, versions 5.2.x prior to 5.2.3 are vulnerable to CSRF attacks through CORS preflight requests that target Spring MVC (spring-webmvc module) or Spring WebFlux (spring-webflux module) endpoints. Only non-authenticated endpoints are vulnerable because preflight requests should not incl...
PUBLISHED: 2020-01-17
Eclipse Memory Analyzer version 1.9.1 and earlier is subject to a deserialization vulnerability if an index file of a parsed heap dump is replaced by a malicious version and the heap dump is reopened in Memory Analyzer. The user must chose to reopen an already parsed heap dump with an untrusted inde...
PUBLISHED: 2020-01-17
It was found that the Red Hat Enterprise Linux 8 kpatch update did not include the complete fix for CVE-2018-12207. A flaw was found in the way Intel CPUs handle inconsistency between, virtual to physical memory address translations in CPU's local cache and system software's Paging structure entries...
PUBLISHED: 2020-01-17
** REJECT ** DO NOT USE THIS CANDIDATE NUMBER. ConsultIDs: CVE-2008-1382. Reason: This candidate is a reservation duplicate of CVE-2008-1382. Notes: All CVE users should reference CVE-2008-1382 instead of this candidate. All references and descriptions in this candidate have been removed to prevent ...
PUBLISHED: 2020-01-17
Eclipse Memory Analyzer version 1.9.1 and earlier is subject to a cross site scripting (XSS) vulnerability when generating an HTML report from a malicious heap dump. The user must chose todownload, open the malicious heap dump and generate an HTML report for the problem to occur. The heap dump could...