Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Application Security

4/29/2019
10:25 AM
Larry Loeb
Larry Loeb
Larry Loeb
50%
50%

NIST Offers Improved Software Testing

Combinatorial testing is a software testing method that the National Institute of Science and Technology (NIST) likes a lot.

Combinatorial testing is a method for more effective software testing that the National Institute of Science and Technology (NIST) likes a lot.

They ran studies from 1999 to 2004 showing that most software bugs and failures are caused by one or two parameters of input, with progressively fewer by three or more.

They call this the interaction rule, and it is a roadmap for more efficient testing than those given with conventional methods. NIST says that it has multiple studies showing fault detection results equal to exhaustive testing with a 20X to 700X reduction in test set size.

The software industry often spends seven to 20 times as much money rendering safety-critical software reliable as it does on more conventional code, so improvements in testing have been called for by designers.

This is not fuzzing, as the term is commonly used in software development. Fuzzing (or fuzz testing) is an software testing technique using invalid, unexpected, or random data as inputs to a program. The program is then monitored for faults such as crashes, failing built-in code assertions, or potential memory leaks. The NIST approach uses normal and valid ranges of inputs for its testing and observation.

Fuzzing checks for what happens when input goes out of bounds and is "unsanitary," while NIST techniques check for what happens when a program's inputs are within norms but changing relative to other inputs.

Anyway, NIST announced the first revision since 2015 of its Automated Combinatorial Testing for Software, or ACTS.

Researchers from the University of Texas at Arlington, Adobe Systems Inc. and Austria's SBA Research announced at last week's IEEE International Conference on Software Testing, Verification and Validation in China that they had updated a version of Combinatorial Coverage Measurement (CCM) to include extensions to it by SBA Research.

NIST's internal tools were able to handle software that had a few hundred input variables, but SBA Research developed another new tool that can examine software that has up to 2,000. This increase can generate a test suite for up to five-way combinations of input variables.

The two tools can be used in a complementary fashion: While the NIST software can measure the coverage of input combinations, the SBA algorithm can extend coverage to thousands of variables. While the SBA Research algorithm is not an official part of the ACTS test suite, the team says that it has plans to bundle it with ACTS in the future.

NIST will make the algorithm available to any developer who requests it.

"Before we revised CCM, it was difficult to test software that handled thousands of variables thoroughly," NIST mathematician Raghu Kacker said in a statement. "That limitation is a problem for complex modern software of the sort that is used in passenger airliners and nuclear power plants, because it's not just highly configurable, it's also life critical. People's lives and health are depending on it."

— Larry Loeb has written for many of the last century's major "dead tree" computer magazines, having been, among other things, a consulting editor for BYTE magazine and senior editor for the launch of WebWeek.

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 7/2/2020
Ripple20 Threatens Increasingly Connected Medical Devices
Kelly Sheridan, Staff Editor, Dark Reading,  6/30/2020
DDoS Attacks Jump 542% from Q4 2019 to Q1 2020
Dark Reading Staff 6/30/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
How Cybersecurity Incident Response Programs Work (and Why Some Don't)
This Tech Digest takes a look at the vital role cybersecurity incident response (IR) plays in managing cyber-risk within organizations. Download the Tech Digest today to find out how well-planned IR programs can detect intrusions, contain breaches, and help an organization restore normal operations.
Flash Poll
The Threat from the Internetand What Your Organization Can Do About It
The Threat from the Internetand What Your Organization Can Do About It
This report describes some of the latest attacks and threats emanating from the Internet, as well as advice and tips on how your organization can mitigate those threats before they affect your business. Download it today!
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-9498
PUBLISHED: 2020-07-02
Apache Guacamole 1.1.0 and older may mishandle pointers involved inprocessing data received via RDP static virtual channels. If a userconnects to a malicious or compromised RDP server, a series ofspecially-crafted PDUs could result in memory corruption, possiblyallowing arbitrary code to be executed...
CVE-2020-3282
PUBLISHED: 2020-07-02
A vulnerability in the web-based management interface of Cisco Unified Communications Manager, Cisco Unified Communications Manager Session Management Edition, Cisco Unified Communications Manager IM & Presence Service, and Cisco Unity Connection could allow an unauthenticated, remote attack...
CVE-2020-5909
PUBLISHED: 2020-07-02
In versions 3.0.0-3.5.0, 2.0.0-2.9.0, and 1.0.1, when users run the command displayed in NGINX Controller user interface (UI) to fetch the agent installer, the server TLS certificate is not verified.
CVE-2020-5910
PUBLISHED: 2020-07-02
In versions 3.0.0-3.5.0, 2.0.0-2.9.0, and 1.0.1, the Neural Autonomic Transport System (NATS) messaging services in use by the NGINX Controller do not require any form of authentication, so any successful connection would be authorized.
CVE-2020-5911
PUBLISHED: 2020-07-02
In versions 3.0.0-3.5.0, 2.0.0-2.9.0, and 1.0.1, the NGINX Controller installer starts the download of Kubernetes packages from an HTTP URL On Debian/Ubuntu system.