Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

News

10/29/2013
01:39 PM
George Crump
George Crump
Commentary
50%
50%

Quick Guide To Flash Storage Latency Wars

Because latency is the key performance differentiator in server-side flash, SSD, PCIe and memory bus flash storage vendors are competing on speed.

Server-side flash is all about performance, and the key performance differentiator is latency. Latency is the time required to complete a transaction between a host and a storage system, such as a read or write operation. As a result, a "latency war" has broken out in the flash community about the best way to reduce flash latency.

Latency actually has several levels. The first level is the time it takes the media to position and be ready to respond to an I/O request. With a hard drive, this latency was the end of the discussion, as the milliseconds it took for a hard drive to get in position overshadowed any other latency in the storage communication chain.

Solid state disk (SSD) drives -- which are flash inside a hard drive container -- eliminated that device latency. SSDs can put themselves in position instantly since they have no moving parts, no platters to rotate. But that exposed other areas of latency in the storage protocol stack. For example, the time it takes for the I/O to work its way through the overhead of SCSI became noticeable.

This led to the introduction and rapid adoption of PCIe-based flash. Most of these cards eliminated the storage protocol stack altogether. The communication was direct to the application or operating system over the PCIe bus. Eliminating the storage protocol stack meant that special drivers had to be created for the various operating systems that a data center might have.

Vendors then introduced API sets that would allow users to write directly to the PCIe flash card from within the application for a further reduction in latency, not only avoiding the storage I/O stack but also avoiding the operating system itself.

What PCIe began to show, though, was that there was another level of latency. The PCIe bus itself. The PCIe bus routes through a motherboard-based fabric that allows multiple cards to share PCIe bandwidth. Some higher end servers will have multiple PCIe hubs in order to better route data. But even a high-end server when burdened with lots of I/O can become PCIe bottlenecked and lead to latency. (Remember that PCIe supports more than storage I/O.)

The next tier in latency elimination is to eliminate the PCIe bus altogether. Several vendors are introducing memory-bus-based flash storage. These flash storage devices come in a memory DIMM form factor and can act as storage with a device driver, similar to a PCIe SSD. Even more interesting, with a tweak to server system BIOS, it can act as main memory to the server. Imagine 400 GBs of "RAM" via flash on a single DIMM. Using the memory bus provides even more I/O channels and greater bandwidth; it was, after all, designed to support DRAM.

In both implementation modes, the use cases are very interesting. The ability to create very dense servers with terabytes (if not petabytes) of flash capacity in a 1U system changes the data center design game quite a bit.

There is no single perfect solution, as SSD, PCIe flash and memory flash all have their ideal use cases, and many data centers may have a mixture of all three. Designing applications for extreme high performance with almost zero latency is now reality. But their use is not limited to the performance fringe. For example imagine using these technologies to design a single physical server to support 10,000 plus desktops. We have the processing power available to us, but the latency of storage is no longer the roadblock it always was.

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
samicksha
50%
50%
samicksha,
User Rank: Apprentice
10/31/2013 | 11:25:23 AM
re: Quick Guide To Flash Storage Latency Wars
Any review on Tape storage,as i don't think it going away any time soon, as it is still the most cost-effective and simplest way to archive huge amounts of data.
D. Henschen
50%
50%
D. Henschen,
User Rank: Apprentice
10/31/2013 | 10:49:46 AM
re: Quick Guide To Flash Storage Latency Wars
I heard an enlightening cost comparison the other day by Ari Zilka, CTO at Hortonworks. 1Terabyte of RAM = $70,000; 1TB of Flash = $8,000 to $20,000, depending on quality/discounts; 1 TB of hard drive = $60 to $100 depending on quality/performance. Maybe somebody would quibble with the exact figures, but it's clear there is a cost to real-time performance.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Ninja
10/31/2013 | 3:58:11 AM
re: Quick Guide To Flash Storage Latency Wars
Of course, this all stands to change over the next several years as SSDs gradually become obsolete (except, possibly, in hybrid setups) under Moore's Law. http://www.enterpriseefficienc...
beachscape
50%
50%
beachscape,
User Rank: Apprentice
10/31/2013 | 2:25:02 AM
re: Quick Guide To Flash Storage Latency Wars
Great summary describing latency in storage. It would be good to have a table comparing latencies of HDDs, SSDs, PCIe and flash memory DIMMs.
Mobile Banking Malware Up 50% in First Half of 2019
Kelly Sheridan, Staff Editor, Dark Reading,  1/17/2020
Exploits Released for As-Yet Unpatched Critical Citrix Flaw
Jai Vijayan, Contributing Writer,  1/13/2020
Microsoft to Officially End Support for Windows 7, Server 2008
Kelly Sheridan, Staff Editor, Dark Reading,  1/13/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: This comment is waiting for review by our moderators.
Current Issue
The Year in Security: 2019
This Tech Digest provides a wrap up and overview of the year's top cybersecurity news stories. It was a year of new twists on old threats, with fears of another WannaCry-type worm and of a possible botnet army of Wi-Fi routers. But 2019 also underscored the risk of firmware and trusted security tools harboring dangerous holes that cybercriminals and nation-state hackers could readily abuse. Read more.
Flash Poll
[Just Released] How Enterprises are Attacking the Cybersecurity Problem
[Just Released] How Enterprises are Attacking the Cybersecurity Problem
Organizations have invested in a sweeping array of security technologies to address challenges associated with the growing number of cybersecurity attacks. However, the complexity involved in managing these technologies is emerging as a major problem. Read this report to find out what your peers biggest security challenges are and the technologies they are using to address them.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-7227
PUBLISHED: 2020-01-18
Westermo MRD-315 1.7.3 and 1.7.4 devices have an information disclosure vulnerability that allows an authenticated remote attacker to retrieve the source code of different functions of the web application via requests that lack certain mandatory parameters. This affects ifaces-diag.asp, system.asp, ...
CVE-2019-15625
PUBLISHED: 2020-01-18
A memory usage vulnerability exists in Trend Micro Password Manager 3.8 that could allow an attacker with access and permissions to the victim's memory processes to extract sensitive information.
CVE-2019-19696
PUBLISHED: 2020-01-18
A RootCA vulnerability found in Trend Micro Password Manager for Windows and macOS exists where the localhost.key of RootCA.crt might be improperly accessed by an unauthorized party and could be used to create malicious self-signed SSL certificates, allowing an attacker to misdirect a user to phishi...
CVE-2019-19697
PUBLISHED: 2020-01-18
An arbitrary code execution vulnerability exists in the Trend Micro Security 2019 (v15) consumer family of products which could allow an attacker to gain elevated privileges and tamper with protected services by disabling or otherwise preventing them to start. An attacker must already have administr...
CVE-2019-20357
PUBLISHED: 2020-01-18
A Persistent Arbitrary Code Execution vulnerability exists in the Trend Micro Security 2020 (v160 and 2019 (v15) consumer familiy of products which could potentially allow an attacker the ability to create a malicious program to escalate privileges and attain persistence on a vulnerable system.