Security pros responsible for log analysis and digital forensic investigations today have so much data to analyze that it can be difficult to make heads or tails of it without the proper tools to parse, prioritize, and identify the valuable information.

Sometimes obscure log entries can be easily deciphered with a simple search on the Internet. But other times, there are too many results and it’s hard to wade through them to find the correct information. Many organizations have adopted security information and event management (SIEM) solutions to help with the correlation and prioritization of security data in order to turn it into actionable information. Once properly configured and tuned, SIEMs can certainly make a big difference. But often, the SIEM's greatest feature turns out to be the ability to take the data and visualize it in a way that the analysts can easily spot patterns or peaks in activity indicating a problem.

Data visualization, or the simplest terms, the visual representation of data, is nothing new. The last two decades have seen an increase in interest in it as researchers, security pros, and vendors have worked to visualize computer-related data in meaningful ways. In 2004, I saw the first data visualization presentation focused on security data visualization at a small hacker conference in Atlanta conference called Interz0ne. Greg Conti gave a fascinating talk that showed many different graphical representations of port scans and attacks that I'd analyzed on a regular basis using an intrusion detection system (IDS), packet sniffer, and network flow data. (PDF). The way the activity popped out was eye-opening.

A few years later in 2007, Greg published the excellent book, "Security Data Visualization: Graphical Techniques for Network Analysis," through No Starch Press. The book provided an intriguing walkthrough of different tools and methods for visualizing everything from network packets and binary files to IDS and firewall logs. Similarly, another excellent book, "Applied Security Visualization," followed a year later.

During this same time, security tool vendors were incorporating data visualization techniques—often to the benefit of the analysts using the system, but sometimes it was poorly designed and served as more of a misleading distraction. In too many cases were graphs and pie charts were added, but provided no meaningful information to the security pros using the system.

Most of the SIEM solutions I've reviewed, used, or seen in action during client engagements have actually put a lot of thought into their dashboard design and serve as the best example of successful data visualization efforts. They focus on taking large amounts of data, normalizing it, correlating it, and presenting the results in a dashboard with graphs and charts. SIEM offerings from companies like ArcSight, NitroSecurity, Splunk, and Tenable Network Security are just a few examples.

Data visualization techniques have been making their way into other areas besides network and log analysis. Digital forensics has seen an increase in interest in timelines and graphical representation of data in the last three years. The number of times timelines have been mentioned in Harlan Carvey's Windows Incident Response blog and the SANS Computer Forensics and Incident Response blog can attest to that fact.

Timeline research has especially benefited digital forensics as free and open source tools have been developed to meet the needs of security investigators. The tools tie together filesystem activity with logs from network devices, Windows Event Log, and services like IIS and Apache. The resulting timeline can be viewed in its native text format or loaded into software like Excel for sorting, filtering, and graphing. Investigators can use the timeline identify patterns and a suspect's activity across many sources of data.

A good example is the recent release of an Excel colored timeline template and a SANS forensic blog entry from Rob Lee, SANS Faculty Fellow. The blog provides links and instructions for using timeline tools to generate data that is then loaded into the Excel template. The different types of data represented in the timeline is color-coded to help investigators track activity like email and chat usage, the opening and modification of files, USB drive usage, and Windows account activity.

Commercial forensic and incident response tools are also seeing new visualization features being added. Just this week, AccessData announced the availability of the "FTK Add-On: AccessData Visualization" to its forensic product FTK (Forensic Toolkit), that according to its datasheet, adds graphs, pie charts, and treemaps for visualizing email and file activity and contents.

Data visualization can be an extremely useful tool during log review, forensic analysis, and other security activities where large amounts of data are involved. Relationships between people and placed are suddenly realized when using a tool like Maltego. Port scans and brute force attacks can easily be traced through the graphs in a SIEM.

It helps to understand when visualization works and when it doesn't, and that can be helped by using the available tools, researching available books and blog entries, and seeing what works best for your environment.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.