Perimeter
4/16/2012
11:30 AM
Commentary
Commentary
Commentary
50%
50%

Log Standards: Put Up, Shut Up, Give Up, Or Throw Up?

Do we need logging standards, or should we just follow the leaders to help direct our logging efforts?

Syslog was developed in the 1980s, and it probably took roughly five minutes of use before someone started complaining about how it wasn't capable enough. When the IETF drafted RFC3164 in 2001, essentially declaring syslog the de facto standard for log transmission, people immediately started talking about how to make a different and "better" standard.

Fast forward to 2012. Syslog is still the de facto log sending standard, but other technologies and methods have emerged to make the transportation and digestion of system logs easier -- and far more customizable. Some standards didn't quite make it, though -- and thanks to my good friend Dr. Anton Chuvakin, we have a detailed listing of headstones. Defense Advanced Research Projects Agency’s (DARPA's) Common Intrusion Detection Framework (CIDF) eventually became the Intrusion Detection Message Exchange Format (IDMEF), which was never really adopted by anyone. MITRE had Common Intrusion Event List (CIEL), but even that was cancelled early on in the process.

The current project-to-standard efforts continue to be lead by the usual suspects. MITRE has the Common Event Expression (CEE) standard, and The Open Group has the XDAS specification -- the two front-runners for something better. Balázs Scheidler’s syslog-ng extends the original syslog model with content-based filtering, rich filtering capabilities, and flexible configuration options, and it adds important features to syslog, like using TCP for transport. Also, Rainer Gerhards' rsyslog supports multithreading, message filtering, and a fully configurable output format. Several vendors also tossed their standards into the ring to help expedite syslog's demise, including IBM (CBE), Webtrends (WELF), ArcSight (CEF), eIQNetworks (OLF), Cisco (SDEE), and Q1 Labs (LEEF), to name a few. Vendors have also exposed APIs to allow third-party products to subscribe to generated logs, typically an XML formatted file with a RESTful API.

So what do you choose? My historical advice to buyers, developers, and vendors was always to look to the infrastructure vendors because they traditionally dictated what event formats and log transport mechanisms would be supported.

The reality in 2012, however, is that infrastructure providers like Cisco, Juniper, and the rest will continue to make do with syslog; though some will argue that they are exploring new methods, the fact remains that syslog will never go away. Application logging, on the other hand, has emerged as a much more complex problem and will likely change the way we generate and consume logs. If you adopt a yet-to-be ratified standard or, even more importantly, a yet-to-be adopted format, you may impact your product’s ability to join in on an enterprise monitoring ecosystem.

The best advice I can give to developers is to design logging mechanisms that are future-proof, i.e., support legacy syslog and explore emerging standards. Also, as a developer, be open to re-evaluating the logging mechanisms you’ve implemented and never settle on the current implementation.

For buyers, make sure the products you use fit into your existing monitoring ecosystem and that you’ve selected a vendor that is open to evolving. If you can get them to commit to it in the contract, then you get bonus points.

Andrew Hay is senior analyst with 451 Research's Enterprise Security Practice (ESP) and is an author of three network security books. Follow him on Twitter: @andrewsmhay

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
MROBINSON000
50%
50%
MROBINSON000,
User Rank: Apprentice
5/2/2012 | 1:31:24 PM
re: Log Standards: Put Up, Shut Up, Give Up, Or Throw Up?
Andrew,

Thank you for sharing some very interesting insights. We would like to
add an application securityGÇÖs company point of view. We believe that logging
functions should be centralized in order for them to behave similarly through
the application and is an essential step in centralizing information security
functionality.-á You can read more here: http://blog.securityinnovation...
Register for Dark Reading Newsletters
White Papers
Cartoon
Current Issue
Dark Reading Tech Digest, Dec. 19, 2014
Software-defined networking can be a net plus for security. The key: Work with the network team to implement gradually, test as you go, and take the opportunity to overhaul your security strategy.
Flash Poll
Title Partner’s Role in Perimeter Security
Title Partner’s Role in Perimeter Security
Considering how prevalent third-party attacks are, we need to ask hard questions about how partners and suppliers are safeguarding systems and data.
Video
Slideshows
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2014-4467
Published: 2015-01-30
WebKit, as used in Apple iOS before 8.1.3, does not properly determine scrollbar boundaries during the rendering of FRAME elements, which allows remote attackers to spoof the UI via a crafted web site.

CVE-2014-4476
Published: 2015-01-30
WebKit, as used in Apple iOS before 8.1.3; Apple Safari before 6.2.3, 7.x before 7.1.3, and 8.x before 8.0.3; and Apple TV before 7.0.3, allows remote attackers to execute arbitrary code or cause a denial of service (memory corruption and application crash) via a crafted web site, a different vulner...

CVE-2014-4477
Published: 2015-01-30
WebKit, as used in Apple iOS before 8.1.3; Apple Safari before 6.2.3, 7.x before 7.1.3, and 8.x before 8.0.3; and Apple TV before 7.0.3, allows remote attackers to execute arbitrary code or cause a denial of service (memory corruption and application crash) via a crafted web site, a different vulner...

CVE-2014-4479
Published: 2015-01-30
WebKit, as used in Apple iOS before 8.1.3; Apple Safari before 6.2.3, 7.x before 7.1.3, and 8.x before 8.0.3; and Apple TV before 7.0.3, allows remote attackers to execute arbitrary code or cause a denial of service (memory corruption and application crash) via a crafted web site, a different vulner...

CVE-2014-4480
Published: 2015-01-30
Directory traversal vulnerability in afc in AppleFileConduit in Apple iOS before 8.1.3 and Apple TV before 7.0.3 allows attackers to access unintended filesystem locations by creating a symlink.

Best of the Web
Dark Reading Radio
Archived Dark Reading Radio
If you’re a security professional, you’ve probably been asked many questions about the December attack on Sony. On Jan. 21 at 1pm eastern, you can join a special, one-hour Dark Reading Radio discussion devoted to the Sony hack and the issues that may arise from it.