With a federal agency deadline for Federal Information Security Management Act (FISMA) compliance reporting through the new automated CyberScope tool already five months past, many security experts believe the government still has a long way to go in its quest to establish standards and implement continuous monitoring across the board.

Their claims are bolstered by recent delays in the publication of final standards from the National Institute for Standards and Technology (NIST) for continuous monitoring in the Federal space and by a recent report from Office of Management and Budget (OMB) that shows only a minority of agencies currently monitor continuously in a meaningful way. CyberScope is a mandatory new online FISMA reporting application for the feds.

Scott Crawford, analyst with Enterprise Management Associates (EMA), says one of the difficulties has been in the implementation of Security Content Automation Protocol (SCAP) standards to streamline manual and automated inputs of agency data for FISMA reporting.

"One of the things causing some confusion are SCAP requirements: how do they apply them, how do they meet them, and do they need to bring in even more tools to meet those requirements," he says. "The concept behind SCAP is very valid: it sets a model for how standardized data formats can enable better integration and automation of security management and security processes, but it seems like it was a rather hasty development, a hasty deployment and maybe not a realistic time line for adoption."

This hastiness is evidenced by several delays from NIST in publishing a final draft of a document outlining guidance in continuous monitoring best practices for FISMA compliance. Initially slated for publication in March, that date was pushed back to May and then ultimately a deadline was set for September.

Meanwhile, the OMB's Fiscal Year 2010 Report to Congress on the Implementation FISMA, out just last month, reported that only 29 percent of agencies are currently compliant with continuous monitoring tools and practices in place.

At the moment, most agencies are just now in the information-gathering stage of deploying continuous monitoring programs, says Mike Yaffe, product marketing manager for Core Security Technologies. "You can just feel the vibe when you're at a show and everybody is asking the same question, 'So, how do you help with continuous monitoring?' and they're all talking about continuous monitoring," Yaffe says.

According to Dr. Mike Lloyd, chief scientist for RedSeal Systems, the gradual uptake following a mandate such as this is the way of the world in the government space. He believes that federal agencies will show improvement year-over-year, even if adoption isn't at the hoped-for 100 percent right away.

"It's one of these situations where there's a mandate and some people will meet it and some of them won't this calendar year," he says. "If we could see half of the agencies getting their initial reports in the [CyberScope] framework this year, that is an outcome we can be proud of and that's a realistic idea of how adoption will work."

He says that a slow adoption rate is to be expected considering where most agencies are coming from. "Most organizations don't even know how many hosts are on the network," he says. "When you're at that fundamental a state of challenge, it is pretty hard to measure elaborate security properties of your overall defensive posture when you don't even know what the object is you're trying to defend."

Even those organizations that do have some kind of continuous monitoring and measurement capabilities built into their security programs tend to suffer from gaps in visibility says Bill Geimer, president of IronVine Security, a federal integrator specializing in security and compliance infrastructure.

"They'll have huge blind spots and be accepting risks they don't even know were there because they're not measuring them," he says. "Sometimes they'll have a tool that only gives them one view, like they'll know their Microsoft patches are within sixty days of compliance but won't know that vulnerabilities exist for Java, Adobe or other applications that are non-Microsoft because they may not be measured by their SMS reports."

Based on his work in the space, Lloyd would guess that approximately half of federal agencies are at that limited visibility level of maturity. Another quarter has better knowledge of their environments but are still overwhelmed by all of the data feeds that they're collecting due to insufficient automated analytics. He guesses that only about a quarter of agencies truly have the kind of visibility and automatic number-crunching necessary to break down security data into a simple and continuous metric that can affect meaningful change in the organization.

While EMA's Crawford wouldn't go so far as to guess how many years it will take to truly get all agencies on board with the continuous monitoring initiative, he believes that it may be a mistake to ever seek an elusive state of compliance. The government would be best served to see this as an ongoing process instead, he says. "This is not going to be a particularly easy or straightforward problem to solve," he warns. "I think it should be seen as an ongoing process, regardless."

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.