Nevertheless, monitoring still does have a place in the overall risk-management framework. The problem is that most organizations confuse logging and log storage with effective monitoring.
"It can be very difficult for any size network to have a reliable logging infrastructure, and this is required by lots of compliance regulations," Gula says. "However, getting logs is much different than analyzing logs and looking for evidence of attacks and abuse."
7. Overbuying to get a specific compliance-oriented feature.
If your organization spends significant amounts of dough on tools that duplicate security features sported by tools they've already purchased just so you can nab one specific compliance-focused feature, then that's a dead giveaway you've fallen in the chec-kbox compliance trap, says Marcus Carey, security researcher at Rapid7.
It happens all the time, Carey says. "You call up a vendor because they're the only one that does X that you specifically need to satisfy some specific requirement of the regulation," he says. "You discover you can't buy X alone, so you end spending six figures on a solution that duplicates most of what you already have installed."
8. Generating a low access revocation rate.
Making information accessible on a need-to-know basis is a security fundamental, which is why access reviews are often included within so many regulatory requirements. Unfortunately, though, most check-box compliance organizations perform these perfunctorily to cover their derrieres. So unless your organization is extremely mature in its security processes, if an access review doesn't yield many changes, it was probably done just to satisfy requirements, Garbis says.
"If an organization is doing a good job of meeting compliance goals, each review process should generate a not-insignificant number of changes that have to be fulfilled," he says. "If a review process generates a very low number of follow-up changes, it's likely that this was a check-box review."
9. Spending more on system maintenance than security operation.
Check-box compliance tends to lure organizations into tons of make-work projects, so that in the long run they end up spending more time maintaining systems they buy to check those boxes than they do in actually making sure the organization stays secure, Carey says.
"[These] organizations have to hire full-time employees dedicated to systems that don't work," he says, "so they have a huge capital expense, man hours, and consulting hours on something that is effectively useless."
10. Systems cause more audit problems than solutions.
In the same vein, Carey says these check-box-prone organizations also tend to see more audit problems come out of their compliance systems than solutions.
"I see misconfigured or poorly designed systems causing a nightmare for organizations trying to explain every false-positive to an auditor," he says. "It's unfortunate the solution they bought to pass audits can ruin them when they actually procure it. That's a double whammy."
Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.