Seat of the pants is no way to prioritize security spending and set project precedence. But that's exactly how some CISOs are doing business.

Michael A. Davis, CTO of CounterTack

January 21, 2011

12 Min Read

Don't Gut It Out Don't Gut It Out

It's no secret that some companies excel at information security while others run around stamping out fires and never get ahead. What's the secret of first-rate IT risk managers? To find out, we interviewed a half-dozen CISOs from various industries. We didn't talk about specific types of threats. Rather, we wanted to understand exactly how these successful security leaders prioritize money and manpower.

One thing we all have in common is data overload. Infosec leaders have their go-to sources for cyberintelligence, like vendors, newsgroups, the National Institute of Standards and Technology, and regulatory bodies. But at some point, we all find ourselves overwhelmed. Call it the "needle in a needle stack" conundrum: You know there's a ton of threats out there, many of them potentially damaging to your company, but if you pick the wrong needle at the wrong time, the stack may just fall and cause death by a thousand cuts.

And there are plenty of reasons a CISO might select the wrong needle.

We'll admit that the media doesn't always help. Early last year, my phone was ringing off the hook after the Google attacks, with information security pros asking whether APT--advanced persistent threat--was the most immediate danger to their companies. Now, data loss prevention and distributed denial of service are back in the spotlight, courtesy of WikiLeaks.

We're also all over the map with risk assessments. Every company we reviewed had some type of risk management framework, but the devil is in the details. We saw no uniform best practices. Our recent InformationWeek Analytics IT Risk Management Survey, available later this month, shows that the most popular way to measure risk, by far, is qualitative categorization of high, medium, or low. In our experience, some companies have rudimentary internal risk assessment systems, supplemented by an external vendor or third party, such as Gartner. On the other end of the spectrum are companies that deploy extensive, 50-plus-question surveys and use a stringent, quantitative approach where every response has a weight; the overall tally denotes a project's risk. Fewer than half of our survey respondents, all of whom play roles in assessing risk at their companies, use such a quantitative method.

What's interesting is that the CISOs we spoke with agree that neither a quantitative nor a qualitative approach is much help with prioritization. Quantitative risk analysis is not the be-all and end-all--just because a risk is scored at 98 out of 100 doesn't mean it will be remediated. For one thing, the business significantly influences whether to spend money. And most surprising to us, in the end, many CISOs say they ignore vendor input, media reports, pundit white papers, even all their own data and make gut decisions.

Let's be clear: Gut decisions aren't useful. Very often they're based on a confirmation bias--the tendency for people to favor information that confirms their preconceptions or hypotheses, regardless of whether the information is true. If you have a confirmation bias and think laptop theft is the largest concern, whether it is or not, you'll find a way to get encryption to be the highest-priority project.

Beyond Gut Instinct

Five Ways to Get Rational About Risk

Become an InformationWeek Analytics subscriber and get our full report on reducing security data overload. This report includes 13 pages of action-oriented analysis. What you'll find:

  • Why you need an emergency risk assessment process

  • Exclusive sneak peek at results of our risk management poll

Get This And All Our Reports


Avoiding confirmation bias is difficult. The first step is to realize that we're all prone to it. If you have a tendency to collect a lot of information and then ignore it, or you always find yourself debating the rest of the organization on which threats are most imminent, you may be more susceptible than most. Try this exercise: Ask your peers whether they think you frequently make decisions based on instinct or intuition. And don't ignore the feedback--it's almost impossible to build trust with an information source, such as your risk assessment team, if you have this tendency.

In our experience, organizations with the best risk reduction processes don't follow any one person's gut or arguments. Their CISOs aren't so afraid to talk to peers and co-workers. They don't buy into the "always greener on the other side" argument, assuming rival organizations' risk management initiatives are more mature because those CISOs have bigger budgets, more resources, or better executive support. They take ownership.

And, most important, they aren't afraid to share decision-making power. In fact, our experience and research show that the most successful CISOs are collaborative and likely to bring various teams to the table. That's critical, because our InformationWeek Analytics 2011 Outlook Survey shows that demand for IT services is on the rise. Your ability to assess the risks associated with all these new programs better keep up.

chart: In this downturn, what's happening to demand for IT at your company?

Alter Egos

Most people see infosec pros as crusaders, fearlessly taking on evil malware distributors, fraudsters, and organized crime. But the truth is closer to this: We aren't so much superheroes as people with good intentions and way too much on our plates. As foes go, an ever-growing pile of threat notices and "expert" advice is almost as dangerous as some identity theft syndicate in Belarus. Sure, we manage through it and make decisions every day that affect our organizations' well-being. But wouldn't we do better with a set of best practices that could provide clarity in the midst of a crisis and help us budget for the right technology products?

Yeah, we thought so. Here are our five best practices to get there.

>> 1. Business leaders own the data, so make sure they take seats at the table.

Read: Even if you need to drag them there. "The business has to come first, and threats must be analyzed with business impact in mind," says the CISO of a $1.7 billion community bank who works very hard to ensure that business leaders fully comprehend the information at hand.

"Collaboration is essential to prioritization, as collaboration creates an atmosphere for questioning why we are doing things," he says. "There are no dumb questions." It's the job of every CISO to translate threats, impacts, and probabilities honestly and deliver that data clearly to executive management. However, most collaboration efforts by CISOs we work with are anything but clear. Rather, they tend to involve a lengthy PowerPoint presentation, a smattering of industry data, and a forced--if not wholly fabricated--ROI.

Stop with the insanity already. Executives don't care about industry data; their business is always different or better or stronger, no matter how similar it is to that company across town that just got breached. Don't tell the CEO that "98% of financial firms are subject to XYZ." She'll immediately assume you're in the rarefied 2% exempt from that mess. Instead, use scenarios that you have actually piloted internally. Involving business leaders in your discovery enhances their connection to the risk, the threat, and most important the results. So back away from the Twitter feed and start interacting directly with your colleagues.

chart: What's your main method to measure IT risks?

>> 2. Pick your sources carefully.

Confirmation bias doesn't stop at which threat to prioritize but often extends to which vendor or threat information source to listen to. Everyone wants to be the go-to resource, but consider motivation. If a CISO has a good relationship with a sales rep, for example, that vendor may get most of the security team's attention. In fact, vendors are, by far, the No. 1 source of trusted information among the CISOs we spoke with--a situation that, frankly, amazes us given all the top-notch security communities and conferences and the fact that partnering with the wrong vendor can waste a lot of money and time. A few CISOs we spoke with told horror stories of ripping and replacing gear from vendors that provided bad information. This does nothing to increase the CFO's view of IT's business savvy.

When we probed deeper about trusted information sources, we found that security engineers often talk about successes and failures with security engineers from other companies, but CISOs and directors of security don't share well. Fortunately, this trend seems to be changing, as many of the CISOs we spoke with had dedicated time and travel budget to participate in a peer group in 2011.

Robert Allen, CISO at CNA Insurance, recently participated in a CISO summit in Chicago. While he notes that conferences are never perfect, Allen was able to gather ideas that will shape his security plans.

>> 3. Embrace transparency.

5 Fast Tips To Effectively Assess Risk

If you use an outside vendor for security risk assessments, consider switching consultants every other year for a fresh perspective. Ask senior, nontechnical executives what keeps them awake at night. Their responses to this simple query are often eye-opening. To assess the risk posed by employees, consider sending a mock phishing e-mail to see who bites. Again, you may be surprised. Be discrete, but be clear about policies. Align assessed risk to the value of data. Sounds basic, but it doesn't always happen. Get your terminology in line with how the business thinks. The language of risk management isn't native to IT; the Factor Analysis of Information Risk (FAIR) framework of interconnected models is a great resource for CISOs looking to get up to speed.

"Awareness builds trust, and trust allows us to have an open and honest conversation with the business so the threats that are urgent and important are addressed," Allen says. His advice: Focus on the present--not on what could be but on what is. Continually measure and monitor your risk profile and the current threat level to reduce the chance of making a gut decision.

Of course, the only way to reduce threat noise is to filter it. GRC (governance, risk, and compliance) tools from companies such as Archer and Relational Security can help risk assessment and mitigation based on your reality, and thus can also support transparency, but don't get caught in the weeds of managing such a system. The data is what's important, not the GRC product's bells and whistles.

>> 4. Always test new threats to figure out if you have compensating controls.

Once you identify a threat, evaluate the real-world risk to your organization. Don't just listen to the risk assessment team and accept the probability and impact they suggest. Use Metasploit, or run the virus in a VM and see what happens. You may be surprised that the effect on your network is much different from what the vendor, analyst, or article says it will be. Got an emergency, where your team is running around in a panic? That's the best time to stop and think. We've seen countless instances where a team forgot or ignored compensating controls when assessing a risk during a perceived crisis.

>> 5. Don't wait for a crisis to set goals.

The best way to prioritize is to be proactive. However, it's human nature to dither when we have no time pressures. Adding to the malaise for CISOs is the fear we'll make a bad decision. So instead, we end up with paralysis by analysis. Should we buy DLP or IPS? Focus on hardening our servers or on policies and documenting processes? Every CISO we talked with had a war story of wasting resources on a technology that didn't help with the problem at hand. Many times, business leaders forced them to choose product X because of price; other times, they admit they got sold on it by vendor Y. Risk management and resource prioritization aren't sexy. But listening to your gut just isn't effective. There are too many variables, and attackers are too wily, numerous, and adaptive.

Test. Put processes in place. Accept transparency because it will increase collaboration with business leaders, which will then enable better prioritization of risks. Only once risks are identified and ranked should you look to external resources. It's your only hope to avoid security information overload.

InformationWeek: Mon. dd, 2011 Issue

InformationWeek: Mon. dd, 2011 Issue

Download a free PDF of Dark Reading January Digital Issue
(registration required)

About the Author(s)

Michael A. Davis

CTO of CounterTack

Michael A. Davis has been privileged to help shape and educate the globalcommunity on the evolution of IT security. His portfolio of clients includes international corporations such as AT&T, Sears, and Exelon as well as the U.S. Department of Defense. Davis's early embrace of entrepreneurship earned him a spot on BusinessWeek's "Top 25 Under 25"
list, recognizing his launch of IT security consulting firm Savid Technologies, one of the fastest-growing companies of its decade. He has a passion for educating others and, as a contributing author for the *Hacking Exposed* books, has become a keynote speaker at dozens of conferences and symposiums worldwide.

Davis serves as CTO of CounterTack, provider of an endpoint security platform delivering real-time cyberthreat detection and forensics. He joined the company because he recognized that the battle is moving to the endpoint and that conventional IT security technologies can't protect enterprises. Rather, he saw a need to deliver to the community continuous attack monitoring backed by automated threat analysis.

Davis brings a solid background in IT threat assessment and protection to his latest posting, having been Senior Manager Global Threats for McAfee prior to launching Savid, which was acquired by External IT. Aside from his work advancing cybersecurity, Davis writes for industry publications including InformationWeek and Dark Reading. Additionally, he has been a partner in a number of diverse entrepreneurial startups; held a leadership position at 3Com; managed two Internet service providers; and recently served as President/CEO of the InClaro Group, a firm providing information security advisory and consulting services based on a unique risk assessment methodology.

Keep up with the latest cybersecurity threats, newly discovered vulnerabilities, data breach information, and emerging trends. Delivered daily or weekly right to your email inbox.

You May Also Like


More Insights