There is a general misunderstanding about what makes a vulnerability dangerous. Hype and publicity tend to be focused on the most advanced threats and tactics. In response to this, security teams focus more on controlling these advanced attacks rather than the more mundane ones, largely because the business supports these sensational cases more easily — at least until the memory has faded.
Consequently, security controls are often incomplete at the lower levels, leaving a wobbly foundation to build more advanced controls to counter more advanced threats. The result: Threats can breach at modest levels anyway, and the advanced controls become symbols of overspending and poor execution.
Often, it is the simplest vulnerabilities that are leveraged to breach a system. The reason for this: The easier a vulnerability is to exploit, the higher the number of threat actors that can, and actually will, exploit that vulnerability. It is a simple numbers game, yet the CISO and security team have a real problem framing this security story in a way that is both accurate and meaningful for executive leadership.
What I've discovered in almost two decades of attack simulation (including penetration testing and red/blue/purple teaming) as well as developing and advising these programs globally is that breaking in is still relatively easy. This does not mean that all the security work organizations have done is wasted or poor.
Often, security programs have covered many solid things, but they are uncalibrated and unbalanced, and they have not been integrated effectively. Foundationally, security should be about basic coverage (closing all the easy doors) before it is about elite capability (closing some of the doors very securely).
When done right, attack simulation can measure individual control performance. Equally valuable, it can measure the performance of parts of the security ecosystem (that is, prevent, detect, respond), and the ecosystem as a whole (impact mitigation). By doing so, it can also strongly indicate budget and resource performance, including over- and underspending. It is the ultimate form of security program assurance.
Here are three issues that undermine successful attack simulation and its strategic and tactical influence in business.
1. Attack simulations are often pitched and perceived as "advanced vulnerability assessment." This almost forces the business to treat them as a commodity. They are poorly scoped, funded, and resourced, and thus can mimic only modest or unrealistic threat scenarios. Tactically, this provides a false sense of security but also eviscerates its unique strategic value proposition — controls and program assurance.
2. Second are what I call the "Hack Olympics." White-hat hackers are very proud and competitive and like to show off to their peers. They will often try fancy new attack vectors for a quick "breach win" and not explore a myriad of more modest, but more common, breach scenarios. Often, this behavior is supported by their employers because they want to demonstrate strong value back to the customer — and to one-up competitors — and they believe that is done by doing something that less capable (that is, commodity) testers cannot do. In their view, this helps justify increased rates (rightly so) and should lead to customer loyalty (not necessarily).
The good news is that we can integrate the above issues into a business-savvy win. Modest cost/sophistication attack simulations can be framed as exactly that: efforts to cost-effectively discover modest breach scenarios. These can cover, say, the bottom 70% of scope. Next, leverage more sophisticated resources for the top 30%. This model demonstrates strong strategic and tactical value, as well as shrewd budget utilization.
3. Uninspired and stagnant reports are globally pervasive and undermine even great attack simulations. Reports fail to calibrate the difficulty level to breach and affect high-value business assets. They do not thoroughly explain the attacker’s decision process/tree/options, as well as which controls frustrated them and which controls could have and should have but did not — and why. Such reports rarely link the story of threat benefit (how hard are threats willing to try?) to business impact (how much do I really care?). An attack simulation report should wrap around a story arc like Ocean’s 11. It should be gripping and easy to understand to executives (goals and impacts to both sides), while showing a decision and execution path for SecOps to effect change.
Undeniably, attack simulation is a critical component of a robust security program. However, several issues undermine the quality and influence of these attack scenarios. This leads to inconsistent security capability, unbalanced budget allocation, uncalibrated security strategy, fear of breach at any moment, and frustration with SecOps, the CISO, and business executive leadership.