Part 1 of a two-part series about securing machine learning.

Gary McGraw Ph.D., Co-Founder, Berryville Institute of Machine Learning

June 12, 2020

5 Min Read

Every decade or so, we get an opportunity to build security in as a new field blossoms into the world. Mobile security provided one such opportunity about a decade ago. Though security people have been known to complain about the state of mobile phone security compared with, say, Windows 95, that's like comparing day to night.

Fast forward to today. One of the most important technical developments we're currently experiencing in hyper-real time is the "AI Renaissance." AI and, in particular, machine learning (ML) have been around for decades, but a resurgence in ML technology driven by incredible modern computational power and unimaginably large collections of well-organized data has even the most skeptical greybeards impressed.

Of course, security has an important role to play in ML (not using ML for security, mind you, but rather securing ML technology itself). In this two-part series, I'll explore ML security through the lens of building security in. We'll start with a quick refresher on software security.

7 Touchpoints for Software Security
In my 2007 book Software Security, I define and discuss seven touchpoints for software security. The touchpoints were intentionally created with reference to software artifacts (think source code or test cases) as opposed to any particular process for creating the artifacts. As a result, development life cycles like the waterfall methodology. In fact, the touchpoints are not a software methodology at all; they are meant to be process-agnostic.

Touchpoints are a mix of destructive and constructive activities. Destructive activities are about attacks, exploits, and breaking software. These kinds of things are represented by the black hat (offense). Constructive activities are about design, defense, and functionality. These are represented by the white hat (defense). Both hats are necessary:

My yin/yang cowboy hats logo above is based on the understanding that good security blends offense and defense in an inextricable balance. 

In the distant past, I presented the software security touchpoints in order from left to right. Although that works well, a better pedagogical approach is to order the touchpoints by their natural utility and present them in some sort of ranking. Some touchpoints are by their very nature more powerful than others, and the most powerful ones should be adopted first.

Here are the touchpoints, in what I consider the best order of effectiveness: 

1. Code review: Reviewing source code with an automated tool like Coverity, Fortify, or SpotBugs is a great way to carry out this touchpoint. This touchpoint finds (and hopefully fixes) security bugs.

2. Architectural risk analysis (ARA): Sometimes improperly called "threat modeling," ARA seeks to identify, rank, and mitigate design flaws. Design flaws account for 50% of software security defects in most software.

3. Penetration testing: Dynamically probing a running system for security problems is an important activity but often happens after the fact, making fixes that are much more difficult and costly. Note that pen testing should definitely not be the first and only touchpoint you employ.

4. Risk-based security tests: Testing security functionality, probing design flaws identified in an ARA, and regression testing against security bugs make up risk-based security testing.

5. Abuse cases: Many projects build use cases to understand how a system is meant to work. Abuse cases help determine what a system will do when it is under attack. Thinking about misuse before it happens is very powerful.

6. Security requirements: Just how secure does your system need to be? Who will attack it and why?  And most importantly, how much can you afford to do about it? These are questions addressed in security requirements.

7. Security operations: Even a system that is designed for security, properly pen tested, and debugged during development will be subject to attack. Keeping an eye on a running system (and making that task easier) is part of security ops.

In the field, the touchpoints have evolved over time into well-understood practices (some better understood and practiced than others). Many aspects of the touchpoints can, for example, be found in the 119 activities defined and measured by the BSIMM.

When a new technology wave sweeps over the security discipline - such as mobile code security, IoT security, or ML security - one important exercise is to think about how the seven touchpoints can be applied in order to make security progress.

When it comes to many technologies, source-code analysis is the easiest security touchpoint to apply first. Why that is the case should be obvious: Regardless of the process you may have used to come up with your code, your code can be subjected to static analysis. That is, just about every software project has code. Well, to a point: Static analysis of a dynamic node.js assembly may not be possible depending on when, where, and how the assembly is put together. In fact, the move to dynamic languages is having a deep impact on the base effectiveness of code review using a static analysis tool. 

Likewise, a DevOps approach elevates the importance of security operations (touchpoint 7), which is now defined in code itself. Containers are code, and container configuration is code. Container orchestration is code, too! So securing a system by design obviously must include operational aspects that may have been left to the IT guys in the past.

All details aside, there is a basic fundamental agreement that the software security touchpoints have a critical role to play in securing software (a field sometimes called application security).

In part two of this article, we'll explore these touchpoints and the very idea of how building in security apply to ML. 

VIRTUALSUMMIT_DR20_320x50.jpg

 

 

 

 

 

 

Learn from industry experts in a setting that is conducive to interaction and conversation about how to prepare for that "really bad day" in cybersecurity. Click for more information and to register

 

About the Author(s)

Gary McGraw Ph.D.

Co-Founder, Berryville Institute of Machine Learning

Gary McGraw is co-founder of the Berryville Institute of Machine Learning where his work focuses on machine learning security. He is a globally recognized authority on software security and the author of eight best selling books on this topic. His titles include Software Security, Exploiting Software, Building Secure Software, Java Security, Exploiting Online Games, and 6 other books; and he is editor of the Addison-Wesley Software Security series.  Dr. McGraw has also written over 100 peer-reviewed scientific publications. Gary serves on the Advisory Boards of Calypso AI, Legit, Irius Risk, Maxmyinterest, Protopia AI, and Red Sift.  He has also served as a Board member of Cigital and Codiscope (acquired by Synopsys) and as Advisor to CodeDX (acquired by Synopsys), Black Duck (acquired by Synopsys), Dasient (acquired by Twitter), Fortify Software (acquired by HP), and Invotas (acquired by FireEye). Gary produced the monthly Silver Bullet Security Podcast for IEEE Security & Privacy magazine for thirteen years. His dual PhD is in Cognitive Science and Computer Science from Indiana University where he serves on the Dean’s Advisory Council for the Luddy School of Informatics, Computing, and Engineering. 

Keep up with the latest cybersecurity threats, newly discovered vulnerabilities, data breach information, and emerging trends. Delivered daily or weekly right to your email inbox.

You May Also Like


More Insights