Developers: The Cause of and Solution to Security's Biggest Problems

Figure 1:

When cybersecurity professionals look across the technology landscape and witness the crush of vulnerabilities, the risky configurations, the poorly protected APIs, and the insecure application architectures, it can be easy to blame developers for them all. 

In the wake of that finger-pointing, security pros tend to look for ways to work around developers, to build systems and procedures that are as likely to block devs from getting valuable work done as they are to protect these engineers from themselves. 

However, there's a sea change in the air.

Many veteran security pros who have been there and done that say the industry is looking at this in the wrong way. True, many security problems today arise as a by-product of a developer's work. But more than anything else, many of the issues have their roots in security's inability to adjust to a code-based world. 

"As security professionals, we've been trying to avoid applications and code for a long time. It was complicated. We didn't understand it. We didn't know what was going on inside it," says Hillel Solow, co-founder and CTO of Protego Labs. "We built an entire practice around, 'How do we build security controls around things without having to worry about what's happening inside them?' We put things in the front and the back, on the bottom and the top."

That approach is falling apart at the seams as security professionals deal with what Solow calls the "revolution of everything as code." The implications of poorly secured code creep far beyond the limited confines of Layer 7. With increasing enterprise reliance on cloud-native technologies like containerization and serverless deployments, the growing use of infrastructure-as-code to spin up and down new computing instances, the rise in software-defined networking, and the explosion in complexities from layering embedded systems upon embedded systems, everything always comes back to code. 

"We've moved over the past 30 years from hardware to software, layer by layer. And we're now at the point where the vast majority of what we deploy in our value is lines of code, whether it's, you know, C code, or Python code, or cloud formation templates, or Terraform, or something else," he says. "Those of us in security who don't understand code are really struggling to keep up."

Solow and many others like him believe the only way security can possibly keep up is if they stop blocking developers and start enlisting their help. Forward-looking technology experts predict that smart developers are the industry's best hope for solving some of the biggest security problems today. 

Making Developers Your Highest Hiring Priority
Seeking out developer assistance starts from within. Many longtime appsec advocates believe security managers need to kick off a change in tactics and strategy by putting developer resumes at the top of the stack for new hire candidates.

"Hiring developers on your team allows you to better speak the language of developers and better influence software development to positively impact security," says Zane Lackey, co-founder and CSO at Signal Sciences. As a former CISO at Etsy, Lackey says one of the best hires he ever made was when he snagged someone internally from his firm's software engineering team for the security crew to focus on development tasks that would positively impact security. 

These kinds of tasks can include building security tools and integrations, but they also extend far beyond that. 

Security developers can bring a fresh perspective to the table when the security team convenes to make policies and procedures — they'll help everyone line up security requirements with business priorities and development realities. They may also take a hand in transforming those developer-friendly policies into security-as-code and what Lackey calls a "golden path" of secure configurations, common libraries, or third-party code that can be shared across engineering projects, encryption standards, and so on. 

"Investing in bringing developers on those security teams can help them build things that are going to be directly consumed by engineers," Lackey says.

He is far from an outlier in this view that security needs to hire more developers. Hit up security and DevOps conferences today, and you'll increasingly run across security managers who are pushing hard for the industry to prioritize development experience.

"I only hire developers; I don't hire security people anymore," says John Melton, application security senior manager at Oracle NetSuite. "If you're a security person and you can't code, you should learn how, or you should hire people on your team who know how to code."

As Melton explains, the lack of development knowledge is endemic in the security world, and it's hurting security teams in so many ways. He's far from the only one to voice those concerns. According to Larry Maccherone, who runs the DevSecOps transformation at Comcast as senior director in the technology and product division's security and privacy group, a lack of developers on security teams does the most damage to the team's credibility. 

"I believe you can't keep telling developers how to do their job if you aren't also writing code," says Maccherone, who, similar to Melton, almost exclusively only hires developers these days. Maccherone says it is much easier to train a developer in security fundamentals and mindset than it is to teach a security vet how to code. He thinks the only way to bring the kind of street cred to the table necessary for eye-to-eye conversations between security and software engineers is to bring experienced devs into the security fold.  

In the process, the industry may well be able to kill two birds with one stone. With the industry struggling to make qualified security hires, it may be time to stop looking for security unicorn hires and start recruiting trainable developers to fill in some vital gaps. 

It doesn't even necessarily have to be as extreme as how Melton or Maccherone approach things. 

{Continued on next page}

According to Protego Labs' Solow, the role of a security software engineer is not necessarily a novel idea. For a long time security teams have employed stray developers now and again to create scripts and build out security systems for them. But they tend to pigeonhole these specialists. Rather than starting with developer new hires, he suggests security managers set out to retrain the ones who may already be hiding in the proverbial basement.  

"Organizations have known they needed an engineering team to build stuff for them, but those developers were never part of the policy-making or a part of the larger security practice," he says. "You probably have some really talented developers that know security. How about finding an expanded role for them? Embrace the developers you have at home first — that's going to be the quickest win."

These existing resources can then be the vanguard for setting new application security policies and procedures before adding new developers to the security team's ranks.

Democratizing Security Across the Development Corps
Of course, hiring developers internally to the security team is just the start to making the most headway on cyber-risk. The whole point of getting that internal help from devs is to leverage them as force multipliers. A small team of security developers can be leaned on to start the collaboration necessary to extend security into the hearts and minds of all of the other nonsecurity developers out in the organization. 

"The only way for security to scale is by enabling the rest of the business to be security self-sufficient," Lackey explains. 

The biggest security salvation for large enterprises lies in the power of developers at large — the software engineers delivering software to business stakeholders and customers. Security cynics may say these developers don't care about security, but that has never been true. 

"Developers care about creating a high-quality product, and they're well aware that if their product gets hacked, then that's not quality," says Tanya Janca, CEO and co-founder of early stage startup Security Sidekick and a former application security advocate at Microsoft. "But I also think that most of them feel pretty frustrated and overwhelmed that they don't know all the things they need to do or have the tools to do them. From what I see, there's little to no support for them."

As an example, Janca relates the story of a place she worked at a while back where the developers "rolled their own crypto because they couldn't get a straight answer from the security team about what to use."  

This brings home the role that security should be playing to help all developers in the organization fulfill their own drive and desire to deliver quality (and, consequently, secure) software. Rather than being the gatekeepers of yesteryear, Maccherone says security should be coaches and toolsmiths for developers. 

"We start off by saying, 'Security trusts you.' We trust that you are close to the business and will trade off security risks with other risks, and that's OK. We want to make sure you understand what you're doing when you make that trade-off," Maccherone says. "We're going to advise you on that risk and also lower the cost of you adopting these right practices.

This makes it possible to democratize a lot of the daily security work out to well-trained and well-supported engineers on the ground. At Comcast, the security team has done so with a systematic training program that has had the security team scale out security training and build up security champions across hundreds of development teams and legions of developers using just a small team of security development enablers. 

Related Content:

(Image: maciek905 via Adobe Stock)

This free, all-day online conference offers a look at the latest tools, strategies, and best practices for protecting your organization’s most sensitive data. Click for more information and, to register, here.