A Day in the Life of a DevSecOps Manager

Ari Kalfus is DevSecOps manager at digital health company Rally Health. He leads the application security direction for the enterprise and coordinates its internal and external penetration testing programs.

Dark Reading: What are some of the first things you do when you get to work?
Ari Kalfus: Check my email, check Slack notifications, plan which gif from PG Tips I want to add to my async standup notes.

DR: When you start your day, do you feel good about what you'll be dealing with at work? Or concerned?
AK: Most days are good days. I think this is really dependent on the culture of the company you work for more so than something specific to the practice of DevSecOps. If you're supported by your company, it doesn't necessarily matter that you have a ton of thorns to pluck during the workday. But if you're not supported by your work environment, the most trivial of tasks will be grating.

DR: Break down your day for us.
AK:

10 a.m.— Time to start working. I participate in the security leadership sync meeting and discuss anything of primary concern for our department this week. For the next couple hours, I may have one-to-one conversations with my team or with partners in others areas of the organization, or perhaps an hour or two to make progress on an individual technical task.

Lunch! — My partner exclaims at how little water I've drunk so far this day. I used to be the one that made pointed looks at her while I refilled my water bottle yet again. The pandemic has affected us all.

Afternoon — The afternoon is typically either scheduled over with multiple meetings with groups looking for application security insight to an issue they are working on, communicating with other groups on a DevSecOps effort that spans across the org, or perhaps dedicated focus time to think about and make more longer-term plans for the appsec program or tackle a particularly thorny engineering problem to spare my team.

DR: Are there any particular focus areas or red flags that indicate to you it may be a trying day on the job?
AK: This may just be my personal view on application security, but I feel a component of DevSecOps is ensuring the security practices we require of engineering demonstrably reduce business risk. Some days, I have meetings with external customers, compliance folks, or others who do not necessarily agree. Those days are always trying.

DR: Examples, please?
AK: I don't care, for example, that an internal service uses TLS 1.1 to talk to another internal service. [This is a manufactured example.] It would be great if they moved to the company standard of 1.2, but I'm not going to be on top of the team to migrate. I do care that those two services encrypt sensitive data across that communication channel – so I want to ensure they are using the cryptographic library our DevSecOps team maintains.

However, for some people a number above zero on a report means bad. Zero means good! It's hard to come to a common understanding and path forward in those conversations. I think a defining principle when it comes to effective DevSecOps as opposed to more traditional application security approaches is the acceptance that context matters, and we should prioritize our time and our efforts in line with that context.

DR: What does a DevSecOps team do, really?
AK: The goal of a DevSecOps team, in my view, is embedding application security into development through enablement, iteration, and continuous feedback – also sometimes called "shifting security left." This requires talking to other folks and making sure you can offer them something that solves your problem while enabling them to solve theirs.

No one wants to "stop" producing value to take care of security concerns, which can often be how it feels to interact with security teams. Everyone already has a full roadmap. Why does this security concern need to be addressed now? Through a DevSecOps philosophy, which mostly means taking agile principles from engineering and applying them to security work, I use those aforementioned days of meetings to determine how a particular security concern can be mitigated or eradicated without adding friction to the development pipeline.

DR: How do you that? Address security, avoid friction, make everyone happy?
AK: This means ignoring many of the more traditional, legacy application security practices, like static scans that take hours and mostly produce false positives, in favor of products or custom implementations that are highly targeted toward specific classes of vulnerabilities present in our environment.

Our DevSecOps team, for example, can write a cryptography library for engineering that uses standard libraries in an appropriate manner, avoiding common implementation mistakes that could lead to data exposure.

Sometimes we may mandate a particular approach, but typically we offer a library like this to engineering and sell it as saving them development time. "Don't do it yourself! Just call the library method and move on with your roadmap! By the way, do this and our data protection security concerns get resolved." This lets engineers produce their features faster while largely eliminating a particular security issue from our environment.

This only works if the library or process you offer actually makes development easier for engineers. So my team's work is a collaboration with the engineers in our org to ensure the libraries, products, and tools we offer to engineering are serving their purpose – eliminating classes of security issues while enabling engineering to perform their work with less friction.

DR: So that's how you make engineers happy. What about keeping your own team happy?
AK: The other side of this is, the DevSecOps/Appsec team is going to be many orders of magnitude smaller than the engineering organization. In order to be effective at delivering solutions, we need to minimize the amount of operational burden our solutions create for our team.

This usually means turning to cloud-native tools that enable us to offer functionality while minimizing operational overhead – integrate webhooks and offer APIs that trigger AWS Lambda functions, for example, that allow us to scale across all of engineering without adding the burden of managing servers or dealing with limited load capacity. I'm not claiming cloud-native tools are magic solutions, but they allow for architectures that minimize our overhead beyond what's capable with traditional servers. We still have our share of bugs.  

While I wouldn't claim that an appsec team has more "customers" than an engineering team – you hopefully have more paying customers than you do engineers in your company – our customers sit, physically or virtually, right next to us, so if we produce content that isn't reliable or causes friction, we will hear about it! So we have to have really excellent engineering processes on our own projects such that we can be extremely confident about the reliability and functionality of the things we push into developers' workstreams.

DR: At the end of the day, what do you consider a "win"? How do you measure success?
AK: Are there fewer security concerns this month than last month? Are engineers today happy or upset with the processes we impose on them?