Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


End of Bibblio RCM includes -->
05:10 PM
Connect Directly

SolarWinds Experimenting With New Software Build System in Wake of Breach

CISO of SolarWinds now has complete autonomy to stop product releases if security concerns exist, CEO says.

SolarWinds is experimenting with a completely new software build process that CEO Sudhakar Ramakrishna says is designed to ensure much better security against intrusions of the sort that the company disclosed last December.

In addition, SolarWinds' CISO has been given full autonomy to stop product releases from happening purely due to time-to-market reasons. A new committee for cybersecurity has also been established at the board level, which includes the CEO and two CIOs, Ramakrishna said in comments during a virtual panel discussion this week involving security leaders from multiple organizations.

Related Content:

How SolarWinds Busted Up Our Assumptions About Code Signing

Special Report: How Data Breaches Affect the Enterprise

New From The Edge: A Day in the Life of a DevSecOps Manager

The measures are part of several changes that Ramakrishna says he has implemented since SolarWinds disclosed a breach of its systems three months ago that resulted in malware called Sunburst being distributed to some 18,000 customers worldwide. A substantially smaller number of them—including FireEye and Mimecast—were subsequently targeted for further compromise.

The intrusion involved attackers gaining access to SolarWinds' software-build environment and injecting the Trojan into automatic updates of the company's Orion network management software.

According to Ramakrishna, the company's investigation shows the attackers somehow gained access to its build environment and managed to inject code dubbed Sunspot into a single source-code file that was fetched from the file system as software was being compiled. The attackers later used Sunspot to inject the Sunburst Trojan into the Orion updates.

The attack had nothing to do with SolarWinds' source-code control systems, and neither did it result in any source code getting changed: "What simply happened was during the automated build process, the Sunspot code, which was sitting in memory quietly was watching for one file to be fetched," Ramakrishna said. The malware then essentially flipped the file and the compilation just continued. "So, you would not find source code logs; you could not go back in time and look at it," he said.

The entire process of modifying one file with malware happened in a window of a few milliseconds - and in memory - as the software was being compiled. The software was later digitally signed and sent out in automated fashion to customers of Orion.

Multiple Build Systems and Pipelines

To ensure that attackers cannot pull off a similar caper, SolarWinds is seeing if it can design its software-build systems and pipelines a bit differently.

Instead of a single build system in a single location, the company is looking at possibly running two or three parallel build systems through two or three parallel build chains. The goal is to see if SolarWinds can establish software integrity across multiple pipelines to avoid supply chain attacks of the kind it experienced a few months ago, Ramakrishna says. An attacker would have to be right three different times, identically, to be able to conduct an attack like the recent one with Orion. SolarWinds has also implemented a two-way hashing algorithm to further establish the integrity of its software.

The idea is to get to a point where code-signing certificates won't be the only way to establish software integrity. SolarWinds wants to "actually build a level of non-repudiation in the code to say, 'I underwrite and undersign what I am delivering,'" Ramakrishna said.

As part of the effort, SolarWinds is also implementing processes to ensure its CISO has more independence and authority. "We are creating an independent organization to build that level of capability and comfort," Ramakrishna said. "Having that level of independence, confidence, and aircover is supremely important. Otherwise, [security] becomes a cost line-item and they get sidelined."

SolarWinds is still in the process of trying to figure out how the attacker—believed to be a sophisticated nation-state-backed actor—managed to infiltrate the company's build environment in the first place.

At one point, incident investigators had as many as 16 different hypotheses for might have happened. They have now whittled that number down to three by a process of elimination, but have not yet been able to narrow it down any further.

Ramakrishna said that the three current theories are that the attackers either got in via a very targeted spear-phishing attack; or by exploiting an unpatched vulnerability in some third-party software at that point in time; or via a credential compromise.

He said that available evidence suggests the attacker behind the operation is very sophisticated and highly organized. 

Jai Vijayan is a seasoned technology reporter with over 20 years of experience in IT trade journalism. He was most recently a Senior Editor at Computerworld, where he covered information security and data privacy issues for the publication. Over the course of his 20-year ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Author
4/1/2021 | 7:42:12 PM
Positive step
This is good news and hopefully other companies will follow this example. Securing the software supply chain and the code signing infrastructure should be part of the security operation of every company that operates a service or develops software.
I Smell a RAT! New Cybersecurity Threats for the Crypto Industry
David Trepp, Partner, IT Assurance with accounting and advisory firm BPM LLP,  7/9/2021
Attacks on Kaseya Servers Led to Ransomware in Less Than 2 Hours
Robert Lemos, Contributing Writer,  7/7/2021
It's in the Game (but It Shouldn't Be)
Tal Memran, Cybersecurity Expert, CYE,  7/9/2021
Register for Dark Reading Newsletters
White Papers
Current Issue
The 10 Most Impactful Types of Vulnerabilities for Enterprises Today
Managing system vulnerabilities is one of the old est - and most frustrating - security challenges that enterprise defenders face. Every software application and hardware device ships with intrinsic flaws - flaws that, if critical enough, attackers can exploit from anywhere in the world. It's crucial that defenders take stock of what areas of the tech stack have the most emerging, and critical, vulnerabilities they must manage. It's not just zero day vulnerabilities. Consider that CISA's Known Exploited Vulnerabilities (KEV) catalog lists vulnerabilitlies in widely used applications that are "actively exploited," and most of them are flaws that were discovered several years ago and have been fixed. There are also emerging vulnerabilities in 5G networks, cloud infrastructure, Edge applications, and firmwares to consider.
Flash Poll
How Enterprises are Developing Secure Applications
How Enterprises are Developing Secure Applications
Recent breaches of third-party apps are driving many organizations to think harder about the security of their off-the-shelf software as they continue to move left in secure software development practices.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2023-03-17
The Bookly plugin for WordPress is vulnerable to Stored Cross-Site Scripting via the full name value in versions up to, and including, 21.5 due to insufficient input sanitization and output escaping. This makes it possible for unauthenticated attackers to inject arbitrary web scripts in pages that w...
PUBLISHED: 2023-03-17
The WP Express Checkout plugin for WordPress is vulnerable to Stored Cross-Site Scripting via the ‘pec_coupon[code]’ parameter in versions up to, and including, 2.2.8 due to insufficient input sanitization and output escaping. This makes it possible for authenti...
PUBLISHED: 2023-03-17
A vulnerability was found in SourceCodester Student Study Center Desk Management System 1.0. It has been rated as critical. This issue affects the function view_student of the file admin/?page=students/view_student. The manipulation of the argument id with the input 3' AND (SELECT 2100 FROM (SELECT(...
PUBLISHED: 2023-03-17
A vulnerability classified as critical has been found in SourceCodester Student Study Center Desk Management System 1.0. Affected is an unknown function of the file Master.php?f=delete_img of the component POST Parameter Handler. The manipulation of the argument path with the input C%3A%2Ffoo.txt le...
PUBLISHED: 2023-03-17
A vulnerability classified as critical was found in SourceCodester Student Study Center Desk Management System 1.0. Affected by this vulnerability is an unknown functionality of the file admin/?page=reports&date_from=2023-02-17&date_to=2023-03-17 of the component Report Handler. The manipula...