Global TeamCity Exploitation Opens Door to SolarWinds-Style Nightmare

APT29, the notorious Russian advanced persistent threat behind the 2020 SolarWinds hack, is actively exploiting a critical security vulnerability in JetBrains TeamCity that could open the door to rampant software supply chain attacks.

That's the word from CISA, the FBI, the NSA, and a host of international partners, who said in a joint alert today that APT29 (aka CozyBear, the Dukes, Midnight Blizzard, or Nobelium) is hammering servers hosting TeamCity software "at a large scale" using the unauthenticated remote code execution (RCE) bug. According to the feds, the exploitation of the issue, tracked as CVE-2023-42793 (CVSS score of 9.8), started in September after JetBrains patched the flaw and Rapid7 released a public proof-of-concept (PoC) exploit for it; but now, it has grown to be a worrying global phenomenon that could result in widespread damage.

The affected platform is a software development lifecycle (SDLC) management tool, which houses everything from source code to signing certificates. Successful incursions could give cyberattackers access to that valuable data, but could also provide a way to alter software compilations and deployment processes — raising the possibility that another SolarWinds-type attack wave could be in the offing.

"[An exploit] may allow for deploying a malicious update which, in the simplest scenario, could execute adversary tools resulting in enabling access to devices or whole networks," according to Wednesday's joint alert on the TeamCity attacks. "In more complicated scenarios, access to the build pipeline could allow for compromising compiled source code and for introduction of almost indetectable modification to software — such as minuscule changes to cryptography protocols that could enable decryption of the protected data."

Persistent TeamCity Backdoors Withstand Patching

In the SolarWinds incident, APT29 was able to stow away on legitimate SolarWinds software updates, landing automatically on legions of victim networks. From the 18,000 compromised, the group cherry-picked targets for second-wave incursions, successfully infiltrating several US government agencies and tech companies including Microsoft and FireEye (now part of Trellix).

For now, the TeamCity attacks have not yet gone that far. But APT29, which the agencies have linked to Russia's Foreign Intelligence Service (SVR), has "been observed using the initial access gleaned by exploiting the TeamCity CVE to escalate its privileges, move laterally, deploy additional backdoors, and take other steps to ensure persistent and long-term access to the compromised network environments," according to the alert.

And indeed, if you're a nation-state threat looking for prime lurking opportunities, one of the benefits of using the exploit is the fact that patching alone won't mitigate the danger. As JetBrains pointed out in its original bug advisory, "Any backdoors are likely to persist and remain undetected after the TeamCity upgrade or security patch plugin are subsequently applied, leaving environments at risk of further exploitation."

According to Shadowserver, there are at first glance at least 800 unpatched TeamCity software instances worldwide exposed to the Internet; it's unclear how many instances have been patched but may remain compromised. And of course, that number doesn't take into account unexposed instances that are reachable by sophisticated adversaries with prior access to corporate networks.

Flurry of APTs Target Developers Through CVE-2023-42793

APT29 is not the only state-sponsored cyberthreat to take notice of the tantalizing prizes on offer in vulnerable TeamCity instances. In October, Microsoft's Threat Intelligence Center pointed to several North Korea-backed APTs, including Lazarus Group (aka Diamond Sleet, Hidden Cobra, or Zinc) and its offshoot Andariel (aka Onyx Sleet or Plutonium), using the TeamCity vuln to install persistent backdoors.

And in some cases, there is more than one Big Bad at work. Researchers at cybersecurity firm Fortinet — which issued a deep-dive on Wednesday into the mechanics of a real-world incident at a US biomedical manufacturing company, along with indicators of compromise (IoC) and mitigation guidance — noted that "observed exploitation originated from multiple disparate threat actors who employed numerous diverse post-exploitation techniques in an attempt to gain a foothold in the victim network."

How to Protect Against JetBrains TeamCity Cyberattacks

To combat the danger posed by the TeamCity bug — i.e., "enormous damages for the economy, civilian organizations, or public safety," according to the joint alert — organizations should start by patching any vulnerable instances (to version 2023.05.4). From there, conducting active threat hunting based on the IoCs to uncover and remove persistent backdoors should be a top priority, according to Fortinet and Microsoft, both of which offer exhaustive guidance on that front. Both the TeamCity server and build agents should be vetted for signs of trouble.

JetBrains, in its CVE-2023-42793 security advisory, recommended that any publicly accessible servers be removed from the reach of the Internet while teams carry out patching and compromise investigations.

The company also warned that while researchers have observed Windows-based TeamCity environments being actively exploited, "this doesn't rule out Linux-based TeamCity environments also being exploited in similar ways."