Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Cloud

9/2/2020
10:00 AM
Robert Brennan
Robert Brennan
Commentary
100%
0%

Why Kubernetes Clusters Are Intrinsically Insecure (& What to Do About Them)

By following best practices and prioritizing critical issues, you can reduce the chances of a security breach and constrain the blast radius of an attempted attack. Here's how.

Teams new to Kubernetes often deploy clusters in an insecure way by default because they don't know what they don't know. Unless you've got a team of battle-hardened Kubernetes experts, you're bound to run into trouble. For example, it's not always obvious when a Kubernetes deployment is overpermissioned, and often the easiest way to get something working is to give it root access or cluster-admin permissions.

But just because the site is working doesn't mean your job is done. If you haven't tightened your security posture adhering to best practices, it's only a matter of time before you start learning lessons the hard way, whether that's through a denial-of-service (DoS) attack or something more severe. The good news is Kubernetes comes with built-in security tooling and a growing ecosystem of both open source and commercial solutions. You can build a security strategy that enables rapid development while maintaining a strong security posture.

Related Content:

How to Secure Your Kubernetes Deployment

Why DevSecOps Is Critical for Containers and Kubernetes

Stay Up-to-Date
The Kubernetes ecosystem is constantly evolving. But that doesn't just mean new features. It means bugs are squashed and security holes are patched every day. It's critical to stay up-to-date at every level of the stack. Once a vulnerability is announced, it's only a matter of time before someone designs an exploit. There are three kinds of updates to focus on:

1. Kubernetes itself. There's a new minor version of Kubernetes every quarter, and patch releases come out on a semiregular basis. Make sure you're monitoring CVE feeds for any new issues. Even better, many managed Kubernetes providers like GKE offer the ability to automatically upgrade your nodes, which helps ensure you're not running any known vulnerabilities.

2. Add-ons. You've likely installed some third-party tools such as nginx-ingress or cert-manager, which come with their own vulnerabilities and release cadence. Routinely check these for updates, or use a tool like Nova to monitor them for you.

3. Container Image Vulns. Each of the individual container images running in your cluster can have vulnerable software installed. You can run a container scanning solution like Trivy to help catch issues, but the best way to stay ahead is to ensure you're always on the latest version of the image. For images that are published with semver, you should at least unpin the patch version, so you're always pulling in any bug fixes.

Limiting the Blast Radius.
Kubernetes can't secure application code; there's nothing to stop developers from introducing bugs that might grant an attacker access to the host machine or an internal API. However, Kubernetes can place strong limits on the blast radius of an attack. When implemented, security teams can start to look at Kubernetes as an opportunity to improve security rather than as a threat.

Role-based access control (RBAC) is the first line of defense. It decides which Kubernetes resources a particular container should have access to and what it should be allowed to do with them. Some workloads might need to view application logs, while others need cluster-admin permission to create and delete other workloads. When creating RBAC profiles, make sure to adhere to the principle of least privilege.

Security teams must also pay close attention to the deployment configuration attached to each of your workloads. If a container is running as root, has access to the host's filesystem, or has some other security flaw, the attack can quickly spread throughout the cluster, compromising every workload. Tools like Polaris can help automate these checks.

Finally, look into additional lockdown mechanisms like Network Policy (which limits traffic in and out of particular pods) or Workload Identity (which ties RBAC to your cloud provider's authentication mechanism, such as IAM on Google Cloud or AWS).

Limiting Network Traffic
A DoS attack is one of the easiest to implement. An attacker simply has to clog your servers with traffic, preventing legitimate traffic from getting in. A genius of Kubernetes is its ability to autoscale to meet traffic increases, but scaling up is costly, and it doesn't happen instantaneously. Without the right limits in place, a DoS attack is sure to create some pain for your Ops team. The right ingress policy — including per-IP limits on the number of concurrent connections, on requests per second/minute/hour, and on the size of request bodies — will give a layer of protection against this type of threat and the cost associated. Depending on your ingress provider, these limits can also be configured per-application, and even per-IP or per-path, giving you flexibility to continue allowing certain users or certain endpoints to scale up quickly.

Maintaining Cluster Security
Even though your cluster will inevitably contain a few hidden vulnerabilities, by following the guidelines above you can strongly mitigate a successful attack and the magnitude of the fallout. As with any software environment, you should treat each workload as compromised, and work to contain it. But once you've built your cluster and tightened its security, how do you keep it that way? Developers are constantly shipping new code and configuration — how can you be sure it's in-line with your policies? Furthermore, how do you enforce those policies across many clusters and lines of business?

Look to a partner or platform to continuously monitor your cluster and enforce security policies. Without the right help, securing Kubernetes clusters can be a manual, time-consuming, and error-prone activity. But with the right help, you can deploy rapidly and without fear.

Robert Brennan is Director of Open Source Software at Fairwinds. Brennan works on the open source projects Polaris, which helps identify security misconfiguration, and Fairwinds Insights, a platform for Kubernetes security. View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 9/25/2020
9 Tips to Prepare for the Future of Cloud & Network Security
Kelly Sheridan, Staff Editor, Dark Reading,  9/28/2020
Malware Attacks Declined But Became More Evasive in Q2
Jai Vijayan, Contributing Writer,  9/24/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
Special Report: Computing's New Normal
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
How IT Security Organizations are Attacking the Cybersecurity Problem
How IT Security Organizations are Attacking the Cybersecurity Problem
The COVID-19 pandemic turned the world -- and enterprise computing -- on end. Here's a look at how cybersecurity teams are retrenching their defense strategies, rebuilding their teams, and selecting new technologies to stop the oncoming rise of online attacks.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-15216
PUBLISHED: 2020-09-29
In goxmldsig (XML Digital Signatures implemented in pure Go) before version 1.1.0, with a carefully crafted XML file, an attacker can completely bypass signature validation and pass off an altered file as a signed one. A patch is available, all users of goxmldsig should upgrade to at least revisio...
CVE-2020-4607
PUBLISHED: 2020-09-29
IBM Security Secret Server (IBM Security Verify Privilege Vault Remote 1.2 ) could allow a local user to bypass security restrictions due to improper input validation. IBM X-Force ID: 184884.
CVE-2020-24565
PUBLISHED: 2020-09-29
An out-of-bounds read information disclosure vulnerabilities in Trend Micro Apex One may allow a local attacker to disclose sensitive information to an unprivileged account on vulnerable installations of the product. An attacker must first obtain the ability to execute low-privileged code on the ...
CVE-2020-25770
PUBLISHED: 2020-09-29
An out-of-bounds read information disclosure vulnerabilities in Trend Micro Apex One may allow a local attacker to disclose sensitive information to an unprivileged account on vulnerable installations of the product. An attacker must first obtain the ability to execute low-privileged code on the ...
CVE-2020-25771
PUBLISHED: 2020-09-29
An out-of-bounds read information disclosure vulnerabilities in Trend Micro Apex One may allow a local attacker to disclose sensitive information to an unprivileged account on vulnerable installations of the product. An attacker must first obtain the ability to execute low-privileged code on the ...