Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Cloud

9/2/2020
10:00 AM
Robert Brennan
Robert Brennan
Commentary
100%
0%

Why Kubernetes Clusters Are Intrinsically Insecure (& What to Do About Them)

By following best practices and prioritizing critical issues, you can reduce the chances of a security breach and constrain the blast radius of an attempted attack. Here's how.

Teams new to Kubernetes often deploy clusters in an insecure way by default because they don't know what they don't know. Unless you've got a team of battle-hardened Kubernetes experts, you're bound to run into trouble. For example, it's not always obvious when a Kubernetes deployment is overpermissioned, and often the easiest way to get something working is to give it root access or cluster-admin permissions.

But just because the site is working doesn't mean your job is done. If you haven't tightened your security posture adhering to best practices, it's only a matter of time before you start learning lessons the hard way, whether that's through a denial-of-service (DoS) attack or something more severe. The good news is Kubernetes comes with built-in security tooling and a growing ecosystem of both open source and commercial solutions. You can build a security strategy that enables rapid development while maintaining a strong security posture.

Related Content:

How to Secure Your Kubernetes Deployment

Why DevSecOps Is Critical for Containers and Kubernetes

Stay Up-to-Date
The Kubernetes ecosystem is constantly evolving. But that doesn't just mean new features. It means bugs are squashed and security holes are patched every day. It's critical to stay up-to-date at every level of the stack. Once a vulnerability is announced, it's only a matter of time before someone designs an exploit. There are three kinds of updates to focus on:

1. Kubernetes itself. There's a new minor version of Kubernetes every quarter, and patch releases come out on a semiregular basis. Make sure you're monitoring CVE feeds for any new issues. Even better, many managed Kubernetes providers like GKE offer the ability to automatically upgrade your nodes, which helps ensure you're not running any known vulnerabilities.

2. Add-ons. You've likely installed some third-party tools such as nginx-ingress or cert-manager, which come with their own vulnerabilities and release cadence. Routinely check these for updates, or use a tool like Nova to monitor them for you.

3. Container Image Vulns. Each of the individual container images running in your cluster can have vulnerable software installed. You can run a container scanning solution like Trivy to help catch issues, but the best way to stay ahead is to ensure you're always on the latest version of the image. For images that are published with semver, you should at least unpin the patch version, so you're always pulling in any bug fixes.

Limiting the Blast Radius.
Kubernetes can't secure application code; there's nothing to stop developers from introducing bugs that might grant an attacker access to the host machine or an internal API. However, Kubernetes can place strong limits on the blast radius of an attack. When implemented, security teams can start to look at Kubernetes as an opportunity to improve security rather than as a threat.

Role-based access control (RBAC) is the first line of defense. It decides which Kubernetes resources a particular container should have access to and what it should be allowed to do with them. Some workloads might need to view application logs, while others need cluster-admin permission to create and delete other workloads. When creating RBAC profiles, make sure to adhere to the principle of least privilege.

Security teams must also pay close attention to the deployment configuration attached to each of your workloads. If a container is running as root, has access to the host's filesystem, or has some other security flaw, the attack can quickly spread throughout the cluster, compromising every workload. Tools like Polaris can help automate these checks.

Finally, look into additional lockdown mechanisms like Network Policy (which limits traffic in and out of particular pods) or Workload Identity (which ties RBAC to your cloud provider's authentication mechanism, such as IAM on Google Cloud or AWS).

Limiting Network Traffic
A DoS attack is one of the easiest to implement. An attacker simply has to clog your servers with traffic, preventing legitimate traffic from getting in. A genius of Kubernetes is its ability to autoscale to meet traffic increases, but scaling up is costly, and it doesn't happen instantaneously. Without the right limits in place, a DoS attack is sure to create some pain for your Ops team. The right ingress policy — including per-IP limits on the number of concurrent connections, on requests per second/minute/hour, and on the size of request bodies — will give a layer of protection against this type of threat and the cost associated. Depending on your ingress provider, these limits can also be configured per-application, and even per-IP or per-path, giving you flexibility to continue allowing certain users or certain endpoints to scale up quickly.

Maintaining Cluster Security
Even though your cluster will inevitably contain a few hidden vulnerabilities, by following the guidelines above you can strongly mitigate a successful attack and the magnitude of the fallout. As with any software environment, you should treat each workload as compromised, and work to contain it. But once you've built your cluster and tightened its security, how do you keep it that way? Developers are constantly shipping new code and configuration — how can you be sure it's in-line with your policies? Furthermore, how do you enforce those policies across many clusters and lines of business?

Look to a partner or platform to continuously monitor your cluster and enforce security policies. Without the right help, securing Kubernetes clusters can be a manual, time-consuming, and error-prone activity. But with the right help, you can deploy rapidly and without fear.

Robert Brennan is Director of Open Source Software at Fairwinds. Brennan works on the open source projects Polaris, which helps identify security misconfiguration, and Fairwinds Insights, a platform for Kubernetes security. View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
When It Comes To Security Tools, More Isn't More
Lamont Orange, Chief Information Security Officer at Netskope,  1/11/2021
US Capitol Attack a Wake-up Call for the Integration of Physical & IT Security
Seth Rosenblatt, Contributing Writer,  1/11/2021
IoT Vendor Ubiquiti Suffers Data Breach
Dark Reading Staff 1/11/2021
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
2020: The Year in Security
Download this Tech Digest for a look at the biggest security stories that - so far - have shaped a very strange and stressful year.
Flash Poll
Assessing Cybersecurity Risk in Today's Enterprises
Assessing Cybersecurity Risk in Today's Enterprises
COVID-19 has created a new IT paradigm in the enterprise -- and a new level of cybersecurity risk. This report offers a look at how enterprises are assessing and managing cyber-risk under the new normal.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2021-3113
PUBLISHED: 2021-01-17
Netsia SEBA+ through 0.16.1 build 70-e669dcd7 allows remote attackers to discover session cookies via a direct /session/list/allActiveSession request. For example, the attacker can discover the admin's cookie if the admin account happens to be logged in when the allActiveSession request occurs, and ...
CVE-2020-25533
PUBLISHED: 2021-01-15
An issue was discovered in Malwarebytes before 4.0 on macOS. A malicious application was able to perform a privileged action within the Malwarebytes launch daemon. The privileged service improperly validated XPC connections by relying on the PID instead of the audit token. An attacker can construct ...
CVE-2021-3162
PUBLISHED: 2021-01-15
Docker Desktop Community before 2.5.0.0 on macOS mishandles certificate checking, leading to local privilege escalation.
CVE-2021-21242
PUBLISHED: 2021-01-15
OneDev is an all-in-one devops platform. In OneDev before version 4.0.3, there is a critical vulnerability which can lead to pre-auth remote code execution. AttachmentUploadServlet deserializes untrusted data from the `Attachment-Support` header. This Servlet does not enforce any authentication or a...
CVE-2021-21245
PUBLISHED: 2021-01-15
OneDev is an all-in-one devops platform. In OneDev before version 4.0.3, AttachmentUploadServlet also saves user controlled data (`request.getInputStream()`) to a user specified location (`request.getHeader("File-Name")`). This issue may lead to arbitrary file upload which can be used to u...