Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Cloud

9/2/2020
10:00 AM
Robert Brennan
Robert Brennan
Commentary
100%
0%

Why Kubernetes Clusters Are Intrinsically Insecure (& What to Do About Them)

By following best practices and prioritizing critical issues, you can reduce the chances of a security breach and constrain the blast radius of an attempted attack. Here's how.

Teams new to Kubernetes often deploy clusters in an insecure way by default because they don't know what they don't know. Unless you've got a team of battle-hardened Kubernetes experts, you're bound to run into trouble. For example, it's not always obvious when a Kubernetes deployment is overpermissioned, and often the easiest way to get something working is to give it root access or cluster-admin permissions.

But just because the site is working doesn't mean your job is done. If you haven't tightened your security posture adhering to best practices, it's only a matter of time before you start learning lessons the hard way, whether that's through a denial-of-service (DoS) attack or something more severe. The good news is Kubernetes comes with built-in security tooling and a growing ecosystem of both open source and commercial solutions. You can build a security strategy that enables rapid development while maintaining a strong security posture.

Related Content:

How to Secure Your Kubernetes Deployment

Why DevSecOps Is Critical for Containers and Kubernetes

Stay Up-to-Date
The Kubernetes ecosystem is constantly evolving. But that doesn't just mean new features. It means bugs are squashed and security holes are patched every day. It's critical to stay up-to-date at every level of the stack. Once a vulnerability is announced, it's only a matter of time before someone designs an exploit. There are three kinds of updates to focus on:

1. Kubernetes itself. There's a new minor version of Kubernetes every quarter, and patch releases come out on a semiregular basis. Make sure you're monitoring CVE feeds for any new issues. Even better, many managed Kubernetes providers like GKE offer the ability to automatically upgrade your nodes, which helps ensure you're not running any known vulnerabilities.

2. Add-ons. You've likely installed some third-party tools such as nginx-ingress or cert-manager, which come with their own vulnerabilities and release cadence. Routinely check these for updates, or use a tool like Nova to monitor them for you.

3. Container Image Vulns. Each of the individual container images running in your cluster can have vulnerable software installed. You can run a container scanning solution like Trivy to help catch issues, but the best way to stay ahead is to ensure you're always on the latest version of the image. For images that are published with semver, you should at least unpin the patch version, so you're always pulling in any bug fixes.

Limiting the Blast Radius.
Kubernetes can't secure application code; there's nothing to stop developers from introducing bugs that might grant an attacker access to the host machine or an internal API. However, Kubernetes can place strong limits on the blast radius of an attack. When implemented, security teams can start to look at Kubernetes as an opportunity to improve security rather than as a threat.

Role-based access control (RBAC) is the first line of defense. It decides which Kubernetes resources a particular container should have access to and what it should be allowed to do with them. Some workloads might need to view application logs, while others need cluster-admin permission to create and delete other workloads. When creating RBAC profiles, make sure to adhere to the principle of least privilege.

Security teams must also pay close attention to the deployment configuration attached to each of your workloads. If a container is running as root, has access to the host's filesystem, or has some other security flaw, the attack can quickly spread throughout the cluster, compromising every workload. Tools like Polaris can help automate these checks.

Finally, look into additional lockdown mechanisms like Network Policy (which limits traffic in and out of particular pods) or Workload Identity (which ties RBAC to your cloud provider's authentication mechanism, such as IAM on Google Cloud or AWS).

Limiting Network Traffic
A DoS attack is one of the easiest to implement. An attacker simply has to clog your servers with traffic, preventing legitimate traffic from getting in. A genius of Kubernetes is its ability to autoscale to meet traffic increases, but scaling up is costly, and it doesn't happen instantaneously. Without the right limits in place, a DoS attack is sure to create some pain for your Ops team. The right ingress policy — including per-IP limits on the number of concurrent connections, on requests per second/minute/hour, and on the size of request bodies — will give a layer of protection against this type of threat and the cost associated. Depending on your ingress provider, these limits can also be configured per-application, and even per-IP or per-path, giving you flexibility to continue allowing certain users or certain endpoints to scale up quickly.

Maintaining Cluster Security
Even though your cluster will inevitably contain a few hidden vulnerabilities, by following the guidelines above you can strongly mitigate a successful attack and the magnitude of the fallout. As with any software environment, you should treat each workload as compromised, and work to contain it. But once you've built your cluster and tightened its security, how do you keep it that way? Developers are constantly shipping new code and configuration — how can you be sure it's in-line with your policies? Furthermore, how do you enforce those policies across many clusters and lines of business?

Look to a partner or platform to continuously monitor your cluster and enforce security policies. Without the right help, securing Kubernetes clusters can be a manual, time-consuming, and error-prone activity. But with the right help, you can deploy rapidly and without fear.

Robert Brennan is Director of Open Source Software at Fairwinds. Brennan works on the open source projects Polaris, which helps identify security misconfiguration, and Fairwinds Insights, a platform for Kubernetes security. View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
Commentary
How SolarWinds Busted Up Our Assumptions About Code Signing
Dr. Jethro Beekman, Technical Director,  3/3/2021
News
'ObliqueRAT' Now Hides Behind Images on Compromised Websites
Jai Vijayan, Contributing Writer,  3/2/2021
News
Attackers Turn Struggling Software Projects Into Trojan Horses
Robert Lemos, Contributing Writer,  2/26/2021
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win an Amazon Gift Card! Click Here
Latest Comment: George has not accepted that the technology age has come to an end.
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you today!
Flash Poll
How Enterprises are Developing Secure Applications
How Enterprises are Developing Secure Applications
Recent breaches of third-party apps are driving many organizations to think harder about the security of their off-the-shelf software as they continue to move left in secure software development practices.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2021-26814
PUBLISHED: 2021-03-06
Wazuh API in Wazuh from 4.0.0 to 4.0.3 allows authenticated users to execute arbitrary code with administrative privileges via /manager/files URI. An authenticated user to the service may exploit incomplete input validation on the /manager/files API to inject arbitrary code within the API service sc...
CVE-2021-27581
PUBLISHED: 2021-03-05
The Blog module in Kentico CMS 5.5 R2 build 5.5.3996 allows SQL injection via the tagname parameter.
CVE-2021-28042
PUBLISHED: 2021-03-05
Deutsche Post Mailoptimizer 4.3 before 2020-11-09 allows Directory Traversal via a crafted ZIP archive to the Upload feature or the MO Connect component. This can lead to remote code execution.
CVE-2021-28041
PUBLISHED: 2021-03-05
ssh-agent in OpenSSH before 8.5 has a double free that may be relevant in a few less-common scenarios, such as unconstrained agent-socket access on a legacy operating system, or the forwarding of an agent to an attacker-controlled host.
CVE-2021-3377
PUBLISHED: 2021-03-05
The npm package ansi_up converts ANSI escape codes into HTML. In ansi_up v4, ANSI escape codes can be used to create HTML hyperlinks. Due to insufficient URL sanitization, this feature is affected by a cross-site scripting (XSS) vulnerability. This issue is fixed in v5.0.0.