DEV Community: Fairwinds

Discover the Top 5 Kubernetes Security Mistakes You’re (Probably) Making

Fairwinds — Mon, 29 Nov 2021 22:29:00 +0000

Written By: Robert Brennan

It’s not an exaggeration to state that cloud native technology is revolutionizing how organizations are developing and delivering applications. As organizations increasingly adopt microservices and containers, many are turning to Kubernetes for container orchestration. Kubernetes controls both resource allocation and traffic management for cloud applications and microservices, providing critical capabilities for running applications in a 24x7 world. K8s enables auto-scaling, auto-recovery, and more. While the benefits of Kubernetes are impressive, many organizations struggle with five common Kubernetes security mistakes. Does your organization?

Kubernetes Challenges for Organizations

Kubernetes is complex, and requires considerable learning and practice before teams gain confidence in their Kubernetes environment. If you’re just starting out, you may lack the tools, processes, and experience necessary to successfully launch Kubernetes environments. Not only that, there’s a considerable culture change that must occur in development, operations, and security teams, because Kubernetes and containers present a new approach for deploying applications. These changes mean that operations and security teams question whether the applications and data will be secure when the organization adopts microservices, containers, and Kubernetes to develop and deploy applications.

Security in a Cloud Native World

In the cloud native model, many of the traditional security tools and processes are no longer the right choice, while at the same time, containers create new blind spots and attack surfaces. Getting the visibility you need across containers and clusters poses an additional challenge. In the new paradigm, developers may find that it’s now necessary to take responsibility for some of the new security challenges, which is a role most devs are unaccustomed to and may be reluctant to embrace.

So what are the most common Kubernetes security mistakes that most organizations make?

Granting access to the host node — it’s easy to give admin level access to applications, but it can increase your risk of attack.
Assuming the operations team is aligned with security — Kubernetes offers many configuration options, which offer a lot of flexibility — and complexity — that security teams need to understand.
Running containers with known vulnerabilities — Kubernetes uses containers to deliver applications, but many teams aren’t aware of the known vulnerabilities that might be exposed in those containers.
Expecting security by default using native controls — while Kubernetes does offer native security features, many are not enabled by default.
Moving to production before you are ready — many teams, in their understandable excitement to get applications up and running in Kubernetes, rush to push apps live, leading to security gaps.

These five mistakes can be avoided by continuously scanning your clusters in dev and production environments. Identifying them is half the battle. Learn how you can identify mistakes and remediate them. To get started on improving your Kubernetes security, learn more about the top five mistakes you’re probably making — and get the information you need to fix them. Read the white paper.

Written By: Robert Brennan

Kubernetes Best Practices for Security

Fairwinds — Wed, 22 Apr 2020 16:24:06 +0000

Kubernetes is the dominant container orchestration solution — adoption now stands at 86% of the market. The genius of Kubernetes is its ability to provide you with a framework to run distributed systems resiliently. However, it introduces a level of complexity that can be overwhelming. By following Kubernetes best practices around security, reliability, efficiency and monitoring, teams can set themselves up for a successful transition. In a series of blog posts, we’ll cover each of these topics, starting with security.

Kubernetes Best Practices: Security

Kubernetes abstracts away just enough of the infrastructure layer so that developers can freely deploy, while ops teams retain access to important governance and risk controls. The challenge is that development teams new to Kubernetes may neglect some critical security features. Often the easiest way to get something working is to soften its security.

Let’s look at three common security challenges, and how to overcome them.

A Good Burst vs. a Bad Burst

Kubernetes responds well to bursts in traffic — whether good or bad. In the event you see a legitimate burst of traffic, Kubernetes will scale up to meet the increase in demand. Your application will consume more resources in your cluster without any degradation of performance. That’s a major benefit. However, in the event of a denial-of-service (DoS) attack, Kubernetes will do exactly the same thing, and you’ll pay for that traffic overload.

K8S Best Practice #1 — Set limits against:

the number of concurrent connections per IP address
the number of requests each user can make per second, minute, or hour
the size of request bodies
and tune these limits for individual hostnames and paths

Granting Safe Levels of Access

The easiest way to deploy a new application or provision a new user is to give away admin permissions. But it’s also the most dangerous way — if an attacker gains access to that account, they’ll have access to everything.

K8S Best Practice #2- Employ role based access controls (RBAC) to adhere to the principle of least privilege.

RBAC allows you to grant users granular access to Kubernetes API resources. You should define access profiles using Roles or ClusterRoles. Using Roles, you'll grant access to a single namespace. With ClusterRoles, you can grant access to resources without namespaces, like Nodes and PersistentVolumes, as well as all namespaced resources.

While RBAC configuration can be confusing and verbose, tools like rbac-manager can help simplify the syntax. This helps prevent mistakes and provides a clearer sense for who has access to what.

The end result? By only granting workloads the permissions they need to do their job, you’ll limit the amount of damage an attacker can do to your Kubernetes environment.

Keep Kubernetes Secrets, Secret

If you are using Kubernetes infrastructure-as-code (IaC) patterns, you benefit from having a completely reproducible environment. But there’s a catch — part of your infrastructure likely includes Kubernetes Secrets, which store and manage sensitive information, such as passwords, OAuth tokens, and ssh keys. And you shouldn't be adding Secrets to your IaC repository.

K8S Best Practice #3 — It’s tempting to check your Kubernetes secrets into your infrastructure-as-code repository so that your builds are 100% reproducible.

But if you care about security, don’t. Once checked in, your secrets are permanently exposed to anyone with access to your Git repository.

The solution is to split the difference: encrypt all your secrets so you can safely check them into your repository without fear of exposing them. Then you’ll then only need access to a single encryption key to “unlock” your IaC repository, and have perfectly reproducible infrastructure. Open source tools like Mozilla’s SOPS can help with this.

You can read more best practices for k8s security by checking out how we implement security for our customer’s managed Kubernetes deployments.

Dig deeper into Kubernetes Best Practices for Security: Ensure cluster security with k8s secrets, network policies, and workload identity

Written By: Robert Brennan

Practical Tips from Engineering Leaders for Working From Home During Uncertain Times

Fairwinds — Mon, 20 Apr 2020 18:54:11 +0000

As COVID-19 spreads across the globe and many organizations have asked non-essential employees to work from home, many of us are adjusting to the new norm of conducting business where we also live our lives. As a remote-first workplace, Fairwinds has considerable experience in helping employees be successful working from home. We recently hosted a panel discussion, “Remote Tips & Trips for Your Engineering Team” that had great insight for any person working from home.

Transitioning to Home-Based Work

These are uncertain times. We all have a lot of unanswered questions, which, when coupled that with setting up a new work environment, can overwhelm even the most unflappable teammates.

“Recognizing that there is an emotional toll where it’s affecting people who might already have the systems in place to work from home, is important,” said panel moderator, Kendall Miller, president at Fairwinds.

Now, because many schools are closed, a good number of us are juggling child care responsibilities within our work days. Also, for those of us with an at-home partner, we are figuring out who works when, trying to schedule calls around each other and working to be patient with seeing each other 24/7.

Employers and employees alike need to consider comfort. Sitting — or standing — at a new workstation (kitchen tables included) could take a toll. Pay attention to what your body is telling you and talk to your manager if you need ergonomic support. And, as simple as it sounds, setting routines and patterns for your day of at-home work is important. Kristina Kemmer, Director of Engineering at Zapier noted, “I make sure to shower every day and actually get dressed to go to work because I need to feel a separation of home life and work life. I try not to look at Slack after hours and I think it’s really good to have cut off times.”

Create — or Fold Into — a Remote Culture

You may not be able to extend free lunches or massages to remote workers, but you can help them focus on creating a culture that works for teams outside of the office. “People who have been working remotely for a while have built up communications patterns — phrases they use to say what they are working on, emojis that mean certain things, etc. If you haven’t had a remote team/workforce, sit down and figure out what exists and how to get new members into those patterns,” said Sarah Zelechoski, VP of engineering at Fairwinds.

Whether those patterns are standing Zoom rooms where people can join in at any time or ‘water cooler’ video chats to help people feel connected, don’t forget to focus on culture.

Overcommunicate

It’s easy to overlook how much we communicate — even unknowingly — when we are together in an office setting. When we are dispersed, we have to make sure to remember to share slight details that might be important.

Brandon Jung, vice president of alliances at GitLab said, “If you lead people, the biggest thing to remember is to overcommunicate. Don’t forget how important it is and that things are written down. Reach out to touch base with people, particularly with these changes.”

Zelchoski echoed this sentiment noting that for all employees, “Communication, openness and status have to be completely different. You have to be more transparent. Otherwise, you will be off in your own corner doing things; you will feel isolated and people will not integrate you into the communication path.”

Getting Work Done — Practical Points

Remote teams have to work with unique considerations such as teams in different time zones. “I’ve managed and worked with people from APAC, North America and European time zones at the same time and scheduling a meeting for those three time zones simultaneously is nearly impossible. Someone is going to be up at midnight,” said Kate Taggard, engineering manager at Stripe. Her solution? Limiting managers to choosing team members from two of the three time zones to make scheduling and working together easier.

To this end, transitioning to a remote workforce means that it’s time to put collaboration and communication tools to work for you. Not only do they help you accomplish your work, but they ensure that your team communicates and is working in tandem with one another. Swarna Podila, senior director of community for the Cloud Foundry Foundation shared, “There is this pairing culture where engineers from different regions and time zones are paired together. They use Zoom calls and other tools they have found useful, like whiteboarding. I don’t personally whiteboard — I have reMarkable paper where I draw on it and it syncs up to my Google Drive which is already in a shared folder.”

The panel, all of whom have engineering backgrounds, also recommended creating processes that are documented and communicated so all team members know what they are. From there, the group suggested their favorite tools, including:

Krisp.ai which blocks background noise during calls.
Retrium which allows for easy retros for remote teams.
GitLab for DevOps.

We may be working as remote teams for a longer period of time than originally anticipated. Putting in the effort now to help teams be successful is worth it — both to the people and the bottom line. Check out the full panel, “ Remote Tips & Trips for Your Engineering Team.” And, be sure to add your favorite remote work tools to our list.

Fairwinds—The Kubernetes Enablement Company
ClusterOps Managed Kubernetes— ClusterOps is a fully-managed Kubernetes cluster management tool that integrates infrastructure as code, open source software, and SRE expertise as a subscription service.
ClusterOps Kubernetes Advisory—ClusterOps Advisory integrates Kubernetes expertise and open source software so you can confidently run reliable, scalable, and secure Kubernetes clusters.
Fairwinds Insights—We integrate trusted tools, collaboration workflows, and expertise into a single monitoring platform, so workloads always stay secure, reliable, and efficient.