Kubernetes, often abbreviated as K8s, is a powerful container orchestration system that has emerged as the go-to platform for containerized microservice workloads. K8s offers a rich set of features for deploying applications, services, and configurations. However, as applications grow in complexity, handling configuration files can become overwhelming.
The two of the most popular tools which are used to simplify the management of configurations in K8s are Helm and Kustomize. Both are having its own set of pros and cons.
In this blog, we will discuss those details and explore how they complement each other.
Helm
Helm is often referred to as the “package manager for K8s” which simplifies the installation and management of K8s applications using the concept of templating. It also introduces the concept of charts, which are pre-configured templates for K8s resources as a single unit. You can also define reusable, parameterized templates and share them with your team, significantly simplifying the process of deploying applications with consistent configurations. You can also manage release versioning with rollbacks.
example: Consider you have an application that consists of a deployment, a service, and an ingress. With Helm, you can create a chart containing all these resources under templates and parameterize certain things that you may want to change based on the values you pass while you are installing like environment-based values.
Kustomize
Kustomize is a native K8s tool for customizing K8s objects. While Helm works better in the templating, Kustomize works in the concept of overlays. Kustomize introduces a template-free way to customize application configuration without modifying the original resource files using a declarative approach. Kustomize employs overlays to apply specific changes for different environments, making it ideal for managing configurations across multiple environments of your application.
example: Consider the same application above and if we want to change, add, or remove some configuration. we can use Kustomize to set up the base resources and patch the configuration on top of the base resources based on different overlays. Kustomize also benefits in moving the repeated parts of the config across resources to the overlays layer to easily manage those configs.
The Powerful Synergy
While Helm and Kustomize are powerful on their own, combining their strengths, we can create an efficient, and scalable approach to K8s resource management. Let’s discuss how to take leverage of this. There are multiple ways we can combine these two tools, we are going to discuss two of those options.
Sample application
A simple nginx web server with a k8s deployment and a service associated with that.
# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: "{{ .Values.image.repo }}:{{.Values.image.tag}}"
imagePullPolicy: "{{ .Values.image.pullPolicy }}"
ports:
- containerPort: 80
---
# service.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80Helm files
Helm chart setup with container image configurations templated and passed as values files while installing the chart.
# Chart.yaml
apiVersion: v2
name: kustomized-helm
description: Sample project with Kustomise + Helm
type: application
version: 0.1.0
appVersion: "0.0.1"
---
# values.yaml
image:
repo: nginx
tag: 1.14.2
pullPolicy: AlwaysKustomize files
Kustomize overlays set up with the number of replicas, common labels, name suffixes, and the container security configuration patched based on the environment.
# overlays/dev/kustomization.yaml
bases:
- ../../base
nameSuffix: "-dev"
commonLabels:
env: dev
patches:
- path: dev.yaml
target:
kind: Deployment
---
# overlays/dev/dev.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: placeholder
spec:
replicas: 1
---
# overlays/prod/kustomization.yaml
bases:
- ../../base
nameSuffix: "-prod"
commonLabels:
env: prod
patches:
- path: prod.yaml
target:
kind: Deployment
---
# overlays/prod/prod.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: placeholder
spec:
replicas: 5
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
template:
spec:
containers:
- name: nginx
securityContext:
allowPrivilegeEscalation: false1. Template First Approach
In the template first approach, we will first use the helm to generate the template output from the charts and then use kustomize to set up the overlays based on the environment.
├── base #empty base, helm output will be updated here
│ └── kustomization.yaml
├── Chart.yaml
├── kustomization.yaml
├── overlays
│ ├── dev
│ │ ├── dev.yaml
│ │ └── kustomization.yaml
│ └── prod
│ ├── kustomization.yaml
│ └── prod.yaml
├── templates #application
│ ├── deployment.yaml
│ └── service.yaml
└── values.yaml
# base/kustomization.yaml
resources:
- out.yaml- Generating the chart output by helm template and pipe the output to the base/out.yaml.
helm template . > base/out.yaml- Next by using the generated output from the helm as the base, running kustomize to patch based on the environment.
kustomize build overlays/dev (or) kustomize build overlays/prod
# pipe it to kubectl for apply
| kubectl apply -f -pros:
- Can fully utilize helm features.
cons:
- No release versioning and rollbacks.
2 . Overlay First Approach
├── base #application in kustomize base │ ├── deployment.yaml │ ├── kustomization.yaml │ └── service.yaml ├── Chart.yaml ├── overlays │ ├── dev │ │ ├── dev.yaml │ │ └── kustomization.yaml │ └── prod │ ├── kustomization.yaml │ └── prod.yaml ├── templates #empty helm templates, kustomize outpu will be updated here └── values.yaml
# base/kustomization.yaml
resources:
- deployment.yaml
- service.yaml- Generate the kustomize output based on the environment and pipe the output to the templates directory.
kustomize build overlays/dev > templates/out.yaml
(or)
kustomize build overlays/prod > templates/out.yaml- Use the generated output to run the helm templates.
helm template . (or) helm upgrade webserver . -n webserver --installpros:
- Helm’s packaging and versioning
- Release versioning and rollbacks
cons:
- Limited helm benefits (kustomize conflicts with the helm templating)
Summary
There are so many use cases and possibilities when we combine these two with some limitations. But it really helps when you will change the configurations too often or if you have different environment-based changes or if you want to add cross-cutting fields or restrict resources to a namespace.
I hope this blog post is useful.
References
https://jfrog.com/blog/power-up-helm-charts-using-kustomize-to-manage-kubernetes-deployments/
