I was genuinely surprised by the scarcity of CDK8s examples available. Consequently, I decided to pitch in and share what I've learned by presenting how I setup a basic Prometheus deployment using CDK8s.
What is CDK8s i hear you ask?
CDK8s is a development framework tailored for Kubernetes. It enables users to define Kubernetes manifests using a library of constructs in their preferred programming language. In essence, CDK8s is akin to AWS CDK but designed for Kubernetes.
Prerequisites
Before we dive into the tutorial, make sure you have the following prerequisites:
- Basic understanding of Kubernetes and Prometheus
- CDK8s CLI installed
- kubectl installed
- A Running Kubernetes cluster (Minikube works well)
1. Initialise your CDK8s application
Begin by setting up your CDK8s app using the init command. While I'll be using TypeScript, there are alternative options available.
cdk8s init typescript-app
This will create a folder structure which looks something like:
my-cdk8s-app/
├── imports/
├── dist/
├── cdk8s.yaml
├── package.json
├── tsconfig.json
└── main.ts
The key file here is main.ts, this contains our CDK8s chart. Charts are just a container for your CDK8s code, they are synthesised into a single K8s manifest.
2. Add imports for necessary constructs
Import the required constructs at the top of your main.ts file to abstract elements like configmaps, volumes, deployments, etc.
main.ts
import {
ClusterRole,
ApiResource,
NonApiResource,
ServiceAccount,
Deployment,
Volume,
ConfigMap }
from 'cdk8s-plus-25'
3. Create a namespace for Prometheus and configure the chart
Run the following command in your terminal to create a namespace for Prometheus:
kubectl create namespace monitoring
Modify your main.ts file to ensure all API objects use this namespace:
new MyChart(app, 'CDK8s-Prometheus', { namespace: 'monitoring' })
4. Configure RBAC
For Prometheus to access the Kubernetes API, create a service account, cluster role, and role binding:
const serviceAccount = new ServiceAccount(this, 'service-account')
const clusterRole = new ClusterRole(this, 'cluster-role')
clusterRole.allowRead(ApiResource.NODES)
clusterRole.allowRead(ApiResource.SERVICES)
clusterRole.allowRead(ApiResource.ENDPOINTS)
clusterRole.allowRead(ApiResource.PODS)
clusterRole.allowRead(ApiResource.INGRESSES)
clusterRole.allowGet(NonApiResource.of('/metrics'))
clusterRole.bindInNamespace('monitoring', serviceAccount)
5. Create a file for Prometheus configuration
Add a prometheus.yml file to the root directory of your project with the following content. This is a simple setup where Prometheus scrape its own metrics
prometheus.yml
global:
scrape_interval: 5s
scrape_configs:
- job_name: prometheus
static_configs:
- targets:
- localhost:9090
6. Create volumes for storage and config
Define two volumes to be mounted in the pods later—one for Prometheus config and another for storing metrics data:
const configMap = new ConfigMap(this, 'configmap')
configMap.addFile(join(__dirname + '/prometheus.yml'))
const configVolume = Volume.fromConfigMap(this, 'config-volume', configMap)
const dataVolume = Volume.fromEmptyDir(this, 'data-volume', 'data')
7. Create our deployment
Specify the deployment details for Prometheus in your main.ts file:
const deployment = new Deployment(this, 'deployment', {
containers: [
{
name: 'prometheus',
image: 'prom/prometheus',
args: ['--config.file=/etc/prometheus/prometheus.yml'],
portNumber: 9090,
securityContext: {ensureNonRoot: false}
}
]
})
deployment.containers[0].mount('/etc/prometheus/', configVolume)
deployment.containers[0].mount('/data/', dataVolume)
8. Deploy your application
With everything set up, deploy your Prometheus instance:
Run:
kubectl apply -f dist/
Test your Prometheus deployment by accessing the UI using port forwarding:
kubectl port-forward <POD NAME> 8080:9090 -n=monitoring
Access the UI via http://localhost:8080
Congratulations! You've successfully set up a basic Prometheus deployment using CDK8s. From here, you can explore further customisation and optimisation based on your specific needs.
Top comments (0)