DEV Community

Janak Shrestha
Janak Shrestha

Posted on

Day 58: Deploy Grafana on Kubernetes Cluster

The Nautilus DevOps teams is planning to set up a Grafana tool to collect and analyze analytics from some applications. They are planning to deploy it on Kubernetes cluster. Below you can find more details.

1.) Create a deployment named grafana-deployment-datacenter using any grafana image for Grafana app. Set other parameters as per your choice.

2.) Create NodePort type service with nodePort 32000 to expose the app.

You do not need to make any configuration changes inside the Grafana app once deployed; just make sure you can access the Grafana login page.

What We Will Build

We are going to deploy Grafana on Kubernetes with the following configuration:

Component Specification
Deployment Name grafana-deployment-datacenter
Image grafana/grafana:latest
Container Port 3000
Service Type NodePort
NodePort 32000

Architecture Overview 

┌─────────────────────────────────────────────────────────────────────────────┐
│                         Kubernetes Cluster                                 │
│                                                                              │
│  ┌────────────────────────────────────────────────────────────────────────┐ │
│  │              Deployment: grafana-deployment-datacenter               │ │
│  │              Replicas: 1                                             │ │
│  │                                                                       │ │
│  │  ┌──────────────────────────────────────────────────────────────────┐ │ │
│  │  │  Pod: grafana-deployment-datacenter-xxxxxxxxxx-xxxxx            │ │ │
│  │  │  Container: grafana-container                                    │ │ │
│  │  │  Image: grafana/grafana:latest                                   │ │ │
│  │  │  Port: 3000                                                      │ │ │
│  │  └──────────────────────────────────────────────────────────────────┘ │ │
│  └────────────────────────────────────────────────────────────────────────┘ │
│                                    │                                         │
│                                    ▼                                         │
│  ┌────────────────────────────────────────────────────────────────────────┐ │
│  │              Service: grafana-service-datacenter                     │ │
│  │              Type: NodePort                                          │ │
│  │              Port: 3000 -> TargetPort: 3000                         │ │
│  │              NodePort: 32000                                        │ │
│  └────────────────────────────────────────────────────────────────────────┘ │
│                                    │                                         │
│                                    ▼                                         │
│                    http://<node-ip>:32000                                  │
└─────────────────────────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Prerequisites

Before proceeding, ensure you have:

  • Access to a Kubernetes cluster
  • kubectl configured on your jump-host
  • Sufficient permissions to create deployments and services

Step-by-Step Implementation

Step 1: Create the Deployment YAML

Create a file named grafana-deployment.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grafana-deployment-datacenter
  labels:
    app: grafana
spec:
  replicas: 1
  selector:
    matchLabels:
      app: grafana
  template:
    metadata:
      labels:
        app: grafana
    spec:
      containers:
      - name: grafana-container
        image: grafana/grafana:latest
        ports:
        - containerPort: 3000
Enter fullscreen mode Exit fullscreen mode

YAML Breakdown

Deployment Metadata

metadata:
  name: grafana-deployment-datacenter
  labels:
    app: grafana
Enter fullscreen mode Exit fullscreen mode

This defines the deployment name and adds a label app: grafana for identification.

Replicas

spec:
  replicas: 1
Enter fullscreen mode Exit fullscreen mode

A single replica is sufficient for testing purposes.

Selector

selector:
  matchLabels:
    app: grafana
Enter fullscreen mode Exit fullscreen mode

The deployment manages pods that match the label app: grafana.

Pod Template

template:
  metadata:
    labels:
      app: grafana
  spec:
    containers:
    - name: grafana-container
      image: grafana/grafana:latest
      ports:
      - containerPort: 3000
Enter fullscreen mode Exit fullscreen mode

The pod template defines the container using the official Grafana image, listening on port 3000.

Step 2: Create the Service YAML

Create a file named grafana-service.yaml with the following content:

apiVersion: v1
kind: Service
metadata:
  name: grafana-service-datacenter
spec:
  type: NodePort
  selector:
    app: grafana
  ports:
    - port: 3000
      targetPort: 3000
      nodePort: 32000
Enter fullscreen mode Exit fullscreen mode

YAML Breakdown

Service Metadata

metadata:
  name: grafana-service-datacenter
Enter fullscreen mode Exit fullscreen mode

This defines the service name.

Service Type

spec:
  type: NodePort
Enter fullscreen mode Exit fullscreen mode

NodePort exposes the service on a static port on each node.

Selector

selector:
  app: grafana
Enter fullscreen mode Exit fullscreen mode

The service routes traffic to pods with the label app: grafana.

Port Configuration

ports:
  - port: 3000
    targetPort: 3000
    nodePort: 32000
Enter fullscreen mode Exit fullscreen mode
  • port: The service port
  • targetPort: The container port
  • nodePort: The port on each node (must be between 30000 and 32767)

Step 3: Apply the Configurations

Apply the deployment:

kubectl apply -f grafana-deployment.yaml
Enter fullscreen mode Exit fullscreen mode

Output:

deployment.apps/grafana-deployment-datacenter created
Enter fullscreen mode Exit fullscreen mode

Apply the service:

kubectl apply -f grafana-service.yaml
Enter fullscreen mode Exit fullscreen mode

Output:

service/grafana-service-datacenter created
Enter fullscreen mode Exit fullscreen mode

Verification

Step 1: Verify the Deployment 

kubectl get deployments
Enter fullscreen mode Exit fullscreen mode

Output:

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
grafana-deployment-datacenter   1/1     1            1           30s
Enter fullscreen mode Exit fullscreen mode

Step 2: Verify the Pods 

kubectl get pods
Enter fullscreen mode Exit fullscreen mode

Output:

NAME                                             READY   STATUS    RESTARTS   AGE
grafana-deployment-datacenter-xxxxxxxxxx-xxxxx   1/1     Running   0          30s
Enter fullscreen mode Exit fullscreen mode

Step 3: Verify the Service 

kubectl get services
Enter fullscreen mode Exit fullscreen mode

Output:

NAME                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
grafana-service-datacenter   NodePort    10.43.60.68     <none>        3000:32000/TCP   8s
kubernetes                   ClusterIP   10.43.0.1       <none>        443/TCP          33m
Enter fullscreen mode Exit fullscreen mode

Step 4: Inspect Service Details 

kubectl describe service grafana-service-datacenter
Enter fullscreen mode Exit fullscreen mode

Output:

Name:                     grafana-service-datacenter
Namespace:                default
Selector:                 app=grafana
Type:                     NodePort
IP:                       10.43.60.68
Port:                     <unset>  3000/TCP
TargetPort:               3000/TCP
NodePort:                 <unset>  32000/TCP
Endpoints:                10.22.0.9:3000
Enter fullscreen mode Exit fullscreen mode

Step 5: Get Node IP 

kubectl get nodes -o wide
Enter fullscreen mode Exit fullscreen mode

Output:

NAME        STATUS   ROLES           AGE   VERSION        INTERNAL-IP      EXTERNAL-IP
jump-host   Ready    control-plane   34m   v1.34.1+k3s1   10.244.226.158   <none>
Enter fullscreen mode Exit fullscreen mode

Step 6: Access Grafana

Grafana

Grafana can now be accessed at:

http://10.244.226.158:32000
Enter fullscreen mode Exit fullscreen mode

Default Credentials:

  • Username: admin
  • Password: admin

Grafana Access Options

Method 1: NodePort Access

Access Grafana directly using the node IP and NodePort:

http://<node-ip>:32000
Enter fullscreen mode Exit fullscreen mode

Method 2: Port Forwarding

Forward the service port to your local machine:

kubectl port-forward service/grafana-service-datacenter 3000:3000
Enter fullscreen mode Exit fullscreen mode

Then access: http://localhost:3000

Method 3: Ingress (Advanced)

For production environments, consider using an Ingress controller to expose Grafana with a domain name and TLS.

Grafana Default Configuration

When Grafana starts for the first time:

Setting Default Value
Username admin
Password admin
Port 3000
Data Directory /var/lib/grafana
Logs Directory /var/log/grafana

First-Time Login

  1. Navigate to the Grafana URL
  2. Enter username admin and password admin
  3. You will be prompted to change the password
  4. After changing the password, you can start configuring data sources and dashboards

Troubleshooting

Pod Fails to Start 

# Check pod status
kubectl get pods

# View pod details
kubectl describe pod <pod-name>

# View pod logs
kubectl logs <pod-name>
Enter fullscreen mode Exit fullscreen mode

Service Not Accessible 

# Check service details
kubectl describe service grafana-service-datacenter

# Check endpoints
kubectl get endpoints grafana-service-datacenter

# Verify pod labels match service selector
kubectl get pods --show-labels
Enter fullscreen mode Exit fullscreen mode

NodePort Conflict 

# Check if NodePort is in use
kubectl get services --all-namespaces | grep 32000

# Verify NodePort range
# NodePort must be between 30000 and 32767
Enter fullscreen mode Exit fullscreen mode

Image Pull Issues 

# Check image pull policy
kubectl describe pod <pod-name> | grep -A 5 "Events"

# Verify image exists
docker pull grafana/grafana:latest
Enter fullscreen mode Exit fullscreen mode

Key Takeaways

What We Learned

  1. Grafana Deployment: Grafana runs on port 3000 and uses the official grafana/grafana:latest image
  2. NodePort Service: Exposes applications on a static port (32000 in this case)
  3. Service Discovery: The service uses labels to route traffic to the correct pods
  4. Access Methods: Grafana can be accessed via NodePort, port forwarding, or Ingress

Best Practices

  1. Resource Limits: Always set resource requests and limits for production deployments
  2. Persistent Storage: Use PersistentVolumeClaims for Grafana data persistence
  3. Secrets: Store Grafana admin credentials in Kubernetes Secrets
  4. Ingress: Use Ingress with TLS for production environments
  5. Monitoring: Monitor Grafana itself using the Kubernetes monitoring stack

Summary

In this challenge, we successfully:

  • Created a Kubernetes deployment for Grafana
  • Exposed Grafana using a NodePort service on port 32000
  • Verified the deployment and service status
  • Accessed the Grafana login page

Top comments (0)