DEV Community

Cover image for Installing Elasticsearch, Logstash, and Kibana (ELK Stack) on Kubernetes
Vultr profile image Sanskriti Harmukh
Sanskriti Harmukh for Vultr

Posted on with Aashish Chaurasiya • Originally published at docs.vultr.com

Installing Elasticsearch, Logstash, and Kibana (ELK Stack) on Kubernetes

#kubernetes #elasticsearch #logging #devops

The Elastic (ELK) Stack — Elasticsearch, Logstash, Kibana, and Beats — collects, processes, stores, and visualizes logs in real time. This guide deploys it on Kubernetes via the Elastic Cloud on Kubernetes (ECK) operator: Elasticsearch for storage/search, Logstash for pipeline processing, Filebeat as a per-node log shipper, and Kibana for visualization, all behind Traefik with Let's Encrypt TLS.

Prerequisites: a Kubernetes cluster (3+ nodes, 4GB RAM each minimum), kubectl/helm configured, a domain for Kibana (e.g. kibana.example.com).

Install Elasticsearch

ECK handles Elasticsearch cluster config, scaling, and upgrades automatically.

1. Install the operator — check the ECK releases page for the current version:

$ kubectl create -f https://download.elastic.co/downloads/eck/3.4.0/crds.yaml
$ kubectl apply -f https://download.elastic.co/downloads/eck/3.4.0/operator.yaml
$ kubectl get -n elastic-system pods
Enter fullscreen mode Exit fullscreen mode

Confirm elastic-operator-0 is Running.

2. Define the Elasticsearch cluster:

$ nano elasticsearch.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 9.4.0
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 40Gi
        storageClassName: standard
Enter fullscreen mode Exit fullscreen mode

Replace standard with an actual StorageClass from your cluster — run kubectl get storageclass to see what's available. node.store.allow_mmap: false avoids memory-map permission issues in containers.

$ kubectl apply -f elasticsearch.yaml
Enter fullscreen mode Exit fullscreen mode

Wait ~2 minutes, then verify:

$ kubectl get pods -l elasticsearch.k8s.elastic.co/cluster-name=quickstart
$ kubectl get service quickstart-es-http
$ kubectl get pvc
$ kubectl get elasticsearch
Enter fullscreen mode Exit fullscreen mode

Single-node setups show yellow health (replicas can't be placed) — that's expected. red means check pods/service/PVC.

3. Grab the generated elastic user password — you'll need it for Elasticsearch and Kibana:

$ PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')
$ echo $PASSWORD
Enter fullscreen mode Exit fullscreen mode

Install Logstash

Logstash ingests from Filebeat, filters/enriches, and forwards to Elasticsearch.

$ nano logstash.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: logstash.k8s.elastic.co/v1alpha1
kind: Logstash
metadata:
  name: quickstart
spec:
  version: 9.4.0
  count: 1
  elasticsearchRefs:
  - name: quickstart
    clusterName: quickstart
  pipelines:
  - pipeline.id: main
    config.string: |
      input {
        beats {
          port => 5044
        }
      }
      filter {
        if [kubernetes][namespace] {
          mutate {
            add_field => { "environment" => "%{[kubernetes][namespace]}" }
          }
        }
      }
      output {
        elasticsearch {
          hosts => [ "${QUICKSTART_ES_HOSTS}" ]
          user => "${QUICKSTART_ES_USER}"
          password => "${QUICKSTART_ES_PASSWORD}"
          ssl_certificate_authorities => "${QUICKSTART_ES_SSL_CERTIFICATE_AUTHORITY}"
          index => "logstash-%{+YYYY.MM.dd}"
        }
      }
  services:
  - name: beats
    service:
      spec:
        type: ClusterIP
        ports:
        - port: 5044
          name: beats
          protocol: TCP
          targetPort: 5044
  volumeClaimTemplates:
  - metadata:
      name: logstash-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 40Gi
      storageClassName: standard
Enter fullscreen mode Exit fullscreen mode

elasticsearchRefs auto-injects connection credentials as env vars. The pipeline listens for Filebeat on port 5044, tags each log with its Kubernetes namespace, and writes to a daily logstash-YYYY.MM.dd index.

$ kubectl apply -f logstash.yaml
$ kubectl get pods -l logstash.k8s.elastic.co/name=quickstart
$ kubectl get logstash
Enter fullscreen mode Exit fullscreen mode

Expect green health, 1/1 available.

Install Filebeat

Runs as a DaemonSet, tailing /var/log/containers/ on every node and enriching with Kubernetes metadata.

$ nano filebeat.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: quickstart
spec:
  type: filebeat
  version: 9.4.0
  config:
    filebeat.inputs:
    - type: filestream
      id: kubernetes-container-logs
      paths:
      - /var/log/containers/*.log
      parsers:
      - container: {}
      prospector.scanner.symlinks: true
      processors:
      - add_kubernetes_metadata:
          host: ${NODE_NAME}
          matchers:
          - logs_path:
              logs_path: /var/log/containers/
      - drop_event.when:
          or:
          - equals:
              kubernetes.namespace: "kube-system"
          - equals:
              kubernetes.namespace: "kube-public"
          - equals:
              kubernetes.namespace: "elastic-system"
          - equals:
              kubernetes.namespace: "kube-node-lease"
    output.logstash:
      hosts: ["quickstart-ls-beats:5044"]
  daemonSet:
    podTemplate:
      spec:
        serviceAccountName: filebeat
        automountServiceAccountToken: true
        terminationGracePeriodSeconds: 30
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        containers:
        - name: filebeat
          env:
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          securityContext:
            runAsUser: 0
          volumeMounts:
          - name: varlogcontainers
            mountPath: /var/log/containers
          - name: varlogpods
            mountPath: /var/log/pods
          - name: varlibdockercontainers
            mountPath: /var/lib/docker/containers
        volumes:
        - name: varlogcontainers
          hostPath:
            path: /var/log/containers
        - name: varlogpods
          hostPath:
            path: /var/log/pods
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
Enter fullscreen mode Exit fullscreen mode

drop_event.when filters out system namespace noise. hostNetwork: true + runAsUser: 0 are needed for node-level log access.

RBAC:

$ nano filebeat-rbac.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: filebeat
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: filebeat
rules:
- apiGroups: [""]
  resources: [namespaces, pods, nodes]
  verbs: [get, list, watch]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: filebeat
subjects:
- kind: ServiceAccount
  name: filebeat
  namespace: default
roleRef:
  kind: ClusterRole
  name: filebeat
  apiGroup: rbac.authorization.k8s.io
Enter fullscreen mode Exit fullscreen mode 
$ kubectl apply -f filebeat-rbac.yaml
$ kubectl apply -f filebeat.yaml
$ kubectl get pods -A -l common.k8s.elastic.co/type=beat
$ kubectl get beat
Enter fullscreen mode Exit fullscreen mode

One Filebeat pod per node, green health.

Install Kibana 

$ nano kibana.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: quickstart
  namespace: default
spec:
  version: 9.4.0
  count: 1
  elasticsearchRef:
    name: quickstart
  http:
    tls:
      selfSignedCertificate:
        disabled: true
Enter fullscreen mode Exit fullscreen mode

TLS is disabled here because Traefik handles termination downstream.

$ kubectl apply -f kibana.yaml
$ kubectl get pods --selector='kibana.k8s.elastic.co/name=quickstart'
$ kubectl get service quickstart-kb-http
$ kubectl get kibana
Enter fullscreen mode Exit fullscreen mode

Secure with Traefik + cert-manager

1. Install Traefik as Ingress:

$ helm repo add traefik https://traefik.github.io/charts
$ helm repo update
$ helm install traefik traefik/traefik \
  --namespace traefik \
  --create-namespace \
  --set "ports.web.exposedPort=80" \
  --set "ports.websecure.exposedPort=443" \
  --set "ports.web.http.redirections.entryPoint.to=websecure" \
  --set "ports.web.http.redirections.entryPoint.scheme=https" \
  --set persistence.enabled=false
$ kubectl get all -n traefik
Enter fullscreen mode Exit fullscreen mode

Wait ~3 minutes for the LoadBalancer, then get its IP and point your DNS provider's A record at it:

$ kubectl get service traefik --namespace traefik
Enter fullscreen mode Exit fullscreen mode

2. Install cert-manager — check the releases page for the current version:

$ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.20.2/cert-manager.yaml
$ kubectl get pods -n cert-manager
Enter fullscreen mode Exit fullscreen mode

3. Create a ClusterIssuer:

$ nano issuer.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    email: admin@example.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: traefik
Enter fullscreen mode Exit fullscreen mode 
$ kubectl apply -f issuer.yaml
Enter fullscreen mode Exit fullscreen mode

4. Create the Ingress. Replace kibana.example.com:

$ nano kibana-ingress.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kibana
  namespace: default
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  ingressClassName: traefik
  tls:
    - hosts:
        - kibana.example.com
      secretName: kibana-tls
  rules:
    - host: kibana.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: quickstart-kb-http
                port:
                  number: 5601
Enter fullscreen mode Exit fullscreen mode 
$ kubectl apply -f kibana-ingress.yaml
$ kubectl get ingress -n default
$ kubectl get certificate -n default
Enter fullscreen mode Exit fullscreen mode

kibana-tls should show READY: True — give cert-manager a few minutes if not.

5. Confirm TLS works:

$ curl -I https://kibana.example.com
Enter fullscreen mode Exit fullscreen mode

Access Kibana 

$ kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}' && echo
Enter fullscreen mode Exit fullscreen mode

Visit https://kibana.example.com, log in as elastic with that password, Explore on my own.

Explore logs:

  1. Main menu → Analytics → Discover.
  2. Select (or create) the logstash-* data view — index pattern logstash-*, time field @timestamp.
  3. Filter with queries like kubernetes.namespace: default, kubernetes.pod.name: demo-app*, or message: error.
  4. Adjust the time range picker, click any log entry to expand full metadata.

Troubleshooting

Elasticsearch pod stuck Pending — kubectl describe pod quickstart-es-default-0. Usually insufficient node resources, or the PVC's StorageClass doesn't exist (kubectl get storageclass to check).

Elasticsearch yellow/red — kubectl get elasticsearch. Yellow is normal for single-node (no room for replicas); scale nodeSets.count to 3+ for green in production. Red means check kubectl logs quickstart-es-default-0 for OOM or storage failures.

Filebeat not collecting — verify RBAC:

$ kubectl get serviceaccount filebeat
$ kubectl get clusterrole filebeat
$ kubectl get clusterrolebinding filebeat
Enter fullscreen mode Exit fullscreen mode

Reapply filebeat-rbac.yaml if anything's missing; check kubectl logs -l common.k8s.elastic.co/type=beat.

Certificate not issued — kubectl describe certificate kibana-tls -n default, kubectl describe certificaterequest -n default, kubectl logs -n cert-manager -l app=cert-manager. Usually DNS not propagated yet (dig kibana.example.com) or port 80 not reachable for the HTTP-01 challenge.

"Kibana server is not ready yet" — confirm kubectl get elasticsearch shows green; if it does but Kibana still fails, check kubectl logs -l kibana.k8s.elastic.co/name=quickstart.

LoadBalancer stuck <pending> — needs cloud-provider LoadBalancer integration; check kubectl describe service traefik -n traefik if it doesn't resolve within 5 minutes.

Next Steps

Logs flow from every node through Filebeat → Logstash → Elasticsearch, visualized in Kibana over TLS. From here:

  • Scale nodeSets.count to 3+ for a green, replicated Elasticsearch cluster
  • Add more Logstash filters (grok, geoip) for richer log parsing
  • Set up Kibana alerting rules on error-rate thresholds

For the full guide, visit the original article on Vultr Docs.

Top comments (0)