DEV Community

Cover image for Installing Elasticsearch, Logstash, and Kibana (ELK Stack) on Kubernetes
Vultr profile image Sanskriti Harmukh
Sanskriti Harmukh for Vultr

Posted on with Aashish Chaurasiya Originally published at docs.vultr.com

Installing Elasticsearch, Logstash, and Kibana (ELK Stack) on Kubernetes

#kubernetes #elasticsearch #logging #devops

The Elastic (ELK) Stack — Elasticsearch, Logstash, Kibana, and Beats — collects, processes, stores, and visualizes logs in real time. This guide deploys it on Kubernetes via the Elastic Cloud on Kubernetes (ECK) operator: Elasticsearch for storage/search, Logstash for pipeline processing, Filebeat as a per-node log shipper, and Kibana for visualization, all behind Traefik with Let's Encrypt TLS.

Prerequisites: a Kubernetes cluster (3+ nodes, 4GB RAM each minimum), kubectl/helm configured, a domain for Kibana (e.g. kibana.example.com).

Install Elasticsearch

ECK handles Elasticsearch cluster config, scaling, and upgrades automatically.

1. Install the operator — check the ECK releases page for the current version:

$ kubectl create -f https://download.elastic.co/downloads/eck/3.4.0/crds.yaml
$ kubectl apply -f https://download.elastic.co/downloads/eck/3.4.0/operator.yaml
$ kubectl get -n elastic-system pods
Enter fullscreen mode Exit fullscreen mode

Confirm elastic-operator-0 is Running.

2. Define the Elasticsearch cluster:

$ nano elasticsearch.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 9.4.0
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 40Gi
        storageClassName: standard
Enter fullscreen mode Exit fullscreen mode

Replace standard with an actual StorageClass from your cluster — run kubectl get storageclass to see what's available. node.store.allow_mmap: false avoids memory-map permission issues in containers.

$ kubectl apply -f elasticsearch.yaml
Enter fullscreen mode Exit fullscreen mode

Wait ~2 minutes, then verify:

$ kubectl get pods -l elasticsearch.k8s.elastic.co/cluster-name=quickstart
$ kubectl get service quickstart-es-http
$ kubectl get pvc
$ kubectl get elasticsearch
Enter fullscreen mode Exit fullscreen mode

Single-node setups show yellow health (replicas can't be placed) — that's expected. red means check pods/service/PVC.

3. Grab the generated elastic user password — you'll need it for Elasticsearch and Kibana:

$ PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')
$ echo $PASSWORD
Enter fullscreen mode Exit fullscreen mode

Install Logstash

Logstash ingests from Filebeat, filters/enriches, and forwards to Elasticsearch.

$ nano logstash.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: logstash.k8s.elastic.co/v1alpha1
kind: Logstash
metadata:
  name: quickstart
spec:
  version: 9.4.0
  count: 1
  elasticsearchRefs:
  - name: quickstart
    clusterName: quickstart
  pipelines:
  - pipeline.id: main
    config.string: |
      input {
        beats {
          port => 5044
        }
      }
      filter {
        if [kubernetes][namespace] {
          mutate {
            add_field => { "environment" => "%{[kubernetes][namespace]}" }
          }
        }
      }
      output {
        elasticsearch {
          hosts => [ "${QUICKSTART_ES_HOSTS}" ]
          user => "${QUICKSTART_ES_USER}"
          password => "${QUICKSTART_ES_PASSWORD}"
          ssl_certificate_authorities => "${QUICKSTART_ES_SSL_CERTIFICATE_AUTHORITY}"
          index => "logstash-%{+YYYY.MM.dd}"
        }
      }
  services:
  - name: beats
    service:
      spec:
        type: ClusterIP
        ports:
        - port: 5044
          name: beats
          protocol: TCP
          targetPort: 5044
  volumeClaimTemplates:
  - metadata:
      name: logstash-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 40Gi
      storageClassName: standard
Enter fullscreen mode Exit fullscreen mode

elasticsearchRefs auto-injects connection credentials as env vars. The pipeline listens for Filebeat on port 5044, tags each log with its Kubernetes namespace, and writes to a daily logstash-YYYY.MM.dd index.

$ kubectl apply -f logstash.yaml
$ kubectl get pods -l logstash.k8s.elastic.co/name=quickstart
$ kubectl get logstash
Enter fullscreen mode Exit fullscreen mode

Expect green health, 1/1 available.

Install Filebeat

Runs as a DaemonSet, tailing /var/log/containers/ on every node and enriching with Kubernetes metadata.

$ nano filebeat.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: quickstart
spec:
  type: filebeat
  version: 9.4.0
  config:
    filebeat.inputs:
    - type: filestream
      id: kubernetes-container-logs
      paths:
      - /var/log/containers/*.log
      parsers:
      - container: {}
      prospector.scanner.symlinks: true
      processors:
      - add_kubernetes_metadata:
          host: ${NODE_NAME}
          matchers:
          - logs_path:
              logs_path: /var/log/containers/
      - drop_event.when:
          or:
          - equals:
              kubernetes.namespace: "kube-system"
          - equals:
              kubernetes.namespace: "kube-public"
          - equals:
              kubernetes.namespace: "elastic-system"
          - equals:
              kubernetes.namespace: "kube-node-lease"
    output.logstash:
      hosts: ["quickstart-ls-beats:5044"]
  daemonSet:
    podTemplate:
      spec:
        serviceAccountName: filebeat
        automountServiceAccountToken: true
        terminationGracePeriodSeconds: 30
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        containers:
        - name: filebeat
          env:
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          securityContext:
            runAsUser: 0
          volumeMounts:
          - name: varlogcontainers
            mountPath: /var/log/containers
          - name: varlogpods
            mountPath: /var/log/pods
          - name: varlibdockercontainers
            mountPath: /var/lib/docker/containers
        volumes:
        - name: varlogcontainers
          hostPath:
            path: /var/log/containers
        - name: varlogpods
          hostPath:
            path: /var/log/pods
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
Enter fullscreen mode Exit fullscreen mode

drop_event.when filters out system namespace noise. hostNetwork: true + runAsUser: 0 are needed for node-level log access.

RBAC:

$ nano filebeat-rbac.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: filebeat
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: filebeat
rules:
- apiGroups: [""]
  resources: [namespaces, pods, nodes]
  verbs: [get, list, watch]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: filebeat
subjects:
- kind: ServiceAccount
  name: filebeat
  namespace: default
roleRef:
  kind: ClusterRole
  name: filebeat
  apiGroup: rbac.authorization.k8s.io
Enter fullscreen mode Exit fullscreen mode 
$ kubectl apply -f filebeat-rbac.yaml
$ kubectl apply -f filebeat.yaml
$ kubectl get pods -A -l common.k8s.elastic.co/type=beat
$ kubectl get beat
Enter fullscreen mode Exit fullscreen mode

One Filebeat pod per node, green health.

Install Kibana 

$ nano kibana.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: quickstart
  namespace: default
spec:
  version: 9.4.0
  count: 1
  elasticsearchRef:
    name: quickstart
  http:
    tls:
      selfSignedCertificate:
        disabled: true
Enter fullscreen mode Exit fullscreen mode

TLS is disabled here because Traefik handles termination downstream.

$ kubectl apply -f kibana.yaml
$ kubectl get pods --selector='kibana.k8s.elastic.co/name=quickstart'
$ kubectl get service quickstart-kb-http
$ kubectl get kibana
Enter fullscreen mode Exit fullscreen mode

Secure with Traefik + cert-manager

1. Install Traefik as Ingress:

$ helm repo add traefik https://traefik.github.io/charts
$ helm repo update
$ helm install traefik traefik/traefik \
  --namespace traefik \
  --create-namespace \
  --set "ports.web.exposedPort=80" \
  --set "ports.websecure.exposedPort=443" \
  --set "ports.web.http.redirections.entryPoint.to=websecure" \
  --set "ports.web.http.redirections.entryPoint.scheme=https" \
  --set persistence.enabled=false
$ kubectl get all -n traefik
Enter fullscreen mode Exit fullscreen mode

Wait ~3 minutes for the LoadBalancer, then get its IP and point your DNS provider's A record at it:

$ kubectl get service traefik --namespace traefik
Enter fullscreen mode Exit fullscreen mode

2. Install cert-manager — check the releases page for the current version:

$ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.20.2/cert-manager.yaml
$ kubectl get pods -n cert-manager
Enter fullscreen mode Exit fullscreen mode

3. Create a ClusterIssuer:

$ nano issuer.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    email: admin@example.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: traefik
Enter fullscreen mode Exit fullscreen mode 
$ kubectl apply -f issuer.yaml
Enter fullscreen mode Exit fullscreen mode

4. Create the Ingress. Replace kibana.example.com:

$ nano kibana-ingress.yaml
Enter fullscreen mode Exit fullscreen mode 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kibana
  namespace: default
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  ingressClassName: traefik
  tls:
    - hosts:
        - kibana.example.com
      secretName: kibana-tls
  rules:
    - host: kibana.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: quickstart-kb-http
                port:
                  number: 5601
Enter fullscreen mode Exit fullscreen mode 
$ kubectl apply -f kibana-ingress.yaml
$ kubectl get ingress -n default
$ kubectl get certificate -n default
Enter fullscreen mode Exit fullscreen mode

kibana-tls should show READY: True — give cert-manager a few minutes if not.

5. Confirm TLS works:

$ curl -I https://kibana.example.com
Enter fullscreen mode Exit fullscreen mode

Access Kibana 

$ kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}' && echo
Enter fullscreen mode Exit fullscreen mode

Visit https://kibana.example.com, log in as elastic with that password, Explore on my own.

Explore logs:

  1. Main menu → Analytics → Discover.
  2. Select (or create) the logstash-* data view — index pattern logstash-*, time field @timestamp.
  3. Filter with queries like kubernetes.namespace: default, kubernetes.pod.name: demo-app*, or message: error.
  4. Adjust the time range picker, click any log entry to expand full metadata.

Troubleshooting

Elasticsearch pod stuck Pending — kubectl describe pod quickstart-es-default-0. Usually insufficient node resources, or the PVC's StorageClass doesn't exist (kubectl get storageclass to check).

Elasticsearch yellow/red — kubectl get elasticsearch. Yellow is normal for single-node (no room for replicas); scale nodeSets.count to 3+ for green in production. Red means check kubectl logs quickstart-es-default-0 for OOM or storage failures.

Filebeat not collecting — verify RBAC:

$ kubectl get serviceaccount filebeat
$ kubectl get clusterrole filebeat
$ kubectl get clusterrolebinding filebeat
Enter fullscreen mode Exit fullscreen mode

Reapply filebeat-rbac.yaml if anything's missing; check kubectl logs -l common.k8s.elastic.co/type=beat.

Certificate not issued — kubectl describe certificate kibana-tls -n default, kubectl describe certificaterequest -n default, kubectl logs -n cert-manager -l app=cert-manager. Usually DNS not propagated yet (dig kibana.example.com) or port 80 not reachable for the HTTP-01 challenge.

"Kibana server is not ready yet" — confirm kubectl get elasticsearch shows green; if it does but Kibana still fails, check kubectl logs -l kibana.k8s.elastic.co/name=quickstart.

LoadBalancer stuck <pending> — needs cloud-provider LoadBalancer integration; check kubectl describe service traefik -n traefik if it doesn't resolve within 5 minutes.

Next Steps

Logs flow from every node through Filebeat → Logstash → Elasticsearch, visualized in Kibana over TLS. From here:

  • Scale nodeSets.count to 3+ for a green, replicated Elasticsearch cluster
  • Add more Logstash filters (grok, geoip) for richer log parsing
  • Set up Kibana alerting rules on error-rate thresholds

For the full guide, visit the original article on Vultr Docs.

Top comments (0)