Observability - 5(Logging using EFK)

Mastering Observability with the EFK Stack in Kubernetes: A Technical Guide

In modern cloud-native environments, maintaining visibility into your applications is essential. The EFK stack—Elasticsearch, Fluent Bit, and Kibana—has become a go-to solution for centralized log management and troubleshooting in Kubernetes environments. Here’s a technical dive into how the EFK stack works and how to set it up in Kubernetes.

Why EFK for Observability?

Observability relies on three pillars: metrics, logs, and traces. Logs, in particular, are crucial for debugging, performance analysis, and security audits. Traditional logging methods aren’t practical in a microservices architecture because logs are scattered across numerous services. Here, a centralized logging system like EFK streamlines the process.

• Elasticsearch acts as the log store, providing powerful search capabilities.
• Fluent Bit is a lightweight log processor and forwarder, ideal for Kubernetes deployments.
• Kibana provides visualization, enabling teams to query and visualize logs.

Step 1: Setting Up the EFK Stack on Kubernetes

Prerequisites

Ensure you have:

• A Kubernetes cluster with Helm installed.
• Permissions to create roles, PersistentVolumeClaims, and DaemonSets.

Step 2: Deploying Elasticsearch with Helm

1. Add the Elastic Helm Repository:

   helm repo add elastic https://helm.elastic.co
   helm repo update

1. Install Elasticsearch: Deploy Elasticsearch with persistent storage (e.g., using Amazon EBS in EKS):

   helm install elasticsearch elastic/elasticsearch \
       --set persistence.storageClass="gp2" \
       --set persistence.size="20Gi" \
       --namespace logging --create-namespace

This setup ensures Elasticsearch has enough space and durability to store logs.

Step 3: Deploy Fluent Bit

1. Add Fluent Bit’s Helm Chart:

   helm repo add fluent https://fluent.github.io/helm-charts
   helm repo update

1. Configure Fluent Bit: Fluent Bit is deployed as a DaemonSet to collect logs from all Kubernetes nodes. Update values.yaml to set Fluent Bit to forward logs to Elasticsearch:

   output:
     host: "elasticsearch.logging.svc.cluster.local"
     port: 9200
     match: "*"
     logstash_prefix: "kubernetes-logs"

1. Install Fluent Bit:

   helm install fluent-bit fluent/fluent-bit --namespace logging -f values.yaml

Fluent Bit will collect, parse, and forward logs to Elasticsearch, indexing them with a kubernetes-logs prefix.

Step 4: Deploy Kibana

1. Install Kibana:

   helm install kibana elastic/kibana --namespace logging

Kibana provides the interface for querying and visualizing logs. Access Kibana by forwarding its service port or exposing it via an Ingress.

Step 5: Testing the EFK Stack

1. Verify Fluent Bit’s Log Collection: Check Fluent Bit’s logs to confirm it’s collecting logs:

   kubectl logs -l app.kubernetes.io/name=fluent-bit -n logging

1. Querying Logs in Kibana: In Kibana, create an index pattern to match kubernetes-logs*. Use filters and visualizations to analyze application logs, helping identify issues or monitor performance.

Conclusion

The EFK stack offers a powerful, scalable logging solution for Kubernetes environments. By deploying Elasticsearch, Fluent Bit, and Kibana, you can centralize logs from across your cluster, enabling fast troubleshooting and insights into your applications. With this setup, your team will have a robust toolset to maintain application observability, reduce downtime, and optimize performance in a cloud-native environment.