DEV Community

Cover image for Running Elasticsearch on Kubernetes from Azure Kubernetes Service
Aravind Putrevu
Aravind Putrevu

Posted on • Originally published at elastic.co

Running Elasticsearch on Kubernetes from Azure Kubernetes Service

#elasticsearch #kubernetes #azure #observability

It's safe to say that Kubernetes is the de facto standard for orchestrating containers and the applications running in them. As the standard, a variety of managed services and orchestration options are available to choose from. In this blog post, we're going to take a look at running the Elastic Stack on Azure Kubernetes Service (AKS) using Elastic Cloud on Kubernetes (ECK) as the operator.

Elastic Cloud on Kubernetes, is the official operator for running the Elastic Stack on Kubernetes. ECK helps to manage, scale, upgrade, and deploy the Elastic Stack securely. In the steps below, we will deploy ECK on AKS and then use that deployment to collect logs, metrics, security events from a virtual machine on Azure.

Here's what we'll do:

  1. Create an Azure Kubernetes Service cluster
  2. Install Elastic Cloud on Kubernetes
  3. Create an Elasticsearch cluster
  4. Deploy Kibana
  5. Create an Azure VM for us to monitor
  6. Deploy Metric beat to collect VM metrics and events

Deploying AKS, ECK, Elasticsearch, and Kibana

Note: You need to have Azure Account and the Azure CLI for Microsoft Azure installed to run some platform-specific commands. This helps you to create your cluster using this Azure CLI command.

Step 1: Create an AKS cluster 

az aks create --resource-group resourceGroupName --name clusterName --node-count 3 --generate-ssh-keys
Enter fullscreen mode Exit fullscreen mode

Step 2: Connect to the AKS cluster 

az aks get-credentials --resource-group resourceGroupName --name clusterName
Enter fullscreen mode Exit fullscreen mode

Step 3: Install the ECK operator 

kubectl apply -f https://download.elastic.co/downloads/eck/1.1.2/all-in-one.yamlkubectl -n elastic-system logs -f statefulset.apps/elastic-operator
Enter fullscreen mode Exit fullscreen mode

Step 4: Create an Elasticsearch cluster with an external IP

We're using the default load balancer that is available with Azure Kubernetes Service.

cat <<EOF | kubectl apply -f -apiVersion: elasticsearch.k8s.elastic.co/v1 
kind: Elasticsearch 
metadata: 
  name: quickstart 
spec: 
  version: 7.9.2 #Make sure you use the version of your choice 
  http: 
    service: 
      spec: 
        type: LoadBalancer #Adds a External IP 
  nodeSets: 
  - name: default 
    count: 1 
    config: 
      node.master: true 
      node.data: true 
      node.ingest: true 
      node.store.allow_mmap: false 
EOF
Enter fullscreen mode Exit fullscreen mode

Step 5: Monitor the cluster creation 

kubectl get elasticsearch
Enter fullscreen mode Exit fullscreen mode 
kubectl get pods -w
Enter fullscreen mode Exit fullscreen mode

Step 6: Check the logs of the pod created 

kubectl logs -f quickstart-es-default-0
Enter fullscreen mode Exit fullscreen mode 
kubectl get service quickstart-es-http
Enter fullscreen mode Exit fullscreen mode

Step 7: Retrieve the password of Elasticsearch cluster 

PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode)
Enter fullscreen mode Exit fullscreen mode 
curl -u "elastic:$PASSWORD" -k "https://<IP_ADDRESS>:9200"
Enter fullscreen mode Exit fullscreen mode

Note: The public IP address of Elasticsearch can be picked by running:

kubectl get svc quickstart-es-http

Step 8: Deploy Kibana 

cat <<EOF | kubectl apply -f - 
apiVersion: kibana.k8s.elastic.co/v1 
kind: Kibana 
metadata: 
  name: quickstart 
spec: 
  version: 7.9.2 #Make sure Kibana and Elasticsearch are on the same version. 
  http: 
    service: 
      spec: 
        type: LoadBalancer #Adds a External IP 
  count: 1 
  elasticsearchRef: 
    name: quickstart 
EOF
Enter fullscreen mode Exit fullscreen mode

Step 9: Monitor the Kibana deployment 

kubectl get kibana
Enter fullscreen mode Exit fullscreen mode

Ingesting and analyzing Azure metrics

Now that we have created an Elasticsearch cluster with Kibana in AKS, let's go ahead and ingest some observability data from Azure Cloud itself. Filebeat and Metricbeat make this easy by coming with out-of-the-box an Azure Module, helping to easily gather logs (activity, sign in, audit) and metrics (vm, container registry, billing) from Azure Cloud Platform.

In this tutorial, we will install Metricbeat on an Azure VM and enable the Azure cloud module. Before that, we also need to have credentials to authenticate with Azure Monitor REST API which uses Azure Resource Manager authentication model.

We need to have client_id, client_secret, subscription_id, tenant_id which can be obtained by creating an Azure Active Directory App. You can use this guide to Azure AD application and service principal that can access resources.

Step 1: Create a Azure VM and SSH into the VM 

az vm create \ 
  --resource-group myResourceGroup \ 
  --name myVM \ 
  --image UbuntuLTS \ 
  --admin-username azureuser \ 
  --generate-ssh-keys ssh azureuser@<IP_ADDRESS>
Enter fullscreen mode Exit fullscreen mode

Step 2: Install Metricbeat 

wget https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-7.9.2-amd64.deb
Enter fullscreen mode Exit fullscreen mode

Step 3: Configure Elasticsearch and Kibana credentials in Metricbeat

This helps us to ship the data to Elasticsearch Cluster created on AKS as well as load dashboards in Kibana.

vim /etc/metricbeat/metricbeat.yml
Enter fullscreen mode Exit fullscreen mode 
 setup.kibana: 
   host: "https://<public_ip_addr>:5601"
Enter fullscreen mode Exit fullscreen mode

Note: The public IP address of Kibana can be picked by running:

kubectl get "kubectl get svc quickstart-kb-http"

vim /etc/metricbeat/modules.d/azure.yml.disabled
Enter fullscreen mode Exit fullscreen mode

Replace the client_id, client_secret, subscription_id, tenant_id for all the metricsets listed in the yml file. A sample might look like this.

- module: azure 
  metricsets: 
  - monitor 
  enabled: true 
  period: 300s 
  client_id: '8dec1ab1-1691-48a6-af43-f87de68e971b' 
  client_secret: '~fwL-MhOcguaD2yK1e_.OWHhhqwdp-p974' 
  tenant_id: 'aa40685b-417d-4664-b4ec-8f7640719adb' 
  subscription_id: '70bd6e77-4b1e-4835-8896-db77b8eef364' 
  refresh_list_interval: 600s 
  resources: 
  - resource_query: "resourceType eq 'Microsoft.DocumentDb/databaseAccounts'" 
    metrics: 
    - name: ["DataUsage", "DocumentCount", "DocumentQuota"] 
      namespace: "Microsoft.DocumentDb/databaseAccounts"
Enter fullscreen mode Exit fullscreen mode

Step 4: Enable Azure Module and start Metricbeat 

cd /usr/bin/
Enter fullscreen mode Exit fullscreen mode 
./metricbeat modules enable azure
./metricbeat setup --dashboards
./metricbeat -e
Enter fullscreen mode Exit fullscreen mode

Step 5: Monitor Metrics of Azure in Kibana

Log into Kibana and head over to Dashboards. Search for "Azure" to look at several preconfigured dashboards regarding storage, database, billing. Here is what a sample monitoring dashboard would look like:

azure-metrics

Wrapping up

And that's that! You have successfully built a secure Elastic Stack deployment on a managed Kubernetes service. You can also deploy other applications like Elastic APM or Elastic Workplace Search. In addition to that, you can also enable cross-cluster search and replication, which enables you to deploy an Elastic Stack on Kubernetes cluster across regions to serve users.

We encourage you to try ECK for yourself (on any Kubernetes service), and if you have further questions related to this blog post, you can always reach us out on our community discussion forums or Slack workspace.

If you'd like to learn more about how Elastic can help with Kubernetes observability, check out our Best of Elastic Observability webinars series

You can follow me at Aravind Putrevu

Top comments (0)

Read next

thenjdevopsguy profile image

Running Stormforge with Karpenter and Kubecost

Michael Levan -

justinepdevasia profile image

Building HashiCorp Nomad Cluster in Vultr Cloud using Terraform

justinepdevasia -

jadelassery profile image

How to monitor Openshift using Datadog Operator

Jade -

sre_panchanan profile image

Kubernetes Networking Deep Dive - Services, Ingress, and Gateway API

Panchanan Panigrahi -