Running Kibana, Elastic, and APM with Docker Compose

Prerequisites

Ensure that Docker and Docker Compose are installed. If not, visit the official Docker website for installation instructions: Docker Installation.

Step 1: Create a docker-compose File

Create a file named elastic-stack.yml and paste the following content, do update the password where it says :

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.13.4
    container_name: elasticsearch
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=true
      - xpack.security.transport.ssl.enabled=false
      - ELASTIC_USERNAME=elastic
      - ELASTIC_PASSWORD=<password>
    ports:
      - 9200:9200
    volumes:
      - /data:/usr/share/elasticsearch/data
    networks:
      - elastic

  kibana:
    image: docker.elastic.co/kibana/kibana:8.13.4
    container_name: kibana
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
      - ELASTICSEARCH_USERNAME=elastic
      - ELASTIC_PASSWORD=<password>
      - ELASTICSEARCH_SERVICE_TOKEN=<token>
    ports:
      - 5601:5601
    networks:
      - elastic

  apm-server:
    image: docker.elastic.co/apm/apm-server:8.13.4
    container_name: apm-server
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
      - ELASTICSEARCH_USERNAME=elastic
      - ELASTIC_PASSWORD=<password>
      - kibana.host=http://kibana:5601
    ports:
      - 8200:8200
    networks:
      - elastic

networks:
  elastic:
    driver: bridge

Step 2: Start the Stack

Run the following command to start the Elastic stack:

sudo docker-compose -f elastic-stack.yml up -d

Step 3: Check Kibana Logs

Check the Kibana logs using:

sudo docker-compose -f elastic-stack.yml logs kibana

You might encounter the following error:
"value of "elastic" is forbidden. This is a superuser account that cannot write to system indices that Kibana needs to function."

If so, generate a service token:

sudo docker exec -it elasticsearch elasticsearch-service-tokens create elastic/kibana kibana-system

• generate a token using

sudo docker exec -it elasticsearch elasticsearch-service-tokens create elastic/kibana kibana-system


Step 4: Update the Token in docker-compose

Replace the environment variable in the Kibana service with the newly generated token:

kibana:
  image: docker.elastic.co/kibana/kibana:8.13.4
  container_name: kibana
  environment:
    - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
    - ELASTICSEARCH_SERVICE_TOKEN=<token>
  ports:
    - 5601:5601
  networks:
    - elastic

Restart kibana using

sudo docker-compose -f elastic-stack.yml restart kibana

Step 5: Verify the Stack

Check the status of the running containers:

docker ps

Your services should now be accessible on the following ports:

• Elastic: 9200
• Kibana: 5601
• APM: 8200

Resolving Token Issues

If the generated token doesn’t work, you can manually create a Kibana system user:

curl -X POST "localhost:9200/_security/user/kibana_system" -H "Content-Type: application/json" -u elastic:password -d'
{
  "password": "<password>",
  "roles": ["kibana_system"]
}'

Update your docker-compose file to use the newly created user credentials:

  kibana:
    image: docker.elastic.co/kibana/kibana:8.13.4
    container_name: kibana
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
      - ELASTICSEARCH_USERNAME=elastic
      - ELASTIC_PASSWORD=<password>

Restart the stack:

sudo docker-compose -f elastic-stack.yml down
sudo docker-compose -f elastic-stack.yml up -d

Now your stack should function as expected.