There are many Docker logging platforms available with agents that can be installed to track metrics and monitor the performance and availability of your clusters.
But sometimes it’s just not worth the costs of such services depending on the scale of your operations. A more feasible option could be hosting your own monitoring and logging solution using Prometheus, Grafana, Alertmanager, cAdvisor, and NodeExporter.
If you still feel like it’s an overkill, you may want to settle down for a quick-and-dirty solution to monitor Docker events and send the logs to your destination of choice.
Docker has an API that allows listening to its events and comes with an array of options such as filtering based on conditions like event type and formatting the output to your liking.
With a little bit of bash scripting, we can put together a simple service that listens for particular event statuses and triggers a cURL call to alert you.
For this example, I’m filtering all events leading to containers shutting down either normally or due to an issue such as running out of memory.
Create the docker-monitor.sh
script:
#!/bin/bash
# Monitor all Docker events
docker events --filter 'event=die' --filter 'event=exec_die' --filter 'event=kill' --filter 'event=stop' --format '{{.ID}}' | while read event
do
container_id=$event
# Get the image name of the started container
image=`docker inspect --format='{{.Name}}' $container_id`
# Send a cURL call to alert you that something's not right!
curl https://mywebsite.com/log
done
Create a service at /etc/systemd/system/docker-monitor.service
to automatically start the script:
[Unit]
Description=Docker Monitor Service
[Service]
Type=forking
ExecStart=/path/to/docker-monitor.sh
[Install]
WantedBy=multi-user.target
Reload systemd
:
systemctl daemon-reload
Start your service:
systemctl start docker-monitor
Enable the service for automatic start on reboot:
systemctl enable docker-monitor
This post was originally published on my blog where I write all about tech.
Top comments (1)
AFAIK you can't really even get this info from cAdvisor. It's surprising there is no explicit daemon option to just have events pushed out with other logs.