DEV Community


Posted on

Prometheus and Grafana, Part 2

The Prometheus.yml:

# my global config
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.

  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

      - targets: ['localhost:9090']

  - job_name: 'node-exporter'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s
    metrics_path: /metrics

      - targets:
        - <PUBLIC IP>:9100
Enter fullscreen mode Exit fullscreen mode

This is a good starting place for a monitoring system for Linux boxes

Running the services in docker is also advised as it simplifies the rollout.

Docker node exporter:

sudo docker run --restart always -d --net="host" --pid="host"  -v "/:/host:ro,rslave"  prom/node-exporter:latest  --path.rootfs=/host
Enter fullscreen mode Exit fullscreen mode

Docker prometheus:

sudo docker service create --replicas 1 --name dockerprometheus     --mount type=bind,source=/home/ubuntu/prometheus/prometheus.yml,destination=/etc/prometheus/prometheus.yml     --publish published=9090,target=9090,protocol=tcp     prom/prometheus
Enter fullscreen mode Exit fullscreen mode

When you get around to making panels in Grafana, these queries are very useful:

100 - ((node_filesystem_avail_bytes{mountpoint="/",fstype!="rootfs"} * 100) /            node_filesystem_size_bytes{mountpoint="/",fstype!="rootfs"})
Enter fullscreen mode Exit fullscreen mode

Enter fullscreen mode Exit fullscreen mode

OR you could just setup a template for Node Exporter:
To import a new dashboard for Node Exporter, go to Dashboards -> Manage
Image description

Enter 405 for the Grafana ID.
Image description

Pick up the name and choose a Data Source for the dashboard.
Image description

To make an alert, start a NEW dashboard and NEW panel. Template dashboards like above will not work.

Enter the query for the item you want to track and press the Alert tab.
That should do it for a basic monitoring service.

Top comments (0)