DEV Community: Beyza

Grafana and Prometheus Setup on Ubuntu 22.04

Beyza — Sun, 10 Dec 2023 12:31:48 +0000

This tutorail will show you how you can create a Grafana Server and how to download Grafana agents and Prometheus to your servers.

Start Prometheus Main server

wget https://github.com/prometheus/prometheus/releases/download/v2.48.0/prometheus-2.48.0.linux-amd64.tar.gz
tar xvfz prometheus-*.tar.gz
cd prometheus-*-amd64/
screen -S prometheus => creates a new screen in you server
Update prometheus yaml file:
nano /prometheus-2.48.0.linux-amd64/prometheus.yml

Example:

# my global config
global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["localhost:9090"]

  # configs for scraping node exporter metrics
  - job_name: node
    static_configs:
      - targets: ['localhost:9100']

Run ./prometheus --config.file=prometheus.yml

http://public ip:9090/metrics You should be able to see Prometheus metrics.

(Use Ctrl a+d = go to main screen)

Download Node exporter:

screen -S node-exporter
wget https://github.com/prometheus/node_exporter/releases/download/v1.5.0/node_exporter-1.5.0.linux-amd64.tar.gz
tar xvfz node_exporter-*.*-amd64.tar.gz
cd node_exporter-*.*-amd64
./node_exporter

http://<public ip>:9100/metrics Prometheus metrics

(Ctrl a+d = Go to main screen)

Install Grafana sudo yum install -y https://dl.grafana.com/enterprise/release/grafana-enterprise-9.5.2-1.x86_64.rpm sudo systemctl start grafana-server.service

This is your Grafana server: http://public ip:3000/

Setup your admin name and pass

Username: admin
Password: mypass

Note: service grafana-server restart = If you make any changes to the main server's configuration, be sure to run this command afterwards.

Resources:

The upcoming tutorials will cover the topics of creating dashboards, alerting systems, and integrating with Slack.

Create ELK Kibana Watcher and Send Email Reports as PNG

Beyza — Thu, 07 Dec 2023 12:41:31 +0000

A tutorial on the process of sending a Kibana dashboard to email addresses as a (PNG) image is not readily available. After extensive experimentation and research, I have produced a comprehensive guide on this topic. The tutorial outlines the step-by-step instructions for this task, and will provide a much-needed resource for those interested in achieving this objective.

Steps:

Go to Your account/Contacts
Add your email here
Confirm the email coming from Elastic.
Go to Stack Manager > Watcher
Create a new Watcher
Edit yout watch
Example:

{
  "trigger": {
    "schedule": {
      "interval": "24h"
    }
  },
  "input": {
    "none": {}
  },
  "condition": {
    "always": {}
  },
  "actions": {
    "email_admin": {
      "email": {
        "profile": "standard",
        "attachments": {
          "error_report.png": {
            "reporting": {
              "url": "reporting-url (1)",
              "interval": "260s",
              "auth": {
                "basic": {
                  "username": "user (2)",
                  "password": "passw (3)"
                }
              }
            }
          }
        },
        "to": [
          "'me <me@mail.com>, you <you@mail.com>'"
        ],
        "subject": "Monitoring Report"
      }
    }
  }
}

There are some fields that you need to fill:

STEP (1)
Go one of your dashboards
Click Share > PNG Reports > Advanced Options > Copy Post Url

STEP (2) and (3)
Create user with privileges,
go to: management/security/users

Create a user

(UTC timezone)

Save your watcher and exit to main page. That way you will get PNG reports to your email everyday.

Main Source:
https://www.elastic.co/guide/en/elasticsearch/reference/current/trigger-schedule.html

basic docker commands #class-notes

Beyza — Tue, 26 Apr 2022 15:46:39 +0000

Container Logic

Without containerizing:

src:kablosuzkedi

With containerizing:

src: kablosuzkedi

NOTE: https://hub.docker.com/ -> The holy documentation.

Commands

$ docker pull ubuntu |or| docker run ubuntu $ docker pull mongo $ docker run redis
…

It pulls ubuntu/mongo/redis from dockerhub. docker run controls the local to check if exists, if not it installs it on docker hub.

$ docker images
shows the installed images

$ docker run -it ubuntu

opens the ubuntu terminal (-it means interactive terminal)

$ docker ps $ docker ps -a

this commands shows which container(s) are active right now.
-a refers to “all” and it shows all containers which were active before (it also shows active ones too).

$ docker run -it -- name my_terminal ubuntu

renames ubuntu to my_terminal (!. double - - before name)

$ docker run -it my_terminal

now the terminal can use this name.

$ docker start my_terminal $ docker stop my_terminal

starts and stops terminals in the background.

$ docker stop e7g

Container ID also can use instead of the name, same thing happens (the Container ID can learn by docker ps -a command).

$ docker rm my_terminal $ docker rm e7g $ docker rm eg7 gh4 77v 22w fg4 #for multiple containers $ docker container rm $(docker container ls -aq) #for all containers

removes the specific container.

$ docker run -d mongo

detach mode works in the background.

$ docker container logs g87dw

shows logs.

$ docker run -it user/lets-meet-app

if there is an interactive app wanted to use, -it flag must be written (otherwise it will just print the code).

$ docker inspect ubuntu

prints its attributes.

Port Mapping

$ docker run -p 3000:3000 node #or another custom app, user/my-app $ docker run -p 3001:3000 node $ docker run -p 3002:3000 node

Now the app can use on port 3000, 3001, 3002 (host:container).

#try localhost:3000 , localhost:3001 and localhost:3002 … Same result.

Volume Mapping and Connecting Containers

= Notes =
• Containers work as a process (like downloaded Spotify, MS Word…)
• Host containers work stateless… So they don't record or save any data on them. When the container stops all info will be gone forever… To prevent this situation, volumes are used.

$ docker volume create first-volume

creates a volume

$ docker container run -it -v first-volume:/sample alpine sh

the first field is the name of the volume (first-volume). The second field is the path where the file or directory are mounted in the container (/sample)
A simple exercise for volume mapping

docker volume create my-volume
docker container run -it -v my-volume:/sample alpine sh
cd sample
echo "hello from the first container" >> file1.txt
exit
docker container rm <ContainerId>

docker volume inspect my-volume
cd /var/lib/docker/volumes/my-volume/_data
cat file1.txt

or

docker container run -it -v first-volume:/try1 alpine sh
cd try1
cat file1.txt

touch file2.txt
echo "hello from the second container" >> file2.txt
exit

docker container run -it -v first-volume:/try2 ubuntu sh
cd try2
ls

$ docker run --name mysql-server -v /opt/data:/etc/mysql/conf.d -p 3306:3306 -e MYSQL_ROOT_PASSWORD=test147 -d mysql

-e stands for: env variable, -p: for port, - -name : for rename, -v : volume. MYSQL_ROOT_PASSWORD is necessary for the password.

$ docker rmi redis

rmi = remove image, removes images.

Networking in Docker

There are 3 basic network drivers in Docker, but the most common use is the bridge network.

$ docker network ls

$ docker run -it -d -- name cont1 alpine ash $ docker run -it -d -- name cont2 alpine ash

run 2 containers with alpine image.

$ docker network inspect bridge

shows the details of the bridge network.

$ docker inspect cont2 | grep IPAddress

getting the ip address of cont2.

$ docker attach cont1

attaching containers.

$ ping -c 5 <cont2 ip address>

-c is stand for 5 hops.

! Note
If we try to ping the 2nd container by name, it will give a "bad address" error because default bridge network containers can only communicate with their IP addresses..

$ docker stop cont1 cont2 $ docker rm con1 cont2

stops and removes containers.

$ docker network create --driver bridge mynet

creates a network bridge

$ docker network connect mynet cont3

connects a container to my network

Docker Compose

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services.

Then, with a single command, you create and start all the services from your configuration. To learn more about all the features of Compose. Docs

Example usage

docker-compose.yml

version: "3.9"  # optional since v1.27.0
services:
  web:
    build: .
    ports:
      - "8000:5000"
    volumes:
      - .:/code
      - logvolume01:/var/log
    links:
      - redis
  redis:
    image: redis

volumes:
  logvolume01: {}

$ docker compose up

starts and runs the app.

Plus: Dockerfile

A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image.

Dockerfile example

FROM python:alpine
COPY . /app
WORKDIR /app 
RUN pip install -r requirements.txt
EXPOSE 80
CMD python ./app.py

$ docker build -f /path/to/a/Dockerfile .

Docs

Note for AWS VM users:

Instead of using sudo every command, write these.

$ sudo usermod -a -G docker ec2-user $ newgrp docker

Sources:

https://docs.docker.com/engine/reference/commandline/rm/
https://www.youtube.com/watch?v=ZeYIp1PrWXc&t=730s
Clarusway pre-classes -> https://clarusway.com/