DEV Community: Shubham Patil

Install and Setup Grafana & Prometheus on Ubuntu 20.04 | 22.04/EC2

Shubham Patil — Fri, 15 Mar 2024 06:27:15 +0000

What is Prometheus ?
Prometheus is a open source Linux Server Monitoring tool mainly used for metrics monitoring, event monitoring, alert management, etc.
Prometheus has changed the way of monitoring systems and that is why it has become the Top-Level project of Cloud Native Computing Foundation (CNCF).
Prometheus uses a powerful query language i.e. “PromQL”.
In Prometheus tabs are on and handles hundreds of services and microservices.
Prometheus use multiple modes used for graphing and dashboarding support.
Prometheus Components

Prometheus Server Prometheus server is a first component of Prometheus architecture. Prometheus server is a core of Prometheus architecture which is divided into several parts like Storage, PromQL, HTTP server, etc. In Prometheus server data is scraped from the target nodes and then stored int the database. 1.a. Storage Storage in Prometheus server has a local on disk storge. Prometheus has many interfaces that allow integrating with remote storage systems. 1.b. PromQL

Prometheus uses its own query language i.e. PromQL which is very powerful querying language.
PromQL allows the user to select and aggregate the data.

Service Discovery Next and very important component of Prometheus Server is the Service Discovery. With the help of Service discovery the services are identified which are need to scraped. To Pull metrics, identification of services and finding the targets are compulsory needed. Through Service discovery we monitor the entities and can also locate its targets.
Scrape Target Once the services are identified and the targets are ready then we can pull metrics from it and can scrape the target. We can export the data of end point using node exporters. Once the metrics or other data is pulled, Prometheus stores it in a local storage.
Alert Manager Alert Manager handles the alerts which may occurs during the session. Alert manager handles all the alerts which are sent by Prometheus server. Alert manager is one of the very useful component of Prometheus tool. If in case any big error or any issue occurs, alert manager manage those alerts and contact with human via E-mail, Text Messages, On-call, or any other chat application service.
User Interface User interface is also a important component as it builds a bridge between the user and the system. In Prometheus, user interface are note that much user friendly and can be used till graph queries. For good exclusive dashboards Prometheus works together with Grafana (visualization tool). Using Grafana over Prometheus to visualize properly we can use custom dashboards. Grafana dashboards displays via pie charts, line charts, tables, good data graphs of CPU usage, RAM utilization, network load, etc with indicators. Grafana supports and run with Prometheus by querying language i.e. PromQL. To fetch data from Prometheus and to display the results on Grafana dashboards PromQL is used. What is Grafana ? Grafana is a free and open source visualization tool mostly used with Prometheus to which monitor metrics. Grafana provides various dashboards, charts, graphs, alerts for the particular data source. Grafana allows us to query, visualize, explore metrics and set alerts for the data source which can be a system, server, nodes, cluster, etc. We can also create our own dynamic dashboard for visualization and monitoring. We can save the dashboard and can even share with our team members which is one of the main advantage of Grafana. What is Node Exporter ? Node exporter is one of the Prometheus exporters which is used to expose servers or system OS metrics. With the help of Node exporter we can expose various resources of the system like RAM, CPU utilization, Memory Utilization, disk space. Node exporter runs as a system service which gathers the metrics of your system and that gathered metrics is displayed with the help of Grafana visualization tool. Prerequisites Ubuntu with 22.04 Version Root user account with sudo privilege. Prometheus system user and group. Sufficient storage on your system and good internet connectivity. Ports Required- 9090 (Prometheus), 3000 (Grafana), 9100 (Node Exporter) We will update the system repository index by using the following command.

Step #1:Creating Prometheus System Users and Directory
Create a system user for Prometheus

sudo useradd --no-create-home --shell /bin/false prometheus

Output

Create the directories in which we will be storing our configuration files and libraries:

sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus

Output

Set the ownership of the /var/lib/prometheus directory

sudo chown prometheus:prometheus /var/lib/prometheus

Output

Step #2:Download Prometheus Binary File
Now we will download the latest version of Prometheus. We can copy the download link as per our Operating System from Prometheus download page

Using below command we can download Prometheus, here we are downloading Prometheus 2.46 version, you use above link to download specific version.

You need to inside /tmp :

cd /tmp/

Download the Prometheus setup using wget

wget https://github.com/prometheus/prometheus/releases/download/v2.46.0/prometheus-2.46.0.linux-amd64.tar.gz

Output

Extract the files using tar

tar -xvf prometheus-2.46.0.linux-amd64.tar.gz

Move the configuration file and set the owner to the prometheus user

cd prometheus-2.46.0.linux-amd64

sudo mv console* /etc/prometheus

sudo mv prometheus.yml /etc/prometheus

sudo chown -R prometheus:prometheus /etc/prometheus

Output

Move the binaries and set the owner

sudo mv prometheus /usr/local/bin/

sudo chown prometheus:prometheus /usr/local/bin/prometheus

Output

Step #3:Prometheus configuration file
We have already copied /opt/prometheus-2.26.0.linux-amd64/prometheus.yml file /etc/prometheus directory, verify if it present and should look like below and modify it as per your requirement.


sudo nano /etc/prometheus/prometheus.yml

Step #4:Creating Prometheus Systemd file
Create the service file


[Unit]

Description=Prometheus

Wants=network-online.target

After=network-online.target

[Service]

User=prometheus

Group=prometheus

Type=simple

ExecStart=/usr/local/bin/prometheus \

— config.file /etc/prometheus/prometheus.yml \

— storage.tsdb.path /var/lib/prometheus/ \

— web.console.templates=/etc/prometheus/consoles \

— web.console.libraries=/etc/prometheus/console_libraries

[Install]

WantedBy=multi-user.target

Output

Reload systemd

sudo systemctl daemon-reload

Start, Enable, Status Prometheus service

sudo systemctl start prometheus

sudo systemctl enable prometheus

sudo systemctl status prometheus

Launch the Prometheus server by executing the following command:


./prometheus –config.file=prometheus.yml (or) ./prometheus

Verify Prometheus Installation:

Open your web browser and access http://server-ip-address:9090. If Prometheus is running correctly, you will see the Prometheus web interface.

Steps to Install and set up the Grafana on Ubuntu 20.04 and 22.04/EC2

Grafana
What is Grafana?
Grafana is an open-source platform for data visualization, monitoring, and analysis. It allows you to create and display interactive, customizable dashboards with real-time monitoring and analytics capabilities. Grafana supports a wide range of data sources, including time-series databases and relational databases.
With Grafana, you can easily create visualizations and alerts based on your data, as well as collaborate with other team members through sharing and annotations. It also provides advanced features like plugins, annotations, and panels, which allow you to extend the platform’s capabilities and create custom visualizations and integrations.
Grafana is widely used in industries such as IT, finance, healthcare, and more, for monitoring and analyzing various metrics, logs, and sensor data. It is known for its user-friendly interface, ease of use, and powerful features, making it a popular choice for data visualization and monitoring.
Installation of Grafana
Update your Ubuntu system using the following command:

sudo apt-get update

Getting packages on Ubuntu distributions
Add the Grafana repository to your Ubuntu installation:

sudo apt-get install -y apt-transport-https
sudo apt-get install -y software-properties-common wget
sudo wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

Install Grafana as a service
Now that all the packages are available, it is time for you to install them.
Update your package list using the following command:

sudo apt-get update

Install the Grafana package using the following command:

sudo apt-get install grafana

Check Grafana version
You can use the Grafana version command to check the currently installed version.

grafana-server -v

Start the Grafana service using the following command:

sudo systemctl start grafana-server

Enable the Grafana service to start on boot:
To start it and make sure that the service is always available every time the machine is restarted, type the command:

sudo systemctl enable grafana-server

Verify your Grafana installation
Check the status of the Grafana service to make sure it is running.
To verify it, run the following command:

sudo systemctl status grafana-server

Grafana should run automatically, but if this is not the case, make sure to start it.

sudo service grafana-server start

In the future to stop and restart the Grafana, the commands are:
Restart

sudo service grafana-server restart

Stop

sudo service grafana-server stop

Allow Grafana TCP port 3000 in the Firewall
Grafana's default HTTP port is 3000, you’ll need to allow access to this port on the firewall.
If your firewall is UFW type the following commands:

sudo ufw allow 3000/tcp

That’s it! You have now installed Grafana on Ubuntu. You can now start using it to create dashboards and visualize your data.
Set up Grafana

Access the Grafana web UI by visiting http://localhost:3000 in your web browser.

Dashboard Grafana

Setup for Data Source

After Setup Final Dashboard Output:=

Containerizing PHP,Apache, and MySQL

Shubham Patil — Tue, 27 Feb 2024 12:57:55 +0000

Introduction
Containerizing a PHP, Apache, and MySQL stack for hosting a blog is a great way to ensure consistency across different environments and simplify deployment processes. Docker is a popular tool for containerization that can help achieve this. Here's a step-by-step guide to containerizing such a stack:

The project structure is as follows :

/php-apache-mysql/
├── apache
│ ├── apache_php.conf
│ └── Dockerfile
├── Cache
│ ├── cbc42e4c979f69f0_0
│ └── index-dir
│ └── the-real-index
├── docker-compose.yml
├── Network Persistent State
├── php
│ └── Dockerfile
├── public
│ ├── config.php
│ ├── create.php
│ ├── delete.php
│ ├── dump
│ │ └── dump.sql
│ ├── error.php
│ ├── index.php
│ ├── read.php
│ └── update.php
└── README.md

Once this structure is replicated or cloned(using "git clone https://github.com/shub6059/php_apache_docker.git") with these files, Docker and Docker compose is installed locally, you can simply run "docker-compose up" from the root of the project to run this project, and point your browser to http://localhost:8080 to see the project running.
Docker Compose

Docker compose allows us to define the dependencies for the services, networks, volumes, etc as code.
docker-compose.yml
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
version: "3" services: php: build: './php/' volumes: - ./public:/var/www/html/ apache: build: './apache/' depends_on: - php - mysql ports: - "8080:80" volumes: - ./public:/var/www/html/ mysql: image: mysql:5.7 restart: always ports: - "3306:3306" volumes: - db_data:/var/lib/mysql # Initially creates employees table - ./public/dump:/docker-entrypoint-initdb.d/ environment: MYSQL_ROOT_PASSWORD: "passwd" MYSQL_DATABASE: "mydb" MYSQL_USER: "user1" MYSQL_PASSWORD: "passwd" volumes: db_data:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Here we are decoupling Apache, PHP and Mysql by building them out into separate containers.We will be using the below Dockerfiles to decouple them :
apache/Dockerfile
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
FROM httpd:2.4.33-alpine RUN apk update; \ apk upgrade; COPY apache_php.conf /usr/local/apache2/conf/apache_php.conf RUN echo "Include /usr/local/apache2/conf/apache_php.conf" \ >> /usr/local/apache2/conf/httpd.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
php/Dockerfile
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
FROM php:7.2.7-fpm-alpine3.7 RUN apk update; \ apk upgrade; RUN docker-php-ext-install mysqli
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Networking

We're going to have Apache proxy connections which require PHP rendering to port 9000 of our PHP container, and then have the PHP container serve those out as rendered HTML.
We need an apache vhost configuration file that is set up to proxy these requests for PHP files to the PHP container. So in the Dockerfile for Apache we have defined above, we add this file and then include it in the base httpd.conf file. This is for the proxying which allows us to decouple Apache and PHP. Here we called it apache_php.conf and we have the proxying modules defined as well as the VirtualHost.
apache/apache_php.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ServerName localhost LoadModule deflate_module /usr/local/apache2/modules/mod_deflate.so LoadModule proxy_module /usr/local/apache2/modules/mod_proxy.so LoadModule proxy_fcgi_module /usr/local/apache2/modules/mod_proxy_fcgi.so <VirtualHost *:80> # Proxy .php requests to port 9000 of the php-fpm container ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://php:9000/var/www/html/$1 DocumentRoot /var/www/html/ <Directory /var/www/html/> DirectoryIndex index.php Options Indexes FollowSymLinks AllowOverride All Require all granted </Directory> # Send apache logs to stdout and stderr CustomLog /proc/self/fd/1 common ErrorLog /proc/self/fd/2 </VirtualHost>
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Volumes

Both the PHP and Apache containers have access to a "volume" that we define in the docker-compose.yml file which maps the public folder of our repository to the respective services for them to access. When we do this, we map a folder on the host filesystem (outside of the container context) to inside of the running containers.This allows us to edit the file outside of the container, yet have the container serve the updated PHP code out as soon as changes are saved.
And we also use "db_data" volume to store the contents of database, so that even if containers are terminated the data won't get deleted.
PHP CRUD application

We'll use the following PHP application to demonstrate everything:
"dump.sql" creates a table 'employees' in the mydb database.
"config.php" connects the php code with the 'employees' table of mydb database.
"index.php" creates the frontend grid which displays records from "employees" table.
"create.php" generates web form that is used to insert records in the employees table.
"read.php" retrieves the records from the employees table.
"update.php" updates records in employees table.
"delete.php" deletes records in employees table.
"error.php" will be displyed if a request is invalid.
STEP :- Clone repo
git clone https://github.com/shub6059/php_apache_docker.git

Open in VS Code:

this type open clone repo structure
Start create Container

get data in Browser

Open url see output
http://localhost:8080/