ELK ( Elasticsearch, Logdash, and Kibana) is one of the popular log management platforms for log analysis. But for this project I will be using Laravel Monolog for handling the logs instead of Logdash.
Before getting into coding let’s know a little about the platforms.
Monolog :
Monolog is a PHP logging library that’s included in Laravel. This library is widely customizable and used because of the easy handle management and it provides build in handler that can easily help anyone to send logs to multiple destinations.
Here’s Monolog source code and Laravel documentation for monolog.
Elasticsearch :
Elasticsearchis a free and open-source analytics and search engine that returns and stores data in the JSON index format. Users may easily store, search and analyze huge quantities of data thanks to this engine. Because it uses an index rather than directly querying the text data, it is able to respond to search results in milliseconds. Simply put, Elasticsearchfunctions as a server that can handle JSON queries and return JSON information.
Kibana:
Kibana is a visual interface tool that lets you work with log data that has been mass-stored in Elasticsearch clusters to explore, visualize, and generate reports. Kibana's primary functionalities are data analysis and querying. Furthermore, you may see data in several ways with Kibana's visualization tools by utilizing heat maps, line graphs, pie charts, histograms, and geospatial support. You can look for root cause diagnostics in the data stored in Elasticsearch using a variety of techniques.
Kibana documentation
Ok!! Let's start coding
Step 01: Run Kibana and Elasticsearch with docker
I am using docker to run Laravel , Kibana, Elasticsearch . I hope you already know how to run Laravelprojects in docker. For Kibana and Elasticsearch, Here's my docker-compose code: -
kibana:
image: kibana:8.11.1
container_name : kibana_container
ports:
- "5601:5601"
environment:
ELASTICSEARCH_HOSTS: http://elasticsearch:9200
ELASTICSEARCH_URL: http://elasticsearch:9200
networks:
- laravel
elasticsearch:
image: elasticsearch:8.11.1
container_name: elasticsearch_container
ports:
- "9200:9200"
environment:
- http.host=0.0.0.0
- transport.host=127.0.0.1
- discovery.type=single-node # To run single node
- xpack.security.enabled=false # To remove SSL complexity
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- "./elk-data:/usr/share/elasticsearch/data"
networks:
- laravel
PS : I am keeping all my container in laravelnetwork.
Now docker-compose up --build to see Kibanaand Elasticsearch is up and running.
Elasticsearch will run on : http://localhost:9200/
Kibana will run on : http://localhost:5601/
Step 02: Setup env file
In .env add this lines:
ELASTIC_HOST=elasticsearch_container:9200
ELASTIC_LOGS_INDEX=laravel_logs
LOG_CHANNEL=elastic
Step 03: Create config file
Create a config file inside config/elastic_log.php . Write this in that file
<?php
return [
'host' => env('ELASTIC_HOST'),
'index' => 'laravel_logs',
'prefix' => 'api',
'type' => '_doc',
];
Step 04: Create ElasticLog Provider
Create a Laravel provider with this command
php artisan make:provider ElasticlogProvider
and put this block of code inside boot function
$index = rtrim(config('elastic_log.prefix'), '_') . '_' . config('elastic_log.index');
$type = config('elastic_log.type');
$this->app->bind(Client::class, function ($app) {
return ClientBuilder::create()->setHosts([config('elastic_log.host')])->setHttpClient(new HttpClient(['verify' => false]))->build();
});
$this->app->bind(ElasticsearchFormatter::class, function ($app) use ($index, $type) {
return new ElasticsearchFormatter($index, $type);
});
$this->app->bind(ElasticsearchHandler::class, function ($app) use ($index, $type) {
return new ElasticsearchHandler($app->make(Client::class), [
'index' => $index,
'type' => $type,
'ignore_error' => false,
]);
});
Step 04: Create Log channel
In config/logging.php add a new monolog channel named elastic under channels array
'elastic' => [
'driver' => 'monolog',
'handler' => ElasticsearchHandler::class,
'level' => 'debug',
'formatter' => ElasticsearchFormatter::class,
],
Step 05: Create a custom command
Create a Laravel custom command with
php artisan make:command ElasticLogSetup
In handle function add this :
$index = rtrim(config('elastic_log.prefix'), '_') . '_' . config('elastic_log.index');
if (!$this->client->indices()->exists(['index' => $index])) {
$this->client->indices()->create([
'index' => $index,
]);
}
Make sure to change the signature of the command :
protected $signature = 'elastic:log_setup';
Now run the command :
php artisan elastic:log_setup
Step 06: Add view data for viewing in Kibana
Check the Kibana container is running. If it's working go to
http://localhost:5601/app/kibana#/management/kibana/index_patterns
And add the api_laravel_logs index into view
Done
Kibana now set up with Laravel project, You can stream logs with default Laravel Log facade.
Thank you !!!
Top comments (0)