DEV Community: Tahsin Ahmed

Laravel log stream to Kibana Elasticsearch Monolog

Tahsin Ahmed — Fri, 08 Dec 2023 10:52:00 +0000

ELK ( Elasticsearch, Logdash, and Kibana) is one of the popular log management platforms for log analysis. But for this project I will be using Laravel Monolog for handling the logs instead of Logdash.
Before getting into coding let’s know a little about the platforms.

Monolog :

Monolog is a PHP logging library that’s included in Laravel. This library is widely customizable and used because of the easy handle management and it provides build in handler that can easily help anyone to send logs to multiple destinations.
Here’s Monolog source code and Laravel documentation for monolog.

Elasticsearch :

Elasticsearchis a free and open-source analytics and search engine that returns and stores data in the JSON index format. Users may easily store, search and analyze huge quantities of data thanks to this engine. Because it uses an index rather than directly querying the text data, it is able to respond to search results in milliseconds. Simply put, Elasticsearchfunctions as a server that can handle JSON queries and return JSON information.

Kibana:

Kibana is a visual interface tool that lets you work with log data that has been mass-stored in Elasticsearch clusters to explore, visualize, and generate reports. Kibana's primary functionalities are data analysis and querying. Furthermore, you may see data in several ways with Kibana's visualization tools by utilizing heat maps, line graphs, pie charts, histograms, and geospatial support. You can look for root cause diagnostics in the data stored in Elasticsearch using a variety of techniques.
Kibana documentation

Ok!! Let's start coding

Step 01: Run Kibana and Elasticsearch with docker

I am using docker to run Laravel , Kibana, Elasticsearch . I hope you already know how to run Laravelprojects in docker. For Kibana and Elasticsearch, Here's my docker-compose code: -

 kibana: 
    image: kibana:8.11.1
    container_name : kibana_container
    ports:
      - "5601:5601"
    environment:
      ELASTICSEARCH_HOSTS: http://elasticsearch:9200
      ELASTICSEARCH_URL: http://elasticsearch:9200
    networks:
      - laravel
  elasticsearch: 
    image: elasticsearch:8.11.1
    container_name: elasticsearch_container
    ports:
      - "9200:9200"
    environment:
      - http.host=0.0.0.0
      - transport.host=127.0.0.1
      - discovery.type=single-node # To run single node 
      - xpack.security.enabled=false # To remove SSL complexity
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - "./elk-data:/usr/share/elasticsearch/data"

    networks:
      - laravel

PS : I am keeping all my container in laravelnetwork.
Now docker-compose up --build to see Kibanaand Elasticsearch is up and running.
Elasticsearch will run on : http://localhost:9200/
Kibana will run on : http://localhost:5601/

Step 02: Setup `env` file

In .env add this lines:

ELASTIC_HOST=elasticsearch_container:9200
ELASTIC_LOGS_INDEX=laravel_logs
LOG_CHANNEL=elastic

Step 03: Create config file

Create a config file inside config/elastic_log.php . Write this in that file

<?php

return [
    'host' => env('ELASTIC_HOST'),
    'index' => 'laravel_logs',
    'prefix' => 'api',
    'type' => '_doc',
];

Step 04: Create ElasticLog Provider

Create a Laravel provider with this command

php artisan make:provider ElasticlogProvider

and put this block of code inside boot function

        $index = rtrim(config('elastic_log.prefix'), '_') . '_' . config('elastic_log.index');
        $type = config('elastic_log.type');

        $this->app->bind(Client::class, function ($app) {
            return ClientBuilder::create()->setHosts([config('elastic_log.host')])->setHttpClient(new HttpClient(['verify' => false]))->build();
        });

        $this->app->bind(ElasticsearchFormatter::class, function ($app) use ($index, $type) {
            return new ElasticsearchFormatter($index, $type);
        });

        $this->app->bind(ElasticsearchHandler::class, function ($app) use ($index, $type) {
            return new ElasticsearchHandler($app->make(Client::class), [
                'index'        => $index,
                'type'         => $type,
                'ignore_error' => false,
            ]);
        });

Step 04: Create Log channel

In config/logging.php add a new monolog channel named elastic under channels array

'elastic' => [
            'driver' => 'monolog',
            'handler' => ElasticsearchHandler::class,
            'level' => 'debug',
            'formatter' => ElasticsearchFormatter::class,
 ],

Step 05: Create a custom command

Create a Laravel custom command with

php artisan make:command ElasticLogSetup

In handle function add this :

$index = rtrim(config('elastic_log.prefix'), '_') . '_' . config('elastic_log.index');

if (!$this->client->indices()->exists(['index' => $index])) {
            $this->client->indices()->create([
                'index' => $index,
            ]);
        }

Make sure to change the signature of the command :

   protected $signature = 'elastic:log_setup';

Now run the command :

php artisan elastic:log_setup

Step 06: Add view data for viewing in Kibana

Check the Kibana container is running. If it's working go to

http://localhost:5601/app/kibana#/management/kibana/index_patterns

And add the api_laravel_logs index into view

Done

Kibana now set up with Laravel project, You can stream logs with default Laravel Log facade.

Thank you !!!

Resources :

How to open Atom editor from terminal in Windows

Tahsin Ahmed — Thu, 13 Oct 2022 06:55:35 +0000

Step 01:

Got to windows start and type run or type click win + R
This will open up run prompt.

Step 02:

Type SystemPropertiesAdvanced.exe and hit enter System properties window will show up.

Step 03:

Then go

Environment Variables >> Path
click edit then new and paste

C:\Users\<user>\AppData\Local\atom\app-x.xx.x\resources\cli

Save and Done.
Now you can use atom . to open atom with bash or terminal.

_P.S : To know your atom version go to _
C:\Users\<user>\AppData\Local\atom\
You will find your app folder with version with it.

DAY 01 - 100 Days With Vue

Tahsin Ahmed — Wed, 24 Nov 2021 08:11:57 +0000

At last, I have started to learn Vue.js. Well, starting is the hard part of learning. Today I have learned about how to integrate Vue CDN and add Vue property to your project.
It's simple just add Vue CDN at the end of your Html file. Vue is ready for use.

Vue CDN

<script src="https://unpkg.com/vue@next" defer></script>

Adding Vue property in a particular HTML section

In your app.js (External JS file) file you have to create a Vue app and mount it into a particular section. Well you can use a variable to declare the Vue app

const app = Vue.createApp({});

To add this Vue app into your HTML code you need to use the mount function to this object. Mount accepts a string value of a CSS selector in which this Vue code applies. Here we have a section that has an id named #user-goal

const app = Vue.createApp({});
app.mount('#user-goal');

Nice ! Vue is mounted on that section

"data" property of Vue object

"data" is a predefined property of the Vue object which holds all the variables of the Vue app. "data" property always returns an object which holds a collection of variable names and values in a key-value pair. the variable value can be anything String, Number
, Boolean etc.

const app = Vue.createApp({
   data(){
      return {
        variable_name : "variable_value"
     }
  }
});
app.mount('#user-goal');

"Interpolations" Showing data property value

"Interpolations" is a fancy name of handlebars, if you want to show your variable value into HTML. Just use double curly braces and write the variable name which you had declared on the data property.

    <p>{{ variable_name }}</p>

"Interpolations" tells Vue to replace the variable name with the actual value of it. so you can't use "Interpolations" outside Vue mounted section.

"v-bind" attribute of Vue

Well, sometimes we need to pass value to Html attribute. For example, let's say you want to return a link to an anchor tag 'href' attribute, but if you use "Interpolations" it won't work. You need to use the "v-bind" attribute for that.
"v-bind" is a Vue attribute that helps to bind any Html attribute to Vue variables. Example :

    <p> <a v-bind:href="variable_name">Link </a></p>

P.S: There are more uses of v-bind and we can use the shorter version of v-bind, But for now, let's stick with this.

"v-html" attribute of Vue

Now let's say you want to show an Html code using a variable. If you use only "Interpolations" it will show markup value and won't fulfill the purpose. For that, you need to use v-html attribute.
Example:
In app.js

const app = Vue.createApp({
   data(){
      return {
        variable_name : "<h1> Hello this is value </h1>"
     }
  }
});
app.mount('#user-goal');

In index.html

     <p v-html="variable_name"></p>

"methods" property of Vue object

Till now, we have used only static data. We need to add some dynamic feel to it. To do this, we need to use the "methods" property of the Vue object. "methods" is a predefined property that accepts only JS objects that hold the collection of functions.

const app = Vue.createApp({
   data(){
      return {
        variable_name : "<h1> Hello this is value </h1>"
     }
  },
methods: {
    function_name: function () {
     //do something
    }
  }
});
app.mount('#user-goal');

We can use direct methods in "Interpolations"
In index.html

     <p> {{ function_name() }}</p>

Use "data" property into "methods"

Basically we need to use variable into methods for that we just need to use "$this" keyword.

const app = Vue.createApp({
   data(){
      return {
        variable_name : "<h1> Hello this is value </h1>"
     }
  },
methods: {
    function_name: function () {
      $this.variable_name = "new value"
    }
  }
});
app.mount('#user-goal');

Github Practice Code

Tahsin-Ahmed52225 / 100days-with-Vue

Learning Vue JS

Tahsin Ahmed — Thu, 18 Nov 2021 10:25:56 +0000

Hello World,
I was curious to learn Vue.Js for a long time, But unfortunately, I couldn't manage time to start this.
So I took a challenge to Learn Vue.Js. I will be describing what I have learned and also going to share codes with you.
I would appreciate any kind of advise, remarks and Highly appreciate if you guys join with me in this journey
Wish me Luck <3

How to increase max upload size in All in one wp migration plugin

Tahsin Ahmed — Sun, 04 Apr 2021 07:41:22 +0000

While migrating WordPress site from local environment to server or server to server All in one wp migration is one of the best options,
But the problem is it has an upload limit of 512M which is not enough while working with a large project. To exceed this limit it offers a paid solution.
Increasing PHP and httaccess file upload limit doesn't work on it.
To extend this we have to use an older version of this plugin
I recommend Version 6.77 because after that this plugin restricted some features that are use full.
Download Link

After installing this, Goto

wp-content > plugins > all-in-one-migration > constant.php

Change this value according to your need

define('AI1WM_MAX_FILE_SIZE', 1024 * 1024 * 1024);

PS: values are in bytes

Oola your upload limit is extended.