DEV Community: Minhaz

Java Spring Boot use Seq for logging

Minhaz — Mon, 21 Oct 2024 21:32:46 +0000

The primary motivation is that I use SEQ for logging, but I couldn't find a quick complete guide to use SEQ with java spring boot using gradle. So here I'm documenting my findings.

Steps

I'm using this configuration for the project

Add these to `build.gradle` for dependency

repositories {
    ...
    maven { url 'https://jitpack.io' }
}

dependencies {
    ...
    implementation 'com.github.serilogj:serilogj:v0.6.1'
}

Now just configure the logger

// import serilogj.Log;
// import serilogj.LoggerConfiguration;
// import serilogj.events.LogEventLevel;
// import static serilogj.sinks.seq.SeqSinkConfigurator.seq;

// setting up the logger
Log.setLogger(new LoggerConfiguration()
        .writeTo(seq("http://localhost:5341/"))
        .setMinimumLevel(LogEventLevel.Verbose)
        .createLogger());

// using the logger
var logger = Log.getLogger().forContext(SpringApplication.class);
logger.information("Hello World");

With this setup the logger should work

You can find the code here

Reference

Java Spring Boot use log4j in a gradle project

Minhaz — Fri, 18 Oct 2024 20:27:21 +0000

The motivation here is that I've spent just too much time trying to figure out how to add logging with log4j in spring boot in gradle. Here the gradle part was the one giving me the most grief because all the documentations I could find was for maven.

So I'm documenting my findings here so that next person who is looking for this might find my post and will be able to figure it out quickly.

This is the configuration I'm using for the demo. I've tested the same implementation in java 17 as well.

Add this line to the `build.gradle`

configurations {
    all*.exclude module : 'spring-boot-starter-logging'
}

dependencies {
    ...
    implementation "org.springframework.boot:spring-boot-starter-log4j2"
}

Create a file named `log4j2.xml` in the `src/main/resources` folder and add this to the file


<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - Message: %msg%n"/>
        </Console>
    </Appenders>
    <Loggers>
        <Root level="trace">
            <AppenderRef ref="console"/>
        </Root>
    </Loggers>
</Configuration>

Your setup is done, now you can add logger from log4j and it will use this xml configuration. Notice that I've added a Message: in every message. So if you don't want it, you can just remove it. I've added it there so that I can test whether or not the project was using this xml file.

Now to test you can just use the log4j logger. You can add it in the `main` method.

// import org.apache.logging.log4j.LogManager;
// import org.apache.logging.log4j.Logger;

Logger logger = LogManager.getLogger(SpringApplication.class);
logger.info("Hello World!");

The logger setup should work as expected. Notice the Message: part.

You can find the project here

References

Reset password in uptime kuma

Minhaz — Thu, 17 Oct 2024 19:14:10 +0000

The purpose of this is to reset the password of an uptime kuma installation. The motivation behind this is that I forget my password for the uptime kuma very frequently. So I need to reset them every 4-6 months (⓿_⓿).

So I'm just documenting the flow here for anyone that needs it.

if you are running the uptime kuma in a docker, then at first go inside the container

docker exec -it uptime-kuma bash

Install sqlite in the container

apt install sqlite

Locate the `kuma.db` file and open it in sqlite, for my case it is in the `/app/data`

sqlite3 /app/data/kuma.db

You can list all the users using this query and note the id

SELECT * FROM user;

Run this command to update the password

UPDATE user SET password="$2y$10$IWoZl5q9Tvvp1wxROvi4hOul7XP.rfyrvm4xbm7ufVANke1nfvLIu" WHERE id=1;

If you want to set a specific password then just go to this website and generate the bcrypt password. Then put it inside the sql password="<encrypted_password>".

Now you can just log in to your user account using the password `password`, don't forget to change the password after you log in.

Thanks for reading, let me know if you face any issue (it can be any webdev/devops related issue) and a star on the repo will be much appreciated :)

Reference

How to enable mongodb query logging in reactive java for quarkus with panache

Minhaz — Sun, 07 Jul 2024 19:51:55 +0000

Keeping it here so that anyone after me can find these configs easily.

At my job I've been writing a microservice with java with reactive style. I'm also using mongodb with panache. Recently I had a problem where one of my queries were working perfectly from mongo compass but with panache it wasn't working as expected. So I needed to see the query that panache was generating.

The quarkus's official document provides a way to log mongodb query in output. Unfortunately it only works for non reactive java and the guide doesn't properly mention this.



quarkus.log.category."io.quarkus.mongodb.panache.common.runtime".level=DEBUG

At first I thought for some reason maybe my project configuration was wrong or maybe configs were conflicting and that's why it wasn't showing the logs. I never thought what reactive coding was the problem. So I debugged the issue in other ways and found the proper solution for my problem.

After that I got somewhat bothered and kept asking myself why the query logging wasn't working and got frustrated. Then in an eureka moment I looked at the configuration key carefully and I can clearly see that the log was enabling a category io.quarkus.mongodb.panache.common.runtime so I went looking for that package. From my jetbrains IDE I found it and I can clearly see that there is a common.runtime.CommonPanacheQueryImpl and another common.reactive.runtime.CommonReactivePanacheQueryImpl.

So I tried to filter by that category and it worked :)
So the solution is to just enable DEBUG on this category. So add this line to your application.properties.



quarkus.log.category."io.quarkus.mongodb.panache.common.reactive.runtime".level=DEBUG

Now the logging is working

Bonus

While searching I've found another way to enable query logging, but it is logging from the mongodb driver. To enable it just add this to the application.properties



quarkus.log.category."org.mongodb.driver.protocol.command".level=DEBUG

The result looks like this, it also outputs the result from the db.

Just thought that I'd post these configs here as a reference to someone who is doing reactive java with panache. As I've invested one of my weekends figuring this out as well as other things. I'll probably make a longer post on my journey as a Senior Software Engineer to debug a panache capability issue and how a junior might navigate these and what were my step by step thought process during this.

I've learned all these because I kept asking myself "why wouldn't it work?" and it kept bothering me.

References

Quick guide to setting up an shortcut/alias in windows

Minhaz — Mon, 01 Jul 2024 18:06:54 +0000

This is a quick 30 second guide to setting up an alias in windows. All this is applicable for windows.

Motivation

The main motivation behind this is that I wanted to simplify the ping command. So that I'll just type a single command and it'll execute the ping www.google.com -t, which I use to check if my internet connection is stable or not.
The motivation behind this post is that I couldn't find a quick guide/overview to setup alias or terminal shortcuts when I searched google. So hopefully this will be helpful to juniors to get started.

Open up a powershell and type away.

Basic alias by creating simple shortcut.

If you want to just create an alias for a command the script is like this.
If I want to reference the command echo using a shortcut like ec the command is like this

Set-Alias -Name ec -Value echo

Now the ec command will refer to the echo command, and the echo command is there as well, it didn't go anywhere. To try and see if this works you can try this

ec "Hello World"

Little bit extended - call simple commands with shortcut

To create the final alias, I'll have to use a powershell function like this.

Function PingGoogle { ping www.google.com -t }

Here the function's name is PingGoogle and it executes the ping www.google.com -t command. We can try it by just entering the PingGoogle command after pasting the method.

Now we'll have to make an alias to this command. Like this

Set-Alias -Name pingg -Value PingGoogle

Here the pingg alias is referring to the PingGoogle function. To test this we can just use pingg command now.

Bonus - How to persist these changes.

You'll notice that if you close your terminal then the changes go away, your shortcut won't work.
So to persist these changes across all terminals you'll have to save them in your terminal profile.

To do that type this

echo $profile

This will show you the location of your powershell profile file. Go to that location and open that file.
Or open that file with this

notepad $profile

A file will open in notepad.

Now paste your code in that notepad and save in the same location. Now what will happen is that every time your terminal runs it will also load codes from this file. That's why your changes that your pasted here will persist between sessions.

Reference

How to know if you have the XZ vulnerability and what you can do about it

Minhaz — Tue, 02 Apr 2024 14:12:20 +0000

If you have been on the internet recently you should already know what happened with the XZ vulnerability. Basically this vulnerability was introduced very recently and not all distros are affected by it.

Only the versions 5.6.0 and 5.6.1 is affected. So if you aren't using these and you haven't upgraded recently then you are safe.

If you are ubuntu user this shouldn't affect you as the most recent version of xz available for ubuntu is version 5.4.

Still I'll document here how to detect and downgrade specific packages if something like this occur.

So the package that was the problem is the xz-utils pacakge.

At first we'll have to which version we are using.

We can do it using the following commands

xz --version

apt-cache showpkg xz-utils

apt-cache policy xz-utils

If the currently used version is 5.6.0 or 5.6.1 we'll have to downgrade.

To downgrade first we'll need to know which versions are available.

We can do that using

apt search xz-utils

Now when we've selected a version to downgrade to we can downgrade using this

sudo apt-get install xz-utils=<version>

A sample of this is like this

sudo apt-get install xz-utils=5.2.5-2ubuntu1

Although not recommended, but if you want to hold automatic upgrade for this package you can do it like this.

sudo apt-mark hold xz-utils

or to revert the hold you can use this

sudo apt-mark unhold xz-utils

You can follow this for more specific instructions for other distros.

Instantly preview rendered liquid template

Minhaz — Wed, 20 Mar 2024 17:16:25 +0000

Liquid is a template language created by shopify. In my use case I use it for generate html that is almost similar looking but differs in data. So when iterating over my HTML, I need to preview the changes I made combined with my data.

I achieve this using 2 VS code extensions.

One is Shopify Liquid Preview this enables us to connect our data with the template file.
Another one is HTML Preview this will help us preview data connected html file inside vs code.

Install these 2 extensions.

So at first create a html file. Mine is named index.html

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Preview Liquid Template Instantly in vscode</title>
</head>

<body>
    <div style="width:100%; text-align: center;">
        <h1>Greetings from {{name}}</h1>
        <h2>{{post.title}}</h2>
        <h3>{{post.description}}</h3>
        {%- if conditional.enabled -%}
        <p>This section is only visible when conditional is enabled</p>
        {%- endif -%}
    </div>
</body>

</html>

Next create a json file that contains your data that will be connected with the HTML file. Mine is named data.json

{
  "name": "Minhazul Hayat Khan",
  "post": {
    "title": "Preview Liquid Template instantly in vscode",
    "description": "This uses 2 vscode extensions to directly preview the liquid template with data connected"
  },
  "conditional": {
    "enabled": true
  }
}

Now go to your HTML file and press "CTRL + SHIFT + P" and select the "Shopify Liquid: Open Preview to the side" option.

After selecting it, select the json file you'd like to connect with the html

If everything is done properly it will generate a Preview with the connected data to the side.

Now click on the Preview html file and press CTRL + SHIFT + p again and select the HTML: Open Preview option.

It will show you the HTML's preview.

I had to struggle with checking and verifying the template changes for a very long time. I hope this helps someone. All the codes shown here can be found in my repo here.

Reference

https://github.com/minhaz1217/devops-notes/tree/master/70.%20instantly%20preview%20rendered%20liquid%20template

Send fluent bit collected logs to Seq

Minhaz — Tue, 02 Jan 2024 20:26:23 +0000

Here I will setup fluent-bit and seq both with docker and push logs through fluent bit to to seq.
I will also show how to setup fluent bit and seq with docker.

Setup a docker network for the containers

docker network create fluent-bit_seq

Setting up Seq

At first setup a hashed password to be used

PH=$(echo 'seqPass%%' | docker run --rm -i datalust/seq config hash)

Make sure that the password variable is ok

echo $PH

Run the container

docker run --name seq -d --network fluent-bit_seq \
    -p8080:80 --restart unless-stopped \
    -e ACCEPT_EULA=Y -e SEQ_FIRSTRUN_ADMINPASSWORDHASH="$PH" \
    datalust/seq

Now browse to localhost:8080 and login into Seq using username=admin password=seqPass%%

Setting up Fluent Bit

Designate a folder where you'll store the fluent bit configs. In my case the directory is D:/fluent-bit

Set the directory where fluent bit's config will come from

export sharedFolder=/var/fluent-bit_seq

Start a temporary container to copy default configs from

docker run -d --rm --name temp cr.fluentbit.io/fluent/fluent-bit

Copy the configs to your designated folder

docker cp temp:/fluent-bit/etc/ $sharedFolder

Stop the temporary container

docker stop temp

Now start fluent bit with the config folder mounted as a volume

docker run -dti --name fluent-bit --network fluent-bit_seq \
    -v $sharedFolder:/fluent-bit/etc \
      cr.fluentbit.io/fluent/fluent-bit

By default fluent bit is configured to output to std out. So see docker log to see what fluent bit is logging.

docker logs fluent-bit

Configuring Fluent Bit to send logs to Seq

Go to the fluent bit configuration directory and search for this section

# fluent-bit.conf
[OUTPUT]
    name  stdout
    match *

Replace this section with this and save

# fluent-bit.conf
[OUTPUT]
    Name             http
    Match            *
    Host             seq
    Port             5341
    URI              /api/events/raw?clef
    Format           json_lines
    Json_date_key    @t
    Json_date_format iso8601
    Log_response_payload False

Now restart the fluent bit container for the changes to take effect.

docker restart fluent-bit

Now browse to localhost:8080 and login into Seq using username=admin password=seqPass%% and see that the logs are being written into Seq

Thanks for reading, Happy Tinkering :)

References

Github repo

Setting up fluent bit using docker

Minhaz — Sat, 30 Dec 2023 00:30:52 +0000

Purpose

This is a step by step guide about how to setup fluent bit with docker.

The fluent bit official installation guide only had documentation about how to run fluent bit using docker but it didn't mention how to set it up.

So this is my solution to how to setup fluent bit using docker commands only. No docker compose.

Steps

Designate a folder where you'll store the fluent bit configs. In my case the directory is `D:/fluent-bit`

Start a temporary container to copy default configs from

docker run -d --rm --name temp cr.fluentbit.io/fluent/fluent-bit

Copy the configs to your designated folder

docker cp temp:/fluent-bit/etc/ D:/fluent-bit

Stop the temporary container

docker stop temp

Now start fluent bit with the config folder mounted as a volume

docker run -dti --name fluent-bit -v D:/fluent-bit:/fluent-bit/etc cr.fluentbit.io/fluent/fluent-bit

By default fluent bit is configured to output to std out. So see docker log to see what fluent bit is logging.

docker logs fluent-bit

You'll see something like this

Now to make sure that fluent bit is indeed taking configs from our folder. Go to the designated folder and open the fluent-bit.conf folder

Look for this string

tag cpu.local

Change it to this

tag CPU_LOCAL

Save the file and restart the container

docker restart fluent-bit

Now look at docker logs

docker logs fluent-bit

Notice that the logs are different.

References

MongoDB giving user permission to view only single collection.

Minhaz — Sun, 24 Dec 2023 21:33:50 +0000

Purpose

Here I will install mongo and set it up in a way where an user can only read and write a single collection or multiple collection but not all.

As a result it will increase security of the db.

Steps

Install mongodb

docker run -dit --name mongo -e MONGO_INITDB_ROOT_USERNAME=admin -e MONGO_INITDB_ROOT_PASSWORD=mongoAdmin33 mongo

Go into the docker container's mongosh

docker exec -it mongo bash -c "mongosh mongodb://admin:mongoAdmin33@localhost:27017/?authSource=admin"

Use this command and see that all the dbs are visible

show dbs;

It should show 3 collections like this -

Switch to a db using

use appleCollection;

Insert some data

db.apples.insertMany([
    {
        title: "Green Apple"
    },
    {
        title: "Red Apple"
    },
    {
        title: "Black Apple"
    }
])

Create an user and give him read write access for only this collection

db.createUser(
   {
     user: "apple_admin",
     pwd: "apple_admin_pass",
     roles: [ "readWrite", "dbAdmin" ]
   }
)

It should say

Exit from the current mongosh using

exit

Login into mongosh using the new apple admin account

docker exec -it mongo bash -c "mongosh mongodb://apple_admin:apple_admin_pass@localhost:27017/?authSource=appleCollection"

Please notice that we are using appleCollection as the auth source.

Now use the command to see dbs;

show dbs;

Only one db should be visible.

Select data from this db.

db.apples.find({});

Switch to a different database

use randomDatabase;

Query this database and it should show error.

show collections

Reference

This was one of my projects that I tried out first here. Here you can also find how to give a user multiple database’s permission.
devops-notes/61. setup mongodb and give specific user permission to specific collection. I will update the github if I find something.

[Github]Giving user permission to specific database

Enjoy and let me know if any problem occurs.

DEV Community: Minhaz

Java Spring Boot use Seq for logging

Steps

I'm using this configuration for the project

Add these to build.gradle for dependency

Now just configure the logger

Reference

Java Spring Boot use log4j in a gradle project

This is the configuration I'm using for the demo. I've tested the same implementation in java 17 as well.

Add this line to the build.gradle

Create a file named log4j2.xml in the src/main/resources folder and add this to the file

Now to test you can just use the log4j logger. You can add it in the main method.

References

Reset password in uptime kuma

if you are running the uptime kuma in a docker, then at first go inside the container

Install sqlite in the container

Locate the kuma.db file and open it in sqlite, for my case it is in the /app/data

You can list all the users using this query and note the id

Run this command to update the password

Now you can just log in to your user account using the password password, don't forget to change the password after you log in.

Reference

How to enable mongodb query logging in reactive java for quarkus with panache

Bonus

References

Quick guide to setting up an shortcut/alias in windows

Motivation

Basic alias by creating simple shortcut.

Little bit extended - call simple commands with shortcut

Bonus - How to persist these changes.

Reference

How to know if you have the XZ vulnerability and what you can do about it

Instantly preview rendered liquid template

Reference

Send fluent bit collected logs to Seq

Setup a docker network for the containers

Setting up Seq

Setting up Fluent Bit

Configuring Fluent Bit to send logs to Seq

Thanks for reading, Happy Tinkering :)

References

Setting up fluent bit using docker

Purpose

Steps

Designate a folder where you'll store the fluent bit configs. In my case the directory is D:/fluent-bit

Start a temporary container to copy default configs from

Copy the configs to your designated folder

Stop the temporary container

Now start fluent bit with the config folder mounted as a volume

By default fluent bit is configured to output to std out. So see docker log to see what fluent bit is logging.

Now to make sure that fluent bit is indeed taking configs from our folder. Go to the designated folder and open the fluent-bit.conf folder

Look for this string

Change it to this

Save the file and restart the container

Now look at docker logs

References

MongoDB giving user permission to view only single collection.

Purpose

Steps

Install mongodb

Go into the docker container's mongosh

Use this command and see that all the dbs are visible

Switch to a db using

Insert some data

Create an user and give him read write access for only this collection

Exit from the current mongosh using

Login into mongosh using the new apple admin account

Now use the command to see dbs;

Select data from this db.

Switch to a different database

Query this database and it should show error.

Reference

Add these to `build.gradle` for dependency

Add this line to the `build.gradle`

Create a file named `log4j2.xml` in the `src/main/resources` folder and add this to the file

Now to test you can just use the log4j logger. You can add it in the `main` method.

Locate the `kuma.db` file and open it in sqlite, for my case it is in the `/app/data`

Now you can just log in to your user account using the password `password`, don't forget to change the password after you log in.

Designate a folder where you'll store the fluent bit configs. In my case the directory is `D:/fluent-bit`