DEV Community: Kathir

Terraform

Kathir — Thu, 16 Jul 2026 16:13:54 +0000

Terraform is an infrastructure as code tool that lets you build, change, and version cloud and on-prem resources safely and efficiently.

Infrastructure as Code

Instead of describing infrastructure in a document like

Server: 4CPUs, Memory: 8GB, Ubuntu, Firewall: Port 80,

We write it as code:

resource "google_compute_instance" "web" {
  machine_type = "e2-standard-2"

  boot_disk {
    initialize_params {
      image = "ubuntu-2204-lts"
    }
  }
}

Terraform versions the infrastructure as Code (Terraform configuration files) that define the cloud or on-premises resources. By storing these files in Git, we can track, review, compare and revert infrastructure changes just like application code.

Terraform doesn't version the infrastructure like VM itself. It versions the code that defines the infrastructure. That lets you track, review and roll back infrastructure changes using a version control system like Git.

Terraform State File

The Terraform state file (terraform.tfstate) is a file that stores Terraform's knowledge of the infrastructure it has created and manages.

Without state file, Terraform would forget everything it had created.

It contains:

Resources

google_compute_instance.web

Name

web-server

ID

123456789

Zone

asia-sout1-a

Machine Type

e2-medium

Status

Running

ID = Also knowns as Resource Id used for updation even if the machine type changes in the VM. Resource Id would be constant so we can change the correct resource by looking the Id.

Where is state file stored?

Local state: It lives on computer.
Remote state (in companies): Stored remotely such as Google Cloud Storage, AWS S3, Azure Storage.

Remote storage allows everyone on the team to work with the same infrastructure safely.

Without state file: Duplicate resource may be created since it didn't know anything about current state.

Even though state file extension is .tfstate, its contents are in JSON.

Terraform Workflow

1. Write Terraform code

Create configuration files.

In this phase, nothing has been created.

2. Format the code

terraform fmt

Terraform automatically:

fixes indentation
aligns equal signs
improves readability

3. Validate

terraform validate

Terraform checks:

Syntax
missing arguments
invalid references

4. Initialize

terraform init

Terraform init helps to downloads everything needed or the project.

It downloads provider plugin, initializes the backend and modules and create .terraform directory.

It is like installing dependencies from the internet.

Eg:

provider "google" {}

Terraform downloads the Google Cloud provider plugin.

Without it, terraform doesn't know how to talk to Google Cloud APIs.

5. Plan

It is the terraform's decision making step.

It compares current and desired state.

Internally, it performs

Terraform code
      ↓
Read state file
      ↓
Ask Google cloud
      ↓
Compare
      ↓
Generate Execution plan

HashiCorp calls this an execution plan, which previews the infrastructure changes before anything modified.

6. Review

User's team reviews the plan.

It is one of the terraform's biggest safety features.

7. Apply

terraform apply

Terraform reads the plan, calls google cloud apis, Creates resource, waits until they are ready, writes the stae file.

8. Update State

After a successful apply:

terraform.tfstate is updated.

9. Destroy

If the environment is no longer needed. Then use

terraform destroy

.terraform/ Directory

.terraform/ directory is like node_modules where the downloaded dependencies and all plugin, modules, metadata will be stored.

.terraform.lock.hcl

.terraform.lock.hcl is Terraform's dependency lock file. It stores the provider name, the exact provider version, and checksums (hashes) of the provider plugins.

During future terraform init runs, Terraform uses this file to download the same provider version and verifies the downloaded plugin against the stored checksums.

This ensures everyone on the team and CI/CD pipelines use identical provider versions, preventing unexpected behavior, compatibility issues, and bugs caused by different provider versions.

Terraform Configuration Files

main.tf

This is the main configuration file.

It contains the infrastructure wants terraform to create.

variables.tf

Instead of hardcoding values, we declare variable.

What values are changeable?

terraform.tfvars

This file provides values for the variables.

It provides answers for the variables.tf.

outputs.tf

Used to display useful information after deployment.

providers.tf

Specifies which cloud provider terraform should use.

Microsoft SQL Server and its Components

Kathir — Tue, 14 Jul 2026 15:17:16 +0000

SQL Server Instance

An SQL Server instance is an independent SQL Server environment running on a machine. Multiple instances can coexist on the same server, each with its own databases, users, configuration, and services, allowing different applications or environments to be isolated while sharing the same hardware.

Microsoft SQL Server

Microsoft SQL Server is a Relational Database Management System (RDBMS).

Applications and Tools → Connect to a SQL Server Instance → Connect to a Database → Communicate using Transact-SQL (T-SQL)

The application connects to a SQL Server instance. It selects a database within that instance. It sends T-SQL commands (queries and updates). SQL Server executes those commands and returns the results to the application.

SQL Server is a complete database product. It includes:

Database Engine
SQL Server Agent
Integration Services (SSIS)
Analysis Services (SSAS)
Reporting Services (SSRS)
Full-Text Search

The Database Engine is a component of SQL Server. It runs as a SQL Server service on the operating system.

Microsoft uses the same Database Engine technology in several products and platforms, including:

SQL Server (installed on your own machine or server)
Azure SQL Database (managed cloud database)
Azure SQL Managed Instance (cloud service with high SQL Server compatibility)
Microsoft Fabric SQL Database (SQL capabilities within Microsoft Fabric)

SQL Server Components

1. Database Engine

The Database Engine is the core service of Microsoft SQL Server. It is the component that actually stores, retrieves, updates, and secures the data.

A Database Engine is the software that manages data stored on disk.

What does the Database Engine do?

When an application sends a T-SQL query, the Database Engine:

Receives the query.
Checks whether the user has permission.
Finds the best way to execute the query.
Reads or modifies the data.
Returns the results.

Responsibilities

Stores data
Executes T-SQL queries
Manages transactions
Provides security
Performs performance optimization (efficient way to execute a query)
Backup and recovery

Main Internal Components of the Database Engine

1. Query Processor

Parses and executes T-SQL queries.
Creates execution plans.

2. Storage Engine

Reads and writes data to disk.
Manages pages, indexes, and files.

3. Transaction Manager

Ensures data consistency using ACID properties.

4. Security Manager

Handles authentication and authorization.

2. Machine Learning Services

Machine Learning Services is a feature in SQL Server that gives the ability to run Python and R scripts with relational data.

It can be used with open-source packages and frameworks, as well as Microsoft Python and R packages, for predictive analytics and machine learning.

The scripts are executed on data without moving the data outside SQL Server or over the network.

Note

The Java Language Extension is a feature of SQL Server used for executing external Java code.

The default runtime used in it is Zulu OpenJRE, and it can also be changed.

3. SQL Server Integration Services (SSIS)

SQL Server Integration Services (SSIS) is one of the major components of Microsoft SQL Server.

Its purpose is moving, transforming, and integrating data between sources.

If you have data in different places like:

Excel files
CSV files
MySQL databases
Oracle databases
SQL Server databases
APIs

SSIS moves the data from these different sources into SQL Server.

SSIS follows the ETL Process

Extract

Extract data from different sources.

Transform

Modify the data.

Examples:

Remove duplicate rows
Convert text to uppercase
Change date formats
Calculate totals
Replace NULL values

Load

Store the processed data into the SQL Server database.

SSIS does not store data. Its job is to move and transform data.

SSIS can do

Import Excel and CSV files
Copy data between databases
Merge multiple data sources
Clean and validate data
Schedule data imports
Automate repetitive data movements (moving data from one location to another)

4. SQL Server Analysis Services (SSAS)

SQL Server Analysis Services (SSAS) is a component of Microsoft SQL Server used for analyzing large amounts of data and helping users make business decisions.

It is designed for analytics.

Types of Analysis in SSAS

SSAS supports two main models.

1. Multidimensional Model (Cube)

It is called a Cube, which simply means data is organized along multiple dimensions.

Dimensions

Something you want to analyze.

Examples:

Product
Customer
Year

Measures

Measures are numbers that you calculate.

Examples:

Sales
Profit
Quantity

Based on dimensions and measures, SSAS precomputes all the values so that results can be returned without waiting time.

2. Tabular Model

Instead of keeping many pre-made summaries, the model organizes all records into highly optimized columns and performs calculations very quickly whenever they are requested.

It is currently the most commonly used model since it uses DAX, which is also used by Power BI.

SSAS is an analytic engine. It creates analytical models.

5. SQL Server Reporting Services (SSRS)

SQL Server Reporting Services (SSRS) is a component of Microsoft SQL Server used to create, manage, and deliver reports.

SSRS can export reports as:

PDF
Excel
Word
CSV
XML
HTML (Web Page)

It can create:

Bar charts
Pie charts
Line charts
Gauge charts
Maps

It also allows interactive reports such as:

Search
Filter
Sort
Drill down into details

SSRS runs queries, retrieves data, formats it, and displays or exports it.

6. SQL Server Replication Service

Replication means copying and synchronizing data between databases.

It is the process of copying data from one database to another and keeping the copies updated.

Types of Replication in SQL Server

1. Snapshot Replication

Copies all the data at once at fixed intervals.
Example: Once a week or once a month.
Every snapshot copies the entire database again.

Best used when:

The database is small.
Data does not change frequently.

Not recommended for large databases.

2. Transactional Replication

Transactional Replication sends only the changes made to the database instead of copying the entire database.

Whenever data is inserted, updated, or deleted, those changes are quickly sent to the subscribers.

Best for:

Large databases
Frequently changing data
Real-time reporting
High-speed synchronization

Disadvantage

One-way updates only (Publisher → Subscriber).

3. Merge Replication

In Merge Replication, both databases can make changes.

SQL Server combines (merges) the changes.

If both databases update the same record differently, a conflict occurs, and SQL Server resolves it using conflict resolution rules.

Best for:

Mobile applications
Salespeople working offline
Branch offices
Point-of-Sale (POS) systems

SQL Server Replication Components

Replication involves three main roles.

1. Publisher

The main database that owns the original data.

2. Distributor

Stores replication information and delivers changes.

3. Subscriber

Receives the copied data.

Terminated Services in SQL Server 2025 (Available in SQL Server 2022)

7. Data Quality Services (DQS)

Used for cleaning and standardizing data.

8. Master Data Services (MDS)

Used for managing master data.

Master Data is data that is used by multiple databases.

Examples:

Company list
Product list

Reason for Termination

Low adoption compared to other SQL Server services like the Database Engine.
Shift toward Microsoft Fabric and Azure.
Better cloud solutions are available.
AI has changed data quality processes.
High maintenance cost.

Conclusion

Microsoft SQL Server is much more than just a database. Along with the Database Engine, it provides powerful services for data integration (SSIS), analytics (SSAS), reporting (SSRS), machine learning, and replication. Understanding these components helps you build, manage, analyze, and maintain enterprise-level database solutions more effectively.

As Microsoft continues shifting its focus toward Azure and Microsoft Fabric, some legacy services such as Data Quality Services (DQS) and Master Data Services (MDS) are being retired, while the core Database Engine remains the foundation across Microsoft's SQL platforms.

Project Object Model

Kathir — Fri, 10 Jul 2026 17:29:18 +0000

Yesterday, we went through Maven. Now, we are going to learn about the Project Object Model (POM).

POM

Project object model is the fundamental unit of work in Maven. It is an XML file that contains information about the project and configuration details used by Maven to build the project.

Super POM

Super POM is the default POM that maven provides internally. Every maven project inherits from the Super pom even if there is no mentioning.

pom.xml doesn't start from scratch. Maven first loads the Super POM and then merges the pom.xml with it.

Super POM contains some defaults:

Central Repository= Even if the pom.xml doesn't mention a specific repository, maven downloads dependencies from maven central because super POM includes it.
Default Build directory=> Super POM tells that the target is where build should be present.
Plugin Reposity=> The super pom tells Maven where to download plugins such as surefire plugin, compiler plugin, JAR plugin.

Effective POM

Super POM +pom.xml is what Maven actually uses during the build.

Minimal POM

The minimum requirement for a POM:

project (root)
modelVersion= should be set to 4.0.0 which is the version of POM XML model
groupId
artifactId
version = which is the project version

Project Object Model

When Maven reads pom.xml, it converts pom.xml into a Java Object internally.

<project xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.mycompany.app</groupId>
    <artifactId>my-app</artifactId>
    <version>1</version>
</project>

Internally maven creates something like this

Project project=new Project();
project.setModelVersion("4.0.0");
project.setGroupId("com.mycompany.app");
project.setArtifactId("my-app");
project.setVersion("1.0");

The structure of the Java object of maven conversion is called the POM model.

A POM requires groupId, artifactId, and version and these helps to form the project's fully qualified artifact name. It is in the form of:

<groupId>:<artifactId>:<version>

Artifact name example:

com.mycompany.app:my-app:1

POM helps to configure details by using defaults if they are not specified.

Example:

Every maven project has packaging type. If it is not specified in the POM then the default value be "jar".

Project Inheritance

It means that one project (child) can inherit configuration from another project (parent). This helps avoid repeating the same configurationa cross multiple projects.

Project inheritance can be done using super pom and parent pom.

Parent POM

It is created with the help xml tag <parent></parent>

<project xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>com.mycompany.app</groupId>
        <artifactId>my-app</artifactId>
        <version>1</version>
        <relativePath>../parent/pom.xl>/relativePath>
    </parent>
    <artifactId>my-module</artifactId>
</project>

Elements in the POM that are merged are the following:

dependencies
developers adn contributors
plugin lists
plugin execustion with matching ids
plugin configuration
resources

Super POM cannot be edited but can be override in pom.xml file.

Project Aggregation

Project Aggregation is similar to Project Inheritance. But instead of specifying the parent POM from the module, it specifies modules from parent POM. By, doing it the parent project knows its modules. If a mavent command is invoked against parent project, that Maven command will then be executed to the paren'ts modules as well.

It is mainly used in microservices. Since, it contains many services like web,common,api,services as projects. It is hard to run each one separately like mvn install. That is hard to maintain so we create a common parent and invoke all of it.

Parent pom.xml

<modules>
    <module>library-common</module>
    <module>library-api</module>
    <module>library-service</module>
    <module>library-web</module>
<modules>

This tells maven that these are the project that belong together.

Project Model Variables

Maven allows you to reference any single-value filed from the project's POM using variables.

Syntax:

${project.properyName}

Example:

${project.groupId}

Special Maven Variables

1. project.basedir

Represents the directory where the project is located.

2. project.baseUri

Same as project.basedir but returned as a URI (Uniform Resource Identifier= It is a string that identifes a resouce.

Syntax:

schem://path

Example:

file:/C:/projects/MyApp

3. maven.build.timestamp

Represents the UTC timestamp whent he maven build starts.

Default output timestamp

yyyy-mm-dd'T'HH:mm:ss'Z'

We can customize the formal by using properties.

<properties>
    <maven.build.timestamp.format>
        yyyy-MM-dd'T'HH:mm:ss'Z'
    </maven.build.timestamp.format>
<properties>

Maven Properties

Defining own reusable variable can done via <properties> section.

Syntax:

<properties>
    <propertyName>value</properyName>
</properties>

We can use it as

${propertyName}

UTC memory tip

M=Month
m=Minute
H=24-hours
h=12-hours
D=Day of year
d=Day of month
S=Smallest visible unit in UTC:Millisecond
s=second
Z=Numeric zone offset(+0530)
z=Zone name (IST,UTC)

Patterns:

MMM= Jul
MMMM=July
yyyy=2026
yy=26
hh:mm:ss a=02:35:20 PM

T=> Separator between date and time (ISO 8601 format)

Why use UTC (Coordinated Universal Time)?

It provides a common time reference so sytems in different countries record the same moment consistenly.

POM non-default fields

1. `<name>`

This is the human-readable name of the project.

Used in Generated documentation, IDE's display, Reports.

2. `<url>`

Project homepage or repository link. Only used for documentation so it can be any link.

3. Properties

This element contains value placeholder accessible anywhere within a POM.

4. dependencies

This element's childrent list dependencies.

5. build

This element hanles things like declaring your project's directory structure and managing plugins.

Spring Boot, Spring Framework, Java, and Maven – Beginner Notes

Kathir — Thu, 09 Jul 2026 16:48:32 +0000

Spring Boot Overview

Spring Boot is maintained by VMware.

Before understanding Spring Boot, it helps to know how Java evolved.

Java (Java SE) = The programming language and the basic platform.
Java EE = Java along with Enterprise APIs.
Jakarta EE = The modern continuation of Java EE.

The Spring Framework is the core framework, whereas Spring Boot is an opinionated layer built on top of the Spring Framework. It automatically configures many components, manages dependencies, provides embedded servers, and makes it much easier to build and run Spring applications.

Spring itself is built on top of Java and frequently relies on Jakarta EE specifications internally.

Internally, Spring Boot uses:

jakarta.servlet for handling HTTP requests.
jakarta.persistence for JPA entities.
jakarta.validation for Bean Validation.

Rather than requiring developers to work directly with Jakarta APIs, Spring provides its own programming model using annotations such as:

@RestController
@Autowired

It also offers Spring Data JPA while internally using Jakarta technologies.

What is Maven?

Maven is primarily a project management and build automation tool. It helps manage:

Builds
Dependencies
Documentation
Software Configuration Management (SCM)
Releases
Reporting
Distribution

Maven improves the build process by following standard conventions and best practices, helping teams develop faster while increasing the chances of successful builds.

Artifact and Resolution

Artifact

In Maven, an artifact is anything that Maven either produces or downloads.

Resolution

Resolution refers to the process of locating and downloading project dependencies.

Maven Configuration Levels

Maven configuration exists at three different levels.

Project Level

Most of the static configuration is stored inside the pom.xml file.

Static configuration means the settings are common for everyone working on the project.

Installation Level

Configuration that is applied once for a Maven installation.

User Level

Configuration specific to an individual user.

Configuration file:

${user.home}/.m2/settings.xml

Maven Repositories

Maven works with three types of repositories.

Local Repository (stored on your computer)
Central Repository (available on the Internet)
Remote Repository (maintained by a company or organization)

A repository is simply a folder where Maven stores downloaded dependencies and generated artifacts.

The default Local Repository location is:

${user.home}/.m2/repository/

Proxy Server

Normally, when downloading a file, your computer sends the request directly to the website.

However, many companies do not allow employees to access the Internet directly. Instead, every request must pass through a proxy server.

A proxy server acts as a middleman between your computer and the Internet.

Companies use proxy servers for:

Security
Monitoring
Access Control
Caching

Maven Proxy Configuration

Sometimes Maven displays errors such as:

"Could not transfer artifact"
"Connection timed out"

These usually indicate that the company network uses a proxy server which blocks Maven from downloading dependencies.

To solve this, configure the proxy in Maven.

Edit:

${user.home}/.m2/settings.xml

Inside this file, configure the proxy using fields like:

id
active
protocol
host (hostname)
port
username
password

Configure Parallel Artifact Resolution

By default, Maven downloads up to five artifacts (from different groups) simultaneously.

To change the thread pool size temporarily:

mvn -Dmaven.artifact.threads=1 verify

For a permanent configuration, use the MAVEN_OPTS environment variable.

export MAVEN_OPTS=-Dmaven.artifact.threads=3

Maven Archetype

An Archetype is Maven's project templating toolkit.

A Maven Archetype is a predefined project template used to generate the initial structure of a Maven project.

Start the interactive wizard using:

mvn archetype:generate

The wizard asks for values such as:

Archetype
GroupId
ArtifactId
Version
Package Name

A non-interactive example:

mvn archetype:generate \
-DgroupId=com.example \
-DartifactId=my-app \
-DarchetypeArtifactId=maven-archetype-quickstart \
-DarchetypeGroupId=org.apache.maven.archetypes \
-DarchetypeVersion=1.5 \
-DinteractiveMode=false

Here, -D means Define a system property.

For example, -DgroupId=com.example creates the system property named groupId with the value com.example.

The generated project structure looks like:

my-app/
├── pom.xml
└── src
    ├── main
    │   └── java
    │       └── com/example/App.java
    └── test
        └── java
            └── com/example/AppTest.java

Common Maven Archetypes

Archetype	Purpose
`maven-archetype-quickstart`	Simple Java Application
`maven-archetype-webapp`	Java Web Application
`maven-archetype-simple`	Basic Maven Project

Create Your Own Maven Archetype

Convert an existing project into an Archetype using:

mvn archetype:create-from-project

Then install the generated Archetype.

cd target/generated-sources/archetype
mvn install

What is MVN?

mvn is not a Java class.

It is a shell script on Linux/macOS and a batch file (mvn.cmd) on Windows.

Its responsibility is to start the Maven Java application.

Launching Maven Archetype

Step 1 – Execute `mvn`

The mvn script (or mvn.cmd on Windows):

Finds the Java installation.
Sets the required environment variables.
Launches the Maven Java program.

At this point, one Java application starts: Maven itself.

Step 2 – Maven Starts

The JVM loads the Maven CLI class org.apache.maven.cli.MavenCli.

This is the main Java application responsible for Maven execution.

Step 3 – Maven Parses the Command

For example:

Plugin: archetype
Goal: generate

Step 4 – Maven Loads the Archetype Plugin

If the plugin is not already available in the local repository, Maven downloads it.

After downloading, Maven loads the plugin classes into the same JVM where the Maven CLI is already running.

Step 5 – Maven Calls the Plugin

The plugin receives all the supplied values and generates the project.

Flow of `mvn spring-boot:run`

mvn spring-boot:run
        │
        ▼
Shell Script (mvn)
        │
        ▼
Starts JVM #1
        │
        ▼
MavenCli
        │
        ▼
Reads pom.xml
        │
        ▼
Loads Spring Boot Maven Plugin
        │
        ▼
Compiles the Project
        │
        ▼
Starts JVM #2
        │
        ▼
DemoApplication.main()
        │
        ▼
SpringApplication.run()
        │
        ▼
Embedded Tomcat Starts

When running mvn spring-boot:run, two JVMs are usually involved.

Why Doesn't Maven Run the Application in the Same JVM?

Maven is a build tool, whereas the application is the software being built.

Keeping them separate provides several advantages.

If the application crashes, Maven continues running.
The application's classpath remains isolated from Maven's classpath.
The application can be stopped and restarted independently.

Commands and Number of JVMs

Command	JVMs
`mvn compile`	Usually 1 (Maven only)
`mvn archetype:generate`	1 (Maven + Archetype Plugin in the same JVM)
`mvn spring-boot:run`	Usually 2 (Maven + Spring Boot Application)
`mvn test`	Usually 2 (Maven + Test JVM)
`java -jar app.jar`	1 (Application only)

Where:

JVM #1 = Maven itself.
JVM #2 = Spring Boot application.

pom.xml

The pom.xml file contains the Project Object Model (POM) for a project.

The POM is the fundamental unit of Maven.

It is important to remember that Maven is project-centric, meaning everything revolves around the concept of a project.

The POM contains every important piece of information related to the project.

What's Next?

Tomorrow we will go through the complete details of pom.xml in depth.

Spring Boot Notes — Day 1 | My Learning Notes from the Spring Boot Documentation

Kathir — Wed, 08 Jul 2026 16:48:25 +0000

Note: I am currently going through the Spring Boot documentation and documenting my learning journey. These are my Day 1 notes. I have organized my notes into a blog format without changing the concepts or content.

What is Spring?

Spring is a Java framework. Provides features like Dependency Injection (IoC), MVC, Security, Data Access, REST APIs, Transactions, AOP (Aspect Oriented Programming).

Spring is very powerful but it requires lot of configurations like configure DispatcherServlet, Tomcat, XML, Beans, Database, Properties, Jackson, Logging, Context.

Developers spent more time configuring the configurations than coding. So, the spring team developed Spring Boot.

Spring = Powerful Engine

Spring Boot = Spring + Automation + Defaults

Boot doesn't replace Spring. It sits on tops of Spring.

Spring Boot doesn't create application for you. It is an assistant and you still write the code but removes repetitive works.

Stand-alone Application

With Spring Boot you can build stand-alone application.

Stand-alone application = Before Spring Boot, You had to install Tomcat, Jetty, WildFly, GlassFish and then deploy .war into the server.

You depended on external server but stand-along application do have everything it needs inside it.

Example:

myapp.jar

├── Application code
├── Spring
├── Tomcat
├── Libraries
├── Configurations
└── Dependencies

myapp.jar contains Application code, Spring, Tomcat, Libraries, Configurations, Dependencies.

Production Grade Application

It helps to build production grade application.

(Application suitable for running at real-time environment).

Production application need:

Security
Monitoring
Logging
Error Handling
Health Checks
Metrics
External Configuration
Performance
Scalability

Logging

Logging- It means recording what your application is doing from start to end.

These recording messages is called logs.

Without logging, You wouldn't know if something went wrong.

Logback is the default logging implementation used by Spring Boot since it is fast, reliable, easy configuration and well integrated with SLF4J.

Other logging framework:

Log4j

SLF4J

SLF4J = Simple Logging Facade For Java.

It is not a logging implementation but a common interface.

Code uses SLF4J and it forwards logging calls to the actual logging framework.

Database Connection Pool

Database Connection Pool = It is a collection of pre-created database connections that remain available while the application is running, so they can be reused instead of creating a new connection for every database request.

Some connection pool are:

Hikari
C3P0
DBCP

HikariCP

HikariCP = It is a high performance connection pool.

Spring Boot prefers it since it is very fast, low memory usage, excellent performance under heavy load and have simple configuration.

JSON

JSON stands for JavaScript Object Notation.

It is the standard format for exchanging data between systems.

JSON libraries:

Jackson

It converts Objects to JSON.
Json back to java objects automatically.

Gson

Created by Google.
Also converts objects to JSON.

JSON-B

JSON binding.

It is a java standard unlike jackson which not java standard api but a third party library.

Embedded Server

Embedded Server => It is a web server that is packaged inside your application and starts automatically when your application starts.

You don't have to manually install or configure a separate web server.

Spring Boot default uses Tomcat as an embedded server becuase it is mature, stable and widely used but you can replace it with:

Jetty

Known for beging light weight and embeddable.

Undertow

A high performance web server forcused on speed and low resource usage.

Metrics

Metric = Micrometer is the mertics library use by spring boot.

It collects information different parts of the application.

Spring Boot Defaults

Spring Boot uses Defaults but you can also change them by your configurations.

Spring Platform

It is a Spring Ecosystem.

The Spring platform is the broader colllection of Spring projects built around the Spring framework.

It includes:

Spring Framework (core)
Spring Boot
Spring Data
Spring Security
Spring cloud-microservices
Spring Batch- batch processing
Spring Integration-enterprise integration
Spring WebFlux-reactive web applications
Spring AOP
Spring JDBC
Spring ORM

Simple Analogy

Think of a smartphone:

Android OS = Spring Framework

Android ecosystem
(Playstore, Maps, Gmail, etc.)
        =
Spring Platform

Boot is built on spring framework and integrates the spring ecosystem.

Inegrates means spring boot makes different spring projects work together easy with little or no configurations.

GraalVM

GraalVM is a high performance Java Virtual Machine and development platform that can run Java and several other languages.

Its most popular feature is that it can compile java pplications into native executables (.exe/ Linux binary).

Problem with JVM

The JVM provides many features but it takes hundreds of milli seconds or even a few seconds for large appliations and memory usage is relatively high and a jdk or jre must be installed on the machine.

For long-running applications, JVM is best since startup time is not implicible.

GraalVM Native Image

Spring Boot applications can be converted into a Native Image.

native-image is a tool provided by Graal VM.

Converts jar to .exe.

Why GraalVM Native Image?

For serverless like aws lambda the application only starts when someone sends a request so graal vm natie image is highly recommended.

Suppose application takes 4 sec to start and the user waits for that 4 sec before the actual work even begins.

This is called cold start.

Memory also decrease for native image compared to spring boot application.

Microservices are best for GraalVM native image.

Build Native Image

Run:

mvn -Pnative native:compile

Instead of producting .jar file it produces:

target/
└── application.exe

Conclusion

Today I learned why Spring Boot was created, how it removes repetitive configurations, how stand-alone applications work, production-grade features, logging, connection pooling, JSON libraries, embedded servers, metrics, the Spring Platform, and how GraalVM Native Image helps reduce startup time and memory usage for serverless applications and microservices.

These are my Day 1 learning notes while going through the official Spring Boot documentation. In the upcoming days, I'll continue exploring more Spring Boot concepts in depth.

Day 2: Understanding MySQL Client-Server Communication, Protocols, and Pages

Kathir — Tue, 07 Jul 2026 15:55:20 +0000

📚 Table of Contents

Introduction
Why do we need Protocol?
What Happens When Java Connects?
Complete MySQL Request Flow
MySQL Server Components
- Connection Manager
- Authentication & Authorization
- SQL Parser
- Preprocessor
- Optimizer
- Executor
- Storage Engine
Pages in MySQL
Inside a Page
How Data is Organized
Why Pages Instead of Rows?
Pages Can Store More Than Rows
Conclusion

Introduction

Welcome to Day 2 of my MySQL learning journey.

In the previous blog, I learned about MySQL Storage Engines and MVCC.

Today, I learned how a Java application communicates with a MySQL server, why protocols are necessary, what happens internally when a connection is established, and how InnoDB stores data using pages.

In this blog, I'll be covering:

Why do we need Protocols?
What happens when Java connects to MySQL?
MySQL Request Flow
MySQL Server Components
What are Pages in MySQL?
Why does InnoDB use Pages?

Why do we need Protocol?

Protocols are simply a set of rules.

Imagine two people who speak two different languages. In this way, they can't understand each other. To be able to understand each other, they need a common language.

This problem takes place in computers also.

Java applications don't understand how MySQL stores data, and MySQL doesn't understand Java objects.

So both agree on one common language:

The MySQL Client/Server Protocol

This protocol defines:

How to connect
How to authenticate
How to send SQL queries
How to receive results
How to report errors
How to close the connection

Without this protocol, Java and MySQL could never communicate.

What Happens When Java Connects?

Consider the following Java code:

Connection conn = DriverManager.getConnection(
    "jdbc:mysql://192.168.1.10:3306/company",
    "root",
    "password"
);

Step 1

The JDBC Driver extracts:

IP Address: 192.168.1.10
Port: 3306
Database: company
Username: root

Step 2

The operating system creates a TCP Socket.

A socket is one endpoint of a network communication channel.

Step 3

TCP performs the Three-Way Handshake to establish a reliable connection.

Complete MySQL Request Flow

Java Application
        │
        ▼
JDBC Driver
        │
        ▼
MySQL Client/Server Protocol (Application Layer)
        │
        ▼
TCP (Reliable Transport)
        │
        ▼
IP (Routing)
        │
        ▼
Ethernet / Wi-Fi
        │
        ▼
Internet / LAN
        │
        ▼
MySQL Server Socket
        │
        ▼
Connection Manager
        │
        ▼
Authentication & Authorization
        │
        ▼
SQL Parser
        │
        ▼
Preprocessor
        │
        ▼
Optimizer
        │
        ▼
Executor
        │
        ▼
Storage Engine
        │
        ▼
Data Files on Disk

MySQL Server Components

Connection Manager

Responsibilities:

Manages client sessions
Manages client threads

Authentication & Authorization

Responsibilities:

Confirms user identity
Checks user permissions

SQL Parser

Responsibilities:

Validates SQL syntax
Creates a Parse Tree

Preprocessor

Responsibilities:

Resolves table names
Resolves column names
Checks object existence

Optimizer

Responsibilities:

Chooses an efficient execution plan
Decides whether to use indexes
Determines join order
Optimizes query execution

Executor

Responsibilities:

Executes the chosen execution plan

Storage Engine

Responsibilities:

Reads the actual data
Writes the actual data

Pages in MySQL

InnoDB does not store tables as one big continuous block.

Instead, it divides the table into Pages.

The table is simply a collection of many pages.

A Page is a fixed-size block of storage (typically 16 KB in InnoDB) that contains parts of a table, parts of an index, or other internal data.

Inside a Page

Suppose a row is about 100 bytes.

A 16 KB Page contains:

16 * 1024 = 16384 bytes

Number of rows per page:

16384 / 100 = 163 rows

So, one page can contain approximately 163 rows.

The next page contains the next set of rows.

A row is smaller than a page, and a page contains many rows.

How Data is Organized

Database
    │
    ▼
Tables
    │
    ▼
Pages
    │
    ▼
Rows

A database contains tables.

A table is stored as many pages.

Each page contains many rows.

Why Pages Instead of Rows?

Suppose you want to find a particular row with:

SELECT * FROM Employee WHERE id = 150;

Would the SSD read only Row 150?

No.

SSDs and HDDs read data in blocks, not individual rows.

So, MySQL reads the entire page, even if you only wanted one row.

Then it finds Row 150 inside that page.

Pages Can Store More Than Rows

A page doesn't always store table rows.

It can also store:

Indexes
Internal metadata
Other internal data structures

Conclusion

Today, I learned about pages in MySQL and how Java connectivity works with MySQL.

Thank you for reading!

This is Day 2 of my MySQL learning journey.

In the next blog, I'll continue exploring more internal concepts of MySQL as I progress through my learning journey.

Day 1: Understanding MySQL Storage Engines and MVCC

Kathir — Mon, 06 Jul 2026 16:05:49 +0000

📚 Table of Contents

Introduction
What is InnoDB?
What is a Storage Engine?
MySQL Server Architecture
MySQL Storage Engines
- InnoDB
- MyISAM
- MEMORY Engine
- CSV Engine
- ARCHIVE Engine
- NDB Cluster
NDB Cluster Architecture
Advantages of NDB Cluster
Disadvantages of NDB Cluster
Real World Use Cases
RAM + Disk Checkpoints
Example Hardware Requirement
Can SQL Node and Management Node be on the Same Machine?
MVCC (Multi-Version Concurrency Control)
Why is MVCC Needed?
Benefits of MVCC
How MySQL Implements MVCC
What Happens When a Row is Updated?
Hidden Columns in Every InnoDB Row
Read View
Conclusion

Introduction

When learning MySQL, one of the first concepts you should understand is the Storage Engine. A storage engine determines how MySQL stores, retrieves, and manages data. Different storage engines are designed for different workloads, such as transaction processing, read-heavy applications, temporary data storage, historical data, and distributed databases.

In this blog, I'll be covering:

What is a Storage Engine?
Different MySQL Storage Engines
InnoDB
MyISAM
MEMORY Engine
CSV Engine
ARCHIVE Engine
NDB Cluster
MVCC (Multi-Version Concurrency Control)

What is InnoDB?

InnoDB is a default storage engine in MySQL. Think of it as the component that decides how your data is stored on disk and how it is managed.

Without a storage engine, MySQL wouldn't know how to physically store or retrieve your data.

What is a Storage Engine?

A storage engine is the software component of a database management system responsible for how data is physically stored, retrieved, indexed and managed. It also controls features such as transactions, concurrency, crash recovery, and data access methods.

A storage engine is software inside MySQL responsible for:

Storing data on disk
Retrieving data
Managing indexes
Handling transactions
Recovering from crashes
Controlling concurrent access

Storage Engine = The internal component that stores and manages data.

Database = The complete software (MySQL, PostgreSQL, MongoDB, etc.)

Not every database supports multiple storage engines like MySQL does. Many databases have one built-in storage engine.

MySQL Server Architecture

MySQL Server
│
├── SQL Engine
│
└── Data Storage (Storage Engines)
     │
     ├── InnoDB
     ├── MyISAM
     ├── MEMORY
     ├── CSV
     ├── ARCHIVE
     └── NDB Cluster

MySQL Server = SQL Engine + Data Storage

MySQL Storage Engines

1. InnoDB

InnoDB =

Foreign keys
Crash recovery
Transactions
ACID
Row locking

Best for:

Banking
E-commerce
ERP
Inventory systems

InnoBase Development => Full form

2. MyISAM

MyISAM

Does not support Transactions
Does not support Foreign Keys
Supports only Table Locking (not Row Locking)
Limited Crash Recovery
Often faster for simple read-only workloads

Best for:

Read-heavy applications
Old web applications
Static databases with frequent SELECT queries but very few updates or deletes

It is lightweight.

Full form

MySQL Indexed Sequential Access Method engine, created by adding indexing and improvements for MySQL.

Unlike other engines, a MyISAM table is stored on your disk as three separate files:

.frm : Stores the table definition
.MYD : Stores the actual MY data
.MYI : Stores the MY index

It was the default storage engine for the MySQL relational database management system versions prior to 5.5 released in December 2009.

3. MEMORY Engine

Everything is stored in RAM.

Advantages

Extremely fast

Disadvantages

Data disappears when MySQL restarts.

Formerly known as Heap.

Creates special-purpose tables with contents that are stored in memory.

Because the data is vulnerable to crashes, hardware issues, or power outages, only use these tables as temporary work areas or read-only caches for data pulled from other tables.

Normally, MySQL stores tables on the hard disk, but for MEMORY the table is stored only in RAM.

MEMORY is used for:

Sessions
Cache
OTPs

Even if MySQL uses RAM it doesn't work like Redis.

MySQL has to read the entire query and has to follow the pipeline even if it is present in memory, but Redis is like a hash table which gives the output.

Even if session and cache may be worked with MEMORY, it is best to use Redis for it.

MEMORY tables cannot be partitioned.

4. CSV Engine

Stores table as a CSV file.

Useful for:

Exporting
Data exchange

When you create a CSV table, the server creates a plain text data file having a name that begins with the table name and has a .CSV extension.

When you store data into the table, the storage engine saves it into the data file in comma-separated values format.

5. ARCHIVE Engine

Designed for:

Historical records
Log storage

Advantages

High compression
Saves disk space

Limitations

Doesn't support UPDATE
Doesn't support DELETE
Doesn't support Transactions
Doesn't support MVCC

Designed for storing large amounts of historical data that is:

Rarely modified
Mostly inserted
Occasionally queried
Highly compressible

Common use cases include:

Audit logs
Login history
Sensor data
Financial transaction history
IoT event logs
Web server logs
Old application events

These records are usually never updated.

6. NDB Cluster

NDB Cluster (Network Database Cluster Engine)

It is a MySQL storage engine designed for:

High availability
Fault tolerance
Distributed databases
In-memory processing
Automated replication

Unlike InnoDB, which stores data on a single MySQL server, NDB stores data across multiple machines (nodes).

Used for applications like:

Banking
Telecom
Stock Exchanges
Airline Reservations

NDB Cluster Architecture

NDB Cluster consists of three node types.

1. Management Node (MGM)

Think of it as the manager.

Responsibilities:

Starts the cluster
Stops the cluster
Stores configuration
Monitors health
Coordinates nodes

It does not store application data.

2. Data Node

They store:

Rows
Indexes
Fragments
Replicas

Usually:

Node A
Node B
Node C
Node D

Data is spread across them.

3. SQL Nodes

These are normal MySQL servers.

Applications connect here.

In NDB, the SQL server doesn't actually store the data.

It acts like a gateway.

Advantages of NDB Cluster

High availability
Automatic replication
Distributed
Very Fast Reads
Fault Tolerance
Scalability

Disadvantages of NDB Cluster

More complex to configure
Requires more RAM
Higher hardware costs
Not commonly used as InnoDB
Some SQL features and performance characteristics differ from InnoDB, so application compatibility should be evaluated.

Real World Use Cases

Commonly used in systems that require continuous availability and low latency, such as:

Telecommunications subscriber databases
Real-time billing systems
Online gaming backends
Financial trading platforms
Airline reservation systems
Large scale authentication services

RAM + Disk Checkpoints

NDB uses RAM + Disk checkpoints.

In NDB, data is primarily stored in RAM for very fast access.

If data were stored only in RAM, then after a power failure data would be lost.

That would be just like the MySQL MEMORY engine.

To prevent this, NDB periodically creates checkpoints.

A checkpoint is simply a snapshot of the current data written from RAM to disk.

Think of it as pressing Save in Microsoft Word and when saved it on disk.

Example Hardware Requirement

Management Node

Machine 1 = Management Server

It does configuration only.

Very little RAM required.

Typically:

1 CPU
1–2 GB RAM
10 GB Disk

First Data Node

Machine 2 = Stores Data

Needs much more RAM.

Example:

4 CPU Cores
16 GB RAM
SSD

Second Data Node

Machine 3 = Replica

Same specifications as the first data node.

Can SQL Node and Management Node be on the Same Machine?

Yes.

For development or testing, you can run:

Machine 1

Management Node
SQL Node

Machine 2

Data Node

Machine 3

Data Node

MVCC (Multi-Version Concurrency Control)

MVCC (Multi-Version Concurrency Control) is a database concurrency mechanism that allows multiple transactions to access the same data simultaneously without blocking each other.

Instead of overwriting data immediately, the database keeps multiple versions of a row.

Why is MVCC Needed?

Imagine two users.

Transaction A is reading a customer's balance.

Transaction B is updating that balance.

Without MVCC

A might have to wait until B finishes (locking), reducing concurrency.

With MVCC

A reads the old version of the row.
B writes a new version of the row.
Both proceed without blocking each other.

Benefits of MVCC

Readers don't block writes.
Writes don't block readers.
High concurrency.
Consistent snapshots for transactions.
Fewer lock conflicts.

How MySQL Implements MVCC

MySQL uses Undo Logs to reconstruct older versions of rows.

Each transaction reads the appropriate version based on its snapshot.

MVCC is implemented by the MySQL storage engine using:

Undo Logs
Transaction IDs
Read Views (Snapshots)

What Happens When a Row is Updated?

When a row is updated:

The new value goes to the data page (in memory first).
The old value is copied into the Undo Log.

MVCC uses the Undo Log to reconstruct old versions for transactions that started earlier.

Hidden Columns in Every InnoDB Row

Every InnoDB row has hidden columns.

DB_TRX_ID = Transaction that last modified the row.
DB_ROLL_PTR = Pointer to Undo Log.
DB_ROW_ID = Internal row ID (if no Primary Key).

The user never sees them.

Read View

Read View = Snapshot

Conclusion

Today, I learned about MySQL Storage Engines and MVCC. I explored the purpose of storage engines, the characteristics of different MySQL storage engines, the architecture of NDB Cluster, and how MVCC enables high concurrency by maintaining multiple versions of rows. These concepts form the foundation for understanding how MySQL manages data efficiently and reliably.

Polling and Events

Kathir — Tue, 23 Jun 2026 03:09:09 +0000

Short Polling and Long Polling

Short Polling
Short polling is a technique where the client repeatedly sends HTTP requests at fixed intervals, and the server responds immediately whether or not new data is available. It doesn't wait for the results, and the result can be generated between any iteration of request response. The waiting time of request and response were assigned by the client or server.

Request data:
Client--------->Server
Client<---------Server (No Data)
Client--------->Server
Client<---------Server (No Data)

Iteration continuous until data is available
Client--------->Server
Client<---------Server (Sent Data)

Long Polling
Long polling is a type of HTTP request where the request is sent by the client to the server, and the connection is open until the result is generated. Only when the proper result generates, the server sent the response to client.

Request data:
Client---------->Server
Wait until data is available....
Client<----------Server (Sent data)

Event Stream

Event Stream
Also known as Server-Sent Events (SSE). Example of Event Stream is "ChatGPT". The server sent the response to the client continuously as the result generation is in progress and during that the time the connection between client and server is active. This prevents the client to wait for a huge amount of time until it completes a large process and sent the result. A limitation of Event Stream is that browsers may restrict the number of simultaneous connections per domain.
Example: Opening many ChatGPT tabs may slow your browser or consume more network resources, but this is not a limitation of Event Streams themselves.

Request data:
Client---------->Server

Client<--------Server (Data until completed-Ex:10% completed data)
Client<--------Server (Data until completed-30%)
Client<--------Server (Data until completed-80%)
Client<--------Server (Data until completed-100%)
Connection closes