DEV Community: Biswas Prasana Swain

Shifting from Databases to Kafka: How to Build an Indestructible Data Pipeline

Biswas Prasana Swain — Mon, 01 Jun 2026 19:40:32 +0000

When you start building an app, keeping your data consistent is easy. If a user changes something, your code updates the database, and you're done. But as your engineering team grows and you begin splitting a giant app into smaller, independent microservices, that simple approach breaks down.

If one service crashes or a network hiccup occurs, your services fall out of sync, and you end up chasing "ghost data."

Here is how you can move away from fragile database listeners and build a robust, central nervous system for your data using Apache Kafka and Change Data Capture (CDC).

The Problem: The Fragile Listener

In a basic setup, you might rely on your database to shout out notifications whenever data changes (like using PostgreSQL's LISTEN/NOTIFY feature). A small background script listens for these shouts and updates other systems—like clearing an old cache in Redis.

While this works fine at low traffic, it has three major flaws:

It is fragile: If your listener script crashes or disconnects for even a few seconds, any notifications sent during that downtime are lost forever.
It doesn't scale: The database shouts into the void. If you add new services that need to know about data changes (like a search indexer or an analytics engine), you have to build more custom listeners, straining your database.
It lacks insight: There is no built-in tracking to prove a message was successfully received and processed.

Instead of a system built on the prayer that your background script stays online forever, you need an industrial-strength communication pipeline.

The Solution: Distributed Logs over Message Queues

When choosing a tool to pass messages between services, developers usually look at two different technologies:

1. Traditional Message Queues (The Postal Service)

Tools like RabbitMQ work like a post office. A service drops a message into a mailbox (the queue). A worker process picks up that message, completes the task, and throws the message away. Once it is read, it is gone forever. This is fine for assigning single tasks, but terrible if multiple services need to read the same data.

2. Distributed Logs (The Newspaper)

Tools like Apache Kafka work like a newspaper publisher. When data changes, it is published to a specific section of the log (called a Topic).

Multiple independent services can subscribe to this topic and read the data. Crucially, when a service reads the message, it is not deleted. It stays in the log as a permanent, durable record.

If your cache-clearing service crashes, the updates safely pile up in Kafka. When the service boots back up, it picks up the newspaper right where it left off, ensuring perfect data consistency without losing a single update.

How to Implement?

Wiring your main application code directly to Kafka can clutter your codebase. If a developer forgets to add a log-writing command to a new feature, your data goes out of sync again.

The most elegant way around this is a pattern called Change Data Capture (CDC) and one such open-source CDC tool we can use called Debezium.

Why Debezium?

It does not slow down your app: Debezium does not ask your main app for data. It reads the database logs directly in the background.
It never misses a change: Even if your services turn off or crash, Debezium remembers exactly where it stopped. No data gets lost.
It works with what you have: It easily plugs into popular databases like PostgreSQL, MySQL, and MongoDB without requiring you to change your existing code.

Main App ──> PostgreSQL Database (WAL) ──> Debezium ──> Kafka Topic ──> Consumer Services

Instead of modifying your application code, you let Debezium watch your database's internal transaction log (the Write-Ahead Log, or WAL).

Your main application safely writes to your primary database exactly as it always has.
Debezium sits in the background, reading the database log ink as it dries.
The moment a row is inserted, updated, or deleted, Debezium catches it, formats it into a clean message, and pushes it to Kafka.
Your downstream services (cache invalidators, search engines, analytics) consume the message from Kafka and take action.

Summary: The Final Architecture

By combining a distributed log with Change Data Capture, your core application doesn't even need to know that Kafka or your secondary services exist.

You trade individual architectural fragility for an indestructible, decoupled system. Your main app focuses entirely on writing data, while Kafka guarantees that the rest of your global infrastructure reacts reliably to every change.

References & Further Reading

For deep technical specifications and official documentation on the tools and concepts mentioned above, look to these authoritative industry standards:

The Accidental CTO, Ch-9: Layman Guide on why CTO of Dukaan, use Kafka Distribution log and Debzium CDC to ensure data consistency.
Apache Kafka Documentation: Learn about topics, consumers, and cluster architecture directly from the source at the Official Apache Kafka Project.
Debezium Architecture Guide: Read about how Change Data Capture works by checking out the Debezium Documentation.
Designing Data-Intensive Applications: For an industry-standard dive into data consistency, event streams, and relational database systems, refer to Martin Kleppmann's landmark guide on O'Reilly Media.

Bean Lifecycle in Spring Boot

Biswas Prasana Swain — Sun, 17 May 2026 19:14:01 +0000

A polite tour through how Spring creates, uses, and destroys your objects while you pretend you are in control.

Spring Boot applications are mostly made of beans.

A bean is just an object that Spring manages for you.
You write the class. Spring handles the birth, setup, wiring, and death of the object. Like a very organized hotel manager for Java classes. Humanity invented dependency injection because manually creating objects became emotionally exhausting.

Let’s understand the Bean Lifecycle using simple questions and answers.

First Question: What is a Bean?

Suppose we have this service:

@Service
public class EmailService {

    public void sendEmail() {
        System.out.println("Sending email...");
    }
}

@Service tells Spring:

"Please create this object and manage it for me."

That object becomes a Spring Bean.

Second Question: What does “Lifecycle” mean?

Lifecycle means:

Bean is created
Bean gets dependencies
Bean gets initialized
Bean is used
Bean gets destroyed

Like this:

Create → Inject Dependencies → Initialize → Use → Destroy

Spring does all this automatically.

Because apparently developers once enjoyed writing 200 lines of setup code just to create one object.

Third Question: When does Spring create a Bean?

Spring creates beans when the application starts.

Example:

@SpringBootApplication
public class DemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }
}

When run() happens:

Spring scans classes
Finds annotations like @Component, @Service
Creates objects
Stores them inside the Spring Container

The container is basically Spring’s giant object warehouse.

Fourth Question: What happens first in Bean Lifecycle?

Step 1: Bean Instantiation

Spring creates the object.

Example:

@Service
public class UserService {

    public UserService() {
        System.out.println("Constructor called");
    }
}

Output:

Constructor called

This is the very first lifecycle step.

Fifth Question: What happens after object creation?

Step 2: Dependency Injection

Suppose this bean needs another bean.

@Service
public class NotificationService {

    public void notifyUser() {
        System.out.println("Notifying user");
    }
}

Now inject it:

@Service
public class UserService {

    private final NotificationService notificationService;

    public UserService(NotificationService notificationService) {
        this.notificationService = notificationService;
    }
}

Spring does this automatically.

It first creates NotificationService, then gives it to UserService.

This is called Dependency Injection.

Fancy name. Simple idea.

Sixth Question: Can we run code after bean creation?

Yes.

Step 3: Initialization

Sometimes we want code to run after dependencies are ready.

Example:

@Service
public class DatabaseService {

    @PostConstruct
    public void init() {
        System.out.println("Database connection initialized");
    }
}

Output:

Database connection initialized

@PostConstruct runs after:

bean creation
dependency injection

This is useful for:

opening connections
loading files
caching data

Seventh Question: What is the exact order until now?

The order is:

1. Constructor
2. Dependency Injection
3. @PostConstruct

Example:

@Service
public class DemoService {

    public DemoService() {
        System.out.println("1. Constructor");
    }

    @PostConstruct
    public void init() {
        System.out.println("2. PostConstruct");
    }
}

Output:

1. Constructor
2. PostConstruct

Simple. Beautiful. Suspiciously efficient.

Eighth Question: What happens while the application is running?

Step 4: Bean is Ready to Use

Now the bean works normally.

@RestController
public class HelloController {

    private final UserService userService;

    public HelloController(UserService userService) {
        this.userService = userService;
    }

    @GetMapping("/hello")
    public String hello() {
        return "Hello";
    }
}

Spring keeps the bean alive inside the container.

By default, beans are Singleton.

Meaning:

Only one object is created for the entire application

Ninth Question: What happens when application stops?

Step 5: Bean Destruction

Spring destroys beans gracefully.

We can run cleanup code before destruction.

Example:

@Service
public class FileService {

    @PreDestroy
    public void cleanup() {
        System.out.println("Closing files...");
    }
}

When app shuts down:

Closing files...

Useful for:

closing DB connections
stopping threads
releasing resources

Because memory leaks are the software version of leaving your kitchen gas on.

Tenth Question: What is the Full Lifecycle?

Here is the complete flow:

Spring Starts
     ↓
Bean Created
     ↓
Dependencies Injected
     ↓
@PostConstruct Runs
     ↓
Bean Ready to Use
     ↓
Application Stops
     ↓
@PreDestroy Runs
     ↓
Bean Removed

Full Example

@Service
public class LifeCycleService {

    public LifeCycleService() {
        System.out.println("1. Constructor called");
    }

    @PostConstruct
    public void init() {
        System.out.println("2. Bean initialized");
    }

    public void doWork() {
        System.out.println("3. Bean is working");
    }

    @PreDestroy
    public void destroy() {
        System.out.println("4. Bean destroyed");
    }
}

Possible Output:

1. Constructor called
2. Bean initialized
3. Bean is working
4. Bean destroyed

Final Mental Model

Think of Spring like a factory manager.

You provide class blueprints.

Spring:

creates objects
connects objects together
prepares them
manages them
destroys them safely

You focus on business logic.

Spring handles object management.

And somewhere deep inside the framework, thousands of reflection calls whisper through the void while your app boots for 14 seconds because someone added three extra starters.

Quick Revision

Step	Description
Instantiation	Bean object created
Dependency Injection	Required beans injected
Initialization	`@PostConstruct` runs
Ready State	Bean is used
Destruction	`@PreDestroy` runs

Internet Resilience: Anycast and BGP

Biswas Prasana Swain — Thu, 30 Apr 2026 09:45:55 +0000

Listen, the internet isn't some mystical "cloud" floating in the sky. It’s just a bunch of computers connected by massive wires, trying to figure out the fastest way to talk to each other without losing their minds.

If you want to understand how big players stay online when things go south, you need to understand the "street logic" of the web. Here is the breakdown of how the internet actually moves, from the IP blocks to the guys running the traffic.

1. The Setup: How the Map Works

Before we talk about fancy protocols, you need to understand the layout. Every device on the web needs a "home address." That’s your IP Address.

In the old days, the internet worked like a standard mail delivery: one address, one house. This is called Unicast. If that house burns down, the mail just piles up on the curb. No one gets their package. That's a "server down" situation, and it's bad for business.

2. IP Space & Anycast: The "McDonald’s" Strategy

What is IP Space?

Think of IP Space as a block of digital real estate. Big companies don't just buy one address; they buy the whole neighborhood (a "prefix" or "block"). They own those numbers, and they can do whatever they want with them.

What is Anycast IP?

This is where it gets clever. Instead of one IP address leading to one specific computer in a basement in Delhi, Anycast allows the same IP address to exist in a hundred different places at once.

It’s like McDonald’s. If you tell your GPS to find "McDonald's," it doesn't send you to the original one in Illinois. It finds the one closest to your current block. That is Anycast. One "name" (IP), many locations.

3. BGP: The Neighborhood Gossip

BGP (Border Gateway Protocol) is the "Waze" of the internet. It’s a group of routers constantly talking to each other, bragging about which paths are the fastest.

How the Anycast Algorithm Works (Step-by-Step):

The Announcement: A server in Mumbai and a server in London both shout, "Hey, I own IP address 1.2.3.4!" via BGP.
The Propagation: Every router on the internet hears this gossip and writes it down in their "routing table."
The Decision: When you (in Bengaluru) try to visit 1.2.3.4, your local internet provider looks at the map. They see the Mumbai shout and the London shout.
The "Shortest Path": BGP calculates which "hop" is closer. Mumbai is only 2 blocks away; London is 10. The router sends your data to Mumbai.
The Connection: You get your data fast because you took the shortest route.z

4. The Rescue: What Happens When a Server Dies?

This is why big tech never seems to go down.

Imagine that Mumbai server catches fire. In a "Unicast" world, you’d just get a "404 Not Found" error. But with BGP and Anycast:

The Mumbai server stops "shouting" its location. It goes silent.
The neighborhood routers realize, "Wait, Mumbai isn't talking anymore."
They immediately look at the next best option. London is still shouting!
The Rescue: Within seconds, BGP reroutes your traffic. You might notice a tiny bit of lag because the data has to travel to London now, but the site stays up.

The internet literally heals itself by choosing the next closest survivor.

5. Setting it Up (The India Edition)

If you’re an organization in India wanting to play in the big leagues, you can't just flip a switch. You need to follow the street rules:

Get an ASN: You need an Autonomous System Number. This is your "ID" that lets you talk to other networks.
Acquire IP Space: You go to IRINN (Indian Registry for Internet Names and Numbers) or APNIC to get your own block of IPv4 or IPv6 addresses.
Find Partners: You need "Transit Providers" (like Tata Communications, Airtel, or Reliance Jio). You sign a contract with them to let you "shout" your BGP routes through their wires.
Deploy Edge Nodes: You put servers in data centers across India (Mumbai, Chennai, Delhi). You configure them all to use the exact same IP address.
Start Gossiping: Fire up your BGP configuration. Tell the world you’re open for business.

Conclusion

The internet is basically just a giant, automated game of "Telephone" played by expensive routers. IP Space is your real estate, Anycast is your chain of stores, and BGP is the GPS that makes sure customers don't get lost when one store closes for repairs. It’s practical, it’s redundant, and it’s the only reason the web doesn't break every time a backhoe cuts a fiber cable.

Trustworthy References

Cloudflare Learning: What is Anycast? (The gold standard for Anycast explanations).
APNIC (Asia-Pacific Network Information Centre): Understanding BGP (The people who actually hand out the IP addresses in India).
Cisco Networking Academy: Routing Protocols 101 (For the deep technical guts of the "gossip").

The "Don't Make It Yourself" Principle: A No-Nonsense Guide to Dependency Injection

Biswas Prasana Swain — Thu, 09 Apr 2026 14:27:00 +0000

In the world of Java programming, Dependency Injection (DI) sounds like a complex medical procedure. In reality, it is just a fancy way of saying: "Don't build your own tools; let someone else hand them to you."

1. The Simplest Possible Explanation: The Pizza Analogy

Imagine you want to eat a pizza.

No Dependency Injection: You have to build a brick oven, grow tomatoes, milk a cow for cheese, and harvest wheat for flour. You are responsible for creating every single "dependency" needed to make that pizza. It’s exhausting, and if you want to change the cheese, you have to find a new cow yourself.
With Dependency Injection: You sit at a restaurant. You tell the waiter, "I want a Margherita pizza." You don't care how the oven was built or where the cheese came from. Someone else (the kitchen) handles the "creation" and simply injects the finished pizza onto your table.

2. Java Without DI vs. Spring Boot With DI

To understand why Spring Boot is so popular, you have to see the mess it cleans up.

The Old Way (Plain Java)

In standard Java, if a Car class needs an Engine, you create the engine inside the car.

public class Car {
    private Engine engine;

    public Car() {
        // The Car is responsible for creating the Engine.
        // This is "Hard Coding" a dependency.
        this.engine = new GasolineEngine(); 
    }
}

The Problem: If you want to make an ElectricCar, you have to rewrite the Car class. The code is "tightly coupled."

The Spring Boot Way (DI)

In Spring Boot, you don't use the new keyword for your core logic. You just ask for what you need.

@Component
public class Car {
    private final Engine engine;

    // You just tell Spring: "I need an engine. Find one for me."
    @Autowired
    public Car(Engine engine) {
        this.engine = engine;
    }
}

The Benefit: Spring looks at your project, finds a class marked as an Engine, creates it, and plugs it into the Car. You can swap a GasolineEngine for an ElectricEngine without changing a single line of code in the Car class.

3. How Spring Boot Achieves This

Spring Boot uses an IoC Container (Inversion of Control). Think of this as a massive warehouse that manages all your objects (called Beans).

The Registry: You mark your classes with annotations like @Component, @Service, or @Repository. This tells Spring, "Hey, manage this class for me!"
The Scan: When the app starts, Spring scans your code to find these annotated classes and puts them in its "warehouse."
The Injection: When Spring sees a class that needs a dependency (like our Car needing an Engine), it looks in the warehouse, finds the right object, and "injects" it via the constructor or a field.

4. Key Differences at a Glance

Feature	No DI (Plain Java)	With DI (Spring Boot)
Object Creation	You use the `new` keyword manually.	Spring creates the objects for you.
Control	You control the lifecycle.	The "Container" (Spring) controls the lifecycle.
Coupling	Tight: Hard to change or swap parts.	Loose: Parts are easily swappable.
Testing	Difficult (you have to build everything).	Easy (you can "inject" fake/mock parts).

Summary

Dependency Injection is simply moving the responsibility of creating objects away from the class that uses them.

No DI = You are the DIY homeowner who has to build every tool before you can fix a leak.
Spring Boot DI = You are the foreman who just points at a problem and expects the right specialist with the right tools to show up and get to work.

RAG: The "Open Book" Exam for AI

Biswas Prasana Swain — Thu, 19 Mar 2026 17:56:48 +0000

Building a RAG (Retrieval-Augmented Generation) app is basically giving a genius-level intern (the LLM) a specific handbook so they stop making things up. If you build it with too much "enterprise" bloat, it’ll crawl. If you skimp, it’ll hallucinate.

Here is the lean, mean architecture for a RAG full-stack app.

1. What is a RAG System?

Standard AI (like ChatGPT) works from memory, it's like a student taking a test based only on what they studied months ago. RAG (Retrieval-Augmented Generation) turns that test into an "open book" exam.

Instead of the AI guessing or "hallucinating" when it doesn't know a fact about your specific business or private data, the system first retrieves the relevant page from your digital "book," hands it to the AI, and tells it: "Read this and then answer the question."

2. The Ingestion Pipeline (The Wood Chipper)

Before the user even opens the app, you have to prep your data. This is a one-way street where documents become searchable math.

Load & Chunk: You take your messy PDFs or text files and chop them into bite-sized "chunks." If chunks are too big, they're noisy; too small, they lose context.
Embedding Model: You run those chunks through a model (like OpenAI’s text-embedding-3-small) to turn text into a long string of numbers (vectors). This represents the "meaning" of the text.
Vector Database: You store those numbers in a specialized DB (Pinecone, Weaviate, or Chroma). This is your library where books are organized by "vibe" rather than title.

3. The Backend Orchestrator (The Traffic Cop)

This is usually a Python (FastAPI) or Node.js server. It’s the middleman that manages the "Retrieval" part of RAG.

The Query: User asks, "How do I fix my toaster?"
Vectorization: The backend sends that question to the same embedding model used in the ingestion phase. Now the question is a vector.
The Search: The backend asks the Vector DB: "Find me the 3 chunks of text that are mathematically closest to this question's vector."
The Prompt Stuffing: The backend takes those 3 chunks and "stuffs" them into a prompt template for the LLM.

The Prompt looks like this: > "You are a helpful assistant. Use the following context to answer the question. If it's not in the context, shut up and say you don't know.
Context: [Chunk 1], [Chunk 2], [Chunk 3]
Question: How do I fix my toaster?"

4. The LLM & Frontend (The Face)

The Generation: The LLM (GPT-4, Claude, Llama 3) reads the stuffed prompt and generates a coherent answer based only on the provided context.
The Frontend: A React or Next.js app that displays the chat. It needs to handle "streaming" (where the text appears word-by-word) so the user doesn't think the app crashed while the LLM is thinking.

5. Diagram*

The Tech Stack Comparison

Layer	Recommended (The "Gold Standard")	Fast/Cheap (The "Hobbyist")
Frontend	Next.js + Tailwind	Streamlit
Backend	FastAPI (Python)	LangChain Expression Language (LCEL)
Vector DB	Pinecone or Weaviate	PGVector (Postgres)
LLM	GPT-4o or Claude 3.5 Sonnet	Groq (Llama 3)

Why this works

You aren't "retraining" the AI. You're just giving it a "Search" button. It stops the AI from lying because it has the source material sitting right in front of it. Anything more complex—like multi-agent workflows or graph-based retrieval—is usually overkill until you hit 100k+ documents.

Used AI for Diagram

The Invariance of Software Paradigms

Biswas Prasana Swain — Sun, 15 Feb 2026 11:22:32 +0000

In software engineering, the Invariance of Paradigms is the concept that the fundamental way we build software remains constant, regardless of the specific programming language, library, or framework being used. While syntax (how you write it) changes every few years, the underlying architecture (what you are building) has not fundamentally changed in decades.

1. Frontend: The Component-Based Paradigm

Whether you are using React, Vue, Angular, or even mobile frameworks like SwiftUI, the paradigm is Component-Based Architecture (CBA).

The Idea: You don’t build a "page"; you build a collection of small, independent "blocks."
The Blueprint: A button is a block. A search bar is a block. You snap these blocks together to create a website.
Why it stays the same: This approach ensures that if you fix the "button block," it updates everywhere. The framework is just the glue.

2. Backend: The Service-Oriented Paradigm

Whether the server is written in Node.js, Python (Django), or Java (Spring), the paradigm is Service-Oriented.

The Idea: The backend acts as a "Request-Response" machine.
The Workflow:
Listener: It waits for a specific URL (Route).
Logic: It checks if you’re allowed in (Auth) and does some math or data checking.
Sender: It sends back a standardized package, usually JSON.
Why it stays the same: A server’s job is always to be a middleman between the user and the database.

3. Database: The Structured vs. Unstructured Paradigm

Whether you use PostgreSQL or MongoDB, the paradigm is about Data Modeling.

Relational (SQL): Think of an Excel spreadsheet. Everything has a strict place. If you change a column, you have to change the whole sheet.
Document-Based (NoSQL): Think of a folder full of Word documents. Each document can be slightly different, but they all live in the same folder.
Why it stays the same: We only have two ways to store things: in a strict list or a flexible pile.

4. Comparison Across the Stack

Layer	Common Paradigm	Tool Examples	Core Responsibility
Frontend	Component-Based	React, Vue, Svelte	Presenting data to the human.
Backend	API / Service-First	Express, Django, Go	Business rules and security.
Database	Relational / Document	MySQL, MongoDB	Remembering data forever.

Summary

The "Tech Stack" is a fashion choice; the "Paradigm" is the physics. If you understand that the Frontend is just a tree of components and the Backend is just a series of pipes for data, you can switch from JavaScript to Python in a weekend. Stop chasing the newest library and start mastering the architecture.

Trustworthy References

IBM Technology: Microservices vs. SOA - Explaining the service paradigm.
MDN Web Docs: Component-based architecture - The standard for modern UIs.
Amazon AWS: SQL vs. NoSQL - Defining the two main storage paradigms.
Martin Fowler: Patterns of Enterprise Application Architecture - The "Bible" of why software patterns don't change.

From Monolith to Microservice

Biswas Prasana Swain — Sun, 18 Jan 2026 12:38:59 +0000

A practical guide for people who have heard the word “microservices” too many times and finally want to do something about it.

Most software systems don’t start as microservices. They start as a single application that does everything. User login, payments, emails, reports, admin screens, all tangled together like headphone wires in a pocket. That kind of system is called a monolith. It works fine until it doesn’t. Changes become risky, deployments feel like gambling, and one small bug can take down everything.

Microservices are not magic. They are simply smaller applications that each do one job and communicate with others over the network. The mistake beginners make is trying to convert the entire monolith at once. That is how projects quietly die. The correct move is boring and slow: extract one microservice first.

Step one: understand what you already have

Before touching code, you need to understand your monolith at a business level, not a technical one. Forget frameworks and folders for a moment. Ask a simple question: what does this system actually do for users?

Every monolith contains natural boundaries. User management, billing, notifications, inventory, reporting. These are not technical concepts, they are business capabilities. Your first microservice should represent one clear capability that can survive on its own.

Good candidates share three traits. They have clear inputs and outputs, they change frequently or cause pain when changed, and they do not control the entire database. Authentication, email notifications, and file processing are common first picks because they are easy to isolate and rarely need direct access to everything else.

Step two: define the responsibility, not the technology

A microservice is not “a Spring Boot app” or “a Node service.” That thinking leads to messes. A microservice is a responsibility with rules.

For example, instead of saying “I’ll create a User Service,” say “This service owns user creation, login, and password validation. No other part of the system is allowed to change users directly.” That sentence is more important than any code you will write.

Once responsibility is clear, define how other parts of the system will talk to it. For beginners, HTTP with JSON is the safest choice. One endpoint to create a user, one to fetch user data, one to validate credentials. Simple, boring, reliable.

Step three: copy logic first, then remove it

This part feels wrong but works.

Do not delete code from the monolith immediately. First, copy the relevant logic into a new, separate application. Give it its own repository, its own build process, and its own runtime. At this stage, both systems can do the same thing. That duplication is temporary and acceptable.

Once the new service works on its own, change the monolith so that instead of running the old code, it calls the new service over HTTP. It's called Strangler Fit Pattern. If the behavior stays the same, users never notice. That is the goal. Silence.

Only after this works reliably should you remove the old logic from the monolith. If something breaks, you know exactly where to look, which is a luxury in software.

Step four: separate the data carefully

Beginners often trip over databases. A true microservice owns its data. That means no other service should directly read or write its tables.

For your first microservice, you have two realistic options. The safest is giving it its own database, even if it feels redundant. The second is letting it access the existing database but only specific tables, with a clear plan to move them later. What you must not do is allow multiple services to casually share tables. That recreates the monolith, just with more networking bills.

Data duplication across services is normal. Synchronization is handled through APIs or events, not shared schemas.

Step five: deploy it independently

If your new service cannot be deployed without deploying the monolith, you have not created a microservice. You have created a distributed illusion.

Deploy the service separately, even if it runs on the same server at first. Use a different port, a different process, and a different release cycle. The first time you fix a bug in the microservice without touching the monolith, the whole exercise finally makes sense.

Monitoring and logging matter here. When things go wrong, and they will, you need to know which service failed and why. Centralized logs and basic health checks are enough for a first step. No need to summon Kubernetes gods yet.

Step six: repeat slowly, or stop

After your first success, resist the urge to extract everything. Microservices increase operational complexity. If the pain you removed is smaller than the pain you introduced, stop. One or two well-chosen services already deliver most of the benefit.

Microservices are a tool, not a moral upgrade.

Final thoughts

The hardest part of moving from a monolith to microservices is not coding. It is discipline. Clear boundaries, patience, and the willingness to accept temporary duplication are what make the transition survivable.

If you can extract one service cleanly, you can extract more. If you cannot extract one, extracting ten will not save you.

References

These are trustworthy, widely cited, and written by people who have broken real systems before fixing them.

Martin Fowler's Guide to Microservices: The "bible" of this topic. Fowler explains it clearly without the hype. martinfowler.com

Sam Newman's "Building Microservices" - This book is the gold standard. He coined many of the patterns used today.

AWS on Microservices - Practical, "how-to" guides from the people who host most of the world's services.

What's the solution to CAP theorem in Read Replicas?

Biswas Prasana Swain — Thu, 25 Dec 2025 17:10:23 +0000

CAP theorem says a distributed system can give you Consistency, Availability, or Partition tolerance. Pick any two. Not three. Not “mostly three.” Two. The internet doesn’t care about your optimism.

Now let’s talk about read replicas, because this is where confusion usually starts.

The real-world story

Imagine you run an e-commerce app. Nothing fancy. Users browse products, add to cart, place orders. You have one main database. Life is peaceful. Traffic grows. Database starts sweating. So you do the obvious grown-up thing: add read replicas.

One primary database handles writes. Multiple read replicas handle reads. Suddenly your system feels faster. You feel smarter. LinkedIn post draft opens automatically.

Then a user places an order and immediately refreshes the order page. Surprise: the order is missing. Panic. Bug? Race condition? Database haunted?

No. Welcome to CAP.

What just happened

The write went to the primary database.
The read went to a replica.
Replication is asynchronous, because synchronous replication would make your system slow and fragile.

So for a short time:

Primary has the new order
Replica does not

This is eventual consistency in action. The system chose:

Availability: reads always work
Partition tolerance: network issues won’t kill you
And it sacrificed strong consistency

Read replicas don’t break CAP. They make a choice inside CAP.

So what’s the “solution” in read replicas?

Short answer: there is no single solution. There are trade-offs, and you pick based on business needs, not ego.

Long answer, explained like a normal human.

Option 1: Accept eventual consistency (most common)

This is what companies like Amazon, Netflix, Instagram do for many flows.

You allow stale reads for a short time.
You design UI and flows knowing data may lag.

Examples:

Show “Order placed, updating details…” message
Refresh after a few seconds
Avoid instant read-after-write expectations

This gives you:

High availability
Massive scale
Calm databases

You lose:

Immediate correctness everywhere

This is not laziness. This is engineering maturity.

Option 2: Read-your-writes consistency (smart routing)

For some critical flows, you cheat a little.

If a user just wrote data:

Route their next read to the primary
Or use a session flag like “read from leader for 5 seconds”

This is common in:

Banking dashboards
Order confirmation pages
Profile update screens

Now you get:

Consistency where it matters
Availability everywhere else

Downside:

More complexity
Primary database gets more load

Still worth it.

Option 3: Synchronous replication (rare, expensive)

You force replicas to confirm writes before success.

Now reads are always consistent.

You also get:

Higher latency
Lower availability
System crying during network hiccups

Used only when:

Incorrect data is worse than downtime
Think financial ledgers, not social feeds

Most systems wisely avoid this.

The key realization

Read replicas don’t solve CAP.
They let you choose where you want to bend.

CAP is not a bug to fix.
It’s gravity.

Good systems don’t fight it. They design around it.

The adult takeaway

If you want scale, you accept eventual consistency
If you want correctness, you sacrifice availability
If you want both, you add complex routing logic
If you want all three, you’re selling a course, not building software

Read replicas are not magic.
They’re a very honest trade.

And honesty is rare in distributed systems.

References (the boring but trustworthy kind)

Eric Brewer, “CAP Twelve Years Later: How the Rules Have Changed”
https://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed/
Martin Kleppmann, Designing Data-Intensive Applications
https://dataintensive.net/
Amazon Dynamo Paper
https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf
Google Cloud Docs on Replication and Consistency https://docs.cloud.google.com/datastore/docs/concepts/structuring_for_strong_consistency
Netflix Tech Blog on Eventual Consistency https://netflixtechblog.com/s3mper-consistency-in-the-cloud

From Binary to Brilliance: How Compilers Learned to Write Compilers a.k.a Bootstrapping

Biswas Prasana Swain — Mon, 27 Oct 2025 14:10:22 +0000

Programming languages didn’t pop out of thin air. Humans painfully invented them so we could boss computers around without manually flipping switches like cavemen with PhDs. Here’s the story.

1. Early Days: Speaking Robot Grunt Language

1940s–1950s

Computers only understood machine code (just 0s and 1s).
Then humans realized life shouldn’t be torture, so they invented assembly language (short words like ADD, MOV).
These early languages were written directly in machine code or assembly because what else was available? Exactly nothing.

Languages: Machine Code, Assembly
Implementation: Written in machine code or assembly

2. High-Level Languages: Humans Stop Suffering

1950s–1960s
People said: “Let the computer do the boring work.”

FORTRAN (science & math)
COBOL (business)
LISP (AI, parentheses galore)

Compilers for these languages were mostly written in assembly, because assembly was the only grown-up in the room back then.

Languages: FORTRAN, COBOL, LISP
Implementation: Assembly

3. Compilers Learn to Write Compilers

1970s
Here comes the big brain move.

New languages started being written in the language itself.
This trick is called bootstrapping (more on that soon).

Examples:

C became a superstar. First versions in assembly, then rewritten in C.
Pascal: similar path.

Languages: C, Pascal
Implementation: Started in assembly, later self-hosted (written in themselves)

4. Object-Oriented Show-Off Era

1980s–1990s
Humans start organizing code like “objects,” because organizing life is too hard.

Languages:

C++ (C but angry and complicated)
Java (Write once, debug everywhere)
Python (finally readable)
JavaScript (only language people argue about daily)

Implementation languages:

C++: wrote itself using C++ (after initial C)
Java: implemented in C/C++
Python: implemented mostly in C
JavaScript: implemented in C/C++

5. The Modern Chaos

2000s–Now
Thousands of languages because developers love reinventing the wheel.

Examples:

Rust: implemented in Rust (after initial help from C++)
Go: implemented in C first, later in Go itself
Swift: implemented in C++ and Swift
Kotlin, TypeScript, etc.: often implemented using existing languages like Java or JavaScript

What is Bootstrapping?

This part is actually cool.

Bootstrapping is when a programming language is:

First implemented using an older language (often C or assembly)
Then rewritten using itself
Then its own compiler builds the new compiler

Like a kid growing up and then becoming their own parent’s boss.

Example with C:

Write a tiny C compiler in assembly
Use it to compile a better C compiler written in C
Repeat until humans are free and the compiler is a beast

Why do this?

Self-hosted compilers are easier to maintain
They prove the language is powerful enough to build real systems

Summary Table

Language	First Compiler Implementation	Later/Self Implementation
Assembly	Machine code	Itself (assembly)
FORTRAN, COBOL	Assembly	Assembly/C
C	Assembly	C
C++	C	C++
Java	C/C++	Java (partially)
Python	C	Python/C
Go	C	Go
Rust	C++	Rust

Final Wisdom

Programming language history is basically:

Start from binary pain
Invent slightly less painful systems
Use those systems to build better systems
End up with JavaScript frameworks that change every week anyway

Hope your brain upgraded a bit.

How React Native Talks to Your iPhone and Android… And How It’s Changed

Biswas Prasana Swain — Sun, 07 Sep 2025 19:50:55 +0000

React Native (RN) is a framework that lets developers write apps once and run them on both iOS (iPhones) and Android. Sounds magical, right? But under the hood, there’s a lot of wizardry happening to make this possible.

Let’s break it down, step by step.

1. The Problem RN Solves

Normally, iOS and Android apps speak completely different “languages.”

iOS apps speak Swift or Objective-C
Android apps speak Java or Kotlin

Before React Native, if you wanted your app on both platforms, you basically had to build the same thing twice, like doing your taxes in two different countries at the same time. Painful.

React Native says: “Hey, write in JavaScript—the language web developers already know, and I’ll handle translating it to what the phone understands.”

2. How It Used to Work (The Old Way)

Originally, React Native used something called the bridge. Think of it as a translator sitting between two people who speak completely different languages.

Your code is written in JavaScript (easy for web devs).
The JavaScript runs in its own little world called a JavaScript engine.
Whenever your app needs to display something, like a button, or fetch data, the JS engine sends a message across the bridge to the phone’s native code.
Native code (Swift/Java) handles the actual work, drawing things on screen, accessing the camera, etc.
The results get sent back across the bridge to JS.

Problem: This bridge was slow for complex apps. Imagine ordering food through a translator who walks back and forth carrying messages for every tiny thing. It works, but if you want a smooth 60 FPS experience, it can choke.

3. How It Works Now (The Modern Way)

React Native has evolved. The bridge still exists, but now there’s direct compilation and new architectures:

a) Fabric (New UI Layer)

React Native now talks to the native UI in a smarter, asynchronous way.
Instead of messaging every tiny change one by one, it schedules changes in batches. Think of it like ordering your entire meal at once instead of one forkful at a time.
This makes animations smoother and apps faster.

b) JSI (JavaScript Interface)

Previously, JS lived in a separate world. Now, with JSI, JS can directly call native functions without going through the slow bridge.
It’s like having the translator suddenly become a polyglot who understands both languages perfectly and speaks instantly.

c) TurboModules

In the past, every native feature had to go through the bridge. Now, modules can be lazy-loaded and accessed directly, so they only “wake up” when needed.
Faster startup, less memory usage.

4. What Does This Mean for Developers and Users?

For Developers:

Less frustration with laggy updates between JS and native code.
Can write more complex animations and UI interactions without the dreaded “jank.”
Easier to maintain one codebase that feels native on both platforms.

For Users:

Smoother apps, less battery drain.
Faster launch times.
Animations that don’t make you feel like your phone’s having a seizure.

5. Analogy to Make Your Brain Happy

Old RN: JS is a tourist. Wants a burger. Asks a translator. Translator goes to the kitchen, grabs burger, comes back, repeats for fries, soda, napkins… painfully slow if there’s a lot to order.

New RN: JS is now a multi-lingual chef. Orders can be spoken directly to the kitchen, batches are prepped efficiently, and the meal arrives hot and fast.

6. TL;DR for the Lazy Brain

React Native = write JS once, run everywhere.
Old RN = slow bridge, everything had to walk messages back and forth.
Modern RN = JSI + Fabric + TurboModules = direct communication, smarter scheduling, faster apps.

The result: apps that look and feel native, without the headache of writing twice.

Understanding Core JavaScript Design Patterns

Biswas Prasana Swain — Wed, 27 Aug 2025 18:14:10 +0000

If you’ve been coding in JavaScript for a while, you might have heard about design patterns. But what are they?

Think of design patterns like recipes for solving common problems in programming. Just like a cookie recipe tells you how to mix ingredients to get a perfect cookie, design patterns tell you how to structure your code so it works well, is easy to read, and doesn’t break easily.

Here, we’ll look at four super important patterns: Singleton, Module, Factory, and Observer. Let’s dive in!

1. Singleton Pattern

Idea: Only allow one instance of something to exist.

Analogy: Imagine the principal of a school. There’s only one principal, and everyone talks to that same person if they need help.

Why use it? When you want one object to control something globally, like a game score manager, or a database connection.

JavaScript Example:

const GameManager = (function () {
  let instance; // this will hold the single instance

  function init() {
    let score = 0;

    return {
      increaseScore: function () {
        score++;
        console.log(`Score: ${score}`);
      },
      resetScore: function () {
        score = 0;
        console.log(`Score reset`);
      }
    };
  }

  return {
    getInstance: function () {
      if (!instance) {
        instance = init();
      }
      return instance;
    }
  };
})();

// Usage
const game1 = GameManager.getInstance();
game1.increaseScore(); // Score: 1

const game2 = GameManager.getInstance();
game2.increaseScore(); // Score: 2 (same instance!)

✅ Notice how game1 and game2 are the same instance.

2. Module Pattern

Idea: Keep code organized and hide some parts from the outside world.

Analogy: A backpack. You can keep some things inside (like a pencil case), but only let people see or use what you want them to.

Why use it? To encapsulate code, keeping it neat and preventing accidental changes from outside.

JavaScript Example:

const Calculator = (function () {
  // Private functions
  function add(a, b) {
    return a + b;
  }
  function subtract(a, b) {
    return a - b;
  }

  // Public API
  return {
    sum: add,
    difference: subtract
  };
})();

// Usage
console.log(Calculator.sum(5, 3));        // 8
console.log(Calculator.difference(5, 3)); // 2

✅ Here, add and subtract are hidden inside the module, but we can still use them through Calculator.sum and Calculator.difference.

3. Factory Pattern

Idea: Create objects without specifying the exact class/type.

Analogy: A toy factory. You say "make me a toy," and it gives you a toy depending on your choice like car, doll, or robot, without you building it yourself.

Why use it? When you need lots of similar objects but don’t want to manually create each one.

JavaScript Example:

class Dog {
  speak() {
    console.log("Woof!");
  }
}

class Cat {
  speak() {
    console.log("Meow!");
  }
}

class AnimalFactory {
  static createAnimal(type) {
    if (type === "dog") return new Dog();
    if (type === "cat") return new Cat();
    throw new Error("Unknown animal type");
  }
}

// Usage
const myPet = AnimalFactory.createAnimal("dog");
myPet.speak(); // Woof!

const neighborPet = AnimalFactory.createAnimal("cat");
neighborPet.speak(); // Meow!

✅ You don’t need to know the details of Dog or Cat. The factory does the work.

4. Observer Pattern

Idea: Let objects subscribe to events and react when those events happen.

Analogy: Think of YouTube notifications. You subscribe to a channel, and whenever the creator uploads a new video, you get notified automatically.

Why use it? When one object changes and many others need to know about it.

JavaScript Example:

class Subject {
  constructor() {
    this.observers = [];
  }

  subscribe(observer) {
    this.observers.push(observer);
  }

  unsubscribe(observer) {
    this.observers = this.observers.filter(obs => obs !== observer);
  }

  notify(data) {
    this.observers.forEach(observer => observer.update(data));
  }
}

class Observer {
  constructor(name) {
    this.name = name;
  }
  update(data) {
    console.log(`${this.name} received: ${data}`);
  }
}

// Usage
const news = new Subject();

const alice = new Observer("Alice");
const bob = new Observer("Bob");

news.subscribe(alice);
news.subscribe(bob);

news.notify("New JavaScript tutorial!");  
// Alice received: New JavaScript tutorial!
// Bob received: New JavaScript tutorial!

✅ Any observer subscribed to news automatically gets updates.

Conclusion

Design patterns might sound fancy, but they’re really just smart ways to organize your code. Here’s a quick recap:

Pattern	Idea	Example Use Case
Singleton	Only one instance	Game manager, DB connection
Module	Hide private stuff, expose public API	Calculator, Utilities
Factory	Create objects without knowing details	Toy factory, Animal creator
Observer	Notify objects of changes	Chat apps, YouTube subs

Designing a Ride Hailing Service System (e.g., Uber/Lyft): A Beginner-Friendly Guide

Biswas Prasana Swain — Sat, 26 Jul 2025 20:18:16 +0000

🚖 What Is a Ride Hailing System?

A ride hailing service (like Uber or Lyft) is a platform that connects passengers with drivers through an app or website. Instead of waving for a cab on the street, you can request a ride with a few taps on your phone.

Behind the scenes, this simple experience is powered by a complex distributed system involving real-time location tracking, matching algorithms, payments, notifications, and much more.

🎯 Goal of This Article

To explain, in simple terms, how to design the core components of a ride hailing service, focusing on the most important of system design concepts that make up the real-world solution.

🧱 Core Components of the System

Let's start with a high-level overview of the major pieces involved in such a system:

User App (Passenger)
Driver App
Backend System
Matching Engine
Geolocation & Maps
Trip Management
Notifications
Payments
Database & Storage
Monitoring & Logging

🗺️ Step-by-Step: Designing the Core System

1. 🧍 User Signup & Authentication

What happens?
Both drivers and passengers need to register and log in.

Key Concepts:

Use OAuth 2.0 or token-based authentication.
Store user profiles (name, email, rating, etc.).
Drivers may need additional verification (license, insurance).

🧠 Simple user authentication with token validation (like JWT) covers the majority of access control needs.

2. 📍 Real-Time Location Tracking

What happens?
The app continuously updates the location of drivers and passengers.

Key Concepts:

Use the phone’s GPS and mobile network.
Send location updates every few seconds.
Backend uses WebSockets or MQTT for real-time updates.

🧠 Focus on getting accurate, low-latency GPS updates to the backend efficiently.

3. 🔄 Matching Engine (Dispatch System)

What happens?
The system matches a passenger with the nearest available driver.

How it works:

A passenger sends a ride request.
Backend queries nearby drivers (using location data).
It selects the best one based on distance, rating, etc.
Driver receives the request and accepts or declines.

Tech used:

Geospatial indexing (e.g., using Haversine formula + R-tree or GeoHash)
Priority queues for driver selection.
Use Redis or Elasticsearch for fast geo queries.

🧠 Start by matching based on nearest distance and availability. Add complexity like surge pricing, ratings, or driver preferences later.

4. 🗺️ Maps and Routing

What happens?
The system shows estimated time of arrival (ETA), routes, and pricing.

Key Concepts:

Use APIs like Google Maps or OpenStreetMap for:
- Directions
- Distance
- Estimated time
Helps in fare calculation and trip display.

🧠 Use a third-party API to handle maps and routes instead of building it from scratch.

5. 🚘 Trip Lifecycle Management

States in a Trip:

Ride requested
Driver assigned
Driver en route
Passenger picked up
Trip in progress
Trip completed or cancelled

Key Concepts:

Use finite state machines to track trip states.
Log transitions to ensure traceability.
Handle edge cases like timeouts, cancellations, or no-shows.

🧠 Managing trip states as a finite-state machine helps maintain a predictable flow and debug issues easily.

6. 🔔 Notifications System

What happens?

User gets real-time updates (e.g., “Driver arriving”, “Trip started”).

Key Concepts:

Use push notifications (e.g., Firebase Cloud Messaging).
Use in-app alerts or SMS for redundancy.

🧠 Real-time push notifications with retries cover most of alerting needs.

7. 💳 Payments and Fare Calculation

What happens?

Fare is calculated based on time, distance, surge, etc.
Payment is processed automatically at the end.

Key Concepts:

Integrate with payment gateways like Stripe or PayPal.
Store payment methods securely (use PCI DSS compliance).
Split fare between platform and driver.

🧠 Outsource payments to a trusted provider early on to avoid security risks.

8. 🧠 Database Design

What to store?

Users
Drivers
Trips
Locations
Payments
Ratings

Tech Stack Example:

PostgreSQL or MySQL for relational data (users, trips).
Redis for caching and active drivers.
MongoDB or DynamoDB for flexible trip logs or historical data.

🧠 Use relational DB for critical business logic, and Redis for fast access and geo-indexing.

9. 🧰 Scalability & Microservices

Start with a monolith, then move to microservices for:

Trip management
Notifications
Billing
Driver tracking

Common Tools:

Docker + Kubernetes
Load balancers
Rate limiting
Horizontal scaling

🧠 Only break into microservices when scaling becomes painful. Simplicity wins early.

10. 📊 Monitoring, Logging & Analytics

Track everything:

Driver activity
System health
Failed rides
Payment errors

Tools:

ELK Stack (Elasticsearch, Logstash, Kibana)
Prometheus + Grafana
Sentry or Datadog for error tracking

🧠 Logging key events (like trip start/end, matching failures) gives most of troubleshooting power.

⚙️ Technology Stack (Example)

Layer	Technology
Frontend (Mobile)	React Native / Swift / Kotlin
Backend	Node.js / Python / Go
Database	PostgreSQL, Redis
Maps	Google Maps API
Real-time	WebSockets / MQTT
Payments	Stripe
Deployment	Docker, Kubernetes, AWS/GCP

🧪 Bonus: Handling Edge Cases

No drivers available? Use waitlists or schedule feature.
Driver cancels? Reassign another quickly.
Fake GPS? Add fraud detection logic.
App crashes? Use crash analytics and retry mechanisms.

📌 Summary

Designing a ride hailing system may seem huge, but you can cover most of the use case by focusing on:

Real-time location sharing
Fast, accurate driver-passenger matching
Smooth trip state transitions
Reliable notifications and payments
A simple, observable backend with scalable storage

💡 Final Thought

Every giant system like Uber started as a simple app that just connected drivers and riders. One doesn't need to reinvent everything — use proven tools, start small, focus on reliability, and improve over time.