DEV Community: Mohd Saquib Mansoori

Map First, Walk Second: What a Librarian Taught Me About Postgres

Mohd Saquib Mansoori — Mon, 20 Jul 2026 16:39:48 +0000

A few weeks back I was staring at an EXPLAIN ANALYZE output, feeling pretty good about myself. I knew the two big scan types — Seq Scan and Index Scan — and I thought that covered it. Full table, or use the index. Simple.

Then I ran a query and saw this:

Bitmap Index Scan
Bitmap Heap Scan

Two lines I didn't recognize. My first reaction was to just ignore it and move on — but that never sits right with me, so I went down the rabbit hole. What I found actually changed how I think about databases.

Picture a librarian, not a computer

Here's the mental model that finally made it click for me.

Imagine you ask a librarian for every book about "Engineering." A librarian doing an Index Scan would work like this: check the catalog, walk to shelf 12, grab the book, walk back, check the catalog again, walk to shelf 47, grab the next book, walk back... For 10 books, that's fine. For 10,000 books scattered across the building? You'd fire that librarian.

A Seq Scan librarian does the opposite — ignores the catalog entirely and just walks every single shelf in the building, checking every book. Wasteful if you only needed a handful, but honestly not bad if you needed most of the books anyway.

Then there's the librarian who does something smarter. Before walking anywhere, they sit down with the catalog and write out a full list: "shelf 4, shelf 9, shelf 9 again, shelf 12, shelf 30." Then they walk the building once, in order, picking up every book along the way without doubling back.

That third librarian is a Bitmap Scan. And once I saw it that way, the Postgres version stopped feeling abstract.

What Postgres is actually doing

It happens in two acts.

Act one — Bitmap Index Scan. Postgres goes through the index, but instead of fetching rows as it finds them, it just marks down where the matches live — a simple map of "yes, this page has something I need." No data is touched yet. Just the map.

Act two — Bitmap Heap Scan. Now Postgres walks the actual table, but it follows that map in physical order instead of chasing matches one by one. Fewer detours, fewer random jumps, way less wasted motion.

That's it. That's the whole trick — look before you leap, and leap in order.

Why this matters more than it sounds

Random disk access is one of the slowest things a database can do. Every time it jumps somewhere new, there's a cost. So Postgres' planner is constantly asking: "is this query going to touch so few rows that random jumping is fine? Or so many rows that I should just read everything? Or is it somewhere annoyingly in the middle?"

That "annoying middle" is exactly where bitmap scans live. Say you've got a million employees and you're looking for everyone in Engineering:

SELECT * FROM employees WHERE department = 'Engineering';

Only a tiny sliver of the table matches? Index Scan — a few random jumps are cheap enough.
About 15% of the table matches? Bitmap Scan — too many for random jumps, too few to justify reading it all.
More than half the table matches? Seq Scan — at that point, just read everything.

The part that actually impressed me

Here's what made me sit up, though. Bitmaps aren't limited to one index — Postgres can build two of them and merge them before ever touching the table:

SELECT * FROM employees
WHERE department = 'Engineering' AND salary > 100000;

One bitmap from the department index, one from the salary index, combined with a quick AND — and only then does it go fetch rows. A regular index scan has no clean way to do that. This, to me, is the real reason bitmap scans exist.

One honest caveat

A bitmap only tells Postgres which pages to check — it still has to visit the table and confirm each row is actually visible (thanks to how MVCC works under the hood). And if the matching set is so big the bitmap can't track individual rows anymore, Postgres quietly downgrades to tracking whole pages instead. You'll spot this in the plan as Recheck Cond — Postgres double-checking the condition itself once it gets to the page, since the bitmap only promised "something here," not "this exact row."

Where I landed

I used to think "index = automatically fast." Turns out that's only true when few rows match. Once the number of matches climbs, blindly following the index becomes its own kind of slow — and that's the gap bitmap scans are built to fill.

So next time Bitmap Index Scan shows up in your EXPLAIN, don't skim past it. It's Postgres quietly doing the librarian thing — mapping first, walking second.

Transactions Behind the Scenes: What Really Happens in a Database

Mohd Saquib Mansoori — Tue, 14 Jul 2026 07:00:05 +0000

What Actually Happens When You Hit COMMIT

I used to think of COMMIT as a light switch. You flip it, the data is saved, done. It wasn't until I actually dug into how Postgres handles a transaction internally that I realized how much engineering is packed into that one keyword — and how much of it exists purely to survive the worst possible moment: your server crashing mid-write.

This post is my attempt to explain that process the way I wish someone had explained it to me — with a real multi-table example, not a toy one.

Setting up a real scenario

Say we're running a small e-commerce backend with three tables:

users(id, name, balance)
orders(id, user_id, amount, status)
transactions(id, user_id, amount, type)

A user places an order. In your application code this probably happens inside a single DB transaction, something like:

BEGIN;

UPDATE users SET balance = balance - 500 WHERE id = 1;

INSERT INTO orders (user_id, amount, status)
VALUES (1, 500, 'placed');

INSERT INTO transactions (user_id, amount, type)
VALUES (1, 500, 'debit');

COMMIT;

Three separate writes, but they need to succeed or fail together. If the balance gets deducted but the order never gets recorded, you've got a very confused customer and a support ticket. This is exactly what a transaction is for — a group of operations that the database treats as one atomic unit, guaranteeing the classic ACID properties: atomicity, consistency, isolation, and durability.

But "guaranteeing durability" is doing a lot of work in that sentence. How does the database actually make sure committed data survives a crash a millisecond later? That's the interesting part.

Nothing happens directly on disk

The first thing that surprised me: none of those three statements touch your disk directly. Disk I/O is slow — genuinely, painfully slow compared to RAM — so Postgres (like basically every serious database) keeps a chunk of memory called the buffer cache (or buffer pool) where it pulls in the relevant data pages before touching them.

So when the UPDATE runs, Postgres loads the page containing user id = 1's row into the buffer cache, and modifies it there. Same for the new rows going into orders and transactions. At this point, the data on disk is completely unaware anything has changed. The only place these changes exist is in memory, and technically only in the database's memory — your terminal, your app, everything else has no idea either.

These modified-but-not-yet-saved pages have a name: dirty pages. It's a slightly dramatic term for "we changed this in RAM but haven't written it back yet," but it's the term you'll see in every Postgres internals doc, so it's worth knowing.

The part that actually protects you: WAL

Here's the problem this creates: if the server loses power right now, those dirty pages evaporate. RAM doesn't survive a crash. So how does a COMMIT ever mean anything?

The answer is Write-Ahead Logging, usually just called WAL. Before Postgres considers a transaction committed, it writes a compact record of exactly what changed to a log file — not the data pages themselves, just enough information to redo the change later. Something conceptually like:

TXN 101:
- users.balance: 1000 → 500
- orders: new row inserted
- transactions: new row inserted

This log entry first lands in a small WAL buffer in memory too. So at this point you might reasonably ask: if the log is also just sitting in memory, what have we actually gained?

This is the detail that took me a while to really internalize: when you run COMMIT, Postgres doesn't write your dirty data pages to disk. It writes the WAL entry to disk, and calls fsync to force the OS to physically flush it — not just hand it to a write buffer, but actually get it onto the storage device. Only once that fsync succeeds does Postgres tell your application "commit successful."

The dirty pages themselves — the actual users, orders, and transactions rows — can sit in memory for a while longer. That's the trick. Writing one small sequential log entry is much cheaper than writing several scattered data pages, so WAL lets Postgres give you durability guarantees fast without paying the cost of a full page flush on every single commit.

So the rule that actually matters is this: a commit is a promise that the WAL entry is safely on disk — not that your table's data pages are.

What crash recovery actually looks like

This log-first approach is also exactly what makes recovery possible. Two scenarios worth walking through:

The server crashes before COMMIT finishes. The WAL entry never made it to disk, and the dirty pages were only ever in memory. Both are gone. When Postgres restarts, it finds no trace of transaction 101 anywhere, so as far as the database is concerned, that transaction never happened. Your app would have seen the crash and (hopefully) never gotten a success response, so this is actually the "safe" failure — nothing inconsistent, nothing half-applied.

The server crashes after COMMIT succeeds, but before the dirty pages are flushed to disk. Now the WAL entry is safely on disk, but the actual table pages might not be yet. On restart, Postgres reads the WAL, finds transaction 101 marked complete, and replays those changes against the data files — a step called REDO. Because the log already told it exactly what changed, it doesn't need to guess or ask your application to resend anything. The database ends up in the same state it would have been in if it had flushed everything immediately, just via a slightly delayed path.

This is really the whole point of WAL: it turns "did my expensive multi-page write survive the crash" into "did my one small log entry survive the crash," which is a much easier problem to solve reliably.

Checkpoints: cleaning up so recovery stays fast

If Postgres kept every dirty page in memory forever and only ever relied on WAL replay, the log would grow without bound, and a crash after weeks of uptime could mean replaying an enormous amount of history on restart. That's where checkpoints come in.

Periodically (Postgres does this on a timer and also based on how much WAL has accumulated), it writes all the current dirty pages from memory out to disk for real, then marks a point in the WAL saying, essentially, "everything before this line is now safely reflected in the data files." Anything before that checkpoint no longer needs to be replayed during recovery — only WAL written after the last checkpoint matters.

This is the mechanism that keeps crash recovery from turning into a multi-hour ordeal on a busy production database. Without it, restart time would grow in proportion to how long the server had been running.

Putting the whole flow together

Strip away the terminology and the sequence for our order example looks like this:

The relevant rows for users, orders, and transactions are pulled into the buffer cache.
Each row gets modified in memory, becoming a dirty page.
Before commit, a WAL entry describing these changes is written.
On COMMIT, that WAL entry (not the data pages) is fsync'd to disk. The transaction is now durable.
At some point later — could be seconds, could be during the next checkpoint — the dirty pages themselves get flushed to disk.
If a crash happens in between, Postgres uses the WAL to redo whatever didn't make it to the data files yet.

A path most systems don't take: shadow paging

It's worth knowing there's an entirely different strategy some older or specialized systems use instead of WAL, called shadow paging. Rather than modifying pages in place and logging the change, the database writes an entirely new copy of the page with the changes applied, leaving the original untouched. Commit is then just a matter of atomically swapping a pointer from the old page to the new one.

It's conceptually simpler in some ways — no REDO or UNDO logic needed, since the old version is still sitting there intact until the swap happens. But it comes at a real cost: every update means duplicating a page rather than modifying it, which gets expensive fast in terms of both storage and I/O once you're dealing with high write volume. That's part of why WAL-based approaches won out in most mainstream relational databases, Postgres included.

Why this is worth actually understanding

You can build plenty of features without ever thinking about buffer pools or fsync calls. But the moment something goes sideways in production — a pod gets OOM-killed mid-transaction, a disk fills up, a deploy restarts the DB at the wrong second — knowing what's actually guaranteed at each stage is the difference between confidently saying "that data is safe" and just hoping it is.

That gap, more than anything else, is what separates someone who runs queries against a database from someone who actually understands what it's doing underneath them.

Creating a small project on Counting Using Bash Scripting

Mohd Saquib Mansoori — Sat, 22 Jul 2023 10:00:07 +0000

Title: Building a Simple Timer Project Using Bash Scripting

Introduction:
In the world of programming, mastering various languages is essential for expanding your skills and enhancing problem-solving abilities. Bash scripting is one such powerful tool that comes in handy for automating tasks, managing processes, and creating small projects. In this blog, we'll explore how to build a simple yet practical timer project using Bash scripting. Whether you're a beginner or an experienced developer, this project will not only be a fun challenge but also a great way to strengthen your Bash skills.

Prerequisites:
Before diving into the project, make sure you have basic knowledge of Bash scripting and a Linux or macOS environment. Bash is a popular shell and scripting language that comes pre-installed on most Unix-based systems.

Project Overview:
The goal of this project is to create a timer that allows users to set a specific time interval and receive a notification when the timer expires. We'll use the sleep command to achieve this, along with some basic user input and output handling in Bash.

Step 1: Getting Started
To begin, open your terminal and create a new file called timer.sh using your favorite text editor. This file will contain the Bash script that defines the timer's functionality.

Step 2: Setting the Timer
In this step, we'll ask the user to input the desired time interval for the timer. We'll use the read command to prompt the user for input and store it in a variable.

#!/bin/bash

echo "Welcome to the Bash Timer!"

# Ask the user to enter the time interval
read -p "Enter the time in seconds: " time_interval

Step 3: Timer Logic
With the user's input, we'll implement the timer logic using the sleep command. The sleep command suspends the script's execution for a specified number of seconds.

# Check if the input is a positive integer
if ! [[ "$time_interval" =~ ^[0-9]+$ ]]; then
    echo "Error: Please enter a positive integer."
    exit 1
fi

# Display a confirmation message to the user
echo "Timer set for $time_interval seconds. Press Ctrl+C to stop the timer."

# Start the timer
sleep "$time_interval"

# Timer expires - display the notification
echo "Timer expired! Time's up!"

Step 4: Make the Script Executable
Before we can run the script, we need to make it executable using the chmod command.

chmod +x timer.sh

Step 5: Run the Timer
You're now ready to run the timer! Execute the script in your terminal by typing:

./timer.sh

Conclusion:
Congratulations! You've successfully built a simple timer project using Bash scripting. This project may seem modest, but it lays the groundwork for more complex and useful applications. Feel free to expand this project by adding features like sound notifications, multiple timers, or integrating it with other scripts. With Bash scripting, the possibilities are endless!

Remember, practice is key to mastering any programming language. So, keep exploring, experimenting, and building more projects to enhance your skills and become a proficient developer.

If there is any thing wrong or is not correct, let me know.