DEV Community: Deepika Pusala

Transactions, Isolation Levels, MVCC, Locking, Triggers, Audit Logs & CAP Theorem in PostgreSQL 🔒

Deepika Pusala — Thu, 12 Nov 2026 06:36:01 +0000

🔒 Isolation, MVCC, Locking, Triggers, Audit Logging & CAP Theorem

This is one of the MOST important database topics for a Full Stack Developer. 💪

Imagine you're building an Instagram-like Social Feed. Thousands of users might like posts, comment, follow people and update profiles at the same time. Now imagine:

🛒 Two users try to buy the last available product
🏦 Two requests update the same bank account
❤️ Someone likes a post while another person deletes it
💥 A transaction fails halfway through
👀 Two people read the same data while a third is modifying it

How does the database prevent incorrect data, conflicts and inconsistent results? That's exactly what we'll learn, from absolute zero, with PostgreSQL examples and connections to FastAPI and SQLAlchemy. 🚀

📚 Table of Contents

What is a transaction?
ACID: Isolation and Durability
What is concurrency?
Dirty read
Non-repeatable read
Phantom read
Isolation levels
MVCC
Row-level locking and deadlocks
Database triggers
Audit logging
CAP theorem
ACID vs CAP
Connecting it all to FastAPI
Revision sheet
Interview questions
Quiz
Practice checklist (two-session lab included!)

1️⃣ What is a transaction?

📦 A transaction is a group of database operations treated as one logical unit of work.

Transfer ₹1,000 from Account A to Account B:

Account	Before	After
A	₹5,000	₹4,000
B	₹2,000	₹3,000

Two actions are needed: subtract from A, add to B. But what if ₹1,000 is deducted from A, the server crashes, and B never receives it? 😱 Money has disappeared. Transactions prevent this.

BEGIN                  → start transaction
  UPDATE Account A     → deduct ₹1,000
  UPDATE Account B     → add ₹1,000
COMMIT                 → save changes ✅
ROLLBACK               → undo changes ❌

BEGIN;

UPDATE accounts
SET balance = balance - 1000
WHERE id = 1;

UPDATE accounts
SET balance = balance + 1000
WHERE id = 2;

COMMIT;

If an error occurs before committing:

ROLLBACK;

Command	Meaning
`BEGIN`	Starts a transaction
`COMMIT`	Permanently saves the transaction's changes
`ROLLBACK`	Cancels uncommitted changes
`SAVEPOINT`	Creates a point inside a transaction you can partially roll back to

📍 Bonus: SAVEPOINT

BEGIN;

INSERT INTO orders (id, user_id) VALUES (1, 10);

SAVEPOINT after_order;

INSERT INTO order_items (order_id, product_id) VALUES (1, 999);  -- suppose this fails

ROLLBACK TO SAVEPOINT after_order;   -- undo only the item insert

COMMIT;                               -- the order itself is still saved

🧠 A transaction is not necessarily one SQL query. It can contain many.

2️⃣ ACID: Isolation and Durability 🧪

ACID is a set of four properties that keep transactions reliable:

⚛️ A: Atomicity. All operations succeed or none do.
✅ C: Consistency. Data follows database rules before and after.
🚦 I: Isolation. Concurrent transactions don't interfere incorrectly.
💾 D: Durability. Once committed, changes survive crashes (under the database's durability guarantees).

You likely know Atomicity and Consistency already. Today we go deep on Isolation and Durability.

2.1 🚦 Isolation: what does it mean?

Two people use the app at the same time:

User A wants to update a product price
User B wants to read that price

Isolation decides how transactions interact when they run concurrently. The goal: never show users incorrect or incomplete intermediate data.

Product 101 costs ₹500.

Transaction A:

BEGIN;

UPDATE products
SET price = 700
WHERE id = 101;
-- not committed yet!

Meanwhile, Transaction B:

SELECT price
FROM products
WHERE id = 101;

Should B see ₹500 or ₹700? 🤔 It depends on the isolation level, which is the most important part of today's lesson.

2.2 💾 Durability: what does it mean?

Once a transaction is successfully committed, its changes should survive a crash or restart.

User updates their profile name
Database executes COMMIT
Database server crashes 💥
Server restarts
The updated name is still there ✅

How? PostgreSQL uses:

📓 Write-Ahead Logging (WAL): records changes in a log before the data pages need to be written
🔁 Recovery: replays the log after a crash
🏁 Checkpoints: limit how much work recovery must replay
🔄 Storage synchronization: persists required data per configured durability settings

📓 What is WAL?

Think of WAL as a diary kept by the database. Before changes are safely reflected in the main data files, PostgreSQL writes the needed information to its log.

1. Application runs an UPDATE
2. PostgreSQL records the change in WAL
3. COMMIT is acknowledged once required WAL durability conditions are met
4. Data pages are written to their main files LATER
5. If a crash happens → PostgreSQL replays WAL during recovery

🎤 Interview point: COMMIT does not necessarily mean every modified table page has already reached its final location. WAL is what makes crash recovery possible.

⚠️ Durability also depends on correct configuration and reliable storage. For example, weakening synchronous_commit weakens the guarantee.

3️⃣ What is concurrency? 🚗🚕🚙

Concurrency means multiple transactions execute during overlapping periods of time.

Your Instagram-like app at one moment:

👤 User A likes a post
💬 User B comments on the same post
✏️ User C updates their profile
📰 User D views the home feed

All these requests hit the backend at nearly the same time, and your backend sends queries for all of them.

Why is it difficult?

Because transactions may:

Read the same row
Update the same row
Insert new rows
Delete existing rows
Depend on each other's data

Without proper concurrency control, problems occur. Let's meet the three classic read problems. 👇

4️⃣ Dirty Read 🧟

A dirty read happens when a transaction reads data modified by another transaction that has NOT committed yet.

In simple words: you read someone else's temporary, unconfirmed changes.

Price starts at ₹500.

Transaction A:

BEGIN;

UPDATE products
SET price = 900
WHERE id = 1;
-- not committed

Transaction B:

SELECT price FROM products WHERE id = 1;
-- imagine it reads 900

Transaction A then:

ROLLBACK;
-- price is back to 500

But B already used ₹900, a value that was never permanently accepted. That's a dirty read.

1. A: changes price 500 → 900 (not committed)
2. B: reads 900
3. A: ROLLBACK (price back to 500)
4. ❌ B used data that never officially existed

🐘 PostgreSQL note: dirty reads never happen in PostgreSQL. Even if you request READ UNCOMMITTED, it behaves like READ COMMITTED.

5️⃣ Non-repeatable Read 🔄

A non-repeatable read happens when the same transaction reads the same row twice and gets different values, because another transaction committed an update in between.

Price starts at ₹500.

Transaction A:

BEGIN;

SELECT price FROM products WHERE id = 1;
-- 500

Transaction B:

BEGIN;

UPDATE products
SET price = 700
WHERE id = 1;

COMMIT;

Transaction A reads again:

SELECT price FROM products WHERE id = 1;
-- 700 😮

Same transaction, same SQL, same row, different result.

Transaction A:  first read  → ₹500
Transaction B:  updates to ₹700 and COMMITS
Transaction A:  second read → ₹700   ← non-repeatable read

This matters when your app expects data to stay stable throughout a transaction.

6️⃣ Phantom Read 👻

A phantom read happens when a transaction runs the same query twice and gets a different set of rows, because another transaction inserted, deleted or changed rows matching the condition.

employees

id	department	salary
1	IT	₹50,000
2	IT	₹60,000
3	HR	₹40,000

Transaction A:

BEGIN;

SELECT * FROM employees WHERE department = 'IT';
-- Employee 1, Employee 2

Transaction B:

INSERT INTO employees (id, department, salary)
VALUES (4, 'IT', 55000);

COMMIT;

Transaction A runs the same query again:

SELECT * FROM employees WHERE department = 'IT';
-- Employee 1, Employee 2, Employee 4  👻

Employee 4 is the phantom row.

⚖️ Non-repeatable read vs phantom read

Problem	What changes?
Non-repeatable read	A previously read row's values change
Phantom read	The set of rows returned by a query changes

🧠 Non-repeatable read = same row, different values. Phantom read = different collection of matching rows.

🐘 PostgreSQL detail: Repeatable Read prevents phantom reads thanks to its snapshot-based MVCC implementation (even though the SQL standard would allow them at that level). Serializable also prevents them, and additionally prevents serialization anomalies.

7️⃣ Isolation Levels 🎚️ (the most important section!)

An isolation level decides how much protection a transaction gets from changes made by other concurrent transactions. The SQL standard defines four:

Read Uncommitted
Read Committed
Repeatable Read
Serializable

7.1 🟥 Read Uncommitted (weakest)

In theory it allows dirty reads. But PostgreSQL treats it as Read Committed, so dirty reads can't occur.

7.2 🟨 Read Committed (PostgreSQL default)

A transaction only sees data committed before the start of each individual SQL statement.

Notice: each individual statement. Every statement gets a fresh snapshot of committed data.

-- Transaction A
BEGIN;
SELECT price FROM products WHERE id = 1;   -- ₹500

-- Transaction B
UPDATE products SET price = 700 WHERE id = 1;
COMMIT;

-- Transaction A again
SELECT price FROM products WHERE id = 1;   -- ₹700 (fresh snapshot!)

Therefore:

✅ Dirty reads: prevented
⚠️ Non-repeatable reads: possible
⚠️ Phantom reads: possible
⚠️ Serialization anomalies: possible

Great for: normal web apps, social feeds, ordinary CRUD.

7.3 🟧 Repeatable Read (stronger)

The transaction keeps seeing the same snapshot from its first query onward.

-- Transaction A
BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT price FROM products WHERE id = 1;   -- 500

-- Transaction B
UPDATE products SET price = 700 WHERE id = 1;
COMMIT;

-- Transaction A again
SELECT price FROM products WHERE id = 1;   -- still 500 ✅

A continues reading its original snapshot. In PostgreSQL this uses MVCC (coming up in section 8).

Repeatable Read prevents dirty reads, non-repeatable reads and (in PostgreSQL) phantom reads. But it does not prevent every anomaly:

🩺 Write skew example: two transactions each read that at least two doctors are on duty, then each marks a different doctor off duty. Both commit, and now nobody is on duty. Only Serializable catches this.

Also, if a Repeatable Read transaction tries to update a row that another transaction changed after its snapshot, PostgreSQL raises:

ERROR: could not serialize access due to concurrent update

and your app must retry.

7.4 🟩 Serializable (strongest)

Successfully committed concurrent transactions produce an outcome equivalent to some serial (one-after-another) execution.

Two users try to buy the last product. Both check stock, both proceed, and boom, wrong result. Serializable detects dangerous interactions and may abort one transaction. The application must retry it.

BEGIN ISOLATION LEVEL SERIALIZABLE;

SELECT quantity FROM products WHERE id = 1;

UPDATE products
SET quantity = quantity - 1
WHERE id = 1;

COMMIT;

On a conflict, PostgreSQL may return:

ERROR: could not serialize access
due to read/write dependencies among transactions

You must retry the complete transaction.

🧠 Serializable does not mean transactions literally run one by one. PostgreSQL runs them concurrently and uses Serializable Snapshot Isolation (SSI) to detect dangerous conflicts.

7.5 📊 Comparison table (interview favorite!)

Isolation level	Dirty read	Non-repeatable read	Phantom read
Read Uncommitted*	Possible by the standard	Possible	Possible
Read Committed	❌ No	✅ Yes	✅ Yes
Repeatable Read (PostgreSQL)	❌ No	❌ No	❌ No
Serializable	❌ No	❌ No	❌ No

* PostgreSQL treats Read Uncommitted as Read Committed.

The table covers these three read anomalies. Serializable also prevents the broader class of serialization anomalies (like write skew).

🪜 Hierarchy

Read Uncommitted  → weakest protection
Read Committed    → prevents dirty reads
Repeatable Read   → stable transaction snapshot
Serializable      → strongest isolation guarantees

⚖️ Higher isolation generally means more conflict handling, possible retries and extra overhead. It doesn't automatically mean every query is slower.

🛠️ Setting the isolation level

-- per transaction
BEGIN ISOLATION LEVEL REPEATABLE READ;

-- or right after BEGIN
BEGIN;
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;

In SQLAlchemy:

from sqlalchemy.ext.asyncio import create_async_engine

engine = create_async_engine(
    "postgresql+asyncpg://user:pass@localhost/db",
    isolation_level="REPEATABLE READ",   # applies to the whole engine
)

# or per connection
async with engine.connect() as conn:
    conn = await conn.execution_options(isolation_level="SERIALIZABLE")

8️⃣ MVCC: Multi-Version Concurrency Control 🧬

MVCC lets multiple transactions work with different versions of the same data.

Product 1 costs ₹500. Transaction A updates it to ₹700. Transaction B wants to read it. How can PostgreSQL let both happen without unnecessarily blocking each other? Through MVCC.

Conceptually, PostgreSQL keeps different row versions:

Old row version  (visible to older snapshots)   → Product 1, ₹500
        │  Transaction A updates the price
        ▼
New row version  (visible to newer snapshots)   → Product 1, ₹700

📝 This is a simplified picture. PostgreSQL actually stores tuple versions in table storage along with transaction visibility information.

The key ideas:

👴 An older transaction can keep seeing the old committed version
👶 A newer transaction sees the newly committed version
📖 Readers generally don't block writers
✍️ Writers generally don't block ordinary readers

📸 What is a snapshot?

A snapshot is the database's view of which transactions and row versions are visible to a particular query or transaction.

A starts at 10:00
B updates a row at 10:01 and commits
A (under Repeatable Read) keeps seeing its original snapshot
A new transaction starting at 10:02 sees the updated value

🧹 Does PostgreSQL keep old versions forever?

No. A process called VACUUM cleans up obsolete row versions once they're no longer needed. Old versions consume storage, and a long-running transaction can stop PostgreSQL from cleaning some of them, causing table bloat.

💡 Developer takeaway: MVCC improves concurrency, but avoid long-running transactions. Never keep a transaction open while waiting on a user or an external API.

9️⃣ Row-Level Locking 🔐

Two people update the same product at once:

id = 1, quantity = 10

A wants 9, B wants 8. How does the database coordinate? Through locks.

🔒 A lock controls access to database resources while transactions operate on them.

9.1 🧱 What is a row-level lock?

A row-level lock protects a particular row from conflicting operations.

BEGIN;

SELECT *
FROM products
WHERE id = 1
FOR UPDATE;

This locks the selected row. Other transactions trying for conflicting locks on that row may have to wait, but they can still work on other rows.

ID	Product	Lock status
1	Laptop	🔒 Locked
2	Mouse	🟢 Available
3	Keyboard	🟢 Available

9.2 ⭐ `SELECT ... FOR UPDATE`

One of the most important commands for backend developers:

BEGIN;

SELECT quantity
FROM products
WHERE id = 1
FOR UPDATE;

UPDATE products
SET quantity = quantity - 1
WHERE id = 1;

COMMIT;

What happens:

PostgreSQL finds the row
It acquires a row-level lock
Other transactions attempting conflicting updates or row locks wait
Your transaction updates the quantity
COMMIT releases the lock

Use it for:

🏦 Banking transactions
📦 Inventory management
🎟️ Booking and ticket reservation
👛 Wallet balance updates
🧠 Any "read a row, then decide based on its value" logic

9.3 🎛️ Row-lock modes

Command	Purpose
`FOR UPDATE`	Strong lock; blocks conflicting modifications and row-locking
`FOR NO KEY UPDATE`	Weaker; useful when key columns aren't changing
`FOR SHARE`	Blocks updates/deletes while allowing other shared locks
`FOR KEY SHARE`	Protects key values against changes that affect foreign keys

As a beginner, focus on FOR UPDATE.

🎁 Bonus: `NOWAIT` and `SKIP LOCKED`

-- Fail immediately instead of waiting
SELECT * FROM products WHERE id = 1 FOR UPDATE NOWAIT;

-- Skip rows that are already locked (great for job queues!)
SELECT * FROM jobs
WHERE status = 'pending'
ORDER BY id
LIMIT 1
FOR UPDATE SKIP LOCKED;

9.4 💀 What is a deadlock?

A deadlock happens when two or more transactions wait for each other forever.

A locks Row 1
B locks Row 2
A now wants Row 2
B now wants Row 1

Transaction A: owns Row 1, waiting for Row 2
                  ▲                │
                  │                ▼
Transaction B: owns Row 2, waiting for Row 1

        🔁 circular waiting

PostgreSQL detects the deadlock and aborts one transaction with:

ERROR: deadlock detected   (SQLSTATE 40P01)

🛡️ How to prevent deadlocks

Always acquire locks in a consistent order (for example, lower account ID first)
Keep transactions short
Avoid unnecessary locks
Never wait for external API calls while holding database locks
Retry on appropriate transient errors

If every transaction locks Account A first and Account B second, you reduce the risk of opposite lock ordering.

🔟 Database Triggers ⚡

A trigger is a database mechanism that automatically executes a function when a particular event happens.

Think of it as an automatic reaction. Common events:

INSERT
UPDATE
DELETE
TRUNCATE (statement-level triggers)

Real-world example

Whenever a user changes their username, you want to record the old and new username. You could write that in every endpoint. But what if another developer updates the table directly with SQL? That code gets skipped. A trigger enforces it at the database level.

10.1 🔄 How a trigger works

Database event:   UPDATE users
        ▼
Trigger fires:    PostgreSQL detects the UPDATE
        ▼
Trigger function: records old and new values
        ▼
Audit table:      change history is stored

10.2 🧭 Types of triggers

By timing:

⏪ BEFORE: runs before the operation
⏩ AFTER: runs after the operation
🔀 INSTEAD OF: runs instead of the operation (commonly on views)

By execution level:

🧍 ROW-level: once for every affected row
📣 STATEMENT-level: once for the entire statement

UPDATE users
SET active = false
WHERE city = 'Bengaluru';

If 100 users match: a ROW trigger fires 100 times, a STATEMENT trigger fires once.

10.3 🆕🆚 `NEW` and `OLD`

OLD → the previous version of a row
NEW → the new version of a row

OLD.username = deepika
NEW.username = deepika_p

Event	Available
`INSERT`	`NEW`
`UPDATE`	`OLD` and `NEW`
`DELETE`	`OLD`

10.4 🧪 Hands-on: build an audit trigger

Step 1: the users table

CREATE TABLE users (
    id       SERIAL PRIMARY KEY,
    username VARCHAR(100),
    email    VARCHAR(255)
);

Step 2: the audit table

CREATE TABLE user_audit (
    id           SERIAL PRIMARY KEY,
    user_id      INTEGER,
    old_username VARCHAR(100),
    new_username VARCHAR(100),
    changed_at   TIMESTAMPTZ DEFAULT NOW()
);

Step 3: the trigger function

CREATE OR REPLACE FUNCTION log_username_changes()
RETURNS TRIGGER AS $$
BEGIN
    IF OLD.username IS DISTINCT FROM NEW.username THEN
        INSERT INTO user_audit (user_id, old_username, new_username)
        VALUES (OLD.id, OLD.username, NEW.username);
    END IF;

    RETURN NEW;
END;
$$ LANGUAGE plpgsql;

IS DISTINCT FROM is a NULL-safe "not equal" comparison
RETURN NEW; lets the update proceed (required in this kind of trigger)

Step 4: attach the trigger

CREATE TRIGGER username_audit_trigger
AFTER UPDATE ON users
FOR EACH ROW
EXECUTE FUNCTION log_username_changes();

Step 5: test it

INSERT INTO users (username, email)
VALUES ('deepika', 'deepika@example.com');

UPDATE users
SET username = 'deepika_p'
WHERE id = 1;

SELECT * FROM user_audit;

user_id	old_username	new_username
1	deepika	deepika_p

🎉 The backend developer never manually inserted an audit record in the UPDATE.

⚠️ Important: trigger actions generally run in the same transaction as the statement that fired them. If that transaction rolls back, the audit row rolls back too. Triggers are great for tracking changes, but they are not a substitute for tamper-proof security auditing.

To remove a trigger:

DROP TRIGGER username_audit_trigger ON users;

1️⃣1️⃣ Audit Logging 📝

Audit logging means keeping a record of important actions and changes in a system.

In your social app, you may want to know:

Who changed a username?
Who deleted a post?
Who changed account privacy?
When was an email address changed?
What was the previous value, and the new one?

User	Action	Old value	New value
101	Username update	deepika	deepika_p
102	Privacy update	Public	Private
103	Profile update	Old bio	New bio

What should an audit log contain?

audit_id
user_id
action
entity_type
entity_id
old_value
new_value
timestamp
request_id

audit_id → unique audit record ID
user_id → who performed the action
action → what happened
entity_type → which kind of resource changed
entity_id → which specific resource changed
old_value / new_value → before and after
timestamp → when
request_id → which API request caused it

⚖️ Application-level vs trigger-based logging

Application-level logging	Database trigger logging
Written in Python/FastAPI	Written in database functions
Captures business context	Captures row changes
Flexible for complex workflows	Runs automatically on configured DB events
Can be skipped if a code path misses it	Runs for applicable changes regardless of entry point
Easy to include API request info	Needs extra design to capture request context

In real systems, both are often used together.

🎁 Bonus: giving triggers request context

A trigger doesn't know which app user made the change. One approach: set a transaction-local setting from your backend, then read it inside the trigger.

-- In your app, at the start of the transaction:
SET LOCAL app.current_user_id = '101';

-- Inside the trigger function:
-- current_setting('app.current_user_id', true)

🔐 Security point: don't casually store passwords, raw auth tokens or other secrets in audit logs. Restrict access and protect sensitive logged data.

1️⃣2️⃣ CAP Theorem 🌐

Now we enter distributed systems. Don't worry, we'll start from zero.

12.1 🖥️ What is a distributed system?

One server:

Frontend (React) → Backend (FastAPI) → Database (PostgreSQL)

Now millions of users arrive. One server isn't enough, so you add more:

Users
  ▼
Load Balancer
  ├─ Server 1
  ├─ Server 2
  └─ Server 3
        ▼
Distributed database / replicas

Your application now runs across multiple machines communicating over a network. But networks fail! 💥

Server 1 can't talk to Server 2
A cable fails
A data center loses connectivity
A connection becomes extremely slow

These are network partitions, and this is where CAP becomes important.

12.2 📐 What is CAP theorem?

In a distributed data system, when a network partition occurs, you cannot guarantee both strong Consistency and Availability for every request.

CAP = Consistency, Availability, Partition tolerance.

🅲 Consistency (in CAP)

Every successful read sees the latest successfully committed write, or an appropriate failure is returned.

A user changes their username from deepika to deepika_p. Your database has Nodes A, B and C. Under strong consistency, once the update is acknowledged, any successful read from any node returns deepika_p, never the old name.

🚨 CAP consistency ≠ ACID consistency. ACID consistency is about constraints and valid states. CAP consistency is about visibility and ordering of data across distributed nodes.

🅰 Availability

Every request received by a non-failing node gets a non-error response, even if that response doesn't contain the latest data.

If Node A is unreachable but Node B can still serve the profile request, the system is available. But an available response isn't necessarily the newest value.

🅿 Partition Tolerance

The system keeps operating despite communication failures between groups of nodes.

Network group A: Node A, Node B
        ✂️  communication broken  ✂️
Network group B: Node C, Node D

The nodes are all running, but they can't talk to each other. That's a partition.

12.3 ⚖️ The CAP trade-off

When a partition occurs, you have two broad choices:

Option 1: prioritize Consistency (CP)

Don't allow potentially stale or conflicting reads
Some requests may wait or fail
Consistency over availability

Example: a strongly consistent banking operation may refuse to proceed without quorum or a valid leader.

Option 2: prioritize Availability (AP)

Keep responding to requests
Some responses may be stale
Replicas reconcile later

Example: a social feed shows a slightly older like count for a moment.

❓ "Pick any two of three?" That's a simplification. The accurate version: when a network partition occurs, a distributed system must trade off strong consistency against availability. Partition tolerance isn't something you can simply switch off, since communication failures are possible. And when there's no partition, the trade-off looks different.

12.4 🌍 Real-world examples

Use case	Typical priority
🏦 Bank balance transfers	Strong consistency
📦 Inventory reservations	Strong consistency
📰 Social media feed	Availability (staleness OK)
❤️ Like counts	Often eventual consistency
🎯 Product recommendations	Availability
🧭 Distributed coordination	Strong consistency

These are tendencies, not absolute labels for whole products. One application can use different consistency strategies for different features.

12.5 ⏳ What is eventual consistency?

If no new updates happen and communication is restored, all replicas eventually converge to the same value.

You like a post:

Primary:  Likes = 101
Replica:  Likes = 100     ← hasn't received the update yet

...a moment later...

Primary:  Likes = 101
Replica:  Likes = 101     ← converged ✅

Strong consistency	Eventual consistency
Reads reflect the latest committed write	Reads may temporarily be stale
More coordination required	Allows more independent operation
Good for financial operations	Good for feeds, analytics, some counters
May reject/delay requests during partitions	Can keep serving during partitions

In your social feed:

📝 Creating a post must reliably save
🔒 Privacy settings need careful consistency
❤️ Like counts can tolerate a small delay
🎯 Recommendations can update asynchronously

Backend developers choose guarantees based on business requirements.

1️⃣3️⃣ ACID vs CAP: don't confuse them! ⚔️

ACID	CAP
About transaction reliability	About distributed system trade-offs
Atomicity	Consistency
Consistency	Availability
Isolation	Partition tolerance
Durability	(no direct equivalent)

ACID Isolation → transactions don't interfere incorrectly
CAP Consistency → latest values observed across distributed nodes
ACID Durability → committed transactions survive failures
CAP Partition Tolerance → handling network communication failures

They are not interchangeable concepts.

1️⃣4️⃣ Connecting it all to your FastAPI project 🔌

Imagine users can reserve limited promotional slots. Only one slot is left, and two users try to reserve it simultaneously.

Without concurrency control:

User A reads available_slots = 1
User B reads available_slots = 1
User A reserves it
User B also reserves it
💥 Overbooked!

✅ Solution 1: row-level locking

BEGIN;

SELECT available_slots
FROM promotions
WHERE id = 1
FOR UPDATE;

UPDATE promotions
SET available_slots = available_slots - 1
WHERE id = 1
  AND available_slots > 0;

COMMIT;

The lock makes competing transactions wait. Also check whether the UPDATE actually affected a row. If it didn't, no slot was available.

✅ Solution 2: a single conditional UPDATE (often simplest!)

UPDATE promotions
SET available_slots = available_slots - 1
WHERE id = 1
  AND available_slots > 0
RETURNING available_slots;

This is atomic:

Row returned → reservation succeeded ✅
No row returned → no slot available ❌

A very useful backend technique. 🧠

🐍 FastAPI + SQLAlchemy version

from sqlalchemy import select


async def reserve_slot(session, promotion_id: int):
    async with session.begin():
        result = await session.execute(
            select(Promotion)
            .where(Promotion.id == promotion_id)
            .with_for_update()
        )
        promotion = result.scalar_one_or_none()

        if promotion is None:
            raise ValueError("Promotion not found")

        if promotion.available_slots <= 0:
            raise ValueError("No slots available")

        promotion.available_slots -= 1
        return promotion

Code	Meaning
`session.begin()`	Starts a transaction
`select(Promotion)`	Builds a SELECT query
`.where(...)`	Filters the requested promotion
`.with_for_update()`	Requests a row-level lock
`scalar_one_or_none()`	Gets one result or `None`
`available_slots -= 1`	Updates the value
End of `with` block	Commits on success, rolls back on exception

⚠️ session.begin() raises an error if a transaction has already started on that session (for example, a dependency already ran a query). If your project's session dependency auto-begins, use the explicit await session.commit() / await session.rollback() style instead. Stay consistent with your project's pattern.

🔁 Retrying on serialization failures and deadlocks

Serializable (and Repeatable Read) can abort a transaction by design, so your code should retry:

from sqlalchemy.exc import DBAPIError

MAX_RETRIES = 3

async def run_with_retry(session_factory, work):
    for attempt in range(1, MAX_RETRIES + 1):
        async with session_factory() as session:
            try:
                async with session.begin():
                    return await work(session)
            except DBAPIError as exc:
                # 40001 = serialization_failure, 40P01 = deadlock_detected
                code = getattr(exc.orig, "sqlstate", None) or getattr(exc.orig, "pgcode", None)
                if code in ("40001", "40P01") and attempt < MAX_RETRIES:
                    continue
                raise

💡 Retry the entire transaction, never just the failed statement.

1️⃣5️⃣ Complete revision sheet 🧾

Concept	Simple meaning
Transaction	Group of operations treated as one unit
`BEGIN`	Start a transaction
`COMMIT`	Save changes
`ROLLBACK`	Undo uncommitted changes
Isolation	Controls how concurrent transactions interact
Dirty read	Reading uncommitted changes
Non-repeatable read	Same row returns different values
Phantom read	Same query returns a different set of rows
Read Committed	Each statement sees a fresh committed snapshot
Repeatable Read	The transaction sees a stable snapshot
Serializable	Concurrent execution behaves like serial execution
MVCC	Maintains multiple row versions
Row-level locking	Locks individual rows against conflicts
Deadlock	Transactions wait on each other's locks
Trigger	Auto-runs database logic on an event
Audit logging	Records who changed what and when
WAL	Log used for crash recovery
Durability	Committed data survives crashes (under configured guarantees)
Distributed system	Multiple machines working together
CAP theorem	Consistency vs Availability during partitions
Eventual consistency	Replicas eventually converge

1️⃣6️⃣ Important interview questions 🎤

Q1. Read Committed vs Repeatable Read?
Read Committed takes a new snapshot for every statement. Repeatable Read (in PostgreSQL) uses a stable snapshot for the whole transaction.

Q2. What is a dirty read?
Reading data modified by another transaction before it commits.

Q3. Non-repeatable vs phantom reads?
Non-repeatable reads involve changed values in an existing row. Phantom reads involve changes in the set of rows matching a query.

Q4. What is MVCC?
Multi-Version Concurrency Control. It lets transactions access different versions of rows, improving concurrency between readers and writers.

Q5. What is SELECT FOR UPDATE?
It acquires row-level locks on selected rows so competing transactions can't perform conflicting operations on them until the lock is released.

Q6. What is a deadlock?
Transactions waiting for locks held by each other, so none can progress. The database detects it and aborts one.

Q7. What is a database trigger?
A mechanism that automatically executes a trigger function when a specified database event occurs.

Q8. What is audit logging?
Recording important changes and actions: who did them, when, and what changed.

Q9. What is WAL?
Write-Ahead Logging. PostgreSQL records changes in a log to support crash recovery and durability.

Q10. Explain CAP theorem.
During a network partition, a distributed system must choose between guaranteeing strong consistency for successful requests and guaranteeing availability for every request.

Q11. What is eventual consistency?
Replicas may temporarily differ but converge once updates stop and synchronization succeeds.

Q12. ACID consistency vs CAP consistency?
ACID consistency means transactions preserve database rules and constraints. CAP consistency means reads across distributed nodes respect the latest successful write under the chosen guarantee.

Q13. What is write skew, and which isolation level prevents it?
Two transactions read overlapping data and then update different rows based on what they read, producing an invalid combined result. Only Serializable prevents it.

Q14. Should you retry a failed Serializable transaction?
Yes. Retry the entire transaction, since serialization failures are an expected part of the model.

1️⃣7️⃣ Test your understanding! 🧪

Answer without scrolling up. (Answers are at the bottom.)

1. Transaction A updates a price but hasn't committed. Transaction B reads that value. What is this?

A) Dirty read
B) Phantom read
C) Deadlock
D) Durability

2. Transaction A reads the same row twice but sees ₹500 and then ₹700 after another transaction commits. What is this?

A) Dirty read
B) Non-repeatable read
C) Phantom read
D) Deadlock

3. Which isolation level provides a stable transaction snapshot in PostgreSQL?

A) Read Uncommitted
B) Read Committed
C) Repeatable Read
D) None

4. Which SQL command locks selected rows against conflicting modifications?

A) SELECT DISTINCT
B) SELECT FOR UPDATE
C) SELECT COUNT
D) SELECT LIMIT

5. What happens to trigger-generated audit rows if the triggering transaction rolls back?

A) They remain permanently
B) They are automatically emailed
C) They are rolled back too
D) Nothing happens

6. Which PostgreSQL mechanism maintains multiple row versions?

A) WAL
B) MVCC
C) CAP
D) JSON

7. During a network partition, what trade-off does CAP describe?

A) Atomicity vs Durability
B) Consistency vs Availability
C) Speed vs Storage
D) SQL vs NoSQL

8. What does WAL primarily help PostgreSQL achieve?

A) Frontend rendering
B) Password hashing
C) Crash recovery and durability
D) API routing

👀 Click to reveal answers

A: dirty read
B: non-repeatable read
C: Repeatable Read
B: SELECT FOR UPDATE
C: they roll back too (same transaction)
B: MVCC
B: Consistency vs Availability
C: crash recovery and durability

1️⃣8️⃣ What to practice as a Full Stack Developer 🛠️

Practice in PostgreSQL using pgAdmin or psql, with two query windows open side by side.

✅ Practical checklist

[ ] Create two transactions and demonstrate COMMIT and ROLLBACK
[ ] Reproduce a non-repeatable read using two sessions
[ ] Compare Read Committed and Repeatable Read using two sessions
[ ] Try SELECT FOR UPDATE with two simultaneous transactions
[ ] Create a deadlock intentionally and observe PostgreSQL's error
[ ] Create a PostgreSQL trigger for username changes
[ ] Create an audit table and verify old/new values
[ ] Observe WAL and understand how PostgreSQL performs recovery
[ ] Explain CP and AP with a real-world example

🧪 Mini lab: non-repeatable read (Read Committed)

-- Setup (run once)
CREATE TABLE products (id INT PRIMARY KEY, price INT);
INSERT INTO products VALUES (1, 500);

Step	Session 1	Session 2
1	`BEGIN;`
2	`SELECT price FROM products WHERE id = 1;` → 500
3		`UPDATE products SET price = 700 WHERE id = 1;` (autocommits)
4	`SELECT price FROM products WHERE id = 1;` → 700 😮
5	`COMMIT;`

🧪 Mini lab: same test under Repeatable Read

Reset the price to 500, then:

Step	Session 1	Session 2
1	`BEGIN ISOLATION LEVEL REPEATABLE READ;`
2	`SELECT price FROM products WHERE id = 1;` → 500
3		`UPDATE products SET price = 700 WHERE id = 1;`
4	`SELECT price FROM products WHERE id = 1;` → still 500 ✅
5	`COMMIT;` then `SELECT` again → 700

🧪 Mini lab: `FOR UPDATE` blocking

Step	Session 1	Session 2
1	`BEGIN;`
2	`SELECT * FROM products WHERE id = 1 FOR UPDATE;`
3		`BEGIN;`
4		`UPDATE products SET price = 999 WHERE id = 1;` → hangs, waiting ⏳
5	`COMMIT;`	Session 2's update now proceeds
6		`COMMIT;`

🧪 Mini lab: create a deadlock 💀

INSERT INTO products VALUES (2, 800);

Step	Session 1	Session 2
1	`BEGIN;`	`BEGIN;`
2	`UPDATE products SET price = 1 WHERE id = 1;`
3		`UPDATE products SET price = 2 WHERE id = 2;`
4	`UPDATE products SET price = 3 WHERE id = 2;` → waits ⏳
5		`UPDATE products SET price = 4 WHERE id = 1;` → 💥 deadlock detected

PostgreSQL aborts one of them. Run ROLLBACK; in the aborted session. Then think: how would consistent lock ordering have avoided this?

🎯 Final thoughts

💡 My biggest advice: don't just memorize the four isolation levels. Run the same scenario in two query windows. Watching Transaction A and Transaction B happen at the same time makes isolation, locking and concurrency click.

Connect it to your stack:

⚡ FastAPI handles incoming concurrent requests
🧪 SQLAlchemy manages database transactions
🐘 PostgreSQL handles isolation, MVCC, locks and triggers
📓 WAL helps with durability
🌐 Distributed architecture introduces CAP trade-offs

Together, these concepts form a major part of backend engineering. 🏆

🙌 Enjoyed this post?

❤️ Drop a reaction
🦄 Save it for revision
💬 Comment with questions, and I'll answer them!

Happy coding! 👩‍💻✨

postgres #sql #database #backend #concurrency

CQRS & Event-Driven Thinking: Commands, Queries, Domain Events & the Outbox Pattern Explained 📨

Deepika Pusala — Thu, 12 Nov 2026 06:32:07 +0000

📨 CQRS & Event-Driven Thinking: Let's Learn from ZERO!

These are important backend architecture concepts, especially with FastAPI, PostgreSQL, SQLAlchemy, repositories, services and a social media app. We'll cover:

🔀 What Command-Query Separation (CQS) means
🧩 What CQRS is and how it differs from CQS
⚔️ Commands vs queries
📣 What domain events are
🌐 How event-driven architecture works
🏢 Real examples using an Instagram-like app
✅ Advantages, disadvantages, use cases and interview questions

We'll start simple and gradually move towards real backend code. 🚀

1. 🤔 First, Understand the Problem

You're building an Instagram-like app. A user can:

Create a post
Like a post
Comment on a post
Follow another user
View their home feed
Search for posts
View someone's profile

All of these touch the database. But are they all doing the same kind of work? No! There are two fundamentally different kinds of operations:

Kind	Examples	Effect
✍️ Writing / changing data	Create a post, like a post, delete a comment, follow a user	Changes the database
👀 Reading / retrieving data	Get all posts, view comments, get a profile, search posts	Retrieves information without changing it

This distinction is the foundation of Command-Query Separation.

2. 🔀 Command-Query Separation (CQS)

Command: an instruction to perform an action
Query: a request to retrieve information
Separation: keeping these two responsibilities apart

📌 Command-Query Separation means separating operations that change data from operations that only read data. A command changes the state of the system. A query returns information but does not change the state.

State means the current stored condition of your application. For example, a post:

{
  "id": 101,
  "caption": "My first post",
  "likes_count": 10
}

Its current state: the post has 10 likes.

2.1 ✍️ Command example: like a post

The frontend sends POST /posts/101/like. The backend:

Identifies the logged-in user
Checks the post exists
Checks the user hasn't already liked it
Inserts a like record
Updates the like count, if your design stores one

Before:  { "id": 101, "likes_count": 10 }
After:   { "id": 101, "likes_count": 11 }

The database state changed, so this is a Command.

2.2 👀 Query example: get post details

The frontend sends GET /posts/101. The backend finds the post, retrieves its caption, author and like count, and returns:

{
  "id": 101,
  "caption": "My first post",
  "likes_count": 11
}

Did we change anything? No. So this is a Query.

2.3 📊 Visual comparison

✍️ COMMAND (write)	👀 QUERY (read)
Changes application state	Does not change application state
Create, Update, Delete, Like	Get, List, Search, Retrieve

🧠 Command = change something. Query = ask for something.

2.4 📋 More examples

Operation	Type	Reason
Create post	Command	Inserts data
Edit caption	Command	Updates data
Delete post	Command	Removes data
Like post	Command	Changes like state
Follow user	Command	Changes follow relationship
Get profile	Query	Reads data
Get feed	Query	Retrieves posts
Search users	Query	Reads matching records
Get comments	Query	Retrieves comments

📝 A query can have internal work such as logging or caching, but its purpose must not be to change the business data being queried.

3. 🤷 Why Do We Need CQS?

Imagine this function:

def get_post(post_id):
    post = database.get_post(post_id)

    if post.likes_count == 0:
        post.likes_count = 1
        database.save(post)

    return post

What's wrong? It's named get_post(), so we expect it to retrieve a post. But it also changes the database! Someone calling get_post() might accidentally modify application data. A better design:

def get_post(post_id):
    return database.get_post(post_id)


def like_post(post_id):
    post = database.get_post(post_id)
    post.likes_count += 1
    database.save(post)
    return post

get_post() → Query
like_post() → Command

✅ Main benefit: your code becomes easier to understand, maintain, test and debug.

4. 🧩 What is CQRS?

📌 CQRS = Command Query Responsibility Segregation. It's an architectural pattern based on the same idea as CQS.

CQS is a principle about separating read and write operations
CQRS takes that further by letting the read and write sides have different models, responsibilities, and sometimes different data stores

4.1 🏛️ Traditional architecture

Frontend (React)
     ▼
Backend API (FastAPI)
     ▼
PostgreSQL Database   ← same database for reading and writing

Writing means insert/update/delete, reading means select. This is completely valid and works very well for many apps. But if your app becomes extremely popular (1 million users, 10 million posts, millions of feed requests, thousands of new posts and likes every minute) the database must handle heavy reads and writes together. That's where CQRS can become useful.

4.2 🧱 CQRS architecture

                    Frontend (React)
                 ┌────────┴────────┐
              Command            Query
          (write request)    (read request)
                 ▼                  ▼
         Command Handler      Query Handler
   (validates and processes   (processes read
            writes)              requests)
                 ▼                  ▼
          Write Model          Read Model
          (PostgreSQL)    (optimized for retrieval)

(Conceptual. Read and write models may use the same database or different ones.)

Write side: handles commands, validates business rules, changes application state, saves data.

Read side: handles queries, retrieves information, returns data in a format suitable for the frontend, and can be optimized for fast reads.

4.3 🗄️ Does CQRS require two databases?

No! A very important interview question. There are two common implementations:

Option A: same database, separate models. Command and query logic are separate, but they share the same underlying database. For example, the command handler uses the SQLAlchemy ORM to create a post, while the query handler uses optimized SQL to retrieve posts.

Option B: separate read and write databases. Write model: PostgreSQL. Read model: Redis, Elasticsearch, or a separate PostgreSQL database. The read database may hold precomputed or denormalized information for faster retrieval. Use this when you need independent scaling or specialized read storage.

4.4 📖 What is a Read Model?

📌 A read model is a representation of data designed specifically for reading.

Suppose your database has:

users:

id	username
1	deepika
2	arjun

posts:

id	user_id	caption
101	1	My first post
102	2	Weekend vibes

likes:

id	post_id	user_id
1	101	2
2	101	1

To display a feed you may need to join several tables and count likes. Your frontend wants:

[
  {
    "post_id": 101,
    "username": "deepika",
    "caption": "My first post",
    "likes_count": 2
  }
]

This combined structure is a read model. It's shaped for the frontend's needs rather than matching your original tables. Instead of repeating expensive joins and calculations on every feed request, you can maintain a read-optimized representation, for example a precomputed like count or a denormalized feed projection.

4.5 ✍️ What is a Write Model?

The write model is the representation used to perform and enforce changes. When a user likes a post, the write side must ensure the post exists, the user exists, the user hasn't already liked it, and the like is stored correctly. It maintains business rules and data integrity.

Write Model	Read Model
Handles changes	Handles retrieval
Processes commands	Processes queries
Enforces business rules	Optimizes presentation
Focuses on consistency	Focuses on efficient reads

4.6 📍 When should we use CQRS?

Read-heavy apps: millions of users frequently retrieve information
Complex queries: many joins, aggregations and calculations
Different scaling needs: read and write traffic scale independently
Different data representations: the frontend needs data structured differently from the normalized write database
Complex business workflows: many operations and rules affect the same domain

When should you avoid it? For a small CRUD app, CQRS adds unnecessary complexity. You'd maintain two models, command handlers, query handlers, event processing, synchronization, and error handling for failed projections. A normal layered FastAPI app is often enough.

⚠️ CQRS is an architectural choice, not a requirement for every backend.

5. 📣 What is Event-Driven Thinking?

A user creates a post and the backend saves it. After that, several other things may need to happen:

The post should appear in the user's feed
Followers may receive notifications
The profile's post count may increase
Analytics may record the new post
An image-processing service may generate thumbnails

Should the post-creation function directly perform all of these? It could, but if the notification service is slow or temporarily unavailable, should the user wait for it before their request finishes? Probably not. That's where event-driven thinking helps.

5.1 📌 What is an Event?

📌 An event represents something that has already happened.

Examples: UserRegistered, PostCreated, PostLiked, CommentAdded, UserFollowed, OrderPlaced, PaymentCompleted.

Notice the past tense. An event is a completed fact. PostCreated means a post has already been created. It doesn't mean someone is requesting creation. That distinction is extremely important.

5.2 ⚔️ Command vs Event

✍️ Command	📣 Event
`CreatePost`: "Please create a new post."	`PostCreated`: "A new post has been successfully created."
An instruction that requests something happen	A fact that announces something happened

Command	Event
`RegisterUser`	`UserRegistered`
`CreatePost`	`PostCreated`
`LikePost`	`PostLiked`
`AddComment`	`CommentAdded`
`FollowUser`	`UserFollowed`
`PlaceOrder`	`OrderPlaced`
`MakePayment`	`PaymentCompleted`

🧠 Command = request to do something. Event = notification that something has happened.

5.3 🌐 How does event-driven architecture work?

1. User creates a post          → React frontend
        ▼
2. CreatePost command           → backend receives the instruction
        ▼
3. Command Handler              → validates and saves the post
        ▼
4. PostCreated event            → a fact is published after successful creation
        ▼
5. Event Bus / Event Broker     → distributes the event to interested consumers
        ├── 🔔 Notifications
        ├── 🖼️ Media
        └── 📊 Analytics

One event can trigger several independent reactions.

A. Event Producer: the component that creates and publishes an event. Here, the post creation service saves the post and publishes PostCreated.

B. Event: carries information about what happened:

{
  "event_id": "evt_001",
  "event_type": "PostCreated",
  "post_id": 101,
  "user_id": 1,
  "created_at": "2026-11-12T10:30:00Z"
}

(An illustrative payload.) It tells other components which event occurred, which post was created, which user created it and when. An event should contain enough information for consumers to do their work, without unnecessarily exposing private data.

C. Event Bus / Event Broker: a communication system that receives events from producers and delivers them to interested consumers. Technologies include RabbitMQ, Apache Kafka, Redis Streams, AWS EventBridge. For a small app you can even use a simple in-process event dispatcher with no broker at all.

D. Event Consumer: a component that receives and reacts to an event. When PostCreated occurs, the notification consumer sends notifications, the analytics consumer records the activity, and the media consumer processes images. Each has its own responsibility.

🔗 The event producer doesn't need to know the internal implementation of every consumer, which reduces coupling.

6. 🏷️ What are Domain Events?

📌 A domain event is an event that represents something meaningful that happened in the application's business domain.

A domain is the business area or problem your app is built to handle:

Application	Domain
Instagram-like app	Social networking
Banking app	Banking
E-commerce app	Online shopping
Hospital app	Healthcare management
Food delivery app	Food ordering and delivery

In a social media app the domain includes users, posts, comments, likes, follows and notifications. When a user follows another user and the operation succeeds, we can represent that important business fact as the domain event UserFollowed.

6.1 📋 Examples of domain events

Social media:

Domain Event	Meaning
`UserRegistered`	A new account was registered
`ProfileUpdated`	A profile was updated
`PostCreated`	A new post was created
`PostDeleted`	A post was deleted
`PostLiked`	A user liked a post
`CommentAdded`	A comment was added
`UserFollowed`	A user followed another user
`UserUnfollowed`	A user unfollowed another user

E-commerce:

Domain Event	Meaning
`ProductAddedToCart`	A product was added
`OrderPlaced`	An order was placed
`PaymentCompleted`	Payment succeeded
`OrderCancelled`	An order was cancelled
`ItemShipped`	An item was shipped

6.2 ⚖️ Domain event vs command vs query

Concept	Meaning	Example
Command	Request to perform an action	`LikePost`
Query	Request to retrieve data	`GetPost`
Domain Event	Something meaningful happened	`PostLiked`

A complete sequence:

A user clicks Like
The backend receives LikePost
The backend checks the rules
The backend saves the like
The backend generates PostLiked
Other components react to PostLiked

LikePost (Command)
     ▼
Business logic + database write
     ▼
PostLiked (Domain Event)
     ▼
Event consumers: 🔔 Notification · 📊 Analytics

The command asks the system to do something. The event records that the system successfully did it.

6.3 🤔 Are domain events always published to Kafka or RabbitMQ?

No. A domain event is a concept. It can exist entirely inside your application. For example, your Python app can create a PostCreated event object and pass it to an internal event dispatcher. You only need Kafka, RabbitMQ or another broker when your architecture requires the capabilities they provide.

7. ⏳ Synchronous vs Asynchronous Event Processing

7.1 🐢 Synchronous processing

The caller waits for the operation to finish before continuing:

def create_post(post_data):
    post = save_post(post_data)
    send_notification(post)
    update_analytics(post)
    return post

Save post
Wait for notification
Wait for analytics
Return response

If notification takes 3 seconds and analytics takes 2, the request may take about 5 seconds longer than the database save itself. The user waits for all of it.

7.2 🐇 Asynchronous event processing

def create_post(post_data):
    post = save_post(post_data)

    publish_event(
        "PostCreated",
        {"post_id": post.id},
    )

    return post

Event consumers process their tasks independently (notifications, analytics, image processing), and the request doesn't have to wait for them.

Synchronous:   Save post → Send notification → Update analytics → Return response (after all)

Asynchronous:  Save post → Publish event → Return response
                              ├── Notification
                              └── Analytics   (processed independently)

📝 Asynchronous doesn't automatically mean a separate server or message broker. Python background tasks, queues and distributed brokers are different implementation options. For reliable processing across services, a durable message broker is often preferable to an in-memory background task.

7.3 🕐 What is Eventual Consistency?

A user creates a post. The write database saves it immediately, but your separate read database or feed projection may take a moment to receive and process PostCreated. For a brief moment the write database says the post exists, while the read database hasn't caught up. After the event is processed, both agree.

Time T1 (write completed)   Write DB: Post exists (ID 101)   Read DB: Post not yet visible
Time T2 (event processed)   Write DB: Post exists (ID 101)   Read DB: Post exists (ID 101)

📌 Eventual consistency: the system may temporarily show slightly outdated information, but if processing succeeds and continues, the read side eventually catches up.

For analytics, recommendation updates and notification delivery, a small delay is often acceptable. But for critical operations such as checking an account balance before a withdrawal, you generally need strong consistency at the point of decision.

8. 💥 What Happens If Event Processing Fails?

Your app saves a post in PostgreSQL, then tries to publish PostCreated to RabbitMQ, but RabbitMQ is temporarily down. The post exists, yet the event was never published, so the notification service may never learn about it. That's a reliability problem. One important solution is the Transactional Outbox Pattern.

8.1 📤 Transactional Outbox Pattern

Instead of saving business data and publishing the event as two unrelated operations, save both the business data and an event record in the same database transaction.

posts:

id	caption
101	My first post

outbox_events:

id	event_type	payload	published
1	PostCreated	`post_id:101`	false

The process:

Start a database transaction
Insert the post
Insert the PostCreated event into the outbox table
Commit the transaction
A background worker reads unpublished events
The worker publishes them to the broker
The worker marks them as published after successful delivery

BEGIN TRANSACTION
  Insert post
  Insert event
COMMIT              ← both records saved together
     ▼
Background Worker
     ▼
Message Broker

Even if the broker is down, the event remains safely stored in PostgreSQL and can be published later.

⚠️ Caveat: a worker can publish an event and crash before marking it as published, so the event may be delivered again. That's why consumers should be idempotent.

8.2 🔁 What is Idempotency?

📌 Idempotency means performing the same operation multiple times produces the same intended result as performing it once.

Suppose PostLiked is delivered twice. If your consumer blindly increments the like count each time, the count goes up twice. That's wrong. A safer design:

Store processed event IDs
Check whether an event was already processed
If yes, skip it
If not, process it and record its ID

9. ⚡ How Does This Fit into FastAPI Architecture?

9.1 🥞 Traditional layered architecture

For POST /posts:

frontend → FastAPI Router → Post Service → Post Repository → PostgreSQL

The router receives the request, the service implements business logic, and the repository handles database operations.

9.2 🧩 Introducing CQRS

Separate the write and read responsibilities:

                     FastAPI
              ┌─────────┴─────────┐
        Command Router       Query Router
              ▼                   ▼
       Command Handler       Query Handler
              ▼                   ▼
       Write Repository      Read Repository
              ▼                   ▼
          Database            Database

Command handler:

class CreatePostCommand:
    def __init__(self, user_id, caption):
        self.user_id = user_id
        self.caption = caption


def handle_create_post(command, repository):
    post = repository.create(
        user_id=command.user_id,
        caption=command.caption,
    )

    return post

Query handler:

def handle_get_post(post_id, repository):
    return repository.get_by_id(post_id)

(Simplified examples, not a complete production implementation.)

CreatePostCommand represents the instruction
handle_create_post() processes the write
handle_get_post() retrieves information

You can have separate command and query handlers even when both use the same SQLAlchemy session and PostgreSQL database.

9.3 📣 Introducing domain events

Extend the command handler:

def handle_create_post(command, repository, event_publisher):
    post = repository.create(
        user_id=command.user_id,
        caption=command.caption,
    )

    event = {
        "event_type": "PostCreated",
        "post_id": post.id,
        "user_id": command.user_id,
    }

    event_publisher.publish(event)

    return post

CreatePost Command
     ▼
Command Handler
     ▼
Post Repository
     ▼
PostgreSQL
     ▼
PostCreated Event
     ▼
Event Publisher
     ▼
Event Consumers

⚠️ For production reliability, you'd normally use the transactional outbox pattern instead of relying on a direct database-write-then-publish sequence.

10. 🆚 Domain Events vs Integration Events

Related, but not identical.

Domain event: represents an important business fact inside a particular domain. It might be used by your app's internal business logic.

PostCreated(
    post_id=101,
    user_id=1,
)

Integration event: communicates something that happened to another system or independent service. It might be published through RabbitMQ or Kafka so another service can consume it.

{
  "event_type": "PostCreated",
  "post_id": 101,
  "user_id": 1
}

Domain Event	Integration Event
Internal to a domain	Shared across service boundaries
Represents business facts	Communicates business facts to other systems
Can be an internal Python object	Usually has a serialized message format
May never leave the application	Intended for external consumers

A domain event can lead to an integration event, but not every domain event needs to be published externally.

11. ⚔️ CQRS vs Event-Driven Architecture vs Event Sourcing

These three are often discussed together but solve different problems.

🅰️ CQRS. Separate reading and writing responsibilities. CreatePost → command, GetPost → query. Focus: how your app handles reads and writes.

🅱️ Event-Driven Architecture (EDA). Components communicate and react to events. PostCreated triggers notifications, analytics and image processing. Focus: how components communicate and respond to changes.

🅲 Event Sourcing. A different concept. Don't confuse it with EDA. Normally your database stores the current state:

{
  "account_id": 1,
  "balance": 700
}

With event sourcing, instead of treating current state as the primary record, you store the sequence of events that produced it:

Event	Amount
`AccountCreated`	0
`MoneyDeposited`	+1000
`MoneyWithdrawn`	-200
`MoneyWithdrawn`	-100

The balance is calculated by replaying the events: 1000 − 200 − 100 = 700.

Traditional database:  Current balance = ₹700
Event sourcing:        Deposited ₹1,000 → Withdrawn ₹200 → Withdrawn ₹100 → Current balance = ₹700

Why use it? Complete history of changes, ability to reconstruct previous states, audit trails, and understanding exactly how a value changed over time.

But it adds complexity: event versioning, event replay, data migration, handling duplicate events and maintaining projections. You don't need event sourcing just because you're using events.

Concept	Main responsibility
CQS	Separate reads and writes at the operation level
CQRS	Separate read and write models/responsibilities
Event-Driven Architecture	Communicate through events
Domain Events	Represent important business facts
Event Sourcing	Store state changes as an event history

🎯 Very important: CQRS and EDA can be used together, but neither requires Event Sourcing.

12. 🧵 Practical Example: A Social Media App

Imagine Deepika likes Arjun's post.

Step 1: the frontend sends a command

POST /posts/101/like
Authorization: Bearer access_token

Step 2: FastAPI receives the request

@router.post("/posts/{post_id}/like")
async def like_post(
    post_id: int,
    current_user=Depends(get_current_user),
):
    return await service.like_post(
        post_id,
        current_user.id,
    )

Step 3: the service processes the command. It checks the post exists, checks whether the user already liked it, applies business rules, and creates the like record.

Step 4: the database saves the change

likes
id | post_id | user_id
1  | 101     | 1

Step 5: a domain event is generated

{
  "event_type": "PostLiked",
  "post_id": 101,
  "user_id": 1
}

Step 6: event consumers react. The notification service informs the post owner, analytics records the interaction, and a read projection updates the like count, if the app maintains one.

Step 7: the user retrieves the post

GET /posts/101

This is a Query. The query handler returns:

{
  "post_id": 101,
  "caption": "My first post",
  "likes_count": 15,
  "liked_by_me": true
}

This one example demonstrates CQS, CQRS, domain events, event-driven architecture, event consumers and eventual consistency.

📝 If liked_by_me must update immediately, read it from the authoritative write-side data, or otherwise ensure the response reflects the successful write. Don't assume an asynchronously updated read model is immediately current.

13. 📖 Important Terms

Term	Beginner-friendly meaning
Command Handler	Processes a command
Query Handler	Processes a query
Write Model	Handles data modifications
Read Model	Represents data for retrieval
Event Producer	Creates and publishes events
Event Consumer	Receives and processes events
Event Bus	Distributes events
Event Broker	Messaging infrastructure for event delivery
Domain Event	Important business occurrence
Event Payload	Data contained in an event
Eventual Consistency	Different data representations become consistent over time
Idempotency	Repeating an operation doesn't produce duplicate effects
Event Sourcing	Stores events as the primary history of state changes
Transactional Outbox	Reliably stores events alongside database changes
Message Queue	Holds messages until consumers process them

14. 🧭 When Do These Concepts Work Together?

Situation	Useful concept
Separating `get_post()` and `create_post()`	CQS
Different read and write models	CQRS
Sending notifications after a post is created	Event-driven architecture
Representing a successful follow action	Domain events
Updating a read database asynchronously	CQRS + events
Keeping event publication reliable	Transactional outbox
Preventing duplicate message processing	Idempotency
Keeping complete historical state changes	Event sourcing

15. 🎤 Interview Questions and Answers

Q1. What is Command-Query Separation?
A design principle that separates operations into Commands, which modify application state, and Queries, which retrieve information without modifying business state.

Q2. What is CQRS?
Command Query Responsibility Segregation: an architectural pattern that separates read and write responsibilities, potentially using different models and data stores.

Q3. What's the difference between CQS and CQRS?
CQS is a principle applied at the operation or method level. CQRS is an architectural pattern that applies the separation more broadly to the application's read and write models.

Q4. What is a Domain Event?
Something important that has already happened in the business domain, such as PostCreated, PaymentCompleted or UserRegistered.

Q5. What's the difference between a Command and an Event?
A Command requests an action. An Event represents a fact that the action has happened.

Q6. What is Event-Driven Architecture?
An approach in which components communicate by publishing and consuming events. It helps decouple components and allows independent processing.

Q7. What is an Event Broker?
Infrastructure that receives and distributes messages between producers and consumers, such as RabbitMQ and Kafka.

Q8. What is Eventual Consistency?
Different data representations may temporarily contain different information but eventually become consistent after updates are processed.

Q9. What is the Transactional Outbox Pattern?
A reliability pattern that stores business data and the corresponding event record in the same database transaction. A background worker later publishes the event.

Q10. Does CQRS require two databases?
No. CQRS can use the same database with separate read and write models. Separate databases are optional.

Q11. Is Event-Driven Architecture the same as Event Sourcing?
No. EDA uses events for communication between components. Event sourcing stores events as the primary record of state changes.

Q12. What are the disadvantages of CQRS?
Architectural complexity, additional code, synchronization challenges, potential eventual consistency, and more difficult debugging.

16. 🧪 Test Your Understanding!

Try these before peeking at the answers! ❤️

A user requests their profile details. What is this?
- A) Command
- B) Query
- C) Domain Event
A user clicks the Like button. What is the incoming instruction?
- A) PostLiked
- B) LikePost
- C) GetPost
The backend successfully saves a post. What event can represent this?
- A) CreatePost
- B) GetPost
- C) PostCreated
A post creation event triggers notifications and analytics independently. Which architecture is this?
- A) Event-Driven Architecture
- B) Event Sourcing
- C) Normalization
Does CQRS always require two databases?
- A) Yes
- B) No
What helps reliably publish events after a database transaction?
- A) B-tree index
- B) Transactional Outbox
- C) JWT
What does Eventual Consistency mean?
- A) Data never changes
- B) All systems update at exactly the same time
- C) Data representations may temporarily differ but eventually catch up

👀 Click to reveal answers

B: Query. It reads data without changing it
B: LikePost is the command. PostLiked is the event that follows
C: PostCreated. Events are past-tense facts
A: Event-Driven Architecture
B: No. The same database with separate models is a valid CQRS implementation
B: Transactional Outbox
C: Representations may temporarily differ but eventually catch up

17. 🏁 Remember These Five Things

🔀 CQS = separate reading and writing operations
🧩 CQRS = separate read and write responsibilities/models
✍️ Command = an instruction to perform an action
📣 Domain Event = something meaningful that happened
🌐 EDA = components react to events and communicate through them

If this post helped you, drop a ❤️ or 🦄 and share it with a friend learning backend architecture. Happy coding! 👩‍💻✨

PostgreSQL Query Optimization: Indexes, EXPLAIN ANALYZE, B-tree vs GIN vs GiST & the N+1 Problem 🚀

Deepika Pusala — Thu, 12 Nov 2026 06:26:09 +0000

🚀 PostgreSQL Query Optimization: Let's Learn from ZERO!

Imagine your social media app has only 30 posts today. Everything is fast. Now imagine it grows to:

👥 10,000 users
📝 1 million posts
❤️ 10 million likes
💬 Millions of comments

Suddenly some API requests become slow, the database takes longer to retrieve data, and your server does more work than necessary.

📌 Query optimization is about making database operations faster and more efficient without changing the correctness of the results.

We'll learn everything from zero, using PostgreSQL and examples from a social-feed project. 🎯

1. 🤔 What Does Query Optimization Mean?

Take a posts table:

id	user_id	caption
1	101	My first post
2	102	Beautiful day
3	101	Weekend vibes
4	103	Coding time
5	101	Hello world

You run:

SELECT *
FROM posts
WHERE user_id = 101;

You want all posts by user 101. But how does the database find them?

It can inspect every row one by one
It can use an index to find matching rows more directly

The second approach can be much faster for suitable queries. That's where indexing and query optimization come in.

1.1 📨 What is a database query?

A query is simply a request sent to the database:

SELECT * FROM users;                      -- get all users
SELECT * FROM posts WHERE id = 10;        -- get post 10
SELECT * FROM likes WHERE post_id = 25;   -- get all likes of post 25

Every time your FastAPI backend needs data, it may execute queries:

User opens the app
      ▼
React frontend requests /feed
      ▼
FastAPI receives the request
      ▼
Service layer executes a database query
      ▼
PostgreSQL retrieves matching posts
      ▼
FastAPI returns a JSON response
      ▼
React displays posts

If the database query is slow, the API response may also become slow.

⚠️ Query optimization isn't only about writing shorter SQL. It's about reducing unnecessary database work.

2. 📚 What is a Database Index?

This is the foundation of the whole topic. Imagine a 1,000-page textbook and you want the topic "Normalization".

Without an index: you start at page 1, then page 2, page 3... until you find it on page 650. That's like a Sequential Scan
With an index: the back of the book says Normalization → Page 650. You go straight there. That's like an indexed lookup

Normalization  → Page 650
Transactions   → Page 720
Primary Key    → Page 100
SQL Joins      → Page 250

📌 A database index is an additional data structure that helps PostgreSQL locate rows more efficiently.

CREATE INDEX idx_posts_user_id
ON posts(user_id);

This creates an index on user_id. Now this query can potentially use it:

SELECT *
FROM posts
WHERE user_id = 101;

Without an index:

Check row 1 → Check row 2 → Check row 3 → Check row 4 → ...

With an index:

Search the index
   ▼
Find user_id = 101
   ▼
Locate matching row positions
   ▼
Retrieve matching rows

🧠 Important things to remember

An index is separate from the actual table data
An index helps speed up certain SELECT queries
An index consumes additional storage
INSERT, UPDATE and DELETE may become slower because indexes also need maintenance
Not every query benefits from an index

🏆 Golden rule: index columns based on actual query patterns, not randomly.

3. 🔍 EXPLAIN: Understanding How PostgreSQL Executes Queries

You write a query, but how do you know PostgreSQL runs it efficiently? You can't assume a query is fast just because it looks simple. PostgreSQL provides EXPLAIN.

3.1 🤔 What is EXPLAIN?

📌 EXPLAIN shows the query execution plan: how PostgreSQL plans to retrieve the requested data.

EXPLAIN
SELECT *
FROM posts
WHERE user_id = 101;

Possible output:

Seq Scan on posts
  Filter: (user_id = 101)

A. Seq Scan. Seq means sequential. PostgreSQL reads through the table's rows to find matching records.

posts table
Row 1 → Check
Row 2 → Check
Row 3 → Check
...

Is Seq Scan bad? NO! This is extremely important. If your table has only 10 rows, using an index might actually cost more than checking all 10. PostgreSQL intelligently chooses between execution strategies.

B. Filter. Filter: (user_id = 101) means PostgreSQL keeps rows satisfying user_id = 101 and discards the others.

3.2 📋 Common scan types

Scan type	Meaning
Seq Scan	Reads table rows sequentially
Index Scan	Uses an index and retrieves table rows
Index Only Scan	Can retrieve required data directly from the index
Bitmap Index Scan	Uses an index to identify matching row locations
Bitmap Heap Scan	Retrieves matching table rows using a bitmap

Sequential Scan: Seq Scan on posts reads the table and checks rows
Index Scan: Index Scan using idx_posts_user_id on posts uses an index to find matching rows, then retrieves the actual table rows
Index Only Scan: if the index contains all the columns the query needs, PostgreSQL may answer without visiting the table's data pages. It can still need visibility checks depending on the visibility map
Bitmap Index Scan: useful when many matching rows must be retrieved. PostgreSQL first identifies matching row locations using the index, then fetches table pages efficiently

3.3 🗺️ What is a query execution plan?

📌 A query execution plan is the sequence of operations PostgreSQL chooses to execute your query.

Index Scan using idx_posts_user_id
  on posts
  Index Cond: (user_id = 101)

Index Cond: the condition used to search through the index
Filter: an additional condition applied to rows after they are retrieved

For example:

SELECT *
FROM posts
WHERE user_id = 101
AND is_deleted = false;

PostgreSQL might use an index on user_id, then filter out deleted posts.

3.4 💰 What does "cost" mean?

Index Scan (cost=0.15..8.20 rows=5 width=120)

Part	Meaning
`0.15`	Estimated startup cost
`8.20`	Estimated total cost
`rows=5`	Estimated number of returned rows
`width=120`	Estimated average row width in bytes

⚠️ PostgreSQL's cost is an internal estimate, not milliseconds. It helps PostgreSQL compare different possible plans.

4. ⏱️ EXPLAIN ANALYZE: Measuring Actual Execution

A very common interview question: what's the difference between EXPLAIN and EXPLAIN ANALYZE?

EXPLAIN shows the estimated plan and doesn't normally execute the query:

EXPLAIN
SELECT *
FROM posts
WHERE user_id = 101;

EXPLAIN ANALYZE actually executes the query and shows the plan along with measured statistics:

EXPLAIN ANALYZE
SELECT *
FROM posts
WHERE user_id = 101;

Index Scan using idx_posts_user_id
  on posts
  (cost=0.15..8.20 rows=5 width=120)
  (actual time=0.025..0.041 rows=5 loops=1)

Planning Time: 0.120 ms
Execution Time: 0.065 ms

Term	Meaning
`actual time`	Measured execution time
`rows`	Actual number of rows produced
`loops`	Number of times the operation executed
`Planning Time`	Time spent planning the query
`Execution Time`	Time spent executing the query

🎯 Interview important: EXPLAIN = what PostgreSQL estimates it will do. EXPLAIN ANALYZE = what PostgreSQL actually did, with measured statistics.

🚨 Warning: EXPLAIN ANALYZE executes the query. For SELECT that's generally safe. For INSERT, UPDATE or DELETE it really performs the modification. Wrap it in a transaction and ROLLBACK if you want to test without keeping changes.

BEGIN;

EXPLAIN ANALYZE
UPDATE posts SET is_deleted = true WHERE id = 10;

ROLLBACK;

You can also ask for buffer statistics:

EXPLAIN (ANALYZE, BUFFERS)
SELECT *
FROM posts
WHERE user_id = 101;

BUFFERS shows information about database pages accessed from shared buffers and storage, which helps you spot queries doing too much disk or memory I/O.

5. 🧰 Types of Indexes: B-tree vs GIN vs GiST

PostgreSQL supports different index types because different queries search for different kinds of data: a user ID, a word inside a text document, locations within a distance, elements inside a JSON document. One index type isn't suitable for every situation.

5.1 🌳 B-tree Index

The most commonly used. B-tree stands for Balanced Tree and it's PostgreSQL's default index type.

CREATE INDEX idx_users_email
ON users(email);

This creates a B-tree index by default.

Imagine these numbers: 10, 20, 30, 40, 50, 60, 70. A simplified tree:

            40
          /    \
        20      60
       (smaller) (larger)

(A simplified binary-tree illustration. Real B-trees have multiple keys and children per node.) The database navigates the tree instead of checking every value.

B-tree is excellent for:

-- 1. Equality searches
SELECT * FROM users WHERE id = 10;

-- 2. Range searches
SELECT * FROM posts WHERE id > 100;

-- 3. Sorting
SELECT * FROM users ORDER BY created_at;

-- 4. Minimum and maximum values
SELECT MIN(created_at) FROM posts;

It's particularly useful for integers, IDs, email addresses, dates and timestamps, numeric values, and sorting and range conditions. Whenever you search, compare or sort ordinary scalar values, B-tree is usually the first index type to consider.

5.2 📇 GIN Index

GIN stands for Generalized Inverted Index. It's especially useful for searching inside collections of values or documents.

A post has tags:

Caption: Learning PostgreSQL     Tags: ["database", "backend", "python"]
Caption: Learning React          Tags: ["frontend", "javascript", "web"]

You want all posts containing the tag backend. Use PostgreSQL's array type:

CREATE TABLE posts (
    id SERIAL PRIMARY KEY,
    caption TEXT,
    tags TEXT[]
);

INSERT INTO posts (caption, tags)
VALUES
(
    'Learning PostgreSQL',
    ARRAY['database', 'backend', 'python']
),
(
    'Learning React',
    ARRAY['frontend', 'javascript', 'web']
);

Query:

SELECT *
FROM posts
WHERE tags @> ARRAY['backend'];

@> means contains: "give me posts whose tags contain backend". Now create a GIN index:

CREATE INDEX idx_posts_tags
ON posts USING GIN(tags);

PostgreSQL can use it for supported array containment queries.

Why "inverted"? Think of a dictionary. Instead of storing Post 1 → database, backend, python, an inverted index conceptually maps each value back to the records containing it:

database   → Post 1
backend    → Post 1
python     → Post 1
frontend   → Post 2
javascript → Post 2
web        → Post 2

Where is GIN useful? PostgreSQL arrays, JSONB data, full-text search, and searching for elements within collections. A JSONB example:

CREATE TABLE products (
    id SERIAL PRIMARY KEY,
    details JSONB
);

INSERT INTO products(details)
VALUES ('{"color": "black", "brand": "Nike"}');

CREATE INDEX idx_products_details
ON products USING GIN(details);

SELECT *
FROM products
WHERE details @> '{"color": "black"}';

This searches for JSON documents containing the specified JSON structure.

🧠 GIN is not a replacement for B-tree. They solve different search problems.

5.3 🗺️ GiST Index

GiST stands for Generalized Search Tree. It's a flexible indexing framework for different kinds of data and search operations. One great example is geographical data.

Imagine finding "all users within 5 km of Bengaluru Palace". That's a spatial search, and a B-tree isn't designed to efficiently handle every kind of spatial relationship. GiST can support spatial indexing through extensions such as PostGIS.

CREATE INDEX idx_locations_geom
ON locations USING GIST(geom);

A spatial query:

SELECT *
FROM locations
WHERE ST_DWithin(
    geom,
    ST_SetSRID(ST_MakePoint(77.59, 12.97), 4326)::geography,
    5000
);

This finds locations within 5,000 metres of the point, assuming geom is stored with a geography-compatible coordinate reference system.

Where is GiST useful? Geographic and spatial searches, geometry operations, range types, and certain similarity searches using suitable operator classes.

⚖️ B-tree vs GIN vs GiST

Feature	B-tree	GIN	GiST
Main purpose	Ordinary values	Elements inside collections/documents	Flexible search structures
Equality	Yes	Supported operators	Operator-class dependent
Range	Excellent	Not generally	Supported for suitable types
Sorting	Yes	Generally no	Generally no
Arrays	Limited	Excellent	Operator-dependent
JSONB	Some operations	Excellent for supported operators	Operator-dependent
Spatial search	No general spatial support	No general spatial support	Excellent with PostGIS

🎯 Interview takeaway:

B-tree → IDs, emails, dates, range searches

GIN → arrays, JSONB, full-text search

GiST → spatial and specialized searches

📝 PostgreSQL also has BRIN and hash indexes. You may meet them in advanced work, but B-tree, GIN and GiST are the important three here.

6. 🧱 Composite Indexes

Another VERY important concept. Take this table:

id	user_id	created_at	caption
1	101	Nov 1	Hello
2	102	Nov 2	Beautiful
3	101	Nov 3	Coding
4	103	Nov 4	Weekend
5	101	Nov 5	Learning

You frequently run:

SELECT *
FROM posts
WHERE user_id = 101
ORDER BY created_at DESC;

That's two operations: find posts of user 101, then sort them newest first. You could create two separate indexes:

CREATE INDEX idx_posts_user
ON posts(user_id);

CREATE INDEX idx_posts_created
ON posts(created_at);

But PostgreSQL might need to combine index information or do extra sorting. There's a better approach.

6.1 🤔 What is a composite index?

📌 A composite index is an index created using multiple columns.

CREATE INDEX idx_posts_user_created
ON posts(user_id, created_at DESC);

One index, two columns. Conceptually it's organized by user_id first, then created_at:

user_id = 101
    ├── Nov 5
    ├── Nov 3
    └── Nov 1

user_id = 102
    └── Nov 2

user_id = 103
    └── Nov 4

The actual index is ordered by the first column, then the second column within matching first-column values. PostgreSQL can potentially find a user's posts and retrieve them in date order without a separate sorting step.

6.2 ⚠️ Column order matters!

CREATE INDEX idx_posts_user_created
ON posts(user_id, created_at);

This index is particularly useful for:

-- Both columns
SELECT * FROM posts
WHERE user_id = 101 AND created_at > '2026-01-01';

-- First column only
SELECT * FROM posts
WHERE user_id = 101;

But what about this?

-- Second column only
SELECT * FROM posts
WHERE created_at > '2026-01-01';

A regular B-tree composite index is generally much less effective here, because it's ordered primarily by user_id.

📝 PostgreSQL 18 added some B-tree skip-scan capabilities, but don't assume a composite index is an equally good substitute for a separate index on its second column.

🔑 The Leftmost Prefix Rule

CREATE INDEX idx_example
ON posts(user_id, created_at, is_deleted);

Query condition	Suitability
`user_id`	✅ Good
`user_id` + `created_at`	✅ Good
`user_id` + `created_at` + `is_deleted`	✅ Good
`created_at` alone	⚠️ Generally less suitable
`is_deleted` alone	⚠️ Generally less suitable

This is a general guideline, not an absolute rule.

📍 When to create composite indexes

When your app repeatedly runs queries using the same combination of columns. A social feed might have:

SELECT *
FROM posts
WHERE user_id = 101
AND is_deleted = false
ORDER BY created_at DESC
LIMIT 20;

A possible index:

CREATE INDEX idx_posts_user_deleted_created
ON posts(user_id, is_deleted, created_at DESC);

But always check your actual query patterns and data distribution before deciding the column order.

⚠️ More indexes do not automatically mean better performance.

7. 🐌 The N+1 Query Problem

🎯 One of the most common database performance problems in apps using ORMs such as SQLAlchemy, Django ORM and Hibernate.

Your home feed contains 10 posts. Each has post info, author info, comments and likes. A beginner might write:

posts = session.query(Post).limit(10).all()

for post in posts:
    author = session.get(User, post.user_id)
    print(post.caption, author.username)

Let's count the queries.

Step 1: fetch all posts (1 query)

SELECT * FROM posts LIMIT 10;

Step 2: fetch each author (10 more queries)

SELECT * FROM users WHERE id = 101;
SELECT * FROM users WHERE id = 102;
SELECT * FROM users WHERE id = 103;
-- ...and so on

Total: 1 + 10 = 11 queries. This is the N+1 Query Problem:

1 query fetches the main records
N additional queries fetch related data for each record

Fetch 100 posts and you might run 101 queries. Fetch 1,000 and you might run 1,001. That seriously affects performance.

7.1 🤔 Why is N+1 a problem?

If every query takes 10 ms:

10 posts:   11 queries × 10 ms  = 110 ms
100 posts: 101 queries × 10 ms = 1010 ms

(An illustration assuming sequential queries with equal latency. Real timings vary.) Every unnecessary database round trip adds overhead. Even if each query is individually fast, hundreds of calls make your API slow.

7.2 ✅ How do we solve N+1?

Solution 1: use a JOIN

SELECT
    posts.id,
    posts.caption,
    users.username
FROM posts
JOIN users
ON posts.user_id = users.id
LIMIT 10;

PostgreSQL retrieves post and author information in one query.

Solution 2: eager loading in SQLAlchemy. SQLAlchemy supports two common strategies: joinedload() and selectinload().

A. joinedload()

from sqlalchemy import select
from sqlalchemy.orm import joinedload

stmt = (
    select(Post)
    .options(joinedload(Post.author))
    .limit(10)
)

posts = session.scalars(stmt).all()

SQLAlchemy loads posts and their authors using a JOIN, so related data arrives as part of the query. For collection relationships, joined loading can multiply result rows, and SQLAlchemy requires .unique() when consuming results in that case.

B. selectinload()

from sqlalchemy import select
from sqlalchemy.orm import selectinload

stmt = (
    select(Post)
    .options(selectinload(Post.comments))
    .limit(10)
)

posts = session.scalars(stmt).all()

First query:

SELECT * FROM posts LIMIT 10;

Second query:

SELECT * FROM comments
WHERE post_id IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10);

Instead of 10 separate comment queries, SQLAlchemy fetches related comments in one batch. A major improvement. 🎉

⚖️ `joinedload` vs `selectinload`

Feature	`joinedload`	`selectinload`
Strategy	SQL JOIN	Separate batch query
Number of queries	Usually fewer	Usually more than one
Many-to-one	Often excellent	Also suitable
One-to-many	Can duplicate parent rows	Often preferable
Large collections	Can produce huge joined results	Often more efficient

Practical guideline:

Use joinedload() for many-to-one or one-to-one relationships when appropriate
Use selectinload() for one-to-many and many-to-many relationships in many common cases

💡 Neither is universally best. Measure the actual workload.

7.3 📱 N+1 in a social-feed app

For GET /feed, the response contains posts, authors, comments and like counts:

[
  {
    "id": 1,
    "caption": "Beautiful day",
    "author": { "username": "deepika" },
    "comments": [],
    "likes_count": 25
  }
]

A poor implementation queries all four things separately for every post. A better one:

Fetches posts using pagination
Eager-loads authors
Batch-fetches comments if needed
Calculates like counts using aggregation or a grouped query
Builds the response without additional queries inside a loop

SELECT
    post_id,
    COUNT(*) AS likes_count
FROM likes
WHERE post_id IN (1, 2, 3, 4, 5)
GROUP BY post_id;

That calculates counts for multiple posts in one query. This is the kind of thinking expected from a good backend developer! 🧠

8. ⚖️ Performance vs Storage Trade-offs

If indexes are so useful, why not index every column? Because indexes are not free.

8.1 💾 Indexes consume storage

posts table        → 500 MB
user_id index      → 100 MB
created_at index   → 120 MB
tags GIN index     → 200 MB
-----------------------------
Total              → 920 MB

Your table was 500 MB, but with three indexes storage grew to 920 MB in this simplified example. Real sizes depend on data types, values, table structure and index configuration.

8.2 🐢 Indexes can slow down INSERT

INSERT INTO posts(user_id, caption)
VALUES (101, 'My new post');

Without indexes:  Insert row → Done

With indexes:     Insert row
                    ▼
                  Update user_id index
                    ▼
                  Update created_at index
                    ▼
                  Update other indexes
                    ▼
                  Done

Every relevant index needs maintenance, so more indexes can mean more write overhead.

8.3 🐢 Indexes can slow down UPDATE and DELETE

UPDATE posts
SET user_id = 102
WHERE id = 10;

If user_id is indexed, PostgreSQL must maintain that index as the indexed value changes. DELETE operations must also maintain index entries.

8.4 🐇 Indexes improve read performance

If your app frequently runs SELECT * FROM posts WHERE user_id = 101;, an appropriate index can significantly reduce the work needed to find those posts. So there's a trade-off:

✅ Advantages	❌ Disadvantages
Faster suitable reads	Additional storage
Faster filtering	Slower writes
Efficient sorting	Index maintenance
Faster joins in suitable cases	More database complexity

The right approach: don't create an index simply because a column exists. Create indexes based on:

Frequently executed queries
WHERE conditions
JOIN conditions
ORDER BY conditions
Actual query execution plans
The balance between reads and writes

9. 🛠️ Other Important Query Optimization Techniques

Indexes are only one part of optimization.

9.1 🚫 Avoid `SELECT *`

SELECT *
FROM users
WHERE id = 10;

This retrieves every column. If the users table has id, username, email, password_hash, bio, profile_picture, created_at, address, preferences but the frontend only needs username and profile_picture:

SELECT username, profile_picture
FROM users
WHERE id = 10;

Why?

Less data transferred
Less memory used
Less unnecessary processing
May allow an index-only scan if a suitable covering index exists and visibility conditions permit

📝 SELECT * is not automatically slow. It's inefficient when you retrieve data you don't need.

9.2 📄 Use `LIMIT` and pagination

With 1 million posts, this is problematic for a feed:

SELECT * FROM posts;

You don't need a million posts at once. Instead:

SELECT *
FROM posts
ORDER BY created_at DESC
LIMIT 20;

For large datasets, cursor pagination is often preferable to deep OFFSET pagination:

SELECT *
FROM posts
WHERE created_at < '2026-11-01 10:00:00'
ORDER BY created_at DESC
LIMIT 20;

In production, use a stable unique tie-breaker such as id:

SELECT *
FROM posts
WHERE (created_at, id) < (:cursor_created_at, :cursor_id)
ORDER BY created_at DESC, id DESC
LIMIT 20;

A matching index:

CREATE INDEX idx_posts_created_id
ON posts(created_at DESC, id DESC);

9.3 🔀 Avoid unnecessary JOINs

SELECT *
FROM posts
JOIN users ON posts.user_id = users.id
JOIN comments ON posts.id = comments.post_id
JOIN likes ON posts.id = likes.post_id;

This can produce a huge number of rows. If one post has 5 comments and 10 likes, joining both collections produces 50 combinations for that single post. This is called row multiplication. Only join the information you actually need, and use separate aggregate queries or appropriate loading strategies when that's better.

9.4 🧮 Use aggregation instead of loading everything

Need only the number of likes?

❌ Inefficient: fetch all rows and count them in Python.

SELECT *
FROM likes
WHERE post_id = 10;

✅ Better: let PostgreSQL aggregate.

SELECT COUNT(*)
FROM likes
WHERE post_id = 10;

9.5 🔧 Avoid functions on indexed columns when possible

SELECT *
FROM users
WHERE LOWER(email) = 'abc@gmail.com';

A normal B-tree index on email might not be usable for this expression. Create an expression index:

CREATE INDEX idx_users_lower_email
ON users(LOWER(email));

Now the query can potentially use it.

10. 🧪 A Practical Optimization Exercise

CREATE TABLE posts (
    id SERIAL PRIMARY KEY,
    user_id INTEGER,
    caption TEXT,
    created_at TIMESTAMP,
    is_deleted BOOLEAN DEFAULT FALSE
);

Your app frequently runs:

SELECT id, user_id, caption, created_at
FROM posts
WHERE user_id = 101
AND is_deleted = false
ORDER BY created_at DESC
LIMIT 20;

Step 1: check the original execution plan

EXPLAIN ANALYZE
SELECT id, user_id, caption, created_at
FROM posts
WHERE user_id = 101
AND is_deleted = false
ORDER BY created_at DESC
LIMIT 20;

Observe: Is PostgreSQL using Seq Scan? How many rows are examined? Is there a Sort operation? How long does execution take?

Step 2: create a composite index

CREATE INDEX idx_posts_user_deleted_created
ON posts(user_id, is_deleted, created_at DESC);

Step 3: run EXPLAIN ANALYZE again and compare. You may see an Index Scan instead of a Sequential Scan, and possibly no separate Sort. But if the table is small, PostgreSQL may still choose Seq Scan. That doesn't mean your index is broken.

Step 4: understand the result

Before	After
Possible sequential scan	Possible index scan
May inspect many rows	Can navigate to matching entries
May require sorting	Index may provide ordering
Less index storage	Additional index storage

That's how you investigate and optimize real database queries. 🔬

11. ⚠️ Common Mistakes

Mistake	Better approach
Creating indexes on every column	Index based on query patterns
Assuming Seq Scan is always bad	Check table size and execution cost
Using `SELECT *` everywhere	Fetch only required columns
Fetching all records	Use pagination
Ignoring N+1 queries	Use eager loading and batch queries
Creating too many indexes	Balance read and write performance
Ignoring composite index order	Design indexes around actual conditions
Optimizing without measuring	Use `EXPLAIN ANALYZE`
Assuming an index always improves speed	Compare actual execution plans

12. 🧵 How It All Connects in a Real Backend

Your mentor asks you to optimize GET /feed, which retrieves posts, authors and likes. Here's the workflow:

🐢 Identify the slow API. GET /feed takes too long
🔎 Inspect the SQL queries. Check which queries SQLAlchemy generates
🔁 Find unnecessary database calls. Is there an N+1 problem?
📊 Analyze execution plans. Use EXPLAIN ANALYZE and inspect scans, rows, sorting and buffers
🗂️ Choose suitable indexes. B-tree for ordinary lookups, GIN for supported JSONB/array searches, GiST for spatial or specialized operations
📦 Optimize data retrieval. Use JOINs, eager loading, aggregation and pagination appropriately
✅ Measure again. Compare performance and make sure results are still correct

💡 Database query time and API response time are not the same. Your API might be slow because of database execution, network latency, Python processing, JSON serialization or external API calls. Don't blame PostgreSQL for every slow endpoint!

13. 🎤 Interview Questions and Answers

Q1. What is query optimization?
Improving database query performance by reducing unnecessary operations, improving execution plans and retrieving data efficiently.

Q2. What is an index?
An additional data structure that helps a database locate records faster, without scanning the entire table in suitable cases.

Q3. What is EXPLAIN?
It displays the execution plan PostgreSQL estimates it will use to execute a query.

Q4. What is EXPLAIN ANALYZE?
It executes the query and displays the actual execution statistics along with the plan.

Q5. What's the difference between B-tree and GIN?
B-tree suits ordinary values, equality, ranges and sorting. GIN is useful for searching elements within arrays, JSONB documents and full-text search.

Q6. What is GiST?
A flexible indexing framework for specialized search operations, including spatial queries with PostGIS.

Q7. What is a composite index?
An index created using multiple columns, for example CREATE INDEX idx_user_date ON posts(user_id, created_at);

Q8. Why does column order matter in a composite index?
A B-tree composite index is ordered primarily by its first column, then subsequent columns, so the order affects which queries can efficiently use it.

Q9. What is the N+1 query problem?
It happens when an application runs one query to retrieve parent records and then an additional query for each parent to retrieve related data.

Q10. How do you solve N+1 in SQLAlchemy?
Use eager loading strategies such as joinedload() and selectinload(), or suitable batch queries.

Q11. Why shouldn't we create indexes on every column?
Indexes consume storage and increase INSERT, UPDATE and DELETE overhead because the indexes must be maintained.

Q12. What is an Index Only Scan?
A scan that can obtain the required data from the index without fetching the corresponding table rows, subject to visibility checks.

14. 📝 Final Revision Sheet

Concept	Remember this
Query Optimization	Make database queries efficient
Index	Helps locate records faster
`EXPLAIN`	Estimated execution plan
`EXPLAIN ANALYZE`	Actual execution statistics
Seq Scan	Reads table rows sequentially
Index Scan	Uses an index to locate records
Index Only Scan	Retrieves required data from the index when possible
B-tree	IDs, equality, ranges, sorting
GIN	Arrays, JSONB, full-text search
GiST	Spatial and specialized searches
Composite Index	Multiple columns in one index
Leftmost Prefix	Leading columns matter
N+1	One main query plus N additional queries
`joinedload`	Eager loading using JOIN
`selectinload`	Eager loading using batch SELECT
Storage trade-off	Indexes consume extra disk space
Write trade-off	Index maintenance adds write overhead

15. 🏋️ Practice Checklist

[ ] Create a PostgreSQL table containing 100,000 sample posts
[ ] Run EXPLAIN on a query without an index
[ ] Create a B-tree index on user_id
[ ] Run EXPLAIN ANALYZE again and compare plans
[ ] Create a composite index on user_id and created_at
[ ] Create a GIN index on an array column and test @>
[ ] Understand a GiST index using a PostGIS example
[ ] Reproduce an N+1 problem using SQLAlchemy
[ ] Fix it using joinedload() or selectinload()
[ ] Compare query execution time, index size and write overhead

💡 Don't just memorize B-tree, GIN, GiST and EXPLAIN. Actually run queries in PostgreSQL and watch how the execution plan changes. That's how you develop real database optimization skills.

🎯 A good backend developer doesn't just make queries work. They make queries work **efficiently* as the application grows.*

If this post helped you, drop a ❤️ or 🦄 and share it with a friend learning backend development. Happy coding! 👩‍💻✨

Structural & Behavioral Design Patterns in Python: Repository, Factory, Strategy, Decorator & Adapter Explained 🧩

Deepika Pusala — Thu, 12 Nov 2026 06:24:14 +0000

🧩 Structural & Behavioral Design Patterns: Let's Learn from ZERO!

Design patterns teach you how to organize real-world application code so it is:

📖 Easy to understand
🛠️ Easy to maintain
🧪 Easy to test
➕ Easy to extend with new features
🔗 Less tightly coupled
♻️ Reusable across different parts of an application

1. 🤔 First, What Are Design Patterns?

Imagine you're building an Instagram-like app with login, creating posts, fetching posts, likes, image uploads, notifications and follows. At first you might write everything in one function:

def create_post():
    # Validate user
    # Check authentication
    # Save post to database
    # Upload image
    # Send notification
    # Return response
    pass

It works at first. But when the app grows to 100 features:

💥 Changing one feature might break another
🧪 Testing becomes difficult
🔁 Code becomes duplicated
🧶 Database logic gets mixed with business logic
😩 Adding new functionality becomes complicated

📌 A design pattern is a reusable solution or blueprint for a commonly occurring software design problem.

⚠️ A design pattern is NOT a library, framework, or ready-made code you copy. It's an approach to organizing your code.

1.1 🗂️ Three Categories of Design Patterns

Category	Concerned with	Examples
🏭 Creational	How objects are created	Factory, Singleton, Builder
🧱 Structural	How classes and objects are organized and connected	Repository, Adapter, Decorator, Facade
🎭 Behavioral	How objects communicate and distribute responsibilities	Strategy, Observer, Command, State

Our focus today:

Pattern	Category	Main purpose
Repository	Architectural / structural	Separates database operations
Factory	Creational	Creates objects
Strategy	Behavioral	Changes algorithms easily
Decorator	Structural	Adds functionality without changing original code
Adapter	Structural	Makes incompatible interfaces work together

📝 Repository is often called a design pattern, but it's more specifically a data-access architectural pattern. Factory is creational, while Strategy is behavioral.

2. 🧹 Separation of Concerns (SoC)

Before individual patterns, one foundational principle. Think of a restaurant 🍽️: the waiter takes orders, the chef prepares food, the cashier handles payments, the storekeeper manages ingredients. Would you ask one person to do everything? Probably not. Software should work the same way.

📌 Separation of Concerns means dividing an application into different parts, where each part handles a specific responsibility.

In a social-feed backend:

Component	Responsibility
🛣️ Controller / Router	Handles HTTP requests
🧠 Service	Contains business logic
📚 Repository	Communicates with the database
🗄️ Model	Represents database structure
✔️ Schema	Validates request and response data

A user creates a post:

⚛️ Frontend (React)       → user clicks Create Post
        ▼
🛣️ Controller / Router    → receives the HTTP POST request
        ▼
🧠 Service                → checks business rules, decides what should happen
        ▼
📚 Repository             → performs database operations
        ▼
🐘 PostgreSQL             → stores the post

This is called Layered Architecture.

3. 🥞 Layered Architecture: Controller → Service → Repository

Layered architecture divides an app into layers, each with its own responsibility. The most common backend structure is Controller (or Router) → Service → Repository → Database.

3.1 🛣️ Controller Layer

📌 A controller is the entry point through which an external client communicates with the backend.

In FastAPI we generally use APIRouter and route functions:

from fastapi import APIRouter, Depends

router = APIRouter()


@router.post("/posts")
def create_post(post_data: PostCreate):
    return post_service.create_post(post_data)

React sends a POST request
FastAPI receives it
The controller gets the request data
It calls the service
It returns the service result

The controller should NOT contain all business logic or SQL queries. Think of it as a receptionist 🛎️: it receives your request and sends it to the right department, without personally doing every job.

3.2 🧠 Service Layer

The service layer contains the app's business logic, the actual rules that define how your app behaves. In Instagram, for example:

A user cannot create a post without authentication
A user cannot edit another person's private post
A post must have valid content or media
A user cannot like the same post twice

class PostService:
    def __init__(self, post_repository):
        self.post_repository = post_repository

    def create_post(self, user_id, content):
        if not content.strip():
            raise ValueError("Post cannot be empty")

        post = self.post_repository.create(
            user_id=user_id,
            content=content,
        )

        return post

PostService handles post-related business logic
__init__() receives the repository
create_post() checks whether content is empty
If valid, it asks the repository to save the post

Notice: the service does not write SQL. It delegates database work to the repository.

3.3 📚 Repository Layer

📌 A repository handles communication between your application and the database. It sits between business logic and data storage.

class PostRepository:
    def __init__(self, db):
        self.db = db

    def create(self, user_id, content):
        post = Post(
            user_id=user_id,
            content=content,
        )

        self.db.add(post)
        self.db.commit()
        self.db.refresh(post)

        return post

Create a Post database object
Add it to the database session
Commit the transaction
Refresh the object to retrieve updated database values
Return the saved post

3.4 🤔 Why separate these three layers?

Imagine one function doing everything:

@router.post("/posts")
def create_post(data: PostCreate):
    # Validation
    # Authentication
    # Business rules
    # SQL queries
    # Database commit
    # Response formatting
    pass

That's tightly coupled code. If you change your database and database code is scattered everywhere, you modify many functions. With layered architecture, database changes are concentrated mainly in the repository and persistence configuration.

Controller	Service	Repository
Handles HTTP	Handles business rules	Handles data access
Receives requests	Makes decisions	Executes queries
Calls service	Calls repository	Talks to database
Returns responses	Returns business results	Returns stored data

🎤 Interview question: Why shouldn't controllers directly communicate with databases?

Answer: It mixes HTTP handling, business logic and data-access responsibilities. Separating them improves maintainability, testability and reusability, and reduces coupling.

4. 📚 Repository Pattern

Structural / data-access pattern

4.1 🚨 The problem

Without the Repository Pattern:

@router.get("/posts")
def get_posts(db: Session):
    posts = db.query(Post).all()
    return posts

The controller talks directly to the database. With 20 endpoints needing database access, you'd repeat database operations everywhere, causing duplication and tight coupling.

4.2 ✅ The solution

Create a separate repository class that handles all database operations for posts.

❌ Without Repository Pattern:  Controller ──▶ Database
✅ With Repository Pattern:     Controller ──▶ Service ──▶ PostRepository ──▶ Database

4.3 🧵 Full example

The repository:

class PostRepository:
    def __init__(self, db):
        self.db = db

    def get_all(self):
        return self.db.query(Post).all()

    def get_by_id(self, post_id):
        return (
            self.db.query(Post)
            .filter(Post.id == post_id)
            .first()
        )

    def create(self, post):
        self.db.add(post)
        self.db.commit()
        self.db.refresh(post)
        return post

    def delete(self, post):
        self.db.delete(post)
        self.db.commit()

The service:

class PostService:
    def __init__(self, repository):
        self.repository = repository

    def get_all_posts(self):
        return self.repository.get_all()

The controller:

@router.get("/posts")
def get_posts():
    return post_service.get_all_posts()

4.4 ✅ Advantages

Database logic stays in one place
Easier unit testing
Reduces duplication
Business logic doesn't need to know SQL details
Easier to maintain

4.5 🤔 Can we change databases with zero changes?

Not automatically! PostgreSQL and MongoDB have different data models and query behavior. The Repository Pattern isolates much of that difference, but changing databases may still require changes to repositories, models, transactions and other persistence code.

📝 A repository is not just a folder containing SQL queries. It represents the data-access operations the rest of the application needs.

5. 🏭 Factory Pattern

Creational pattern

5.1 🚨 The problem

Your app supports email, SMS and push notifications. You could create objects directly everywhere:

email = EmailNotification()
sms = SMSNotification()
push = PushNotification()

But with 50 places creating notifications, what happens when the creation process changes? You must update many places.

5.2 🤔 What is the Factory Pattern?

📌 The Factory Pattern provides a separate component or method responsible for creating objects. Instead of creating objects yourself everywhere, you ask a factory to create the appropriate one.

🍕 Think of ordering pizza or a burger. You don't walk into the kitchen and make it yourself. The kitchen receives your order and creates the item. That's a factory.

5.3 🐍 Python example

class EmailNotification:
    def send(self, message):
        print("Sending email:", message)


class SMSNotification:
    def send(self, message):
        print("Sending SMS:", message)


class PushNotification:
    def send(self, message):
        print("Sending push notification:", message)

The factory:

class NotificationFactory:
    @staticmethod
    def create_notification(notification_type):
        if notification_type == "email":
            return EmailNotification()
        elif notification_type == "sms":
            return SMSNotification()
        elif notification_type == "push":
            return PushNotification()
        else:
            raise ValueError("Invalid notification type")

Use it:

notification = NotificationFactory.create_notification("email")
notification.send("Your post was liked!")

Sending email: Your post was liked!

5.4 🔍 The flow

Application: "I need an email notification."
        ▼
NotificationFactory → decides which object to create
        ▼
EmailNotification object → created and returned

5.5 📍 When to use a Factory

Creating different notification types
Creating different payment processors
Creating different database clients
Creating different file parsers
Creating different authentication providers

5.6 ⚖️ Factory vs normal object creation

Normal creation	Factory
Application creates objects directly	Factory creates objects
Creation logic may be duplicated	Creation logic is centralized
Application knows concrete classes	Application can depend on a common interface

🧠 Factory focuses on HOW objects are created.

6. 🎯 Strategy Pattern

Behavioral pattern. One of the MOST important patterns for backend development.

6.1 🚨 The problem

A payment system supports UPI, credit card, debit card and PayPal. Each processes payments differently. Without the Strategy Pattern:

def make_payment(payment_type, amount):
    if payment_type == "upi":
        print("Processing UPI payment")
    elif payment_type == "card":
        print("Processing card payment")
    elif payment_type == "paypal":
        print("Processing PayPal payment")

Fine at first. But with 20 payment methods the function becomes huge, and every new method means modifying the same function.

6.2 🤔 What is the Strategy Pattern?

📌 The Strategy Pattern lets you define multiple algorithms, put each in a separate class, and select the appropriate one at runtime.

In simple words: same task, different ways of performing it. 🚌 To travel to college you can choose bus, car, bike or metro. The destination is the same, but the method changes.

6.3 🐍 Implementation

Step 1: create different strategies

class UpiPayment:
    def pay(self, amount):
        print(f"Paid {amount} using UPI")


class CardPayment:
    def pay(self, amount):
        print(f"Paid {amount} using Card")


class PayPalPayment:
    def pay(self, amount):
        print(f"Paid {amount} using PayPal")

Each class implements the same operation, pay(), but performs it differently.

Step 2: create the context class (the class that uses a selected strategy)

class PaymentProcessor:
    def __init__(self, strategy):
        self.strategy = strategy

    def make_payment(self, amount):
        self.strategy.pay(amount)

We pass a payment strategy object into PaymentProcessor. It doesn't need to know whether it's UPI, Card or PayPal. It only needs an object that supports pay().

Step 3: use different strategies

upi = UpiPayment()
processor = PaymentProcessor(upi)
processor.make_payment(500)

Paid 500 using UPI

Change the strategy:

card = CardPayment()
processor = PaymentProcessor(card)
processor.make_payment(1000)

Paid 1000 using Card

PaymentProcessor (Context) → uses a selected payment strategy
        ├── UPI     → pay()
        ├── Card    → pay()
        └── PayPal  → pay()

6.4 📱 Strategy in a social-feed project

Different feed-generation algorithms:

Chronological: newest posts first
Popularity-based: most liked posts first
Personalized: posts based on user interests

Instead of one huge function, create separate strategies:

class ChronologicalFeed:
    def generate(self, posts):
        return sorted(
            posts,
            key=lambda post: post.created_at,
            reverse=True,
        )


class PopularityFeed:
    def generate(self, posts):
        return sorted(
            posts,
            key=lambda post: post.like_count,
            reverse=True,
        )

A context uses either one:

class FeedService:
    def __init__(self, strategy):
        self.strategy = strategy

    def get_feed(self, posts):
        return self.strategy.generate(posts)

service = FeedService(ChronologicalFeed())
feed = service.get_feed(posts)

You can change the strategy without rewriting FeedService.

6.5 ✅ Advantages

Easy to add new algorithms
Avoids huge if-elif conditions
Reduces code duplication
Makes algorithms independently testable
Supports the Open/Closed Principle

6.6 📖 Important terminology

Term	Meaning
Strategy	An interchangeable algorithm
Concrete Strategy	The actual implementation, such as UPI or Card
Context	The class that uses the strategy
Runtime selection	Choosing which strategy to use while the app runs

🎤 Interview answer: The Strategy Pattern defines a family of interchangeable algorithms, encapsulates each in a separate class, and lets the application select one at runtime without changing the client code.

7. 🎁 Decorator Pattern

Structural pattern. It lets you add extra functionality to existing objects without modifying their original classes.

7.1 🚨 The problem

A coffee-ordering app has a simple coffee. Customers want milk, chocolate, whipped cream, milk and chocolate, and so on. One approach is a class for each combination:

class SimpleCoffee:
    pass


class CoffeeWithMilk:
    pass


class CoffeeWithChocolate:
    pass


class CoffeeWithMilkAndChocolate:
    pass

With 15 toppings you may end up with a huge number of combinations. This is the class explosion problem. 💥

7.2 🤔 What is the Decorator Pattern?

📌 The Decorator Pattern dynamically adds responsibilities or functionality to an object by wrapping it inside another object.

Take an existing object, wrap it, and give it extra functionality without changing the original. ☕ You buy a coffee, then add milk, chocolate and cream. Your original coffee still exists. You're just adding features to it.

7.3 🐍 Python implementation

Step 1: the basic component

class Coffee:
    def cost(self):
        return 100

    def description(self):
        return "Basic Coffee"

Step 2: a decorator for milk

class MilkDecorator:
    def __init__(self, coffee):
        self.coffee = coffee

    def cost(self):
        return self.coffee.cost() + 20

    def description(self):
        return self.coffee.description() + " + Milk"

Step 3: a decorator for chocolate

class ChocolateDecorator:
    def __init__(self, coffee):
        self.coffee = coffee

    def cost(self):
        return self.coffee.cost() + 30

    def description(self):
        return self.coffee.description() + " + Chocolate"

Step 4: combine them

coffee = Coffee()
coffee = MilkDecorator(coffee)
coffee = ChocolateDecorator(coffee)

print(coffee.description())
print(coffee.cost())

Basic Coffee + Milk + Chocolate
150

Basic Coffee        ₹100
   ▼ wrapped by
MilkDecorator       +₹20
   ▼ wrapped by
ChocolateDecorator  +₹30
   ▼
Final coffee        ₹150

coffee = MilkDecorator(coffee) passes the original coffee into MilkDecorator. Then coffee = ChocolateDecorator(coffee) passes the milk-decorated coffee into ChocolateDecorator. This is called wrapping.

7.4 ⚡ Real-world example: FastAPI decorators

You've probably already used decorators!

@router.get("/posts")
def get_posts():
    return {"message": "Posts"}


@router.post("/posts")
def create_post():
    return {"message": "Created"}

Python decorators are a language feature that can wrap or modify function behavior. FastAPI uses them to register route functions and associate HTTP methods and paths with them. More examples:

@staticmethod
def create():
    pass


@property
def username(self):
    return self._username


@functools.lru_cache
def get_settings():
    return Settings()

⚠️ Important distinction: the Decorator Design Pattern is a particular object-composition pattern. Not every Python @decorator is an implementation of the GoF Decorator Pattern.

7.5 ✅ Advantages

Adds functionality dynamically
Avoids modifying existing classes
Avoids creating many subclasses for combinations
Supports the Open/Closed Principle
Allows multiple decorators to be combined

7.6 📍 When to use it

Logging
Caching
Authentication wrappers
Authorization checks
Adding features to objects
Compression and encryption wrappers

8. 🔌 Adapter Pattern

Structural pattern. One of the easiest to understand with a real-life example.

8.1 🚨 The problem

Your phone has a USB-C port, but your charger has a different connector. Can you connect them directly? No! You need an adapter that converts one interface into another compatible one. Software works the same way.

8.2 🤔 What is the Adapter Pattern?

📌 The Adapter Pattern lets two incompatible interfaces work together by introducing a middle component that converts one interface into the one expected by the other. Think of it as a translator between two incompatible systems. 🗣️

8.3 🐍 Python example

Your application expects every payment gateway to have pay(amount). But an external provider has make_payment(value), so your app can't use it where pay() is expected.

Step 1: the external payment provider

class ExternalPaymentGateway:
    def make_payment(self, value):
        print(f"External payment: {value}")

Step 2: create an adapter

class PaymentAdapter:
    def __init__(self, gateway):
        self.gateway = gateway

    def pay(self, amount):
        self.gateway.make_payment(amount)

Step 3: use the adapter

external_gateway = ExternalPaymentGateway()
adapter = PaymentAdapter(external_gateway)

adapter.pay(500)

External payment: 500

The application calls adapter.pay(500)
The adapter receives the amount
The adapter converts the call into gateway.make_payment(500)
The external gateway processes the payment

Your Application      → calls pay(amount)
        ▼
Payment Adapter       → translates the method call
        ▼
External Payment Gateway → receives make_payment(value)

8.4 🌍 Real-world examples

Your app might integrate Razorpay, Stripe, PayPal or different cloud storage providers. Each has different APIs and method names. Create adapters that expose a common interface to your application:

class StripeAdapter:
    def __init__(self, stripe_client):
        self.client = stripe_client

    def charge(self, amount):
        return self.client.create_payment_intent(
            amount=amount,
        )

Your app uses charge() without needing to understand all the provider-specific details.

📝 In production you'd also handle currency units, authentication, errors and payment verification properly.

8.5 ✅ Advantages

Integrates third-party services
Reuses existing code
Avoids modifying external libraries
Reduces coupling between your app and external systems
Makes switching providers easier

🎤 Interview answer: The Adapter Pattern converts the interface of an existing class into another interface expected by the client, allowing incompatible classes to work together.

9. 🧬 Composition Over Inheritance

An object-oriented design principle, and VERY important for writing clean code.

9.1 👪 What is inheritance?

Inheritance lets one class acquire properties and methods from another:

class Animal:
    def eat(self):
        print("Animal is eating")


class Dog(Animal):
    def bark(self):
        print("Dog is barking")

dog = Dog()
dog.eat()
dog.bark()

Animal is eating
Dog is barking

Dog inherits eat() from Animal. This is an IS-A relationship: a dog is an animal.

9.2 🧩 What is composition?

📌 Composition means creating a class using objects of other classes. Instead of inheriting functionality, a class contains or uses another object to perform a task. This is a HAS-A relationship.

A car has an engine. A car is not an engine. 🚗

class Engine:
    def start(self):
        print("Engine started")


class Car:
    def __init__(self):
        self.engine = Engine()

    def start_car(self):
        self.engine.start()
        print("Car is ready")

car = Car()
car.start_car()

Engine started
Car is ready

self.engine = Engine() means the Car contains an Engine object and uses its functionality instead of inheriting from it.

9.3 ⚖️ Inheritance vs composition

Inheritance:  Animal ◀── inherits from ── Dog     (IS-A)
Composition:  Car ── HAS-A ──▶ Engine             (uses another object)

9.4 🤔 Why is composition often preferred?

A notification system has:

class EmailNotification:
    pass


class SMSNotification:
    pass


class PushNotification:
    pass

You want logging on every notification. One approach is inheritance:

class EmailWithLogging(EmailNotification):
    pass


class SMSWithLogging(SMSNotification):
    pass


class PushWithLogging(PushNotification):
    pass

Now add logging, caching, retry and monitoring, and you may create a huge number of subclasses. That's the inheritance explosion problem. Use composition instead:

class Logger:
    def log(self, message):
        print("LOG:", message)


class NotificationService:
    def __init__(self, logger):
        self.logger = logger

    def send(self, message):
        self.logger.log("Sending notification")
        print("Notification sent:", message)

logger = Logger()
service = NotificationService(logger)
service.send("Hello!")

Now you can replace the Logger with another implementation without changing the notification service.

9.5 ✅ Advantages of composition

More flexible code
Less dependency on parent classes
Easier testing
Easier to replace components
Supports loose coupling
Encourages reusable functionality

9.6 🤷 Does composition mean never use inheritance?

Absolutely not! Inheritance is useful when a genuine IS-A relationship exists: a Dog is an Animal, an AdminUser is a User, a Circle is a Shape. Composition is preferable when you want to reuse behavior or collaborate with other objects without creating an artificial parent-child relationship.

🎤 "Favor composition over inheritance" means prefer combining objects with specific responsibilities over building deep, rigid inheritance hierarchies.

10. 🧵 How All These Patterns Work Together

A user uploads a post containing an image. A well-organized architecture could work like this:

⚛️ React Frontend → user creates a post with an image
        ▼
🛣️ Controller / FastAPI Router → receives and validates the HTTP request
        ▼
🧠 PostService → applies post creation rules
        │
        ├── 🏭 Factory   → creates the appropriate storage provider
        ├── 🔌 Adapter   → converts the storage provider's API into your app's expected interface
        ├── 🎯 Strategy  → selects an image-processing or compression algorithm
        └── 🎁 Decorator → adds optional logging, caching or monitoring around an operation
        ▼
📚 PostRepository → saves post information in PostgreSQL
        ▼
🐘 PostgreSQL → stores post records and image URLs

📝 These are possible applications of the patterns, not a requirement that every service must use all five.

The main lesson: each component has a clear responsibility.

11. ⚖️ Separation of Concerns vs Loose Coupling vs High Cohesion

These three are often discussed together.

11.1 🧹 Separation of Concerns

Each component has a specific responsibility. The controller handles HTTP requests, the service handles business logic, the repository handles database access. Don't mix everything into one class.

11.2 🔗 Loose Coupling

Coupling is how strongly two components depend on each other.

Tight coupling: one component heavily depends on the internal implementation of another
Loose coupling: components interact through well-defined interfaces and have fewer dependencies on each other's internals

❌ Tightly coupled:

class NotificationService:
    def send(self, message):
        email = EmailNotification()
        email.send(message)

Here NotificationService directly creates an EmailNotification, so it's permanently dependent on that concrete class.

✅ Loosely coupled:

class NotificationService:
    def __init__(self, notification):
        self.notification = notification

    def send(self, message):
        self.notification.send(message)

Now you can inject EmailNotification, SMSNotification or PushNotification. The service doesn't need to know which concrete implementation it receives. This is Dependency Injection. 💉

11.3 🎯 High Cohesion

Cohesion is how closely related the responsibilities inside a component are.

High cohesion: a class or module focuses on one closely related set of responsibilities
Low cohesion: a class handles unrelated responsibilities

❌ Bad design:

class UserManager:
    def create_user(self):
        pass

    def send_email(self):
        pass

    def generate_report(self):
        pass

    def process_payment(self):
        pass

✅ Better design:

class UserService:
    def create_user(self):
        pass


class EmailService:
    def send_email(self):
        pass


class ReportService:
    def generate_report(self):
        pass


class PaymentService:
    def process_payment(self):
        pass

📊 Comparison

Concept	Meaning	Main goal
Separation of Concerns	Divide responsibilities	Organized architecture
Loose Coupling	Reduce dependencies	Flexibility
High Cohesion	Keep related responsibilities together	Focused components

🧠 Good software design = separation of concerns + loose coupling + high cohesion + clear responsibilities + easy testing and maintenance.

12. 🥊 Important Differences Between the Patterns

🎯 This is where interviewers may try to confuse you.

12.1 🏭 Factory vs 🎯 Strategy

Both use different classes, but they solve different problems.

Factory	Strategy
Creates objects	Selects behavior
Focuses on object creation	Focuses on algorithms
Which object should I create?	Which algorithm should I use?
Example: create a UPI or Card processor	Example: use a UPI or Card payment strategy

A factory might create a UPI payment object, and the strategy determines how it processes a payment. They can also be used together.

12.2 🔌 Adapter vs 🎁 Decorator

Both can wrap another object, but their purposes differ.

Adapter	Decorator
Changes the interface	Adds functionality
Makes incompatible objects compatible	Enhances existing object behavior
Acts as a translator	Acts as a wrapper
Example: converts `make_payment()` into `pay()`	Example: adds logging to `pay()`

12.3 📚 Repository vs 🧠 Service

Repository	Service
Handles database operations	Handles business rules
Saves, retrieves, deletes data	Decides what should happen
Knows persistence details	Should avoid persistence implementation details
Example: `get_post_by_id()`	Example: `create_post()`

12.4 🧬 Inheritance vs Composition vs Decorator

Concept	Main idea
Inheritance	Get functionality from a parent class
Composition	Use another object as a component
Decorator	Wrap an object to add behavior

13. 🧭 How to Identify Which Pattern to Use

Your problem	Suitable pattern
Database queries are scattered across controllers	📚 Repository
Need to create different notification objects	🏭 Factory
Need multiple feed-generation algorithms	🎯 Strategy
Need to add logging without modifying original classes	🎁 Decorator
A third-party API has an incompatible method	🔌 Adapter
One class inherits too many unrelated behaviors	🧬 Composition
One class is doing everything	🧹 Separation of Concerns / SRP

14. ⚠️ Common Mistakes Beginners Make

❌ Using design patterns everywhere. Not every class needs a Factory, Adapter or Strategy. Use patterns when they solve an actual problem.
❌ Putting business logic inside repositories. Repositories should primarily handle data access.
❌ Putting SQL queries inside controllers. Delegate persistence operations to the repository layer.
❌ Confusing Factory and Strategy. Factory creates objects; Strategy selects interchangeable behavior.
❌ Confusing Adapter and Decorator. Adapter changes compatibility; Decorator adds functionality.
❌ Creating unnecessarily deep inheritance hierarchies. Prefer composition when objects need to collaborate rather than inherit from each other.
❌ Creating huge service classes. Keep related business responsibilities together.

15. 🎤 Interview Questions and Answers

Q1. What are design patterns?
Reusable approaches to commonly occurring software design problems. They help developers write maintainable, flexible and reusable code.

Q2. What is layered architecture?
It divides an application into separate layers such as Controller, Service and Repository, where each layer has a specific responsibility.

Q3. Why do we use the Repository Pattern?
To separate database access logic from business logic, improving maintainability, reusability and testability.

Q4. What is the Factory Pattern?
A centralized way to create objects without exposing all object-creation details to the client.

Q5. What is the Strategy Pattern?
It encapsulates different algorithms into separate interchangeable classes and lets the application select one at runtime.

Q6. What is the Decorator Pattern?
It dynamically adds behavior to an existing object without modifying its original class.

Q7. What is the Adapter Pattern?
It allows incompatible interfaces to work together by converting one interface into another expected interface.

Q8. What is composition over inheritance?
The principle of preferring object composition and collaboration over unnecessary inheritance hierarchies.

Q9. What is dependency injection?
A technique in which an object receives its dependencies from outside rather than creating them internally.

Q10. What are the benefits of separation of concerns?
It improves readability, maintainability, testability and reusability, and makes individual components easier to modify.

16. 📝 Final Revision Sheet

Pattern / principle	Category	Example	Idea
📚 Repository	Structural / data access	`PostRepository`	Separates database operations from business logic
🏭 Factory	Creational	`NotificationFactory`	Creates the appropriate object
🎯 Strategy	Behavioral	Payment strategies	Selects between interchangeable algorithms
🎁 Decorator	Structural	Logging decorator	Adds functionality by wrapping an object
🔌 Adapter	Structural	Payment API adapter	Makes incompatible interfaces work together
🧬 Composition	Design principle	Car HAS an Engine	Builds functionality from collaborating objects
🥞 Layered Architecture	Architectural pattern	Controller → Service → Repository	Separates responsibilities into layers
🧹 Separation of Concerns	Design principle	One focused responsibility per component	Keeps different responsibilities separate

17. 🧪 Test Your Understanding!

Try these before peeking at the answers! ❤️

Your app supports UPI, PayPal and Card payments, and you want to choose a payment algorithm at runtime. Which pattern?
- A) Factory
- B) Strategy
- C) Adapter
- D) Repository
Your controller contains SQL queries. You want to move all database operations into a separate component. Which pattern?
- A) Decorator
- B) Factory
- C) Repository
- D) Adapter
An external API uses make_payment(), but your app expects pay(). What do you use?
- A) Adapter
- B) Strategy
- C) Decorator
- D) Factory
You want to add logging to an existing object without modifying its class. Which pattern?
- A) Repository
- B) Factory
- C) Decorator
- D) Strategy
A Car uses an Engine object instead of inheriting from Engine. What is this?
- A) Inheritance
- B) Composition
- C) Adapter
- D) Factory
You want one class to create EmailNotification, SMSNotification or PushNotification objects. Which pattern?
- A) Strategy
- B) Factory
- C) Repository
- D) Decorator

👀 Click to reveal answers

B: Strategy. Interchangeable algorithms chosen at runtime
C: Repository. It separates data access from the controller
A: Adapter. It translates one interface into another
C: Decorator. It wraps an object to add behavior
B: Composition. This is a HAS-A relationship
B: Factory. It centralizes object creation

18. 🏁 What to Remember

🚫 Don't put everything in one file
🛣️ Keep controllers responsible for HTTP communication
🧠 Keep services responsible for business rules
📚 Keep repositories responsible for database operations
🏭 Use Factory when object creation becomes complex
🎯 Use Strategy when multiple algorithms are interchangeable
🎁 Use Decorator when you need to add behavior without modifying the original object
🔌 Use Adapter when integrating incompatible external systems
🧬 Prefer composition when objects need to collaborate
🔗 Aim for loose coupling and high cohesion

🎯 The ultimate goal isn't to use as many design patterns as possible. It's to write software that is easy to understand, extend, test and maintain.

If this post helped you, drop a ❤️ or 🦄 and share it with a friend learning software design. Happy coding! 👩‍💻✨

SOLID Principles Deep Dive (I & D): Interface Segregation and Dependency Inversion Explained 🧩

Deepika Pusala — Thu, 12 Nov 2026 06:18:16 +0000

🧩 SOLID Principles Deep Dive (I & D): Let's Learn from ZERO!

We've reached the final two principles of SOLID! 🎉 In this post we'll understand Interface Segregation (I) and Dependency Inversion (D) from absolute zero, with Python examples, real-world scenarios, FastAPI applications and interview questions.

🔤 The Five SOLID Principles

Letter	Principle	One-line idea
S	Single Responsibility	A class should have one responsibility
O	Open/Closed	Open for extension, closed for modification
L	Liskov Substitution	Child classes should be usable wherever their parent is expected
I ⭐	Interface Segregation	Don't force a class to implement methods it doesn't need
D ⭐	Dependency Inversion	Depend on abstractions, not concrete implementations

(Today's topics are marked with ⭐.)

🧱 PART 1: Interface Segregation Principle (ISP)

1. 🤔 First, what is an interface?

📌 An interface is a contract that specifies what operations an object should provide.

Think about a TV remote 📺. It has buttons for power, volume, channel and mute. You don't need to know how the TV performs those operations internally. You only need to know which operations are available. That's the basic idea of an interface.

In Python, we commonly represent interfaces using:

Abstract Base Classes (ABC)
Abstract methods using @abstractmethod
Protocols using typing.Protocol

We'll use the ABC approach first.

2. 📜 What is the Interface Segregation Principle?

📌 Clients should not be forced to depend on interfaces they do not use.

In simple English: imagine an interface with 10 methods, but a particular class only needs 3. Should you force that class to implement all 10? NO! Instead, divide the large interface into smaller, specific interfaces, so each class implements only the methods relevant to it. That's ISP.

3. 🚨 The problem, with a real-world example

An employee management system has three kinds of employees: Developer, Manager, Intern. Suppose one common Employee interface contains:

work()
attend_meeting()
approve_leave()
write_code()

But look at what each role really does:

Employee	Work	Meeting	Approve leave	Write code
Developer	✅	✅	❌	✅
Manager	✅	✅	✅	Not necessarily
Intern	✅	✅	❌	✅

A developer or an intern shouldn't be forced to implement approve_leave(). The interface is too large and mixes unrelated responsibilities. That's called a fat interface. 🐷

4. ❌ Without ISP: bad implementation

from abc import ABC, abstractmethod


class Employee(ABC):
    @abstractmethod
    def work(self):
        pass

    @abstractmethod
    def attend_meeting(self):
        pass

    @abstractmethod
    def approve_leave(self):
        pass

    @abstractmethod
    def write_code(self):
        pass

Employee is an abstract class that forces every child to implement all four methods. Now a developer:

class Developer(Employee):
    def work(self):
        print("Developer is working")

    def attend_meeting(self):
        print("Developer is attending meeting")

    def approve_leave(self):
        raise NotImplementedError

    def write_code(self):
        print("Developer is writing code")

See approve_leave()? A developer doesn't approve leave! We're implementing a method the developer doesn't need, which is exactly the problem ISP prevents.

Problems with this design:

Unnecessary methods are forced on classes
Classes contain meaningless implementations
Changes to the large interface can affect unrelated classes
Code becomes harder to maintain
It can cause runtime errors when unsupported methods are called

5. ✅ With ISP: correct implementation

Instead of one big interface, split it into smaller ones:

Before ISP:  Employee → work() · attend_meeting() · approve_leave() · write_code()
                        (ONE LARGE INTERFACE)

After ISP:   Workable            → work()
             MeetingParticipant  → attend_meeting()
             LeaveApprover       → approve_leave()
             CodeWriter          → write_code()
                        (small, focused interfaces)

from abc import ABC, abstractmethod


class Workable(ABC):
    @abstractmethod
    def work(self):
        pass


class MeetingParticipant(ABC):
    @abstractmethod
    def attend_meeting(self):
        pass


class LeaveApprover(ABC):
    @abstractmethod
    def approve_leave(self):
        pass


class CodeWriter(ABC):
    @abstractmethod
    def write_code(self):
        pass

Now each interface has one specific responsibility. Let's build the classes:

class Developer(Workable, MeetingParticipant, CodeWriter):
    def work(self):
        print("Developer is working")

    def attend_meeting(self):
        print("Developer is attending meeting")

    def write_code(self):
        print("Developer is writing code")


class Manager(Workable, MeetingParticipant, LeaveApprover):
    def work(self):
        print("Manager is managing the team")

    def attend_meeting(self):
        print("Manager is attending meeting")

    def approve_leave(self):
        print("Manager approved leave")


class Intern(Workable, MeetingParticipant, CodeWriter):
    def work(self):
        print("Intern is learning and working")

    def attend_meeting(self):
        print("Intern is attending meeting")

    def write_code(self):
        print("Intern is writing code")

Each class implements only the relevant interfaces. Nobody is forced to implement unnecessary methods. That's the Interface Segregation Principle! 🎯

6. 🐷 Fat interface vs thin interface

Fat interface	Thin interface
Contains too many methods	Contains fewer focused methods
Includes unrelated operations	Includes related operations
Forces unnecessary implementations	Classes implement what they need
Difficult to maintain	Easier to maintain

🧠 Fat interface = too many unrelated responsibilities. Thin interface = a small, focused contract.

7. 💳 Another example: a payment system

E-commerce payment methods: credit card, UPI, cash on delivery. Suppose you create one interface:

class PaymentMethod(ABC):
    @abstractmethod
    def pay(self):
        pass

    @abstractmethod
    def refund(self):
        pass

    @abstractmethod
    def scan_qr(self):
        pass

The problem:

Credit cards can pay and refund
UPI can pay, refund and scan QR codes
Cash on delivery might not support online refunds or QR scanning

One interface forces every payment method to support everything. Split it instead:

class Payable(ABC):
    @abstractmethod
    def pay(self):
        pass


class Refundable(ABC):
    @abstractmethod
    def refund(self):
        pass


class QRScannable(ABC):
    @abstractmethod
    def scan_qr(self):
        pass

Now payment implementations choose the capabilities they actually support. This is useful in payment gateways, notification systems, file processing and authentication systems.

8. ⚡ ISP in real full-stack development

In a social-feed backend you might have normal users, admins and content moderators, with operations like create post, like post, comment, delete any user's post, block users and manage reports. Would you give every user every operation? Of course not! A normal user creates posts and comments, while an admin can also delete inappropriate posts and manage users. Separate these capabilities into focused interfaces:

class PostCreator(ABC):
    @abstractmethod
    def create_post(self):
        pass


class PostModerator(ABC):
    @abstractmethod
    def delete_post(self):
        pass


class UserManager(ABC):
    @abstractmethod
    def block_user(self):
        pass

Different classes implement different capabilities, which makes it easier to extend when new roles and permissions arrive.

⚠️ ISP is about designing small, cohesive interfaces. It does not mean every single method needs its own interface. Group methods that naturally belong together.

9. ✅ Advantages of ISP

🧹 Less unnecessary code: classes don't implement methods they don't need
🔗 Loose coupling: classes depend on fewer operations
🛠️ Easy maintenance: changes to one interface are less likely to affect unrelated classes
📖 Better readability: each interface has a clear purpose
🧪 Better testing: test smaller components independently
📈 Better scalability: new capabilities don't require changing a huge interface

10. 🕵️ When should you use ISP?

Use it when:

An interface has too many unrelated methods
Classes implement methods they don't actually support
You see NotImplementedError in classes just because the parent interface forces it
Different clients need different subsets of operations

⚠️ ISP doesn't mean creating hundreds of tiny interfaces without a reason. Over-segregation also makes code hard to understand.

🔄 PART 2: Dependency Inversion Principle (DIP)

🏛️ One of the most important SOLID principles for understanding professional backend architecture.

To understand DIP we need three concepts first: dependency, abstraction vs concretion, and dependency inversion.

1. 🔗 What is a dependency?

class Engine:
    def start(self):
        print("Engine started")


class Car:
    def __init__(self):
        self.engine = Engine()

    def start_car(self):
        self.engine.start()

The car needs an engine to operate, so Car depends on Engine.

📌 When one class needs another class to perform its work, the first class has a dependency on the second.

Car class    (needs an engine to work)
     │  depends on
     ▼
Engine class (provides start() functionality)

But what if you want an electric or hybrid engine? Car directly creates a specific Engine, which makes it tightly coupled to that implementation.

2. 🪢 What is tight coupling?

Coupling describes how strongly two software components depend on each other.

Tight coupling: one class is strongly dependent on the internal implementation of another
Loose coupling: one class depends on a stable contract rather than a particular implementation

class DieselEngine:
    def start(self):
        print("Diesel engine started")


class Car:
    def __init__(self):
        self.engine = DieselEngine()

    def start_car(self):
        self.engine.start()

The problem line:

self.engine = DieselEngine()

Car directly creates a DieselEngine. Tomorrow you need an electric one:

class ElectricEngine:
    def start(self):
        print("Electric engine started")

...and now you must modify the Car class. As the app grows with many implementations, changing them becomes hard when business logic directly creates them. That's tight coupling. 😩

3. 🎭 What is abstraction?

📌 Abstraction means hiding implementation details and exposing only the essential functionality.

When you transfer money in a banking app 🏦, you enter the amount, enter the receiver and click Transfer. Do you need to know the database queries, server communication, transaction validation and network operations? No! You interact with one simple operation, and the complicated internals are hidden. That's abstraction.

⚖️ Abstraction vs implementation

Abstraction	Implementation
What should happen	How it happens
Defines a contract	Provides actual logic
Focuses on functionality	Focuses on internal details
Example: Send notification	Example: Send using email

In Python:

from abc import ABC, abstractmethod


class NotificationService(ABC):
    @abstractmethod
    def send(self, message):
        pass

This is an abstraction. It says "any notification service must provide a send() method", but not how to send. Now implement it in different ways:

class EmailNotification(NotificationService):
    def send(self, message):
        print("Sending email:", message)


class SMSNotification(NotificationService):
    def send(self, message):
        print("Sending SMS:", message)


class PushNotification(NotificationService):
    def send(self, message):
        print("Sending push notification:", message)

These are concrete implementations: each follows the same contract but performs the operation differently.

              NotificationService      ← abstraction · send(message)
             ╱         │          ╲
          Email       SMS         Push     ← concrete implementations

🧱 What is a concretion?

A concretion is an actual class that provides the implementation of an abstraction.

NotificationService → abstraction
EmailNotification, SMSNotification, PushNotification → concrete implementations

🧠 Abstraction defines WHAT. Concretion defines HOW.

4. 📜 What exactly is the Dependency Inversion Principle?

The official definition:

High-level modules should not depend on low-level modules. Both should depend on abstractions.

Abstractions should not depend on details. Details should depend on abstractions.

Translated: your application has two kinds of layers.

High-level module: contains important business logic (e.g. an order service that processes customer orders)
Low-level module: performs technical operations (sending emails, saving to a database, uploading images, sending SMS)

The high-level business logic should not be tightly connected to one specific low-level implementation. Instead, both depend on an abstraction. That's DIP!

5. 🏗️ High-level vs low-level modules

A customer places an order. What should happen?

Validate the order
Calculate the total
Save the order
Send a confirmation email

Component	Type
`OrderService`	🔝 High-level module
`EmailService`	🔧 Low-level module
`DatabaseRepository`	🔧 Low-level module
`PaymentGateway`	🔧 Low-level module

OrderService is high-level because it contains the important business workflow. Email and database services are low-level because they perform specific technical operations used by that workflow.

6. ❌ Without DIP: bad implementation

class EmailService:
    def send_email(self, message):
        print("Email sent:", message)


class OrderService:
    def __init__(self):
        self.email_service = EmailService()

    def place_order(self):
        print("Order placed successfully")
        self.email_service.send_email(
            "Your order has been confirmed"
        )

The key line:

self.email_service = EmailService()

OrderService directly creates an EmailService, so:

It knows about EmailService
It's tightly coupled to EmailService
It can't easily switch to SMS or push notifications

If your manager says "we now want order confirmations through SMS instead of email", you must change OrderService. That's undesirable, because the business workflow shouldn't change just because the notification technology changes. This violates DIP. 🚨

7. ✅ With DIP: correct implementation

Step 1: create an abstraction

from abc import ABC, abstractmethod


class NotificationService(ABC):
    @abstractmethod
    def send(self, message):
        pass

Step 2: create concrete implementations

class EmailService(NotificationService):
    def send(self, message):
        print("Email sent:", message)


class SMSService(NotificationService):
    def send(self, message):
        print("SMS sent:", message)

Step 3: modify the high-level module

class OrderService:
    def __init__(self, notification_service):
        self.notification_service = notification_service

    def place_order(self):
        print("Order placed successfully")
        self.notification_service.send(
            "Your order has been confirmed"
        )

Previously we created the email service inside OrderService. Now we receive the notification service from outside. A major difference! 🌟

Step 4: inject the implementation

email_service = EmailService()
order_service = OrderService(email_service)
order_service.place_order()

Order placed successfully
Email sent: Your order has been confirmed

Want SMS instead?

sms_service = SMSService()
order_service = OrderService(sms_service)
order_service.place_order()

Order placed successfully
SMS sent: Your order has been confirmed

Did we modify OrderService? NO! We just passed a different implementation. That's the power of DIP. 💪

8. 🔍 Exactly what changed?

❌ Without DIP
OrderService (business logic) ──▶ EmailService (specific implementation)
                  tightly coupled

✅ With DIP
OrderService (high-level) ──▶ NotificationService (abstraction / contract) ◀── EmailService
                                                                             ◀── SMSService
                  loosely coupled architecture

Previously: OrderService → EmailService

After applying DIP: OrderService → NotificationService ← EmailService

Both the high-level and low-level components depend on the abstraction. The dependency direction has been inverted. 🔄

9. 💉 What is Dependency Injection (DI)?

You may have heard about Dependency Injection in FastAPI. DIP and DI are related, but they are NOT the same.

📌 Dependency Injection means providing an object or dependency to a class from outside instead of making that class create it internally.

Without DI:

class OrderService:
    def __init__(self):
        self.payment = PaymentService()

With DI:

class OrderService:
    def __init__(self, payment):
        self.payment = payment

⚖️ DIP vs DI

DIP	DI
A design principle	A technique / pattern
Focuses on abstractions	Focuses on providing dependencies
Reduces tight coupling	Helps achieve loose coupling
Architectural guideline	Practical implementation technique

DIP tells you how your architecture should be designed
DI gives you a way to supply the required objects

You can use DI to implement DIP. But simply injecting a concrete class doesn't automatically guarantee DIP. The dependency should be designed around an appropriate abstraction.

10. ⚡ How does this work in FastAPI?

FastAPI has a dependency injection system called Depends():

from fastapi import Depends, FastAPI

app = FastAPI()


class UserService:
    def get_user(self, user_id: int):
        return {
            "id": user_id,
            "name": "Deepika",
        }


def get_user_service():
    return UserService()


@app.get("/users/{user_id}")
def get_user(
    user_id: int,
    service: UserService = Depends(get_user_service),
):
    return service.get_user(user_id)

The flow:

A client sends a GET request
FastAPI sees Depends(get_user_service)
FastAPI calls the dependency provider
The provider creates and returns a UserService object
FastAPI passes that object to the endpoint
The endpoint calls service.get_user()

The endpoint doesn't create the service itself. That's dependency injection.

HTTP Request  GET /users/10
      ▼
FastAPI Router → endpoint receives the request
      ▼
Depends()      → requests the dependency
      ▼
get_user_service() → provides a UserService object
      ▼
Endpoint uses the service → service.get_user(user_id)

📝 Depends() is the DI mechanism. To achieve DIP, you should also design your service and repository dependencies around abstractions when appropriate.

11. 🗄️ A more realistic FastAPI example: the repository pattern

A very important example for backend architecture. Your layers: API/router → service → repository → database. The service needs user data, but it shouldn't need to know which database technology is used. Today it's PostgreSQL. Tomorrow it might be another database, or an in-memory repository for testing.

Step 1: define the repository abstraction

from abc import ABC, abstractmethod


class UserRepository(ABC):
    @abstractmethod
    def get_user(self, user_id: int):
        pass

Step 2: create a concrete implementation

class PostgreSQLUserRepository(UserRepository):
    def get_user(self, user_id: int):
        print("Fetching user from PostgreSQL")
        return {
            "id": user_id,
            "name": "Deepika",
        }

Step 3: create the high-level service

class UserService:
    def __init__(self, repository: UserRepository):
        self.repository = repository

    def get_user_details(self, user_id: int):
        user = self.repository.get_user(user_id)
        return user

Notice repository: UserRepository. The service depends on the abstraction, not directly on PostgreSQLUserRepository.

Step 4: inject the implementation

repository = PostgreSQLUserRepository()
user_service = UserService(repository)

result = user_service.get_user_details(10)
print(result)

Fetching user from PostgreSQL
{'id': 10, 'name': 'Deepika'}

Now replace PostgreSQL with an in-memory implementation:

class InMemoryUserRepository(UserRepository):
    def get_user(self, user_id: int):
        return {
            "id": user_id,
            "name": "Test User",
        }


repository = InMemoryUserRepository()
user_service = UserService(repository)

print(user_service.get_user_details(10))

You changed the repository implementation without changing UserService. That's DIP in action. 🎯

Real-world connection to a social-feed backend:

PostService → post-related business logic
PostRepository → an interface for post data access
PostgresPostRepository → the actual PostgreSQL implementation

The service shouldn't need to know the SQL queries the repository uses.

📝 This is a simplified architectural example. A real SQLAlchemy implementation may use AsyncSession, async repository methods and FastAPI dependencies.

12. 🧪 Why is DIP great for testing?

When testing UserService, it normally connects to PostgreSQL. But what if your database is unavailable, you don't want to create real records, you want predictable test data, or you want to test business logic independently? Inject a fake repository:

class FakeUserRepository(UserRepository):
    def get_user(self, user_id: int):
        return {
            "id": user_id,
            "name": "Testing User",
        }


fake_repository = FakeUserRepository()
service = UserService(fake_repository)

result = service.get_user_details(10)

assert result["name"] == "Testing User"

We tested UserService without connecting to PostgreSQL! That's possible because the service depends on an abstraction, so we can supply a different implementation. A hugely useful technique for unit testing.

13. 🧠 DIP vs DI vs IoC

These three terms confuse everyone. Let's separate them:

Concept	Meaning
DIP	High-level and low-level modules depend on abstractions
DI	Dependencies are provided from outside
IoC	Control over object creation or execution is handed to another component or framework

Inversion of Control (IoC) is the broader concept. In a traditional program you create and manage all objects yourself. With FastAPI, the framework manages dependency resolution when it processes a request. That's IoC. DI is one way to implement IoC, and DIP is an architectural principle that keeps components independent of concrete implementations.

14. ✅ Advantages of DIP

🔗 Loose coupling: business logic isn't tied to specific technologies
🔄 Easy replacement: swap PostgreSQL for another repository implementation
🧪 Easy testing: inject fake or mock implementations
🛠️ Maintainability: technical changes don't necessarily affect business logic
📈 Scalability: add new implementations without rewriting business logic
🔀 Flexibility: switch email providers, payment gateways, storage services and more

15. 🕵️ When should you use DIP?

Use it when:

Business logic directly creates database or external service objects
You want to support multiple implementations
You want to test business logic independently
Your application has multiple architectural layers
You want to avoid tightly coupling services to specific technologies

⚠️ Don't create abstractions for every class unnecessarily. A simple app with no meaningful variation may not need a separate interface for every dependency.

⚠️ PART 3: Common Mistakes and Important Differences

1. ⚔️ ISP vs DIP

ISP	DIP
Interface Segregation Principle	Dependency Inversion Principle
Focuses on interface size and design	Focuses on dependency direction
Avoids unnecessary methods	Avoids dependence on concrete implementations
Uses small, focused interfaces	Uses abstractions between modules
Example: separate payment capabilities	Example: `PaymentService` depends on a `PaymentGateway` abstraction

🧠 ISP: "Don't force me to implement methods I don't need." DIP: "Don't force me to depend on a specific implementation."

2. 📋 Abstraction vs concretion: quick revision

Example	Abstraction	Concrete implementation
Notification	`NotificationService`	`EmailService`
Database	`UserRepository`	`PostgreSQLUserRepository`
Payment	`PaymentGateway`	`RazorpayGateway`
Storage	`FileStorage`	`S3Storage`
Authentication	`AuthProvider`	`GoogleAuthProvider`

📝 These are examples of possible designs. An abstraction doesn't have to be a Python ABC. It can also be a Protocol or another suitable contract.

3. ❌ Common mistakes to avoid

Mistake 1: thinking DIP and DI are the same. They aren't. DIP is a principle; DI is a technique.

Mistake 2: thinking every interface must have only one method. No. Interfaces should group related operations. They should just avoid forcing unrelated operations on clients.

Mistake 3: thinking abstraction means hiding all code. No. Abstraction means exposing essential functionality while hiding unnecessary implementation details.

Mistake 4: thinking DIP means never depending on concrete classes. Not exactly. Concrete implementations must exist and be instantiated somewhere. The important thing is that high-level business logic shouldn't be tightly coupled to them.

Mistake 5: thinking FastAPI's Depends() automatically guarantees DIP. No. It provides dependency injection, but your architecture must still be designed appropriately.

🎤 PART 4: Interview Preparation

Q1. What is the Interface Segregation Principle?
ISP states that clients should not be forced to depend on interfaces they do not use. We divide large interfaces into smaller, focused ones so classes only implement relevant methods.

Q2. What is a fat interface?
An interface containing too many methods, often including unrelated operations that force implementing classes to provide unnecessary functionality.

Q3. What is the Dependency Inversion Principle?
DIP states that high-level modules should not depend directly on low-level modules. Both should depend on abstractions. This reduces coupling and makes applications easier to maintain and test.

Q4. What's the difference between abstraction and implementation?
Abstraction defines what functionality should be provided, while implementation defines how it's performed.

Q5. What is tight coupling?
When one component strongly depends on another component's specific implementation, making changes and replacements difficult.

Q6. What is Dependency Injection?
A technique in which a class receives its required dependencies from an external source instead of creating them internally.

Q7. How does FastAPI support Dependency Injection?
Through Depends(), which lets dependencies such as services, database sessions and authentication logic be provided to endpoints.

Q8. How does DIP improve testing?
Since business logic depends on abstractions, we can inject mock or fake implementations instead of connecting to actual external services or databases.

🧪 PART 5: Test Your Understanding!

Try these before peeking at the answers! ❤️

A class is forced to implement a method that it never uses. Which principle is violated?
- A) DIP
- B) ISP
- C) SRP
- D) OCP
An OrderService directly creates EmailService inside its constructor. What's the main design problem?
- A) Tight coupling
- B) Encapsulation
- C) Polymorphism
- D) Normalization
What should high-level and low-level modules depend on, according to DIP?
- A) Concrete classes
- B) Database tables
- C) Abstractions
- D) Global variables
You pass a fake repository into UserService for testing. What technique are you using?
- A) Inheritance only
- B) Dependency Injection
- C) Normalization
- D) Overloading
Which is a concrete implementation?
- A) NotificationService abstraction
- B) UserRepository contract
- C) EmailService
- D) Payment interface

👀 Click to reveal answers

B: ISP. The class is being forced to depend on a method it doesn't use
A: Tight coupling. The service is bound to one specific implementation
C: Abstractions. Both layers should depend on contracts
B: Dependency Injection. The dependency is supplied from outside
C: EmailService is an actual class that provides the implementation

📝 PART 6: Final Revision, All Five SOLID Principles

Letter	Principle	Idea	Example
S	Single Responsibility	One class should have one reason to change	Separate email sending from order processing
O	Open/Closed	Open for extension, closed for modification	Add a new payment method without modifying existing payment logic
L	Liskov Substitution	Child classes should be replaceable wherever their parent is expected, without breaking correct behavior	Any valid subtype of a payment processor satisfies the payment contract
I	Interface Segregation	Don't force classes to implement methods they don't need	Separate `Workable`, `LeaveApprover` and `CodeWriter` interfaces
D	Dependency Inversion	Depend on abstractions, not concrete implementations	`UserService` depends on `UserRepository`, not `PostgreSQLUserRepository`

🧭 Five Questions to Ask When Designing Your Backend

S: Does this class have too many responsibilities?
O: Can I add new functionality without unnecessarily changing existing code?
L: Can I replace a child implementation without breaking the expected behavior?
I: Am I forcing a class to implement unnecessary methods?
D: Am I depending on a concrete implementation when I should depend on an abstraction?

If you can answer these while designing your FastAPI services, repositories and database layers, you're starting to think like a software engineer, not just someone who writes code. 🧠

If this post helped you, drop a ❤️ or 🦄 and share it with a friend learning software design. Happy coding! 👩‍💻✨

SOLID Principles Deep Dive (S, O, L): Explained from Absolute Zero 🧱

Deepika Pusala — Thu, 12 Nov 2026 06:11:35 +0000

🧱 SOLID Principles Deep Dive (S, O, L): Let's Learn from ZERO!

SOLID shows up in Python, FastAPI, Java, Spring Boot, React architecture and almost every real-world codebase. Interviewers and mentors rarely stop at "define SOLID". They ask why we use it, what problem it solves, and how you apply it in a project. 🎯

In this post we'll cover the first three principles, S, O and L, step by step, with simple examples and practical code. 🚀

1. 🤔 First, What Are SOLID Principles?

Imagine you're building an Instagram-like social media app with registration and login, posts, image uploads, likes, comments, notifications and follows. At first you write everything in one huge class:

class SocialMedia:
    def register_user(self):
        pass

    def login_user(self):
        pass

    def create_post(self):
        pass

    def upload_image(self):
        pass

    def like_post(self):
        pass

    def comment_on_post(self):
        pass

    def send_notification(self):
        pass

It works, right? But when the app grows to 100 features:

😵 Changing login might accidentally affect registration
😵 Changing image upload might affect post creation
🧪 Testing individual features becomes difficult
♻️ Reusing functionality becomes difficult
🐌 Adding new features gets complicated

This is tightly coupled, poorly organized code. We need a better way to design software. That's where SOLID comes in!

🔤 What does SOLID stand for?

Letter	Principle	One-line idea
S	Single Responsibility	One class should have one responsibility
O	Open/Closed	Open for extension, closed for modification
L	Liskov Substitution	Child classes should safely replace parent classes
I	Interface Segregation	Don't force classes to implement methods they don't need
D	Dependency Inversion	Depend on abstractions, not concrete implementations

Today we'll go deep on S, O and L.

2. 1️⃣ S: Single Responsibility Principle (SRP)

2.1 🤔 What does it mean?

📌 A class should have only one reason to change.

Single = one
Responsibility = one specific job
Principle = a rule to follow

In simple English: a class should focus on one particular job instead of handling several unrelated ones.

🍽️ Real-life example: a restaurant chef prepares food. The chef should not also manage customer payments, clean the whole restaurant, maintain the database or handle salaries. Those jobs belong to other people or departments. When one responsibility changes, it shouldn't unnecessarily affect everything else. The same logic applies to classes.

2.2 ❌ SRP with bad code

A user can register, be saved to a database, and receive a welcome email. A beginner might write:

class User:
    def register(self, username, email):
        # Validate user
        print("User registered")

        # Save user to database
        print("Saving user to database")

        # Send welcome email
        print("Sending welcome email")

This class has three different responsibilities:

Functionality	Responsibility
Validate user	Business validation
Save user	Database operations
Send email	Email communication

Now imagine:

📧 The email provider changes from Gmail SMTP to SendGrid → modify User
🗄️ The database changes from PostgreSQL to MySQL → modify User
✅ Validation rules change → modify User

The class has multiple reasons to change. That's an SRP violation. 🚨

2.3 ✅ Correct implementation using SRP

Separate responsibilities into different classes:

UserService     → handles user registration business logic
UserRepository  → database operations
EmailService    → email operations

Class 1: UserRepository (only database operations)

class UserRepository:
    def save_user(self, username, email):
        print("Saving user to database")

Class 2: EmailService (only sending emails)

class EmailService:
    def send_welcome_email(self, email):
        print("Sending welcome email")

Class 3: UserService (coordinates the registration process)

class UserService:
    def __init__(self, repository, email_service):
        self.repository = repository
        self.email_service = email_service

    def register_user(self, username, email):
        print("Validating user")

        self.repository.save_user(username, email)
        self.email_service.send_welcome_email(email)

        print("Registration completed")

Run it:

repository = UserRepository()
email_service = EmailService()

user_service = UserService(
    repository,
    email_service,
)

user_service.register_user(
    "Deepika",
    "deepika@gmail.com",
)

Validating user
Saving user to database
Sending welcome email
Registration completed

Each class now has a clear responsibility.

📝 UserService still coordinates several steps, but its responsibility is the registration use case. Having multiple method calls doesn't automatically violate SRP.

2.4 ✅ Why is SRP important?

Benefit	Explanation
🛠️ Maintainability	Easier to modify individual functionality
♻️ Reusability	Reuse `EmailService` in other modules
🧪 Testability	Test each class independently
📖 Readability	Code becomes easier to understand
🛡️ Reduced side effects	Changes are less likely to break unrelated features

2.5 ⚡ SRP in a FastAPI backend

backend/
└── app/
    ├── api/
    │   └── routes/
    │       ├── auth.py
    │       └── posts.py
    │
    ├── services/
    │   ├── auth_service.py
    │   └── post_service.py
    │
    ├── repositories/
    │   ├── user_repository.py
    │   └── post_repository.py
    │
    └── models/
        ├── user.py
        └── post.py

Layer	Responsibility
🛣️ API routes	Receive HTTP requests and return responses
🧠 Services	Handle business logic
📚 Repositories	Communicate with the database
🗄️ Models	Represent database entities

For example: auth.py handles login and registration endpoints, auth_service.py handles authentication business logic, user_repository.py handles user database operations, and user.py represents the User database model. This separation supports SRP.

🎤 Interview question: What happens if we put database queries, business logic and HTTP request handling inside one route function?

Answer: The route becomes responsible for too many things. It becomes harder to test, maintain, reuse and modify independently.

3. 2️⃣ O: Open/Closed Principle (OCP)

3.1 🤔 What does it mean?

📌 Software entities should be open for extension but closed for modification.

It sounds confusing at first, so let's take each phrase:

Open for extension → we can add new functionality
Closed for modification → we avoid repeatedly changing existing, working code to support every new requirement

⚠️ It does NOT mean existing code can never be changed. It means designing code so that adding a feature usually doesn't require modifying existing, stable logic.

💳 Real-life example: a payment app first supports credit and debit cards. Later the company adds UPI, PayPal and net banking. A poorly designed system forces you to change the same payment-processing function every time. A better design lets you add new payment methods without rewriting the existing payment flow.

3.2 ❌ OCP with bad code

class PaymentService:
    def pay(self, payment_type, amount):
        if payment_type == "card":
            print("Processing card payment")
        elif payment_type == "upi":
            print("Processing UPI payment")
        elif payment_type == "paypal":
            print("Processing PayPal payment")

It works! But tomorrow we add Google Pay, so we modify pay(). Next month another method, and we modify pay() again. The method keeps growing with new conditions, which makes it harder to maintain and increases the chance of breaking existing functionality.

3.3 ✅ Correct implementation using OCP

We use abstraction and polymorphism. Don't worry, we'll explain both.

First, a common interface using Python's ABC. An interface is essentially a contract that says which methods an implementation must provide.

from abc import ABC, abstractmethod


class PaymentMethod(ABC):
    @abstractmethod
    def pay(self, amount):
        pass

ABC lets us define an abstract base class
@abstractmethod declares a method that child classes must implement
PaymentMethod defines the common contract

Now the different payment methods:

class CardPayment(PaymentMethod):
    def pay(self, amount):
        print(f"Paid {amount} using Card")


class UPIPayment(PaymentMethod):
    def pay(self, amount):
        print(f"Paid {amount} using UPI")


class PayPalPayment(PaymentMethod):
    def pay(self, amount):
        print(f"Paid {amount} using PayPal")

The payment service:

class PaymentService:
    def process_payment(self, payment_method, amount):
        payment_method.pay(amount)

Run it:

service = PaymentService()

card = CardPayment()
upi = UPIPayment()
paypal = PayPalPayment()

service.process_payment(card, 1000)
service.process_payment(upi, 500)
service.process_payment(paypal, 2000)

Paid 1000 using Card
Paid 500 using UPI
Paid 2000 using PayPal

Now the business introduces Google Pay. We simply add another class:

class GooglePay(PaymentMethod):
    def pay(self, amount):
        print(f"Paid {amount} using Google Pay")

google_pay = GooglePay()
service.process_payment(google_pay, 1500)

Paid 1500 using Google Pay

Look carefully: we did not modify the existing PaymentService class. We extended the system by adding a new implementation. OCP achieved! 🎉

3.4 🎭 What is polymorphism here?

📌 Polymorphism means one common interface, many different implementations.

payment_method.pay(amount)

The same method call behaves differently depending on the object passed:

Object	What happens?
`CardPayment`	Card payment
`UPIPayment`	UPI payment
`PayPalPayment`	PayPal payment
`GooglePay`	Google Pay payment

Same method name, different behavior. That's polymorphism.

3.5 ✅ Why is OCP important?

➕ Adding new features becomes easier
🧱 Existing tested code stays stable
🔁 Reduces repeated modifications
📈 Improves scalability
🛡️ Reduces the risk of introducing bugs into existing functionality

3.6 📱 OCP in a social media app

Your app supports email notifications at first. Later you add push, SMS and in-app notifications. Instead of one huge function with many if/elif branches, create a common notification interface. Each notification type implements its own sending behavior. When a new type arrives, you add a new implementation and don't rewrite the main notification-processing logic.

🎤 Interview question: Why is OCP useful in a growing application?

Answer: Requirements change frequently. OCP lets developers add new functionality through extension instead of repeatedly modifying existing, stable code, reducing the risk of breaking existing features.

4. 3️⃣ L: Liskov Substitution Principle (LSP)

Probably the trickiest of the three at first. But once you see the example, it's easy. 😄

4.1 🤔 What does it mean?

📌 Objects of a parent class should be replaceable with objects of its child classes without breaking the correctness of the program.

Parent class = base class
Child class = derived class
Substitution = replacing one thing with another

In simple English: if a child class inherits from a parent class, the child should behave in a way that is compatible with what the parent promises. Wherever you use a child object instead of a parent object, the application should keep working correctly.

🧬 First, understand inheritance

Inheritance is when one class acquires properties and methods from another:

class Animal:
    def eat(self):
        print("Animal is eating")


class Dog(Animal):
    def bark(self):
        print("Dog is barking")

Animal is the parent, Dog is the child, and Dog inherits eat():

dog = Dog()
dog.eat()
dog.bark()

Animal is eating
Dog is barking

But LSP asks something more important: can we safely use the child class wherever the parent class is expected?

4.2 🐦 Real-life example: birds

class Bird:
    def fly(self):
        print("Bird is flying")


class Sparrow(Bird):
    def fly(self):
        print("Sparrow is flying")


class Eagle(Bird):
    def fly(self):
        print("Eagle is flying")

Both can fly, so everything is fine. Now:

def make_bird_fly(bird):
    bird.fly()


make_bird_fly(Sparrow())
make_bird_fly(Eagle())

Sparrow is flying
Eagle is flying

The child objects substitute for the parent type without problems. That's LSP. ✅

4.3 ❌ Understanding an LSP violation

Now we introduce a Penguin. Can a penguin fly? No! But our parent class says every Bird has a fly() method.

class Penguin(Bird):
    def fly(self):
        raise Exception("Penguins cannot fly")


penguin = Penguin()
make_bird_fly(penguin)

Exception: Penguins cannot fly

make_bird_fly() expects a bird that can fly. Penguin inherits from Bird but breaks that expectation. The parent promises flying behavior, and the child can't deliver it. This is a classic Liskov Substitution Principle violation. 🚨

4.4 ✅ How do we fix it?

Redesign the hierarchy. Instead of saying every Bird can fly, separate general bird behavior from flying behavior:

class Bird:
    def eat(self):
        print("Bird is eating")


class FlyingBird(Bird):
    def fly(self):
        print("Bird is flying")


class Sparrow(FlyingBird):
    def fly(self):
        print("Sparrow is flying")


class Eagle(FlyingBird):
    def fly(self):
        print("Eagle is flying")


class Penguin(Bird):
    def swim(self):
        print("Penguin is swimming")

Bird           → common bird behavior: eat()
 ├── FlyingBird → fly()
 │     ├── Sparrow
 │     └── Eagle
 └── Penguin    → swim()   (does not promise flying behavior)

Now:

def make_bird_fly(bird: FlyingBird):
    bird.fly()


make_bird_fly(Sparrow())
make_bird_fly(Eagle())

We don't pass Penguin. The design now reflects the actual capabilities of the objects.

💡 LSP isn't simply about whether inheritance exists. It's about whether the child class honors the behavioral contract of the parent class.

4.5 📐 Another important example: Rectangle and Square

A famous interview example. Mathematically a square is a special kind of rectangle, so a developer might try this:

class Rectangle:
    def __init__(self, width, height):
        self.width = width
        self.height = height

    def set_width(self, width):
        self.width = width

    def set_height(self, height):
        self.height = height

    def area(self):
        return self.width * self.height


class Square(Rectangle):
    def set_width(self, width):
        self.width = width
        self.height = width

    def set_height(self, height):
        self.width = height
        self.height = height

Test it:

def calculate_area(rectangle):
    rectangle.set_width(5)
    rectangle.set_height(10)
    return rectangle.area()

With a normal rectangle:

rectangle = Rectangle(2, 3)
print(calculate_area(rectangle))   # 50 ✅

With a square:

square = Square(2, 2)
print(calculate_area(square))      # 100 ❌

Why? Setting the height to 10 also changes the square's width to 10. The function expected a rectangle whose width and height can change independently, and Square breaks that expectation. That's an LSP violation.

🧠 The lesson: a mathematically valid inheritance relationship doesn't always make a good software inheritance relationship.

4.6 🔍 How to spot LSP violations

Warning signs:

A child method throws an exception for behavior the parent promises to support
A child method does nothing when the parent promises meaningful behavior
A child changes the expected output or behavior in an incompatible way
A child requires stricter inputs than the parent
A child returns weaker or incompatible results than the parent contract promises

There are deeper formal rules behind this:

Preconditions should not be strengthened
Postconditions should not be weakened
Parent invariants should be preserved

Term	Simple meaning
Preconditions	What must be true before a method runs
Postconditions	What the method guarantees after execution
Invariants	Rules that must stay true throughout the object's valid lifetime

For example, if a parent method accepts any positive number, a child shouldn't suddenly accept only numbers greater than 100. That would strengthen the precondition and violate LSP.

4.7 ✅ Why is LSP important?

🔒 Makes inheritance reliable
🚫 Prevents unexpected runtime errors
♻️ Improves code reusability
🎭 Makes polymorphism safer
🏗️ Helps you design correct class hierarchies
🧪 Makes unit testing easier, because child objects can be tested against parent expectations

5. ⚖️ Comparing S, O and L

Principle	Main question	Problem it prevents
S: SRP	Does this class have one clear responsibility?	Classes doing too many unrelated jobs
O: OCP	Can I add features without repeatedly modifying stable code?	Constant modification of existing logic
L: LSP	Can a child object replace its parent safely?	Incorrect inheritance and unexpected behavior

One example to remember each:

🅢 SRP: a User class shouldn't handle everything. Separate database operations, email sending and registration.
🅞 OCP: adding Google Pay shouldn't require rewriting PaymentService. Add a new implementation of the common contract.
🅛 LSP: a Penguin shouldn't be forced to implement flying. Design inheritance around actual behavioral capabilities.

6. 🤝 How Are These Principles Related?

They're separate, but they complement each other. Imagine an e-commerce backend with PaymentService, EmailService, OrderService, PaymentMethod, CardPayment and UPIPayment:

SRP: each class has a well-defined responsibility
OCP: you can introduce a new payment method without changing the existing payment-processing logic
LSP: every payment implementation honors the common PaymentMethod contract, so the payment service can safely use any of them

One good design can follow all three principles at once. ✨

7. ⚠️ Common Mistakes with SOLID

Following SOLID doesn't mean creating hundreds of unnecessary classes.

Mistake	Correct understanding
"SRP means one method per class"	No. It means one cohesive responsibility per class
"OCP means never modify existing code"	No. It means designing for extension without unnecessary modification
"LSP means every child must have exactly the same code as its parent"	No. Children can implement things differently but must preserve expected behavior
"More classes always mean better SOLID design"	No. Avoid unnecessary complexity
"Inheritance automatically means LSP is satisfied"	No. Behavioral compatibility is what matters

8. 🎤 Interview Questions and Answers

Q1. What is SOLID?
A collection of five object-oriented design principles that help developers write maintainable, flexible, reusable and scalable software.

Q2. What's the difference between SRP and OCP?
SRP ensures a class has one clear responsibility. OCP allows new functionality to be added without repeatedly modifying existing, stable code.

Q3. What does "one reason to change" mean in SRP?
A class should be responsible for one cohesive area of functionality. If that responsibility changes, the class may need modification.

Q4. How do we implement OCP in Python?
Using abstraction, abstract base classes, inheritance and polymorphism to create extensible designs.

Q5. What is polymorphism?
It lets different objects respond to the same method call in their own ways.

Q6. What happens when LSP is violated?
A child object may cause unexpected errors or incorrect behavior when used in place of its parent object.

Q7. Is a Square a Rectangle? Why can it violate LSP?
Mathematically yes. But if a Rectangle lets width and height change independently, a Square that forces them to stay equal violates the expected behavior of that implementation.

Q8. Are SOLID principles only for Java?
No. They apply to object-oriented design in Python, Java, C++, C# and other languages, and their ideas also help when designing larger application architectures.

9. 🧪 Test Your Understanding!

Try these before peeking at the answers! ❤️

A class handles user registration, database operations and email sending. Which principle is violated?
- A) OCP
- B) SRP
- C) LSP
You add Google Pay by creating a new payment class without modifying PaymentService. Which principle are you following?
- A) SRP
- B) LSP
- C) OCP
A Penguin inherits from Bird, but the parent promises every bird can fly. Which principle is violated?
- A) LSP
- B) OCP
- C) SRP
Different payment classes implement the same pay() method differently. What OOP concept is this?
- A) Encapsulation
- B) Polymorphism
- C) Normalization
A child class requires stricter input conditions than its parent method. Which principle may be violated?
- A) SRP
- B) OCP
- C) LSP

👀 Click to reveal answers

B: SRP. The class has several unrelated reasons to change
C: OCP. You extended the system without modifying existing code
A: LSP. The child can't honor the parent's behavioral contract
B: Polymorphism. One interface, many implementations
C: LSP. Strengthening preconditions breaks substitutability

10. 📝 Final Revision Notes

🅢 Single Responsibility Principle: one class = one clear responsibility = one main reason to change
🅞 Open/Closed Principle: open for extension, closed for unnecessary modification
🅛 Liskov Substitution Principle: child objects must be safely usable wherever their parent objects are expected

💡 Don't just memorize the definitions. Whenever you write a FastAPI service, repository, model or authentication module, ask yourself:

Is this class doing too many jobs?

Can I add a new feature without rewriting existing logic?

If I inherit from another class, will my child class preserve its expected behavior?

That's how you start thinking like a software engineer, not just someone who writes code. 🧠

If this post helped you, drop a ❤️ or 🦄 and share it with a friend learning software design. Happy coding! 👩‍💻✨

Transactions & Basic Indexing in PostgreSQL: COMMIT, ROLLBACK, EXPLAIN & SQLAlchemy ⚛️

Deepika Pusala — Wed, 11 Nov 2026 07:35:30 +0000

⚛️ Transactions & Basic Indexing

This is another VERY important backend concept, especially when you work with PostgreSQL, SQLAlchemy, FastAPI and real-world applications. 💪

Imagine you're building an e-commerce app 🛒. A user buys a product and your app must reduce its stock. Now imagine something breaks halfway:

✅ Product A's stock gets reduced
❌ Product B's stock does not get increased
💥 Your database now holds incorrect information

How do we prevent this? That's exactly where transactions, ROLLBACK, COMMIT and indexing come in.

We'll learn everything from absolute zero, with SQL examples, real-world analogies, PostgreSQL experiments and FastAPI connections. 🚀

📚 Table of Contents

What is a database transaction?
BEGIN, COMMIT and ROLLBACK
Atomicity and Consistency in practice
Hands-on: transfer stock between two products
What are database indexes?
Primary, Unique and Composite indexes
Other index types worth knowing
EXPLAIN and EXPLAIN ANALYZE
Hands-on: add an index and compare plans
Transactions in SQLAlchemy 2.0
Indexes in SQLAlchemy models and Alembic
Real-world usage
Common beginner mistakes
Quick revision sheet
Quiz

1️⃣ What is a database transaction?

Here's a products table:

products

product_id	product_name	stock
1	Laptop	50
2	Mobile	30
3	Headphones	100

A customer buys 5 laptops, so your app runs:

UPDATE products
SET stock = stock - 5
WHERE product_id = 1;

Laptop stock is now 45. But what if...

💥 the app crashes right after this update?
💥 the second query fails?
💥 the database connection drops?

We need a way to make a group of related operations succeed together or fail together. That mechanism is a transaction.

📦 Definition: a database transaction is a group of one or more SQL operations treated as a single logical unit of work.

The principle:

✅ If all operations succeed → save the changes
❌ If something fails → undo the changes made by that transaction

💸 Bank transfer example

Transfer ₹1,000 from Account A to Account B.

Before

Account	Balance
A	₹5,000
B	₹3,000

Two operations are required:

Subtract ₹1,000 from A
Add ₹1,000 to B

Expected after

Account	Balance
A	₹4,000
B	₹4,000

But if only the first operation succeeds:

Account	Balance
A	₹4,000
B	₹3,000

Where did ₹1,000 go? 😱 That's an inconsistent state. A transaction prevents it by making both operations succeed together, or neither.

🔄 How a transaction flows

BEGIN                      → start the transaction
  ├─ Operation 1           → subtract ₹1,000 from A
  ├─ Operation 2           → add ₹1,000 to B
  │
  ├─ all good?  → COMMIT   → save changes permanently ✅
  └─ failure?   → ROLLBACK → undo everything ❌

Three SQL commands drive this: BEGIN, COMMIT and ROLLBACK.

2️⃣ BEGIN, COMMIT and ROLLBACK

2.1 🚦 `BEGIN`: start a transaction

It tells the database: "I'm starting a transaction. Treat the upcoming operations as one unit."

BEGIN;

UPDATE products
SET stock = stock - 5
WHERE product_id = 1;

The database starts tracking the changes. They are not yet permanent.

2.2 💾 `COMMIT`: save the transaction

It tells the database: "Everything worked. Save all changes permanently."

BEGIN;

UPDATE products
SET stock = stock - 5
WHERE product_id = 1;

COMMIT;

Transaction begins
Stock is reduced by 5
The database commits
Changes become permanent ✅

2.3 ↩️ `ROLLBACK`: undo the transaction

It tells the database: "Something went wrong. Undo all changes made in this transaction."

BEGIN;

UPDATE products
SET stock = stock - 5
WHERE product_id = 1;

ROLLBACK;

Moment	Laptop stock
Initially	50
After the `UPDATE` (inside the transaction)	45
After `ROLLBACK`	50 ✅ (restored)

2.4 📊 The three commands

Command	Meaning
`BEGIN`	Start a transaction
`COMMIT`	Save changes
`ROLLBACK`	Undo uncommitted changes

⚠️ PostgreSQL detail: if a statement fails inside a transaction, PostgreSQL marks the transaction as aborted. Until you run ROLLBACK, further ordinary statements in that transaction are rejected.

3️⃣ Atomicity and Consistency in practice ⚛️

Remember ACID?

A: Atomicity
C: Consistency
I: Isolation
D: Durability

Today we focus on Atomicity and Consistency.

3.1 ⚛️ Atomicity

All operations in a transaction happen together, or none of them happen.

Think of an electrical switch: it's ON or OFF, never "half on". 💡

A bank transfer needs both:

UPDATE accounts SET balance = balance - 1000 WHERE account_id = 1;
UPDATE accounts SET balance = balance + 1000 WHERE account_id = 2;

If the second query fails, the first must be undone. That's atomicity.

❌ WITHOUT atomicity
   Account A: ₹5,000 → updated
   Account B: ₹3,000 → FAILED
   Result: money disappeared from the system!

✅ WITH atomicity
   Account A: ₹5,000 → restored
   Account B: ₹3,000 → unchanged
   Result: the whole transaction was rolled back.

3.2 ✅ Consistency

A transaction must take the database from one valid state to another valid state.

Suppose your rule is: stock must never be negative. We enforce it with a constraint:

CHECK (stock >= 0)

Laptop stock is 10, and a customer tries to buy 15:

UPDATE products
SET stock = stock - 15
WHERE product_id = 1;

That would give 10 - 15 = -5. PostgreSQL rejects the operation because it violates the CHECK constraint. The database never enters an invalid state. 🛡️

3.3 ⚖️ Atomicity vs Consistency

Atomicity	Consistency
All or nothing	Valid state → valid state
Focuses on transaction execution	Focuses on database rules
Uses `COMMIT` and `ROLLBACK`	Uses constraints and correct transaction logic

🧠 Atomicity protects a transaction from partial completion. Consistency protects the correctness of the data.

4️⃣ Hands-on: transfer stock between two products 🧪

You have two products:

product_id	product	stock
1	Laptop	50
2	Mobile	30

Goal: move 10 units from Laptop to Mobile. After success: Laptop = 40, Mobile = 40. Then we'll deliberately fail halfway to see ROLLBACK in action.

Step 1️⃣: create the table

CREATE TABLE products (
    product_id   INTEGER PRIMARY KEY,
    product_name VARCHAR(100),
    stock        INTEGER CHECK (stock >= 0)
);

product_id → unique identifier
product_name → the name
stock → available quantity
CHECK (stock >= 0) → blocks negative stock

Step 2️⃣: insert sample data

INSERT INTO products
VALUES
    (1, 'Laptop', 50),
    (2, 'Mobile', 30);

SELECT * FROM products;

product_id	product_name	stock
1	Laptop	50
2	Mobile	30

Step 3️⃣: a successful transfer ✅

BEGIN;

-- Reduce Laptop stock by 10
UPDATE products
SET stock = stock - 10
WHERE product_id = 1;

-- Increase Mobile stock by 10
UPDATE products
SET stock = stock + 10
WHERE product_id = 2;

COMMIT;

Product	Before	After
Laptop	50	40
Mobile	30	40

Both operations succeeded, so we committed.

Step 4️⃣: deliberately fail midway 💥

First, reset the stock:

UPDATE products SET stock = 50 WHERE product_id = 1;
UPDATE products SET stock = 30 WHERE product_id = 2;

Now run this:

BEGIN;

-- Operation 1: reduce Laptop stock (this works)
UPDATE products
SET stock = stock - 10
WHERE product_id = 1;

-- Operation 2: deliberately cause an error (division by zero!)
UPDATE products
SET stock = stock / 0
WHERE product_id = 2;

-- Undo everything
ROLLBACK;

What happens:

Operation 1 executes inside the transaction, and Laptop stock is temporarily 40
Operation 2 fails with division by zero
PostgreSQL marks the transaction as aborted
We run ROLLBACK;, and the first operation is undone too ↩️

Product	Initial	After rollback
Laptop	50	50 ✅
Mobile	30	30 ✅

Atomicity demonstrated! 🎉

💡 Practical detail: after the error, you must send ROLLBACK; as a separate statement. The remaining statements in an aborted transaction can't run normally. (If you paste everything in one go in some clients, the error may stop the script before reaching ROLLBACK. Just run ROLLBACK; yourself afterwards.)

🧪 Bonus: break the CHECK constraint

BEGIN;

UPDATE products
SET stock = stock - 100   -- Laptop only has 50!
WHERE product_id = 1;
-- ERROR: new row violates check constraint

ROLLBACK;

This shows Consistency protecting you, and Atomicity cleaning up. 🛡️

5️⃣ What are database indexes? 📇

Now the second big concept: indexing.

Imagine a library with 1,00,000 books 📚. A student asks: "Where's the book Database Management Systems?"

❌ Without an index: you check Book 1, Book 2, Book 3... up to Book 1,00,000. You may examine almost every book.

✅ With an index: the library has an alphabetical index. You jump to D, find the location, done. Much faster!

📦 Definition: a database index is a data structure that helps the database find rows faster without scanning the entire table.

users

user_id	username	email
1	deepika	deepika@gmail.com
2	ishu	ishu@gmail.com
3	rahul	rahul@gmail.com
4	priya	priya@gmail.com

SELECT *
FROM users
WHERE email = 'ishu@gmail.com';

Without an index → PostgreSQL may scan the whole table
With a suitable index → PostgreSQL can locate the row efficiently

⚠️ Indexes have costs!

Every INSERT, UPDATE or DELETE may also need to update the indexes

Indexes consume storage space

So never create indexes on every column blindly.

6️⃣ Types of indexes you should know

6.1 🆔 Primary Key index

CREATE TABLE users (
    user_id  INTEGER PRIMARY KEY,
    username VARCHAR(100),
    email    VARCHAR(150)
);

PostgreSQL automatically creates a unique B-tree index for the primary key, because every user_id must be unique.

user_id	username
1	Deepika
2	Rahul
3	Priya

You can't insert another row with user_id = 1. The database rejects the duplicate.

Facts to remember:

✅ A primary key can't contain NULL
✅ A table has only one primary key constraint
✅ It can span one or multiple columns
✅ PostgreSQL auto-creates a unique index to enforce it

You generally don't need to manually index a primary key again.

6.2 🦄 Unique index

A unique index prevents duplicate values in the indexed column (or column combination).

CREATE UNIQUE INDEX idx_users_email
ON users(email);

Now:

INSERT INTO users
VALUES (5, 'john', 'deepika@gmail.com');

If that email already exists, PostgreSQL rejects the insert.

Useful for:

📧 Preventing duplicate email accounts
👤 Preventing duplicate usernames
🆔 Preventing duplicate employee IDs
📏 Enforcing uniqueness in business data

Primary Key	Unique Index
Identifies a row	Enforces uniqueness
Cannot contain `NULL`	Can allow `NULL` values
One per table	Multiple unique indexes possible
Automatically creates a unique index	Created explicitly or via a `UNIQUE` constraint

🧠 Nuance: PostgreSQL unique indexes normally allow multiple NULLs, since NULLs aren't considered equal. You can change that with NULLS NOT DISTINCT (PostgreSQL 15+).

A UNIQUE constraint also creates a unique index automatically:

CREATE TABLE users (
    user_id INTEGER PRIMARY KEY,
    email   VARCHAR(150) UNIQUE
);

No need for a second unique index on email.

6.3 🧩 Composite index

A composite index covers multiple columns together. It's especially important in backend work.

orders

order_id	user_id	status	amount
1	101	completed	500
2	102	pending	700
3	101	pending	300
4	103	completed	900

Your app often runs:

SELECT *
FROM orders
WHERE user_id = 101
  AND status = 'pending';

We filter on two columns, so we create one index on both:

CREATE INDEX idx_orders_user_status
ON orders(user_id, status);

🚨 Column order matters!

The index (user_id, status) is useful for:

WHERE user_id = 101

and:

WHERE user_id = 101 AND status = 'pending'

But it is generally less useful for:

WHERE status = 'pending'

Why? The index is organized by user_id first, then status.

📞 Analogy: a phone directory sorted by last name, then first name. Know the last name? Easy. Only know the first name? You'd basically have to read the whole book.

This is the leftmost-prefix principle. For PostgreSQL B-tree indexes, the leading column matters most, though the planner can sometimes still use the index in other ways.

When to use composite indexes

When your app repeatedly filters on multiple columns:

WHERE user_id = 10 AND status = 'active'
WHERE category_id = 5 AND created_at > '2026-01-01'
WHERE tenant_id = 20 AND email = 'abc@gmail.com'

The right column order depends on your actual query patterns.

7️⃣ Other index types worth knowing 🧰

Index type	Purpose
B-tree	General-purpose: equality, range and sorting (the default)
Hash	Equality comparisons
GIN	Searching inside arrays, JSONB and full-text data
GiST	Geometric, range and specialized searches
BRIN	Very large tables whose values correlate with physical order (like timestamps)

For now, focus on B-tree. It's what PostgreSQL uses by default.

CREATE INDEX idx_products_name
ON products USING BTREE (product_name);

Useful for:

SELECT * FROM products WHERE product_name = 'Laptop';

and for range comparisons on sortable values.

8️⃣ EXPLAIN: how does the database run your query? 🔍

You write:

SELECT *
FROM users
WHERE email = 'deepika@gmail.com';

You know what you want. But how does PostgreSQL find it? Does it:

Scan every row?
Use an index?
Use another strategy?

PostgreSQL has a query planner that picks an execution plan. EXPLAIN lets you see it.

📦 Definition: EXPLAIN displays the execution plan PostgreSQL intends to use for a query.

EXPLAIN
SELECT *
FROM users
WHERE email = 'deepika@gmail.com';

Possible output:

Seq Scan on users
  Filter: (email = 'deepika@gmail.com')

8.1 📖 Sequential Scan (Seq Scan)

PostgreSQL reads the table's rows one after another to find matches, like searching a notebook page by page. 📓

EXPLAIN
SELECT *
FROM users
WHERE username = 'deepika';

With no suitable index, PostgreSQL may choose a Seq Scan.

❓ Is a Seq Scan always bad? NO! For a table with only 10 rows, reading everything is cheaper than using an index. Even with an index, PostgreSQL may choose a Seq Scan if it estimates that reading most of the table is cheaper.

8.2 🎯 Index Scan

PostgreSQL uses an index to locate matching rows, then fetches those rows from the table.

CREATE INDEX idx_users_username
ON users(username);

EXPLAIN
SELECT *
FROM users
WHERE username = 'deepika';

Possible output:

Index Scan using idx_users_username on users
  Index Cond: (username = 'deepika')

This tells us PostgreSQL found a suitable index and used it. ✅

8.3 ⚡ Index Only Scan

Sometimes PostgreSQL can answer the query using only the index, without touching the table.

CREATE INDEX idx_users_email
ON users(email);

EXPLAIN
SELECT email
FROM users
WHERE email = 'deepika@gmail.com';

PostgreSQL may choose an Index Only Scan because the requested column is already in the index. (It may still check visibility information via the visibility map.)

8.4 📏 EXPLAIN ANALYZE

EXPLAIN → shows the estimated plan
EXPLAIN ANALYZE → actually runs the query and reports measured details

EXPLAIN ANALYZE
SELECT *
FROM users
WHERE email = 'deepika@gmail.com';

Sample output:

Index Scan using idx_users_email on users
  (cost=0.15..8.17 rows=1 width=...)
  (actual time=0.020..0.022 rows=1 loops=1)
Planning Time: 0.100 ms
Execution Time: 0.050 ms

Term	Meaning
`cost`	Estimated planner cost (not milliseconds)
`rows`	Estimated number of returned rows
`width`	Estimated average row size in bytes
`actual time`	Measured time in milliseconds
`loops`	Times the plan node executed
`Planning Time`	Time taken to create the plan
`Execution Time`	Time taken to run the query

🚨 Very important: EXPLAIN ANALYZE executes the query! Be careful with UPDATE, DELETE and other data-changing statements. Wrap them in a transaction and roll back:

BEGIN;

EXPLAIN ANALYZE
UPDATE products
SET stock = stock - 1
WHERE product_id = 1;

ROLLBACK;

The update runs, then gets undone. 😎 (Transactions and EXPLAIN working together!)

9️⃣ Hands-on: add an index and compare plans 🧪

Let's create a big table so the difference is visible.

Step 1️⃣: create the table

CREATE TABLE users (
    user_id  SERIAL PRIMARY KEY,
    username VARCHAR(100),
    email    VARCHAR(150),
    city     VARCHAR(100)
);

Step 2️⃣: insert 1 lakh (100,000) rows

INSERT INTO users (username, email, city)
SELECT
    'user_' || i,
    'user_' || i || '@gmail.com',
    CASE
        WHEN i % 3 = 0 THEN 'Bangalore'
        WHEN i % 3 = 1 THEN 'Hyderabad'
        ELSE 'Chennai'
    END
FROM generate_series(1, 100000) AS i;

generate_series(1, 100000) is a PostgreSQL function that generates the numbers 1 to 100000. So we insert 1 lakh sample users. 🎉

Refresh planner statistics:

ANALYZE users;

Step 3️⃣: query WITHOUT an email index

EXPLAIN ANALYZE
SELECT *
FROM users
WHERE email = 'user_50000@gmail.com';

You'll likely see:

Seq Scan on users
  Filter: (email = 'user_50000@gmail.com')
  Rows Removed by Filter: 99999

PostgreSQL scanned the table because there's no index on email.

Step 4️⃣: create the index

CREATE INDEX idx_users_email
ON users(email);

ANALYZE users;

Step 5️⃣: run the same query again

EXPLAIN ANALYZE
SELECT *
FROM users
WHERE email = 'user_50000@gmail.com';

Now you'll likely see:

Index Scan using idx_users_email on users

📊 What to compare

	Before index	After index
Scan type	Seq Scan	Index Scan
Rows examined	Whole table	Just the match
Execution time	Slower	Much faster ⚡

Compare: scan type, estimated cost, actual execution time and rows scanned or returned.

⚠️ Heads up: PostgreSQL can still choose a Seq Scan after you create an index. That doesn't mean the index is broken. The planner picks whatever it estimates is cheaper.

For learning only, you can discourage Seq Scans in your current session:

SET enable_seqscan = OFF;
-- ...run EXPLAIN ANALYZE...
RESET enable_seqscan;

🚫 Never use this as a production performance fix. It's only a way to inspect alternative plans.

🎁 Bonus experiment: composite index

-- Before
EXPLAIN ANALYZE
SELECT * FROM users
WHERE city = 'Bangalore' AND username = 'user_300';

-- Create a composite index
CREATE INDEX idx_users_city_username
ON users(city, username);
ANALYZE users;

-- After
EXPLAIN ANALYZE
SELECT * FROM users
WHERE city = 'Bangalore' AND username = 'user_300';

Then try WHERE username = 'user_300' alone and see whether the planner still uses the composite index. This helps you feel the leftmost-prefix rule. 🧠

🔟 Transactions in SQLAlchemy 2.0 🐍

Now let's connect all this to your real stack:

⚡ FastAPI
🧪 SQLAlchemy 2.0
🐘 PostgreSQL
🔄 AsyncSession

In raw SQL we write BEGIN; COMMIT; ROLLBACK;. In SQLAlchemy we use transaction methods or context managers.

10.1 ✅ Basic transaction

from sqlalchemy import update


async def transfer_stock(session):
    async with session.begin():
        await session.execute(
            update(Product)
            .where(Product.id == 1)
            .values(stock=Product.stock - 10)
        )
        await session.execute(
            update(Product)
            .where(Product.id == 2)
            .values(stock=Product.stock + 10)
        )

async with session.begin():

Starts a transaction
If the block finishes normally → COMMIT
If an exception escapes the block → ROLLBACK

async with session.begin()  →  transaction starts
    UPDATE #1
    UPDATE #2
    no exception  →  COMMIT ✅
    exception     →  ROLLBACK ↩️

⚠️ Heads up: session.begin() raises an error if a transaction has already started on that session (for example, because you already ran a query on it). In FastAPI apps where a dependency provides the session, many projects instead use await session.commit() / await session.rollback() explicitly, or session.begin_nested(). Follow whichever pattern your project already uses, consistently.

10.2 💥 Deliberately failing

async def transfer_stock(session):
    async with session.begin():
        await session.execute(
            update(Product)
            .where(Product.id == 1)
            .values(stock=Product.stock - 10)
        )

        raise Exception("Something went wrong!")

        await session.execute(          # never reached
            update(Product)
            .where(Product.id == 2)
            .values(stock=Product.stock + 10)
        )

What happens:

Transaction begins
First stock update executes
An exception occurs
The second update is never reached
SQLAlchemy rolls back the transaction
The first update is undone ↩️

That's atomicity inside your FastAPI backend! 🎉

🚨 Important: the exception must escape the async with block for the automatic rollback. If you catch and swallow it inside the block, the context manager sees a normal exit and commits. And never keep using a failed transaction without rolling it back.

10.3 🧰 The explicit style (very common in FastAPI)

async def create_post(session, user_id: int, caption: str):
    try:
        post = Post(user_id=user_id, caption=caption)
        session.add(post)
        await session.flush()          # get post.id without committing

        session.add(PostMetadata(post_id=post.id, language="en"))

        await session.commit()         # ✅ save everything
        return post
    except Exception:
        await session.rollback()       # ↩️ undo everything
        raise

flush() sends pending SQL to the database inside the transaction, but doesn't commit
commit() makes it permanent
rollback() undoes uncommitted work

1️⃣1️⃣ Adding indexes with SQLAlchemy models and Alembic 🧱

from sqlalchemy import Integer, String
from sqlalchemy.orm import Mapped, mapped_column


class User(Base):
    __tablename__ = "users"

    id: Mapped[int] = mapped_column(Integer, primary_key=True)
    username: Mapped[str] = mapped_column(String(100))
    email: Mapped[str] = mapped_column(String(150), unique=True)

primary_key=True → creates the primary key constraint (and its index)
unique=True → creates a uniqueness constraint (and its unique index)

Creating a normal index

The simplest way is index=True on the column:

username: Mapped[str] = mapped_column(String(100), index=True)

Or declare it explicitly in __table_args__:

from sqlalchemy import Index


class User(Base):
    __tablename__ = "users"

    id: Mapped[int] = mapped_column(primary_key=True)
    username: Mapped[str] = mapped_column(String(100))

    __table_args__ = (
        Index("idx_users_username", "username"),
    )

Creating a composite index

class Order(Base):
    __tablename__ = "orders"

    id: Mapped[int] = mapped_column(primary_key=True)
    user_id: Mapped[int] = mapped_column(Integer)
    status: Mapped[str] = mapped_column(String(20))

    __table_args__ = (
        Index("idx_orders_user_status", "user_id", "status"),
    )

💡 A bare Index("name", User.username) statement on its own line only works when the columns are already attached to a table. Putting indexes inside __table_args__ is the cleanest, most reliable pattern.

🧬 Using Alembic migrations

In real projects, schema changes go through Alembic migrations:

alembic revision --autogenerate -m "add user indexes"

Review the generated file, then apply it:

alembic upgrade head

You can also write a migration by hand:

from alembic import op


def upgrade():
    op.create_index(
        "idx_users_username",
        "users",
        ["username"],
    )


def downgrade():
    op.drop_index(
        "idx_users_username",
        table_name="users",
    )

upgrade() → applies the change
downgrade() → reverses it

Schema changes should be version-controlled just like application code. 📜

1️⃣2️⃣ Where are transactions and indexes used in real life? 🌍

Application	Transactions	Indexes
🏦 Banking	Money transfers	Account ID lookup
🛒 E-commerce	Stock and order updates	Product ID, category
📸 Instagram-like app	Creating a post with related records	User ID, post ID, `created_at`
🍕 Food delivery	Order creation + payment records	Restaurant ID, order status
🏥 Hospital management	Updating patient records	Patient ID
🔐 Authentication	Creating sessions	Email, token hash

📸 In a social-feed backend

When creating a post:

Insert the post record
Insert media records
Insert related metadata
Commit

If any step fails, you want the entire database operation rolled back.

And if you frequently run:

SELECT *
FROM posts
WHERE user_id = 10
ORDER BY created_at DESC;

a composite index designed around that query helps:

CREATE INDEX idx_posts_user_created
ON posts(user_id, created_at DESC);

✅ Always confirm with EXPLAIN ANALYZE on realistic data. Don't guess.

1️⃣3️⃣ Common beginner mistakes ⚠️

❌ Mistake 1: thinking every SQL statement is automatically a big safe transaction.
Without an explicit BEGIN, PostgreSQL treats each statement as its own transaction (autocommit). Database libraries often manage transactions for you, so know what yours does.

❌ Mistake 2: thinking ROLLBACK can undo everything.
It only undoes uncommitted changes in the current transaction. Once committed, an ordinary rollback can't reverse it.

❌ Mistake 3: indexing every column.
Indexes consume storage and slow down writes.

❌ Mistake 4: thinking an index always makes queries faster.
The planner may choose a Seq Scan if it's cheaper.

❌ Mistake 5: thinking COMMIT covers external actions.
A database transaction can't roll back an email, an external payment gateway call or an already-uploaded image. Those need extra patterns such as compensating actions or the transactional outbox pattern.

❌ Mistake 6: confusing consistency with atomicity.

Atomicity = all or nothing
Consistency = valid data before and after

1️⃣4️⃣ Quick revision sheet 🧾

Transactions

Command	Purpose
`BEGIN`	Start a transaction
`COMMIT`	Save changes
`ROLLBACK`	Undo uncommitted changes

ACID

⚛️ Atomicity: all or nothing
✅ Consistency: maintain valid database states
🚦 Isolation: concurrent transactions don't interfere incorrectly
💾 Durability: committed changes survive failures covered by the database's durability guarantees

Indexes

🆔 Primary key index: created automatically
🦄 Unique index: prevents duplicate indexed values
🧩 Composite index: indexes multiple columns together (order matters!)
🌳 B-tree: PostgreSQL's general-purpose default

EXPLAIN

📖 Seq Scan: reads rows sequentially
🎯 Index Scan: uses an index to find rows
⚡ Index Only Scan: answers from the index alone
📏 EXPLAIN ANALYZE: executes and measures the query

1️⃣5️⃣ Test your understanding! 🧪

Answer without scrolling up. (Answers are at the bottom.)

1. You subtract ₹1,000 from Account A, but adding it to Account B fails. What should happen?

A) Commit the first operation
B) Roll back the entire transaction
C) Ignore the error

2. Which SQL command permanently saves a successful transaction?

A) BEGIN
B) ROLLBACK
C) COMMIT

3. What does CHECK (stock >= 0) help ensure?

A) Stock cannot become negative
B) Stock is always increasing
C) Every product has a unique name

4. Which index suits a frequent query filtering by user_id and status together?

A) Composite index
B) Only a primary key index
C) No index can do this

5. What does EXPLAIN ANALYZE do?

A) Only displays the SQL query
B) Executes the query and reports actual execution details
C) Creates an index automatically

👀 Click to reveal answers

B: roll back the entire transaction (atomicity)
C: COMMIT
A: stock can't go negative (consistency)
A: a composite index on (user_id, status)
B: it runs the query and measures real timings

🎯 What to remember as a backend developer

When you build a real application, you use these concepts together:

Transactions protect multiple related database operations
COMMIT saves successful operations
ROLLBACK undoes failed operations
Atomicity prevents partial transactions
Consistency ensures database rules are maintained
Indexes help retrieve records efficiently
EXPLAIN ANALYZE helps investigate query performance
SQLAlchemy transactions manage database operations safely in FastAPI
Alembic manages index and schema changes over time

💡 One final distinction: transactions protect the correctness of your database operations, while indexes help the database retrieve information efficiently. They solve two different problems, and both are essential for reliable, scalable backend systems.

🙌 Enjoyed this post?

❤️ Drop a reaction
🦄 Save it for revision
💬 Comment with questions, and I'll answer them!

Happy coding! 👩‍💻✨

postgres #sql #database #backend #sqlalchemy

Relational Databases, SQL, Normalization & ACID: A Beginner-Friendly Guide 🗄️

Deepika Pusala — Wed, 11 Nov 2026 07:31:03 +0000

🗄️ Relational Databases, SQL, Normalization & ACID

This is one of the most important topics to understand before working with SQLAlchemy, PostgreSQL, FastAPI and backend development. 💪

We'll learn relational database concepts, SQL, normalization and ACID properties from absolute zero. No prior database knowledge is assumed, no skipped fundamentals, and no terminology without explanation.

Along the way we'll connect everything to a real Instagram-like Social Feed project (React + FastAPI + PostgreSQL), so you can see how these ideas are actually used. 🚀

📚 Table of Contents

What is a database?
Tables, rows and columns
Keys (Primary, Foreign, Composite...)
Relationships (1:1, 1:N, N:N)
SQL (DDL, DML, DQL)
JOINs
Aggregations, GROUP BY and HAVING
Normalization (1NF, 2NF, 3NF)
ACID properties and transactions
Constraints
Designing an Instagram database
SQL → SQLAlchemy → FastAPI
Interview questions
Quiz and final revision

1️⃣ First, what is a database?

Imagine you're building Instagram 📸. Your app needs to store:

👤 Users
🖼️ Posts
💬 Comments
❤️ Likes
👥 Followers
🔑 Login sessions

Where does all this go? Into a database.

📦 A database is an organized collection of data that lets applications store, retrieve, update and delete information.

React (Frontend)         → user clicks "Like"
      │  HTTP request / response
      ▼
FastAPI (Backend)        → validates, applies business logic
      │  SQL via SQLAlchemy
      ▼
PostgreSQL (Database)    → stores users, posts, likes, comments...

When a user creates a post:

User types a caption in React
React sends a request to FastAPI
FastAPI validates it
FastAPI talks to PostgreSQL
PostgreSQL stores the post
The backend returns a response
React shows the new post ✅

⚠️ The frontend does not normally talk directly to your database. The backend is the middleman.

1.1 🧩 What is a Relational Database?

A relational database stores data in tables and defines relationships between those tables.

Popular ones:

🐘 PostgreSQL
🐬 MySQL
🪶 SQLite
🏢 Microsoft SQL Server
🏛️ Oracle Database

We'll use PostgreSQL in our examples.

Why "relational"? Because tables connect to each other:

One user can create many posts
One post can have many comments
One post can receive many likes

Instead of one giant table, we split information into logically related tables. That's the central idea of the relational model.

2️⃣ Tables, Rows and Columns

Here's a simple users table:

user_id	username	email	age
1	ishu	ishu@gmail.com	22
2	rahul	rahul@gmail.com	23
3	priya	priya@gmail.com	21

Term	Meaning	Example
📋 Table	Collection of related data	`users`, `posts`, `comments`
➡️ Row (record / tuple)	One complete entry	One user's information
⬇️ Column (attribute)	One property of the data	`username`, `email`, `age`
🔲 Cell	A single value at a row/column intersection	`ishu@gmail.com`
📐 Schema	The blueprint: tables, columns, types, constraints, relationships	The whole structure

2.1 🔤 Data types

Every column has a data type that decides what it can store.

Data type	Meaning	Example
`INTEGER`	Whole numbers	`25`
`VARCHAR(n)`	Variable-length text (max n)	`'Ishu'`
`TEXT`	Text of any length	A post caption
`BOOLEAN`	True or false	`TRUE`
`DATE`	Calendar date	`2026-11-11`
`TIMESTAMP`	Date and time	`2026-11-11 12:30:00`
`DECIMAL`	Exact decimal numbers	`99.50`
`UUID`	Unique identifier	`550e8400-...`

CREATE TABLE users (
    user_id   INTEGER,
    username  VARCHAR(50),
    email     VARCHAR(100),
    age       INTEGER,
    is_active BOOLEAN
);

users → the table name
username → text, max 50 characters
email → text, max 100 characters
is_active → a Boolean

2.2 🕳️ What is `NULL`?

NULL means a value is missing or unknown. It is NOT the same as:

❌ 0
❌ an empty string ''
❌ FALSE

INSERT INTO users (user_id, username, email, age)
VALUES (1, 'ishu', 'ishu@gmail.com', NULL);

To find users whose age is missing:

SELECT *
FROM users
WHERE age IS NULL;

🚨 You cannot write age = NULL. NULL is an unknown value, not an ordinary comparable value. Always use IS NULL / IS NOT NULL.

3️⃣ Keys in Relational Databases 🔑

Imagine two users with the same name:

user_id	username
1	Rahul
2	Rahul

If you want to update the first Rahul's email, how does the database know which one? We need something that uniquely identifies each row. That's what keys do.

3.1 🆔 Primary Key (PK)

A Primary Key is a column (or combination of columns) that uniquely identifies every row in a table.

CREATE TABLE users (
    user_id  INTEGER PRIMARY KEY,
    username VARCHAR(50),
    email    VARCHAR(100)
);

Rules:

Every primary key value must be unique
A primary key can never be NULL
A table has only one primary key constraint (though it may span multiple columns)

user_id (PK)	username
1	Rahul
2	Rahul
3	Priya

Usernames can repeat, but IDs can't.

🎓 Analogy: your college USN. Many students share a name, but every student has a distinct USN.

3.2 🔗 Foreign Key (FK)

A Foreign Key is a column (or group of columns) that references a key in another table (usually its primary key or a UNIQUE key).

Users

user_id (PK)	username
1	Ishu
2	Rahul
3	Priya

Posts

post_id (PK)	caption	user_id (FK)
101	My first post	1
102	Travel memories	1
103	Coding journey	2

The user_id in posts tells us who created the post:

Post 101 → user 1
Post 102 → user 1
Post 103 → user 2

CREATE TABLE posts (
    post_id INTEGER PRIMARY KEY,
    caption TEXT,
    user_id INTEGER,
    FOREIGN KEY (user_id) REFERENCES users(user_id)
);

🛡️ Why is a Foreign Key important?

What if someone tries this?

INSERT INTO posts VALUES (104, 'Hello', 999);

User 999 doesn't exist, so the foreign key constraint blocks the insert. This is called Referential Integrity: relationships between tables always stay valid.

3.3 🗝️ Other important keys

Key	Meaning
Super Key	Any set of columns that uniquely identifies a row
Candidate Key	A minimal super key (no unnecessary columns)
Alternate Key	A candidate key not chosen as the primary key
Composite Key	A key made of multiple columns
Unique Key	A constraint that prevents duplicate values

CREATE TABLE users (
    user_id  INTEGER PRIMARY KEY,
    username VARCHAR(50) UNIQUE,
    email    VARCHAR(100) UNIQUE
);

Here user_id is the primary key, while username and email are unique (and are also candidate keys).

3.4 🧱 Composite Primary Key

Think of a likes table:

user_id	post_id
1	101
1	102
2	101

One user can like many posts, and one post can be liked by many users. But the same user shouldn't like the same post twice.

CREATE TABLE likes (
    user_id INTEGER,
    post_id INTEGER,
    PRIMARY KEY (user_id, post_id)
);

The combination (user_id, post_id) is unique. Neither column needs to be unique on its own.

4️⃣ Relationships in Relational Databases 🤝

There are three main types:

One-to-One (1:1)
One-to-Many (1:N)
Many-to-Many (N:N)

4.1 1️⃣ One-to-One (1:1)

One record in Table A relates to at most one record in Table B, and vice versa. Example: one user has one profile.

users

user_id	username
1	Ishu
2	Rahul

profiles

profile_id	user_id (FK)	bio
101	1	Developer
102	2	Student

CREATE TABLE profiles (
    profile_id INTEGER PRIMARY KEY,
    user_id    INTEGER UNIQUE REFERENCES users(user_id),
    bio        TEXT
);

Why UNIQUE on user_id? Without it, multiple profiles could point to the same user. UNIQUE guarantees at most one profile per user.

Common uses: user profiles, employee details, passport info, account settings.

4.2 🌳 One-to-Many (1:N)

One record in Table A can relate to many records in Table B, but each B record belongs to one A record. Example: one user creates many posts.

users

user_id	username
1	Ishu
2	Rahul

posts

post_id	user_id
101	1
102	1
103	1
104	2

📌 Golden rule: the foreign key goes on the MANY side. So posts.user_id references users.user_id.

More examples:

One teacher → many students
One customer → many orders
One post → many comments
One department → many employees

4.3 🕸️ Many-to-Many (N:N)

Many records in A relate to many records in B. Example: students and courses.

One student can enroll in many courses
One course can have many students

A single foreign key in either table can't model this. So we add a third table called a junction table (also: bridge table or associative table).

students

student_id	name
1	Ishu
2	Rahul

courses

course_id	course_name
10	Python
20	SQL

enrollments (junction)

student_id	course_id
1	10
1	20
2	10

CREATE TABLE enrollments (
    student_id INTEGER REFERENCES students(student_id),
    course_id  INTEGER REFERENCES courses(course_id),
    PRIMARY KEY (student_id, course_id)
);

This turns one many-to-many relationship into two one-to-many relationships. 🎯

Instagram example: Users ↔ Posts is many-to-many through Likes. One user can like many posts, and one post can be liked by many users.

4.4 ⚖️ Quick comparison

Relationship	Example	Implementation
1:1	User → Profile	FK + `UNIQUE`
1:N	User → Posts	FK on the many side
N:N	Students ↔ Courses	Junction table

5️⃣ SQL: Structured Query Language 💬

How do we talk to a relational database? With SQL (Structured Query Language). With SQL we can:

Create tables
Insert, retrieve, update and delete records
Create relationships and apply constraints
Manage transactions

SQL commands are grouped into categories.

5.1 🏗️ DDL: Data Definition Language

DDL defines or changes the structure of database objects.

Command	Purpose
`CREATE`	Creates a table or other object
`ALTER`	Modifies an existing table
`DROP`	Deletes an object
`TRUNCATE`	Removes all rows, keeps the table

-- CREATE
CREATE TABLE users (
    user_id  INTEGER PRIMARY KEY,
    username VARCHAR(50),
    email    VARCHAR(100)
);

-- ALTER: add an age column
ALTER TABLE users
ADD COLUMN age INTEGER;

-- TRUNCATE: remove all rows, keep structure
TRUNCATE TABLE users;

-- DROP: remove the table itself (structure + data)
DROP TABLE users;

5.2 ✏️ DML: Data Manipulation Language

DML manipulates the data inside tables: INSERT, UPDATE, DELETE.

-- INSERT: add data
INSERT INTO users (user_id, username, email)
VALUES (1, 'ishu', 'ishu@gmail.com');

-- UPDATE: modify existing data
UPDATE users
SET username = 'deepika'
WHERE user_id = 1;

-- DELETE: remove data
DELETE FROM users
WHERE user_id = 1;

🚨 Never forget WHERE! Without it, UPDATE changes every row and DELETE removes every row.

5.3 🔎 DQL: Data Query Language

DQL retrieves data. The main command is SELECT.

-- all columns
SELECT * FROM users;

-- specific columns
SELECT username, email
FROM users;

-- filter rows
SELECT *
FROM users
WHERE age > 20;

Useful comparison operators:

Operator	Meaning
`=`	Equal
`!=` or `<>`	Not equal
`>`	Greater than
`<`	Less than
`>=`	Greater than or equal
`<=`	Less than or equal

More filtering tools:

-- AND / OR
SELECT * FROM users WHERE age >= 18 AND is_active = TRUE;

-- IN
SELECT * FROM users WHERE username IN ('ishu', 'rahul');

-- BETWEEN
SELECT * FROM users WHERE age BETWEEN 18 AND 25;

-- LIKE (pattern matching)
SELECT * FROM users WHERE email LIKE '%@gmail.com';

-- ORDER BY + LIMIT
SELECT * FROM users
ORDER BY age DESC
LIMIT 5;

🧠 % in LIKE means "any number of characters".

5.4 🔐 Other SQL categories

Category	Purpose	Examples
DDL	Define structure	`CREATE`, `ALTER`, `DROP`
DML	Change data	`INSERT`, `UPDATE`, `DELETE`
DQL	Query data	`SELECT`
TCL	Control transactions	`BEGIN`, `COMMIT`, `ROLLBACK`
DCL	Control permissions	`GRANT`, `REVOKE`

6️⃣ SQL JOINs 🔀

Data lives in separate tables, so how do we combine it? With JOINs.

We'll use this sample data throughout:

users

user_id	username
1	Ishu
2	Rahul
3	Priya
4	Arjun

posts

post_id	caption	user_id
101	Hello World	1
102	My Travel	1
103	Coding	2
104	Photography	NULL

💡 Post 104 has no owner (user_id is NULL). In a well-designed schema posts.user_id would be NOT NULL, but this orphan row helps us see how each JOIN behaves.

6.1 🤝 INNER JOIN

Returns only rows that have a match in both tables.

SELECT
    users.username,
    posts.caption
FROM users
INNER JOIN posts
    ON users.user_id = posts.user_id;

username	caption
Ishu	Hello World
Ishu	My Travel
Rahul	Coding

Why no Priya or Arjun? They have no matching posts.
Why no post 104? Its user_id matches no user.

🧠 INNER JOIN = only matching records.

6.2 ⬅️ LEFT JOIN (LEFT OUTER JOIN)

Returns:

All rows from the left table
Matching rows from the right table
NULL for right-side columns when there's no match

SELECT
    users.username,
    posts.caption
FROM users
LEFT JOIN posts
    ON users.user_id = posts.user_id;

username	caption
Ishu	Hello World
Ishu	My Travel
Rahul	Coding
Priya	NULL
Arjun	NULL

Priya and Arjun appear because LEFT JOIN protects the left table (users).

🧠 LEFT JOIN = everything from the left + matching data from the right.

Real use case: "Show all users, including those who haven't posted anything."

6.3 ➡️ RIGHT JOIN (RIGHT OUTER JOIN)

The mirror image of LEFT JOIN. Returns all rows from the right table plus matching left rows (NULL where there's no match).

SELECT
    users.username,
    posts.caption
FROM users
RIGHT JOIN posts
    ON users.user_id = posts.user_id;

username	caption
Ishu	Hello World
Ishu	My Travel
Rahul	Coding
NULL	Photography

"Photography" is included because RIGHT JOIN protects every row of posts.

6.4 ↔️ FULL OUTER JOIN

Returns all matches + all unmatched rows from both sides.

SELECT
    users.username,
    posts.caption
FROM users
FULL OUTER JOIN posts
    ON users.user_id = posts.user_id;

username	caption
Ishu	Hello World
Ishu	My Travel
Rahul	Coding
Priya	NULL
Arjun	NULL
NULL	Photography

6.5 📊 JOIN comparison

JOIN	Returns
`INNER JOIN`	Only matching rows
`LEFT JOIN`	All left rows + matching right rows
`RIGHT JOIN`	All right rows + matching left rows
`FULL OUTER JOIN`	All rows from both tables

Easy memory trick:

INNER = match only
LEFT = protect the left side
RIGHT = protect the right side
FULL = protect both sides

💡 PostgreSQL supports all four. Some other databases have limited support or different syntax (for example, MySQL has no FULL OUTER JOIN).

6.6 🧮 Multiple-table JOINs

Real apps often need three or more tables. Let's show the username, post caption and comment:

SELECT
    users.username,
    posts.caption,
    comments.content
FROM users
INNER JOIN posts
    ON users.user_id = posts.user_id
INNER JOIN comments
    ON posts.post_id = comments.post_id;

Users connect to Posts
Posts connect to Comments
We retrieve information from all three tables ✅

7️⃣ SQL Aggregations 📈

Your mentor might ask:

How many users are registered?
How many posts exist?
What's the average number of likes?
Which user has the most posts?

For this we need aggregate functions: calculations that turn many rows into a single result.

Function	Purpose
`COUNT()`	Counts rows / non-NULL values
`SUM()`	Adds numeric values
`AVG()`	Calculates the average
`MIN()`	Finds the smallest value
`MAX()`	Finds the largest value

7.1 🔢 COUNT()

SELECT COUNT(*) FROM users;

Difference:

COUNT(*) → counts all rows
COUNT(age) → counts only rows where age is NOT NULL

7.2 ➕ SUM, AVG, MIN, MAX

Imagine a payments table:

payment_id	amount
1	500
2	1000
3	1500

SELECT SUM(amount) FROM payments;  -- 3000
SELECT AVG(amount) FROM payments;  -- 1000
SELECT MIN(amount) FROM payments;  -- 500
SELECT MAX(amount) FROM payments;  -- 1500

7.3 🗂️ GROUP BY

How many posts has each user created?

post_id	user_id
101	1
102	1
103	1
104	2
105	2
106	3

SELECT
    user_id,
    COUNT(*) AS total_posts
FROM posts
GROUP BY user_id;

user_id	total_posts
1	3
2	2
3	1

GROUP BY user_id puts rows with the same user_id into one group, and COUNT(*) counts rows in each group. AS total_posts gives the result a readable name (an alias).

7.4 🚧 HAVING

We only want users with more than two posts:

SELECT
    user_id,
    COUNT(*) AS total_posts
FROM posts
GROUP BY user_id
HAVING COUNT(*) > 2;

user_id	total_posts
1	3

❓ WHERE vs HAVING (a classic interview question!)

WHERE filters individual rows, before grouping
HAVING filters groups, after GROUP BY

SELECT
    user_id,
    COUNT(*) AS total_posts
FROM posts
WHERE post_id > 100
GROUP BY user_id
HAVING COUNT(*) > 2;

Logical processing order:

FROM: get the source table
WHERE: filter rows
GROUP BY: form groups
HAVING: filter groups
SELECT: produce columns
ORDER BY: sort
LIMIT: restrict output

📝 This is the logical order, not necessarily the database's physical execution order.

8️⃣ Database Normalization 🧹 (VERY IMPORTANT!)

First, the problem normalization solves. A beginner might design one giant table:

user_id	username	post_id	caption	comment	commenter
1	Ishu	101	Hello	Nice!	Rahul
1	Ishu	101	Hello	Amazing!	Priya
1	Ishu	102	Travel	Beautiful!	Rahul
2	Rahul	103	Coding	Great!	Ishu

See the problems? 👀

🔁 Data redundancy: Ishu's username and her post caption repeat in multiple rows. This wastes storage and complicates maintenance.
✏️ Update anomaly: if Ishu changes her username, you must update every row. Miss one and your data becomes inconsistent.
➕ Insertion anomaly: a new user who hasn't posted yet can't be stored cleanly, because the table expects post data.
🗑️ Deletion anomaly: if Rahul's only post is deleted, you may accidentally lose his user details too.

These are called database anomalies, and normalization prevents them.

8.1 What is Normalization?

Normalization is organizing data into well-structured tables to:

✅ Reduce unnecessary duplication
✅ Prevent insertion, update and deletion anomalies
✅ Keep data consistent
✅ Improve data integrity
✅ Represent relationships correctly

We split large, poorly structured tables into smaller related ones. The foundational normal forms are 1NF, 2NF and 3NF.

8.2 1️⃣ First Normal Form (1NF)

Rule: every cell holds a single atomic value. No repeating groups or multi-valued columns.

❌ Not in 1NF

student_id	name	phone_numbers
1	Ishu	9876, 8765
2	Rahul	7654, 6543

Multiple phone numbers are crammed into one cell.

✅ Fixed (1NF)

students

student_id	name
1	Ishu
2	Rahul

student_phones

student_id (FK)	phone_number
1	9876
1	8765
2	7654
2	6543

Now every cell has exactly one value.

🧠 1NF = one value per cell, no repeating groups.

8.3 2️⃣ Second Normal Form (2NF)

Rule: the table must be in 1NF, and every non-key attribute must depend on the whole candidate key, not just part of it.

Two terms first:

Non-key attribute: a column that isn't part of the candidate key
Partial dependency: a non-key attribute depends on only part of a composite key

❌ Not in 2NF — enrollment with composite key (student_id, course_id):

student_id	course_id	student_name	course_name
1	101	Ishu	Python
1	102	Ishu	SQL
2	101	Rahul	Python

student_name depends only on student_id
course_name depends only on course_id

Neither depends on the complete key. That's a partial dependency, and it causes repeated names.

✅ Fixed (2NF)

students

student_id (PK)	student_name
1	Ishu
2	Rahul

courses

course_id (PK)	course_name
101	Python
102	SQL

enrollments

student_id (PK, FK)	course_id (PK, FK)
1	101
1	102
2	101

🧠 2NF = 1NF + no partial dependency.

💡 Nuance: if a table's candidate key is a single column, partial dependency is impossible.

8.4 3️⃣ Third Normal Form (3NF)

Rule: the table must be in 2NF, and have no transitive dependency of non-key attributes on the key.

A transitive dependency means a non-key attribute depends on another non-key attribute instead of directly on the key.

❌ Not in 3NF — employees (PK: employee_id):

employee_id	employee_name	department_id	department_name
1	Ishu	10	Engineering
2	Rahul	10	Engineering
3	Priya	20	HR

Dependencies:

employee_id   → employee_name
employee_id   → department_id
department_id → department_name

So employee_id → department_id → department_name. The department name depends on the department, not directly on the employee. If "Engineering" becomes "Software Engineering", you must update many rows and might miss one.

✅ Fixed (3NF)

employees

employee_id (PK)	employee_name	department_id (FK)
1	Ishu	10
2	Rahul	10
3	Priya	20

departments

department_id (PK)	department_name
10	Engineering
20	HR

Now the department name is stored once.

🧠 3NF = 2NF + no transitive dependency.

📖 Formal definition: for every non-trivial functional dependency X → Y, either X is a superkey, or Y is a prime attribute (part of some candidate key).

8.5 🧬 Functional Dependency (FD)

A functional dependency means one attribute determines another. It's written:

A → B        (read: "A determines B")

Examples:

employee_id   → employee_name
department_id → department_name

If we know employee_id, we can determine the employee_name (assuming employee_id is unique). FDs are the foundation for spotting partial and transitive dependencies.

8.6 📊 1NF vs 2NF vs 3NF

Normal form	Main rule	Problem eliminated
1NF	Atomic values, no repeating groups	Multi-valued cells
2NF	No partial dependency	Partial dependency
3NF	No transitive dependency of non-key attributes	Transitive dependency

Think of it as three stages:

1NF: organize the cells. Store single values, not lists.
2NF: organize the dependencies. Every non-key column depends on the whole key.
3NF: remove indirect dependencies. Each entity gets its own table.

8.7 🛡️ Why normalization is a data-integrity principle

Normalization isn't just about saving space. It's about storing each fact in exactly one appropriate place:

A username belongs to a user
A caption belongs to a post
A department name belongs to a department
A like belongs to a user-post combination

When each fact lives in one place, inconsistent data becomes much harder to create.

⚖️ Note: normalization doesn't mean "split everything always". Real systems sometimes use controlled denormalization for performance, but only when the trade-offs are understood.

9️⃣ ACID Properties and Transactions ⚛️

Imagine a banking app. You transfer ₹1,000 to a friend. Two operations must happen:

Deduct ₹1,000 from your account
Add ₹1,000 to your friend's account

What if the first succeeds, but the app crashes before the second? 😱 Your money is gone and your friend never receives it.

We need guarantees. That's what ACID provides:

A: Atomicity
C: Consistency
I: Isolation
D: Durability

9.1 📦 What is a transaction?

A transaction is a group of database operations treated as one logical unit of work.

BEGIN;

UPDATE accounts
SET balance = balance - 1000
WHERE account_id = 1;

UPDATE accounts
SET balance = balance + 1000
WHERE account_id = 2;

COMMIT;

BEGIN → starts the transaction
first UPDATE → deducts money
second UPDATE → adds money
COMMIT → makes the changes permanent

If something goes wrong:

ROLLBACK;

ROLLBACK cancels all uncommitted changes in the transaction.

9.2 ⚛️ A: Atomicity ("all or nothing")

A transaction either fully completes or has no effect at all.

✅ SUCCESS (both operations succeed)
   Account A: ₹5,000 → ₹4,000
   Account B: ₹2,000 → ₹3,000

❌ FAILURE (everything is rolled back)
   Account A: ₹5,000 → ₹5,000
   Account B: ₹2,000 → ₹2,000

We must never end up with money deducted from A but not added to B.

9.3 ✅ C: Consistency ("valid state to valid state")

A transaction must keep the database's defined integrity rules intact. The database is valid before and after.

CREATE TABLE accounts (
    account_id INTEGER PRIMARY KEY,
    balance    DECIMAL(12,2) CHECK (balance >= 0)
);

If account A has ₹500 and someone runs:

UPDATE accounts
SET balance = balance - 1000
WHERE account_id = 1;

the balance would become -₹500, so the database rejects it because it violates the CHECK constraint.

Consistency rules include primary key uniqueness, foreign key validity, NOT NULL, UNIQUE and CHECK constraints.

⚠️ ACID consistency doesn't mean the database magically understands every business rule. Developers must define the constraints and transaction logic.

9.4 🚦 I: Isolation ("transactions don't interfere incorrectly")

Concurrent transactions should behave correctly without corrupting each other's intermediate work.

Your balance is ₹5,000. Two requests arrive at nearly the same time:

Transaction A withdraws ₹4,000
Transaction B withdraws ₹4,000

If both read ₹5,000 and both proceed, you could withdraw more than you have. That's a concurrency problem, and isolation mechanisms control it.

PostgreSQL supports these isolation levels:

Isolation level	General protection
`READ UNCOMMITTED`	Lowest level; PostgreSQL treats it like Read Committed
`READ COMMITTED`	Prevents dirty reads (PostgreSQL default)
`REPEATABLE READ`	Consistent snapshot for the whole transaction; prevents non-repeatable reads
`SERIALIZABLE`	Strongest; concurrent transactions behave as if run one after another

Key terms:

🧟 Dirty read: reading another transaction's uncommitted changes
🔄 Non-repeatable read: reading the same row twice and getting different committed values
👻 Phantom read: repeating a query and getting extra or missing rows because another transaction inserted/deleted matching rows

PostgreSQL uses MVCC (Multi-Version Concurrency Control), which keeps multiple row versions so readers and writers don't block each other unnecessarily.

Isolation matters most in banking, ticket booking and inventory systems.

9.5 💾 D: Durability ("committed means persisted")

Once a transaction is committed, its changes survive crashes and restarts.

You transfer ₹1,000, the database confirms the commit, and the server crashes right after. After recovery, the transfer must still be there.

PostgreSQL supports durability through Write-Ahead Logging (WAL): changes are recorded in a log before the corresponding data pages are necessarily written to their final location, so the database can recover after a crash.

9.6 🧾 ACID together

Property	Simple meaning	Banking example
Atomicity	All or nothing	Both debit and credit happen
Consistency	Rules remain valid	Balance can't go negative
Isolation	Concurrent operations are controlled	Two withdrawals don't interfere incorrectly
Durability	Committed changes survive failures	A successful transfer stays recorded

Easy memory aid:

A: All or nothing
C: Correct database state
I: Independent execution
D: Data stays committed

🔟 Database Constraints 🚧

A constraint is a rule on a column or table that blocks invalid data.

Imagine a table that allows this:

user_id	email
1	ishu@gmail.com
1	rahul@gmail.com
2	NULL

Duplicate IDs and missing emails! Constraints prevent this.

Constraint	Purpose	Example
`PRIMARY KEY`	Unique row identifier, no NULL	`user_id INTEGER PRIMARY KEY`
`FOREIGN KEY`	Referenced record must exist	`user_id INTEGER REFERENCES users(user_id)`
`UNIQUE`	No duplicate values	`email VARCHAR(100) UNIQUE`
`NOT NULL`	A value is required	`username VARCHAR(50) NOT NULL`
`CHECK`	Enforces a condition	`age INTEGER CHECK (age >= 18)`
`DEFAULT`	Value used when none is given	`is_active BOOLEAN DEFAULT TRUE`

All together:

CREATE TABLE users (
    user_id   INTEGER PRIMARY KEY,
    username  VARCHAR(50)  NOT NULL UNIQUE,
    email     VARCHAR(100) NOT NULL UNIQUE,
    age       INTEGER CHECK (age >= 18),
    is_active BOOLEAN DEFAULT TRUE
);

This is how we enforce data quality at the database level. 🛡️

🧹 Bonus: `ON DELETE` behavior

What happens to a user's posts when the user is deleted? You decide:

CREATE TABLE posts (
    post_id INTEGER PRIMARY KEY,
    user_id INTEGER NOT NULL
        REFERENCES users(user_id) ON DELETE CASCADE,
    caption TEXT
);

ON DELETE CASCADE → deleting a user also deletes their posts
ON DELETE RESTRICT / NO ACTION → block the delete while posts exist
ON DELETE SET NULL → keep posts but clear user_id (column must be nullable)

1️⃣1️⃣ Putting it together: design an Instagram database 📸

Your mentor says: "Design the database for an Instagram-like app." How do you approach it?

Step 1: identify the entities (real-world things we store data about):

👤 User
🖼️ Post
💬 Comment
❤️ Like
👥 Follow

Step 2: create the tables.

1. users

Column	Type	Constraint
user_id	INTEGER	PK
username	VARCHAR	UNIQUE, NOT NULL
email	VARCHAR	UNIQUE, NOT NULL
password_hash	TEXT	NOT NULL

2. posts

Column	Type	Constraint
post_id	INTEGER	PK
user_id	INTEGER	FK, NOT NULL
caption	TEXT
created_at	TIMESTAMP

3. comments

Column	Type	Constraint
comment_id	INTEGER	PK
post_id	INTEGER	FK, NOT NULL
user_id	INTEGER	FK, NOT NULL
content	TEXT	NOT NULL

4. likes

Column	Type	Constraint
user_id	INTEGER	PK, FK
post_id	INTEGER	PK, FK
created_at	TIMESTAMP

5. follows

Column	Type	Constraint
follower_id	INTEGER	PK, FK
following_id	INTEGER	PK, FK
created_at	TIMESTAMP

Notice that in follows, both follower_id and following_id reference the users table. That's a self-referencing relationship.

follower_id	following_id
1	2
1	3
2	1

User 1 follows users 2 and 3
User 2 follows user 1

The composite primary key prevents following the same person twice, and we add a CHECK so nobody follows themselves:

CHECK (follower_id <> following_id)

1️⃣1️⃣.1 🗺️ How the tables relate

USERS (PK: user_id)
   │ 1 : N
   ▼
POSTS (FK: user_id)
   │ 1 : N
   ▼
COMMENTS (FK: post_id, user_id)

USERS ◄── N : N via LIKES (user_id + post_id) ──► POSTS

USERS ◄── self N : N via FOLLOWS (follower_id + following_id) ──► USERS

Users → Posts: one-to-many
Posts → Comments: one-to-many
Users → Comments: one-to-many
Posts → Likes: one-to-many
Users ↔ Posts through Likes: many-to-many
Users ↔ Users through Follows: self-referencing many-to-many

1️⃣1️⃣.2 🧪 Real SQL queries

Display a post with its creator:

SELECT
    posts.post_id,
    posts.caption,
    users.username
FROM posts
INNER JOIN users
    ON posts.user_id = users.user_id;

Count posts per user (including users with zero posts):

SELECT
    users.username,
    COUNT(posts.post_id) AS total_posts
FROM users
LEFT JOIN posts
    ON users.user_id = posts.user_id
GROUP BY users.user_id, users.username;

🤔 Why LEFT JOIN? To include users with no posts.
🤔 Why COUNT(posts.post_id) and not `COUNT()?* For a user with no posts, LEFT JOIN still creates a row with NULL post columns. COUNT(*) would count that row as 1, but COUNT(posts.post_id) correctly returns 0`.

Top 5 most-liked posts:

SELECT
    posts.post_id,
    posts.caption,
    COUNT(likes.user_id) AS total_likes
FROM posts
LEFT JOIN likes
    ON posts.post_id = likes.post_id
GROUP BY posts.post_id, posts.caption
ORDER BY total_likes DESC
LIMIT 5;

Start with posts
Join likes
Group likes by post
Count likes per post
Sort by like count
Return the top five 🏆

Who does user 1 follow? (bonus)

SELECT users.username
FROM follows
INNER JOIN users
    ON follows.following_id = users.user_id
WHERE follows.follower_id = 1;

This is the kind of query you'll meet in real backend apps.

1️⃣2️⃣ How this connects to SQLAlchemy and FastAPI 🔌

You might wonder: "If SQL talks to PostgreSQL, why learn SQLAlchemy?"

SQL → the language for querying relational databases
PostgreSQL → the database system that stores and manages the data
SQLAlchemy → a Python library that lets us work with databases using Python objects and SQL expressions

The SQL table:

CREATE TABLE users (
    user_id  INTEGER PRIMARY KEY,
    username VARCHAR(50)
);

becomes this SQLAlchemy 2.0 ORM model (simplified):

from sqlalchemy import Integer, String
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column


class Base(DeclarativeBase):
    pass


class User(Base):
    __tablename__ = "users"

    user_id: Mapped[int] = mapped_column(Integer, primary_key=True)
    username: Mapped[str] = mapped_column(String(50))

SQL concept	SQLAlchemy equivalent
Table	ORM model (a class)
Column	`mapped_column`
Primary Key	`primary_key=True`
Foreign Key	`ForeignKey()`
`UNIQUE`	`unique=True`
`NOT NULL`	`nullable=False`
Relationship	`relationship()`
`INSERT`	`session.add()`
`UPDATE`	Modify object attributes
`DELETE`	`session.delete()`
`COMMIT`	`session.commit()`
`ROLLBACK`	`session.rollback()`

Creating a user (with an async session):

new_user = User(username="ishu")
session.add(new_user)
await session.commit()

session.add() registers the object with the session
session.commit() commits the transaction (that's ACID in action! ⚛️)
SQLAlchemy generates and runs the SQL for you

🧠 SQLAlchemy doesn't replace SQL. It's a Pythonic way to work with relational databases. Understand the underlying SQL, because SQLAlchemy still talks to the database using SQL.

🔄 How a request flows through your backend

React frontend       → user clicks "Create Post"
      ▼
FastAPI endpoint     → receives and validates the request
      ▼
Service layer        → applies business logic
      ▼
Repository layer     → runs database operations via SQLAlchemy
      ▼
PostgreSQL           → stores the row in the posts table

That's why relational database fundamentals come before SQLAlchemy and Alembic. 🎯

1️⃣3️⃣ Common interview questions 🎤

Q1. Primary Key vs Foreign Key?
A primary key uniquely identifies a row in its own table. A foreign key references a key in another table and maintains referential integrity.

Q2. Can a table have multiple primary keys?
Only one PRIMARY KEY constraint, but it can span multiple columns (a composite primary key).

Q3. Can a foreign key contain NULL?
Yes, unless the column is NOT NULL. A NULL foreign key means no related record is specified.

Q4. DELETE vs TRUNCATE vs DROP?

`DELETE`	`TRUNCATE`	`DROP`
Removes selected rows	Removes all rows	Removes the whole table
Supports `WHERE`	No `WHERE`	Removes structure too
Table remains	Table remains	Table is gone

Q5. WHERE vs HAVING?
WHERE filters rows before grouping. HAVING filters groups after grouping.

Q6. Why do we need normalization?
To reduce redundancy, prevent insertion/update/deletion anomalies and improve data integrity.

Q7. What is a junction table?
A table implementing a many-to-many relationship using foreign keys to the two related tables.

Q8. 2NF vs 3NF?
2NF removes partial dependencies. 3NF removes transitive dependencies of non-key attributes.

Q9. What happens when a transaction fails?
Uncommitted changes can be rolled back, preventing partial application.

Q10. Atomicity vs Durability?
Atomicity: all operations succeed together or none do. Durability: once committed, changes survive failures.

Q11. What is referential integrity?
A guarantee that foreign key values always point to existing rows (or are NULL).

Q12. INNER JOIN vs LEFT JOIN?
INNER JOIN returns only matching rows. LEFT JOIN returns all left rows plus matches, with NULLs where there's no match.

1️⃣4️⃣ Practice quiz 🧪

Answer without scrolling up! (Answers are at the bottom.)

1. Which key uniquely identifies each row in a table?

A) Foreign Key
B) Primary Key
C) Alternate Table
D) Index

2. One user creates multiple posts. What relationship is this?

A) One-to-One
B) One-to-Many
C) Many-to-Many
D) None

3. Which JOIN returns all records from the left table?

A) INNER JOIN
B) RIGHT JOIN
C) LEFT JOIN
D) CROSS JOIN

4. Which normal form eliminates partial dependency?

A) 1NF
B) 2NF
C) 3NF
D) None

5. Which normal form eliminates transitive dependencies of non-key attributes?

A) 1NF
B) 2NF
C) 3NF
D) BCNF only

6. Which SQL clause filters aggregated groups?

A) WHERE
B) ORDER BY
C) HAVING
D) LIMIT

7. Which ACID property means all operations happen or none happen?

A) Consistency
B) Atomicity
C) Isolation
D) Durability

8. What does COMMIT do?

A) Deletes a table
B) Starts a transaction
C) Makes transaction changes permanent
D) Reads records

9. What implements a many-to-many relationship?

A) Only a Primary Key
B) A junction table
C) A single text column
D) DROP TABLE

10. Why do we use a Foreign Key?

A) To sort records
B) To maintain referential integrity
C) To encrypt passwords
D) To create indexes automatically in every database

👀 Click to reveal answers

🧾 Final revision sheet

Concept	What to remember
Relational model	Data organized into related tables
Table	Collection of related records
Row	One record
Column	One attribute
Primary Key	Unique row identifier
Foreign Key	References another table's key
1:1	One record relates to at most one other (FK + UNIQUE)
1:N	One record relates to many (FK on the many side)
N:N	Many relate to many through a junction table
INNER JOIN	Only matching rows
LEFT JOIN	All left rows + matching right rows
RIGHT JOIN	All right rows + matching left rows
FULL JOIN	All rows from both sides
GROUP BY	Groups similar records
HAVING	Filters grouped results
Normalization	Organizes data and reduces anomalies
1NF	Atomic values
2NF	No partial dependency
3NF	No transitive dependency of non-key attributes
Atomicity	All or nothing
Consistency	Preserves database rules
Isolation	Controls concurrent transactions
Durability	Committed changes persist

🚀 What should you learn next?

You now have the conceptual foundation of relational database development. Learn in this order:

SQL CRUD operations and filtering
Joins and subqueries
GROUP BY, HAVING and aggregate queries
Constraints and indexes
Transactions and isolation levels
SQLAlchemy Core and ORM
Alembic migrations

💡 My biggest advice: don't just memorize definitions of INNER JOIN, 1NF, 2NF, 3NF and ACID. Create real tables, insert rows, join them and watch what happens. That's when it all clicks. ❤️

🙌 Enjoyed this post?

❤️ Drop a reaction
🦄 Save it for revision
💬 Comment with questions, and I'll answer them!

Follow me so you don't miss it! 🔔
Happy coding! 👩‍💻✨

Authentication Basics (JWT) & FastAPI Security: A Beginner-Friendly Guide 🔐

Deepika Pusala — Wed, 11 Nov 2026 07:23:56 +0000

🔐 Authentication Basics (JWT) & FastAPI Security

Authentication is not just a login page. 🙅‍♀️

It is about understanding how a backend identifies a user, verifies their identity, issues a token, and protects private API endpoints.

In this guide we'll learn everything from absolute zero:

✅ How JWT works internally
✅ What OAuth2PasswordBearer actually does
✅ Why password hashing is necessary
✅ How Depends() makes authentication reusable
✅ How all these pieces connect in a real React + FastAPI + PostgreSQL project (my Social Feed app)

Simple examples, real-world analogies, Python code and execution flows. No assumed knowledge. Let's go! 🚀

📚 Table of Contents

Authentication vs Authorization
Why do we need authentication?
Token-based authentication
What is JWT?
JWT issuing and verification
PyJWT and python-jose
What is OAuth2?
What is OAuth2PasswordBearer?
What is Depends()?
Password hashing (and why plaintext is a crime)
Passlib, bcrypt and Argon2id
Building the complete FastAPI auth system
Full execution flow
Expiry, access tokens, refresh tokens and logout
JWT vs session-based authentication
HTTP status codes
Common security mistakes
Architecture of a real FastAPI project
Interview questions
Quick quiz and final revision

1️⃣ Authentication vs Authorization

These two words sound similar, but they mean different things.

Imagine Instagram 📸. You open it and enter:

Email: deepika@gmail.com
Password: mypassword123

Instagram checks whether these credentials belong to a registered user. That's Authentication.

Now you're logged in and you try to edit someone else's profile. Instagram checks whether you have permission to do that. That's Authorization.

Concept	Meaning	Example
🪪 Authentication	Who are you?	Verify your login
🚦 Authorization	What can you do?	Can you delete this post?

🧠 Remember: Authentication = identity verification. Authorization = permission checking. Authentication usually happens first, authorization follows.

2️⃣ Why do we need authentication?

Suppose you build a social media app using:

⚛️ React frontend
⚡ FastAPI backend
🐘 PostgreSQL database

Your API has these endpoints:

GET    /feed
POST   /posts
DELETE /posts/15
GET    /users/10

What if anyone can call DELETE /posts/15? 😱 Anyone could delete anyone's post!

The backend needs a way to answer:

Who is making this request?
Are they logged in?
Is their identity valid?
Are they allowed to do this operation?

That's exactly what authentication (and authorization) solves.

3️⃣ What is token-based authentication?

A token is a piece of data that represents a user's authenticated session or identity.

Think of a movie theatre 🎬:

You buy a ticket.
The ticket is given to you.
You show it at the entrance.
Staff verifies it.
You're allowed inside.

You don't buy a new ticket for every section of the theatre. Same idea here:

User enters email and password.
Backend verifies the credentials.
Backend generates a token.
Frontend stores the token.
Frontend sends the token with every later request.
Backend verifies the token before allowing access.

After a successful login the backend returns something like:

{
  "access_token": "eyJhbGciOiJIUzI1NiJ9.example.signature",
  "token_type": "bearer"
}

And the frontend sends it back on future requests:

GET /feed
Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.example.signature

3.1 🏷️ What is the `Authorization` header?

HTTP requests can carry headers, extra information about the request.

GET /profile
Authorization: Bearer abc123
Content-Type: application/json

Authorization → the header name
Bearer → tells the server "I'm giving you a bearer token"
abc123 → the token itself

A bearer token basically means: whoever holds this valid token can use it. 🎟️

⚠️ That's why tokens must be protected. If someone steals your token, they may be able to act as you.

3.2 🔄 The complete authentication flow

User logs in: React sends email and password to POST /login.
Backend verifies credentials: FastAPI fetches the user from PostgreSQL and checks the password hash.
Backend creates a JWT: it creates and signs a token containing claims.
Frontend receives the token: the response contains access_token and token_type.
Frontend makes another request: React sends Authorization: Bearer <token>.
FastAPI verifies the token: it checks the signature, expiry and claims.
Protected endpoint runs: if valid, the backend identifies the user and continues.

Now let's understand JWT itself. 👇

4️⃣ What is JWT (JSON Web Token)?

JWT stands for JSON Web Token. It's a compact format for transferring information between parties as a JSON-based, signed token.

A JWT looks like this:

xxxxx.yyyyy.zzzzz

Three parts separated by dots:

HEADER . PAYLOAD . SIGNATURE

Part	Contains
🧾 Header	Algorithm and token type
📦 Payload	User identity and token claims
✍️ Signature	Cryptographic proof of integrity

4.1 🧾 Header

{
  "alg": "HS256",
  "typ": "JWT"
}

alg → algorithm used to sign the token
HS256 → HMAC using SHA-256
typ → token type (JWT)

4.2 📦 Payload (claims)

Claims are statements about the user or the token.

{
  "sub": "101",
  "role": "user",
  "iat": 1794398200,
  "exp": 1794400000
}

Claim	Meaning
`sub`	Subject, usually the user identifier
`exp`	Expiration time
`iat`	Issued-at time
`iss`	Issuer of the token
`aud`	Intended audience
`jti`	Unique token identifier

These are standard registered claims. You can also add custom claims like role.

🚨 Very important: a JWT payload is NOT encrypted by default. Anyone holding the token can decode the header and payload!

So NEVER put these inside a JWT:

❌ Passwords
❌ Password hashes
❌ Secret API keys
❌ Sensitive personal information

A signed JWT gives you integrity, not confidentiality.

4.3 ✍️ Signature (the security-critical part)

The backend signs the token using a secret key. Conceptually:

Signature = HMAC-SHA256(
    encoded_header + "." + encoded_payload,
    SECRET_KEY
)

Imagine an attacker changes the payload from "sub": "user_101" to "sub": "admin_999". Without the secret key they cannot produce a matching signature, so verification fails. 🛡️

Encoding vs signing vs encryption:

📝 Encoding → makes data representable in a transport format
✍️ Signing → verifies integrity and authenticity
🔒 Encryption → hides data from unauthorized readers

JWTs using HS256 are signed, not encrypted.

5️⃣ JWT issuing and verification

🏭 Issuing

Creating and signing a JWT, usually right after a successful login.

token = create_access_token(user_id=101)

The function receives the identity, builds the payload, adds expiry, signs it and returns the JWT string.

🔍 Verification

Happens whenever a client sends the token to a protected API:

Extract the token
Verify the signature
Check the expiration
Read the user ID from the payload
Identify the corresponding user
Allow or reject the request

A token can be rejected because:

❌ Invalid signature
❌ Expired
❌ Malformed structure
❌ Required claims missing
❌ The user or session is no longer valid

💡 A valid signature does not automatically mean the account is still active. You may also need to check the user or session in your database.

6️⃣ PyJWT and python-jose

You never implement signing algorithms by hand. You use a library. 📦

Library	Purpose
`PyJWT`	Create, encode, decode and verify JWTs
`python-jose`	Create, encode, decode and verify JWTs

Both work with FastAPI. Older tutorials (and older FastAPI docs) use python-jose, while the more actively maintained PyJWT is what the current FastAPI documentation uses. I'll use PyJWT in this article:

pip install pyjwt

If you'd rather use python-jose:

pip install "python-jose[cryptography]"

(The function names encode / decode are almost identical, so everything below translates easily.)

6.1 🏭 Creating a JWT

import jwt
from datetime import datetime, timedelta, timezone

SECRET_KEY = "my-super-secret-key"   # ⚠️ demo only!
ALGORITHM = "HS256"

def create_access_token(user_id: int) -> str:
    expire = datetime.now(timezone.utc) + timedelta(minutes=30)
    payload = {
        "sub": str(user_id),
        "exp": expire,
    }
    return jwt.encode(payload, SECRET_KEY, algorithm=ALGORITHM)

Line-by-line:

SECRET_KEY: the secret used to sign the JWT. In real apps, never hardcode it. Load it from an environment variable. Make it strong, random and private. 🔑
ALGORITHM = "HS256": HMAC SHA-256. The same secret signs and verifies.
expire: the token dies 30 minutes from now. If a token is stolen but expires soon, the damage window is small. ⏳
"sub": str(user_id): by convention sub is a string, so we convert the ID.
jwt.encode(...): encodes the header and payload, creates the signature, joins the three parts and returns the JWT string. ✨

That's JWT issuing!

6.2 🔍 Verifying a JWT

import jwt
from jwt import InvalidTokenError

def verify_access_token(token: str):
    try:
        payload = jwt.decode(
            token,
            SECRET_KEY,
            algorithms=[ALGORITHM],   # 👈 explicit allow-list!
        )
    except InvalidTokenError:
        return None

    user_id = payload.get("sub")
    if user_id is None:
        return None
    return user_id

What jwt.decode() does:

Decodes the token
Checks the signature
Validates exp (it raises an error if the token has expired)
Rejects invalid tokens

⚠️ Notice algorithms=[ALGORITHM]. Always specify the algorithms you accept. Never blindly trust the algorithm declared by an incoming token.

7️⃣ What is OAuth2?

You might be thinking: "We learned JWT, why OAuth2? Aren't they the same?"

NOOO! 🙅 They're different things.

OAuth2 → an authorization framework. It describes how authorization is granted and how tokens are obtained.
JWT → a token format. It describes how a token can be represented.

OAuth2 doesn't require JWT. OAuth2 access tokens can also be opaque random strings.

7.1 🌍 Real-world example

You see a "Continue with Google" button and click it. Google authenticates you and your app receives authorization information through an OAuth-based flow. You never give your Google password to the third-party app. 🎉

One nuance:

OAuth2 primarily handles authorization.
OpenID Connect (OIDC), built on top of OAuth2, adds an identity layer for authentication. Google Sign-In generally uses OIDC.

7.2 ⚡ OAuth2 in FastAPI

FastAPI ships security utilities for common OAuth2-compatible patterns. The most important one is OAuth2PasswordBearer. 👇

8️⃣ What is `OAuth2PasswordBearer`?

OAuth2PasswordBearer is a FastAPI security dependency. Its job:

📍 Tell FastAPI where the token comes from
📤 Extract the bearer token from the Authorization header
📥 Hand that token to your dependency or endpoint
📖 Describe the OAuth2 security scheme in the OpenAPI docs (that green Authorize button in Swagger UI!)

from fastapi.security import OAuth2PasswordBearer

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="login")

What does `tokenUrl="login"` mean?

It tells FastAPI (and the docs UI) that the endpoint used to obtain a token is POST /login.

What does it actually do?

Given this request:

GET /profile
Authorization: Bearer my-secret-token

it extracts my-secret-token (it strips the Bearer prefix).

It does NOT:

❌ Generate the JWT
❌ Verify the JWT signature
❌ Check the user's password
❌ Fetch the user from PostgreSQL
❌ Log the user in

🧠 OAuth2PasswordBearer only extracts the token. You still have to verify it yourself.

8.1 What if the token is missing?

If a request has no Authorization header, OAuth2PasswordBearer automatically responds with HTTP 401 Unauthorized. 🚫

9️⃣ What is `Depends()`?

Before building the full system we need dependency injection, another core FastAPI concept.

Imagine ten protected endpoints:

GET    /profile
GET    /feed
POST   /posts
DELETE /posts/10
POST   /comments
GET    /notifications
...

Every one needs authentication. Would you copy-paste the same token-checking code ten times? 😵 No way. FastAPI gives us:

Depends()

It means: "FastAPI, before running this endpoint, run this dependency and give me its result."

from fastapi import Depends, FastAPI

app = FastAPI()

def verify_user():
    return "Deepika"

@app.get("/profile")
def get_profile(user=Depends(verify_user)):
    return {"user": user}

What happens:

A request hits /profile
FastAPI sees Depends(verify_user)
FastAPI runs verify_user()
It returns "Deepika"
FastAPI passes that value into the user parameter
The endpoint runs

Response:

{
  "user": "Deepika"
}

That's Dependency Injection (DI). 💉

🔗 Connect the dots

Depends() → the dependency injection mechanism

OAuth2PasswordBearer → a dependency that extracts a bearer token

Your custom dependency (get_current_user) → verifies the token and identifies the user

Combining them

from fastapi import Depends
from fastapi.security import OAuth2PasswordBearer

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="login")

@app.get("/profile")
def get_profile(token: str = Depends(oauth2_scheme)):
    return {"received_token": token}

Incoming request
Authorization: Bearer abc123
        │
        ▼
Depends(oauth2_scheme)
        │  extracts "abc123"
        ▼
Endpoint receives token = "abc123"

But we haven't verified that abc123 is a valid JWT yet. That's coming up! 😉

🔟 Password hashing: why we can't store passwords directly

This is a HUGE real-world security concept. 🚨

A user registers with:

Email:    deepika@gmail.com
Password: deepika123

A beginner might store this directly:

id	email	password
1	deepika@gmail.com	deepika123

If your database is ever compromised, the attacker reads everyone's passwords. 💀 Applications must NEVER store plaintext passwords. They store password hashes.

10.1 What is hashing?

Hashing is a one-way transformation of input into a fixed-format output.

Input:  deepika123
Hash:   $argon2id$v=19$m=65536,t=3,p=4$...

(The exact output depends on the algorithm and a random salt.)

You cannot practically reverse a properly designed password hash back to the original password.

Hashing	Encryption
One-way	Reversible with the key
No decryption operation	Can be decrypted
Used for password verification	Used for confidentiality
Example: Argon2id	Example: AES

Password hashing is deliberately designed to make guessing expensive. 💸

10.2 What is bcrypt?

bcrypt is a password-hashing algorithm built specifically for password storage. Its key property is a configurable computational cost: the higher the cost, the more resources each hash needs, making bulk guessing slower.

10.3 🧂 What is a salt?

A salt is a random value added during password hashing.

Two users register with the same password:

User A: deepika123
User B: deepika123

Without salts, a deterministic scheme gives both the same hash. With unique random salts, their stored hashes differ.

Why it matters:

✅ Identical passwords get different hashes
✅ Precomputed rainbow-table attacks become much less useful
✅ Bulk password guessing gets harder

Good libraries generate and manage salts automatically. Don't roll your own.

10.4 ✅ How does verification work?

You do not "decrypt" the stored hash. Instead:

The library reads the algorithm and salt from the stored hash
Hashes the submitted password with those parameters
Compares the results securely
Returns True or False

REGISTRATION                      LOGIN
plain password                    entered password
     │                                 │
     ▼                                 ▼
  hashing                       verify against stored hash
     │                                 │
     ▼                                 ▼
 store the hash                    True / False

1️⃣1️⃣ What is Passlib?

Passlib is a Python library for password hashing and verification. It supports bcrypt, Argon2, PBKDF2 and more, so you never implement hashing by hand.

pip install "passlib[argon2]"

💡 I'm using Argon2id here, a modern, memory-hard algorithm (it needs both CPU and memory to compute, making guessing even more expensive). If you prefer bcrypt, use pip install "passlib[bcrypt]" and schemes=["bcrypt"]. Just make sure your passlib and bcrypt versions are compatible, since newer bcrypt releases have caused warnings or errors with older passlib.

from passlib.context import CryptContext

pwd_context = CryptContext(
    schemes=["argon2"],
    deprecated="auto",
)

CryptContext → Passlib's hashing configuration object
schemes=["argon2"] → our chosen algorithm
deprecated="auto" → lets Passlib flag older schemes that should be upgraded when users log in

Hash a password

def hash_password(password: str) -> str:
    return pwd_context.hash(password)

hashed = hash_password("deepika123")
print(hashed)   # a long Argon2id hash, NOT the original password

Verify a password

def verify_password(plain_password: str, hashed_password: str) -> bool:
    return pwd_context.verify(plain_password, hashed_password)

print(verify_password("deepika123", hashed))      # True ✅
print(verify_password("wrongpassword", hashed))   # False ❌

🧠 Passlib is the library; Argon2id / bcrypt is the algorithm. Don't mix them up!

The important lesson isn't "always use bcrypt". It's use a modern, slow, salted password-hashing algorithm, never plaintext and never ordinary SHA-256.

1️⃣2️⃣ Building a complete FastAPI authentication system 🏗️

Time to combine everything. We'll build:

✅ User registration
✅ Password hashing
✅ User login
✅ Password verification
✅ JWT generation
✅ Token extraction
✅ Token verification
✅ Protected endpoints
✅ A reusable authentication dependency

We'll use an in-memory dictionary so it's beginner-friendly. In a real app, PostgreSQL and a repository layer replace it.

12.1 📁 Folder structure

backend/
│
├── app/
│   ├── main.py
│   │
│   ├── api/
│   │   └── auth.py
│   │
│   ├── core/
│   │   └── security.py
│   │
│   ├── dependencies/
│   │   └── auth.py
│   │
│   ├── models/
│   │   └── user.py
│   │
│   └── schemas/
│       └── auth.py
│
└── requirements.txt

File	Responsibility
`core/security.py`	Password hashing, JWT creation and verification
`api/auth.py`	Login and registration endpoints
`dependencies/auth.py`	Extract token, verify token, identify current user
`models/user.py`	User database model
`schemas/auth.py`	Request and response validation
`main.py`	Creates the FastAPI application

This is separation of concerns: each file has one job. 🎯

Install what we need:

pip install fastapi uvicorn pyjwt "passlib[argon2]" python-multipart

(python-multipart is required because the login endpoint receives form data.)

12.2 🔐 Step 1: `core/security.py`

import os
from datetime import datetime, timedelta, timezone

import jwt
from jwt import InvalidTokenError
from passlib.context import CryptContext

# 🔑 Load from environment, NEVER hardcode in real projects
SECRET_KEY = os.environ.get("JWT_SECRET_KEY", "dev-only-change-me")
ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES = 30

pwd_context = CryptContext(schemes=["argon2"], deprecated="auto")


# 🔒 Password helpers
def hash_password(password: str) -> str:
    return pwd_context.hash(password)


def verify_password(plain_password: str, hashed_password: str) -> bool:
    return pwd_context.verify(plain_password, hashed_password)


# 🎟️ JWT helpers
def create_access_token(user_id: int) -> str:
    now = datetime.now(timezone.utc)
    payload = {
        "sub": str(user_id),
        "iat": now,
        "exp": now + timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES),
    }
    return jwt.encode(payload, SECRET_KEY, algorithm=ALGORITHM)


def decode_access_token(token: str) -> str | None:
    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
    except InvalidTokenError:
        return None
    return payload.get("sub")

Four reusable operations in one place:

hash_password()
verify_password()
create_access_token()
decode_access_token()

12.3 🚪 Step 2: Registration and login routes (`api/auth.py`)

from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordRequestForm
from pydantic import BaseModel

from app.core.security import (
    create_access_token,
    hash_password,
    verify_password,
)

router = APIRouter()

# 🧪 In-memory "database" (use PostgreSQL in real life!)
users: dict[str, dict] = {}


class RegisterRequest(BaseModel):
    email: str
    password: str


@router.post("/register", status_code=status.HTTP_201_CREATED)
def register(data: RegisterRequest):
    if data.email in users:
        raise HTTPException(
            status_code=status.HTTP_400_BAD_REQUEST,
            detail="User already exists",
        )

    users[data.email] = {
        "id": len(users) + 1,
        "email": data.email,
        "hashed_password": hash_password(data.password),
    }
    return {"message": "User registered successfully"}


@router.post("/login")
def login(form_data: OAuth2PasswordRequestForm = Depends()):
    user = users.get(form_data.username)  # username field = our email

    # Same error for "no such user" and "wrong password" 👇
    if not user or not verify_password(form_data.password, user["hashed_password"]):
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Incorrect email or password",
            headers={"WWW-Authenticate": "Bearer"},
        )

    token = create_access_token(user["id"])
    return {"access_token": token, "token_type": "bearer"}

🧐 What is `OAuth2PasswordRequestForm`?

form_data: OAuth2PasswordRequestForm = Depends()

It's another FastAPI-provided dependency that extracts login credentials sent as form data:

form_data.username
form_data.password

The client submits:

POST /login
Content-Type: application/x-www-form-urlencoded

username=deepika@gmail.com&password=deepika123

Notice it says username even if your app identifies users by email. Just treat the username field as the email.

⚠️ Heads up: the OAuth2 Resource Owner Password Credentials grant is discouraged by modern OAuth security guidance for new systems. This form-based pattern is perfect for learning and for a first-party login API, but for production with external identity providers prefer Authorization Code with PKCE. A first-party login API can also simply accept a JSON request body.

12.4 🛡️ Step 3: Reusable authentication dependency (`dependencies/auth.py`)

from fastapi import Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer

from app.api.auth import users
from app.core.security import decode_access_token

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="login")


def get_current_user(token: str = Depends(oauth2_scheme)):
    credentials_error = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Invalid or expired token",
        headers={"WWW-Authenticate": "Bearer"},
    )

    user_id = decode_access_token(token)
    if user_id is None:
        raise credentials_error

    # ✅ Check the user still exists (real apps: query the DB / session)
    user = next((u for u in users.values() if str(u["id"]) == user_id), None)
    if user is None:
        raise credentials_error

    return user

Trace it:

oauth2_scheme → expects a bearer token in the Authorization header
Depends(oauth2_scheme) → FastAPI extracts the token and injects it
decode_access_token(token) → verifies the token and returns the user ID
if user_id is None → invalid or expired token means reject
We look the user up, because a valid signature doesn't prove the account is still active
Return the authenticated user

12.5 🔒 Step 4: Protect an endpoint (`main.py`)

from fastapi import Depends, FastAPI

from app.api.auth import router as auth_router
from app.dependencies.auth import get_current_user

app = FastAPI()
app.include_router(auth_router)


@app.get("/profile")
def get_profile(current_user: dict = Depends(get_current_user)):
    return {
        "message": "Welcome!",
        "user_id": current_user["id"],
        "email": current_user["email"],
    }

That one line:

current_user: dict = Depends(get_current_user)

does a LOT of work. Before /profile runs, FastAPI:

Calls get_current_user()
Extracts the token via oauth2_scheme
Verifies the JWT
Loads the user
Injects the user into your endpoint

✅ Success:

{
  "message": "Welcome!",
  "user_id": 1,
  "email": "deepika@gmail.com"
}

❌ Failure (the endpoint body never runs):

{
  "detail": "Invalid or expired token"
}

12.6 🧪 Try it!

Run the app:

uvicorn app.main:app --reload

Then open http://127.0.0.1:8000/docs, register a user, click Authorize (🔓) and log in. Swagger UI now automatically sends your bearer token. Or test with curl:

# 1. Register
curl -X POST http://127.0.0.1:8000/register \
  -H "Content-Type: application/json" \
  -d '{"email": "deepika@gmail.com", "password": "deepika123"}'

# 2. Login (form data!)
curl -X POST http://127.0.0.1:8000/login \
  -d "username=deepika@gmail.com&password=deepika123"

# 3. Call the protected route
curl http://127.0.0.1:8000/profile \
  -H "Authorization: Bearer <paste_access_token_here>"

⚠️ This is intentionally simplified. The in-memory users dict isn't shared between processes and disappears on restart. A real implementation uses database-backed users and sessions.

12.7 🚦 Bonus: Authorization (role checks)

Authentication tells you who. Now let's check what they can do, by building on get_current_user:

from fastapi import Depends, HTTPException, status

def require_admin(current_user: dict = Depends(get_current_user)):
    if current_user.get("role") != "admin":
        raise HTTPException(
            status_code=status.HTTP_403_FORBIDDEN,
            detail="You don't have permission to do this",
        )
    return current_user


@app.delete("/posts/{post_id}")
def delete_post(post_id: int, admin: dict = Depends(require_admin)):
    return {"message": f"Post {post_id} deleted"}

Dependencies can depend on other dependencies. 🧩 That's what makes Depends() so powerful.

1️⃣3️⃣ Complete execution flow 🧭

Phase A: Registration 📝

User enters email + password
        │
        ▼
POST /register
        │
        ▼
hash_password()  →  plaintext becomes a secure hash
        │
        ▼
PostgreSQL stores: email + password hash

Phase B: Login 🔑

POST /login  (username + password)
        │
        ▼
Backend finds the user, verify_password()
        │
        ▼
create_access_token()  →  JWT created and signed
        │
        ▼
Frontend receives: access_token + "bearer"

Phase C: Accessing a protected endpoint 🚪

React requests /profile
Authorization: Bearer <JWT>
        │
        ▼
OAuth2PasswordBearer  →  extracts the JWT
        │
        ▼
get_current_user()  →  verifies token, identifies user
        │
        ▼
Depends(get_current_user)  →  injects the user into the endpoint
        │
        ▼
✅ Endpoint returns the user's profile

If you understand this flow, you understand the foundation of JWT authentication in FastAPI. 🏆

1️⃣4️⃣ Token expiry, refresh tokens and logout ⏰

Our JWT contains:

{
  "sub": "101",
  "exp": 1794400000
}

Suppose the token is created at 10:00 AM and expires at 10:30 AM:

10:15 AM request → ✅ valid, the request proceeds
10:45 AM request → ❌ expired, the backend rejects it, and the user needs a new access token

14.1 🎫 Access token

A credential presented to an API to access protected resources.

GET /feed
Authorization: Bearer <access_token>

Access tokens are short-lived, so a stolen one is only useful for a limited time (plus any extra revocation checks).

14.2 🔁 Refresh token

A credential used to get a new access token without asking for the password again.

Access token lifetime: 30 minutes
Refresh token lifetime: 7 days

When the access token expires:

Frontend sends the refresh token to a refresh endpoint
Backend validates it
Backend issues a new access token
The user keeps using the app seamlessly ✨

Refresh tokens are especially sensitive. Store them carefully, rotate them when used where appropriate, and make them revocable. For browser apps, a common approach is HttpOnly, Secure, properly configured SameSite cookies.

14.3 🚪 What about logout?

Traditional server-side sessions can be killed by deleting the session on the server. But a self-contained JWT stays cryptographically valid until it expires, even after the user clicks "Logout". The JWT doesn't magically know the user logged out. 🤔

Possible solutions:

Keep access tokens short-lived
Track or revoke sessions server-side
Use a token denylist when appropriate
Revoke refresh tokens
Rotate session credentials

This is why real-world systems often combine tokens with database-backed sessions.

1️⃣5️⃣ JWT vs Session-based authentication ⚔️

Both are widely used in production.

🗄️ Session-based

The backend creates a session in the database:

Session ID	User ID	Expires
abc123	101	7 days

On every request: the frontend sends the session ID → the backend looks it up → checks validity → identifies the user.

🎟️ JWT-based

The backend issues a signed JWT containing identity and expiry. On every request: the frontend sends the JWT → the backend verifies the signature → checks the claims → identifies the user.

JWT	Server-side session
Identity carried in token claims	Identity retrieved via session lookup
Can be verified without a DB lookup	Normally requires a session lookup
Revocation needs extra mechanisms	Revoked by invalidating the session
Self-contained token	Server maintains session state
Common in distributed APIs	Common in traditional web apps

⚖️ Neither is automatically better. The right choice depends on your architecture and security requirements.

💼 In my Social Feed project

My implementation actually uses database-backed sessions (7-day lifetime) with Argon2id for passwords, and SHA-256 hashing of a random session token for lookups. That's different from the JWT example above:

My session implementation	JWT implementation
Random session token	Structured JWT
Hash token for DB lookup	Cryptographically sign token
Database session lookup	Signature verification
Easy server-side revocation	Needs extra revocation strategy

🧠 Why is SHA-256 okay here? Because it's hashing a high-entropy random token, not a human-chosen password. Passwords need slow, salted algorithms like Argon2id or bcrypt. Random tokens don't, since they can't be guessed from a dictionary.

1️⃣6️⃣ HTTP status codes in authentication 📟

Code	Meaning	Example
`200`	OK	Successful login
`201`	Created	Successful registration
`400`	Bad Request	Invalid request
`401`	Unauthorized	Missing or invalid authentication
`403`	Forbidden	Authenticated but lacks permission
`404`	Not Found	User doesn't exist
`422`	Validation Error	Invalid request schema
`500`	Internal Server Error	Unexpected backend failure

The most important distinction: `401` vs `403`

🪪 You try to enter an office without an ID card → 401
🚫 You show a valid ID card but try to enter the server room → 403

Authentication asks: is your identity valid? Authorization asks: do you have permission?

1️⃣7️⃣ Common security mistakes ⚠️

Mistake	Why it's dangerous
Storing plaintext passwords	A DB leak exposes passwords
Using plain SHA-256 for passwords	Too fast, easy to brute-force
Hardcoding JWT secrets	The secret leaks through source code
Never-expiring access tokens	Stolen tokens stay useful forever
Putting passwords in the JWT payload	The payload can be decoded
Accepting any JWT algorithm	Serious verification vulnerabilities
Not checking token expiration	Expired credentials may be accepted
Detailed login errors	Reveals which accounts exist
Storing tokens carelessly	XSS or other attacks can steal them
Forgetting HTTPS	Credentials and tokens can be intercepted

📖 A few more terms worth knowing

HTTPS → encrypts communication between browser and server
CORS → controls which browser origins may access your backend (it is not authentication!)
XSS → malicious scripts executing in a user's browser
CSRF → tricks a browser into making an unwanted authenticated request
Brute-force attack → repeatedly guessing passwords
Rate limiting → restricts excessive requests, like repeated login attempts

🍪 Where should the frontend store tokens?

localStorage is readable by JavaScript, so it's exposed if an XSS vulnerability exists. HttpOnly cookies can't be read by JavaScript, but cookie-based auth needs proper CSRF protection. There's no free lunch. Design it carefully. 🧠

1️⃣8️⃣ How this connects to real FastAPI architecture 🏛️

┌──────────────────────────────────────────────┐
│  API / Router layer                          │
│  Receives login and protected requests       │
│  e.g. auth.py, users.py                      │
├──────────────────────────────────────────────┤
│  Dependency layer                            │
│  Extracts token, identifies the user         │
│  e.g. get_current_user()                     │
├──────────────────────────────────────────────┤
│  Security layer                              │
│  Hash/verify passwords, issue/verify JWTs    │
│  e.g. security.py                            │
├──────────────────────────────────────────────┤
│  Service layer                               │
│  Authentication business logic               │
│  e.g. auth_service.py                        │
├──────────────────────────────────────────────┤
│  Repository / Database layer                 │
│  Fetch users, create sessions, run queries   │
│  e.g. user_repository.py, session_repository.py │
└──────────────────────────────────────────────┘

🧼 Principle: keep the router thin. Delegate business logic to services and database work to repositories.

1️⃣9️⃣ Interview questions & answers 🎤

Q1. What is authentication?
Verifying the identity of a user before allowing access to protected resources.

Q2. What is JWT?
JSON Web Token, a compact, signed token format used to transfer claims between parties.

Q3. What are the three parts of a JWT?
Header, payload and signature.

Q4. Is JWT encrypted?
Not by default. A typical signed JWT has an encoded header and payload plus a signature. The payload can be decoded by anyone.

Q5. What is OAuth2PasswordBearer?
A FastAPI security dependency that extracts a bearer token from the Authorization header and declares an OAuth2 security scheme in the API docs.

Q6. Does OAuth2PasswordBearer verify JWT signatures?
No. It only extracts the token. You need separate verification logic.

Q7. What is Depends()?
FastAPI's dependency injection mechanism, which lets reusable functions and resources be provided to endpoints.

Q8. What is password hashing?
A one-way transformation of a password into a secure stored form that can be verified later without storing the original.

Q9. What is bcrypt?
A password-hashing algorithm designed to make guessing computationally expensive.

Q10. What is a salt?
A random value included in password hashing so identical passwords produce different stored hashes.

Q11. Access token vs refresh token?
An access token accesses protected APIs. A refresh token obtains a new access token.

Q12. What happens when a JWT expires?
The backend rejects it. The client may use a valid refresh mechanism to get a new one.

Q13. JWT vs session authentication?
JWT carries signed claims the server can verify. Sessions store state on the server and look it up via a session ID.

Q14. Why not store passwords directly in PostgreSQL?
A database breach would expose real passwords. We store salted, computationally expensive hashes instead.

Q15. 401 vs 403?
401 means valid authentication is missing. 403 means you're authenticated but lack permission.

2️⃣0️⃣ Quick knowledge test 🧪

Answer without scrolling up! (Answers are below.)

1. What does OAuth2PasswordBearer actually do?

A) Generates a JWT
B) Extracts the bearer token from the Authorization header
C) Hashes the password
D) Connects to PostgreSQL

2. Which three parts make up a JWT?

A) Username, password, signature
B) Header, payload, signature
C) Request, response, token
D) Issuer, database, session

3. Why do we hash passwords?

A) To decrypt them later
B) To make login faster
C) To avoid storing plaintext passwords
D) To create JWTs

4. What does Depends() do in FastAPI?

A) Automatically injects dependency results
B) Creates database tables
C) Encrypts tokens
D) Starts Uvicorn

5. Is JWT payload encryption automatic?

A) Yes, always
B) No, ordinary signed JWTs are not encrypted
C) Only with HS256
D) Only with FastAPI

6. What happens when a JWT expires?

A) It becomes valid forever
B) The backend should reject it
C) It refreshes itself automatically
D) The database deletes the user

👀 Click to reveal answers

🧾 Final revision: everything in one place

Concept	What to remember
Authentication	Verifying who the user is
Authorization	Checking what the user can access
Token	Credential representing an authenticated session or identity
JWT	Signed token format containing claims
Header	Algorithm and token type
Payload	User identity and claims
Signature	Protects token integrity
OAuth2	Authorization framework
`OAuth2PasswordBearer`	Extracts the bearer token
`Depends()`	Dependency injection
Password hashing	Secure password storage
bcrypt	Password-hashing algorithm
Argon2id	Modern memory-hard password-hashing algorithm
Passlib	Python password-hashing library
Access token	Used to access protected resources
Refresh token	Used to obtain new access tokens
`get_current_user()`	Reusable dependency to identify the authenticated user

🎯 Final thoughts

Don't just memorize definitions. Be able to explain the complete journey:

A user clicks Login in React → FastAPI verifies the password hash → a token is issued → React sends it on every request → OAuth2PasswordBearer extracts it → get_current_user() verifies it → Depends() injects the user → the protected /feed endpoint runs. 🚀

If you can narrate that flow confidently, you truly understand authentication.

And remember: JWT-based and database-session-based authentication are different designs. Both are valid. Knowing which one your code uses, and why, is what separates memorizing from understanding. 💡

🙌 Enjoyed this post?

❤️ Drop a reaction
🦄 Save it for revision
💬 Comment with questions, and I'll answer them!

Follow me so you don't miss it! 🔔

Happy coding! 👩‍💻✨

fastapi #python #jwt #security #webdev

SQLAlchemy 2.0 & Alembic Migrations: A Complete Beginner's Guide🗄️

Deepika Pusala — Wed, 11 Nov 2026 07:12:43 +0000

🗄️ SQLAlchemy 2.0 & Alembic Migrations: Let's Learn from ZERO!

Once you understand SQLAlchemy 2.0 + Alembic, you understand how a real Python backend talks to PostgreSQL, stores data, retrieves records, updates them, and safely changes database structure as the app grows. 🚀

No assumptions here. You don't need to know ORMs, sessions, migrations, or database internals. We'll use examples from an Instagram-like SocialFeed app throughout.

1. 🖼️ The Big Picture

SocialFeed stores users, posts, comments, likes, follows, notifications and sessions. All of it must live permanently in PostgreSQL. But your backend is written in Python, while PostgreSQL understands SQL. How do they talk?

🐍 Python / FastAPI     → your application code
        ▼
🧱 SQLAlchemy 2.0       → converts Python database operations into SQL
        ▼
🐘 PostgreSQL           → stores and retrieves your data

🔄 Alembic              → manages changes to your database structure over time

SQLAlchemy helps Python communicate with the database
Alembic helps you modify the database structure safely over time

🧱 PART 1: SQLAlchemy Fundamentals

2. 🤔 What exactly is SQLAlchemy?

📌 SQLAlchemy is a Python library for interacting with relational databases.

It lets you create tables, insert, retrieve, update and delete records, manage connections and transactions, and work with relationships.

Raw SQL to fetch user 10:

SELECT * FROM users WHERE id = 10;

With the SQLAlchemy ORM:

user = session.get(User, 10)

Raw SQL: you write SQL statements yourself
SQLAlchemy ORM: you work with Python classes and objects

⚠️ SQLAlchemy doesn't eliminate SQL. It helps you work with SQL through Python.

3. 🪄 What is an ORM?

ORM = Object Relational Mapping.

Object: Python objects
Relational: relational databases with tables and relationships
Mapping: connecting Python objects to database tables

The users table:

id	username	email
1	deepika	deepika@gmail.com
2	rahul	rahul@gmail.com
3	priya	priya@gmail.com

A Python class representing it:

class User:
    def __init__(self, id, username, email):
        self.id = id
        self.username = username
        self.email = email


user = User(id=1, username="deepika", email="deepika@gmail.com")

Python class  User   (id, username, email)
        ⇅  ORM mapping
Database table  users   (id, username, email)

One User object = one row in the users table

SQLAlchemy ORM makes this mapping automatic. Instead of writing SQL for every operation you write Python:

user.username = "new_username"

SQLAlchemy detects the change and generates an SQL UPDATE when the transaction is flushed.

4. ⚔️ SQLAlchemy Core vs SQLAlchemy ORM

🎯 A very important interview and practical concept.

SQLAlchemy offers two ways of working with a database. Both talk to the same database. The difference is how you express operations.

🅰️ SQLAlchemy Core

A SQL expression toolkit. You work with tables, columns, SQL expressions and statements, with no Python object per row.

from sqlalchemy import select

statement = select(users_table)
result = session.execute(statement)
users = result.all()

select() creates a SELECT statement
users_table represents the database table
execute() runs it
all() retrieves all matching rows

Inserting with Core:

from sqlalchemy import insert

statement = insert(users_table).values(
    username="deepika",
    email="deepika@gmail.com",
)

session.execute(statement)
session.commit()

🅱️ SQLAlchemy ORM

You work with Python classes representing tables:

statement = select(User)
result = session.scalars(statement)
users = result.all()

for user in users:
    print(user.username)
    print(user.email)

User is an ORM model, and SQLAlchemy returns User objects instead of raw column-oriented rows.

📊 Core vs ORM

Feature	Core	ORM
Main approach	Tables and SQL expressions	Python classes and objects
Uses	Table definitions	ORM mapped classes
Returns	Rows / column results	ORM objects
Abstraction	Lower-level	Higher-level
Relationships	Explicit SQL joins	ORM relationships supported
Best for	SQL-heavy operations	Application development

💡 Neither is universally better, and you can combine them. A FastAPI app might use the ORM for normal user and post operations, and Core for bulk inserts or complex reports. Learn the ORM thoroughly first, while understanding Core enough to recognize its SQL expression API.

🏗️ PART 2: Declarative Models

5. 📐 What are Declarative Models?

In SQL you'd write:

CREATE TABLE users (
    id INTEGER PRIMARY KEY,
    username VARCHAR(50) NOT NULL,
    email VARCHAR(100) NOT NULL
);

SQLAlchemy lets you define the same table with a Python class. That's a declarative model.

📌 A declarative model is a Python class that describes a database table, including its columns, data types, constraints and relationships.

Step 1: install

pip install sqlalchemy

Step 2: the imports

from sqlalchemy import String
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column

Import	Purpose
`String`	Defines a string database column
`DeclarativeBase`	Base class for declarative models
`Mapped`	Type annotation for mapped attributes
`mapped_column`	Defines a mapped database column

Step 3: create a Base class

class Base(DeclarativeBase):
    pass

Base is the common parent class for your models. Every model inherits from it:

class User(Base):
    ...


class Post(Base):
    ...


class Comment(Base):
    ...

Because they inherit from Base, SQLAlchemy knows they're ORM models and tracks their table definitions in shared metadata.

Step 4: create the User model

from sqlalchemy import String
from sqlalchemy.orm import Mapped, mapped_column


class User(Base):
    __tablename__ = "users"

    id: Mapped[int] = mapped_column(primary_key=True)

    username: Mapped[str] = mapped_column(
        String(50),
        nullable=False,
    )

    email: Mapped[str] = mapped_column(
        String(100),
        nullable=False,
        unique=True,
    )

Line by line:

class User(Base): a Python class named User that inherits from Base
__tablename__ = "users": the actual database table name (class User ↔ table users)
id: Mapped[int]: id is a mapped Python attribute of type int
mapped_column(primary_key=True): defines the column and makes it the primary key, which uniquely identifies each row. No two rows can share one
nullable=False: the database doesn't allow NULL here
unique=True: duplicate values aren't allowed (two users can't share an email)

🎨 Common column types

SQLAlchemy type	Python type	Database meaning
`Integer`	`int`	Whole number
`String(100)`	`str`	Limited-length text
`Text`	`str`	Longer text
`Boolean`	`bool`	True / False
`DateTime`	`datetime`	Date and time
`Float`	`float`	Floating-point number
`Numeric`	`Decimal`	Precise numeric value
`JSON`	`dict` / `list` etc.	JSON data

📝 Python annotations and database types are related, but not exactly the same thing.

6. 🔍 `Mapped` and `mapped_column` in Depth

You'll see both everywhere in SQLAlchemy 2.0. Why do both exist?

username: Mapped[str] = mapped_column(
    String(50),
    nullable=False,
)

Mapped[str] is the Python-side typing information. It tells developers what type to expect, tells IDEs what to suggest, lets static type checkers check, and tells SQLAlchemy the attribute is mapped.

mapped_column() describes the actual database column: its type, primary key, foreign key, nullability, unique constraints and defaults.

🧠 Mapped[str] = Python-side declaration. mapped_column() = database column configuration.

❓ Optional columns

A user's bio isn't compulsory:

from typing import Optional

bio: Mapped[Optional[str]] = mapped_column(
    String(300),
    nullable=True,
)

In modern Python you can write:

bio: Mapped[str | None] = mapped_column(
    String(300),
    nullable=True,
)

The attribute can hold a string or None, and the database column allows NULL.

🎚️ Default values

is_active: Mapped[bool] = mapped_column(
    default=True,
    nullable=False,
)

You can also configure a database-side default:

server_default="true"

Option	Who applies it?
`default`	SQLAlchemy, when constructing an INSERT without an explicit value
`server_default`	The database itself, when an INSERT omits that column

🔗 PART 3: Relationships Between Models

7. 🗝️ What are Foreign Keys?

A user creates multiple posts.

users:

id	username
1	deepika
2	rahul

posts:

id	content	user_id
101	Hello world	1
102	Beautiful day	1
103	Learning SQLAlchemy	2

user_id tells us who created the post. It's a foreign key referencing the id column of users:

from sqlalchemy import ForeignKey

user_id: Mapped[int] = mapped_column(
    ForeignKey("users.id"),
    nullable=False,
)

This creates a database-level reference and helps maintain referential integrity. For example, you normally can't insert a post with user_id=999 if no such user exists (assuming foreign key enforcement is on).

8. 🔄 What is `relationship()`?

A foreign key connects tables. But what if you want convenient access to related Python objects? That's relationship():

from sqlalchemy import ForeignKey, String
from sqlalchemy.orm import Mapped, mapped_column, relationship


class User(Base):
    __tablename__ = "users"

    id: Mapped[int] = mapped_column(primary_key=True)
    username: Mapped[str] = mapped_column(String(50))

    posts: Mapped[list["Post"]] = relationship(
        back_populates="author",
    )


class Post(Base):
    __tablename__ = "posts"

    id: Mapped[int] = mapped_column(primary_key=True)
    content: Mapped[str] = mapped_column(String(500))

    user_id: Mapped[int] = mapped_column(ForeignKey("users.id"))

    author: Mapped["User"] = relationship(
        back_populates="posts",
    )

User.posts → a user can access their posts
Post.author → a post can access its author

user = session.get(User, 1)

print(user.username)

for post in user.posts:
    print(post.content)

post = session.get(Post, 101)
print(post.author.username)

One-to-many relationship

User (id = 1)  ──▶  Post 101
                ──▶  Post 102
                ──▶  Post 103

One User → many Posts. Each Post → one User.

🧠 ForeignKey defines the database-level relationship. relationship() defines the convenient Python-level object relationship. You usually need both.

9. 🧩 Other Relationship Types

One-to-one: one user has one profile.

User 1 ───── Profile 1

One-to-many: one user has many posts.

User 1 ───── Post 1
        ├─── Post 2
        └─── Post 3

Many-to-many: many users like many posts.

User 1 ───┬── Post 1
          └── Post 2

User 2 ───┬── Post 1
          └── Post 3

Many-to-many relationships generally use an association table. For SocialFeed, a likes table is a natural example:

user_id	post_id
1	101
2	101
1	102

🔌 PART 4: Engine and Database Connections

10. ⚙️ What is a SQLAlchemy Engine?

Before Python can execute queries, it needs a way to connect to PostgreSQL. SQLAlchemy uses an Engine, the central database connection manager. It knows which database to connect to, which driver to use, how to create connections, and how to manage the connection pool.

from sqlalchemy import create_engine

engine = create_engine(
    "postgresql+psycopg://user:password@localhost/mydb"
)

postgresql → database
psycopg → driver
user / password → credentials
localhost → host
mydb → database name

📝 Creating the engine doesn't necessarily open a connection right away. Connections are generally obtained when needed.

🏊 What is a connection pool?

100 API requests each need the database. Would you create a brand-new connection every time? That's inefficient. SQLAlchemy uses connection pooling: a collection of reusable connections.

FastAPI requests:  A · B · C
        ▼
SQLAlchemy Engine  → manages reusable connections
        ▼
Connection 1 · Connection 2 · Connection 3
        ▼
🐘 PostgreSQL

A connection is checked out when needed and returned to the pool when its transaction or usage ends. Pool settings include pool_size, max_overflow, pool_timeout and pool_recycle. You don't need to tune them yet. First understand why the pool exists.

🧑‍💻 PART 5: Session and AsyncSession

🎯 One of the MOST IMPORTANT sections!

11. 🗂️ What is a Session?

A user calls POST /posts. Your backend must connect to the database, insert the post, commit the transaction, and release resources.

📌 A Session is a workspace for interacting with the database through the ORM. It manages ORM objects, database operations, pending changes, transactions and the identity map.

🅰️ Creating a Session

from sqlalchemy.orm import Session, sessionmaker

SessionLocal = sessionmaker(
    bind=engine,
    class_=Session,
)

session = SessionLocal()

sessionmaker is a factory for creating sessions.

🅱️ Adding a record

user = User(
    username="deepika",
    email="deepika@gmail.com",
)

session.add(user)
session.commit()

Create a Python object. It exists only in Python memory and isn't necessarily in PostgreSQL yet
session.add(user). SQLAlchemy tracks the object as pending
session.commit(). SQLAlchemy flushes pending changes and commits the transaction. Now the database stores the record

🅲 What is `flush()`?

🚨 A VERY important distinction!

session.add(user)
session.flush()

SQLAlchemy sends the INSERT to the database within the current transaction, but does NOT commit:

The SQL has been executed
The transaction remains uncommitted
Other transactions generally can't see the change
You can still roll back

session.add(user)
session.flush()

print(user.id)     # the generated ID is available!

session.rollback()

Why is flush() useful? Sometimes you need the generated ID before committing. For example, insert a user, get their ID, then create another record that references it, all in one transaction.

🅳 `commit()`

Commits the current transaction and makes changes permanent (subject to database durability).

🅴 `rollback()`

Something went wrong mid-way and you don't want a half-finished transaction:

session.rollback()

🅵 `close()`

session.close()

Closes the Session and releases its transactional resources. It doesn't destroy your connection pool.

📋 Remember these five methods

Method	Meaning
`add()`	Track an object for persistence
`flush()`	Send pending SQL changes without committing
`commit()`	Commit the transaction
`rollback()`	Undo uncommitted transaction changes
`close()`	Close the Session and release resources

12. 🪪 What is the Identity Map?

A Session maintains an identity map: within a Session, the same database identity is represented by the same Python object.

user1 = session.get(User, 1)
user2 = session.get(User, 1)

print(user1 is user2)   # typically True

If the object is still in the identity map and hasn't expired, SQLAlchemy can return the same Python object. This keeps things consistent: you don't get two separate in-memory objects for the same row in the same Session.

📝 An identity map is not a general-purpose query cache. A new query with different filters may still execute SQL.

13. ⚡ What is `AsyncSession`?

FastAPI supports async endpoints:

@app.get("/users")
async def get_users():
    ...

But using a synchronous database driver inside an async endpoint can block the event loop. SQLAlchemy therefore provides AsyncEngine, AsyncSession and asynchronous execution.

Session	AsyncSession
Synchronous	Asynchronous
`Session`	`AsyncSession`
`create_engine()`	`create_async_engine()`
`session.execute()`	`await session.execute()`
Blocking DB I/O	Non-blocking DB I/O (with an async driver)

🛠️ Creating an AsyncEngine

pip install sqlalchemy asyncpg

asyncpg is an asynchronous PostgreSQL driver.

from sqlalchemy.ext.asyncio import create_async_engine

DATABASE_URL = "postgresql+asyncpg://user:password@localhost/mydb"

engine = create_async_engine(
    DATABASE_URL,
    echo=True,
)

postgresql → database type
asyncpg → async driver
echo=True → prints generated SQL to the console (great for development; don't leave sensitive SQL parameters exposed in production logs)

🏭 Creating an AsyncSession factory

from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker

AsyncSessionLocal = async_sessionmaker(
    bind=engine,
    class_=AsyncSession,
    expire_on_commit=False,
)

expire_on_commit=False means ORM objects keep their already-loaded attribute values after a commit instead of automatically expiring. This matters in async code because accessing an expired attribute could require an implicit database load. It does not automatically load relationships or guarantee every attribute is available.

▶️ Using it

async with AsyncSessionLocal() as session:
    result = await session.execute(select(User))
    users = result.scalars().all()

Why await? Executing a query requires I/O, which takes time. await lets the coroutine suspend while waiting so other tasks can run on the event loop.

14. 🚪 What is `async with`?

async with AsyncSessionLocal() as session:
    ...

This is an asynchronous context manager. It manages the Session's lifecycle, and the Session is closed when the block finishes. It's like:

session = AsyncSessionLocal()

try:
    # Perform database operations
    pass
finally:
    await session.close()

⚠️ Closing a session does not automatically commit your changes. Commit explicitly when appropriate.

🧪 A complete AsyncSession example

async def create_user():
    async with AsyncSessionLocal() as session:
        user = User(
            username="deepika",
            email="deepika@gmail.com",
        )

        session.add(user)

        await session.commit()
        await session.refresh(user)

        return user

refresh() reloads an ORM object's attributes from the database. The database may have generated values such as auto-generated IDs, server-side timestamps or database defaults, and refreshing retrieves them.

15. ⏳ What is `expire_on_commit=False`?

await session.commit()

print(user.username)

By default, SQLAlchemy expires ORM object attributes after a commit. Expiration means SQLAlchemy may need to reload them from the database when you access them.

In sync code this reload happens transparently
In async code implicit database I/O is a problem, because it needs to be awaited

So many async applications configure expire_on_commit=False. But remember: it doesn't make unloaded relationships or deferred columns magically available.

16. 🚨 Don't Share a Session Across Concurrent Tasks

A common beginner mistake is sharing one Session across multiple requests or tasks. Don't!

A SQLAlchemy Session is mutable and stateful. It isn't safe to share concurrently between threads. Likewise, an AsyncSession isn't safe to share across concurrent asyncio tasks:

# ❌ Avoid: both functions use the SAME AsyncSession concurrently
await asyncio.gather(
    do_query(session),
    do_another_query(session),
)

Use separate AsyncSession instances for independent concurrent tasks. This is especially important in real FastAPI apps.

🏗️ PART 6: A Real Async Database Setup

17. 🗂️ The project structure

backend/
│
├── app/
│   ├── api/
│   ├── models/
│   │   ├── user.py
│   │   └── post.py
│   │
│   ├── repositories/
│   │   ├── user_repository.py
│   │   └── post_repository.py
│   │
│   ├── services/
│   ├── schemas/
│   ├── database.py
│   └── main.py
│
├── alembic/
├── alembic.ini
└── tests/

database.py

from sqlalchemy.ext.asyncio import (
    AsyncSession,
    async_sessionmaker,
    create_async_engine,
)

DATABASE_URL = "postgresql+asyncpg://user:password@localhost/mydb"

engine = create_async_engine(
    DATABASE_URL,
    echo=True,
)

AsyncSessionLocal = async_sessionmaker(
    engine,
    class_=AsyncSession,
    expire_on_commit=False,
)

Now a dependency:

async def get_db():
    async with AsyncSessionLocal() as session:
        yield session

yield provides the Session to FastAPI and keeps it open while the request uses it. When the request's dependency scope finishes, the context manager closes the Session.

⚡ Using it in FastAPI

from fastapi import Depends
from sqlalchemy.ext.asyncio import AsyncSession


@app.get("/users")
async def get_users(db: AsyncSession = Depends(get_db)):
    ...

db → the variable holding the session
AsyncSession → the expected type
Depends(get_db) → FastAPI calls the dependency and injects the resulting session

That's dependency injection. You don't manually create a session in every endpoint.

🏛️ Architecture point: one request generally gets its own Session. The Session should not be a global shared object.

🔄 PART 7: Alembic Migrations

18. 🤔 What is Alembic?

📌 Alembic is a database migration tool designed to work with SQLAlchemy. Its purpose is to manage changes to your database schema over time.

A schema describes the structure of your database: tables, columns, data types, primary keys, foreign keys, constraints and indexes.

Version 1: users(id, username)
        ▼  new feature added
Version 2: users(id, username, email)

You could run this by hand:

ALTER TABLE users
ADD COLUMN email VARCHAR(100);

But after hundreds of changes over several months, how do you track:

Which changes have been made?
Which have been applied to production?
Which are pending?
How do you reverse one?
How do your teammates apply the same changes?

Alembic solves this by keeping a history of schema changes through migration files.

19. 📜 What is a migration?

📌 A migration is a Python file containing instructions to change your database schema.

For example, to add email to users, Alembic creates a migration with:

Upgrade: add the email column
Downgrade: remove the email column

🌿 Migrations are like Git commits

Git	Alembic
Commit	Migration revision
Commit history	Migration history
Checkout / revision navigation	Migration version tracking
Apply code changes	Apply schema changes
Revert a commit	Write/run a downgrade where appropriate

⚠️ Migrations can modify real production data and structures. Downgrades aren't always perfectly reversible.

20. 📥 Installing Alembic

pip install alembic
alembic init alembic

backend/
│
├── app/
│   ├── models/
│   ├── database.py
│   └── main.py
│
├── alembic/
│   ├── versions/
│   ├── env.py
│   └── script.py.mako
│
└── alembic.ini

File / folder	Purpose
`alembic.ini`	Alembic configuration
`alembic/env.py`	Environment and migration configuration
`alembic/versions/`	Stores migration files
`script.py.mako`	Template used to generate migration files

env.py is one of the most important Alembic files. It tells Alembic which database to connect to, where the SQLAlchemy model metadata is, whether migrations run synchronously or asynchronously, and how to compare the existing schema with your model definitions.

21. 📄 What is `alembic.ini`?

Alembic's main configuration file, containing the migration script location, database connection configuration and logging settings:

[alembic]

script_location = %(here)s/alembic

prepend_sys_path = .

sqlalchemy.url = postgresql://user:password@localhost/mydb

But our app uses postgresql+asyncpg://... and is asynchronous. We can configure the migration environment to use an async SQLAlchemy engine.

🔐 In a real app, don't hardcode credentials in a committed config file. Read them from environment variables or your settings module:

from app.config import settings

database_url = settings.DATABASE_URL

22. 🎯 What is `target_metadata`?

🚨 A VERY important concept!

Remember our Base class?

class Base(DeclarativeBase):
    pass

SQLAlchemy stores information about every model that inherits from it in Base.metadata, which holds the table definitions SQLAlchemy knows about:

class User(Base):
    __tablename__ = "users"

    id: Mapped[int] = mapped_column(primary_key=True)

Alembic needs to compare two things:

Your current SQLAlchemy model definitions
Your actual database schema

The model metadata provides the first side of that comparison. Inside alembic/env.py:

target_metadata = Base.metadata

⚠️ Alembic must also be able to import your model classes so they're registered in Base.metadata. If you forget to import a model, its table may be missing from the metadata, and autogenerate might not detect it.

🧭 PART 8: The Alembic Migration Workflow

23. 🤖 What is Autogenerate?

You have a User model, and the table already exists in your database:

class User(Base):
    __tablename__ = "users"

    id: Mapped[int] = mapped_column(primary_key=True)
    username: Mapped[str] = mapped_column(String(50))

Now you add an email column to the model:

class User(Base):
    __tablename__ = "users"

    id: Mapped[int] = mapped_column(primary_key=True)
    username: Mapped[str] = mapped_column(String(50))

    email: Mapped[str] = mapped_column(
        String(100),
        nullable=False,
    )

🚨 The key point: PostgreSQL doesn't change just because you edited Python code! SQLAlchemy models describe the desired structure. Alembic migrations change the actual database structure.

Now run:

alembic revision --autogenerate -m "add email to users"

alembic → the command-line tool
revision → create a new migration revision
--autogenerate → compare database schema with model metadata and propose changes
-m → a descriptive message

Alembic generates a file in alembic/versions/:

"""add email to users"""

import sqlalchemy as sa
from alembic import op

revision = "abc123"
down_revision = "previous_revision"


def upgrade():
    op.add_column(
        "users",
        sa.Column(
            "email",
            sa.String(length=100),
            nullable=False,
        ),
    )


def downgrade():
    op.drop_column("users", "email")

revision is the migration's unique ID
down_revision identifies the migration that came before it, which establishes the order:

Migration A
    ▼
Migration B
    ▼
Migration C      (C's down_revision points to B)

upgrade() contains the instructions to move the database forward (add the column)
downgrade() contains instructions to reverse it (drop the column)

Old schema: users(id, username)
      │  upgrade()  ──▶  move forward
      │  downgrade() ◀──  move backward
New schema: users(id, username, email)

⚠️ Adding a nullable=False column to a table that already has rows needs a default (or a multi-step migration), or PostgreSQL will reject it. Always review the generated file.

24. ⬆️ `alembic upgrade head`

Creating a migration file doesn't apply it. You must run:

alembic upgrade head

upgrade → apply migrations in the forward direction
head → move to the latest available migration revision

If you have migrations A, B and C and your database is at A, then alembic upgrade head applies B and C.

🧠 revision --autogenerate creates a migration file. upgrade head applies it to the database. They're two separate steps.

25. ⬇️ `alembic downgrade`

Applied a migration by accident?

alembic downgrade -1

This moves back one revision using the current migration's downgrade() function.

Before:  A → B → C
                 ^ current DB

After:   A → B → C
             ^ current DB

alembic downgrade abc123    # to a specific revision
alembic downgrade base      # back to the original empty base

🚨 A downgrade may delete columns, tables or data. Never blindly downgrade a production database.

26. 📋 Alembic Command Cheat Sheet

Command	Purpose
`alembic init alembic`	Initialize Alembic
`alembic revision -m "message"`	Create an empty migration
`alembic revision --autogenerate -m "message"`	Generate a migration from detected schema differences
`alembic upgrade head`	Apply all pending migrations
`alembic upgrade +1`	Apply the next migration
`alembic downgrade -1`	Reverse one migration
`alembic downgrade base`	Reverse all migrations
`alembic current`	Show the current database revision
`alembic history`	Show migration history
`alembic heads`	Show migration head revisions
`alembic stamp head`	Mark the database at head without executing migrations

🏷️ What is `stamp`?

Say you already have a database with tables, but Alembic isn't tracking its migration history. You can tell Alembic the database should be considered to be at a particular revision:

alembic stamp head

This updates Alembic's version tracking without running the migration scripts. It does NOT create missing tables or modify columns. Only use it when you've verified the actual schema matches the revision you're stamping.

27. 📒 What is `alembic_version`?

How does Alembic know which migrations have already been applied? It uses a special table called alembic_version. For a simple single-head history it holds the current revision ID:

version_num
abc123

When you run alembic upgrade head, Alembic:

Checks the current revision
Determines which migrations are pending
Executes the required upgrade functions
Updates the version tracking table

That prevents applied migrations from re-running. (With multiple heads, the tracking can involve multiple revision IDs.)

28. 🔬 How does Autogenerate actually work?

Your database has users(id, username), but your Python model defines users(id, username, email). Alembic performs a schema comparison:

SQLAlchemy metadata:  id, username, email
Actual database:      id, username
        ▼
Schema comparison
        ▼
Detected difference: new column `email VARCHAR(100)`
        ▼
Alembic proposes a migration operation

⚠️ Autogenerate is not perfect. It detects many common differences but can't understand every database change or your business intention:

Some table renames appear as a drop + create

Some column renames appear as drop + add

Data transformations need manual logic

Some constraint and database-specific changes need manual handling

Always inspect the generated migration before applying it.

⚡ PART 9: Async Alembic Configuration

29. 🤝 How does Alembic work with asyncpg?

Your FastAPI app uses postgresql+asyncpg://..., but Alembic's migration environment must be configured appropriately. A common setup uses async_engine_from_config and runs Alembic's synchronous migration operations through connection.run_sync().

Inside alembic/env.py:

import asyncio

from alembic import context
from sqlalchemy import pool
from sqlalchemy.ext.asyncio import async_engine_from_config

from app.database import Base
from app.models import post, user  # noqa: F401  (registers models)

config = context.config

target_metadata = Base.metadata

Why import user and post? So SQLAlchemy registers these models in Base.metadata.

📝 This example imports Base from app.database. Import it from wherever you defined your Base class.

The synchronous migration operation:

def do_run_migrations(connection):
    context.configure(
        connection=connection,
        target_metadata=target_metadata,
    )

    with context.begin_transaction():
        context.run_migrations()

Getting the asynchronous connection:

async def run_async_migrations():
    connectable = async_engine_from_config(
        config.get_section(config.config_ini_section),
        prefix="sqlalchemy.",
        poolclass=pool.NullPool,
    )

    async with connectable.connect() as connection:
        await connection.run_sync(do_run_migrations)

    await connectable.dispose()

Finally:

def run_migrations_online():
    asyncio.run(run_async_migrations())


run_migrations_online()

Why run_sync() if we're using asyncpg? Alembic's migration context and operations are designed around synchronous-style operations. run_sync() is a bridge: it lets Alembic run its synchronous migration functions on the underlying async connection through SQLAlchemy's async integration. It doesn't mean your app switched to a synchronous PostgreSQL driver.

📝 This is a simplified online-migration setup. Production Alembic environments may add offline migrations, logging, URL loading and transaction handling.

🚫 What is `pool.NullPool`?

NullPool doesn't keep a pool of reusable connections. For a migration command that runs and exits, that's sensible: it opens a connection when needed and closes it when finished. Your FastAPI app can still use its normal connection pool.

🏛️ PART 10: The Repository Pattern

30. 🤔 What is the Repository Pattern?

A user clicks Login in SocialFeed and your backend must find them in PostgreSQL. Where should that query live? You could write it directly in the route:

@app.get("/users/{user_id}")
async def get_user(
    user_id: int,
    db: AsyncSession = Depends(get_db),
):
    result = await db.execute(
        select(User).where(User.id == user_id)
    )

    user = result.scalar_one_or_none()

    return user

That works. But with 50 endpoints, many needing to retrieve users, you'd repeat it, which causes:

🔁 Duplicate code
😩 Difficult maintenance
🧶 Database logic mixed with API logic
🧪 Difficult testing
♻️ Harder code reuse

📌 A repository is a layer responsible for communicating with the database and performing data-access operations. Think of it as the database specialist in your backend.

❌ Without the Repository Pattern
FastAPI Route: SQL queries + business logic + response handling, all mixed together
        ▼
🐘 PostgreSQL

✅ With the Repository Pattern
🛣️ FastAPI Route     → receives the HTTP request
        ▼
🧠 Service Layer     → business rules and application logic
        ▼
📚 Repository Layer  → database queries and data access
        ▼
🐘 PostgreSQL        → stores data

🛣️ Layer 1: API / Router

Receives HTTP requests, validates path/query parameters, obtains dependencies, calls service functions and returns HTTP responses. It shouldn't contain complicated database logic.

🧠 Layer 2: Service

Contains business logic: check whether a user exists, check permissions, enforce application rules, coordinate multiple repositories, decide what happens next.

async def get_user_profile(user_id: int, db):
    user = await user_repository.get_by_id(db, user_id)

    if user is None:
        raise UserNotFoundError()

    return user

Deciding how the app behaves when a user doesn't exist is a business/application-level responsibility.

📚 Layer 3: Repository

Handles database access:

async def get_by_id(db: AsyncSession, user_id: int):
    result = await db.execute(
        select(User).where(User.id == user_id)
    )

    return result.scalar_one_or_none()

It only retrieves a user. It doesn't decide the HTTP status code, how to authenticate someone, or what message to show.

31. 🧵 A Complete Repository Example

app/
│
├── api/
│   └── user_routes.py
│
├── services/
│   └── user_service.py
│
├── repositories/
│   └── user_repository.py
│
├── models/
│   └── user.py
│
└── database.py

Step 1: repositories/user_repository.py

from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession

from app.models.user import User


async def get_by_id(db: AsyncSession, user_id: int):
    result = await db.execute(
        select(User).where(User.id == user_id)
    )

    return result.scalar_one_or_none()


async def get_by_email(db: AsyncSession, email: str):
    result = await db.execute(
        select(User).where(User.email == email)
    )

    return result.scalar_one_or_none()


async def create(db: AsyncSession, username: str, email: str):
    user = User(
        username=username,
        email=email,
    )

    db.add(user)

    await db.flush()

    return user

get_by_id() retrieves a user by ID
get_by_email() retrieves a user by email
create() creates a new User object and adds it to the Session

Notice await db.flush(). Why not commit here? Because the repository doesn't necessarily own the transaction. The service (or a higher-level unit of work) may need several operations together: create a user, create their profile, create their default settings. They might need to succeed or fail together.

Step 2: services/user_service.py

from app.repositories import user_repository


async def get_user_profile(db, user_id: int):
    user = await user_repository.get_by_id(db, user_id)

    if user is None:
        return None

    return user


async def register_user(db, username: str, email: str):
    existing_user = await user_repository.get_by_email(db, email)

    if existing_user is not None:
        raise ValueError("Email already registered")

    user = await user_repository.create(db, username, email)

    await db.commit()

    return user

The service checks whether the email exists, decides whether registration proceeds, calls the repository, and commits.

⚠️ This is a teaching example. In production, the database unique constraint must still enforce email uniqueness, because two simultaneous registrations can race. You'd catch the resulting IntegrityError, roll back, and translate it into an appropriate application error.

Step 3: api/user_routes.py

from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy.ext.asyncio import AsyncSession

from app.database import get_db
from app.services import user_service

router = APIRouter()


@router.get("/users/{user_id}")
async def get_user(
    user_id: int,
    db: AsyncSession = Depends(get_db),
):
    user = await user_service.get_user_profile(db, user_id)

    if user is None:
        raise HTTPException(
            status_code=404,
            detail="User not found",
        )

    return user

The route doesn't know how the SQL query works. It asks the service for a user. That's clean separation of responsibilities. ✨

32. 🤔 Why Shouldn't Repositories Usually Commit?

Your app performs three operations: create_user(), create_profile(), create_settings(). If each repository function commits on its own and:

✅ User creation succeeds
✅ Profile creation succeeds
❌ Settings creation fails

...you've got a partially completed registration. Use one transaction instead:

async with db.begin():
    user = await user_repository.create(
        db,
        "deepika",
        "deepika@gmail.com",
    )

    await profile_repository.create(db, user.id)
    await settings_repository.create(db, user.id)

async with db.begin() manages a transaction:

If the block completes successfully, the transaction commits
If an exception occurs, the transaction is rolled back

That makes several related operations atomic: all succeed together, or the transaction is rolled back.

📝 db.begin() expects the Session not to have already begun a transaction (for example, from an earlier query on the same Session). The exact transaction boundary is an architectural decision. The key is clear ownership, rather than committing randomly inside every repository method.

🔎 PART 11: Important SQLAlchemy Query Methods

33. 🆚 `execute()` vs `scalars()`

You'll see these constantly in SQLAlchemy 2.0.

statement = select(User)
result = await db.execute(statement)

execute() returns a SQLAlchemy Result object containing rows from the query.

🅰️ result.scalars() extracts the first selected element of each row. Since we selected User, it returns User ORM objects:

users = result.scalars().all()

🅱️ result.scalar_one_or_none() is for when you expect zero or one result:

No matching result → None
Exactly one result → that value
More than one → raises MultipleResultsFound

🅲 result.scalar_one() expects exactly one:

No result → raises NoResultFound
One result → returns it
Multiple → raises MultipleResultsFound

🅳 result.scalars().first() returns the first result or None, and doesn't enforce uniqueness.

📋 Quick comparison

Method	Expected result
`all()`	All rows
`first()`	First row or `None`
`scalar_one()`	Exactly one scalar
`scalar_one_or_none()`	Zero or one scalar
`scalars().all()`	All first-column values
`scalars().first()`	First scalar value

🔑 What is `session.get()`?

user = await db.get(User, 1)

Retrieves a User by primary key. It's especially convenient for primary-key lookups, and it can use the Session's identity map, possibly avoiding another SQL query if the object is already present and valid in that Session.

34. 🎛️ Filtering, Ordering and Limiting

Filtering

statement = select(User).where(User.username == "deepika")

SELECT * FROM users WHERE username = 'deepika';

Multiple conditions

from sqlalchemy import and_

statement = select(User).where(
    and_(
        User.is_active == True,
        User.username == "deepika",
    )
)

or simply:

statement = select(User).where(
    User.is_active.is_(True),
    User.username == "deepika",
)

Ordering

statement = select(User).order_by(User.username)
statement = select(User).order_by(User.id.desc())   # descending

Limiting

statement = select(User).limit(10)

SELECT * FROM users LIMIT 10;

Offset

statement = select(User).offset(10).limit(10)

That's the basic idea behind offset pagination. For a SocialFeed project, also learn cursor pagination, which is often preferable for large, changing feeds.

😴 PART 12: Relationships and Async Loading

35. 🐢 What is Lazy Loading?

user = await db.get(User, 1)

The User is retrieved. But what about user.posts? Are the posts automatically fetched? Not necessarily. The default relationship strategy is generally lazy="select", meaning related records load when the relationship is accessed. That's lazy loading.

⚠️ Traditional implicit lazy loading can cause problems in async SQLAlchemy, because accessing a relationship might require database I/O, and normal attribute access can't use await. So async apps should usually plan relationship loading explicitly.

🅰️ `selectinload()`

from sqlalchemy import select
from sqlalchemy.orm import selectinload

statement = select(User).options(selectinload(User.posts))

result = await db.execute(statement)
users = result.scalars().all()

SQLAlchemy retrieves the users
It retrieves their related posts with an additional query, generally using an IN condition
It populates the posts relationships

Now you can access the already-loaded relationships without triggering implicit lazy loading.

🅱️ `joinedload()`

from sqlalchemy.orm import joinedload

statement = select(User).options(joinedload(User.posts))

Loads related objects using a SQL JOIN. When loading collections this way, you generally need:

result = await db.execute(statement)
users = result.unique().scalars().all()

Why unique()? Joining a user with multiple posts can produce repeated rows for the same user in the SQL result, and unique() deduplicates ORM identities.

⚖️ `selectinload()` vs `joinedload()`

Feature	`selectinload`	`joinedload`
Query strategy	Separate SELECT	SQL JOIN
Collection loading	Often efficient	Can multiply result rows
Main advantage	Avoids large joined result sets	Related data in one SQL statement
Typical use	One-to-many collections	Many-to-one or small related loads

💡 Neither is universally better. Choose based on the query and the data size.

🧠 PART 13: Important Database Concepts

36. 🔁 What is a Transaction?

📌 A transaction is a group of database operations treated as one logical unit of work.

A user transfers ₹500 from Account A to Account B:

Deduct ₹500 from A
Add ₹500 to B

What if the first succeeds but the second fails? Money would disappear! Transactions prevent partial updates: both commit, or both roll back. SQLAlchemy manages transactions through Session methods and transaction context managers.

37. 🧪 What is ACID?

Property	Meaning	Example
Atomicity	All or nothing	Both transfer operations succeed or neither does
Consistency	Valid database rules remain satisfied	Foreign keys and constraints are respected
Isolation	Concurrent transactions are controlled	One transaction shouldn't see another's uncommitted changes under normal isolation
Durability	Committed changes persist	A successful transfer remains stored

38. 🚧 What is a Constraint?

📌 A constraint is a database rule that controls what data can be stored.

# Primary key
id: Mapped[int] = mapped_column(primary_key=True)

# Unique constraint
email: Mapped[str] = mapped_column(String(100), unique=True)

# Not null
username: Mapped[str] = mapped_column(String(50), nullable=False)

# Foreign key
user_id: Mapped[int] = mapped_column(ForeignKey("users.id"))

Why do constraints matter? Application-level validation alone can't guarantee database consistency. Two concurrent requests could both check whether an email exists, both find it doesn't, and both try to insert it. A unique database constraint ensures only one succeeds.

39. ⚡ What is an Index?

Your users table has 10 million records and you run:

SELECT * FROM users WHERE email = 'deepika@gmail.com';

Without an appropriate index, PostgreSQL may need to scan many rows.

📌 An index is a data structure that helps the database find matching records efficiently.

email: Mapped[str] = mapped_column(
    String(100),
    unique=True,
    index=True,
)

Or at table level:

from sqlalchemy import Index


class User(Base):
    __tablename__ = "users"

    # columns ...

    __table_args__ = (
        Index("ix_users_username", "username"),
    )

✅ Indexes can speed up certain reads
💾 They consume storage
🐌 They add overhead to inserts, updates and deletes
🎯 Not every column needs one
A unique constraint commonly creates a supporting unique index in PostgreSQL

⚠️ PART 14: Common Mistakes and Best Practices

40. ❌ Common SQLAlchemy Mistakes

Mistake	Correct approach
Sharing one `AsyncSession` between concurrent tasks	Use separate sessions
Forgetting `await` on async database calls	Await asynchronous operations
Forgetting to commit	Commit at the intended transaction boundary
Assuming `flush()` commits	Flush only sends SQL; it doesn't commit
Forgetting to roll back after a transaction failure	Roll back or use transaction context managers
Putting all queries in routers	Use repositories
Assuming model changes update PostgreSQL	Generate and apply migrations
Blindly applying autogenerated migrations	Review them first
Accessing unloaded relationships in async code	Use explicit loading strategies
Storing DB passwords in Git	Use environment configuration

41. 💥 What Happens When a Database Operation Fails?

You insert two users with the same email. PostgreSQL rejects the second insert because of the unique constraint, and SQLAlchemy raises an exception, commonly an IntegrityError. After a failed flush or commit, the Session must be rolled back before it can be used again.

from sqlalchemy.exc import IntegrityError

try:
    db.add(user)
    await db.commit()
except IntegrityError:
    await db.rollback()
    raise

A failed transaction leaves the Session in a failed transaction state until you roll back. Also, don't catch every exception and silently ignore it. Translate expected database errors into meaningful application errors and let unexpected ones be handled appropriately.

42. ⚖️ SQLAlchemy vs Alembic

SQLAlchemy	Alembic
Database toolkit and ORM	Database migration tool
Defines ORM models	Tracks schema changes
Executes queries	Applies migration operations
Creates, reads, updates and deletes records	Adds/removes tables and columns
Manages Sessions and transactions	Tracks schema revision history
Used by application code	Used during database deployment and maintenance

🧠 SQLAlchemy manages data operations. Alembic manages schema evolution. They work together but have different responsibilities.

🧵 PART 15: Putting Everything Together

43. 🛠️ A Complete Real-World Workflow

Your mentor asks you to add a Post table. Here's the full workflow:

1️⃣ Define the SQLAlchemy model in models/post.py

class Post(Base):
    __tablename__ = "posts"

    id: Mapped[int] = mapped_column(primary_key=True)

2️⃣ Register the model. Import it so its table is included in Base.metadata

3️⃣ Generate the migration

alembic revision --autogenerate -m "create posts table"

4️⃣ Review the migration. Inspect the generated upgrade() and downgrade() functions

5️⃣ Apply the migration

alembic upgrade head

PostgreSQL now has the new table. ✅

6️⃣ Create repository functions: create_post(), get_post_by_id(), delete_post()

7️⃣ Create service functions with business rules such as checking ownership and validating permissions

8️⃣ Create FastAPI endpoints

@router.post("/posts")
async def create_post(...):
    ...

9️⃣ Test the functionality. Verify the endpoint works and the data is correctly stored in PostgreSQL

44. 🗺️ How the Whole Architecture Connects

Component	Responsibility
⚡ FastAPI	Handles HTTP requests
✔️ Pydantic	Validates request and response data
🧱 SQLAlchemy ORM	Maps Python objects to database tables
🔧 SQLAlchemy Core	Provides SQL expression tools
🔌 Engine	Manages database connectivity
🗂️ AsyncSession	Manages asynchronous ORM operations and transactions
🚀 asyncpg	Communicates asynchronously with PostgreSQL
📚 Repository	Performs database access
🧠 Service	Implements business rules
🔄 Alembic	Changes the database schema over time
🐘 PostgreSQL	Permanently stores relational data

🎤 PART 16: Interview Questions

1. What is SQLAlchemy?
A Python SQL toolkit and ORM.

2. What's the main difference between Core and ORM?
Core works with tables and SQL expressions. The ORM works with mapped Python classes and objects.

3. What does flush() do?
It sends pending SQL changes to the database within the current transaction, without committing.

4. What is AsyncSession?
A session for asynchronous ORM database operations, used with an async engine and driver such as asyncpg.

5. What is Alembic used for?
Managing database schema migrations over time.

6. What does --autogenerate do?
It compares your model metadata with the actual database and proposes a migration. Always review the result.

7. What does alembic upgrade head do?
It applies all pending migrations up to the latest revision.

8. What is the Repository Pattern?
A layer that encapsulates database access, keeping queries out of routes and business logic.

9. What does target_metadata = Base.metadata do?
It tells Alembic which SQLAlchemy model metadata to compare against the database.

10. Can one AsyncSession be safely shared by concurrent asyncio tasks?
No. Each concurrent task should use its own AsyncSession.

🧪 PART 17: Test Yourself!

Try these before peeking at the answers! ❤️

What is SQLAlchemy?
- A) A Python SQL toolkit and ORM
- B) A database server
- C) A Python web framework
What's the main difference between Core and ORM?
- A) Core uses tables and SQL expressions; ORM uses mapped Python objects
- B) Core is asynchronous; ORM is synchronous
- C) They're completely unrelated tools
What does flush() do?
- A) Permanently commits changes
- B) Sends pending SQL changes within the transaction without committing
- C) Closes the database connection
What is AsyncSession?
- A) A session for asynchronous database operations
- B) A PostgreSQL table
- C) An Alembic migration
What is Alembic used for?
- A) Validating HTTP request bodies
- B) Managing database schema migrations
- C) Creating FastAPI routes
What does --autogenerate do?
- A) Automatically deploys the database
- B) Compares model metadata with the database and proposes a migration
- C) Deletes old migrations
What does alembic upgrade head do?
- A) Applies pending migrations to the latest revision
- B) Deletes all tables
- C) Creates a new SQLAlchemy model
What is the Repository Pattern?
- A) A way to organize Git repositories
- B) A layer that encapsulates database access
- C) A database indexing technique
What does target_metadata = Base.metadata do?
- A) Tells Alembic which SQLAlchemy model metadata to compare
- B) Creates a PostgreSQL database
- C) Commits a transaction
Can one AsyncSession be safely shared by concurrent asyncio tasks?
- A) Yes, always
- B) No, each concurrent task should use its own AsyncSession
- C) Only when using PostgreSQL

👀 Click to reveal answers

A: SQLAlchemy is a Python SQL toolkit and ORM
A: Core is SQL-expression oriented; the ORM maps Python objects
B: flush() sends SQL in the current transaction but doesn't commit
A: AsyncSession handles asynchronous ORM operations
B: Alembic manages schema migrations
B: Autogenerate compares metadata with the database and proposes changes
A: It applies pending migrations up to the latest revision
B: A repository encapsulates database access
A: It gives Alembic the model metadata to compare against the database
B: Sessions are stateful, so use one per concurrent task

🗺️ PART 18: What to Learn Next

Work through this checklist in order:

[ ] Understand SQLAlchemy Core vs ORM
[ ] Create DeclarativeBase and ORM models
[ ] Understand Mapped, mapped_column and column types
[ ] Create primary keys, foreign keys and relationships
[ ] Understand Engine and connection pooling
[ ] Master Session, flush, commit and rollback
[ ] Learn AsyncEngine, AsyncSession and asyncpg
[ ] Learn select(), execute(), scalars() and filtering
[ ] Understand relationship loading strategies
[ ] Create repository and service layers
[ ] Initialize Alembic and configure target_metadata
[ ] Generate and review autogenerated migrations
[ ] Apply upgrades and perform downgrades
[ ] Handle database transactions and IntegrityError

🏁 The 10 Most Important Things to Remember

🧱 SQLAlchemy is a Python SQL toolkit and ORM
🪄 The ORM maps Python classes to database tables
📐 Declarative models define your tables using Python classes
🔧 Core works with SQL expressions and table definitions
🗂️ Session / AsyncSession manage ORM database interactions and transactions
🚀 asyncpg is an asynchronous PostgreSQL driver
🚿 flush() executes pending SQL without committing
🔄 Alembic manages database schema changes through migrations
🤖 Autogenerate proposes migrations by comparing metadata with the actual database
📚 The Repository Pattern separates database access from business logic and API routes

🎯 One final distinction: if you add a bio column to your SQLAlchemy User model, PostgreSQL doesn't automatically get that column. You need to generate and apply an Alembic migration. And when you want to retrieve a user from PostgreSQL, SQLAlchemy's Session or AsyncSession executes the query. These are two different responsibilities.

🚀 What's Next?

Add a Post model to a small project, generate the migration with --autogenerate, read it before applying it, run alembic upgrade head, and then build the repository, service and route on top. Hands-on practice is where it all clicks! 💡

If this post helped you, drop a ❤️ or 🦄 and share it with a friend learning backend development. Happy coding! 👩‍💻✨

Pydantic v2 Deep Dive & Custom Validators: Explained from Absolute Zero 🛡️

Deepika Pusala — Wed, 11 Nov 2026 07:00:03 +0000

🛡️ Pydantic v2 Deep Dive & Custom Validators: Let's Learn from ZERO!

Almost every API you build needs to validate incoming data, reject invalid values, convert data into Python objects, and control what goes back to the frontend. In FastAPI, Pydantic does all of that. 🚀

No Pydantic knowledge is assumed. We'll connect every concept to a React + FastAPI social-feed backend so you see how it works in a real project, not just as definitions.

1. 🤔 What problem does Pydantic solve?

Your React frontend sends this to your FastAPI backend:

{
  "username": "deepika",
  "age": 22,
  "email": "deepika@gmail.com"
}

But what if it accidentally sends this?

{
  "username": 12345,
  "age": "twenty-two",
  "email": "not-an-email"
}

❌ Username is a number instead of a string
❌ Age is text instead of a number
❌ Email is not a valid email address

Should your backend blindly accept it? Absolutely not! 🙅

📌 Pydantic is a Python library for data validation, parsing, and serialization using Python type annotations. In simple words: it checks whether data follows the structure and rules you defined.

Think of Pydantic as a security checkpoint ✈️ at the entrance of your backend:

✅ Correct data → allowed through
❌ Incorrect data → validation error
🔄 Data needing conversion → converted when the rules permit

🤝 Why do we need it in FastAPI?

FastAPI integrates directly with Pydantic. When a request reaches an endpoint, FastAPI can use a Pydantic model to:

Read incoming JSON
Validate its fields
Convert compatible values into Python types when allowed
Give your endpoint a validated Python object
Generate API documentation automatically
Validate and serialize outgoing responses

2. 🏗️ What is a Pydantic model?

First, a normal Python class:

class User:
    def __init__(self, username, age):
        self.username = username
        self.age = age


user = User("deepika", 22)

print(user.username)  # deepika
print(user.age)       # 22

But Python doesn't validate the types here:

user = User(12345, "twenty-two")  # Python happily creates this 😬

Now with Pydantic:

pip install pydantic

from pydantic import BaseModel


class User(BaseModel):
    username: str
    age: int


user = User(username="deepika", age=22)

print(user)

username='deepika' age=22

BaseModel is the main Pydantic class that provides validation and serialization
username: str and age: int are type annotations. They tell Pydantic what to expect

Now try User(username=12345, age="twenty-two") and Pydantic raises a ValidationError. 💥

📌 A Pydantic model is a Python class that inherits from BaseModel and defines the expected structure and validation rules of data.

Incoming Python data / JSON
        ▼
Pydantic BaseModel   → checks fields, types and validation rules
        ├── ✅ Valid data   → Python model object
        └── ❌ Invalid data → ValidationError

3. 🆕 Pydantic v2: What's New and Important?

Pydantic has two major versions: v1 (older API) and v2 (newer implementation, faster, with updated APIs). Focus on v2.

Pydantic v1	Pydantic v2
`@validator`	`@field_validator`
`@root_validator`	`@model_validator`
`.dict()`	`.model_dump()`
`.json()`	`.model_dump_json()`
`.parse_obj()`	`.model_validate()`
`.schema()`	`.model_json_schema()`
`.copy()`	`.model_copy()`

user.dict()          # old
user.model_dump()    # new ✅

💡 In new projects, learn and use the v2 APIs.

4. 🎚️ Required Fields, Optional Fields and Defaults

Not every field in an API request is mandatory. In a registration API, username, email and password are required, but a profile bio might be optional.

from pydantic import BaseModel


class UserCreate(BaseModel):
    username: str
    email: str
    password: str
    bio: str | None = None

username: str → required
bio: str | None = None → can be a string, can be None, and defaults to None if omitted

user = UserCreate(
    username="deepika",
    email="deepika@gmail.com",
    password="secret123",
)

print(user.bio)  # None

⚠️ Optional vs default value (a classic interview question!)

class User(BaseModel):
    bio: str | None        # value may be None, but the field is STILL REQUIRED

class User(BaseModel):
    bio: str | None = None  # the user can omit it ✅

In the first version, the caller must explicitly supply bio, even if the value is None.

🧮 Default values

class User(BaseModel):
    username: str
    is_active: bool = True
    followers_count: int = 0

🏭 Default factories

Need a new list for every object?

from pydantic import BaseModel, Field


class User(BaseModel):
    username: str
    interests: list[str] = Field(default_factory=list)

Field(default_factory=list) creates a fresh empty list whenever a default is needed, so objects never accidentally share the same mutable list. You can also use it for timestamps:

from datetime import datetime, timezone

created_at: datetime = Field(
    default_factory=lambda: datetime.now(timezone.utc)
)

Each object gets its own creation timestamp. ⏱️

5. 📏 `Field()`: Adding Validation Constraints

Imagine someone sends {"username": "", "age": -10}. Technically the username is a string and the age is an integer, but the values make no sense! We need extra rules. That's what Field() is for.

from pydantic import BaseModel, Field


class User(BaseModel):
    username: str = Field(min_length=3, max_length=20)
    age: int = Field(ge=18, le=100)

🔤 String constraints

Constraint	Meaning
`min_length=3`	Minimum 3 characters
`max_length=20`	Maximum 20 characters
`pattern="..."`	Must match a regular expression

username: str = Field(
    min_length=3,
    max_length=20,
    pattern=r"^[a-zA-Z0-9_]+$",
)

This means 3 to 20 characters, using only letters, numbers and underscores.

🔢 Numeric constraints

Constraint	Meaning
`gt=0`	Greater than zero
`ge=18`	Greater than or equal to 18
`lt=100`	Less than 100
`le=100`	Less than or equal to 100
`multiple_of=5`	Must be divisible by 5

price: float = Field(gt=0)        # must be greater than zero
age: int = Field(ge=18, le=100)   # between 18 and 100, inclusive

📚 Collection constraints

tags: list[str] = Field(min_length=1, max_length=5)

The list needs at least 1 item and at most 5. Because of list[str], Pydantic also checks that every item is a string.

🧠 Remember: Field() defines declarative constraints. Validators define custom logic. min_length=3 is a constraint, but "username must not contain a prohibited word" needs custom code.

6. 🧪 What are Custom Validators?

In your social-feed registration API you want to enforce:

Username must not contain spaces
Email must be lowercase
Username must not contain offensive words

Field() can't express all of that. We need Python logic.

📌 A custom validator is a function you write to check whether a value (or a whole model) satisfies your application's rules.

Pydantic v2 gives you two decorators:

@field_validator: validates one field
@model_validator: validates the whole model

7. 🎯 `@field_validator`: Validating Individual Fields

from pydantic import BaseModel, field_validator


class UserCreate(BaseModel):
    username: str

    @field_validator("username")
    @classmethod
    def validate_username(cls, value):
        if " " in value:
            raise ValueError("Username cannot contain spaces")

        return value

Line by line:

@field_validator("username") → "run this function when validating the username field"
@classmethod → in Pydantic v2, field validators generally take cls as the first parameter
def validate_username(cls, value) → value is the value being validated (e.g. "deepika")
if " " in value → our check
raise ValueError(...) → rejects the value. Pydantic catches it and produces a structured ValidationError
return value → allows the value to continue. A field validator must return the value it wants Pydantic to use

You can validate several fields with one validator:

@field_validator("username", "display_name")
@classmethod
def validate_names(cls, value):
    return value.strip()

Testing it:

UserCreate(username="deepika")          # ✅ works
UserCreate(username="deepika rao")      # ❌ ValidationError

💡 raise ValueError() rejects the input. return value lets the validated value continue.

8. 🔀 Field Validator Modes: Before, After, Plain, Wrap

@field_validator supports four modes:

Mode	When does it run?
`before`	Before Pydantic's normal type validation
`after`	After normal type validation
`plain`	Replaces normal validation
`wrap`	Wraps normal validation and controls when it runs

🅰️ `before`: clean raw input

The frontend sends " deepika " and you want to strip the spaces first:

from pydantic import BaseModel, field_validator


class User(BaseModel):
    username: str

    @field_validator("username", mode="before")
    @classmethod
    def clean_username(cls, value):
        if isinstance(value, str):
            return value.strip()

        return value

Incoming value: "   deepika   "
        ▼
Before validator → strip() removes extra spaces
        ▼
Pydantic type validation → checks the result is a string
        ▼
Final value: "deepika"

Why isinstance(value, str)? Because before validation, Pydantic hasn't checked the input yet. It could be a string, integer, list or anything. A before validator must be ready for raw input.

Use before to clean, normalize or preprocess incoming data.

🅱️ `after`: check validated values

class User(BaseModel):
    age: int

    @field_validator("age", mode="after")
    @classmethod
    def validate_age(cls, value):
        if value < 18:
            raise ValueError("User must be at least 18")

        return value

For User(age="22"), in default (non-strict) mode Pydantic converts "22" to 22 first, and then your validator receives 22 and checks the business rule.

Use after for values that have already passed type validation. It's the easiest and safest mode for beginners. ✅

🅲 `plain`: take full control

A plain validator replaces Pydantic's normal validation:

from pydantic import BaseModel, field_validator


class Product(BaseModel):
    quantity: int

    @field_validator("quantity", mode="plain")
    @classmethod
    def validate_quantity(cls, value):
        if isinstance(value, int) and value > 0:
            return value

        raise ValueError("Quantity must be a positive integer")

Even though the field says quantity: int, Pydantic won't run its normal integer validation after this. Your function is responsible for checking the input. Use plain only when you deliberately want to replace normal validation.

🅳 `wrap`: advanced control

A wrap validator receives the incoming value and a handler that runs Pydantic's normal validation. You can run logic before it, after it, or catch errors:

from pydantic import BaseModel, ValidatorFunctionWrapHandler, field_validator


class Product(BaseModel):
    quantity: int

    @field_validator("quantity", mode="wrap")
    @classmethod
    def validate_quantity(cls, value, handler: ValidatorFunctionWrapHandler):
        print("Before normal validation")

        result = handler(value)

        print("After normal validation")

        return result

For Product(quantity="10"):

Before normal validation
        ▼
handler(value) → Pydantic validates and converts "10" to 10
        ▼
After normal validation
        ▼
Return result

If the input is invalid, the handler raises a validation error, which you can catch to implement fallback behavior.

📊 All four modes at a glance

Mode	Idea	Example
`before`	Preprocess first, then normal validation	Remove extra spaces
`after`	Normal validation first, then custom logic	Age must be at least 18
`plain`	Custom validator replaces normal validation	Implement your own full validation
`wrap`	Custom logic controls normal validation	Intercept errors, fallback behavior

💡 Recommendation: become very comfortable with before and after first, then learn plain and wrap.

9. 🧩 `@model_validator`: Validating Multiple Fields Together

A registration form has password and confirm_password:

{
  "password": "hello123",
  "confirm_password": "hello456"
}

Individually both are valid strings. Together they violate a rule: they must match. A field validator can't conveniently solve this. Use @model_validator, which validates the entire model.

from pydantic import BaseModel, model_validator


class UserRegister(BaseModel):
    username: str
    password: str
    confirm_password: str

    @model_validator(mode="after")
    def check_passwords(self):
        if self.password != self.confirm_password:
            raise ValueError("Passwords do not match")

        return self

@field_validator("username") validates one specific field
@model_validator(mode="after") validates the complete model
An after model validator receives the complete model instance, so you access self.password and self.confirm_password
return self returns the validated model instance

⏪ `before` model validators

A before model validator runs before field validation and receives the raw input:

from typing import Any

from pydantic import BaseModel, model_validator


class User(BaseModel):
    username: str
    email: str

    @model_validator(mode="before")
    @classmethod
    def preprocess_data(cls, data: Any):
        if isinstance(data, dict):
            data = data.copy()

            if "user_name" in data and "username" not in data:
                data["username"] = data.pop("user_name")

        return data

If someone sends {"user_name": "deepika", "email": "deepika@gmail.com"}, the validator turns it into {"username": "deepika", ...} and then normal field validation happens. Notice we copy the dictionary instead of modifying the original input.

📊 Model validator modes

Mode	Behavior
`before`	Receives raw input before field validation
`after`	Receives the validated model instance
`wrap`	Can control and wrap the entire model validation process

📝 Unlike field validators, model validators don't have a plain mode.

⚔️ Field validator vs model validator

Feature	Field validator	Model validator
Main purpose	Validate individual fields	Validate relationships between fields
Decorator	`@field_validator`	`@model_validator`
Example	Username cannot contain spaces	Passwords must match
Can access whole model?	Not directly in the normal way	Yes, in `after` mode
Modes	before, after, plain, wrap	before, after, wrap

10. ℹ️ `ValidationInfo`: Accessing Other Field Values

Sometimes you validate one field based on another. For example, reject a display_name identical to the username:

from pydantic import BaseModel, ValidationInfo, field_validator


class User(BaseModel):
    username: str
    display_name: str

    @field_validator("display_name")
    @classmethod
    def validate_display_name(cls, value, info: ValidationInfo):
        username = info.data.get("username")

        if username and value == username:
            raise ValueError("Display name cannot equal username")

        return value

info.data contains the previously validated fields.

⚠️ Field validation normally follows the order in which fields are declared. Here username comes before display_name, so its validated value is available. Reverse the order and username might not be available yet. For relationships between several fields, model_validator(mode="after") is often clearer.

11. 🌍 Custom Validation: A Real-World Example

Registration rules for your social-feed app:

Username has 3 to 20 characters
Username cannot contain spaces
Email must be lowercase
Age must be at least 18
Password has at least 8 characters
Confirm password must match password

from pydantic import (
    BaseModel,
    Field,
    field_validator,
    model_validator,
)


class UserRegister(BaseModel):
    username: str = Field(min_length=3, max_length=20)
    email: str
    age: int = Field(ge=18)
    password: str = Field(min_length=8)
    confirm_password: str

    @field_validator("username")
    @classmethod
    def validate_username(cls, value):
        if " " in value:
            raise ValueError("Username cannot contain spaces")

        return value.strip()

    @field_validator("email")
    @classmethod
    def validate_email(cls, value):
        value = value.strip().lower()

        if "@" not in value:
            raise ValueError("Invalid email")

        return value

    @model_validator(mode="after")
    def validate_passwords(self):
        if self.password != self.confirm_password:
            raise ValueError("Passwords do not match")

        return self

📝 This uses a simple email check to demonstrate custom logic. For production-grade email validation, use EmailStr with the email-validator package. And never store plain-text passwords. Hash them with an algorithm such as Argon2id.

🧭 Where should validation happen?

Not every business rule belongs in Pydantic:

Rule	Best place
Username must be a string	Pydantic
Username length must be 3 to 20	Pydantic
Password confirmation must match	Pydantic
Email must have valid format	Pydantic
Username already exists in database	Service / database
User has permission to delete a post	Authorization / service layer
Post belongs to the logged-in user	Service / database

Pydantic is for validating data structure and input rules. It shouldn't become responsible for database operations, authorization, or application workflows.

12. 🔄 Request Schemas vs Response Schemas

🚨 EXTREMELY IMPORTANT for FastAPI projects!

A user registers and the frontend sends:

{
  "username": "deepika",
  "email": "deepika@gmail.com",
  "password": "secret123"
}

Your database stores:

id	username	email	password_hash
1	deepika	deepika@gmail.com	`$argon2id$...`

Should the backend send that complete record back to React? Never! You must never expose password hashes to the frontend. That's why we separate request and response schemas.

🅰️ Request schema

What the client is allowed or expected to send:

from pydantic import BaseModel, Field


class UserCreate(BaseModel):
    username: str = Field(min_length=3, max_length=20)
    email: str
    password: str = Field(min_length=8)

🅱️ Response schema

What the backend is allowed to return:

from pydantic import BaseModel


class UserResponse(BaseModel):
    id: int
    username: str
    email: str

Notice there's no password field. That's intentional. 🔐

🤔 Why separate them?

Frontend → Backend:   UserCreate     (username, email, password)
        ▼
Backend processes and stores data
        ▼
Database:             id, username, email, password_hash
        ▼
Backend selects safe output fields
        ▼
Backend → Frontend:   UserResponse   (id, username, email)  ✅ no password hash

This gives you security, clear API contracts, better maintainability, independent validation for input and output, and control over exposed fields.

📚 Other common schema types

class UserCreate(BaseModel):
    username: str
    email: str
    password: str


class UserUpdate(BaseModel):
    username: str | None = None
    bio: str | None = None


class UserResponse(BaseModel):
    id: int
    username: str
    email: str
    bio: str | None = None


class UserLogin(BaseModel):
    email: str
    password: str

Schema	Responsibility
`UserCreate`	New user registration
`UserUpdate`	Updating profile details
`UserResponse`	Returning safe user information
`UserLogin`	Accepting login credentials

🧠 A database model and a Pydantic schema are not the same thing. A SQLAlchemy model represents database structure and behavior. A Pydantic model represents the structure and validation of data entering or leaving your API.

13. ⚡ FastAPI `response_model`: How Does It Work?

response_model tells FastAPI which schema should describe and validate the endpoint's output.

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()


class UserCreate(BaseModel):
    username: str
    password: str


class UserResponse(BaseModel):
    id: int
    username: str


@app.post("/users", response_model=UserResponse)
def create_user(user: UserCreate):
    return {
        "id": 1,
        "username": user.username,
        "password": user.password,
    }

The function returns a dict with the password, but UserResponse only has id and username. FastAPI validates the returned data against UserResponse and serializes the response according to it. The extra password field is not included in the normal serialized response:

{
  "id": 1,
  "username": "deepika"
}

✅ Why is `response_model` useful?

Validates outgoing response data
Serializes data into the expected representation
Filters fields that aren't part of the response schema
Generates accurate API documentation
Establishes a clear contract for the frontend

🚨 Security warning: don't rely on response filtering as your only security measure. Never pass secrets into response construction unnecessarily. Build safe response data explicitly and keep sensitive fields out of API-facing schemas.

🆚 Request body vs `response_model`

@app.post("/users", response_model=UserResponse)
def create_user(user: UserCreate):
    ...

user: UserCreate → defines the incoming request body
response_model=UserResponse → defines the outgoing response

🎛️ More `response_model` options

@app.get(
    "/users/{user_id}",
    response_model=UserResponse,
    response_model_exclude_unset=True,
)

Option	Meaning
`response_model_exclude_unset=True`	Excludes fields not explicitly set
`response_model_exclude_none=True`	Excludes fields whose value is `None`
`response_model_exclude_defaults=True`	Excludes values equal to their defaults
`response_model_include={...}`	Includes selected fields
`response_model_exclude={...}`	Excludes selected fields

💡 Prefer creating a dedicated response schema over relying heavily on include/exclude tricks.

14. 📤 Serialization: Python Objects → JSON

user = UserResponse(
    id=1,
    username="deepika",
    email="deepika@gmail.com",
)

This is a Python object, but React expects JSON. Converting it is serialization.

📌 Serialization means converting a Python object or structured data into a format that can be transmitted, stored, or consumed elsewhere.

🅰️ `model_dump()`: model → Python dict

user_data = user.model_dump()
print(user_data)

{"id": 1, "username": "deepika", "email": "deepika@gmail.com"}

It produces a Python dictionary, not a JSON string.

🅱️ `model_dump_json()`: model → JSON string

print(user.model_dump_json())

{"id":1,"username":"deepika","email":"deepika@gmail.com"}

🅲 `model_validate()`: Python data → validated model

user_data = {
    "id": 1,
    "username": "deepika",
    "email": "deepika@gmail.com",
}

user = UserResponse.model_validate(user_data)

🅳 `model_validate_json()`: JSON text → validated model

user_json = """
{
    "id": 1,
    "username": "deepika",
    "email": "deepika@gmail.com"
}
"""

user = UserResponse.model_validate_json(user_json)

📊 Quick comparison

Method	Purpose
`model_dump()`	Model → Python dictionary
`model_dump_json()`	Model → JSON string
`model_validate()`	Python data → validated model
`model_validate_json()`	JSON string/bytes → validated model
`model_copy()`	Create a copy of a model
`model_json_schema()`	Generate JSON Schema

🎛️ Serialization options

user.model_dump(exclude={"email"})
# {"id": 1, "username": "deepika"}

user.model_dump(exclude_none=True)    # skip fields whose value is None
user.model_dump(exclude_unset=True)   # skip fields not explicitly supplied/set

⚠️ exclude_unset and exclude_defaults are different. A field explicitly provided with its default value is still considered set.

15. 🏷️ Aliasing: Different Names in Python and JSON

Your Python backend uses snake_case (created_at), but your React frontend uses camelCase (createdAt). Both mean the same thing. Do you rewrite your Python codebase in camelCase? No! Use aliases.

📌 An alias is an alternative name for a field, used when validating input or serializing output.

🅰️ Basic alias

from pydantic import BaseModel, Field


class Post(BaseModel):
    created_at: str = Field(alias="createdAt")


post = Post(createdAt="2026-11-11")

print(post.created_at)  # 2026-11-11

Python still uses created_at, but incoming data used createdAt.

🅱️ `alias` vs `validation_alias` vs `serialization_alias`

Property	Purpose
`alias`	Alias for both validation and serialization
`validation_alias`	Alternative name when accepting input
`serialization_alias`	Alternative name when producing output

validation_alias: the frontend sends {"userName": "deepika"} but your field is username:

class User(BaseModel):
    username: str = Field(validation_alias="userName")

serialization_alias: you want created_at to appear as createdAt in outgoing JSON:

class Post(BaseModel):
    created_at: str = Field(serialization_alias="createdAt")


post = Post(created_at="2026-11-11")

print(post.model_dump(by_alias=True))
# {"createdAt": "2026-11-11"}

print(post.model_dump())
# {"created_at": "2026-11-11"}

by_alias=True tells Pydantic to use the serialization alias instead of the Python field name.

🅲 Different input and output names

class User(BaseModel):
    username: str = Field(
        validation_alias="userName",
        serialization_alias="displayName",
    )


user = User(userName="deepika")

print(user.model_dump(by_alias=True))
# {"displayName": "deepika"}

Incoming JSON  →  userName
Python model   →  username
Outgoing JSON  →  displayName

🅳 Alias generators

With 50 fields (created_at, updated_at, first_name, ...) you don't want to write each alias by hand:

from pydantic import BaseModel, ConfigDict
from pydantic.alias_generators import to_camel


class User(BaseModel):
    model_config = ConfigDict(
        alias_generator=to_camel,
        validate_by_alias=True,
        validate_by_name=True,
    )

    first_name: str
    last_name: str
    created_at: str

Python field	Alias
`first_name`	`firstName`
`last_name`	`lastName`
`created_at`	`createdAt`

user.model_dump(by_alias=True)
# {"firstName": "Deepika", "lastName": "Rao", "createdAt": "2026-11-11"}

📝 validate_by_name and validate_by_alias are the granular settings in newer Pydantic v2 releases. Older v2 examples commonly use populate_by_name=True.

16. ⚙️ `ConfigDict`: Configuring Pydantic Models

Sometimes you want rules for the entire model: should extra fields be allowed? Should strings be auto-stripped? Should strict validation be on? Can the model be built from object attributes? Pydantic v2 uses ConfigDict:

from pydantic import BaseModel, ConfigDict


class User(BaseModel):
    model_config = ConfigDict(
        extra="forbid",
        str_strip_whitespace=True,
    )

    username: str
    email: str

🚫 Understanding `extra`

The frontend sends:

{
  "username": "deepika",
  "email": "deepika@gmail.com",
  "admin": true
}

But the model defines only username and email. What should happen?

Setting	Behavior
`extra="ignore"`	Ignore extra fields (default)
`extra="forbid"`	Reject extra fields
`extra="allow"`	Allow extra fields and retain them

For registration APIs, extra="forbid" is useful when you want unexpected input rejected rather than silently ignored.

🔧 Other useful settings

model_config = ConfigDict(strict=True)

Strict mode prevents many automatic type conversions. For example, an int field generally won't accept "22" in strict Python-object validation.

model_config = ConfigDict(from_attributes=True)

This lets Pydantic read data from object attributes, which is handy with ORM objects (more on this in section 21).

17. 🪆 Nested Pydantic Models

Real API responses are rarely flat. A post contains an ID, caption, author info and comments. Instead of one giant model, create smaller models and nest them:

from pydantic import BaseModel


class UserResponse(BaseModel):
    id: int
    username: str


class CommentResponse(BaseModel):
    id: int
    content: str
    author: UserResponse


class PostResponse(BaseModel):
    id: int
    caption: str
    author: UserResponse
    comments: list[CommentResponse]

Example response:

{
  "id": 101,
  "caption": "My first post!",
  "author": { "id": 1, "username": "deepika" },
  "comments": [
    {
      "id": 10,
      "content": "Beautiful!",
      "author": { "id": 2, "username": "sneha" }
    }
  ]
}

Pydantic validates the post, the author inside the post, every comment in the list, and the author inside each comment. This is nested model validation. 🎯

18. 🎨 Enums and `Literal` Values

A post's visibility should only be public, private or followers. You don't want the frontend sending "secret_everyone". Use an Enum:

from enum import Enum

from pydantic import BaseModel


class Visibility(str, Enum):
    PUBLIC = "public"
    PRIVATE = "private"
    FOLLOWERS = "followers"


class PostCreate(BaseModel):
    caption: str
    visibility: Visibility

PostCreate(caption="Hello", visibility="public")     # ✅
PostCreate(caption="Hello", visibility="everyone")   # ❌ rejected

For a small fixed set of strings you can use Literal:

from typing import Literal


class PostCreate(BaseModel):
    visibility: Literal["public", "private", "followers"]

19. 🚨 Pydantic Validation Errors

from pydantic import BaseModel, ValidationError


class User(BaseModel):
    username: str
    age: int


try:
    user = User(username="deepika", age="hello")
except ValidationError as error:
    print(error)

error.errors() returns structured information:

type: type of validation failure
loc: location of the invalid field
msg: human-readable message
input: the invalid input
sometimes ctx: additional error context

⚡ What happens in FastAPI?

@app.post("/users")
def create_user(user: UserCreate):
    return user

If the frontend sends invalid JSON data, FastAPI normally returns HTTP 422 Unprocessable Entity:

{
  "detail": [
    {
      "type": "greater_than_equal",
      "loc": ["body", "age"],
      "msg": "Input should be greater than or equal to 18"
    }
  ]
}

The exact structure depends on the failure. The important distinction:

❌ Request validation failure → normally HTTP 422
💥 Invalid data returned by your endpoint that violates response_model → server-side response validation failure, normally HTTP 500

Why 500? Because the server failed to produce the response promised by its own API contract.

20. 📦 `EmailStr`, `HttpUrl` and Other Built-in Types

from pydantic import BaseModel, EmailStr, HttpUrl


class User(BaseModel):
    username: str
    email: EmailStr
    profile_url: HttpUrl

To use EmailStr, install the email validation dependency:

pip install "pydantic[email]"

Type	Purpose
`EmailStr`	Email validation
`HttpUrl`	HTTP/HTTPS URL validation
`AnyUrl`	General URL validation
`IPvAnyAddress`	IPv4 or IPv6 address validation
`PositiveInt`	Integer greater than zero
`NonNegativeInt`	Integer greater than or equal to zero
`PositiveFloat`	Positive floating-point number

from pydantic import BaseModel, PositiveInt


class Post(BaseModel):
    id: PositiveInt
    likes_count: int = 0

💡 You don't need a custom validator for every simple rule. Use Pydantic's built-in types whenever they fit.

21. 🔌 `from_attributes`: Connecting Pydantic with SQLAlchemy

Especially important for a FastAPI + PostgreSQL project. Your SQLAlchemy model is a Python object with attributes (id, username, email), but a Pydantic model normally expects data. How can Pydantic read the SQLAlchemy object's attributes?

from pydantic import BaseModel, ConfigDict


class UserResponse(BaseModel):
    model_config = ConfigDict(from_attributes=True)

    id: int
    username: str
    email: str

response = UserResponse.model_validate(user_object)

Pydantic reads user_object.id, user_object.username, user_object.email and creates a validated Pydantic object.

🐘 PostgreSQL        → stored user record
        ▼
🗄️ SQLAlchemy ORM    → Python database object
        ▼
✔️ Pydantic model_validate()   (from_attributes=True)
        ▼
📤 UserResponse      → validated response object
        ▼
JSON response → ⚛️ React frontend

📌 from_attributes=True does not execute database queries. SQLAlchemy retrieves the object, and Pydantic reads its attributes.

22. 🗂️ Pydantic Schemas in a Real FastAPI Folder Structure

backend/
│
├── app/
│   ├── api/
│   │   └── routes/
│   │       └── users.py
│   │
│   ├── schemas/
│   │   ├── __init__.py
│   │   └── user.py
│   │
│   ├── models/
│   │   └── user.py
│   │
│   ├── services/
│   │   └── user_service.py
│   │
│   └── main.py
│
└── tests/
    └── test_user_schemas.py

Folder / file	Responsibility
`schemas/`	Pydantic request and response models
`models/`	SQLAlchemy database models
`api/routes/`	FastAPI endpoint definitions
`services/`	Application business logic
`tests/`	Tests for schemas and functionality

schemas/user.py

from pydantic import BaseModel, EmailStr, Field


class UserCreate(BaseModel):
    username: str = Field(min_length=3, max_length=20)
    email: EmailStr
    password: str = Field(min_length=8)


class UserResponse(BaseModel):
    id: int
    username: str
    email: EmailStr

api/routes/users.py

from fastapi import APIRouter

from app.schemas.user import UserCreate, UserResponse

router = APIRouter()


@router.post("/users", response_model=UserResponse)
def create_user(user: UserCreate):
    # Call service layer here

    return {
        "id": 1,
        "username": user.username,
        "email": user.email,
    }

Your route doesn't contain every validation rule. Pydantic schemas handle data validation, while services handle business logic.

🔍 What happens when the frontend calls this API?

React sends JSON
FastAPI receives the request
FastAPI validates the body using UserCreate
If validation fails, FastAPI returns a 422 error
If it succeeds, the endpoint receives a UserCreate object
The endpoint calls the service layer
The service performs application logic and database operations
The endpoint returns data
FastAPI validates and serializes it using UserResponse
React receives JSON

23. 🧪 Testing Pydantic Validators with pytest

Knowing how to write validators is good. Knowing how to test them is better.

from pydantic import BaseModel, Field, field_validator


class UserCreate(BaseModel):
    username: str = Field(min_length=3, max_length=20)

    @field_validator("username")
    @classmethod
    def validate_username(cls, value):
        if " " in value:
            raise ValueError("Spaces are not allowed")

        return value

tests/test_user_schema.py:

import pytest
from pydantic import ValidationError


def test_valid_username():
    user = UserCreate(username="deepika")

    assert user.username == "deepika"


def test_username_with_spaces():
    with pytest.raises(ValidationError):
        UserCreate(username="deepika rao")


def test_username_too_short():
    with pytest.raises(ValidationError):
        UserCreate(username="ab")

pytest.raises(ValidationError) means "we expect this code to raise a validation error." If it does, the test passes. If the code unexpectedly accepts invalid data, the test fails.

What to test:

✅ Valid input
❌ Missing required fields
❌ Incorrect types
📏 Minimum and maximum constraints
🧪 Custom validator failures
🧩 Cross-field validation
🏷️ Serialization and aliases

24. 🚀 Important Advanced Concepts

🅰️ Strict validation

By default, Pydantic tries to convert compatible values:

class User(BaseModel):
    age: int


User(age="22")   # "22" is converted into 22

To reject conversions:

class User(BaseModel):
    age: int = Field(strict=True)   # "22" is now rejected

Or for the whole model: model_config = ConfigDict(strict=True).

🅱️ Computed fields

Include a value calculated from other fields:

from pydantic import BaseModel, computed_field


class User(BaseModel):
    first_name: str
    last_name: str

    @computed_field
    @property
    def full_name(self) -> str:
        return f"{self.first_name} {self.last_name}"


user = User(first_name="Deepika", last_name="Rao")

print(user.full_name)  # Deepika Rao

user.model_dump() includes full_name in the serialized output.

🅲 JSON Schema

print(User.model_json_schema())

JSON Schema describes field names, types, required fields, constraints and descriptions. FastAPI uses this to generate the OpenAPI docs that power Swagger UI and ReDoc.

🅳 Custom error types

from pydantic_core import PydanticCustomError

raise PydanticCustomError(
    "username_taken",
    "Username '{username}' is already taken",
    {"username": "deepika"},
)

Useful when you want a specific, machine-readable validation error type.

🅴 Validation context

User.model_validate(
    {"username": "deepika"},
    context={"allowed_usernames": ["deepika", "sneha"]},
)

A validator can read this through info.context, which helps with dynamic rules. But rules requiring database queries should usually stay in your service layer.

25. 📋 Complete Pydantic v2 Cheat Sheet

Concept	What it does	Example
`BaseModel`	Creates Pydantic models	`class User(BaseModel)`
`Field()`	Defines field constraints	`Field(min_length=3)`
`field_validator`	Validates individual fields	`@field_validator("username")`
`model_validator`	Validates multiple fields together	`@model_validator(mode="after")`
`ValidationInfo`	Accesses validation information	`info.data`
`ConfigDict`	Configures model behavior	`extra="forbid"`
`EmailStr`	Validates email addresses	`email: EmailStr`
`from_attributes`	Reads attributes from objects	`from_attributes=True`
`validation_alias`	Changes accepted input field name	`Field(validation_alias="userName")`
`serialization_alias`	Changes serialized field name	`Field(serialization_alias="createdAt")`
`model_dump()`	Converts model to dictionary	`user.model_dump()`
`model_dump_json()`	Converts model to JSON string	`user.model_dump_json()`
`model_validate()`	Validates Python data	`User.model_validate(data)`
`model_validate_json()`	Validates JSON data	`User.model_validate_json(data)`
`model_copy()`	Copies a model	`user.model_copy()`
`computed_field`	Adds calculated serialized fields	`@computed_field`

26. 🎤 Common Interview Questions

1. What is Pydantic?
A Python library for data validation, parsing and serialization based on type annotations.

2. What's the difference between BaseModel and a normal Python class?
A normal class stores whatever you pass without checking types. A BaseModel validates (and where allowed, converts) data against its annotations and constraints, and provides serialization methods.

3. What's the difference between field_validator and model_validator?
field_validator validates individual fields. model_validator validates the whole model, so it can check relationships between fields, such as password and confirm password.

4. What's the difference between before and after validators?
before runs before Pydantic's type validation and receives raw input, which suits cleaning and normalizing. after runs after type validation and receives the already validated value, which suits business rules.

5. Why do we separate request and response schemas?
Request schemas validate what clients send. Response schemas control what the API returns, preventing sensitive fields such as password hashes from being exposed, and giving the frontend a clear contract.

6. What does response_model do in FastAPI?
It tells FastAPI which schema describes and validates the output. FastAPI validates and serializes the returned data against it, filters fields not in the schema, and documents the response in OpenAPI.

7. What is serialization?
Converting a Python object or model into a format that can be transmitted or stored, such as a dictionary (model_dump()) or a JSON string (model_dump_json()).

8. What's the difference between alias and serialization_alias?
alias applies to both validation (input) and serialization (output). serialization_alias only changes the field name in outgoing data, used with by_alias=True.

9. What is from_attributes=True?
A ConfigDict setting that lets Pydantic read values from object attributes, such as SQLAlchemy ORM objects, when using model_validate().

10. What's the difference between ValidationError and HTTP 422?
ValidationError is the Pydantic exception raised when validation fails. HTTP 422 is the response FastAPI sends to the client when request validation fails. A response-validation failure in your own endpoint is usually a server error (500).

27. 🏋️ Practical Exercises

Try these in a separate Python file:

Create a Product model with name, price and quantity. Price must be greater than zero
Create a username field_validator that removes extra spaces
Create an email validator that converts emails to lowercase
Create a registration model with password and confirm_password. Reject mismatches
Create UserCreate and UserResponse schemas. Never expose the password in the response
Create a Post model with a nested UserResponse author and a list of comments
Use validation_alias and serialization_alias to convert username into userName on input and displayName on output
Create a FastAPI POST endpoint using a request schema and response_model
Enable extra="forbid" and test what happens when an unknown field is submitted
Write pytest tests for valid and invalid usernames

28. 🏁 What to Remember

🏗️ Pydantic models define the structure of API data
📏 Field constraints enforce basic rules such as minimum lengths and positive numbers
🧪 Custom validators enforce additional input rules
📥 Request schemas define what the frontend can send
📤 Response schemas define what the backend returns
🔄 Serialization converts Python model data into dictionaries or JSON
🏷️ Aliases let Python and JavaScript follow different naming conventions
⚡ FastAPI integrates all of this into the request-response lifecycle
🗄️ SQLAlchemy handles database persistence; Pydantic handles API data validation and serialization
🧠 Services handle business logic that shouldn't live inside your schemas

🎯 One important architectural lesson: Pydantic validates whether data is structurally and semantically acceptable. It does not replace database constraints, authentication, authorization, or business logic.

🚀 What's Next?

Implement these concepts in a mini registration and post-creation API: write the schemas, add validators, wire up response_model, and cover it all with pytest. 💡

If this post helped you, drop a ❤️ or 🦄 and share it with a friend learning FastAPI. Happy coding! 👩‍💻✨

FastAPI Fundamentals & Dependency Injection: Explained from Absolute Zero ⚡

Deepika Pusala — Wed, 11 Nov 2026 06:05:45 +0000

⚡ FastAPI Fundamentals & Dependency Injection: Let's Learn from ZERO!

Ever wondered how FastAPI receives requests, validates data, calls your Python functions, connects to databases, handles authentication, and organizes a real-world backend? 🤔

In this post we'll learn it all from scratch, with simple examples, real-life analogies, Python code, and folder structures. No prior backend knowledge needed! 🚀

1. 🤔 What exactly is FastAPI?

Imagine you're building Instagram. Your app has:

⚛️ A React frontend
🐍 A Python FastAPI backend
🐘 A PostgreSQL database

A user clicks the Like button. What happens?

⚛️ Frontend (React)
   User clicks Like
        │  HTTP request
        ▼
⚡ Backend (FastAPI)
   Receives request, validates data, executes logic
        │  Database query
        ▼
🐘 PostgreSQL Database
   Stores the like information
        │
        ▼
⚡ FastAPI returns a response
   200 OK → { "message": "Post liked successfully" }

📌 FastAPI is a modern Python web framework used to build APIs. An API (Application Programming Interface) lets two software applications communicate.

FastAPI helps us:

🛣️ Create API endpoints
📥 Receive data from frontend applications
✔️ Validate incoming data
🧠 Execute business logic
🗄️ Communicate with databases
📤 Return responses in JSON
📖 Automatically generate API documentation

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def home():
    return {"message": "Hello World"}

from fastapi import FastAPI imports the FastAPI class
app = FastAPI() creates our application
@app.get("/") creates a GET endpoint at the root URL
def home() is the Python function that handles the request
return sends the response back to the client

Visit http://127.0.0.1:8000/ and you get:

{
  "message": "Hello World"
}

That's your first FastAPI endpoint! 🎉

2. 📬 Understanding API Parameters

When a frontend calls a backend, it often needs to send information:

Which post should we retrieve?
Which page of posts should we show?
What data should we save?
Which user is making the request?

FastAPI has three parameter types you need to know:

🛤️ Path parameters
❓ Query parameters
📦 Body parameters

2.1 🛤️ Path parameters

Values passed directly inside the URL path. You have 100 posts and want post number 25:

GET /posts/25

Here 25 is the path parameter, because it's part of the URL path itself.

from fastapi import FastAPI

app = FastAPI()


@app.get("/posts/{post_id}")
def get_post(post_id: int):
    return {
        "post_id": post_id,
        "message": "Post retrieved",
    }

{post_id} is a dynamic placeholder
The actual value comes from the URL
FastAPI extracts it and passes it to your function

When you request GET /posts/25, FastAPI internally calls:

get_post(post_id=25)

Response:

{
  "post_id": 25,
  "message": "Post retrieved"
}

Notice we wrote post_id: int. What if somebody sends GET /posts/hello? FastAPI automatically validates the value. Since "hello" can't become an integer, it returns a validation error (HTTP 422). You don't write that validation yourself! ✨

Real-life examples: /users/10, /posts/25, /comments/100, /products/500.

🧠 Path parameters generally identify a particular resource.

2.2 ❓ Query parameters

Extra values passed after a ? in the URL. Want only 10 of your 100 posts?

GET /posts?limit=10
GET /posts?limit=10&skip=20

/posts = endpoint
? = start of query parameters
limit=10 = a query parameter
& separates multiple query parameters

@app.get("/posts")
def get_posts(limit: int = 10, skip: int = 0):
    return {
        "limit": limit,
        "skip": skip,
    }

limit: int = 10 means limit should be an integer, and if the user doesn't provide it, use 10. Same idea for skip, defaulting to 0.

Request	Response
`GET /posts`	`{"limit": 10, "skip": 0}`
`GET /posts?limit=5`	`{"limit": 5, "skip": 0}`
`GET /posts?limit=20&skip=40`	`{"limit": 20, "skip": 40}`

Real-life uses:

Query parameter	Purpose
`?limit=10`	Get 10 records
`?page=2`	Get page 2
`?search=deepika`	Search for something
`?sort=latest`	Sort by latest
`?category=technology`	Filter by category

In a social feed, query parameters are perfect for pagination, filtering and searching, like GET /feed?limit=10&cursor=abc123.

2.3 📦 Body parameters (request body)

A user wants to create a new post and must send a caption, image URL and location. That data usually goes in the request body:

POST /posts

{
  "caption": "Beautiful evening!",
  "image_url": "https://example.com/image.jpg",
  "location": "Bengaluru"
}

from pydantic import BaseModel


class PostCreate(BaseModel):
    caption: str
    image_url: str
    location: str


@app.post("/posts")
def create_post(post: PostCreate):
    return {
        "message": "Post created",
        "caption": post.caption,
    }

PostCreate is a Pydantic model
post: PostCreate tells FastAPI to expect request-body data matching that model
post.caption and post.image_url access the values

2.4 ⚖️ Path vs Query vs Body

Type	Role	Example	Question it answers
🛤️ Path	Identifies which resource	`GET /posts/25`	Which post? Post 25
❓ Query	Controls how we retrieve	`GET /posts?limit=10`	How many posts? 10
📦 Body	Carries data to create/update	`POST /posts` + `{"caption": "My new post"}`	What do we save? The caption

🧠 Quick rule: Path = which resource? · Query = how should we retrieve it? · Body = what data are we sending?

3. ✔️ Pydantic Models: Request and Response Schemas

Imagine your frontend sends:

{
  "username": "deepika",
  "age": "twenty",
  "email": "not-an-email"
}

Age is a string instead of an integer, and the email isn't valid. Should your backend blindly accept that? Obviously not! We need a way to validate incoming data. Enter Pydantic.

3.1 🛂 What is Pydantic?

📌 Pydantic is a Python library for data validation and parsing. It checks whether incoming data follows the structure and rules we've defined.

Think of it as an airport security checkpoint ✈️: before passengers enter, their details are checked. Before request data reaches your business logic, Pydantic validates it.

3.2 🏗️ Creating a Pydantic model

from pydantic import BaseModel


class UserCreate(BaseModel):
    username: str
    age: int
    email: str

BaseModel is Pydantic's base class. We inherit from it to create a model.

Field	Expected type
`username`	String
`age`	Integer
`email`	String

3.3 ✅ Valid data arrives

{
  "username": "deepika",
  "age": 22,
  "email": "deepika@example.com"
}

Pydantic validates it, creates a Python model object, and you access user.username, user.age, user.email.

3.4 ❌ Invalid data arrives

{
  "username": "deepika",
  "age": "hello",
  "email": "deepika@example.com"
}

"hello" can't become an integer, so FastAPI automatically returns:

{
  "detail": [
    {
      "type": "int_parsing",
      "loc": ["body", "age"],
      "msg": "Input should be a valid integer"
    }
  ]
}

💡 FastAPI handles this request-validation error automatically. You don't write manual if checks for every field.

3.5 📏 Validation rules

from pydantic import BaseModel, EmailStr, Field


class UserCreate(BaseModel):
    username: str = Field(min_length=3, max_length=20)
    age: int = Field(ge=18, le=100)
    email: EmailStr

min_length=3: username needs at least 3 characters
max_length=20: username can't exceed 20 characters
ge=18: age must be greater than or equal to 18
le=100: age must be less than or equal to 100
EmailStr: validates email format

📝 EmailStr requires the optional email-validator package.

3.6 🕳️ Optional fields and defaults

class UserCreate(BaseModel):
    username: str
    age: int
    bio: str | None = None

str | None means the value can be a string or absent/null
= None provides a default value

3.7 🔄 Request schemas vs Response schemas

🎯 A VERY important distinction in real backends!

A user creates an account and sends:

{
  "username": "deepika",
  "email": "deepika@example.com",
  "password": "secret123"
}

That's a request schema. But should we return the password? Absolutely not! 🙅 We return:

{
  "id": 101,
  "username": "deepika",
  "email": "deepika@example.com"
}

That's a response schema.

from pydantic import BaseModel


# Incoming data
class UserCreate(BaseModel):
    username: str
    email: str
    password: str


# Outgoing data
class UserResponse(BaseModel):
    id: int
    username: str
    email: str

Connect them in FastAPI:

@app.post("/users", response_model=UserResponse)
def create_user(user: UserCreate):
    return {
        "id": 101,
        "username": user.username,
        "email": user.email,
        "password": user.password,
    }

Even though the function returns a dict containing the password, the response_model filters the output to the fields in UserResponse. The client never sees the password. 🔐

Why separate request and response schemas?

✔️ Validate incoming data
🎛️ Control outgoing data
🔒 Prevent sensitive information from being exposed
📜 Maintain a clear API contract
🧹 Make code easier to maintain

Common naming conventions:

Model	Purpose
`UserCreate`	Incoming user creation data
`UserUpdate`	Incoming user update data
`UserResponse`	Outgoing user data
`PostCreate`	Incoming post creation data
`PostResponse`	Outgoing post data
`LoginRequest`	Login credentials

4. 💉 Dependency Injection (DI): The MOST Important Concept!

Don't worry about the big name. Let's use a simple example.

4.1 🚨 First, the problem

Whenever a user views their feed, your backend needs a database:

def get_feed():
    db = connect_to_database()
    posts = db.get_posts()
    return posts

Now imagine 20 endpoints needing database access. Would you write db = connect_to_database() inside all 20 functions? That causes:

🔁 Repeated code
😩 Hard to maintain
🧪 Hard to test
🔌 Hard to manage database connections

What if one function provided the database connection whenever another function needed it? That's the basic idea of Dependency Injection.

4.2 🧩 What is a dependency?

📌 A dependency is something a function or class needs to do its work.

def get_feed(db):
    posts = db.get_posts()
    return posts

Here db is a dependency, because get_feed() depends on it. Other examples:

🗄️ Database session
👤 Authenticated user
⚙️ Configuration settings
📝 Logger
🌍 External API client
⚡ Cache connection

4.3 💉 What is Dependency Injection?

📌 Dependency Injection means: instead of creating everything a function needs by itself, we provide those things from outside.

❌ Without DI:

def get_feed():
    db = connect_to_database()
    return db.get_posts()

The function creates its own dependency.

✅ With DI:

def get_feed(db):
    return db.get_posts()

The dependency is provided from outside. This is more flexible. For example, during testing you can pass in a fake database instead of the real one. 🧪

4.4 🎯 What is `Depends()` in FastAPI?

FastAPI has a built-in function, Depends(), to declare dependencies:

from fastapi import Depends, FastAPI

app = FastAPI()


def get_database():
    return "Database connection"


@app.get("/posts")
def get_posts(db=Depends(get_database)):
    return {
        "database": db,
        "message": "Posts retrieved",
    }

get_database() is our dependency provider. Now the interesting part:

def get_posts(db=Depends(get_database)):

Depends(get_database) tells FastAPI: "Before executing get_posts(), run get_database() and inject its result into the db parameter."

Client sends GET /posts
        ▼
FastAPI sees Depends(get_database)
        ▼
Executes get_database()
        ▼
Injects the returned value into db
        ▼
Executes get_posts(db)
        ▼
Response returned

You never manually call get_database() inside get_posts(). FastAPI resolves it for you. That's the beauty of Depends()! ✨

4.5 🗄️ A practical example: database dependency

With PostgreSQL and SQLAlchemy, we create a database session provider:

from app.database import SessionLocal


def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()

db = SessionLocal() creates a new database session (a temporary working connection to the database)
yield db hands the session to the function that requested it
finally: db.close() closes the session after the request finishes, even if an exception occurs

🤔 Why yield instead of return? A generator dependency lets FastAPI run cleanup code after the request completes.

Now use it in an endpoint:

from fastapi import Depends
from sqlalchemy.orm import Session


@app.get("/posts")
def get_posts(db: Session = Depends(get_db)):
    posts = db.query(Post).all()
    return posts

db: Session is the expected SQLAlchemy session type
Depends(get_db) tells FastAPI how to provide it
db.query(Post).all() retrieves all Post records

We never created the session manually. FastAPI manages its lifecycle through the dependency.

📝 This is a simplified synchronous SQLAlchemy example. Async SQLAlchemy uses an async session and an async dependency.

4.6 🔐 DI for authentication

A user wants to create a post. Before allowing it we check: Is the user logged in? Is their token valid? Which user is this? Create a dependency:

def get_current_user():
    # Validate access token
    # Find user in database
    # Return authenticated user
    return current_user

Then use it:

@app.post("/posts")
def create_post(current_user=Depends(get_current_user)):
    return {
        "message": "Post created",
        "user": current_user.username,
    }

What happens:

The client sends a request
FastAPI sees Depends(get_current_user)
It runs the authentication dependency
The dependency validates the user's identity
It returns the authenticated user
FastAPI injects that user into current_user
The endpoint executes

If authentication fails, the dependency can raise an HTTP 401 or 403, and the endpoint won't run normally. With 15 endpoints needing authentication, you reuse the same dependency instead of repeating token logic 15 times. ♻️

5. 🔄 Inversion of Control (IoC)

You understand DI, so IoC is easier now.

5.1 🤔 What is IoC?

📌 Inversion of Control means transferring the responsibility of managing something from your code to an external framework or system.

❌ Without IoC, you decide when to create the session, how to pass it, and when to close it:

def get_posts():
    db = create_database_session()
    try:
        return db.get_posts()
    finally:
        db.close()

✅ With IoC using FastAPI:

def get_posts(db=Depends(get_db)):
    return db.get_posts()

FastAPI manages dependency resolution and the cleanup lifecycle for the request. You simply declare what you need. Control has shifted from your endpoint to the framework. That's Inversion of Control. 🎛️

5.2 🔗 Relationship between IoC and DI

Concept	Meaning
IoC	The general principle of handing over control
DI	A technique used to implement IoC
`Depends()`	FastAPI's mechanism for declaring dependencies

Inversion of Control (IoC)   → the general design principle
        ▼
Dependency Injection (DI)    → a way to achieve IoC
        ▼
FastAPI Depends()            → a built-in dependency mechanism

🎤 Interview-ready explanation

"Inversion of Control is a design principle where the responsibility of managing dependencies is transferred from our application code to the framework. In FastAPI, we achieve this using Dependency Injection through Depends(). For example, instead of manually creating database sessions inside every route, we declare Depends(get_db), and FastAPI resolves and manages the dependency."

6. 🗂️ APIRouter: Organizing API Endpoints

As your app grows, you might write everything in one file:

from fastapi import FastAPI

app = FastAPI()


@app.get("/users")
def get_users():
    pass


@app.post("/users")
def create_user():
    pass


@app.get("/posts")
def get_posts():
    pass


@app.post("/posts")
def create_post():
    pass


@app.get("/comments")
def get_comments():
    pass

Imagine 100 endpoints in one file! 😵 We need to split them by responsibility. That's APIRouter.

6.1 🤔 What is APIRouter?

📌 APIRouter is a FastAPI class that organizes related endpoints into separate modules.

Think of dividing a company into departments: the user department handles users, the post department handles posts, the comment department handles comments, the auth department handles authentication.

6.2 📁 Recommended folder structure

backend/
└── app/
    ├── api/
    │   ├── users.py       # User endpoints
    │   ├── posts.py       # Post endpoints
    │   └── comments.py    # Comment endpoints
    └── main.py            # Main application

6.3 🛠️ Creating a router

users.py:

from fastapi import APIRouter

router = APIRouter()


@router.get("/")
def get_users():
    return {"message": "All users"}


@router.get("/{user_id}")
def get_user(user_id: int):
    return {"user_id": user_id}


@router.post("/")
def create_user():
    return {"message": "User created"}

router = APIRouter() creates a separate router. Instead of @app.get("/") we write @router.get("/"), because these endpoints belong to the router, not directly to the main app.

6.4 🔌 Connecting the router to the main app

main.py:

from fastapi import FastAPI

from app.api import users

app = FastAPI()

app.include_router(
    users.router,
    prefix="/users",
    tags=["Users"],
)

include_router() registers the router with the main app
prefix="/users" means every endpoint in that router starts with /users
tags=["Users"] groups these endpoints under a Users section in Swagger docs

Router endpoint	Final endpoint
`@router.get("/")`	`GET /users/`
`@router.get("/{user_id}")`	`GET /users/{user_id}`
`@router.post("/")`	`POST /users/`

6.5 ✅ Why use APIRouter?

🧩 Modularity: separate related endpoints
🛠️ Maintainability: easier to modify code
♻️ Reusability: routers can be included in different apps
📖 Readability: easier for developers to understand
📈 Scalability: suitable for large backends

⚠️ Important: APIRouter does not create a separate server or application. It organizes routes inside one FastAPI application.

7. 📖 Automatic Interactive Documentation: Swagger UI and ReDoc

One of FastAPI's coolest features! You create 50 endpoints. How will your frontend developer know which endpoints exist, which methods they support, what parameters and request body they need, and what they return? You could write docs manually, but FastAPI generates them automatically! 🪄

7.1 🧪 Swagger UI

An interactive interface to explore and test your API. After starting your app, visit:

http://127.0.0.1:8000/docs

You can:

View all endpoints
Expand an endpoint
See required parameters
Enter request data
Click Try it out
Click Execute
View the response and HTTP status code

You can test your backend without writing any frontend code! 🎉

7.2 📚 ReDoc

Another automatically generated documentation interface:

http://127.0.0.1:8000/redoc

It presents your API in a more reference-oriented, organized layout.

7.3 ⚖️ Swagger UI vs ReDoc

Feature	Swagger UI	ReDoc
URL	`/docs`	`/redoc`
Interactive testing	Yes	Primarily documentation-focused
API exploration	Yes	Yes
Layout	Interactive interface	Reference-style documentation
Generated automatically	Yes	Yes

Both are generated from FastAPI's OpenAPI schema.

7.4 📜 What is OpenAPI?

📌 OpenAPI is a standard specification that describes your API: available endpoints, HTTP methods, request parameters, request body schemas, response schemas and authentication requirements.

FastAPI generates it automatically at:

http://127.0.0.1:8000/openapi.json

Swagger UI and ReDoc read this schema to display the docs.

7.5 🪄 How does FastAPI generate docs automatically?

@app.get("/posts/{post_id}")
def get_post(post_id: int):
    return {"post_id": post_id}

FastAPI understands the HTTP method (GET), the endpoint (/posts/{post_id}), the parameter name (post_id) and its type (integer). Likewise:

class PostCreate(BaseModel):
    caption: str
    image_url: str

FastAPI understands the request-body structure, and:

@app.post("/posts", response_model=PostResponse)

tells it the expected response schema. It collects all this into OpenAPI docs. That's why type hints and Pydantic models matter so much in FastAPI. 💡

8. 🎯 Single Responsibility Principle (SRP)

SRP is one of the five SOLID principles of object-oriented design. Let's focus on just this one.

8.1 🤔 What is SRP?

📌 A function, class, or module should have one clear responsibility, or one reason to change.

Think of a restaurant 🍽️: the chef prepares food, the cashier handles payments, the waiter takes orders, the manager manages operations. If one person did everything, it'd be chaos. Similarly, we shouldn't put all backend responsibilities into one function.

8.2 ❌ The problem: without SRP

@app.post("/posts")
def create_post(post: PostCreate):
    # Validate user
    # Connect to database
    # Check image URL
    # Create post in database
    # Send notification
    # Return response
    return {"message": "Post created"}

This function handles too many responsibilities:

Receiving the HTTP request
Validating user permissions
Handling database operations
Executing post creation logic
Sending notifications
Returning the HTTP response

Change notification logic? Modify this function. Change database logic? Modify it again. It becomes hard to maintain. 😩

8.3 ✅ Applying SRP: separate responsibilities

🛣️ Router Layer      posts.py             Receives HTTP requests, reads parameters, calls services, returns responses
        ▼
🧠 Service Layer      post_service.py      Contains business logic and rules
        ▼
📚 Repository Layer   post_repository.py   Handles database operations (insert, update, retrieve)
        ▼
🗄️ Database           Stores application data

A. Router layer (posts.py)

@router.post("/")
def create_post(
    post: PostCreate,
    db: Session = Depends(get_db),
):
    return post_service.create_post(db, post)

Its job: receive the request, get the request data, obtain the database dependency, call the service, return the result. It doesn't need to know the internal business rules.

B. Service layer (post_service.py)

def create_post(db, post):
    if not post.caption:
        raise ValueError("Caption is required")

    return post_repository.create(db, post)

Its job: implement business rules, decide what should happen, coordinate the required operations (like checking whether a user is allowed to create a post).

C. Repository layer (post_repository.py)

def create(db, post):
    new_post = Post(caption=post.caption)

    db.add(new_post)
    db.commit()
    db.refresh(new_post)

    return new_post

Its job: talk to the database (insert, retrieve, update, delete).

Why is this better? If your database changes, you modify the repository layer without rewriting all your HTTP endpoints. If business rules change, you update the service layer. The app becomes easier to maintain and test. 🧪

📝 These layers are an architectural choice, not a mandatory FastAPI requirement. Small apps can have fewer layers. The key is to avoid mixing unrelated responsibilities.

9. 🧵 Putting Everything Together: A Real FastAPI Example

Let's build a mini Instagram backend with an endpoint to create posts.

backend/
└── app/
    ├── api/
    │   └── posts.py              # Routes
    ├── schemas/
    │   └── post.py               # Pydantic models
    ├── services/
    │   └── post_service.py       # Business logic
    ├── repositories/
    │   └── post_repository.py    # Database operations
    └── main.py                   # Application entry point

Step 1: create Pydantic schemas (schemas/post.py)

from pydantic import BaseModel


class PostCreate(BaseModel):
    caption: str


class PostResponse(BaseModel):
    id: int
    caption: str

Step 2: create the repository (repositories/post_repository.py)

def create_post(db, caption):
    new_post = {
        "id": 1,
        "caption": caption,
    }
    return new_post

📝 A simplified repository using a dictionary instead of a real SQLAlchemy database.

Step 3: create the service (services/post_service.py)

from app.repositories import post_repository


def create_post(db, post):
    return post_repository.create_post(db, post.caption)

Step 4: create the router (api/posts.py)

from fastapi import APIRouter, Depends

from app.schemas.post import PostCreate, PostResponse
from app.services import post_service

router = APIRouter()


def get_db():
    return "Database session"


@router.post("/", response_model=PostResponse)
def create_post(
    post: PostCreate,
    db=Depends(get_db),
):
    return post_service.create_post(db, post)

Step 5: connect the router (main.py)

from fastapi import FastAPI

from app.api import posts

app = FastAPI()

app.include_router(
    posts.router,
    prefix="/posts",
    tags=["Posts"],
)

Start the server:

uvicorn app.main:app --reload

Visit http://127.0.0.1:8000/docs, find POST /posts/, and send:

{
  "caption": "My first post!"
}

Expected response:

{
  "id": 1,
  "caption": "My first post!"
}

9.1 🔍 What happened internally?

🌐 Frontend sends the request: POST /posts/ with JSON data
🛣️ FastAPI identifies the route: it finds the matching endpoint in the posts router
✔️ Pydantic validates request data: checks that the caption is a string
💉 Dependency Injection happens: FastAPI sees Depends(get_db) and resolves the database dependency
🧠 Router calls the service: post_service.create_post(db, post)
📚 Service calls the repository: the repository performs the database operation
📤 Response model validates output: FastAPI uses PostResponse to validate and serialize the response
🔙 Response goes back to the frontend as JSON

This is how everything you've learned works together in a real backend. 🎯

10. 📋 Quick Revision: Everything at a Glance

Concept	What it does	Example
FastAPI	Python framework for APIs	`FastAPI()`
Path parameter	Identifies a resource	`/posts/10`
Query parameter	Filters or controls retrieval	`?limit=10`
Body parameter	Sends structured data	JSON request body
Pydantic	Validates and parses data	`BaseModel`
Request schema	Defines incoming data	`PostCreate`
Response schema	Defines outgoing data	`PostResponse`
`Depends()`	Injects dependencies	`Depends(get_db)`
IoC	Transfers control to a framework	FastAPI manages dependency resolution
APIRouter	Organizes endpoints	`APIRouter()`
Swagger UI	Interactive API testing	`/docs`
ReDoc	API documentation	`/redoc`
SRP	One clear responsibility per component	Separate router and service

11. 🎤 Interview Questions

1. What is FastAPI?
A modern Python web framework used to build APIs. It supports automatic request validation, dependency injection, asynchronous programming and automatic API documentation.

2. What's the difference between path and query parameters?
Path parameters identify a particular resource, whereas query parameters provide additional filtering, sorting or pagination information.

3. What is Pydantic?
A Python data validation library. It validates incoming data based on the types and rules defined in our models.

4. Why do we need separate request and response schemas?
Request schemas validate incoming data, while response schemas control the structure of outgoing data and prevent unwanted fields, such as passwords, from being exposed.

5. What is Dependency Injection?
A technique where required objects or services are provided to a function from outside instead of the function creating them internally.

6. What is Depends() in FastAPI?
FastAPI's built-in mechanism for declaring dependencies. FastAPI resolves them and injects their results into endpoint functions.

7. What is Inversion of Control?
A design principle where control over certain operations is transferred from application code to a framework or external component. FastAPI implements this through dependency injection.

8. What is APIRouter?
It organizes related API endpoints into separate modules, making large applications modular and maintainable.

9. What's the difference between Swagger UI and ReDoc?
Swagger UI provides an interactive interface for exploring and testing APIs. ReDoc provides a documentation-focused interface.

10. What is SRP?
The Single Responsibility Principle says a component should have one clear responsibility. In FastAPI, we apply it by separating route handling, business logic and database operations.

12. 🧪 Test Yourself!

Try these before peeking at the answers! ❤️

You want to retrieve post number 50 using /posts/50. What type of parameter is 50?
- A) Query parameter
- B) Path parameter
- C) Body parameter
Which library validates incoming request data using BaseModel?
- A) SQLAlchemy
- B) Uvicorn
- C) Pydantic
What does Depends(get_db) do?
- A) Deletes the database
- B) Asks FastAPI to provide a database dependency
- C) Creates a new API endpoint
Which component organizes related API endpoints?
- A) APIRouter
- B) BaseModel
- C) ReDoc
Where can you interactively test your FastAPI endpoints by default?
- A) /redoc
- B) /openapi.json
- C) /docs
Which layer should contain business logic?
- A) Router
- B) Service
- C) Frontend

👀 Click to reveal answers

B: 50 is part of the URL path, so it's a path parameter
C: Pydantic validates data using BaseModel
B: It asks FastAPI to resolve and inject the get_db dependency
A: APIRouter groups related endpoints
C: /docs opens Swagger UI for interactive testing
B: The service layer holds business logic

13. 🧭 What to Remember in Your Own Backend

When you open your own FastAPI project, try to identify:

🛣️ api/: where are the routes?
✔️ schemas/: what request and response structures are defined?
🧠 services/: where is the business logic?
📚 repositories/: where are database operations performed?
💉 Depends(): how are dependencies provided?
🗂️ APIRouter: how are endpoints grouped?
🚪 main.py: where are routers registered and the app created?

Once you can spot these in real code, you'll understand how a professional FastAPI backend is structured instead of just memorizing definitions.

🚀 What's Next?

Pick one endpoint and trace its complete flow: frontend request → router → dependency → service → repository → database → response. That's far more valuable than memorizing 50 definitions. 💡

If this post helped you, drop a ❤️ or 🦄 and share it with a friend learning FastAPI. Happy coding! 👩‍💻✨

DEV Community: Deepika Pusala

Transactions, Isolation Levels, MVCC, Locking, Triggers, Audit Logs & CAP Theorem in PostgreSQL 🔒

🔒 Isolation, MVCC, Locking, Triggers, Audit Logging & CAP Theorem

📚 Table of Contents

1️⃣ What is a transaction?

📍 Bonus: SAVEPOINT

2️⃣ ACID: Isolation and Durability 🧪

2.1 🚦 Isolation: what does it mean?

2.2 💾 Durability: what does it mean?

📓 What is WAL?

3️⃣ What is concurrency? 🚗🚕🚙

Why is it difficult?

4️⃣ Dirty Read 🧟

5️⃣ Non-repeatable Read 🔄

6️⃣ Phantom Read 👻

⚖️ Non-repeatable read vs phantom read

7️⃣ Isolation Levels 🎚️ (the most important section!)

7.1 🟥 Read Uncommitted (weakest)

7.2 🟨 Read Committed (PostgreSQL default)

7.3 🟧 Repeatable Read (stronger)

7.4 🟩 Serializable (strongest)

7.5 📊 Comparison table (interview favorite!)

🪜 Hierarchy

🛠️ Setting the isolation level

8️⃣ MVCC: Multi-Version Concurrency Control 🧬

📸 What is a snapshot?

🧹 Does PostgreSQL keep old versions forever?

9️⃣ Row-Level Locking 🔐

9.1 🧱 What is a row-level lock?

9.2 ⭐ SELECT ... FOR UPDATE

9.3 🎛️ Row-lock modes

🎁 Bonus: NOWAIT and SKIP LOCKED

9.4 💀 What is a deadlock?

🛡️ How to prevent deadlocks

🔟 Database Triggers ⚡

Real-world example

10.1 🔄 How a trigger works

10.2 🧭 Types of triggers

10.3 🆕🆚 NEW and OLD

10.4 🧪 Hands-on: build an audit trigger

1️⃣1️⃣ Audit Logging 📝

What should an audit log contain?

⚖️ Application-level vs trigger-based logging

🎁 Bonus: giving triggers request context

1️⃣2️⃣ CAP Theorem 🌐

12.1 🖥️ What is a distributed system?

12.2 📐 What is CAP theorem?

🅲 Consistency (in CAP)

🅰 Availability

🅿 Partition Tolerance

12.3 ⚖️ The CAP trade-off

12.4 🌍 Real-world examples

12.5 ⏳ What is eventual consistency?

1️⃣3️⃣ ACID vs CAP: don't confuse them! ⚔️

1️⃣4️⃣ Connecting it all to your FastAPI project 🔌

✅ Solution 1: row-level locking

✅ Solution 2: a single conditional UPDATE (often simplest!)

🐍 FastAPI + SQLAlchemy version

🔁 Retrying on serialization failures and deadlocks

1️⃣5️⃣ Complete revision sheet 🧾

1️⃣6️⃣ Important interview questions 🎤

1️⃣7️⃣ Test your understanding! 🧪

1️⃣8️⃣ What to practice as a Full Stack Developer 🛠️

✅ Practical checklist

🧪 Mini lab: non-repeatable read (Read Committed)

🧪 Mini lab: same test under Repeatable Read

🧪 Mini lab: FOR UPDATE blocking

🧪 Mini lab: create a deadlock 💀

🎯 Final thoughts

🙌 Enjoyed this post?

postgres #sql #database #backend #concurrency

CQRS & Event-Driven Thinking: Commands, Queries, Domain Events & the Outbox Pattern Explained 📨

📨 CQRS & Event-Driven Thinking: Let's Learn from ZERO!

1. 🤔 First, Understand the Problem

2. 🔀 Command-Query Separation (CQS)

2.1 ✍️ Command example: like a post

2.2 👀 Query example: get post details

2.3 📊 Visual comparison

2.4 📋 More examples

3. 🤷 Why Do We Need CQS?

9.2 ⭐ `SELECT ... FOR UPDATE`

🎁 Bonus: `NOWAIT` and `SKIP LOCKED`

10.3 🆕🆚 `NEW` and `OLD`

🧪 Mini lab: `FOR UPDATE` blocking

⚖️ `joinedload` vs `selectinload`

9.1 🚫 Avoid `SELECT *`

9.2 📄 Use `LIMIT` and pagination