DEV Community: Mohamed Mayallo

Is SQL BETWEEN Inclusive or Exclusive?

Mohamed Mayallo — Mon, 06 Oct 2025 10:20:00 +0000

TL;DR

The SQL BETWEEN operator is inclusive. That means both the lower and upper boundary values are included in the results.

Introduction

When writing SQL queries, many developers wonder: is SQL BETWEEN inclusive or exclusive? The answer is simple: BETWEEN is inclusive. This means that if you write BETWEEN 10 AND 20, both 10 and 20 are part of the results.

SQL BETWEEN Example

Imagine we have a table called products:

id	name	price
1	Pen	5
2	Notebook	10
3	Bag	20
4	Laptop	50


SELECT * 
FROM products
WHERE price BETWEEN 10 AND 20;

Result:

id	name	price
2	Notebook	10
3	Bag	20

Explanation:
Both 10 and 20 are included. That’s why we say SQL BETWEEN is inclusive.

SQL BETWEEN vs Greater Than and Less Than


SELECT * 
FROM products
WHERE price >= 10 AND price <= 20;

This gives the same result as using BETWEEN.

id	name	price
2	Notebook	10
3	Bag	20

So, BETWEEN = >= and <=.

Common Misunderstanding

Some developers think that BETWEEN is exclusive (ignores boundaries). That’s wrong. If you wanted to exclude the edges, you would use > and < instead:


SELECT * 
FROM products
WHERE price > 10 AND price < 20;

Result:

id	name	price
--	--	--

Here no rows match because only a price between 10 and 20 (like 15) would qualify, but our table doesn’t have that value.

Conclusion

So, is SQL BETWEEN inclusive or exclusive? The answer: SQL BETWEEN is inclusive.

It includes both boundary values.
It works the same as >= and <=.
If you need exclusive ranges, use > and <.

This makes BETWEEN a very convenient way to filter data in an inclusive range.

Finally, if you found this article helpful, you can check more here:

Two Ways To Git Cherry-Pick From Another Repo

Mohamed Mayallo — Thu, 02 Oct 2025 07:53:00 +0000

Introduction

When working with multiple repositories, there are times when you want to bring in just one (or a few) commits from another project, without merging the entire repo. In Git, you can cherry-pick from another repo by treating it like a remote.

In this article, I’ll walk you through two steps with examples. Let's jump into them.

Why cherry-pick from an external repo?

You want to reuse a bug fix or feature commit from another project.
You don’t want to merge all changes, just specific commits.
Keeps history clean by pulling only essential changes.

However, cherry-picking across repos should be used carefully, because it can lead to duplicate content and conflicts.

First Way: Using Cherry-Pick

First of all, suppose your current repo is RepoA, and you want a commit from RepoB (on GitHub or Locally).

1. Add the external repo as a remote

git remote add repoB https://github.com/username/RepoB.git

# or if the RepoB is local
git remote add repoB C:/Users/User/RepoB

repoB is just a name (alias) for the external repository.

You can choose any name you like (e.g. external, upstream2, sourceRepo).

If you already have a remote with that name, pick a different alias.

2. Fetch from that remote

git fetch repoB

This will bring all branches and commits from RepoB, so that your local Git can see them. You can now reference branches like repoB/main or repoB/feature-xyz.

3. Identify commit(s) you need

You need the commit hash (SHA) you want to cherry-pick.

List commits in a branch from the external remote:

git log repoB/main --oneline

# or

git log --graph --decorate repoB/main

Once you see the hash (say abc1234), that’s the commit you want to pick.

4. Cherry-pick the commit

Switch to the branch in RepoA where you want to apply the change:

git checkout my-feature-branch

Now cherry-pick:

git cherry-pick abc1234

# or cherry-pick a range
git cherry-pick startSHA..endSHA

# or multiple commits
git cherry-pick abc1234 def5678

This will apply the changes from commit abc1234 into your branch, creating a new commit (with the same content, possibly a different hash).

Then, resolve conflicts if any, or if you decide it's too messy or want to abort:

git cherry-pick --abort

That rolls back to the state before the cherry-pick started.

Finally, don't forget to remove the remote if you no longer need it.

git remote remove repoB

Second Way: Using format-patch

In fact, it is a different way from cherry-picking, but it achieves our last goal.

If you don’t want to add a remote, you can generate a patch file from the other repo and apply it.

1. Generate the patch file:

In RepoB, identify the commit hash you want to pick (as we did before), so export it into a patch file:

So, in RepoB, run:

git format-patch -1 abc1234 --stdout > mypatch.patch

-1 abc1234 means “one commit” (the one we want).

--stdout emits the patch to stdout (redirected to file).

Now you have a physical file (mypatch.patch) in the RepoB that represents the commit.

2. Move this patch file to RepoA

Copy mypatch.patch into RepoA (can be via cp, USB, email, etc.):

cp /path/to/RepoB/mypatch.patch /path/to/RepoA/

3. Apply the patch in RepoA

Then in RepoA:

git apply mypatch.patch

This method is useful when you can’t or don’t want to maintain a remote link.

Keep In Mind These Tips

Avoid cherry-picking too many commits across repos; it may cause duplicated code or merge confusion.
Always double-check commit context (dependencies); sometimes a commit depends on earlier ones you didn’t cherry-pick.
After you fetch from an external remote, avoid accidentally merging the remote branch unless you intend to.

Conclusion

In summary, to git cherry-pick from an external repo:

Add the external repo as a remote
Fetch it
Identify commit(s)
Run git cherry-pick (or patch method)
Resolve conflicts
Push your changes

Finally, if you found this article helpful, you can check more here:

Untangle Your Code: Chain of Responsibility Design Pattern

Mohamed Mayallo — Tue, 30 Sep 2025 07:22:04 +0000

Introduction

As developers, we’ve all been there: a single function bloated with a lot of responsibilities. It starts simple, and soon, the logic becomes a tangled mess that’s hard to read, maintain, and extend.

The Chain of Responsibility Design Pattern is a behavioral design pattern that offers a clean escape from this complexity.

It lets you pass a request along a chain of handlers. Each handler decides either to process the request or to pass it to the next handler in the chain. This allows you to decouple the sender of a request from its receivers.

Use Case 1: A Messy Signup Form

Imagine you're building a user registration feature. You need to validate the user's email, name, and password. Without a clear pattern, you might end up with something like this:

// A single function handling all validation
function validateRegistration(data: any) {
    // Validation for email
    if (data.email && data.email.includes('@')) {
        console.log("Email seems okay.");
        // Nested validation for name
        if (data.name && data.name.length > 2) {
            console.log("Name is fine.");
            // More nested validation for password
            if (data.password && data.password.length > 5) {
                console.log("Password is valid. All checks passed!");
                return true;
            } else {
                console.log("Error: Password is too short.");
                return false;
            }
        } else {
            console.log("Error: Name is too short.");
            return false;
        }
    } else {
        console.log("Error: Invalid email.");
        return false;
    }
}

const formData = {
    email: "test@test.com",
    name: "test",
    password: "123456"
};

validateRegistration(formData);

This code works, but it's rigid. What if you need to add a new validation rule, like checking for password strength? You’d have to dive back into this nested logic, increasing the risk of breaking something.

The Solution: A Clean Chain of Validators

The Chain of Responsibility pattern solves this by breaking each validation rule into its own handler class. These handlers are then linked together in a sequence, or a "chain."

A request enters the start of the chain and travels from handler to handler until one of them handles it or it reaches the end. For validation, if any handler finds an issue, it can break the chain.

Here is how we can refactor our validation logic using the Chain of Responsibility pattern in TypeScript. This example creates a flexible input validation chain.

// Define the contract for every handler in the chain
interface Handler {
    setNext(handler: Handler): Handler;
    handle(): boolean | undefined;
}

// An abstract base class to provide default chaining behavior
abstract class BaseHandler implements Handler {
    private nextHandler: Handler | undefined;

    // Sets the next handler in the chain and returns it,
    // allowing us to link handlers together fluently.
    public setNext(handler: Handler): Handler {
        this.nextHandler = handler;
        return handler;
    }

    // Executes the next handler if it exists. Subclasses will
    // call this after their own logic passes.
    public handle(): boolean | undefined {
        if (this.nextHandler) {
            return this.nextHandler.handle();
        }
        // If this is the end of the chain, return true to signify success
        return true;
    }
}

// Concrete handler for validating an email
class IsEmailHandler extends BaseHandler {
    private email: string;

    constructor(email: string) {
        super();
        this.email = email;
    }

    public handle(): boolean | undefined {
        const isValid = !!this.email && this.email.includes('@');

        if (isValid) {
            console.log("Valid email, passing to the next validator...");
            // If valid, call the next handler in the chain
            return super.handle();
        }

        console.log("Not a valid email, breaking the chain.");
        return false; // Break the chain
    }
}

// Concrete handler for validating a name
class IsNameValidHandler extends BaseHandler {
    private name: string;

    constructor(name: string) {
        super();
        this.name = name;
    }

    public handle(): boolean | undefined {
        const isValid = !!this.name && this.name.length > 2;

        if (isValid) {
            console.log("Valid name, passing to the next validator...");
            return super.handle();
        }

        console.log("Not a valid name, breaking the chain.");
        return false;
    }
}

// Concrete handler for validating a password
class IsPasswordValidHandler extends BaseHandler {
    private password: string;

    constructor(password: string) {
        super();
        this.password = password;
    }

    public handle(): boolean | undefined {
        const isValid = !!this.password && this.password.length > 5;

        if (isValid) {
            console.log("Valid password, all checks passed!");
            return super.handle();
        }

        console.log("Not a valid password, breaking the chain.");
        return false;
    }
}

// --- Client Code ---
const userFormData = {
    email: "test@test.com",
    name: "test",
    password: "123456"
};

// Create instances of our validators
const emailValidator = new IsEmailHandler(userFormData.email);
const nameValidator = new IsNameValidHandler(userFormData.name);
const passwordValidator = new IsPasswordValidHandler(userFormData.password);

// Build the chain of responsibility
emailValidator.setNext(nameValidator).setNext(passwordValidator);

// Start the validation process from the first handler
console.log("Starting validation...");
const isAllValid = emailValidator.handle();

if (isAllValid) {
    console.log("Registration successful!");
} else {
    console.log("Registration failed.");
}

Yes, for sure, more code, but have a closer look, it is significantly much cleaner. Each class has a single responsibility. Adding a new rule is as simple as creating a new handler class and inserting it into the chain, without modifying any existing code, which applies the Open-Closed Principle.

Use Case 2: API Middleware

This is a very common use case. In web frameworks like Express.js, middleware functions form a chain of responsibility. An incoming HTTP request is passed through a series of middleware functions.

An AuthMiddleware checks for a valid user token.
A RoleMiddleware checks if the user has permission for the action.
A LoginHandler handles the actual login logic.

Each middleware can either pass the request to the next one or end the cycle by sending a response (e.g., a 401 Unauthorized error). This answers the common middleware vs chain of responsibility question: middleware is a powerful, real-world implementation of this pattern.

// --- 1. Define the Request and Middleware Interface ---

// A simple representation of an incoming API request.
// The `user` property is optional because it's added by the AuthMiddleware.
interface ApiRequest {
    headers: {
        token?: string;
    };
    body: {
        data: string;
    };
    user?: {
        id: number;
        role: 'admin' | 'user';
    };
}

// The contract for any handler (middleware) in our chain.
interface Middleware {
    setNext(handler: Middleware): Middleware;
    handle(request: ApiRequest): { success: boolean; message: string; data?: any };
}

// --- 2. Create a Base Middleware Class for common logic ---

abstract class BaseMiddleware implements Middleware {
    private nextHandler: Middleware | null = null;

    // This sets the next middleware in the chain.
    public setNext(handler: Middleware): Middleware {
        this.nextHandler = handler;
        return handler;
    }

    // This runs the next middleware if it exists.
    // Each concrete middleware will call this method if its check passes.
    public handle(request: ApiRequest): { success: boolean; message: string; data?: any } {
        if (this.nextHandler) {
            return this.nextHandler.handle(request);
        }
        // This is the default success response if it's the end of the chain.
        return { success: true, message: 'Request processed successfully.' };
    }
}

// --- 3. Implement Concrete Middleware Handlers ---

/**
 * Checks for a valid authentication token. If found, it "enriches" the
 * request by adding user information before passing it on.
 */
class AuthMiddleware extends BaseMiddleware {
    public handle(request: ApiRequest): { success: boolean; message: string; data?: any } {
        console.log('Checking authentication...');

        // In a real app, you'd validate a real token (e.g., JWT).
        if (request.headers.token === 'my-secret-token') {
            // Token is valid. Attach user info and pass to the next middleware.
            request.user = { id: 1, role: 'admin' };
            console.log('Authentication successful. User attached.');
            return super.handle(request);
        }

        // Token is invalid, so we break the chain here.
        console.log('Authentication failed: Invalid token.');
        return { success: false, message: '401 Unauthorized: Invalid token.' };
    }
}

/**
 * Checks if the user has the required role. This middleware assumes
 * AuthMiddleware has already run and attached a user object.
 */
class RoleMiddleware extends BaseMiddleware {
    private requiredRole: 'admin' | 'user';

    constructor(role: 'admin' | 'user') {
        super();
        this.requiredRole = role;
    }

    public handle(request: ApiRequest): { success: boolean; message:string; data?: any } {
        console.log(`Checking authorization for role: ${this.requiredRole}...`);

        if (request.user?.role === this.requiredRole) {
            // User has the required role. Pass to the next middleware.
            console.log('Authorization successful.');
            return super.handle(request);
        }

        // User does not have the required role, so we break the chain.
        console.log('Authorization failed: Insufficient permissions.');
        return { success: false, message: '403 Forbidden: Insufficient permissions.' };
    }
}

/**
 * This is the final handler, representing the actual business logic
 */
class LoginHandler extends BaseMiddleware {
     public handle(request: ApiRequest): { success: boolean; message: string; data?: any } {
        console.log('Executing the login action...');
        // All checks passed, so process the request.
        return {
            success: true,
            message: 'Data processed by login.',
            data: `Hello, user ${request.user?.id}! You processed: ${request.body.data}`,
        };
     }
}


// --- 4. Client Code: Build and Run the Chain ---

// Create the middleware instances
const authCheck = new AuthMiddleware();
const adminCheck = new RoleMiddleware('admin');
const finalAction = new LoginHandler();

// Build the chain: Auth -> Role -> Action
authCheck.setNext(adminCheck).setNext(finalAction);


// --- TEST CASES ---

console.log('--- Test Case 1: Successful Request ---');
const successfulRequest: ApiRequest = {
    headers: { token: 'my-secret-token' },
    body: { data: 'important data' },
};
let result = authCheck.handle(successfulRequest);
console.log('Result:', result, '\n');


console.log('--- Test Case 2: Authentication Failure ---');
const failedAuthRequest: ApiRequest = {
    headers: { token: 'invalid-token' },
    body: { data: 'some data' },
};
result = authCheck.handle(failedAuthRequest);
console.log('Result:', result, '\n');


console.log('--- Test Case 3: Authorization Failure ---');
// To test this, we'll temporarily change the user's role after a valid auth.
// In a real scenario, this would come from the database lookup in AuthMiddleware.
class MockAuthMiddleware extends BaseMiddleware {
     public handle(request: ApiRequest) {
        request.user = { id: 2, role: 'user' }; // Mock a non-admin user
        return super.handle(request)
     }
}
const mockAuth = new MockAuthMiddleware()
mockAuth.setNext(adminCheck).setNext(finalAction)
const failedRoleRequest: ApiRequest = {
    headers: { token: 'my-secret-token' }, // Auth token is valid
    body: { data: 'admin data' },
};
result = mockAuth.handle(failedRoleRequest);
console.log('Result:', result);

So, as you saw, each handler has its own responsibility, which increases the decoupling. On top of that, you might have noticed from the third test case that it makes unit testing remarkably much easier.

When to Use the Chain of Responsibility Pattern

When you want to decouple the object that sends a request from the objects that process it.
When you need to execute handlers in a specific order.
When you want to add or remove responsible objects at runtime.

Conclusion

In short, the Chain of Responsibility pattern is more than just a theoretical concept; it's a practical solution to a common coding problem. By transforming a monolithic block of logic into a flexible chain of independent handlers, you gain code that is easier to read, test, and extend.

Refer to these resources:

Think about it

If you enjoyed this article, I’d truly appreciate it if you could share it—it really motivates me to keep creating more helpful content!

If you're interested in exploring more, check out these articles.

Thanks for sticking with me until the end—I hope you found this article valuable and enjoyable!

Do You Know The LATERAL JOIN In SQL?

Mohamed Mayallo — Wed, 06 Aug 2025 07:02:00 +0000

Introduction

Have you ever been stuck trying to write a SQL query that needs to perform a sub-calculation for each row of your main table? Maybe you needed to find the last 3 blog posts for every user, or the top 5 products for each category.

These "for-each" style problems can lead to complicated, slow, and hard-to-read queries. This is the exact problem the LATERAL JOIN was designed to solve.

What Does LATERAL Actually Mean?

First, let's clarify a critical point. While it's common to hear "lateral join", the true power comes from the LATERAL keyword itself. It isn't a type of join like INNER or LEFT; it's a modifier that fundamentally changes how a subquery in the FROM clause behaves.

Think of "lateral" as meaning "sideways." A standard subquery in a FROM clause is executed in isolation and cannot reference other tables in the same clause. The LATERAL keyword breaks this rule. It gives a subquery permission to look sideways at columns from tables that appear before it in the FROM list.

This enables a powerful for-each loop pattern directly in your SQL.

A Real-World Business Problem (PostgreSQL)

Let's create an e-commerce scenario with customers, products, and orders to demonstrate LATERAL joins in action using PostgreSQL.

CREATE TABLE customers (
    customer_id INT PRIMARY KEY,
    customer_name VARCHAR(100)
);

CREATE TABLE orders (
    order_id INT PRIMARY KEY,
    customer_id INT,
    order_date DATE,
    amount INT,
    items JSONB, -- To store a list of products in the order
    tags VARCHAR[] -- To store tags like 'gift', 'rush_delivery'
);

-- Seed the tables with versatile data
INSERT INTO customers (customer_id, customer_name) VALUES
(1, 'Alice'),
(2, 'Bob'),
(3, 'Charlie');

INSERT INTO orders (order_id, customer_id, order_date, amount, items, tags) VALUES
(101, 1, '2025-01-15', 25, '[{"product": "Mouse", "price": 25.00}]', '{"sale"}'),
(102, 1, '2025-06-20', 120,'[{"product": "Webcam", "price": 45.00}, {"product": "Keyboard", "price": 75.00}]', '{"work", "rush_delivery"}'),
(103, 1, '2025-07-01', 15,'[{"product": "Mousepad", "price": 15.00}]', '{}'),
(104, 2, '2025-02-25', 250,'[{"product": "Monitor", "price": 250.00}]', '{}'),
(105, 2, '2025-07-10', 80,'[{"product": "Keyboard", "price": 80.00}]', '{"urgent"}'),
(106, 3, '2025-04-05', 120,'[{"product": "Headphones", "price": 120.00}]', '{"gift", "rush_delivery"}');

The LATERAL Use Cases

So, let's solve some challenges:

1- Selecting Top N Rows per Group

Goal: Find the 2 most recent orders for each customer.

When you encounter this problem, you might think of some available options. Let's examine each option, and finally, we will use the LATERAL option.

Option 1: The Window Function Approach

This is a common way to solve top-N problems, but it requires multiple logical steps.

WITH RankedOrders AS (
    -- Step 1: Scan all orders and assign a rank to each one within its customer group
    SELECT
        c.customer_name,
        o.order_date,
        o.items,
        ROW_NUMBER() OVER(PARTITION BY c.customer_id ORDER BY o.order_date DESC) as rn
    FROM
        customers c
    JOIN
        orders o ON c.customer_id = o.customer_id
)
-- Step 2: Select from the ranked results and filter for the ones you want
SELECT
    customer_name,
    order_date,
    items
FROM
    RankedOrders
WHERE
    rn <= 2;

Why it's less ideal: While it works perfectly, the logic is indirect. You have to create a temporary ranked dataset of all orders and then query it again.

Option 2: The Correlated Subquery Approach

A traditional correlated subquery is often used to fetch a single piece of related data.

-- This only works for a single value, not a full row.
SELECT
    c.customer_name,
    (SELECT o.order_date FROM orders o WHERE o.customer_id = c.customer_id ORDER BY o.order_date DESC LIMIT 1) AS latest_order_date
FROM
    customers c;

Why it fails for this problem: The fundamental limitation of a scalar subquery (one used in a SELECT list) is that it must return exactly one column and one row. It's physically impossible to use this technique to fetch the two full order records (with multiple columns like order_date and items) that our problem requires. It's the wrong tool for the job.

Option 3: The GROUP BY Approach

GROUP BY is built for aggregation, not for retrieving individual rows from a group.

-- This finds the latest date, but loses all other details of that order.
SELECT
    c.customer_name,
    MAX(o.order_date) AS latest_order_date
FROM
    customers c
JOIN
    orders o ON c.customer_id = o.customer_id
GROUP BY
    c.customer_name;

Why it fails for this problem: The entire purpose of GROUP BY is to collapse many rows into a single summary row. It can give you the latest date, the total count, or the average amount, but it discards the individual rows in the process. You cannot use it to select the "top 2" original rows from the group.

Option 4: The LATERAL Solution

SELECT
    c.customer_name,
    recent_orders.order_date,
    recent_orders.items
FROM
    customers c
LEFT JOIN LATERAL (
    SELECT
        o.order_date,
        o.items
    FROM
        orders o
    WHERE
        o.customer_id = c.customer_id
    ORDER BY
        o.order_date DESC
    LIMIT 2
) AS recent_orders ON true;

As you see, it reads like a direct translation of our request: "For each customer, get their two most recent orders."

Tip: The ON TRUE clause is used since we’re not joining on specific columns but including the subquery’s results.

But why is it better than the Window Function approach? The LATERAL approach feels more natural because it integrates the filtering (LIMIT 2) directly into the "for-each-customer" step, avoiding the separate ranking-then-filtering process.

Result:

customer_name	order_date	items
Alice	2025-07-01	`[{"product": "Mousepad", "price": 15.00}]`
Alice	2025-06-20	`[{"product": "Webcam", "price": 45.00}, {"product": "Keyboard", "price": 75.00}]`
Bob	2025-07-10	`[{"product": "Keyboard", "price": 80.00}]`
Bob	2025-02-25	`[{"product": "Monitor", "price": 250.00}]`
Charlie	2025-04-05	`[{"product": "Headphones", "price": 120.00}]`

2- Unpacking JSONB Order Details

Goal: Find all customers who have ever ordered a 'Keyboard'.

SELECT DISTINCT -- Use DISTINCT to only list each customer once
    c.customer_name
FROM
    orders o
JOIN
    customers c ON o.customer_id = c.customer_id
-- For each order 'o', expand its items into a table called 'item_details'
JOIN LATERAL jsonb_to_recordset(o.items) AS item_details(product TEXT, price DECIMAL) ON true
WHERE
    item_details.product = 'Keyboard';

The function jsonb_to_recordset turns a JSON array into a virtual table. LATERAL lets us run this function for every single order row.

Result:

customer_name
Alice
Bob

3- Searching Array Data With UNNEST

Goal: Find the names and order dates for all orders tagged with 'rush_delivery'.

SELECT
    c.customer_name,
    o.order_date,
    tag_value
FROM
    orders o
JOIN
    customers c ON o.customer_id = c.customer_id
-- For each order 'o', expand its tags array into a single-column table
, LATERAL unnest(o.tags) AS tag_value
WHERE
    tag_value = 'rush_delivery';

The unnest function expands an array into a set of rows. LATERAL lets us do this for each order's tags array.

However, PostgreSQL allows you to omit LATERAL in some cases implicitly. When you use a set-returning function like unnest() in the FROM clause with a table column as argument (e.g., o.tags), PostgreSQL treats it as a LATERAL join implicitly. So, in this case you can remove LATERAL and the query will work; but it is best practice to keep

Tip: Keep in mind that the , is a shorthand for CORSS JOIN; which means it should be CROSS JOIN LATERAL.

Result:

customer_name	order_date	tag_value
Alice	2025-06-20	rush_delivery
Charlie	2025-04-05	rush_delivery

4- Using Functions and Calculations per Row

Goal: Apply a 10% discount if the order amount exceeds 100.

SELECT 
    c.customer_name,
    o.amount,
    d.discount
FROM customers c
LEFT JOIN orders o ON c.customer_id = o.customer_id
CROSS JOIN LATERAL (
    SELECT CASE 
        WHEN o.amount > 100 THEN o.amount * 0.1 
        ELSE 0 
    END AS discount
) d;

Result:

customer_name	amount	discount
Alice	25	0
Alice	120	12
Alice	15	0
Bob	250	25
Bob	80	0
Charlie	120	12

LATERAL JOIN vs. Correlated Subqueries

Correlated subqueries and LATERAL JOINs in SQL both allow a subquery to reference columns from a preceding table; however, they differ in their capabilities.

Correlated Subqueries:

A traditional correlated subquery must return a single scalar value (one column, one row) for each row processed by the outer query.
They are typically found in the SELECT list, WHERE clause, or HAVING clause.
It runs repeatedly for each row, which might lead to scanning the target table multiple times.

LATERAL JOIN:

A LATERAL JOIN can return multiple columns and multiple rows from the subquery for each row of the outer query.
It is placed in the FROM clause, acting as a join between the outer table and the result of the LATERAL subquery.
Allows PostgreSQL to plan the subquery execution more efficiently, potentially reducing table scans.

When to Avoid LATERAL?

An expert knows when not to use a tool. LATERAL is a specialized solution, and using it for simpler problems is inefficient and overly complex. Let's introduce some use cases.

Case 1: For the "Top-1" Problem, Use `DISTINCT ON`

If you only need the single most recent record, DISTINCT ON is purpose-built, more declarative, and often faster.

-- Better way to get ONLY the single most recent order
SELECT DISTINCT ON (c.customer_id)
    c.customer_name,
    o.order_date
FROM
    customers c
JOIN
    orders o ON c.customer_id = o.customer_id
ORDER BY
    c.customer_id, o.order_date DESC;

Why it's better: DISTINCT ON is the idiomatic PostgreSQL solution for the "top-1" problem. Its intent is clearer, and the query planner can often optimize it more effectively.

Case 2: For Simple Aggregates, Use `GROUP BY`

If you only need a summary calculation (COUNT, SUM, AVG) and not the details of the individual rows, GROUP BY is the correct, standard tool.

-- Get the total number of orders for each customer
SELECT
    c.customer_name,
    COUNT(o.order_id) AS total_orders
FROM
    customers c
JOIN
    orders o ON c.customer_id = o.customer_id
GROUP BY
    c.customer_name;

Why it's better: This is the most fundamental and efficient way to perform aggregations. Using LATERAL to count rows would be a convoluted anti-pattern.

Performance Considerations

LATERAL can improve performance over correlated subqueries because PostgreSQL’s query planner can optimize them better. However:

Index Usage: Ensure indexes exist on join conditions (e.g., customer_id in the orders table).
Data Volume: For large datasets, test performance to avoid excessive subquery executions.
Query Complexity: Complex LATERAL subqueries may still be resource-intensive if not optimized.

Tip: Use EXPLAIN to analyze the query plan and ensure indexes are used effectively.

CREATE INDEX idx_orders_customer_id ON orders(customer_id);

Conclusion

LATERAL joins in PostgreSQL are a game-changer for handling complex queries with clarity and efficiency. From extracting JSONB data to selecting top N rows per group, they offer flexibility that traditional joins and correlated subqueries often lack.

By understanding when to use LATERAL joins, optimizing performance with indexes, and avoiding overuse in simple scenarios, you can unlock their full potential.

Try experimenting with LATERAL in your next PostgreSQL project, and see how it simplifies your data challenges!

If you’re interested in exploring more, check out these articles.

Postgres SQL FILTER: A Smarter CASE and CTE Alternative

Mohamed Mayallo — Mon, 30 Jun 2025 07:06:00 +0000

Introduction

When you build reports in SQL, you often need to count or average values based on a condition, like counting only completed orders or averaging amounts for canceled ones.

For years, SQL developers relied on CASE statements or complex Common Table Expressions (CTEs) to handle these scenarios. They work, but it gets messy fast.

There's a better way.

PostgreSQL (and some other databases) support the FILTER clause, a clean and efficient feature for conditional aggregation. In this guide, you’ll learn how to use it to simplify your queries, boost performance, and avoid common mistakes.

The Business Problem: An E-commerce Dashboard

Imagine you run an online store. To understand your business's health, you need a monthly report that shows:

The total revenue from all orders.
The revenue generated specifically from "completed" orders.
The revenue lost from "cancelled" orders.

Getting this data in one clean report requires us to aggregate sums based on the status of each order.

Setting Up Our Example

First, let's create the tables we need. We'll have a customers table and an orders table.

-- Create a table for our customers
CREATE TABLE customers (
    customer_id INT PRIMARY KEY,
    customer_name VARCHAR(100),
    join_date DATE
);

-- Create a table for orders
CREATE TABLE orders (
    order_id INT PRIMARY KEY,
    customer_id INT,
    order_date DATE,
    order_status VARCHAR(20), -- e.g., 'completed', 'pending', 'cancelled'
    amount DECIMAL(10, 2),
    FOREIGN KEY (customer_id) REFERENCES customers(customer_id)
);

-- Insert some sample data to work with
INSERT INTO customers (customer_id, customer_name, join_date) VALUES
(1, 'John Smith', '2023-01-15'),
(2, 'Jane Doe', '2023-02-20');

INSERT INTO orders (order_id, customer_id, order_date, order_status, amount) VALUES
(101, 1, '2024-05-05', 'completed', 150.00),
(102, 2, '2024-05-10', 'completed', 200.50),
(103, 1, '2024-05-12', 'cancelled', 75.25),
(104, 2, '2024-06-01', 'completed', 300.00),
(105, 1, '2024-06-03', 'pending', 50.00),
(106, 2, '2024-06-08', 'cancelled', 120.75),
(107, 1, '2024-06-15', 'completed', 250.00);

Our goal is to create the following monthly summary from this data.

month	total_revenue	completed_revenue	cancelled_revenue
2024-05	425.75	350.50	75.25
2024-06	720.75	550.00	120.75

Solution 1: Using the CASE Statement

The most common method for PostgreSQL conditional count and sum was the CASE statement inside an aggregate function.

-- This query uses CASE to conditionally sum the amounts.
-- It works, but it can get messy.
SELECT
    TO_CHAR(order_date, 'YYYY-MM') AS month,
    -- Sum all orders for total revenue
    SUM(amount) AS total_revenue,
    -- Only sum if the status is 'completed'
    SUM(CASE WHEN order_status = 'completed' THEN amount ELSE 0 END) AS completed_revenue,
    -- Only sum if the status is 'cancelled'
    SUM(CASE WHEN order_status = 'cancelled' THEN amount ELSE 0 END) AS cancelled_revenue
FROM
    orders
GROUP BY
    month
ORDER BY
    month;

This query gets the job done. However, for every conditional sum, you have a CASE...WHEN...THEN...ELSE...END block. If you have many conditions, the SELECT list becomes long, hard to read, and difficult to maintain.

Solution 2: Using Common Table Expressions (CTEs)

Another approach is to use CTEs to pre-aggregate each status separately and then join them.

-- This query uses CTEs, which is even more verbose.
-- It breaks the logic into steps, but is much longer.
WITH monthly_totals AS (
    SELECT
        TO_CHAR(order_date, 'YYYY-MM') AS month,
        SUM(amount) AS total_revenue
    FROM orders
    GROUP BY month
),
completed AS (
    SELECT
        TO_CHAR(order_date, 'YYYY-MM') AS month,
        SUM(amount) AS completed_revenue
    FROM orders
    WHERE order_status = 'completed'
    GROUP BY month
),
cancelled AS (
    SELECT
        TO_CHAR(order_date, 'YYYY-MM') AS month,
        SUM(amount) AS cancelled_revenue
    FROM orders
    WHERE order_status = 'cancelled'
    GROUP BY month
)
SELECT
    mt.month,
    mt.total_revenue,
    COALESCE(c.completed_revenue, 0) AS completed_revenue,
    COALESCE(cn.cancelled_revenue, 0) AS cancelled_revenue
FROM
    monthly_totals mt
LEFT JOIN completed c ON mt.month = c.month
LEFT JOIN cancelled cn ON mt.month = cn.month
ORDER BY
    mt.month;

This approach is extremely verbose. It requires multiple scans of the orders table (though the optimizer might fix this) and complex joins. It's simply too much code for such a common task.

The Better Way: The FILTER Clause

Now, let's solve the same problem using the FILTER clause.

-- This is the modern, clean, and efficient way.
-- Notice how much easier it is to read.
SELECT
    TO_CHAR(order_date, 'YYYY-MM') AS month,
    -- Sum all orders for total revenue
    SUM(amount) AS total_revenue,
    -- The FILTER clause applies a WHERE condition to the aggregation
    SUM(amount) FILTER (WHERE order_status = 'completed') AS completed_revenue,
    SUM(amount) FILTER (WHERE order_status = 'cancelled') AS cancelled_revenue
FROM
    orders
GROUP BY
    month
ORDER BY
    month;

The result is identical, but the query is dramatically cleaner. The FILTER (WHERE ...) syntax is attached directly to the SQL aggregate function FILTER, making the intent crystal clear: "Sum the amount, but only for rows that meet this condition." This makes it perfect for creating SQL pivot with filter-like reports.

FILTER vs. CASE: A Closer Look

Readability is a huge win, but what about performance?

In the Postgres FILTER vs CASE debate, according to jOOQ’s benchmark, FILTER often has a performance edge. While logically similar, FILTER provides a more direct instruction to the PostgreSQL query planner.

Why? The CASE statement is a general-purpose expression that can be used anywhere, so the optimizer has to work harder to understand its context within an aggregate function. The FILTER clause, however, was designed specifically for this purpose, allowing for a more direct and optimized execution path.

FILTER vs. CTEs

When it comes to CTE vs FILTER, the choice is even clearer for conditional aggregation. The CTE approach requires multiple subqueries and joins, which adds significant overhead both in terms of code length and planner complexity. The FILTER clause accomplishes the same goal in a single, streamlined scan of the table, making it vastly more efficient and maintainable.

Using FILTER with Multiple Conditions

What if you need to check multiple criteria? The FILTER clause supports standard AND and OR logic inside its WHERE clause, making a PostgreSQL filter multiple conditions check straightforward.

-- Example of filtering on multiple conditions
SELECT
    COUNT(*) FILTER (WHERE order_status = 'completed' AND amount > 200) AS large_completed_orders,
    COUNT(*) FILTER (WHERE order_status = 'pending' OR order_status = 'shipped') AS in_progress_orders
FROM
    orders;

Conclusion

The PostgreSQL FILTER clause is more than just syntactic sugar. It is a powerful tool for writing cleaner, more readable, and more performant SQL. By allowing you to apply specific conditions directly to aggregate functions, it solves the problem of PostgreSQL's conditional aggregation in a way that is far superior to older methods.

For any data analyst or developer working with PostgreSQL, mastering the FILTER clause is essential. It simplifies complex reports, speeds up queries, and makes your SQL code a joy to read and maintain. The next time you find yourself writing a CASE statement inside a SUM() or COUNT(), give FILTER a try. You won't look back.

Finally, if you found this article helpful, you can check more here:

Can GraphQL Call REST API?

Mohamed Mayallo — Wed, 18 Jun 2025 07:00:00 +0000

Introduction

Yes, GraphQL can call REST APIs. In fact, this is a common way to use GraphQL in real projects. It lets you use your existing REST services while giving frontend apps the flexibility of GraphQL.

In this article, you’ll learn how to call REST from GraphQL using a very simple and practical method.

Why Use GraphQL with REST?

Many apps already have REST APIs. Instead of replacing them completely, you can wrap those APIs with GraphQL. This lets frontend developers ask only for the data they need, without changing your backend.

You get the best of both worlds:

Use existing REST endpoints
Expose a cleaner GraphQL API to clients

Let’s Build It Step-by-Step

We’ll create a GraphQL server using Node.js, and we’ll call a public REST API:

https://jsonplaceholder.typicode.com/users/:id

This is a fake REST API that returns fake user data. It’s perfect for testing.

Step 1: Set Up Project

Make a new folder and install the packages:

mkdir graphql-rest-example
cd graphql-rest-example
npm init -y
npm install express express-graphql graphql axios

Step 2: Create the GraphQL Server

Create a file called index.js:

const express = require('express');
const { graphqlHTTP } = require('express-graphql');
const { buildSchema } = require('graphql');
const axios = require('axios');

// Step 1: Define GraphQL schema
const schema = buildSchema(`
  type User {
    id: ID
    name: String
    username: String
    email: String
  }

  type Query {
    user(id: ID!): User
  }
`);

// Step 2: Define resolver that calls REST API
const root = {
  user: async ({ id }) => {
    const response = await axios.get(`https://jsonplaceholder.typicode.com/users/${id}`);

    const user = await response.data;
    return {
      id: user.id,
      name: user.name,
      username: user.username,
      email: user.email
    };
  }
};

// Step 3: Create Express server
const app = express();
app.use(
  '/graphql',
  graphqlHTTP({
    schema,
    rootValue: root,
    graphiql: true // Enable UI
  })
);

app.listen(4000, () => {
  console.log('GraphQL server running at http://localhost:4000/graphql');
});

Step 3: Run the Server

Start the server:

node index.js

Then open your browser and go to:

http://localhost:4000/graphql

You’ll see a GraphQL UI where you can run this query:

{
  user(id: 1) {
    id
    name
    email
  }
}

It will return this result:

{
  "data": {
    "user": {
      "id": "1",
      "name": "Leanne Graham",
      "email": "Sincere@april.biz"
    }
  }
}

This proves that you can call a REST API from GraphQL using a resolver.

Conclusion

If you were wondering can GraphQL call REST API? Now you know the answer is yes, and it's quite simple. By calling REST endpoints from your resolvers, you can keep your backend logic and still enjoy the benefits of GraphQL on the frontend.

This method is clean, fast to implement, and great for gradually modernizing your stack.

Finally, if you found this article useful, you can check more here:

Are Nx Monorepo Configurations Really Complex?

Mohamed Mayallo — Tue, 17 Jun 2025 17:57:02 +0000

Introduction

This is the second part of our Nx monorepo series. If you haven't yet, you might want to check out the first article, Nx Monorepo Guide: React & Node Fullstack App, where I talked about setting up an Nx workspace.

Now, we'll dive into something more specific, yet very important: how TypeScript works inside Nx. Many developers, myself included, often get lost in all the tsconfig files. Do you find yourself wondering what each one does? I know it can be too much to handle at first.

Why Nx TypeScript Configuration Feels Complex

The complexity comes from Nx managing TypeScript across dozens of apps and libraries. Unlike traditional single-project setups with one tsconfig.json, Nx monorepos can have 50+ configuration files scattered throughout your workspace.

The challenge is that each project needs its own settings while sharing common configurations. This creates a web of interconnected config files that can feel overwhelming when you're just trying to import a utility function.

The good news is that Nx handles most of this complexity automatically for you, but understanding how things work under the hood helps you troubleshoot issues and make informed decisions about your workspace structure.

Common Configuration Pitfalls

Treating configs as independent: Developers sometimes copy settings between projects, unaware that Nx relies on inheritance, so their changes can unintentionally impact the entire workspace.
Manually editing path mappings: Adding paths manually to fix import errors, when Nx auto-generates these mappings based on the workspace structure.
Circular dependencies: Creating innocent-looking imports between libraries that later cause dependency cycles with cryptic error messages.
Build vs runtime confusion: Setting up configs perfectly for development, then wondering why production builds fail or deployed apps can't resolve modules.

The Inheritance Chain Explained

Think of Nx TypeScript configuration like a family tree. At the workspace root, tsconfig.base.json sets defaults and path mappings that flow down to every project.

Base level: Workspace-wide settings that all projects inherit
Project level: Each app/library extends the base config with its own specific needs
Purpose-specific: Individual projects often have multiple configs (build, test, type-checking) that extend from their main project config

When you change the base configuration, it affects every project that inherits from it. This is powerful but requires careful thinking about where changes should be made.

Understanding the Configuration Files

Nx organizes TypeScript configurations for good reason. It wants to make sure different parts of your monorepo can share code and build efficiently. It is very smart about this. Moreover, this organization prevents conflicts and ensures consistency.

Root Level `tsconfig.base.json`: The Global Foundation

The tsconfig.base.json file is the cornerstone of your workspace's TypeScript configuration. It's where you define the global settings that will be inherited by all the applications and libraries within your monorepo. This is the place to specify:

Common compiler options: Settings like target, module, strict, and esModuleInterop that you want to apply universally.
Path aliases: To simplify import statements across your projects, you can define path mappings here. For example, you can map @my-org/my-lib to the actual path of the library, making it easier to reference shared code.

By centralizing these common configurations, you ensure consistency and reduce redundancy across your entire codebase.

Root Level `tsconfig.json`: Enabling Project References

More recently, Nx has introduced a root-level tsconfig.json to leverage TypeScript project references. This feature helps to improve build times and enforce clearer boundaries between your projects.

The primary role of this root tsconfig.json is to enumerate all the projects (applications and libraries) in your workspace. It contains a references array that points to the tsconfig.json file of each individual project.

This allows TypeScript-aware tools and editors to understand the dependency graph of your monorepo, leading to more efficient compilation and a better development experience.

Project Level `tsconfig.json`: The Project Entry Point

Every project (be it an application or a library) has its own tsconfig.json file. This file acts as the main configuration entry point for that specific project.

It typically contains references to the other specialized tsconfig.*.json files within the same project directory, such as tsconfig.app.json or tsconfig.lib.json, and tsconfig.spec.json. Or it might include references to the tsconfig.*.json files from the other dependent projects.

Project Level `tsconfig.app.json` & `tsconfig.lib.json`: Application and Library Specifics

Depending on whether you've generated an application or a library, you will find either a tsconfig.app.json or a tsconfig.lib.json file. These files are tailored for the specific needs of their respective project types:

tsconfig.app.json: This file contains TypeScript compiler options that are specific to an application. For instance, it might include settings related to the application's build output or specific environment requirements.
tsconfig.lib.json: Similarly, this file holds configurations that are particular to a library. This could include settings for generating declaration files (.d.ts) or other library-focused compiler options.

Both of these files extend the root tsconfig.base.json and can override any of the global settings as needed for that particular project.

Project Level `tsconfig.spec.json`: Fine-Tuning for Tests

For your test files, Nx generates a tsconfig.spec.json. This configuration file is specifically designed for your testing environment. It allows you to have a distinct TypeScript setup for your tests, which can be useful for:

Including test-specific files and type definitions.
Configuring a different module system or other compiler options that are better suited for your testing framework (like Jest or Vitest).

Like the other project-level tsconfig files, tsconfig.spec.json also extends the base configuration, ensuring that your tests are compiled with the necessary settings without affecting your application or library source code.

Root Level `nx.json`: The Workspace "Brain"

At the root of your workspace, the nx.json file acts as the central configuration hub or the "brain" for the Nx CLI. While tsconfig.base.json manages global TypeScript settings, nx.json manages workspace-wide operational settings.

Its key responsibilities include:

Task Dependencies: Defining the dependency order for tasks. For example, ensuring a library is always built before an application that uses it.
Caching Configuration: Specifying which tasks are cacheable. This is a core feature of Nx that saves immense amounts of time by not re-running tasks (like builds or tests) if the source code hasn't changed.
Default Runners and Executors: Setting the default tools used to run tasks across your projects.
Plugins and Generators: Configuring Nx plugins that extend the workspace's capabilities, like adding support for new frameworks or tools.

In short, nx.json orchestrates how Nx operates on a global level, making your development workflow fast and consistent.

Project Level `project.json` or `package.json`: The Project's Instruction Manual

Inside each application and library folder, you'll find a file that serves as the instruction manual for that specific project. This configuration tells Nx everything it needs to know about the project, most importantly, what "targets" it has. A target is simply a task that you can run, such as:

build: To compile the application or library.
test: To execute unit tests.
lint: To check for code style and errors.

This configuration can be defined in one of two ways. One approach is a dedicated project.json file. Alternatively, the configuration can live inside the project's package.json file, under a special "nx": {} property. Both methods achieve the same goal and are valid ways to configure a project in a workspace.

For each target, the configuration defines the executor (the tool that runs the task, like Webpack or Jest) and any specific options that tool needs to operate on that project.

Nx's Superpower: Inferring Tasks Automatically

Here is where Nx truly shines. While you can explicitly define every task for every project, you often don't have to. Nx has the powerful ability to infer tasks automatically by detecting the presence of tool-specific configuration files.

For example, if you add a jest.config.ts file to your project, Nx will see it and automatically create a test target for you behind the scenes. If you add an .eslintrc.json file, a lint target is instantly made available. This works for a wide range of common development tools.

This inference capability is a key advantage, as it dramatically reduces boilerplate configuration and keeps your project.json files lean. You only need to manually define targets when you have a complex setup or need to override the inferred behavior. It’s a smart system that provides structure when you need it and gets out of your way when you don't.

[newsletter_form form="2"]

Path Aliases vs. Project References: Which Should You Choose?

This is a common question in Nx and TypeScript in general. Both path aliases and project references help you organize imports. However, they do it in very different ways, and have different benefits.

What are Path Aliases?

Path aliases let you create shortcuts for import paths. Instead of a long relative path like ../../../libs/my-lib/src, you can use @my-org/my-lib.

This makes your import statements much cleaner and easier to read. I always use them; they really improve code readability. They're like giving your home a nickname instead of saying its full address every time.

// Before Path Alias
import { Button } from '../../../../packages/shared-ui/src/lib/shared-ui'; // This path is long and hard to read.

// After Path Alias (defined in tsconfig.base.json)
import { Button } from '@my-awesome-nx-repo/shared-ui'; // This is much shorter and clearer, thanks to the alias.

Pros:
- Cleaner import statements.
- Easier to move files around without updating import paths.
- Works well for smaller monorepos or just for readability.
Cons:
- TypeScript treats them as simple text replacements; no strong compile-time checks for module existence.
- Might not leverage Nx's caching as effectively for builds.

What are TypeScript Project References?

TypeScript Project References are a more robust way to link projects. They allow TypeScript to understand the dependencies between different projects. This means TypeScript can check types across project boundaries. Also, it only recompiles changed projects, which saves a lot of time. This is a game-changer for large monorepos.

To use project references, you add a references array to your tsconfig.json files.

// apps/my-react-app/tsconfig.json
{
  "extends": "../../tsconfig.base.json",
  "files": [],
  "include": [],
  "references": [
    {
      "path": "../../packages/shared-ui" // This links to the 'shared-ui' project. TypeScript will understand this project's types and changes.
    }
  ]
}

Pros:
- Improved build performance through incremental builds.
- Stronger type checking across projects.
- Better IDE support for navigation and refactoring.
- Essential for very large workspaces, as Nx's build system uses these for the dependency graph.
Cons:
- Can be more complex to set up initially.
- Requires all referenced projects to also be TypeScript projects.

The Verdict: Path Aliases vs. Project References

For small projects or just for making imports pretty, path aliases are fine. However, for most Nx monorepos, especially bigger ones, you should definitely use TypeScript Project References.

Nx itself heavily relies on them for its build system and caching. It’s the expert choice for performance and type safety.

Troubleshooting Common TypeScript Issues

Even with a good setup, you might face issues. I've spent countless hours debugging weird TypeScript errors in Nx. Here are some common problems and how to fix them.

Import Resolution Problems

One of the most common issues is TypeScript not being able to find your imports. This usually looks like: "Cannot find module '@my-org/my-lib' or its corresponding type declarations."

Possible Causes and Solutions:

Incorrect paths in tsconfig.base.json: Double-check your path aliases. Make sure the path actually points to the src/index.ts of your library.

// Check this part carefully
"paths": {
  "@my-org/my-lib": ["libs/my-lib/src/index.ts"] // Ensure this path correctly points to your library's entry file.
}

Missing Project Reference: If you're using project references, ensure the references array in the consumer project's tsconfig.json correctly lists the library.

// apps/my-react-app/tsconfig.json
"references": [
  { "path": "../../packages/shared-ui" } // Verify the path to the referenced library is accurate.
]

Caching Issues: Sometimes Nx's cache can get stale. A simple nx reset or rm -rf node_modules dist .nx/cache followed by npm install (or yarn) and nx build <project-name> often solves it.

Build Errors and Solutions

Sometimes TypeScript compiles fine, but Nx build commands fail. Or you get runtime errors related to types.

Compiler Errors: Check your tsconfig.app.json or tsconfig.lib.json files. Are your include and exclude arrays correct? Sometimes a test file gets included in a build.

// Ensure test files are excluded from build
"exclude": ["jest.config.ts", "src/**/*.test.ts"] // Prevents test-related files from being bundled into your production build.

Version Mismatches: Ensure your TypeScript version is consistent across your workspace. Using different versions can cause strange issues. Check your package.json.
Circular Dependencies: If project A depends on project B, and B depends on A, TypeScript (and Nx) will struggle. Nx helps detect this, but it's best to fix the code structure.

IDE Integration Issues

Your IDE (like VS Code) uses tsconfig.json files to give you autocompletion and error hints. If your IDE isn't working correctly, it's often a configuration problem.

Restart IDE: The simplest fix. Close and reopen VS Code.
Workspace tsconfig.json: Make sure the root tsconfig.json correctly references all projects. This is what your IDE uses to get a global view.
Nx Console extension: If you're using VS Code, install the Nx Console extension. It helps a lot with understanding and managing Nx projects.

Caching and Performance in Nx

Nx is famous for its performance, and caching is a big part of that. Understanding how it works with TypeScript projects can significantly speed up your development.

How Nx Caching Works

Nx uses a computation cache. When you run a command (like nx build my-app), Nx first checks if it has already built that project with the same inputs. If yes, it just restores the output from its cache instead of running the build again. This includes source files, package.json dependencies, tsconfig files, and even environment variables. It saves so much time.

Inputs: The files and configurations that define a task's output.
Outputs: The generated files (e.g., dist folder).
Hashing: Nx computes a hash of all inputs. If the hash hasn't changed, the output is pulled from cache.

To check what Nx is caching:

nx show cache # Displays general information about the Nx cache.
nx graph --files-json=true # Generates a JSON representation of the dependency graph, which influences caching decisions.

Performance Tuning Your TypeScript Project

You can optimize your Nx TypeScript projects for even better performance.

Leverage Project References: As discussed, project references enable incremental builds, where only affected projects are recompiled. This is a huge performance gain.
Correct include/exclude: Don't include unnecessary files in your tsconfig files. For example, don't include test files in your main build configuration.
Optimize compilerOptions:
- skipLibCheck: true: Speeds up compilation by skipping type checking of declaration files from node_modules. Most times, this is safe to use.
- isolatedModules: true: Ensures every file can be compiled independently, which is good for tools like Babel.
Clean node_modules: Occasionally clear your node_modules and reinstall. Sometimes old packages or faulty caches can slow things down.
Use the Daemon: Nx Daemon runs in the background and keeps a dependency graph in memory, making subsequent commands faster. It's usually on by default.

// nx.json - Ensure caching is enabled for targets
{
  "tasksRunnerOptions": {
    "default": {
      "runner": "nx/tasks-runners/default",
      "options": {
        "cacheableOperations": ["build", "test", "lint", "e2e"] // Explicitly marks these tasks as cacheable, so Nx will store their outputs for faster re-execution.
      }
    }
  }
}

cacheableOperations: Lists which tasks Nx should try to cache.

Conclusion

Nx TypeScript configuration might seem daunting at first, but once you understand the inheritance pattern and common pitfalls, it becomes much more manageable. Remember that Nx does the heavy lifting automatically - your job is simply to understand the system well enough to troubleshoot when things go wrong and make informed decisions about your workspace structure.

The key is starting simple and gradually building complexity as your monorepo grows. With these fundamentals in place, you'll spend less time fighting configuration issues and more time building great applications.

If you want to dive deeper, have a look at these resources:

Think about it

If you enjoyed this article, I’d truly appreciate it if you could share it—it really motivates me to keep creating more helpful content!

If you’re interested in exploring more, check out these articles.

Thanks for sticking with me until the end—I hope you found this article valuable and enjoyable!

SQL COALESCE in Postgres: A Simple Guide

Mohamed Mayallo — Mon, 09 Jun 2025 16:15:11 +0000

Introduction

In a database, we often find empty places in our tables. These are not zeros or spaces; they are special markers called NULL. A NULL value means "no data." This can be a problem.

Let's see how the SQL COALESCE function can help us solve this common issue using Postgres examples.

Our Small Shop Example

Imagine you have a small online shop. You have a table in your database called products. This table stores information about your products, like their name, their normal price, and a special discount price.

First, let's create our products table.

CREATE TABLE products (
    id SERIAL PRIMARY KEY,
    product_name VARCHAR(100),
    regular_price NUMERIC(10, 2),
    discount_price NUMERIC(10, 2)
);

Now, let's add some data to our table. Notice that some products have a discount_price and some do not (their discount_price is NULL).

INSERT INTO products (product_name, regular_price, discount_price) VALUES
('Cool T-Shirt', 20.00, 15.00),
('Jeans', 50.00, NULL),
('Hat', 15.00, 12.50),
('Socks', 8.00, NULL);

Now our table is ready. We have some products with discounts and some without.

The Problem with NULL

Let's say you want to show your customers the final price. If a product has a discount, you show the discount_price. If it does not, you show the regular_price.

You might try to select the discount_price like this:

SELECT product_name, discount_price FROM products;

The result will look like this:

product_name	discount_price
Cool T-Shirt	15.00
Jeans	NULL
Hat	12.50
Socks	NULL

This is a problem. You cannot show "NULL" to your customers on your website. They will be confused. You need to show the regular_price when the discount_price is NULL.

A First Try: The CASE Statement

One way to solve this is using a CASE statement. The CASE statement goes through conditions and returns a value when the first condition is met.

Here is how you can use CASE to show the correct price:

SELECT
    product_name,
    CASE
        WHEN discount_price IS NOT NULL THEN discount_price
        ELSE regular_price
    END AS final_price
FROM
    products;

This query says: check the discount_price. If it is not NULL, then use it. If it is NULL, then use the regular_price.

This works! The result is:

product_name	final_price
Cool T-Shirt	15.00
Jeans	50.00
Hat	12.50
Socks	8.00

But the CASE statement can be long to write, especially if you have many conditions. This brings us to a better way. When we compare PostgreSQL Coalesce vs Case, we see that COALESCE is much simpler for this task.

The Best Solution: The COALESCE Function

The COALESCE function in PostgreSQL is a perfect and simple solution for this problem. It returns the first non-NULL value from a list of arguments.

The syntax is very easy: COALESCE(argument_1, argument_2, ...)

Let's write our query again, but this time using COALESCE.

SELECT
    product_name,
    COALESCE(discount_price, regular_price) AS final_price
FROM
    products;

This query does the same thing as the CASE statement. It looks at discount_price first. If discount_price is not NULL, it returns that value. If it is NULL, it moves to the next argument, which is regular_price, and returns that value.

The result is exactly the same, but the query is much shorter and easier to read.

product_name	final_price
Cool T-Shirt	15.00
Jeans	50.00
Hat	12.50
Socks	8.00

Using COALESCE with Many Values

The power of COALESCE does not stop there. You can use it with many values. This is helpful when you need to coalesce multiple columns.

Let's add a new column to our table called clearance_price.

ALTER TABLE products ADD COLUMN clearance_price NUMERIC(10, 2);

UPDATE products SET clearance_price = 10.00 WHERE product_name = 'Cool T-Shirt';

Now, the T-Shirt has a regular_price, a discount_price, and a clearance_price. We want to show the clearance_price if it exists. If not, the discount_price if it exists. If not, then the regular_price.

COALESCE makes this very easy.

SELECT
    product_name,
    COALESCE(clearance_price, discount_price, regular_price) AS final_price
FROM
    products;

The function will check each column in order (clearance_price, then discount_price, then regular_price) and return the very first price that is not NULL.

Conclusion

The SQL COALESCE function is a powerful tool in PostgreSQL. It helps you write clean, simple, and readable SQL. When you need to replace NULL with a default value, COALESCE is often the best choice.

It is simpler than writing long CASE statements and makes your code better. So next time you have a problem with NULL values, remember the SQL COALESCE function.

Finally, if you found this article useful, you can check more here:

Nx Monorepo Guide: React & Node Fullstack App

Mohamed Mayallo — Thu, 05 Jun 2025 07:00:00 +0000

What is a Monorepo and Why Nx?

Ever find yourself checking many projects, each in its own GitHub repo, making updates feel like fixing one thing just makes another pop up? I know I used to.

It's often difficult to keep track of all the different versions of things or remember where that one shared piece of code actually lived. This scattered setup really tests your patience, doesn't it? Well, there's another way.

Understanding Monorepo Architecture

A monorepo holds all code for many projects in one single version control repository. All the different parts of a big system, like a website, an API, or even a mobile app, live side-by-side. For example, Google, Facebook, and Microsoft all use monorepos for parts of their massive codebases. This approach can really speed up things.

Consider a shared component library. In a traditional setup, you'd publish it to npm, then each project would install it. With a monorepo, you just import it directly, no publishing needed. This removes so much friction! However, managing changes across many projects in one place needs special tools. Otherwise, it becomes a mess, this is where Nx shines.

What Makes Nx Special for Developers

Smart monorepo tool: Nx is more than just a basic tool; it’s designed for managing monorepos, especially for JavaScript and TypeScript projects.
Understands project structure: Nx maps out how your projects are connected and knows exactly what needs to be rebuilt when you make even a small change.
Saves time and resources: By rebuilding only what’s necessary, Nx speeds up builds and testing, saving time and computing power.
Computation caching: Nx remembers tasks (like builds or tests) it has already done; if the code hasn’t changed, it skips the task and uses cached results instead.
Built-in code generators and plugins: Supports popular tools like React, Angular, Node.js, and Jest, which helps keep code clean, consistent, and easy to maintain across your entire project.
Supports clean testing setup: Nx makes it easier to set up and manage your testing framework.

Nx vs Other Monorepo Tools Comparison

Many tools support monorepos, but they offer different levels of functionality. Here's how Nx compares to some of the most popular options.

Nx vs Lerna

Lerna was one of the first monorepo tools, focusing on managing and publishing packages. Nx goes much further by including a full build system and more.

Key differences:

Build system: Nx includes smart task orchestration and caching; Lerna does not.
Code generation: Built into Nx; not available in Lerna.
Dependency graph: Nx understands task relationships; Lerna tracks only package dependencies.
Plugin ecosystem: Nx supports many frameworks and tools; Lerna is limited to package-related tasks.
CI/CD integration: Nx has built-in support for affected-only commands; Lerna needs manual setup.
Development activity: Nx is actively maintained and growing; Lerna is stable but less active.

Summary: Nx is an all-in-one solution, while Lerna often requires other tools to match the same functionality.

Nx vs Turborepo

Turborepo is known for its speed, especially with caching and parallel task execution. Nx offers similar performance but provides a broader developer experience.

Key differences:

Focus: Turborepo is minimal and fast; Nx is full-featured and flexible.
Tooling: Nx includes code scaffolding, plugins, and workspace generators; Turborepo does not.
Use case: Turborepo is great if you already have a strong setup; Nx helps guide structure from the start.

Summary: Turborepo is like a high-speed engine; Nx is a full vehicle ready for production use.

Nx vs npm Workspaces

npm Workspaces help link local packages inside a monorepo, letting you manage dependencies across projects.

Key differences:

Scope: npm Workspaces handle package linking only; Nx adds intelligent build orchestration and caching.
Features: Nx provides code generation, task execution, dependency graphs, and CI optimizations—none of which are part of npm Workspaces.
Integration: Nx works with npm Workspaces (and pnpm/Yarn) as a base, then builds on top of them.

Summary: npm Workspaces handle the basics; Nx turns that foundation into a powerful monorepo platform.

Setting Up Your First Nx Workspace

Ready to get your hands dirty? Setting up an Nx workspace isn't hard, I promise. It's a quick process to get a powerful development environment ready. You'll soon see how easy it is to manage different apps and libs in one place.

Please find the complete code here.

Creating Your Nx Workspace with CLI

Now, let's create our first Nx Monorepo. It's done with a single command. Open your terminal or command prompt and type this:

npx create-nx-workspace@latest my-awesome-nx-repo

npx: This runs the create-nx-workspace package without installing it globally.
create-nx-workspace@latest: Ensures you get the most recent version of the Nx workspace generator.
my-awesome-nx-repo: This is the name of your new workspace folder. You can name it whatever you like.

When you run this, Nx will ask you a few questions:

"Need to install the following packages: create-nx-workspace": Yes, if you want to install it globally.
"Which stack*?"*: None, will set up each app individually.
"Prettier for code formatting?" Yes.
"Which CI provider?": Do it later.
"Remote caching using Nx Cloud?": No.

After this, Nx will install everything. It takes a minute or two. You now have a working Nx workspace! Go into the folder: cd my-awesome-nx-repo.

Essential Nx Configuration Files Explained

Nx uses a few main configuration files to know how to build, test, and lint your projects. Understanding these files helps you work better with Nx.

nx.json: This is the heart of your Nx workspace configuration. It defines projects, their types (application or library), and how they relate to each other. It also sets up caching strategies and task runners.
package.json: This file, at the root of your workspace, lists all the global dependencies for your entire monorepo. Instead of each app having its own node_modules, Nx (via Yarn/pnpm/npm workspaces) often hoists them to the root.
tsconfig.base.json: This is the top-level TypeScript configuration. It defines common settings for all TypeScript projects in your workspace. Keep in mind, every project has its own tsconfig file to override this top configuration file.
tsconfig.json: Acts as the project references entry point for TypeScript in a monorepo. Its references value lists all apps and libs, so TypeScript understands the project graph and can do incremental builds.

Nx CLI Commands Every Developer Should Know

Nx uses a command line interface (CLI) to manage your workspace. These commands are your daily tools for working with Nx.

nx generate [generator] or nx g [generator]: Creates new applications, libraries, components, etc.
- Example: nx g @nx/react:app my-new-react-app (creates a new React app)
- Example: nx g @nx/js:lib shared-data (creates a new JavaScript/TypeScript library)
nx serve [project-name]: Starts the development server for a specific application.
- Example: nx serve my-react-app
nx build [project-name]: Builds a project for production.
- Example: nx build my-express-api
nx test [project-name]: Runs tests for a project.
- Example: nx test my-react-app
nx lint [project-name]: Runs linting checks for a project.
- Example: nx lint shared-ui
nx graph: Visualizes the dependency graph of your workspace in your browser. This is super cool and really helps you understand how things connect.
nx affected:[command]: Runs a command only on projects that have changed since your last commit (or a specific commit). This is where Nx's intelligence truly shines for CI/CD.
- Example: nx affected:build (builds only changed projects)
- Example: nx affected:test (tests only changed projects)

These commands, especially nx affected, are why large monorepos with Nx can still have fast CI pipelines. You only build and test the parts that are truly impacted.

Note: If you are using VS Code, there is a very handy extension that will make your life way easier; it is Nx Console; believe me, you will like it.

Creating Your First Node.js Backend App

Now that we have a workspace, let's add a Node.js backend. This is often the first step for a lot of web applications. You'll see how easy it is to add new parts to your Nx Monorepo.

Generating a Node.js Application

We can generate a new Node.js application using the Nx CLI. This will create a basic Express.js app for us. It sets up all the boilerplate, so you don't have to. It's truly a time saver.

npm i @nx/node -D  # Install the needed plugin
npx nx generate @nx/node:app packages/my-backend --framework=express # Setup our backend

@nx/node:app: This is the generator for creating Node.js applications.
packages/my-backend: The name of your new Node.js application inside the packages directory.
--framework=express: Tells Nx to set up the app using the Express.js framework. You could also choose nest or none.

Nx will add a new folder inside packages/ named my-backend, complete with a basic src/ directory and setup files. It also adds a new entry in packages.json workspaces for my-backend (or you will find "packages/*") and updates tsconfig.json with a path reference.

Note: You can list the available plugins using this command nx list.

Understanding the App Structure

Inside packages/my-backend, you'll find a simple Express app structure. It's designed to be clean and extensible.

packages/my-backend/
├── src/*
├── package.json
├── .spec.swcrc
├── tsconfig.json
├── tsconfig.app.json
└── tsconfig.spec.json

src/: Your main app.
package.json: The "nx": { "targets": { … } } section in package.json allows you to define, configure, and customize the various tasks (targets) related to your Node.js app. This is key for automating workflows like building, testing, serving, and linting within an Nx workspace.
.spec.swcrc: The configuration file used by SWC to optimize and configure the transpilation of your test files during unit tests.
tsconfig.json: Base TypeScript configuration for the app (common settings for all files in the app).
tsconfig.app.json: TypeScript configuration specifically for compiling the app's source code (src/*.ts).
tsconfig.spec.json: TypeScript configuration for compiling the app’s test files (src/*.spec.ts).

Mostly, you will find these files in all of your newly generated applications or libraries.

Setting Up Basic Express Server

Nx already generates a basic Express server for you. To run it, simply use: npx nx serve my-backend. You'll see the server start up and listen on http://localhost:3000.

If you navigate there in your browser, you should see the "Hello API" message. Pretty neat, right?

Adding React Components to Your Monorepo

What about adding a frontend? Nx makes it just as easy to add React applications and shared React components. This is where the true power of a monorepo starts to show itself.

Creating a Shared React Library

Let's make a shared library for our React components. This library will live in libs/ and can be used by any React application in our monorepo. It’s like building a reusable Lego set that many different structures can use.

npm i @nx/react -D  # Install the needed plugin
npx nx generate @nx/react:lib libs/shared-ui --style=css

@nx/react:lib: This generator creates a React library.
shared-ui: The name of our new library in the libs folder.
--style=css: Specifies CSS as the styling approach. You could choose scss, less, or styled-components too.

Nx creates libs/shared-ui with a basic component, tests, and its own configuration files.

Building Your First Reusable Component

Inside libs/shared-ui/src/lib/ you'll find a shared-ui.tsx file (or similar, depending on Nx version). Let's modify it to be a simple Button component.

/* libs/shared-ui/src/lib/shared-ui.tsx */
import styles from './shared-ui.module.css'; // Assuming you have a CSS module

/* eslint-disable-next-line */
export interface ButtonProps {
  text: string;
  onClick: () => void;
}

export function Button(props: ButtonProps) {
  return (
    <button className={styles['my-button']} onClick={props.onClick}>
      {' '}
      {/* Render a button */}
      {props.text} {/* Display button text */}
    </button>
  );
}

export default Button; // Export for easier import

And a simple CSS file to go with it:

/* libs/shared-ui/src/lib/shared-ui.module.css */
.my-button {
  background-color: blue;
  color: white;
  padding: 10px 15px;
  border-radius: 5px;
  cursor: pointer;
  border: none;
}

.my-button:hover {
  opacity: 0.8;
}

Now, other projects can use this Button component.

Creating A New React App

Similar to our backend app, let's create the frontend app.

npx nx generate @nx/react:app packages/my-react-app --style=css

Note: if you encountered an error like this:

An error occurred while processing files for the @nx/vite/plugin
libs/shared-ui/vite.config.ts: Cannot find module 'ajv/dist/core'

Simply, run this command npm install ajv.

Importing Components Across Apps

To use our new Button component in my-react-app, we simply import it using its alias defined in package.json name property; @my-awesome-nx-repo/shared-ui.

First, find packages/my-react-app/src/app/app.tsx (the main app file).

// packages/my-react-app/src/app/app.tsx
import React from 'react';
import { Button } from '@my-awesome-nx-repo/shared-ui'; // Import our shared button
import './app.module.css';

export function App() {
  const handleClick = () => {
    alert('Button clicked from shared library!');
  };

  return (
    <div>
      <h1>Welcome to my-react-app!</h1>
      <Button text="Click Me" onClick={handleClick} /> {/* Use the shared button */}
      <p>This button comes from a shared library.</p>
    </div>
  );
}

export default App;

Now, if you run nx serve my-react-app, you'll see your React app with the shared button. Any change to libs/shared-ui will instantly update in my-react-app during development.

This is because Nx knows the dependency and rebuilds only what's needed. It's a huge time-saver and makes developing complex UIs much more manageable.

Hot Reloading and Development Workflow

Nx provides a smooth development experience with hot reloading. When you run nx serve [project-name], any changes you make to that project, or any library it depends on, will trigger a recompile and refresh your browser (for frontend apps) or restart your server (for backend apps). It is quite magical.

This workflow is incredibly efficient. You modify a shared UI component in libs/shared-ui, and all React apps that use it automatically update. You don't need to rebuild libraries manually or run separate watch commands. Nx handles it all. This rapid feedback loop is a big win for productivity.

[newsletter_form form="2"]

Understanding Nx Workspace Architecture

Photo by Lucas Santos on Unsplash

To truly master Nx, you need to grasp its architectural ideas. It's more than just a collection of files; it's a way of thinking about how the different parts of a software system relate.

Apps (Packages) vs Libraries: When to Use Each

This is a core concept in Nx. Knowing when to make something an app and when it should be a lib is important. It's not just about what it is, but what it does.

Applications (packages/ or apps/):
- Purpose: These are the deployable units of your workspace. They are what users interact with, or what gets deployed to a server.
- Examples: A React web app, a Node.js API server, an Electron desktop app, a mobile app.
- Characteristics: They usually depend on libraries, but other applications generally don't depend on them. They often contain specific configurations for deployment.
Libraries (libs/):
- Purpose: These are reusable, shareable units of code within your workspace. They are meant to be consumed by applications or other libraries.
- Examples: UI component library, data access layer, utility functions, shared types, authentication logic.
- Characteristics: They should be cohesive and focused on a single responsibility. They often depend on other libraries but should generally not depend on applications. This promotes modularity and reusability.

Choosing correctly ensures a cleaner architecture. If you find yourself copying code between applications, you probably need a new library. If a project contains UI components and API logic, it should probably be split into a UI library and an API application. This enforces better separation of concerns.

Dependency Graph Visualization

One of Nx's most powerful features is its ability to build and visualize the dependency graph of your workspace. It maps out how every project (app or lib) connects to others. This graph is not just for show; Nx uses it to figure out what needs to be built or tested when you make a change.

To see your graph, run:

npx nx graph

This command opens a browser window showing a visual representation of your projects and their dependencies. You can click on projects to see their specific connections.

Module Boundaries and Import Restrictions

Nx lets you enforce rules about how projects can import each other. This is called "Module Boundaries." It helps maintain a healthy, layered architecture and prevents spaghetti code. For example, you might want to stop a frontend UI library from importing a backend database library. This is a common problem in big systems, and Nx helps prevent it.

You define these rules in your root .eslint.config.mjs file using enforceBuildableLibDependency and depConstraints.

// .eslint.config.mjs example showing depConstraints

// other configs
{
  files: ['**/*.ts', '**/*.tsx', '**/*.js', '**/*.jsx'],
  rules: {
    '@nx/enforce-module-boundaries': [
      'error',
      {
        enforceBuildableLibDependency: true,
        allow: ['^.*/eslint(\\.base)?\\.config\\.[cm]?js$'],
        depConstraints: [
          // Here are our rules!
          {
            sourceTag: 'scope:frontend', // Projects with this tag
            onlyDependOnLibsWithTags: ['scope:shared', 'scope:frontend'], // Can only depend on these tags
          },
          {
            sourceTag: 'scope:backend',
            onlyDependOnLibsWithTags: ['scope:shared', 'scope:backend'],
          },
          {
            sourceTag: 'scope:shared',
            onlyDependOnLibsWithTags: ['scope:shared'], // Shared libs can only depend on other shared libs
          },
        ],
      },
    ],
  },
},
// other configs

To make these rules work, you add "tags" to your projects in their package.json files.

// packages/my-react-app/project.json
{
  "name": "@my-awesome-nx-repo/my-react-app",
  ... // other details
  "nx": {
    "tags": ["scope:frontend"] // Apply tags here
  }
}

// libs/shared-ui/project.json
{
  "name": "@my-awesome-nx-repo/shared-ui",
  ... // other details
  "nx": {
    "tags": ["scope:shared"] // Apply tags here
  }
}

Try to violate these rules by removing the scope:shared dependent from the React app, then run npx nx lint my-react-app, you will get the following error:

error  A project tagged with "scope:frontend" can only depend on libs tagged with "scope:frontend"

This helps enforce a clean architecture, like preventing an app from depending on a library that is not allowed to be called.

Managing Dependencies in Nx Workspace

In a monorepo, managing dependencies can be done using two strategies:

Centralized Dependency Management: All dependencies are defined in the root package.json, shared across all apps and libraries.

Pros: Simplifies management, ensures consistency, and reduces duplication.
Cons: Less flexibility for individual projects to use different versions of dependencies.

Explicit Project-Specific Dependencies: Dependencies are scoped to individual apps or libraries, allowing each project to manage its own.

Pros: Provides flexibility and isolation for each project.
Cons: Increases complexity, may lead to duplication, and code sharing becomes more challenging.

So, it is a tradeoff as usual, but I would stick to the centralized dependency strategy.

Conclusion

Nx Monorepo isn't a one-size-fits-all solution, but when it fits, it transforms how teams build and scale applications.

It's ideal for large, interconnected projects that benefit from shared code, consistent tooling, and fast CI/CD. While it might be overkill for small or isolated projects.

For most modern development workflows, Nx offers the structure, performance, and flexibility to build with confidence and maintain long-term efficiency.

Think about it

If you enjoyed this article, I’d truly appreciate it if you could share it—it really motivates me to keep creating more helpful content!

If you’re interested in exploring more, check out these articles.

Thanks for sticking with me until the end—I hope you found this article valuable and enjoyable!

TypeScript Type Vs Interface? The Answer Is Type!

Mohamed Mayallo — Tue, 20 May 2025 07:00:00 +0000

Introduction

In TypeScript, you can define custom shapes using type aliases or interface. But when it comes to choosing between them, many developers ask: "TypeScript Type vs Interface?" The answer is increasingly clear: use type.

While both share similarities, type offers more flexibility, supports complex patterns, and aligns better with modern TypeScript and frameworks. In this guide, you'll learn the differences, strengths, and why type is the better default.

What Are Types and Interfaces?

Both type and interface define object shapes or function signatures:

// Using type
type Point = { x: number; y: number };
type SetPoint = (x: number, y: number) => void;

// Using interface
interface Point { x: number; y: number }
interface SetPoint { (x: number, y: number): void }

They look and behave similarly here, but the differences appear in advanced use cases.

Why Type Wins in Modern TypeScript

But why are types preferable?

1- More Expressive

type is more versatile and can describe things that interface can't, like unions, primitives, tuples, mapped types, and conditional types.

// Using `type` to alias primitive types.
// `interface` cannot be used with primitives like `string`, `number`, etc.
type Name = string; 

// Using `type` to define a union type.
// `interface` does not support union types directly.
type Result = { success: true } | { error: string }; 

// Using `type` to define a tuple.
// Tuples can't be defined using `interface`; `type` is required.
type Coordinates = [number, number]; 

// Using `type` with mapped types to create a read-only version of a given type.
// This utilizes TypeScript's advanced type system, which `type` handles more flexibly than `interface`.
type Readonly<T> = { readonly [K in keyof T]: T[K] };

// Conditional type (cannot be achieved using `interface`)
type ResponseData<T extends boolean> = T extends true
  ? { success: true; data: string }
  : { success: false; error: string };

// When the flag is true, the shape has 'data'
type SuccessResponse = ResponseData<true>; // { success: true; data: string }

// When the flag is false, the shape has 'error'
type ErrorResponse = ResponseData<false>; // { success: false; error: string }

Interfaces can’t express these patterns.

2- Safer and More Predictable

Only interface supports declaration merging:

interface Config { debug: boolean }
interface Config { verbose: boolean }
// Becomes: { debug: boolean; verbose: boolean }

This can be helpful in rare cases but risky in most—it leads to hard-to-debug surprises. type prevents accidental redefinitions.

3- Works Everywhere

type can extend both types and interfaces, and you can use intersection (&) for composition.

type A = { a: string };
type B = A & { b: number };

interface I { i: boolean }
type Combined = I & { x: number };

This makes type more consistent for real-world scenarios.

4- Ideal for Functional and React Patterns

Modern frontend libraries and tools like React, Redux, and Zod tend to favor type aliases due to their flexibility, especially when working with discriminated unions, complex props, or functional patterns.

// Simple React props using a type alias
type Props = { children: React.ReactNode };

// Discriminated union for Redux-like actions
type Action =
  | { type: 'start' }
  | { type: 'stop' };

// This is a discriminated union: a union of object types
// that share a common `type` field used to determine the variant.

You can’t define union types or complex props this cleanly with interface.

When Interface Makes Sense

While type is generally more flexible, interface still plays a valuable role in certain scenarios:

Public APIs / Libraries – interface supports declaration merging, which allows libraries to be extended or augmented safely.
OOP patterns where classes implement contracts

interface IUser {
  id: string;
  login(): void;
}

class Admin implements IUser {
  id = 'admin';
  login() {}
}

But even here, you can use type instead.

type IUser = {
  id: string;
  login(): void;
}

class Admin implements IUser {
  id = 'admin';
  login() {}
}

While both versions work, interface tends to be preferred in OOP-heavy codebases for its semantic clarity and support for extension via extends or declaration merging.

Conclusion

In most real-world TypeScript codebases, type outshines interface. It handles everything from unions to tuples, supports mapped and conditional types, and avoids unexpected behaviors like declaration merging.

At the end of the day, unless you’re exposing a public OOP-style API or need declaration merging, stick with type.

Think Of It

If you enjoyed this article, I’d truly appreciate it if you could share it—it really motivates me to keep creating more helpful content!

If you’re interested in exploring more, check out these articles.

And to learn more, please check the TypeScript Documentation:

Thanks for sticking with me until the end—I hope you found this article valuable and enjoyable!

Deep Copy vs Shallow Copy in JavaScript: Complete Guide

Mohamed Mayallo — Sun, 18 May 2025 08:27:00 +0000

Understanding Object References in JavaScript

Have you ever changed something you thought was a separate copy, only to find you messed up the original data too? Yeah, that happens a lot if you don't get how JavaScript handles different kinds of copying; deep copy vs shallow copy.

Value vs Reference Types

JavaScript has these two main ways it thinks about data: by value and by reference. Simple stuff like numbers, strings, booleans, null, and undefined are value types. When you copy them, JavaScript just makes a whole new one. A separate thing entirely. Changing the new one never affects the old one.

let a = 10;
let b = a; // b gets a copy of the value 10
b = 20;
console.log(a); // Output: 10 (a is unchanged)

However, objects, arrays, and functions are reference types. When you assign one variable to another, you're not copying the actual data. Instead, you're copying the reference to where that data lives in the computer's memory.

It's like having two street signs pointing to the same house. You change the house through one sign, the change is there if you use the other sign, too. It can be confusing, is it not?

let obj1 = { name: 'Alice' };
let obj2 = obj1; // obj2 gets a copy of the reference to the object
obj2.name = 'Bob';
console.log(obj1.name); // Output: Bob (obj1 was changed!)

The Problem with Direct Object Assignment

Assigning one object variable to another using the simple equals sign (=) creates this reference copy. This is where things get messy if you aren't expecting it. You think you've made a copy to work with, but you've just made another handle for the exact same object in memory. Modifying the 'copy' variable directly modifies the original object.

Let's say you get some data from an API and store it in a variable, maybe called userData. Then you need to modify this data before displaying it, but you also want to keep the original userData untouched for later use. If you just do let displayData = userData; and then change displayData, you've just changed your original userData.

let originalUser = { id: 1, name: 'Charlie', settings: { theme: 'dark' } };
let modifiedUser = originalUser; // Just copying the reference

modifiedUser.name = 'David'; // Changes originalUser.name too!

console.log(originalUser.name); // Output: David

Therefore, if your goal is to have a separate version of the object that you can change without affecting the source, a direct assignment simply doesn't do the job you think it might.

Copy Operations and Memory Management

JavaScript supports two kinds of copying: shallow and deep. Both involve different levels of memory handling.

Grasping how memory works in each case helps explain why deep copying tends to be slower and more resource-intensive. So, let’s break down both types in more detail.

Shallow Copy: Methods and Limitations

When you make a shallow copy, a new memory is allocated for the top-level object or array structure. However, the nested objects inside still refer to the same memory locations as the original.

It's like a new outer box, but the things inside are the originals. It's faster than a deep copy because it does less work.

Built-in Shallow Copy Techniques

JavaScript offers several easy ways to create shallow copies.

Object.assign(): This method copies enumerable own properties from one or more source objects to a target object. The target object is then returned. It's been around for a while, very reliable for flat objects.

const original = { a: 1, b: { c: 2 } };
const shallowCopy = Object.assign({}, original);

shallowCopy.a = 100; // Changes only shallowCopy.a
shallowCopy.b.c = 200; // Changes original.b.c and shallowCopy.b.c

console.log(original.a); // Output: 1
console.log(original.b.c); // Output: 200 (Still shared!)

Spread syntax (...): This is perhaps the most modern and often cleanest way for arrays and objects. It 'spreads' the properties or elements from the source into a new object or array literal. It does essentially the same thing as Object.assign() for objects and provides a nice syntax for arrays.

const originalArr = [1, { d: 4 }];
const shallowCopyArr = [...originalArr]; // Shallow copy of array

shallowCopyArr[0] = 100; // Changes only shallowCopyArr[0]
shallowCopyArr[1].d = 400; // Changes originalArr[1].d and shallowCopyArr[1].d

console.log(originalArr[0]); // Output: 1
console.log(originalArr[1].d); // Output: 400 (Still shared!)

const originalObj = { a: 1, b: { c: 2 } };
const shallowCopyObj = { ...originalObj }; // Shallow copy of object

shallowCopyObj.a = 100; // Changes only shallowCopyObj.a
shallowCopyObj.b.c = 200; // Changes originalObj.b.c and shallowCopyObj.b.c

console.log(originalObj.a); // Output: 1
console.log(originalObj.b.c); // Output: 200 (Still shared!)

Array methods (slice(), concat()): For arrays specifically, methods like slice() without arguments or concat() with an empty array also create shallow copies. These are classic methods, reliable for many years now.

const originalArrOld = [1, { e: 5 }];
const shallowCopyArrOld1 = originalArrOld.slice();
const shallowCopyArrOld2 = [].concat(originalArrOld);

shallowCopyArrOld1[1].e = 500; // Changes originalArrOld[1].e and shallowCopyArrOld1[1].e
shallowCopyArrOld2[1].e = 600; // Changes originalArrOld[1].e, shallowCopyArrOld1[1].e, and shallowCopyArrOld2[1].e

console.log(originalArrOld[1].e); // Output: 600 (All shared!)

Nested Object Behavior

As you saw, the crucial thing about shallow copies is their behavior with nested objects. The copy process only goes one level deep. For that reason, the nested object { c: 2 }, { d: 4 }, or { e: 5 } is still the exact same object in memory? Changing it through any of the variables (original, shallowCopy, shallowCopyObj, etc.) affects all of them.

This sharing of nested references is the primary reason shallow copy isn't suitable when you need complete independence between the original and the copy, especially when dealing with complex data structures or state management, where immutability is important. It's a common source of bugs.

Deep Copy: Complete Object Duplication

Sometimes you need a true clone, a completely independent replica where changing the copy never impacts the original, no matter how deep the nesting goes. This is where deep copies come in.

A deep copy, on the other hand, allocates entirely new memory for every part of the object structure, including all nested objects and arrays. This requires more processing time and more memory space compared to a shallow copy.

It's like unpacking a box and then making a brand new copy of everything inside and putting it into a new box. This way, nothing in the new box points back to the old box's contents.

JSON Method: Syntax and Limitations

The most common trick developers used for a long time to achieve a deep copy was JSON.parse(JSON.stringify(obj)). How does this work? JSON.stringify() converts a JavaScript object into a JSON string. This string representation contains only the data, not the references. Then, JSON.parse() converts that string back into a new JavaScript object. Since it's built from a string, all the objects and arrays within it are newly created.

const originalDeep = { a: 1, b: { c: 2 } };
const deepCopyJSON = JSON.parse(JSON.stringify(originalDeep));

deepCopyJSON.a = 100; // Changes only deepCopyJSON.a
deepCopyJSON.b.c = 200; // Changes only deepCopyJSON.b.c

console.log(originalDeep.a); // Output: 1
console.log(originalDeep.b.c); // Output: 2 (Original is untouched!)

This method is simple and works well for basic data objects that contain only primitive types, arrays, and other plain objects. However, it has significant limitations you must be aware of:

Loses Data Types: It cannot handle certain JavaScript data types. Functions, undefined, NaN, Infinity, and RegExp objects turn into null or are simply dropped during stringification. Dates become strings. Maps and Sets become empty objects. This is because the JSON format itself doesn't have standard representations for these JavaScript-specific concepts.

const objA = { date: new Date() };
const objB = JSON.parse(JSON.stringify(objA));

console.log(objA.date instanceof Date) // true
console.log(objB.date instanceof Date) // false (Loses the Date type!)

Circular References: Objects with circular references (where a property of an object refers back to the object itself or one of its ancestors) will cause JSON.stringify() to throw an error.

const objA = { name: 'A' };
const objB = { name: 'B', parent: objA };
objA.child = objB;

// JSON.parse(JSON.stringify(objA)); // Throws TypeError

So, while easy, this method is far from a universal deep copy solution. You might run into trouble unexpectedly.

Structured Clone API

Modern JavaScript introduces the [structuredClone()](https://developer.mozilla.org/en-US/docs/Web/API/Window/structuredClone) function, which handles many of the limitations above. It's becoming the standard way to do this.

const original = {
  name: 'Alpha',
  date: new Date(),
  map: new Map([['key', 'value']]),
  set: new Set([1, 2, 3])
};
original.self = original;

const copy = structuredClone(original);

console.log(copy.date instanceof Date); // true
console.log(copy.map instanceof Map);   // true
console.log(copy.set instanceof Set);   // true
console.log(copy.self === copy);        // true (circular reference preserved)

It supports:

Date, RegExp, Map, Set.
ArrayBuffer, Blob, File, FileList, ImageData, MessagePort, ImageBitmap, Error.
NaN, Infinity.
Circular references.

But it still has drawbacks:

Functions are not allowed: they’ll be dropped or throw an error.

structuredClone({ fn: () => {} }); // Throws DataCloneError

Doesn't preserve the prototype chain: class instances lose their methods.

class MyClass {
  constructor(value) { this.value = value; }
  greet() { console.log('Hello ' + this.value); }
}

const instance = new MyClass('World');
const clone = structuredClone(instance);

console.log(clone.value);           // "World"
console.log(clone.greet);           // undefined
console.log(clone instanceof MyClass); // false

Doesn’t support getters/setters or property descriptors.

class SecretBox {
  constructor() {
    this._value = 'secret';
  }

  get hiddenValue() {
    return this._value;
  }

  set hiddenValue(val) {
    console.log('Trying to set to:', val);
  }
}

const original = new SecretBox();
const clone = structuredClone(original);

// Original still works
console.log(original.hiddenValue);  // Output: 'secret'
original.hiddenValue = 'new';       // Logs: Trying to set to: new

// Clone fails
console.log(clone.hiddenValue);     // Output: undefined (getter is lost)
clone.hiddenValue = 'test';         // No log (setter is gone)

// Prototype check
console.log(clone instanceof SecretBox); // false
console.log(Object.getPrototypeOf(clone) === SecretBox.prototype); // false

It’s a much better option than JSON for most modern needs, but not perfect for all use cases, especially when behavior or prototypes matter.

Library Solutions: Lodash

If you need a more flexible, production-grade deep clone and don't want to write your own, libraries like Lodash are a great choice. These libraries have already solved many of the edge cases that custom implementations miss.

// npm install lodash

class MyClass {
  constructor(value, obj) {
    this.value = value;
    this.obj = obj;
  }
  greet() { console.log('Hello ' + this.value); }
  get val() {
    return this.value;
  }
}

const instance = new MyClass('World');
instance.obj = { greeting: "Hello", date: new Date() };
instance.obj.self = instance.obj;
const clone = _.cloneDeep(instance);
clone.obj.greeting = "Hi";

console.log(clone.value);                        // "World"
console.log(instance.obj.greeting);              // "Hello"
console.log(clone.obj.greeting);                 // "Hi"
console.log(instance.obj.date instanceof Date);  // true
console.log(clone.obj.date instanceof Date);     // true
console.log(clone.obj.self);                     // Handles the circular refs (No errors)
console.log(clone.greet);                        // ƒ greet() (Doesn't lose functions)
console.log(clone instanceof MyClass);           // true
console.log(instance.val);                       // "World"
console.log(clone.val);                          // "World" (Doesn't lose getters and setters)

As you see, it mostly solves all the problems.

Libraries are generally the recommended path for complex deep cloning needs because they handle many tricky situations for you. Why reinvent the wheel when someone else built a perfectly good one, right?

Custom Recursive Implementation

If you need total control, maybe you want to copy specific things and ignore others; you might need to write your own deep copy function.

However, keep in mind, writing a robust custom deep clone function is quite complicated. It needs to account for all built-in types, handle inheritance (prototypes), property attributes, and those circular references.

Because of the complexity, this is usually only worth doing when libraries don't meet your requirements.

When to Use Shallow vs Deep Copy

How do you choose between shallow and deep copy?

It comes down to the structure of your data and what you intend to do with the copy. No single rule applies to every situation.

Use shallow copy when:

Your object or array is flat (no nested objects/arrays).
You only need to modify properties directly on the top level.
Performance is critical, and shared nested references are acceptable or won't be modified.

Use deep copy when:

Your object or array has nested objects or arrays.
You need complete independence between the original and the copy, allowing modification of nested structures without affecting the source.
Immutability of the entire structure is required.

Conclusion

JavaScript offers several approaches to copying objects, each with distinct trade-offs. Shallow copies (like Object.assign() or spread operators) are fast and memory-efficient but only duplicate the top level, leaving nested structures vulnerable to unintended side effects.

On the other side, deep copies provide complete independence by duplicating everything, though at the cost of performance and memory. And we covered some approaches to handle it.

The right choice depends on your specific situation, the complexity of your data, performance needs, and which limitations you can accept.

By understanding these trade-offs, you can make informed decisions that lead to more predictable, bug-free code.

Think about it

If you enjoyed this article, I’d truly appreciate it if you could share it; it really motivates me to keep creating more helpful content!

If you’re interested in exploring more, check out these articles.

Thanks for sticking with me until the end. I hope you found this article valuable and enjoyable!

MongoDB Views or Materialized Views: When to Use?

Mohamed Mayallo — Tue, 13 May 2025 05:59:05 +0000

Intoduction

MongoDB offers powerful tools to simplify data access and improve performance. Among these tools are Standard Views and Materialized Views, two approaches that allow you to shape how data is read and presented, each with its own trade-offs and use cases.

In this article, we’ll explore the differences between the two and when to use each effectively.

I created a small Node.js server to compare between them and guide you how to create and refresh both programmatically, Find it here.

What Is a MongoDB Standard View?

So, you're working with MongoDB and hear about these "view" things. Are they the same? Not quite.

A MongoDB Standard View (or MongoDB View) is essentially a saved query (aggregation pipeline). It acts like a virtual collection. When you query the view, MongoDB runs the saved query against the real collections at that moment.

It's logical: Doesn't store any data itself, just the definition.
It's read-only: You can't write data to a view.
It uses aggregation pipelines: This is how you define what the view shows.
It's metadata: The view definition is stored, not the results. So, zero extra storage footprint for the data itself.

Basically, it's a shortcut; instead of writing a very long, complex aggregation pipeline over and over, you save this definition as a virtual collection for later use.

What is a Materialized View?

Unlike standard views, a materialized view actually stores data. It's basically a regular collection that holds the pre-computed results of an aggregation query.

Guess what? MongoDB doesn't natively support them like some SQL databases do (not yet, anyway). When we talk about materialized views in MongoDB, we mean a pattern you implement yourself, another collection (the materialized view) that holds pre-computed results.

Stores results physically: You run an aggregation query and save its output into a real, separate collection.
Precomputed: The results are already calculated and stored.
Simulates denormalization: Kinda like creating a purpose-built table holding exactly the data you need for fast reads.

So, you're trading real-time data freshness for speed, because reading from this pre-built collection is way faster than running a complex aggregation every time.

MongoDB Views Vs Materialized Views

Well, let's compare them based on some criteria.

Data Freshness

Views: Always show the latest data because they run the query live. What you see is what's in the base collection right now.
Materialized Views: Only as fresh as the last refresh. If you refresh it once a day, the data could be up to 24 hours old.

Performance Implications

Views: Read performance depends entirely on the underlying query and collections. Complex aggregations on large datasets? Will be slow. Simple filter? Probably fast.
Materialized Views: Reads are typically super fast because it's just querying a regular collection with precomputed data. Writes, though? You add overhead because you need to run the aggregation and write the results periodically (the refresh).

Storage Considerations

Views: Tiny footprint (neglicable). Just stores the query definition.
Materialized Views: Can use significant storage, because you're duplicating data (the results of the query). The size depends on the output of your aggregation.

Maintenance Overhead

Views: Almost none. Define it once, use it. Maybe update the definition if your needs change.
Materialized Views: You have to build and manage the refresh process. How often? How to handle failures? What if the source data changes structure? It's definitely more work.

Native Support in MongoDB

Views: Fully supported feature.
Materialized Views: Not supported at this time. It's a pattern. You use standard MongoDB features like aggregation ($out or $merge) and maybe schedulers or Change Streams to build it.

Security & Abstraction Capabilities

Views: Great for this. You can grant read access only to the view, effectively hiding fields or documents from users who shouldn't see them.
Materialized Views: It's just another collection. You need to manage its security access separately, just like any other collection.

When To Use Views vs Materialized Views?

Why bother with regular views then?

Query Abstraction: Hide complex aggregation logic behind a simple view name. Makes life easier for application devs who just want to query order_analytics_view instead of writing a complex pipeline.
Data Security: You can create views that only show certain fields or documents to specific users or roles. Kinda like a security layer without needing complex application logic. Like, maybe show user_basic_info_view to regular staff, but the full users collection only to admins.

On the other hand, materialized views come into play when performance is key, especially for reads.

Speeding up Reads: If you have dashboards or analytics hitting the database with the same heavy aggregations over and over, pre-calculating results in a materialized view makes those reads lightning fast.
Expensive Aggregations: Some aggregations just take a long time and use lots of resources. Running them once and storing the result is way more efficient if the data doesn't need to be absolutely live.
Denormalized Data: Perfect when you need data structured in a specific format, often combining data from multiple collections.

How to Implement a MongoDB View?

Alright, let's make a view. It's pretty straightforward.

Create a View Using MongoDB Shell

Say you have a users collection and just want a view of the active ones.

// Connect to your database first
// Example: use my_database

db.createView(
  "active_users_view",  // The name of your new view
  "users",              // The source collection
  [                     // The aggregation pipeline
    { $match: { status: "active" } } 
  ]
)

Now you can query it like a regular collection: db.active_users_view.find() and you'll only get users with status: "active". MongoDB runs that $match pipeline every time you query the view.

You could even add $group stages or use cool pipeline stages like $facet for multi-faceted results.

Create a View Using MongoDB Driver (With Node.js)

Say you want to create a script to create a view programmatically, you can do the following:

const client = await MongoClient.connect(process.env.MONGODB_URI);
const db = client.db(process.env.DATABASE_NAME);

await db.createCollection("active_users_view", {
    viewOn: "users",
    pipeline: [
      { $match: { status: "active" } } 
    ]
});

The same functionality as before.

Updating or Modifying a View

MongoDB views are immutable. You can't alter an existing view's definition directly. What do you do?

Drop and Recreate: The most common way.

db.active_users_view.drop();
// Now create it again with the new definition 
db.createView("active_users_view", "users", [ /* new pipeline */ ]);

Versioning: Create a new view with a version number (e.g., active_users_v2) and update your application code to use the new one. You can drop the old one later.

How to Implement a MongoDB Materialized View?

MongoDB doesn't have a built-in CREATE MATERIALIZED VIEW command like some SQL databases (not yet, anyway). So we have to implement it manually.

Keep in mind, when we talk about Materialized Views, the create operation is the same as the refresh operation. Because the refresh operation needs an existing collection, and if it is not there, it will create a new one. So they are not separate.

Create/Refresh with Aggregation ($out)

This is the simplest. Run an aggregation pipeline and tell it to write the results to a new collection using the $out stage.

db.orders.aggregate([
  { 
    $match: { status: "completed" } // Only completed orders
  },
  { 
    $group: { 
      _id: "$customerId", // Group by customer
      totalSpent: { $sum: "$amount" } // Sum their order amounts
    } 
  },
  { 
    $out: "customer_spending_mv" // Write results to this collection (materialized view)
  }
]);

Now you have a customer_spending_mv collection, and, you can query it like this db.customer_spending_mv.find()

Warning: $out replaces the entire output collection every time it runs. If the aggregation fails mid-way, you might lose the collection temporarily. Good for infrequent updates or simple use.

Create/Refresh with Aggregation ($merge)

Run an aggregation and merge the results into an existing collection. This is more flexible; you can update existing documents, insert new ones, or even fail the operation based on matching conditions. Better for incremental updates.

db.orders.aggregate([
  { 
    $match: { status: "completed" } // Only completed orders
  },
  { 
    $group: { 
      _id: "$customerId", // Group by customer
      totalSpent: { $sum: "$amount" } // Sum their order amounts
    } 
  },
  {
    $merge: {
      into: "customer_spending_mv", // The target collection
      on: "_id", // Field to match documents on
      whenMatched: "replace", // What to do if a document matches (update it)
      whenNotMatched: "insert" // What to do if no document matches (insert it)
    }
  }
])

Completely Manual Create/Refresh

We knew that the materialized view is just another collection, right? Then, we can write a custom script that queries the source data and updates the target materialized view collection. Makes sense? Gives you maximum control but requires more effort as it is your responsibility to keep both collections in sync.

On-Demand vs. Incremental Refresh Strategies

Since materialized views store data snapshots, you have to decide how to keep them updated.

On-Demand Refresh: You update the view whenever you need to, maybe manually or via an API call. Simple, but data can get quite stale. Useful if updates are infrequent or data changes predictably.
Scheduled Refresh (Batch): Use a scheduler (like cron or MongoDB Atlas Scheduled Triggers) to run your $out or $merge aggregation periodically (e.g., every hour, every night). This is common for reports. Data freshness depends on the schedule frequency.
Incremental Refresh (Near Real-Time): Use MongoDB Change Streams. These let you watch for changes (inserts, updates, deletes) in the source collection(s) in real-time. When a change happens, you trigger a small, targeted update to your materialized view collection. This keeps the view much more up-to-date but is more complex to set up and manage and it is available only on the replica-set setup.

Choosing the right strategy depends completely on how fresh your data needs to be and how much load the refresh process puts on your system.

Performance Benchmark: Views vs Materialized Views

Talking performance is nice, but seeing is believing, right? For that reason, I built a small app where I have the orders collection that has tons of documents. And created a View and a Materialized View to measure the query performance of both. Let's see.

First, run the following commands in sequence:

npm run setup-db # setup the `sales` database
npm run seed-data # seed the `orders` collection
npm run create-standard-view # create the `order_analytics_view` view
npm run create-materialized-view # create the `order_analytics_materialized` materialized view
npm run start # run the Node.js server

Then, open your browser and hit this endpoint:

http://localhost:3000/api/analytics/compare

This request returns 100 documents for both the View, the Materialized View, and some performance metrics. And technically uses the same aggregation query under the hood.

Let's have a look at the performance metrics (in my environment):

"performance": {
    "standardView": {
      "queryTimeMs": 274,
      "resultCount": 100
    },
    "materializedView": {
      "queryTimeMs": 6,
      "resultCount": 100,
      "lastUpdated": "2025-04-20T06:19:40.416Z"
    },
    "comparison": {
      "timeDifferenceMs": 268,
      "percentageDifference": "97.81%"
    }
  }

If you noticed the huge gap in performance between both, the Materialized View is 97.81% faster than the Standard View, and that's expected.

Use Cases For Each Approach

So, when should you pick which?

MongoDB Views – Ideal Scenarios

Role-Based Access: Show different data slices to different user roles without complex app logic. Expose limited_user_profile_view vs full users collection.
Admin Dashboards (Light Filtering): Simple dashboards that need real-time data but don't involve massive aggregations.
Rapid Prototyping: Quickly define a data shape without restructuring collections.
Simplifying Application Queries: Hide complex joins ($lookup) or calculations behind a simple view name.

Materialized Views – Ideal Scenarios

Analytics Dashboards: Dashboards needing fast responses on complex aggregations (e.g., sales summaries, user activity trends). Hourly or daily freshness is often fine.
Heavy Reporting: Generating complex reports that would time out or overload the DB if run live. Run the aggregation overnight into a materialized view.
Time-Series Aggregations: Summarizing large amounts of time-series data (e.g., daily sensor readings into monthly averages).
Denormalization for Microservices: Create a specialized, denormalized collection (your materialized view) optimized for a specific microservice's read patterns, fed from other source collections/services.

Frequently Asked Questions (FAQ)

Q1: Can you create an index on a MongoDB View?

No, you cannot create indexes directly on a MongoDB View itself. Performance depends on the indexes present on the underlying source collection(s) that the view's aggregation pipeline uses.

Q2: Can you update data through a MongoDB View?

No, MongoDB Views are strictly read-only. You cannot perform insert, update, or delete operations through a view. You must modify the data in the base collection(s).

Q3: How often should I refresh a Materialized View in MongoDB?

It depends entirely on your application's tolerance for stale data versus the overhead of the refresh process. It could be every few minutes, hourly, daily, or even weekly. Analyze your specific needs. For near real-time, consider Change Streams, but that's much more complex.

Q4: Does using $out or $merge lock the source collection during materialized view refresh?

The aggregation pipeline reads from the source collection(s), which will acquire the necessary read locks or yield like any other aggregation. The $out stage briefly acquires an exclusive lock on the destination collection while replacing it. $merge acquires locks on the destination collection as it performs writes/updates. The impact on the source collection is primarily the read load of the aggregation itself.

Q5: Is there a size limit for MongoDB Views?

No, because views don't store data. However, the aggregation pipeline defining the view has limits (e.g., 100MB memory limit by default unless allowDiskUse: true is specified, though this option isn't always usable in the first stage of a view definition). The performance will degrade significantly if the view's query processes massive amounts of data without good indexing on the base collection.

Conclusion

Okay. We covered a lot.

We learned that a Standard View is a saved query and a Materialized View is a saved collection.

And, as usual, everything is a trade-off:

Standard Views = Simplicity + Real-time Data (but potentially slow reads)
Materialized Views = Fast Reads (but Complexity + Stale Data + Storage Costs)

So, pick the tool that fits the job. Don't over-engineer if a simple view works, but don't shy away from materialized views when you really need that speed boost.

And keep in mind, you don't always have to choose just one. Sometimes, a mix is best.

Think about it

If you enjoyed this article, I’d truly appreciate it if you could share it—it really motivates me to keep creating more helpful content!

If you’re interested in exploring more, check out these articles.

Thanks for sticking with me until the end—I hope you found this article valuable and enjoyable!

DEV Community: Mohamed Mayallo

Is SQL BETWEEN Inclusive or Exclusive?

TL;DR

Introduction

SQL BETWEEN Example

SQL BETWEEN vs Greater Than and Less Than

Common Misunderstanding

Conclusion

Two Ways To Git Cherry-Pick From Another Repo

Introduction

Why cherry-pick from an external repo?

First Way: Using Cherry-Pick

Second Way: Using format-patch

Keep In Mind These Tips

Conclusion

Untangle Your Code: Chain of Responsibility Design Pattern

Introduction

Use Case 1: A Messy Signup Form

The Solution: A Clean Chain of Validators

Use Case 2: API Middleware

When to Use the Chain of Responsibility Pattern

Conclusion

Think about it

Do You Know The LATERAL JOIN In SQL?

Introduction

What Does LATERAL Actually Mean?

A Real-World Business Problem (PostgreSQL)

The LATERAL Use Cases

1- Selecting Top N Rows per Group

2- Unpacking JSONB Order Details

3- Searching Array Data With UNNEST

4- Using Functions and Calculations per Row

LATERAL JOIN vs. Correlated Subqueries

When to Avoid LATERAL?

Case 1: For the "Top-1" Problem, Use DISTINCT ON

Case 2: For Simple Aggregates, Use GROUP BY

Performance Considerations

Conclusion

Postgres SQL FILTER: A Smarter CASE and CTE Alternative

Introduction

The Business Problem: An E-commerce Dashboard

Setting Up Our Example

Solution 1: Using the CASE Statement

Solution 2: Using Common Table Expressions (CTEs)

The Better Way: The FILTER Clause

FILTER vs. CASE: A Closer Look

FILTER vs. CTEs

Using FILTER with Multiple Conditions

Conclusion

Can GraphQL Call REST API?

Introduction

Why Use GraphQL with REST?

Let’s Build It Step-by-Step

Step 1: Set Up Project

Step 2: Create the GraphQL Server

Step 3: Run the Server

Conclusion

Are Nx Monorepo Configurations Really Complex?

Introduction

Why Nx TypeScript Configuration Feels Complex

Common Configuration Pitfalls

The Inheritance Chain Explained

Understanding the Configuration Files

Root Level tsconfig.base.json: The Global Foundation

Root Level tsconfig.json: Enabling Project References

Project Level tsconfig.json: The Project Entry Point

Project Level tsconfig.app.json & tsconfig.lib.json: Application and Library Specifics

Project Level tsconfig.spec.json: Fine-Tuning for Tests

Root Level nx.json: The Workspace "Brain"

Project Level project.json or package.json: The Project's Instruction Manual

Nx's Superpower: Inferring Tasks Automatically

Path Aliases vs. Project References: Which Should You Choose?

What are Path Aliases?

What are TypeScript Project References?

The Verdict: Path Aliases vs. Project References

Troubleshooting Common TypeScript Issues

Import Resolution Problems

Build Errors and Solutions

IDE Integration Issues

Caching and Performance in Nx

Case 1: For the "Top-1" Problem, Use `DISTINCT ON`

Case 2: For Simple Aggregates, Use `GROUP BY`

Root Level `tsconfig.base.json`: The Global Foundation

Root Level `tsconfig.json`: Enabling Project References

Project Level `tsconfig.json`: The Project Entry Point

Project Level `tsconfig.app.json` & `tsconfig.lib.json`: Application and Library Specifics

Project Level `tsconfig.spec.json`: Fine-Tuning for Tests

Root Level `nx.json`: The Workspace "Brain"

Project Level `project.json` or `package.json`: The Project's Instruction Manual