DEV Community: Madhukar Vissapragada

Builder Design Pattern - Simplified

Madhukar Vissapragada — Fri, 03 Jul 2026 15:31:10 +0000

A real story from building a manuscript submission system at a scientific publishing company.

We were building a system at a Publishing Company that accepts a research manuscript and sends it through an automated scientific publishing pipeline.

Then we had to write the class that an API calls to create a submission.

class ManuscriptSubmissionService:
    def __init__(self, title, abstract, authors, affiliations, keywords,
                 references, funding, ethics, conflicts,
                 supplementary_files, cover_letter, supporting_files,
                 manuscript_attachment):
        self.title = title
        self.abstract = abstract
        self.authors = authors
        self.affiliations = affiliations
        self.keywords = keywords
        self.references = references
        self.funding = funding
        self.ethics = ethics
        self.conflicts = conflicts
        self.supplementary_files = supplementary_files
        self.cover_letter = cover_letter
        self.supporting_files = supporting_files
        self.manuscript_attachment = manuscript_attachment

And creating an object looked like this:

submission = ManuscriptSubmissionService(
    "Covid-19 study",
    "some random abstract goes here",
    "a1, a2, a3",
    "aff1, aff2, aff3",
    "keyword1, keyword2",
    "references1, references2",
    "funding1",
    "ethic1",
    "conflict1",
    "supplementary1",
    "manuscript1",
    "supporting_file1",
    "manuscript_attachment1",
)

Look at that. Thirteen positional arguments. No context. No idea what the fourth parameter is without scrolling back up to the constructor definition every single time.

This is called the monster constructor problem — one init that keeps growing until nobody can read it.

Fix 1: Named Parameters

The first instinct is to use named parameters with defaults:

class ManuscriptSubmissionService:
    def __init__(self, title: str = None, abstract: str = None,
                 authors: list = None, affiliations: list = None,
                 keywords: list = None, references: list = None,
                 funding: str = None, ethics: str = None,
                 conflicts: str = None, supplementary_files: list = None,
                 cover_letter: str = None, supporting_files: list = None,
                 manuscript_attachments: list = None):
        self.title = title
        self.abstract = abstract
        self.authors = authors
        self.affiliations = affiliations
        self.keywords = keywords
        self.references = references
        self.funding = funding
        self.ethics = ethics
        self.conflicts = conflicts
        self.supplementary_files = supplementary_files
        self.cover_letter = cover_letter
        self.supporting_files = supporting_files
        self.manuscript_attachments = manuscript_attachments


submission = ManuscriptSubmissionService(
    title="Covid-19 study",
    abstract="some random abstract goes here",
    authors="a1, a2, a3",
    affiliations="aff1, aff2, aff3",
)

This looks cleaner at the call site. But now everything is optional — None by default. A researcher can submit a manuscript with no title, no authors, no actual manuscript file. The system accepts it silently and breaks somewhere downstream.

The problem: no mandatory field enforcement.

Fix 2: Validation Inside the Constructor

Okay, so add validation:

class ManuscriptSubmissionService:
    def __init__(self, title: str = None, abstract: str = None,
                 authors: list = None, affiliations: list = None,
                 keywords: list = None, references: list = None,
                 funding: str = None, ethics: str = None,
                 conflicts: str = None, supplementary_files: list = None,
                 cover_letter: str = None, supporting_files: list = None,
                 manuscript_attachments: list = None):
        if title is None:
            raise IOError("ManuscriptSubmissionService requires a title")
        if abstract is None:
            raise IOError("ManuscriptSubmissionService requires an abstract")
        if not authors:
            raise IOError("ManuscriptSubmissionService requires authors")
        if not affiliations:
            raise IOError("ManuscriptSubmissionService requires affiliations")
        if not manuscript_attachments:
            raise IOError("ManuscriptSubmissionService requires a manuscript attachment")

        self.title = title
        self.abstract = abstract
        self.authors = authors
        self.affiliations = affiliations
        self.keywords = keywords
        self.references = references
        self.funding = funding
        self.ethics = ethics
        self.conflicts = conflicts
        self.supplementary_files = supplementary_files
        self.cover_letter = cover_letter
        self.supporting_files = supporting_files
        self.manuscript_attachments = manuscript_attachments

This looks like it works. But stop and think about what the constructor is doing now.

It's validating the input AND initialising the object's state. Two responsibilities. One method.

That's a Single Responsibility Principle violation.

And there's a deeper problem hiding here. By the time validation runs, the object already exists in memory. Python allocated space for it the moment ManuscriptSubmissionService(...) was called. You're validating an object that's already been born — and potentially destroying it right after.

An invalid manuscript should never exist in the first place.

Fix 3: Setters

What about an empty constructor and setters?

class ManuscriptSubmissionService:
    def __init__(self):
        self.title = None
        self.abstract = None
        self.authors = []
        self.affiliations = []
        self.keywords = []
        self.references = []
        self.funding = None
        self.ethics = None
        self.conflicts = None
        self.supplementary_files = []
        self.cover_letter = None
        self.supporting_files = []
        self.manuscript_attachments = []

    def set_title(self, title: str):
        self.title = title
        return self

    def set_abstract(self, abstract: str):
        self.abstract = abstract
        return self

    def set_author(self, author: str):
        self.authors.append(author)
        return self

    def set_affiliation(self, affiliation: str):
        self.affiliations.append(affiliation)
        return self

    def set_keyword(self, keyword: str):
        self.keywords.append(keyword)
        return self

    def set_references(self, reference: str):
        self.references.append(reference)
        return self

    def set_funding(self, funding: str):
        self.funding = funding
        return self

    def set_ethics(self, ethics: str):
        self.ethics = ethics
        return self

    def set_conflict(self, conflict: str):
        self.conflicts = conflict
        return self

    def set_supplementary_files(self, supplementary_file: str):
        self.supplementary_files.append(supplementary_file)
        return self

    def set_cover_letter(self, cover_letter: str):
        self.cover_letter = cover_letter
        return self

    def set_supporting_files(self, supporting_file: str):
        self.supporting_files.append(supporting_file)
        return self

    def set_manuscript_attachment(self, manuscript_attachment: str):
        self.manuscript_attachments.append(manuscript_attachment)
        return self

    def validate(self):
        if self.title is None:
            raise IOError("ManuscriptSubmissionService requires a title")
        if self.abstract is None:
            raise IOError("ManuscriptSubmissionService requires an abstract")
        if len(self.authors) == 0:
            raise IOError("ManuscriptSubmissionService requires authors")
        if len(self.affiliations) == 0:
            raise IOError("ManuscriptSubmissionService requires affiliations")
        if len(self.manuscript_attachments) == 0:
            raise IOError("ManuscriptSubmissionService requires a manuscript attachment")


# Usage
submission = ManuscriptSubmissionService()
submission.set_title("Covid-19 study")
submission.set_abstract("This paper studies...")
submission.set_author("Dr. Smith")
submission.set_affiliation("Oxford University")
submission.set_manuscript_attachment("covid19_paper.pdf")
submission.validate()

Now the object exists with no data. Nothing stops you from calling send_to_pipeline(submission) before calling any setters. The half-built object silently enters the publishing pipeline and breaks somewhere downstream.

The problem: object exists in an invalid state before validate() is ever called.

Three Approaches. Three Failures.

Approach	Problem
Monster constructor	Unreadable, positional confusion
Named params + validation in `__init__`	SRP violation, object born before validation
Setters	Invalid object state, nothing enforced

None of them fully solve it. What we actually need is a system that builds the object step by step and only creates it in memory when everything is in place.

This is exactly where the Builder pattern comes in.

What Is the Builder Pattern?

Builder is a creational design pattern — it falls under the umbrella of patterns that control how objects are created.

Its job: separate the construction and validation of an object from its representation. An object should only come to life when it has everything it needs.

Two classes. One job each:

ManuscriptSubmissionService — holds data. Nothing else.
ManuscriptBuilder — sets values, validates, and only then creates the object.

Building It Step by Step

Step 1: Separate the Classes

ManuscriptSubmissionService.__init__ does exactly one thing — stores what it receives. No validation. No logic. One responsibility.

class ManuscriptSubmissionService:
    def __init__(self, title: str, abstract: str, authors: list,
                 affiliations: list, keywords: list, references: list,
                 funding: str, ethics: str, conflicts: str,
                 supplementary_files: list, cover_letter: str,
                 supporting_files: list, manuscript_attachments: list):
        # just stores data — nothing else
        self.title = title
        self.abstract = abstract
        self.authors = authors
        self.affiliations = affiliations
        self.keywords = keywords
        self.references = references
        self.funding = funding
        self.ethics = ethics
        self.conflicts = conflicts
        self.supplementary_files = supplementary_files
        self.cover_letter = cover_letter
        self.supporting_files = supporting_files
        self.manuscript_attachments = manuscript_attachments

    @staticmethod
    def builder():
        return ManuscriptSubmissionService.ManuscriptBuilder()

Step 2: The Builder with Method Chaining

Every setter returns self. This enables method chaining — calling multiple setters in one readable fluent line.

Also notice: the builder initialises all fields to None or empty lists. This way, if someone calls build() without setting a field, our validation runs correctly — no AttributeError, just a clean IOError.

    class ManuscriptBuilder:
        def __init__(self):
            self.title = None
            self.abstract = None
            self.authors = []
            self.affiliations = []
            self.keywords = []
            self.references = []
            self.funding = None
            self.ethics = None
            self.conflicts = None
            self.supplementary_files = []
            self.cover_letter = None
            self.supporting_files = []
            self.manuscript_attachments = []

        def set_title(self, title: str):
            self.title = title
            return self

        def set_abstract(self, abstract: str):
            self.abstract = abstract
            return self

        def set_author(self, author: str):
            self.authors.append(author)
            return self

        def set_affiliation(self, affiliation: str):
            self.affiliations.append(affiliation)
            return self

        def set_keyword(self, keyword: str):
            self.keywords.append(keyword)
            return self

        def set_references(self, reference: str):
            self.references.append(reference)
            return self

        def set_funding(self, funding: str):
            self.funding = funding
            return self

        def set_ethics(self, ethics: str):
            self.ethics = ethics
            return self

        def set_conflict(self, conflict: str):
            self.conflicts = conflict
            return self

        def set_supplementary_files(self, supplementary_file: str):
            self.supplementary_files.append(supplementary_file)
            return self

        def set_cover_letter(self, cover_letter: str):
            self.cover_letter = cover_letter
            return self

        def set_supporting_files(self, supporting_file: str):
            self.supporting_files.append(supporting_file)
            return self

        def set_manuscript_attachment(self, manuscript_attachment: str):
            self.manuscript_attachments.append(manuscript_attachment)
            return self

Step 3: `build()` — Validate First, Create Second

This is the heart of the pattern. Validation happens entirely inside build() — before ManuscriptSubmissionService(...) is ever called. If validation fails, the object is never created. Not even for a millisecond.

        def build(self):
            if self.title is None:
                raise IOError("ManuscriptSubmissionService requires a title")
            if self.abstract is None:
                raise IOError("ManuscriptSubmissionService requires an abstract")
            if len(self.authors) == 0:
                raise IOError("ManuscriptSubmissionService requires authors")
            if len(self.affiliations) == 0:
                raise IOError("ManuscriptSubmissionService requires affiliations")
            if len(self.manuscript_attachments) == 0:
                raise IOError("ManuscriptSubmissionService requires a manuscript attachment")

            # only NOW does the object get created
            return ManuscriptSubmissionService(
                self.title, self.abstract, self.authors, self.affiliations,
                self.keywords, self.references, self.funding, self.ethics,
                self.conflicts, self.supplementary_files, self.cover_letter,
                self.supporting_files, self.manuscript_attachments
            )

Step 4: Block Direct Instantiation

There's still a loophole. Nothing stops someone from bypassing the builder entirely:

# bypasses all validation — object created in invalid state
submission = ManuscriptSubmissionService(None, None, [], [], [], [], None, None, None, [], None, [], [])

Python doesn't support private constructors. But we can enforce it with a UUID token and a custom exception:

from uuid import uuid4, UUID


class InitializationError(Exception):
    """Raised when ManuscriptSubmissionService is instantiated directly instead of via builder."""
    def __init__(self, message: str):
        super().__init__(message)


class ManuscriptSubmissionService:
    _Builder_token = uuid4()  # generated once at class definition time

    def __init__(self, token_: UUID, title: str, ...):
        if token_ != self._Builder_token:
            raise InitializationError(
                "ManuscriptSubmissionService cannot be initialized directly. "
                "Use ManuscriptSubmissionService.builder()"
            )
        # rest of init...

The token is generated once when the class is defined. The builder knows it. Nobody outside does. Direct instantiation raises InitializationError immediately.

Why a custom exception instead of ValueError? In production, specific exceptions mean:

Easier debugging — you know exactly what failed
Callers can catch InitializationError specifically
Cleaner, more descriptive logs

The Complete Implementation

from uuid import uuid4, UUID


class InitializationError(Exception):
    """Raised when ManuscriptSubmissionService is instantiated directly instead of via builder."""
    def __init__(self, message: str):
        super().__init__(message)


class ManuscriptSubmissionService:
    _Builder_token = uuid4()

    def __init__(self, token_: UUID, title: str, abstract: str, authors: list,
                 affiliations: list, keywords: list, references: list,
                 funding: str, ethics: str, conflicts: str,
                 supplementary_files: list, cover_letter: str,
                 supporting_files: list, manuscript_attachments: list):
        if token_ != self._Builder_token:
            raise InitializationError(
                "ManuscriptSubmissionService cannot be initialized directly. "
                "Use ManuscriptSubmissionService.builder()"
            )
        self.title = title
        self.abstract = abstract
        self.authors = authors
        self.affiliations = affiliations
        self.keywords = keywords
        self.references = references
        self.funding = funding
        self.ethics = ethics
        self.conflicts = conflicts
        self.supplementary_files = supplementary_files
        self.cover_letter = cover_letter
        self.supporting_files = supporting_files
        self.manuscript_attachments = manuscript_attachments

    @staticmethod
    def builder():
        return ManuscriptSubmissionService.ManuscriptBuilder()

    class ManuscriptBuilder:
        def __init__(self):
            self.title = None
            self.abstract = None
            self.authors = []
            self.affiliations = []
            self.keywords = []
            self.references = []
            self.funding = None
            self.ethics = None
            self.conflicts = None
            self.supplementary_files = []
            self.cover_letter = None
            self.supporting_files = []
            self.manuscript_attachments = []

        def set_title(self, title: str):
            self.title = title
            return self

        def set_abstract(self, abstract: str):
            self.abstract = abstract
            return self

        def set_author(self, author: str):
            self.authors.append(author)
            return self

        def set_affiliation(self, affiliation: str):
            self.affiliations.append(affiliation)
            return self

        def set_keyword(self, keyword: str):
            self.keywords.append(keyword)
            return self

        def set_references(self, reference: str):
            self.references.append(reference)
            return self

        def set_funding(self, funding: str):
            self.funding = funding
            return self

        def set_ethics(self, ethics: str):
            self.ethics = ethics
            return self

        def set_conflict(self, conflict: str):
            self.conflicts = conflict
            return self

        def set_supplementary_files(self, supplementary_file: str):
            self.supplementary_files.append(supplementary_file)
            return self

        def set_cover_letter(self, cover_letter: str):
            self.cover_letter = cover_letter
            return self

        def set_supporting_files(self, supporting_file: str):
            self.supporting_files.append(supporting_file)
            return self

        def set_manuscript_attachment(self, manuscript_attachment: str):
            self.manuscript_attachments.append(manuscript_attachment)
            return self

        def build(self):
            if self.title is None:
                raise IOError("ManuscriptSubmissionService requires a title")
            if self.abstract is None:
                raise IOError("ManuscriptSubmissionService requires an abstract")
            if len(self.authors) == 0:
                raise IOError("ManuscriptSubmissionService requires authors")
            if len(self.affiliations) == 0:
                raise IOError("ManuscriptSubmissionService requires affiliations")
            if len(self.manuscript_attachments) == 0:
                raise IOError("ManuscriptSubmissionService requires a manuscript attachment")

            return ManuscriptSubmissionService(
                ManuscriptSubmissionService._Builder_token,
                self.title, self.abstract, self.authors, self.affiliations,
                self.keywords, self.references, self.funding, self.ethics,
                self.conflicts, self.supplementary_files, self.cover_letter,
                self.supporting_files, self.manuscript_attachments
            )

Usage — Plain English at the Call Site

submission = (
    ManuscriptSubmissionService.builder()
        .set_title("Covid-19 and its effects on respiratory systems")
        .set_abstract("This paper studies the long-term effects of Covid-19...")
        .set_author("Dr. Smith")
        .set_author("Dr. Jones")
        .set_affiliation("Oxford University")
        .set_affiliation("Imperial College London")
        .set_keyword("covid")
        .set_keyword("pandemic")
        .set_keyword("respiratory")
        .set_funding("WHO Grant 2023")
        .set_manuscript_attachment("covid19_paper.pdf")
        .build()
)

print(submission.title)    # Covid-19 and its effects on respiratory systems
print(submission.authors)  # ['Dr. Smith', 'Dr. Jones']

The Director — Presets for Known Configurations

Builder has an optional companion — the Director. It doesn't build anything itself. It just knows the right combination of steps to create commonly used presets.

we had different submission types — standard journal, fast-track, open access. Each had different default configurations. Instead of copy-pasting builder chains everywhere, the Director encapsulates those presets.

The Director should get a fresh builder for every preset call — not reuse the same builder instance, which would cause state pollution across calls.

class ManuscriptDirector:
    def __init__(self, builder):
        self.__builder = builder

    def standard_submission(self):
        return self.__builder().set_ethics("standard-ethics-declaration").set_conflict("no-conflicts-declared")

    def fast_track_submission(self):
        return self.__builder().set_ethics("expedited-ethics").set_cover_letter("fast-track-request-letter")


    def open_access_submission(self):
        return self.__builder().set_funding("open-access-fund-2024").set_ethics("standard-ethics-declaration")


director = ManuscriptDirector(ManuscriptSubmissionService.builder)

submission = (
    director.standard_submission()
        .set_title("Effects of Climate Change on Biodiversity")
        .set_abstract("This paper explores...")
        .set_author("Dr. Patel")
        .set_affiliation("Cambridge University")
        .set_manuscript_attachment("climate_paper.pdf")
        .build()
)

When do you need a Director? When you have multiple known configurations of the same object and you want a single place to manage those presets. When you only have one configuration — skip it. Don't over-engineer.

The Tradeoffs

What Builder Gives You

✅ Readable object creation
✅ Mandatory field enforcement
✅ No SRP violation
✅ Object never exists in an invalid state
✅ Optional fields with sensible defaults in the builder
✅ Method chaining

What Builder Costs You

1. Boilerplate — the OCP violation

Every new mandatory field means changes in 5 places:

ManuscriptSubmissionService.__init__ — add the parameter
ManuscriptBuilder.__init__ — add default None
Add a new setter method
Add validation in build()
Pass it when calling ManuscriptSubmissionService(...)

This is an Open/Closed Principle violation — adding a field forces modification of existing code in multiple places.

2. More Code for Simple Cases

A class with 2-3 fields doesn't need a Builder. You've added a nested class, a static method, and 3 setters for what could have been a simple constructor with keyword arguments. Apply Builder when the complexity justifies it.

3. Direct Instantiation is Not Truly Private

The UUID token trick works in practice, but it's a convention enforced at runtime — not a language feature. Python doesn't support private constructors. A determined developer can inspect ManuscriptSubmissionService._Builder_token and bypass the guard. This is an accepted limitation of the pattern in Python.

5. No Compile-Time Safety

Unlike Java or Kotlin where you can enforce mandatory fields at compile time, Python's Builder enforcement is entirely at runtime. A missing mandatory field only surfaces when build() is called — not when the setter is forgotten. Type checkers and tests help, but they don't eliminate the gap.

How We Optimized Manuscript Processing at a Scientific Publishing Platform

Madhukar Vissapragada — Sat, 27 Jun 2026 14:14:15 +0000

Section 1: The Domain

We build a product that processes academic manuscripts for scientific publishing companies. Authors submit their manuscripts in the form of DOCX files. Our system converts these DOCX files into XML — the industry standard format used across the entire publishing world.

Every publishing company has their own DTD (Document Type Definition) — a schema that defines the structure of their XML. This means the rules for how a manuscript should be structured, tagged, and formatted are different for every journal and every publisher.

The conversion pipeline

When an author submits a manuscript, here is what happens:

Author submits a DOCX file to our system
The system downloads the journal-specific rules file from S3
The rules file is used to convert the DOCX into a structured XML
During conversion, the system raises annotations and discrepancies — places where the manuscript does not conform to the journal's rules
The XML is then converted into an HTML preview
The author views the HTML on the UI and resolves the annotations and discrepancies raised by the system

The problem

The entire process — from DOCX submission to HTML preview — takes approximately 3 minutes.

The product owner received feedback from authors that 3 minutes is too long. Authors are sitting and waiting, not knowing if their manuscript is being processed correctly. This is a poor experience and directly impacts the author's trust in the platform.

The product owner has asked us to reduce the waiting time.

Section 2: Finding the Bottleneck

As part of improving the system latency we started profiling the pipeline to identify where the 3 minutes were being spent.

We quickly identified that downloading the rules file from S3 on every submission was the biggest contributor to latency.

Why is the download so slow?

The rules files are large — each rules file is close to 1 GB in size. Transferring 1 GB over the network on every single submission adds significant overhead before any actual processing even begins.

Understanding the scale

We receive 3,000 to 5,000 submissions per day
Our system supports 300 different journal types
Based on our metrics, 60 to 70 journals account for 90% of all submissions

This last observation is the key insight. We are downloading 1 GB files from S3 thousands of times a day — but for the same 60 to 70 journals repeatedly. The same rules file is being downloaded over and over again for every submission of the same journal type.

Current architecture

The decision

We decided to break this network latency on every request. Instead of downloading the rules file from S3 on every submission, we would cache the rules files of the top 70 journals locally on the application server.

This would eliminate the redundant S3 downloads for 90% of our submissions and directly reduce the processing time for the majority of authors.

Section 3: Why Local Cache?

Before jumping into the implementation we had to decide what type of cache to use.

Global cache — ruled out

At first glance a global cache seems like a reasonable choice. But when we thought about it carefully we realised it does not solve our problem at all.

With a global cache:

Rules file is downloaded from S3 → stored in global cache server
For every submission, app server downloads the rules file from the global cache server

We have simply moved the bottleneck. We are still transferring 1 GB of data over the network for every request — just from a different source. The latency problem remains unsolved.

Local cache — the right choice

With a local cache, the rules file is stored directly on the application server's HDD. When a submission comes in:

If the rules file is already on the local HDD → use it directly, zero network transfer
If not → download from S3 once, store locally, use it

Every subsequent submission for the same journal type uses the locally cached file with no network overhead at all.

Storage cost

We support 300 journal types but only 60 to 70 journals account for 90% of submissions. If we cache the rules files for these 70 journals:

70 journals × 1 GB per rules file = 70 GB per application server

70 GB on a local HDD is completely acceptable. HDD storage is cheap and this is a one time cost per server.

Local cache is the right fit for our use case.

Section 4: The Consistency Problem

Now that we have decided on local caching, we need to think carefully about cache invalidation. Before choosing an algorithm, we need to understand what kind of consistency our system requires.

Does eventual consistency work here?

Let's think through the scenario.

We cache the rules files locally and start processing manuscripts. Now assume that a domain expert — someone from the manuscript team or a peer reviewer — identifies that one of the journal rules has a bug. A rule has been added incorrectly, changed, or deleted. A separate uploader service fixes the rules file, uploads the new version to S3, and updates the DB.

But here is the problem — our application servers have the old rules file cached locally. Before the cache gets invalidated, our system processes around 100 manuscripts using the wrong rules. The XML is generated, the HTML is rendered, and authors go to the UI and spend time resolving annotations and discrepancies based on incorrect rules.

Later, the quality check team identifies that the XML was generated using outdated rules. Now we have to go back to those authors and tell them their work needs to be redone.

Authors will not accept this. The credibility of the platform is at stake.

Eventual consistency does not fit this use case.

We need immediate consistency

Our system must ensure that the moment a rules file is updated, every subsequent manuscript processed by any application server uses the updated rules. No exceptions.

The natural choice for immediate consistency is Write Through cache — write to both cache and the source of truth atomically. But this is extremely hard to implement in a distributed environment.

With multiple application servers each holding their own local cache, achieving atomic writes across all of them simultaneously is practically impossible. If we use 2 Phase Commit to coordinate these writes, the latency overhead would be so high that it defeats the entire purpose of caching in the first place.

The solution — Write Through with Versioning

We need immediate consistency without the complexity and latency of 2 Phase Commit. The solution is a tweaked approach — write through with versioning.

Before we explain the solution, let us look at how the current code is implemented.

Section 5: The Code

v1 — The naive approach

This is how the system was originally implemented. Every time a manuscript submission comes in, the rules file is hardcoded based on the journal ID.

import os

def handle_request_v1(request):
    author_id = request.author_id
    journal_id = request.journal_id
    manuscript = request.manuscript

    author_info = pgsql.get_author_info(author_id)
    if author_info is None:
        return 404

    journal_info = pgsql.get_journal_info(journal_id)
    if journal_info is None:
        return 404

    rules_file_name = f"{journal_id}_rules.csv"

    if not os.path.exists(rules_file_name):
        download_from_s3([journal_info.rules_file_url])

    convert_to_xml(manuscript, rules_file_name)

The problem with v1:

The rules file name is hardcoded as {journal_id}_rules.csv. If the rules file is updated in S3 and the DB, the file name on disk stays the same. The os.path.exists check returns True — the file exists — but the content is stale. The system silently serves outdated rules with no way of knowing.

This is exactly the eventual consistency problem we described.

v2 — Write Through with Versioning

The key insight is simple — instead of hardcoding the rules file name, we fetch it from the DB on every request. The DB always holds the latest file name. The file name itself acts as the version.

When a domain expert updates the rules file:

The uploader service uploads the new rules file to S3 with a timestamp in the file name — e.g. journal_42_rules_2025-03-16_08:00.csv
On success, the uploader updates the file name in the DB
These two writes do not need to be atomic — if S3 succeeds but DB update fails, the new file is simply not being used yet. It causes no harm.

Now when the next submission comes in:

App server fetches journal_info from DB — gets the new file name
Checks if this file exists on local HDD
It does not — because the old file had a different name
Downloads the new file from S3
Processes the manuscript with the correct updated rules

The old cached file is simply never used again and will eventually be evicted by LRU.

import os

def handle_request_v2(request):
    author_id = request.author_id
    journal_id = request.journal_id
    manuscript = request.manuscript

    author_info = pgsql.get_author_info(author_id)
    if author_info is None:
        return 404

    journal_info = pgsql.get_journal_info(journal_id)
    if journal_info is None:
        return 404

    rules_file_name = journal_info.rules_file_name

    if not os.path.exists(f'/rules_data/{rules_file_name}'):
        download_from_s3([journal_info.rules_file_url])

    convert_to_xml(manuscript, f'/rules_data/{rules_file_name}')

What changed from v1 to v2:

	v1	v2
Rules file name	Hardcoded `{journal_id}_rules.csv`	Fetched from DB `journal_info.rules_file_name`
Versioning	None — stale files served silently	Timestamp in file name acts as version
Consistency	Eventual	Immediate
Cache path	No prefix	`/rules_data/` prefix

The key insight:

We are not caching the file name — the file name is fetched fresh from the DB on every request so it can never be stale. We are only caching the actual rules file content on disk. This gives us immediate consistency without 2 Phase Commit.

Section 6: Eviction, Routing and Tradeoffs

Cache Eviction — LRU

We use LRU (Least Recently Used) eviction. It is the best fit for our use case because every rules file has a timestamp in its name. The operating system automatically maintains read and write timestamps for all files on disk. When the cache storage threshold is reached, we simply evict the file that was least recently accessed to make space for new downloads.

This requires no additional tracking logic — the OS does it for us naturally.

Routing — Round Robin

We use Round Robin for routing requests across application servers.

The reason is that rules files are replicated across all app servers — not sharded. Every app server can serve any journal type.

Why not shard by journal ID? If we route all requests for a specific journal to a specific server, and that journal gets a high volume of submissions, that one server gets bombarded while all others remain idle. This creates an uneven load distribution that defeats the purpose of horizontal scaling.

With Round Robin, any request can go to any server. All servers gradually cache the top 70 journals and serve submissions equally.

The tradeoff — cold start latency

There is one unavoidable tradeoff in this design. The first request for any journal type will always be slow — the rules file has to be downloaded from S3 before processing can begin. Only subsequent requests benefit from the local cache.

This is acceptable in most cases but can be a poor experience if a high volume of submissions is expected for a specific journal — for example, right before a submission deadline.

Overcoming the cold start — warm up strategy

We can proactively eliminate this cold start problem using a cache warm-up background job.

Before a high volume period is expected:

A background job sends dummy small manuscript submissions for the most frequently used journals
The app servers download and cache the rules files
The dummy XML outputs are deleted after caching

When the real high volume of author submissions arrives, the rules files are already cached on all app servers and every submission gets the fast path.

The tradeoff is additional background job infrastructure and a small amount of unnecessary S3 download cost. But the author experience is significantly improved.

Caching — From Fundamentals to Production Systems

Madhukar Vissapragada — Fri, 26 Jun 2026 14:06:02 +0000

Section 1: What is Caching?

Caching is a technique to store a small amount of data for faster retrieval and higher throughput.

Caching is used in two scenarios:

High read frequency — when a particular piece of data is requested a very high number of times
Expensive computation — when fetching data requires complex joins or aggregations — caching the result avoids recomputing it every time

Computer memory hierarchy

To understand why we cache only a small amount of data, we need to look at how computer memory is structured. The higher up in the hierarchy, the faster the read — but the less space available.

Level	Speed	Size
Registers	Ultra fast	Bytes
CPU Cache	Very fast	Kilobytes
RAM	Fast	Gigabytes
Flash / SSD	Slower than RAM	Terabytes
Hard Disk	Slowest	Terabytes

The key insight — the smaller the storage, the faster the retrieval. Caching works by moving frequently accessed data up this hierarchy, closer to the CPU and away from the slow disk.

Section 2: Caching at Various Levels of the Application

Caching does not happen at just one place in a system. It happens at multiple levels — from the browser all the way to the backend database layer.

Frontend caching

As a frontend developer, you can cache data directly in the browser using:

Local Storage — persists even after the browser is closed
Session Storage — cleared when the browser tab is closed
Indexed DB — a full browser-side database for storing large structured data

Third party caching — CDN

Just like DNS is a third party service that resolves domain names, a CDN is a third party service that offers caching as a service.

CDN providers like Akamai, Cloudflare, and AWS Cloudfront have set up edge nodes around the globe. Static content is served directly from these edge nodes — close to the user — instead of travelling all the way to the origin server.

What can be cached in edge nodes? Any static content larger than 10KB — images, videos, code files, web pages, audio files.

CDN providers either own or rent space in data centers around the world to set up these edge nodes. Today, 70% of internet traffic is handled by CDNs — they are the backbone of the modern internet.

Backend caching

On the backend, we have three types of cache:

Local Cache — cache lives inside the application server's memory. Inherently distributed since every app server has its own cache.
Distributed Global Cache — a separate dedicated cache cluster shared by all app servers. Data is either sharded or replicated across cache servers.
Single Global Cache — one dedicated cache server shared by all app servers. No routing algorithm needed.

Section 3: CDN Deep Dive — How Facebook Serves Videos

Let's understand how CDNs work with a real world example.

Upload flow — Madhukar uploads a video

Madhukar sends a video upload request to Facebook's application servers
Facebook's application servers upload the video to a file storage service like S3 and receive back an S3 url
Facebook's application servers send the S3 url to a CDN provider like Akamai to cache the video across their edge nodes
The application server stores all the metadata in the database — video_id, s3_url, cdn_url

View flow — Gopi watches the video
Gopi sends a GET request to Facebook's application servers
Facebook's application servers respond with the CDN url
Gopi's device sends a DNS request to resolve the CDN url to an IP address

How does DNS find the nearest edge node?

Geo DNS
The authoritative DNS servers of the CDN look at the IP address of the user's DNS resolver and return the nearest edge node IP. But there is a problem — if a user changes their DNS server to something like 8.8.8.8, the authoritative server sees Google's IP instead of the user's actual location and may return a far away edge node. This is why Geo DNS is not widely used.

Anycast
All edge nodes share the same IP address. When Gopi's request is sent to that IP, the internet's BGP routers calculate the number of hops required to reach each edge node and automatically route the request to the one with the fewest hops — the nearest edge node. Gopi's DNS settings are completely irrelevant here.

Gopi's request is routed to the nearest edge node via BGP
The video is served from the edge node

Do we cache on the first request?

No. CDNs do not cache on the first or second request. This is called 2 hit caching.

Over time CDN providers observed that many requests are one-hit wonders — a file gets requested once and never again. Caching those wastes precious edge node memory. So CDN providers set a threshold of 2 — only after the third request is the content cached and served from the edge node.

Can the backend server access the CDN?

Technically yes — but CDNs are client facing. They are designed to serve content to end users, not to backend servers.

Section 4: Cache Eviction

Cache memory is limited. When the cache is full and a new record needs to be inserted, the system must remove something to make space. This is called cache eviction.

There are several eviction policies:

LRU — Least Recently Used
Removes the item that has not been accessed for the longest time. The assumption is that if something hasn't been used recently it is unlikely to be used again soon. LRU is the industry standard and is used by the majority of applications.

LFU — Least Frequently Used
Removes the item that has been accessed the fewest number of times overall. Useful when some data is accessed in bursts but overall rarely used.

FIFO — First In First Out
Removes the item that was inserted first regardless of how often it was accessed. Simple but often inefficient — a very popular item that was cached early could get evicted even if it is still heavily used.

Policy	Evicts	Best for
LRU	Least recently accessed	General purpose — industry standard
LFU	Least frequently accessed	Data with stable long term access patterns
FIFO	Oldest inserted item	Simple systems where recency does not matter

Section 5: Cache Invalidation Patterns

Cache invalidation means the data in the cache is no longer valid and needs to be updated or removed. There are four main patterns to handle this.

TTL — Time To Live (Cache Aside)

TTL provides eventual consistency and is ideal for simple DB queries.

Every cached item has an expiry time. After the TTL expires the data is no longer valid.

All write requests go directly to the DB
All read requests go to the cache
- Cache hit → serve from cache
- Cache miss → fetch from DB, asynchronously write to cache The system can serve stale data until the TTL expires. This is acceptable for most use cases where eventual consistency is fine.

Write  →  DB only
Read   →  Cache → hit: serve | miss: fetch from DB → populate cache

Write Around

Write Around is ideal for scenarios where fetching data from the DB is very complex — involving many joins and aggregations.

All write requests go directly to the DB
All read requests go to the cache
A cron job runs periodically, pulls data from the DB and writes it to the cache
Until the cron job runs, the cache will serve stale data
When the first request comes in there is no data in the cache — a 404 is returned to the client

Write  →  DB only
Read   →  Cache → hit: serve | miss: 404
Cron   →  DB → Cache (periodic sync)

Write Through

Write Through provides immediate consistency.

Every write request goes to both the cache and the DB atomically. All read requests go to the cache — which is always fresh.

On a single server — atomic writes are achievable using locks, since both cache and DB share the same memory
On a distributed system — atomic writes require 2 Phase Commit, which increases latency and adds rollback complexity Single server implementation:

# Both writes succeed
lock.acquire()
try:
    write_to_cache(key, value)
    write_to_db(key, value)
finally:
    lock.release()

# Cache write fails
lock.acquire()
try:
    write to cache  ✗  # fails here
    write to DB        # never reached
except CacheError:
    # nothing to rollback
    log("cache write failed")
finally:
    lock.release()

# DB write fails
lock.acquire()
try:
    write to cache  ✓
    write to DB     ✗  # fails here
except DBError:
    rollback cache     # undo cache write
    log("db write failed, cache rolled back")
finally:
    lock.release()

Write  →  Cache + DB atomically
Read   →  Cache always (always fresh)

Write Back

In Write Back all reads and writes go directly to the cache. A background cron job periodically fetches data from the cache and dumps it into the DB.

No data consistency — DB is always behind
No stale reads — since all reads come from cache which has the latest data
Data loss is possible — if the cache crashes before the cron job runs, that window of data is lost
Ideal for use cases where intermediate data points are not critical and some data loss is acceptable

Write  →  Cache only
Read   →  Cache only (always fresh)
Cron   →  Cache → DB (periodic dump)

Conclusion

Caching is not a single technique — it is a layered strategy that operates at every level of a modern application, from the browser to the CDN to the backend database layer.

The choice of cache pattern is always a tradeoff:

If your system can tolerate stale data — TTL or Write Around keeps things simple and scalable
If your system requires immediate consistency — Write Through gives you that guarantee but at the cost of latency and distributed complexity
If your system is write-heavy and can tolerate data loss — Write Back gives you the highest write throughput

The next time your app loads a video in milliseconds, a feed in under a second, or a product page instantly — caching is quietly doing its job at every layer of the stack.

Credits

This blog is based on the HLD curriculum taught at Scaler Academy by instructor Pragy Agarwal.

Consistent Hashing — A System Design Deep Dive

Madhukar Vissapragada — Tue, 16 Jun 2026 16:07:06 +0000

Consistent Hashing — The Algorithm Behind Scalable Systems

Section 1: The Problem

We have customers from Amazon and Flipkart who want to wishlist products from these platforms. Assume that this wishlist feature doesn't exist natively on Amazon or Flipkart. We have built a service where users can save and manage their wishlist from any e-commerce platform — all in one place.

Our service is gaining popularity and experiencing massive scale. To handle this, we first tried vertical scaling — replacing our existing server with a more powerful one. But even that wasn't enough to meet our scale expectations.

So we decided to go with horizontal scaling — adding more servers of the same configuration to share the load.

Vertical scaling — Replacing the existing server with a more powerful server.

Horizontal scaling — Adding more servers of the same existing configuration.

Horizontal scaling solves two problems at once:

Too much traffic — a single server cannot handle all the incoming requests
Too much data — a single server cannot store all the wishlist data

Both of these force us to distribute across multiple servers. But now we have a new problem — we have multiple servers, so which server handles which request? And how do we know which server holds which user's data?

Section 2: What is Sharding?

Before we talk about routing, we need to understand how data is distributed across servers.

If we normalize a table into two separate tables — say users and wishlist — on the same server, we call it normalization.

If we do the same thing but place each table on a different server, we call it separation of concerns or microservices.

If we split the data across multiple tables where the schema remains the same but the data is divided based on a sharding key, and we do this on the same server, we call it horizontal partitioning.

If we do the same thing across different servers, we call it Data Sharding. Data sharding is always done using a sharding key.

Section 3: What makes a good routing algorithm?

Now that we have sharded the data across multiple servers, we need to understand how to route requests correctly.

If user A stores their wishlist data and we route them to Server 5, then the next time user A wants to view their wishlist, we must route that request to Server 5 again — because that is where their data lives. This is the core problem a good routing algorithm must solve.

Before we pick an algorithm, let's define what makes a routing algorithm good:

Fast — computationally efficient to determine which request goes to which server
Equal distribution — load must be distributed equally across all servers
Minimal data movement — when we add or remove servers, the amount of data that needs to move should be minimal
Deterministic — all load balancers must always be in sync without constant communication between them, and requests of the same type must always go to the same server

Section 4: Round Robin

Round Robin sends each request to the next server in sequence using a simple modulo operation.

servers = ["A", "B", "C", "D"]

def handle_request(request):
    key = request.user_id
    total_servers = len(servers)
    server_id = int(key) % total_servers
    server = servers[server_id]
    forward_request(request, server)

Initial distribution with 4 servers:

A → user 0, 4, 8 ...
B → user 1, 5, 9 ...
C → user 2, 6, 10 ...
D → user 3, 7, 11 ...

When server B crashes, N changes from 4 to 3:

A → user 0, 3, 6, 9 ...
C → user 1, 4, 7, 10 ...
D → user 2, 5, 8, 11 ...

Only users 1, 5, 9... should have moved since their server B crashed. But almost every user got remapped to a different server — causing massive unnecessary data movement.

	Round Robin
Fast	✓
Equal distribution	✓
Minimal data movement	✕
Deterministic	✓

Verdict — Round Robin fails on minimal data movement. It works only when the number of servers is fixed and never changes.

Section 5: Bucketing (Range Based)

Bucketing assigns a range of user IDs to each server rather than using modulo.

server_list = ["A", "B", "C", "D"]
total_users = 400

def handle_request(request):
    key = request.user_id
    total_servers = len(server_list)           # 4
    bucket_size = total_users // total_servers  # 400 / 4 = 100
    server_id = math.floor(key / bucket_size)
    server = server_list[server_id]
    forward_request(server, request)

Initial distribution with 4 servers and 400 users:

A → user 0   – 99
B → user 100 – 199
C → user 200 – 299
D → user 300 – 399

When server B crashes, the remaining 3 servers must cover all 400 users:

A → user 0   – 132
C → user 133 – 265
D → user 266 – 399

All buckets change — causing massive unnecessary data movement across every server, not just server B's users.

When new users sign up, we cannot accommodate them on existing servers because the bucket ranges are already decided. It is impossible to add new users without first buying more servers.

	Bucketing
Fast	✓
Equal distribution	✓
Minimal data movement	✕
Deterministic	✓

Verdict — Bucketing is even worse than Round Robin. Not only does it reshuffle all data when a server is added or removed, it cannot even accommodate new users without buying new servers first.

Section 6: Mapping Table

What if instead of a formula-based solution we simply track which user is on which server? The load balancer maintains a hashmap from user_id to server_id.

mapping = {
    "user_A": "server_1",
    "user_B": "server_3",
    "user_C": "server_2",
    ...
}

def handle_request(request):
    server = mapping[request.user_id]
    forward_request(server, request)

When a server crashes, only the users on that server are reassigned to a random available server. No other users are affected.

When a new server is added, we take the total user count, pick a proportional number of random users, and move them to the new server to balance the load.

This gives us minimal data movement — only the affected users move, everyone else stays put.

The fatal flaw — all load balancers must always have the same mapping table. Keeping this table in sync across all load balancers at low latency is practically impossible. The CAP theorem tells us we cannot achieve perfect consistency with low latency at the same time.

	Mapping Table
Fast	✓
Equal distribution	✓
Minimal data movement	✓
Deterministic	✕

Verdict — Mapping Table finally solves minimal data movement but fails on the hardest problem — keeping all load balancers in sync at low latency is impossible.

Section 7: Consistent Hashing

Consistent Hashing is a hash based technique to route requests. It satisfies all four criteria — making it the algorithm of choice for distributed systems.

	Consistent Hashing
Fast	✓
Equal distribution	✓
Minimal data movement	✓
Deterministic	✓

How it works

A hash function is a digest function — it takes anything as input and returns an output within a fixed range. Common hash functions include MD5, SHA256, SHA512, and Murmur3.

In Consistent Hashing, we use a logical hash ring — the output space of the hash function bent into a circle.

We use k hash functions to place each server at k virtual spots on the ring. A good value of k is 32 or 64.
We use 1 hash function to place each user at one spot on the ring.
When a request arrives, we find the first server clockwise from the user's position on the ring using binary search.
That server handles the request and stores the user's data. This means:
All load balancers run the same hash functions and see the same ring — no sync needed
When a server is added or removed, only the users nearest to it on the ring are remapped — minimal data movement
The same user always hashes to the same spot — deterministic ### Implementation

import mmh3
import bisect

def custom_hash(data: str, i: int) -> int:
    # Murmur3 hash with seed i, output range 0..9
    return mmh3.hash(data, seed=i, signed=False) % 10

def build_ring(servers, k):
    ring = []
    for server in servers:
        for index in range(1, k + 1):
            h = custom_hash(str(server), index)
            ring.append((h, server))
    ring = sorted(ring)
    return ring

def route(user):
    user_hash = custom_hash(str(user), 0)
    # Find the first server clockwise using binary search
    index = bisect.bisect_right(ring, (user_hash, user))
    index = index % len(ring)  # wrap around the ring
    server_hash, server = ring[index]
    print(f"{user} hashed to {user_hash} → routed to {server} at position {server_hash}")

servers = ['10.11.1.13', '10.11.2.17', '10.11.13.167', '10.11.12.255', '10.11.1.0']
ring = build_ring(servers, 32)

route('Vishal')
route('Sanjana')

Notice that no matter how many times we call route('Vishal'), he always lands on the same server — that is Consistent Hashing being deterministic.

Adding a new server

When a new server K is added to the ring between user A and server H:

Before: User A → Server H
New server K added between User A and Server H
After:  User A → Server K

User A's existing data moves from Server H → Server K. All other users are completely unaffected. This is minimal data movement — only the users who fall between the new server and its predecessor on the ring are impacted.

When a server crashes

When a server crashes, all the users whose data lived on that server are automatically remapped to the next server clockwise on the ring.

Before: User A → Server H
Server H crashes
After:  User A → Server K  (next server clockwise)

User A's data moves from Server H → Server K
All other users → untouched

In both cases — adding or removing a server — only a small slice of users near that server on the ring are affected. Everyone else continues routing to the same server as before.

This is what makes Consistent Hashing fundamentally better than Round Robin and Bucketing — where any change in the number of servers causes a full reshuffle of almost all users.

Conclusion

Every routing algorithm we evaluated satisfied some criteria but failed on others.

Round Robin uses a simple modulo operation — user_id % N — which means any change in N causes almost all keys to remap to different servers, making it unusable in dynamic environments. Bucketing improves readability but suffers the same remapping problem and additionally cannot accommodate new users without pre-purchasing servers. Mapping Table achieves true minimal data movement but introduces a distributed consistency problem — keeping the hashmap in sync across all load balancers at low latency violates the guarantees of the CAP theorem.

Consistent Hashing resolves all four criteria in a single algorithm. By placing both servers and users on a logical hash ring using k independent hash functions, it guarantees:

O(log Nk) routing time via binary search on a sorted ring
Near-perfect load distribution as k approaches 32 or 64
1/N data movement when a server is added or removed — only the affected segment of the ring remaps
Zero inter-LB communication — since all load balancers share the same hash functions and server list via heartbeat, they independently compute identical routing decisions This is why Consistent Hashing is the foundation of distributed systems at scale — Cassandra, DynamoDB, Redis Cluster, and CDN routing systems like Cloudflare all rely on it.

DEV Community: Madhukar Vissapragada

Builder Design Pattern - Simplified

Fix 1: Named Parameters

Fix 2: Validation Inside the Constructor

Fix 3: Setters

Three Approaches. Three Failures.

What Is the Builder Pattern?

Building It Step by Step

Step 1: Separate the Classes

Step 2: The Builder with Method Chaining

Step 3: build() — Validate First, Create Second

Step 4: Block Direct Instantiation

The Complete Implementation

Usage — Plain English at the Call Site

The Director — Presets for Known Configurations

The Tradeoffs

What Builder Gives You

What Builder Costs You

How We Optimized Manuscript Processing at a Scientific Publishing Platform

Section 1: The Domain

The conversion pipeline

The problem

Section 2: Finding the Bottleneck

Why is the download so slow?

Understanding the scale

Current architecture

The decision

Section 3: Why Local Cache?

Global cache — ruled out

Local cache — the right choice

Storage cost

Section 4: The Consistency Problem

Does eventual consistency work here?

We need immediate consistency

The solution — Write Through with Versioning

Section 5: The Code

v1 — The naive approach

v2 — Write Through with Versioning

Section 6: Eviction, Routing and Tradeoffs

Cache Eviction — LRU

Routing — Round Robin

The tradeoff — cold start latency

Overcoming the cold start — warm up strategy

Caching — From Fundamentals to Production Systems

Section 1: What is Caching?

Computer memory hierarchy

Section 2: Caching at Various Levels of the Application

Frontend caching

Third party caching — CDN

Backend caching

Section 3: CDN Deep Dive — How Facebook Serves Videos

Upload flow — Madhukar uploads a video

View flow — Gopi watches the video

How does DNS find the nearest edge node?

Do we cache on the first request?

Can the backend server access the CDN?

Section 4: Cache Eviction

Section 5: Cache Invalidation Patterns

TTL — Time To Live (Cache Aside)

Write Around

Write Through

Write Back

Conclusion

Credits

This blog is based on the HLD curriculum taught at Scaler Academy by instructor Pragy Agarwal.

Consistent Hashing — A System Design Deep Dive

Consistent Hashing — The Algorithm Behind Scalable Systems

Section 1: The Problem

Section 2: What is Sharding?

Section 3: What makes a good routing algorithm?

Section 4: Round Robin

Section 5: Bucketing (Range Based)

Section 6: Mapping Table

Section 7: Consistent Hashing

How it works

Adding a new server

When a server crashes

Conclusion

Step 3: `build()` — Validate First, Create Second