DEV Community: Chetan Singh

The CAP Theorem: A Guide for Developers Who Build Real-World Systems

Chetan Singh — Wed, 06 Aug 2025 07:06:28 +0000

As software engineers, architects, and data professionals, we build systems that live on networks. And networks, by their very nature, are unreliable. Understanding how to design resilient systems in the face of this unreliability is arguably one of the most critical skills in modern software engineering.

This guide is a deep dive into the CAP Theorem, the fundamental law that governs distributed systems. We won't just define it; we'll build it up from first principles, explore its profound implications, and see how it dictates the architecture of the databases and services you use every day.

Chapter 1: The Genesis of the Problem - Why Distribute?

Before we can understand the solution, we must deeply understand the problem. Let's start with a single server running your application and its database.

The Single-Node Utopia (and its Limits)

Imagine a lone, brilliant baker who runs a single, highly efficient bakery. This is our single-node system.

Simplicity: Everything is in one place. The baker knows the exact state of his inventory at all times.
Inherent Consistency: It's impossible to sell the same cake twice. There is one source of truth.

But success brings problems. The line of customers grows, and the baker can't keep up. He has two options to scale:

Vertical Scaling ("Get a Bigger Oven"): He could buy a much larger, faster oven. In computing, this is like upgrading to a server with a faster CPU, more RAM, or faster SSDs. This works, but only up to a point. You eventually hit a hard ceiling imposed by physics and cost. There's no infinitely large oven you can buy.
Horizontal Scaling ("Open More Branches"): He could hire more bakers and open new, identical branches of his bakery across the city. In computing, this means adding more servers (nodes) to a cluster to share the load.

This second option, horizontal scaling, is the birth of a distributed system.

Definition: Distributed System
A collection of independent computers (nodes) that work together and appear to its users as a single, coherent system.

This approach also solves another critical problem: fault tolerance. If our original baker gets sick (i.e., the server crashes), the business closes. In our distributed bakery, if one baker gets sick, the other branches can still serve customers. The system is resilient to failure.

However, by distributing our system, we've traded a simple set of problems (physical limits) for a new, far more complex set of challenges related to communication and agreement.

Chapter 2: The Three Sacred Promises

When we create a distributed system, we ideally want to provide three guarantees to our users.

1. Consistency (C)

Intuitive Analogy: Every bakery branch has the exact same inventory count at the exact same time. If a cake is sold at Branch A, Branch B knows about it instantly before it talks to its next customer.
Technical Definition: This refers to strong consistency or linearizability. It guarantees that every read operation receives the most recent write's value. Once a write completes, all subsequent reads (no matter which node they hit) will see that value. To the outside world, the system appears to behave as if it were a single, non-distributed machine.

2. Availability (A)

Intuitive Analogy: The bakery is always open for business. A customer can always walk into any branch, place an order, and get a response. The shop never just puts up a "Closed for Internal Issues" sign.
Technical Definition: Every request sent to a non-failing node in the system receives a non-error response. This doesn't guarantee the response contains the most up-to-date information, only that the system is operational and will respond.

3. Partition Tolerance (P)

Intuitive Analogy: A storm knocks out the phone lines between bakery branches. The branches are "partitioned" from each other and cannot communicate. The system must be designed to continue operating in some capacity despite this communication breakdown.
Technical Definition: The system continues to function even when an arbitrary number of messages are dropped or delayed by the network between nodes.

The Crucial Realization: 'P' is a Fact of Life

In any system that communicates over a network (i.e., any distributed system), partitions are inevitable. Routers fail, switches crash, network cables are unplugged, and network latency can spike so high that nodes time out and consider each other "down."

Because network partitions will happen, Partition Tolerance (P) is not a choice. It is a requirement for any real-world distributed system.

Chapter 3: The CAP Theorem - The Inevitable Trade-off

This brings us to the theorem itself, first formulated by Dr. Eric Brewer.

The CAP Theorem
A distributed data store can only simultaneously provide two of the following three guarantees: Consistency (C), Availability (A), and Partition Tolerance (P).

Since we've established that we must tolerate partitions (P), the theorem forces a monumental choice upon us:

When a network partition occurs, you must choose between Consistency and Availability.

Let's illustrate this with our bakery during the "storm."

Scenario 1: Choosing Consistency over Availability (CP)

A partition occurs. Branch A and Branch B cannot communicate. A customer walks into Branch B and asks for the last Red Velvet cake. The baker in Branch B has no way of knowing if that cake was just sold in Branch A.

To guarantee Consistency, the baker must prevent a situation where he sells a non-existent cake. He tries to contact the central database but fails. His only option is to refuse the sale.

Action: The baker tells the customer, "I'm sorry, our inventory system is currently unreachable. I cannot complete your order at this time."
Result: Consistency is maintained (no data corruption), but Availability is sacrificed (a legitimate request was denied).

CP Systems are for when correctness is an absolute, non-negotiable requirement.

Use Cases: Financial ledgers, payment processing, user authentication, master databases for e-commerce inventory.
Technology Examples: etcd, Zookeeper, Consul. Relational databases like PostgreSQL in a default single-master configuration behave as CP systems.

Scenario 2: Choosing Availability over Consistency (AP)

Faced with the same partition, the baker in Branch B decides that serving the customer is the top priority. He checks his local records, which think there is one cake left, and sells it.

Action: The baker sells the cake and the customer leaves happy.
Result: Availability is maintained (the system remained operational for the user). But Consistency is sacrificed. If the cake was also sold at Branch A, the bakery's data is now inconsistent.

This inconsistency must be resolved later when the partition heals. This leads to the concept of eventual consistency, where the system guarantees that eventually, if no new updates are made, all replicas will converge to the same value.

AP Systems are for when downtime is unacceptable and some data staleness is tolerable.

Use Cases: Social media feeds (seeing a 'like' a few seconds late is okay), view counters, real-time analytics, shopping carts (where an item might be found to be out of stock only at the final checkout step).
Technology Examples: Amazon DynamoDB, Apache Cassandra, Riak, CouchDB.

Chapter 4: What About CA? The Mythical Beast

If the choice is between CP and AP, what happened to CA (Consistent and Available)?

A CA system is a system that chooses to forgo Partition Tolerance. The only way to do this is to run all your components on a single machine or within a single, hyper-reliable hardware rack where a network partition is considered impossible.

A traditional, single-instance RDBMS (like MySQL running on one server) is a CA system. It is consistent and available... right up until that single server fails. It has no tolerance for partitions because it has nothing to be partitioned from. The moment you add a replica over a network, you are in the world of P, and the CAP trade-off becomes mandatory.

Chapter 5: Evolving the Conversation with PACELC

The CAP theorem is powerful, but it has a limitation: it only describes the system's behavior during a network partition. What about when the system is running normally? This is where the PACELC theorem provides a more complete picture.

The PACELC Theorem
If there is a Partition, a system must choose between Availability and Consistency.
Else (when operating normally), a system must choose between Latency and Consistency.

The "Else" part is the crucial extension. During normal operation, you face a new trade-off:

Latency (L): How quickly does the system respond to a user request? To achieve very low latency, a system might read from the nearest (but possibly not fully up-to-date) replica.
Consistency (C): Does every read see the absolute latest write? To guarantee this, a read might have to check with a quorum of replicas or travel to a master node, which takes more time (increasing latency).

This gives us a more nuanced way to classify systems:

System	Partition (P) Choice	Normal (E) Choice	Classification
DynamoDB, Cassandra	`A`vailability	`L`atency	PA/EL
MongoDB	`A`vailability	`C`onsistency	PA/EC
VoltDB, BigTable	`C`onsistency	`C`onsistency	PC/EC
PostgreSQL (default)	`C`onsistency	`C`onsistency	PC/EC

Chapter 6: Practical Takeaways for the Modern Developer

These theorems are not just academic. They are practical frameworks for making critical design decisions. When choosing a database or designing a service, ask yourself:

What is the business cost of being wrong? If your system processes payments, inconsistency is catastrophic. You need a CP system. If you're counting video views, a slight undercount for a few minutes is acceptable.
What is the business cost of being unavailable? If your system is the front page of an e-commerce site, being unavailable means losing sales directly. You need an AP system. If it's a batch-processing analytics job, being delayed by a few minutes is fine.
During normal operation, what do my users value more: speed or perfect accuracy? Do they need an instant response (favoring Latency), or can they wait a few hundred milliseconds for a guaranteed-correct answer (favoring Consistency)?

Understanding this spectrum—from CP to AP, and from Latency to Consistency—is the hallmark of a senior engineer. It allows you to move beyond "what's the hot new database?" and instead ask, "What are the fundamental trade-offs this system makes, and do they align with the needs of my application?

Took help of Gemini to write this article. I usually write so that I can come and revise later LOL. Glad if it helps someone!!

How to Manage Background Services with systemctl and systemd (with Celery Example)

Chetan Singh — Wed, 30 Jul 2025 08:49:17 +0000

When deploying services like Celery, Gunicorn, Redis, etc., on a Linux server — you don’t want to manually start them every time your server reboots. That’s where systemd and systemctl come in.

In this guide, I’ll explain how to:

Understand what systemctl is
Create a systemd service for something like Celery
Use essential commands to manage any service
Troubleshoot and view logs

What is systemd and systemctl?

systemd is the init system and service manager used by most modern Linux distributions.
systemctl is the command-line tool to interact with systemd.

It's used to start, stop, restart, enable (auto-start on boot), and debug services.

When Do You Need a systemd Service?
If you're running long-lived processes like:

Celery worker
Gunicorn for a Django/Flask app
Background scripts
Queues / job runners
Redis / PostgreSQL / any server

And you're not using Docker or Kubernetes, it’s best to define a systemd unit file for each of them.

Where Are These Service Files Stored?

| Location                  | Purpose                               |
| ------------------------- | ------------------------------------- |
| `/etc/systemd/system/`    | 🔧 Your custom services (recommended) |
| `/lib/systemd/system/`    | 📦 Package-installed services         |
| `~/.config/systemd/user/` | 👤 Per-user services (less common)    |

Use /etc/systemd/system/ for your custom services like Celery.

🧰 Example: Create a Celery Systemd Service

Assume:

Your app is in /home/ubuntu/yourproject
You’re using a virtual environment at /home/ubuntu/venv
Your Celery app is named yourproject Create the service file:

sudo nano /etc/systemd/system/celery.service

Paste this:
[Unit]
Description=Celery Worker Service
After=network.target

[Service]
User=ubuntu
Group=ubuntu
WorkingDirectory=/home/ubuntu/yourproject
ExecStart=/home/ubuntu/venv/bin/celery -A yourproject worker --loglevel=info
Restart=always

[Install]
WantedBy=multi-user.target

Enable and start the service

# Reload systemd to recognize the new service
sudo systemctl daemon-reload

# Enable to start on boot
sudo systemctl enable celery

# Start it now
sudo systemctl start celery

🧾 Handy systemctl Commands

Command	Description
`sudo systemctl start celery`	Start service
`sudo systemctl stop celery`	Stop service
`sudo systemctl restart celery`	Restart it
`sudo systemctl status celery`	View status
`sudo systemctl enable celery`	Start on boot
`sudo systemctl disable celery`	Remove from startup
`sudo systemctl daemon-reload`	Reload configs (after editing .service)

📑 View Logs with journalctl

# View logs for celery service
sudo journalctl -u celery

# Follow logs in real-time (like tail -f)
sudo journalctl -u celery -f

🧹 Bonus: Reload After Edits

Any time you edit a .service file, run:
sudo systemctl daemon-reload

💭 Conclusion

Using systemd with systemctl helps you make your production deployments clean, auto-recovering, and reboot-friendly.

Instead of manually managing your services, just define them once and let Linux take care of the rest.

_Written using Chatgpt for quick one stop basic guide. _

Just started learning Linux. Thought to share whatever I learned so far. Check it out !!

Chetan Singh — Sat, 14 Jun 2025 07:17:59 +0000

Chetan Singh

Jun 14 '25

Linux Fundamentals: What I Wish Someone Had Told Me When I Started

#linux #devops #programming

Comments 2

5 min read

Linux Fundamentals: What I Wish Someone Had Told Me When I Started

Chetan Singh — Sat, 14 Jun 2025 06:42:15 +0000

The Day I Decided to Befriend the Terminal

Last week, I made a decision that both excited and terrified me: I decided to learn Linux. Not just the surface-level "copy-paste commands from Stack Overflow" approach I'd been using, but really understand what's happening under the hood.
Why? Because every time I deployed code to a server, configured a CI/CD pipeline, or troubleshot a production issue, I felt like I was speaking a foreign language with a thick accent. Sure, I could get things done, but I knew I was missing something fundamental.
So here I am, documenting my journey from the very beginning. If you're in the same boat—comfortable with development but intimidated by Linux—this one's for you.

Linux is Actually Logical

The first thing that clicked for me was understanding that Linux isn't just a random collection of cryptic commands. It's built on a philosophy so elegant it's almost beautiful: do one thing, and do it well.
Think about it like this: instead of having one massive Swiss Army knife that's mediocre at everything, Linux gives you a precision toolbox where each tool excels at its specific job. The magic happens when you start combining these tools—that's where the real power lives.

The City Metaphor of Linux

Hardware is like the city's infrastructure—the roads, power lines, and water systems. It's there, it's essential, but most people don't think about it day-to-day.
The Linux Kernel is like city government. It makes all the important decisions about resource allocation, manages traffic between different parts of the system, and keeps everything running smoothly. When your web browser needs memory, it doesn't directly grab RAM—it politely asks the kernel, which decides whether to grant the request.
System Libraries are like public services—the post office, utilities, and municipal services that programs use to get things done. Your applications don't need to understand the complexities of hardware; they just use these well-defined services.
User Applications are like businesses and residents. They rely on all the layers below but don't need to understand the complex details of city infrastructure.
This mental model transformed how I think about troubleshooting. When something goes wrong, I now think: "Is this a hardware issue (infrastructure), a kernel issue (government), a library issue (public services), or an application issue (business)?"

The Shell

The terminal intimidated me for years. All those black screens with white text looked like something from a 1980s hacker movie. But here's what I learned: the shell is just a conversation interface with your computer.
When you open a terminal, you're essentially starting a chat session with Linux. That prompt—username@hostname:~$—isn't cryptic code. It's Linux politely saying: "Hi [username], you're on [hostname], currently in your home directory (~), and I'm ready to listen ($)."
Every command you type is like asking Linux a question or giving it a task:

whoami → "Who am I?"
pwd → "Where am I right now?"
ls → "What's here?"
cd Documents → "Take me to my Documents folder"

The shell remembers your conversation history, can complete your sentences (tab completion), and even lets you recall and modify previous "statements" using the up arrow. It's like having a very patient, very literal assistant who never forgets anything you've said.

The File System

Coming from Windows, Linux's file system felt alien at first. No C: drive? No D: drive? Just a single forward slash (/) at the root of everything?
But here's the beautiful part: Linux treats everything as a file, organized in one massive, logical tree. Imagine the world's most organized library, where every book, every piece of information, has exactly one correct place, and there's a logical system for finding anything.
The root directory (/) branches out into standard subdirectories:

/home - The residential area where users live
/etc - The rulebook (configuration files)
/usr - The commercial district (user programs)
/var - The dynamic area (logs, temporary files, changing data)

What blew my mind was realizing that this organization isn't arbitrary. When you install a program, it doesn't just dump files randomly. Executables go in /usr/bin, configs go in /etc, documentation goes in /usr/share/doc. This predictability makes the entire system maintainable and troubleshoot-able.
Your current location in this vast library is called your "working directory." You can reference files relative to where you are, or use absolute paths that start from the root. It's like giving someone directions by saying either "go to the kitchen" (relative) or "from the front door, go straight, turn right at the living room, then enter the kitchen" (absolute).

Processes

Every program running on Linux is a process—think of them as the living inhabitants of our Linux city. Each gets a unique Process ID (PID), like a social security number.
What fascinated me was learning that processes have family relationships. When you start a program from the shell, the shell becomes the parent, and your program becomes the child. This family tree affects how processes communicate and share resources.
Processes can be in different states:

Running: Actively using the CPU
Sleeping: Waiting for something (user input, network data, file operations)
Stopped: Paused by a signal
Zombie: Finished but not yet acknowledged by the parent

Understanding this helped me debug a hanging Node.js application last week. Instead of just restarting everything, I could identify the specific process, understand its state, and figure out what it was waiting for.

The Commands That Changed My Daily Workflow
Navigation that feels natural:

pwd          # Where am I?
ls -la       # What's here? (detailed view)
cd ~/project # Take me to my project folder
cd ..        # Go up one level
cd -         # Go back to where I was before

Finding things without clicking through folders:

find . -name "*.js"           # Find all JavaScript files
grep -r "TODO" .              # Find all TODO comments
locate filename               # Find files by name (super fast)

Understanding what's running:

ps aux                        # Show all processes
top                          # Real-time process monitor
htop                         # Even better process monitor

What I'm Learning Next
This is just the beginning. Next week, I'm diving into:

Text processing with pipes and filters
File permissions and security
Environment variables and shell customization
Package management and system maintenance

The Surprising Benefits I Didn't Expect
Learning Linux fundamentals has already improved my development workflow in unexpected ways:

Debugging is faster: I can quickly investigate system resources, process states, and file permissions when things go wrong.
Server management makes sense: Instead of blindly following deployment tutorials, I understand what each command actually does.
Docker and containerization clicked: Understanding processes and file systems made container concepts much clearer.
CI/CD scripts aren't mysterious anymore: I can read and write bash scripts that actually make sense.

This article is part of my "Learning in Public" series where I document my journey toward technical mastery. Follow along for honest insights, practical tips, and the occasional face-palm moment.

Building a Natural Language Task Scheduler with FastAPI, Notion, and LLM

Chetan Singh — Wed, 22 Jan 2025 12:02:25 +0000

Are you tired of manually creating and managing tasks in Notion? What if you could just say "Schedule a meeting with John tomorrow at 3pm" and have it automatically create a task? In this tutorial, we'll build exactly that - a natural language task scheduler that integrates with Notion!

What We're Building

We'll create a system that can:

Create tasks using natural language
Delete tasks when you ask
Reschedule tasks using simple commands
Integrate with your Notion workspace

For example, you can say:

"Schedule team meeting with John tomorrow at 3pm"
"Delete my meeting with Sarah"
"Reschedule project review to next Monday at 2pm"

Prerequisites

Before we start, make sure you have:

Python 3.8+ installed
A Notion account and API key
A Groq API key (for LLM integration)
Basic understanding of Python and APIs

Project Setup

First, create your project structure:

notion-ai-scheduler/
├── app/
│   ├── services/
│   │   ├── notion_service.py
│   │   ├── task_processor.py
│   │   └── scheduler_service.py
│   ├── main.py
├── .env
└── requirements.txt

Install required packages:

pip install fastapi uvicorn notion-client groq python-dotenv pytz

Set up your environment variables (.env):

NOTION_API_KEY=your_notion_api_key
NOTION_DATABASE_ID=your_database_id
GROQ_API_KEY=your_groq_api_key

Step 1: Setting Up Notion

Create a new Notion database with these columns:
Task (Title)
Status (Select: Not Started, In Progress, Done)
Due Date (Date)
Priority (Select: High, Medium, Low)
Create a Notion integration:
Go to https://www.notion.so/my-integrations
Create a new integration
Copy your API key
Share your database with the integration

Step 2: Creating the Notion Service

Create notion_service.py:

from notion_client import Client

class NotionService:
    def __init__(self):
        self.notion = Client(auth=NOTION_API_KEY)
        self.database_id = NOTION_DATABASE_ID

    def create_task(self, title: str, due_date: str = None, priority: str = "Medium"):
        try:
            properties = {
                "Task": {"title": [{"text": {"content": title}}]},
                "Status": {"status": {"name": "Not started"}},
                "Priority": {"status": {"name": priority}}
            }
            if due_date:
                properties["Due Date"] = {"date": {"start": due_date}}

            return self.notion.pages.create(
                parent={"database_id": self.database_id},
                properties=properties
            )
        except Exception as e:
            print(f"Error creating task: {e}")
            raise

    def delete_task(self, task_id: str):
        try:
            return self.notion.pages.update(
                page_id=task_id,
                archived=True
            )
        except Exception as e:
            print(f"Error deleting task: {e}")
            raise

Step 3: Creating the LLM Processor

Create task_processor.py:

from groq import Groq
from datetime import datetime
import pytz

class TaskProcessor:
    def __init__(self):
        self.client = Groq()
        self.ist = pytz.timezone('Asia/Kolkata')

    def process_input(self, user_input: str):
        current_time = datetime.now(self.ist)

        prompt = f"""You are a task management assistant.
        Current time: {current_time.strftime("%Y-%m-%d %H:%M:%S")}

        Convert the task request into a JSON with:
        - title: task description
        - due_date: YYYY-MM-DD HH:MM format or null
        - priority: High/Medium/Low
        - needs_clarification: true if time is vague
        """

        try:
            completion = self.client.chat.completions.create(
                messages=[
                    {"role": "system", "content": prompt},
                    {"role": "user", "content": user_input}
                ],
                model="mixtral-8x7b-32768",
                response_format={"type": "json_object"}
            )

            return json.loads(completion.choices[0].message.content)
        except Exception as e:
            print(f"Error: {str(e)}")
            return None

Step 4: Creating the Scheduler Service

Create scheduler_service.py:

class SchedulerService:
    def __init__(self):
        self.notion = NotionService()
        self.processor = TaskProcessor()

    def create_task_from_text(self, text: str):
        llm_result = self.processor.process_input(text)
        if not llm_result:
            return {"error": "Failed to process text"}

        if llm_result.get('needs_clarification'):
            return {
                "needs_clarification": True,
                "question": llm_result['clarification_question']
            }

        try:
            notion_task = self.notion.create_task(
                title=llm_result['title'],
                due_date=llm_result['due_date'],
                priority=llm_result['priority']
            )
            return {
                "success": True,
                "task": notion_task,
                "message": "Task created successfully"
            }
        except Exception as e:
            return {"error": f"Failed to create task in Notion: {str(e)}"}

Step 5: Setting Up FastAPI

Create main.py:

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI()
scheduler = SchedulerService()

class TextInput(BaseModel):
    text: str

@app.post("/process-text")
def process_text_to_task(text_input: TextInput):
    try:
        result = scheduler.create_task_from_text(text_input.text)
        return result
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

Running the Project

Start the server:

uvicorn app.main:app --reload

Test creating a task:

curl -X POST "http://localhost:8000/process-text" \
-H "Content-Type: application/json" \
-d '{"text": "Schedule team meeting tomorrow at 3pm"}'

Key Features and How They Work

Natural Language Processing

The LLM (Groq) processes natural language input
Extracts key information like dates, times, and priorities
Handles various date formats ("tomorrow", "next Monday", etc.)

Notion Integration

Creates tasks in your Notion database
Manages task status and priority
Handles date and time conversions

Error Handling

Validates input
Handles API errors gracefully
Asks for clarification when needed

Advanced Features

Task Deletion

@app.post("/process-deletion")
def process_deletion(text_input: TextInput):
    try:
        result = scheduler.task_deletion(text_input.text)
        return result
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

Task Rescheduling

@app.post("/reschedule-task")
def reschedule_task(text_input: TextInput):
    try:
        result = scheduler.reschedule_task(text_input.text)
        return result
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

Future Improvements

You could extend this project by:

Adding a frontend interface
Implementing user authentication
Adding support for recurring tasks
Creating task templates
Adding email notifications

Conclusion

We've built a powerful task management system that combines the simplicity of natural language with the organization of Notion. This project demonstrates how to:

Integrate multiple APIs
Process natural language
Handle complex date/time operations
Build a practical tool for daily use

The complete code is available on https://github.com/chetansingh-2/notion-ai-project.

Resources

Happy coding! 🚀

How to make HTML content editable?

Chetan Singh — Mon, 17 Jan 2022 16:48:48 +0000

Ever wondered? Can we make HTML content editable when it runs on a browser? The answer is YES.
By using contenteditable attribute we can specify whether the content of an element will be editable or not.

Syntax:
<element contenteditable="true or false">
We can give any of two values 'true' or 'false'.
'True' tells that element is editable while 'false' specifies that element is not editable.

Let's see:

This is my first post. I would document my learning journey on this platform and I hope it will prove helpful for beginners as I, too, am a beginner.

HEART IT