DEV Community: Deeshath

Stop Chatting With Claude. Start Orchestrating It.

Deeshath — Mon, 06 Apr 2026 09:47:15 +0000

Most developers are using Claude Code like an expensive autocomplete. I run 3 features in parallel with it. Here's the exact setup.

When I first started using Claude Code, I was doing the obvious thing:

Ask → Wait → Review → Repeat

One task. One agent. One thread. Me, waiting.

That's basically hiring a genius and making them work one ticket at a time while you watch.

Here's what I actually do now — and why it's a completely different category of tool.

                YOU (Lead / Orchestrator)
                         │
        ┌────────────────┼────────────────┐
        │                │                │
   project-1/       project-2/       project-3/
   (Agent A)        (Agent B)        (Agent C)
   Feature A        Feature B        Feature C

Each:
- Own repo copy/ worktrees
- Own Claude session
- Own branch + state

The Mental Model That Changed Everything

Claude Code isn't a chat assistant. It has full access to my repo. It reads files, runs commands, writes code, runs tests.

Most people use it like a smarter Stack Overflow. The actual unlock is this:

I'm not writing code anymore. I'm an engineering lead coordinating a team of AI developers.

You stop being the person at the keyboard and start being the person deciding what gets built and whether it's right. The keyboard work? Delegated.

That sounds abstract. Let me make it concrete by walking through my actual workflow.

Step 1: Be Clear on What and Why Before Anything Else

Before I touch anything, I get my intent straight.

I used to jump straight to implementation, and Claude would confidently build the wrong thing. Fast, clean, wrong. Like a contractor who builds the exact wall you described — in the wrong room.

Now: I tell Claude exactly what I want to build and why — the business reason, the constraint, the goal. Not a formal spec doc, not a design review process. Just clarity. What problem are we solving, what does done look like, what are the constraints.

Then I go straight to planning.

What I learned the hard way: "Simple" tasks are exactly where unexamined assumptions cause the most wasted work. Two sentences of context upfront saves an hour of fixing the wrong implementation.

Step 2: Write a Plan Before Writing Code

The plan says how — exact file paths, exact code changes, nothing left to interpretation.

A good plan looks like this:

Task 1: Add user notification feature

Files:
- Modify: src/services/notification_service
- Modify: src/models/user

Changes:
- notification_service: add send_notification method
  [exact diff of what gets added]
- user model: add notification_preferences field
  [exact diff of what changes]

Verify: run tests, confirm passing
Commit: "feat: add user notification"

Every file path is exact. Every change is shown as a diff — not described, shown. No ambiguity about what the agent is supposed to do.

Think of it as writing sheet music, not just saying "play something jazzy." The agent follows the score.

The plan also gets saved to .claude/plans/<feature-name>.md — so if a session gets interrupted (laptop dies, connection drops, you accidentally close the tab like I definitely haven't done), the next session picks up exactly where things left off.

A reviewer subagent reads the plan before execution starts. Its only job: find gaps, missing steps, anything that would cause an agent to get stuck. Fix, re-review, proceed.

Step 3: Isolate — And Know Which Isolation Method to Use

Here's where most people hit a wall they don't understand. Turns out "just run it in parallel" is not a plan.

Git Worktrees — for a single isolated feature

When I'm working on one feature that needs its own branch and clean state:

git worktree add .worktrees/feature-auth -b feature/auth-refactor
cd .worktrees/feature-auth
# install deps, run tests to verify clean baseline

The worktree gets its own directory and branch. Claude can commit, test, and experiment without touching my main working state. Lightweight, clean, no drama.

Separate repo copies — for working on multiple features in parallel

Git worktrees have a hard limitation people don't talk about: you can't run two worktrees in parallel without them stepping on each other.

Worktrees share the underlying git objects. Depending on your stack, this causes conflicts — shared process state, directory locks, test runs fighting over the same resources. Two agents, one worktree setup, instant chaos.

I tried. It was not a fun afternoon.

The solution: full repo copies.

cp -r ~/project ~/project-1
cp -r ~/project ~/project-2
cp -r ~/project ~/project-3

Each copy is a completely independent environment. One agent building feature A, another building feature B, a third on feature C — all simultaneously, each with their own branch, their own state, their own everything.

project/   → my main working copy
project-1/ → Agent working on feature A
project-2/ → Agent working on feature B
project-3/ → Agent working on feature C

While one agent is deep in feature A, I'm reviewing feature B's output. Feature C is running in the background. I'm never waiting on a single thread — and neither is any agent.

To kick one off — open Claude Code in that copy's directory, point it to the plan file, and it runs from there. Each copy is a fully independent Claude Code session.

When an agent finishes, I pull the diff out and raise a PR from that copy.

Why not just worktrees for this too?

Worktrees isolate at the branch level. Copies isolate at the process level. For parallel agents actively building and running tests, you need process-level isolation. Worktrees can't give you that — copies can.

Rule of thumb:

One feature, one agent, sequential → git worktree
Multiple features, multiple agents, parallel → separate repo copies

Step 4: How Each Agent Actually Works Inside a Copy

Each repo copy has one Claude agent. That agent owns its feature end to end. It's not a junior dev you have to babysit — it's a senior dev who happens to need very explicit instructions and will do exactly what you say, for better or worse.

Inside each copy, the agent follows the plan task by task. For each task, I dispatch a fresh subagent with exactly what it needs: the task, the relevant context, the constraints, what to return. Nothing else.

Fresh subagent per task = no context bleed. The agent on task 3 doesn't carry confusion from task 1. Every task starts sharp.

After each task, two review gates before moving on:

Gate 1 — Requirement compliance

Does the code match what was asked? Nothing missing, nothing extra?

If the agent added something clever that wasn't asked for — out. If it skipped something required — back in. "Close enough" fails this gate every time.

Gate 2 — Code quality

Is the implementation actually well-built?

Gate 2 only runs after Gate 1 passes. No point stress-testing the quality of something that built the wrong thing.

Both gates are fresh subagents. Issues found → implementer fixes → reviewer re-checks. Loop until both approve, then next task. No shortcuts.

Meanwhile, project-2 is running the exact same flow for a different feature. And project-3. All simultaneously. All operating from the same coding standards — because each copy carries the full Claude config.

This is the actual parallel. Not just running tests faster. Entire feature tracks shipping at the same time.

The Thing That Makes All of This Work: CLAUDE.md + Skills

Every agent I dispatch — every subagent, every reviewer, every parallel agent in every repo copy — reads my CLAUDE.md before touching anything.

It's the employee handbook for my AI team. Except unlike most employee handbooks, people actually follow this one.

Mine covers:

Architecture conventions (where things live, how they're structured)
What's forbidden (patterns I've banned and exactly why)
Testing strategy (what to test, what not to mock, how tests are structured)
Key commands (how to run tests, lint, build)

Since each repo copy is a full copy of the repo, it carries the full Claude config — CLAUDE.md and all the domain skills configured for that codebase. Skills are reusable instruction sets that auto-trigger based on what you're working on — think of them as specialised playbooks Claude loads when it detects you're touching a specific part of the system.

Every agent in every copy automatically gets the same standards, the same workflows, the same constraints. No setup per copy. No drift between agents.

This is what keeps multiple agents consistent without me managing each one individually. They're not just isolated from each other — they're all working from the same rulebook.

The rules I include aren't just what — they explain why. If Claude understands the reasoning, it makes better calls in gray areas instead of picking the most literal interpretation of your instructions and running with it off a cliff.

What I cut: anything where I'd answer "removing this won't cause Claude to make mistakes." Every unnecessary line dilutes the signal.

What I Still Review Myself

Claude Code is fast. It's not a mind reader, and it's not infallible.

Things I always personally review, regardless of what review gates say:

Authorization logic — never skip, never assume it's right
Anything touching shared state across users or accounts
Core business logic with real edge cases

Two hard rules I enforce on every agent, no exceptions:

No auto-commit. Agents stage changes, present the diff, and wait for my explicit sign-off before anything touches git. I've seen agents commit mid-task with half-finished work. Hard no.

Cleanup before done. Any code added during a session that got replaced or made redundant during iteration gets deleted before the task closes. Agents leave behind ghost methods from earlier attempts like forgotten tabs in a browser. The rule: review everything added in the session, delete anything no longer used.

AI-generated code can be confidently wrong. "Approved" from a reviewer subagent doesn't replace my eyes on critical paths.

What You Actually Walk Away With

Apply these today — regardless of where you are with Claude Code

Start with intent, not implementation. Two sentences of what + why before any prompt saves you an hour of wrong output.
CLAUDE.md is your team's shared brain. Set it up once. Every agent reads it. The more precise it is, the less you babysit.
No auto-commit. No exceptions. Agents stage, you approve, then it goes to git. You're the last line.
Worktrees for one feature, repo copies for parallel ones. Don't fight the tool. Each has a job — and conflating them is a bad afternoon.

Once you're running parallel agents

Plan with exact diffs, not descriptions. Vague plans produce vague code. The agent follows the score — write the score.
Fresh subagent per task. Context bleed is real. Every task starts clean, focused, without baggage from the one before it.
Two review gates, always in order. Right requirements first, then quality. Reviewing quality on the wrong feature is just polishing a mistake.

What Actually Changed

I used to be the bottleneck. I'd write code, review it, test it, fix it — all sequential, all me. One thread. Very human. Very slow.

Now I design, delegate, and validate. I'm often waiting on myself to review output, not waiting on Claude to finish. That's a good problem to have.

The actual unlock isn't "Claude writes faster code."

It's: I stopped being the execution bottleneck and became the decision-maker.

The engineers are faster. The leverage is real. The judgment still has to come from you — and honestly, that's the interesting part of the job anyway.

All patterns in this post come from my actual Claude Code setup — workflows and conventions I run daily on a production codebase.

What's your current Claude Code setup? Still single-threaded or already running parallel? Drop it in the comments — curious what workflows people have settled on.

Database Sharding: Simplifying Data Scalability

Deeshath — Sat, 09 Nov 2024 18:30:37 +0000

Managing data at scale can be a bit like managing a massive library. Imagine your library started with a handful of books, and over time, it grew—slowly at first, then faster and faster until you found yourself lost in the aisles, wondering where to find just one book. Adding more shelves (or servers) helps, but eventually, even more shelves won’t solve the problem. What you really need is a way to organize the books into manageable sections.

That’s essentially what database sharding does. When your database grows too large for a single server to handle, sharding divides the data into smaller, more manageable shards, and these shards are distributed across multiple servers. This makes your system more efficient, faster, and scalable—like organizing the library by genre, so you can find the book you need without wandering for hours.

In this guide, we’ll explore the ins and outs of sharding, its different types, and how you can use it to build a more scalable system. Along the way, we’ll keep things simple, and hopefully make a somewhat dry subject a bit more fun to digest.

What is Database Sharding?

Let’s say you’re the head librarian in a library (your database), and everything was smooth sailing when you only had a few thousand books. But as the collection grew, you noticed things starting to slow down—finding the right book became a challenge. Just like the library’s shelves, a single server can only hold so much data before things grind to a halt.

To solve this, you break the collection up into smaller sections—shards—so that each section (or shard) is easier to manage. Instead of searching through the entire library, you only need to search through the relevant section.

This is exactly what happens with database sharding. It’s the process of dividing a large database into smaller, more manageable pieces (shards), which are then distributed across multiple servers. So, instead of one giant server struggling to handle everything, each shard lives on its own server. The system knows exactly where to look when you make a request for data, just like how a well-organized library catalog can point you to the right section of the library.

Why Shard a Database?

Sharding is helpful when your database becomes too large for a single server to handle. Here are a few signs that it might be time to consider sharding:

1. Scalability

As your data grows, a single server can’t handle all the traffic. Sharding allows you to scale your database horizontally, by distributing the data across multiple servers. It’s like expanding your library by adding multiple branches, each with its own set of books.

2. Performance

With sharding, queries can be processed in parallel. Since the data is distributed across servers, your queries are more efficient, and the load is better balanced. This makes retrieving data faster—like sending multiple librarians to search for a book in different sections, instead of just one librarian running back and forth.

3. Availability

If one shard goes down, the other shards are still functional. It’s like your library remaining open even if one section is temporarily closed for renovation. You don’t lose access to the entire collection when one server fails.

Types of Database Sharding

When it comes to sharding, there are several methods you can use, depending on your data structure and access patterns. Let’s break down the four main types of sharding with examples and a bit of humor to keep you from falling asleep.

1. Horizontal Sharding (Row-Based Sharding)

In horizontal sharding, data is split by rows. Each shard holds a subset of the rows, typically based on a range of data, like user IDs or geographic regions. This works well when your data is uniform and can be evenly distributed across servers.

Example:

Imagine you’re running a huge e-commerce platform with millions of users. You might split the user base like this:

Shard 1: Users with IDs 1–100,000
Shard 2: Users with IDs 100,001–200,000
Shard 3: Users with IDs 200,001–300,000

So, when a request comes in for user data with ID 150,000, the system knows exactly which shard to query. It's like organizing your library by genre, so if you’re looking for sci-fi books, you know exactly which section to head to.

When to Use Horizontal Sharding:

Growing datasets: When your data grows beyond what a single server can handle.
Applications with uniform data: For instance, social media apps or blogs where each user's data is similar.
High-traffic systems: E-commerce sites or news websites that handle tons of requests.

Real-World Example:

A social media platform might use horizontal sharding to manage user profiles. Each shard could hold a different range of user IDs, so as the platform grows, it can scale efficiently.

2. Vertical Sharding (Column-Based Sharding)

In vertical sharding, data is split by columns instead of rows. Each shard holds a subset of the columns, often based on how the data is accessed or updated. This approach is useful when certain parts of your data are more frequently accessed than others.

Example:

Imagine an online banking system with user accounts and transactions. You might organize the data into different shards:

Shard 1: Stores user details (name, address, email)
Shard 2: Stores account balances and transaction history
Shard 3: Stores login activity (last login time, IP address, session data)

With this setup, frequently accessed data, like user information, is separated from less frequently accessed data, like login history, making everything faster and more efficient.

When to Use Vertical Sharding:

Different access patterns: Some columns are used more often than others.
Mixed data types: For example, you might store user info separately from transaction info.
Frequent updates: If some columns change more often than others, you might want to isolate them for performance.

Real-World Example:

An online retailer might separate product data (names, prices) from customer data (addresses, orders) to make product searches faster while keeping customer information updates isolated.

3. Directory-Based Sharding

With directory-based sharding, a central directory keeps track of where each piece of data is stored. Instead of using ranges or hash functions, the directory maps data to a specific shard, offering more flexibility.

Example:

Imagine you have a customer database. Instead of splitting data by user ID or region, you use a directory that maps each customer to a specific shard:

Customer 12345 → Shard 1
Customer 67890 → Shard 2
Customer 11223 → Shard 3

The directory works like a library catalog, helping you track down exactly where each book (or piece of data) is stored. So when you need something, you don’t have to guess—you go straight to the right section.

When to Use Directory-Based Sharding:

Custom distribution: When you need fine-grained control over data placement.
Dynamic rebalancing: When your data changes and you need to move it around.
Complex access patterns: When simple range-based or hash-based methods won’t work.

Real-World Example:

A streaming platform could use directory-based sharding to store content by region, knowing that North American content goes to one shard, and European content to another.

4. Hash-Based Sharding

In hash-based sharding, a hash function is applied to a key (like a user ID or product ID), and the resulting value determines where the data is stored. This ensures that the data is evenly distributed across all shards.

Example:

Let’s say you have a large number of users uploading photos. You could hash each user’s ID and distribute their photos across shards:

Hash(User ID 12345) → Shard 1
Hash(User ID 67890) → Shard 2
Hash(User ID 13579) → Shard 3

This method ensures that no shard gets overloaded with data, keeping things balanced. It’s like randomly assigning genres to different sections of the library, so one section doesn’t get all the heavy traffic.

When to Use Hash-Based Sharding:

Even data distribution: When you want to ensure an even spread of data across shards.
Unpredictable access patterns: If you can’t predict which data will be queried more frequently.
Avoiding hot spots: Hashing helps prevent some shards from becoming more heavily loaded than others.

Real-World Example:

A photo-sharing platform might use hash-based sharding to ensure that user uploads are spread evenly across servers, preventing one shard from becoming overwhelmed by popular images.

Pros and Cons of Sharding

Pros:

Improved performance: Queries are processed faster because data is distributed across servers.
Scalability: You can scale your database horizontally by adding more servers as your data grows.
High availability: If one shard goes down, the others keep functioning, ensuring your system stays operational.

Cons:

Increased complexity: Managing multiple shards can be tricky, especially around data consistency and backups.
Cross-shard queries: Queries that need data from multiple shards can be slower and more complex.
Data imbalance: If your data isn’t evenly distributed, some shards might get overloaded while others are underutilized.

When Should You Use Sharding?

Sharding is great when:

You need to handle a large dataset that can’t fit on a single server.
Your system needs to scale horizontally as your data grows.
You want to distribute traffic more efficiently across multiple servers.
You require high availability, so the system stays up even if part of it goes down.

Conclusion

Think of database sharding as your librarian’s solution to a growing library: splitting the books (data) into smaller, more organized sections (shards) so that you can find what you need quickly, even when the library (database) becomes massive. By choosing the right sharding strategy, you’ll ensure that your application scales smoothly and performs efficiently, no matter how much data you’re dealing with.

So go ahead and start organizing your data—your future self will thank you for it! And remember, the only thing worse than a slow database is a slow librarian. 😉

Happy sharding!

Apache Kafka - KRaft Mode: Setup

Deeshath — Mon, 16 Sep 2024 10:17:02 +0000

A Simple Guide to Apache Kafka: Setting It Up and Testing in KRaft Mode

What is Apache Kafka?

Let’s start simple. Imagine Kafka as a mailing system for your data. Different parts of your system, or even different systems, can communicate with each other by sending and receiving data messages. If you have tons of data being created all the time, like user actions on a website, machine logs, or even weather sensor data, Kafka ensures that this data flows smoothly and quickly to wherever it needs to go.

In essence, Kafka makes sure data gets from one place to another in a way that’s fast, reliable, and scalable. So, whether you’re running a small application or a massive enterprise, Kafka’s got you covered.

Why Should You Use Kafka?

Here are a few good reasons why people use Kafka:

Real-Time Data Processing: Data flows through Kafka almost instantly, making it perfect for live data streams.
Reliable and Scalable: Kafka can handle millions of data messages every second, which makes it great for both small and large systems.
Fault-Tolerant: If something goes wrong, Kafka won’t lose your data – it’s designed to recover and keep things moving.
Decouples Systems: Kafka allows systems to send and receive data without needing to know too much about each other, which keeps everything flexible.

Key Kafka Concepts

Before we jump in, let’s break down some of the basic terms:

Producer: This is the system or service that sends data to Kafka. Think of it like the sender in the mailing system.
Consumer: This is the system that reads or takes the data from Kafka. It’s the recipient.
Topic: A channel or folder where Kafka stores messages. Producers send data to topics, and consumers read data from topics.
Broker: A Kafka server that holds and manages the messages within topics.
Partition: Topics can be split into smaller pieces called partitions, which helps Kafka handle a large amount of data more efficiently.

Kafka in Zookeeper Mode vs. KRaft Mode

Originally, Kafka used Zookeeper to manage its data and operations. Zookeeper was like the manager that made sure everything stayed organized. However, managing Zookeeper and Kafka together could be a bit complex, especially as your system grows.

That’s where KRaft Mode comes in. KRaft Mode lets Kafka manage its own data without needing Zookeeper. This makes the entire setup simpler and more efficient because you’re only managing Kafka now, not both Kafka and Zookeeper.

How Do They Compare?

Feature	Zookeeper Mode	KRaft Mode
Metadata Management	Managed by Zookeeper	Managed directly by Kafka
Consensus Protocol	Uses Zookeeper's method	Kafka uses its own Raft protocol
Architecture	Needs a separate Zookeeper cluster	No Zookeeper needed
Complexity	More components to manage	Simpler, one less system to worry about

Setting Up Kafka in KRaft Mode Using Docker

Now that we have a basic understanding, let’s set up Kafka in KRaft mode using Docker. If you’re not familiar with Docker, don’t worry! It’s a tool that lets us run programs in containers, which are like isolated environments, without needing to install everything directly on our computer.

Prerequisites:

Docker and Docker Compose installed on your machine.

Step 1: Create the Docker Compose File

Create a file called docker-compose.yml in your project directory.
Copy and paste the following into that file:

name: 'stream'
version: '3.8'
services:
  kafka:
    image: confluentinc/cp-kafka:latest
    hostname: kafka
    container_name: kafka
    ports:
      - "9092:9092"
      - "9093:9093"
    environment:
      KAFKA_KRAFT_MODE: "true"  # This enables KRaft mode in Kafka.
      KAFKA_PROCESS_ROLES: controller,broker  # Kafka acts as both broker and controller.
      KAFKA_NODE_ID: 1  # A unique ID for this Kafka instance.
      KAFKA_CONTROLLER_QUORUM_VOTERS: "1@localhost:9093"  # Defines the controller voters.
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_LOG_DIRS: /var/lib/kafka/data  # Where Kafka stores its logs.
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"  # Kafka will automatically create topics if needed.
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1  # Since we’re running one broker, one replica is enough.
      KAFKA_LOG_RETENTION_HOURS: 168  # Keep logs for 7 days.
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0  # No delay for consumer rebalancing.
      CLUSTER_ID: "Mk3OEYBSD34fcwNTJENDM2Qk"  # A unique ID for the Kafka cluster.
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./data:/var/lib/kafka/data  # Store Kafka logs on your local machine.

Step 2: Start Kafka

Once the file is ready, open your terminal (or command line), go to the directory where you saved the docker-compose.yml file, and run this command:

docker-compose up

This command will pull the Kafka image, set everything up, and run Kafka in KRaft mode. Now you have Kafka running locally!

Testing Kafka: Let’s See If It Works!

Great! Now let’s test our Kafka setup to make sure everything is working as expected. We’ll create a topic, send some messages, and then read them back.

Step 1: Access the Kafka Container

To interact with Kafka, we need to get inside the Docker container where Kafka is running. Use this command to access the Kafka container:

docker exec -it kafka bash

Step 2: Create a Topic

Now, let’s create a topic in Kafka. A topic is where messages are stored. We’ll create one called test-topic:

/usr/bin/kafka-topics --create --topic test-topic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

Step 3: Send Messages to Kafka (Producing)

Next, we’ll send some messages to the test-topic. We do this by starting a producer that sends messages to Kafka:

/usr/bin/kafka-console-producer --broker-list localhost:9092 --topic test-topic

Type a few messages and press Enter after each one:

> Hello Kafka!
> This is a test message.

Step 4: Read Messages from Kafka (Consuming)

Now, let’s see if we can read the messages we just sent. To do this, we’ll start a consumer that reads messages from test-topic:

/usr/bin/kafka-console-consumer --bootstrap-server localhost:9092 --topic test-topic --from-beginning

You should see the messages you sent earlier:

Hello Kafka!
This is a test message.

Step 5: Check All Topics

If you want to see all the topics that exist in your Kafka setup, you can list them with this command:

/usr/bin/kafka-topics --list --bootstrap-server localhost:9092

Step 6: Check Logs (Optional)

If you want to check what’s happening behind the scenes, you can always check the Kafka logs:

docker logs kafka

Wrapping It Up

Well done! 🎉 You’ve just set up Kafka in KRaft mode, created a topic, sent messages, and consumed those messages. With KRaft mode, Kafka becomes simpler to manage, since you no longer need Zookeeper.

Kafka is an amazing tool for handling real-time data, and now that you’ve got it running, you can start exploring its potential. Try experimenting with different topics, producing more data, or even connecting Kafka to your own applications to see how it handles real-world use cases.

Good luck, and happy streaming!

REST API using GO(gin) & Mongo

Deeshath — Tue, 27 Jun 2023 04:31:36 +0000

In this blog, we will be building a RESTful API server with two endpoints. The example project will be a repository of data about books in a library.

This includes the following sections:

Design API endpoints.
Create a folder for code.
Connect to MongoDB using qmgo
Write handlers to
- add a new book
- get all books
- get a specific book
- delete a specific book

Prerequisites:

An installation of Go 1.20 or later. For installation instructions, visit.
An installation of MongoDB 6.0 or later. For installation instructions, visit
Any text editor or IDE we have will work fine.

Design API endpoints:

We’ll build an API that provides access to the books in a library. So we’ll need to provide endpoints through which a client can get, add or delete books in a library.

Here are the endpoints we’ll create in this tutorial.

/books

GET – Get a list of all books, returned as JSON.
POST – Add a new book from the request data sent as JSON.

/books/:id

GET – Get a book by its ID, returning the book data as JSON.
PATCH – Update a book by its ID, returning the book data as JSON.
DELETE – Delete a book by its ID, returning the deletion status as JSON.

Create a folder for our code:

To begin, we will create a project for the code we are about to write.

Using the command prompt, create a folder for our code called books.

$ mkdir books
$ cd books

Create a module in which we can manage dependencies.

Run the go mod init command, giving it the path of the module our code will be in.

$ go mod init books
go: creating new go.mod: module books

This command creates a go.mod file in which dependencies we add will be listed for tracking.

Code

Using IDE/text editor, create a file called main.go in the books directory. We’ll write our Go code in this file.

In main.go, at the top of the file, add the following package declaration and main function that will be called when we run the code since a standalone program (as opposed to a library) is always in package main.

package main

func main() {

}

To run the program use go run .

Beneath the package declaration, let's start writing the code for Connecting to MongoDB.

Install the qmgo using the below cmd

go get github.com/gin-gonic/gin

import the "github.com/qiniu/qmgo" package and declare the variable database and collection and initialize them in the main function so that it could be used later on to perform CRUD operations on Data in the Database.

var database *qmgo.Database
var collection *qmgo.Collection

func main() {

    // create new Client
    const databaseURI = "mongodb://localhost:27017"
    fmt.Println("Connecting to database", databaseURI)
    ctx := context.Background()
    connection, err := qmgo.NewClient(ctx, &qmgo.Config{Uri: databaseURI})

    database = connection.Database("test")    // creating Database connection
    collection = database.Collection("books") // get the collection
    defer func() {
        if err = connection.Close(ctx); err != nil {
            fmt.Println("Closing Connection to database", databaseURI)
            panic(err)
        }
    }()

}

Now we proceed on to write the handlers and configuring the app to listen to a http port (in our case 8000) in the main function.

install the gin using the below cmd

go get github.com/gin-gonic/gin
and import it in the main file

    router := gin.Default() // create router using gin

    // register routes
    router.POST("/books", CreateBook)

    fmt.Println("Service is up & running at localhost:8000")
    router.Run(":8000") // register router to port 8000

Here we have registered a POST router let's go ahead and create the CreateBook function to handle the create request

Create a new file books.go in which we will be writing the code for handling the requests

Create the request and response structure for the books:

// form:"title" to map the JSON field name to the struct
// binding:"required" to enforce the value is required
type BookCreateUpdateRequest struct {
    Title  string `form:"title" binding:"required"`
    Author string `form:"author"`
}

// json:"id" to map the struct Name to its Json field name
type BookResponse struct {
    Id        primitive.ObjectID `json:"id"`
    Title     string             `json:"title"`
    Author    string             `json:"author"`
    CreatedAt time.Time          `json:"createdAt"`
    UpdatedAt time.Time          `json:"updatedAt"`
}

Create the Database model structure:

type Book struct {
    field.DefaultField `bson:"inline"`
    Title              string `bson:"title" validate:"required"`
    Author             string `bson:"author"`
}

Let's create the CreateBookmethod to handle book creation requests.

func CreateBook(ctx *gin.Context) {
    var newBook BookCreateUpdateRequest

    // to bind the received JSON to BookRequest to strip the unnecessary fields.
    if err := ctx.ShouldBind(&newBook); err != nil {
        ctx.JSON(http.StatusBadRequest, "Invalid Request")
        return
    }

    // setting data to book model struct
    book := Book{
        Title:  newBook.Title,
        Author: newBook.Author,
    }
    _, err := collection.InsertOne(ctx, &book) //Inserting the Book Data to database

    // to send error response if any error occurs
    if err != nil {
        ctx.JSON(http.StatusInternalServerError, "Something went wrong, Try again after sometime")
        return
    }

    // to send success response on completion
    ctx.JSON(http.StatusCreated, GetBooksResponse(book))
}

func GetBooksResponse(book Book) (bookResponse BookResponse) {
    // setting response for book
    bookResponse = BookResponse{
        Id:        book.DefaultField.Id,
        Title:     book.Title,
        Author:    book.Author,
        CreatedAt: book.CreateAt,
        UpdatedAt: book.UpdateAt,
    }
    return
}

In the above code, we use GetBooksResponse method to set the response that we are to send (we will be using this for other API responses too).

Below are the Handlers for Get, List, Update and Delete.

Get a book detail API:

func GetBook(ctx *gin.Context) {

    // to get and convert the received path variable to  desired type
    bookId, err := primitive.ObjectIDFromHex(ctx.Param("bookId"))
    if err != nil {
        ctx.JSON(http.StatusBadRequest, "Invalid Request")
        return
    }

    //Getting the Book Data from database
    var book Book
    err = collection.Find(ctx, bson.M{"_id": bookId}).One(&book)

    // to send error response if any error occurs
    if err != nil {
        ctx.JSON(http.StatusNotFound, "Book Not Found")
        return
    }

    // to send success response on completion
    ctx.JSON(http.StatusOK, GetBooksResponse(book))
}

In the above code the bson.M{"_id": bookId} is used to find the book by Id and One is used to bind the mongo Data to the book variable

Update a book API:

func UpdateBook(ctx *gin.Context) {

    // to get and convert the received path variable to  desired type
    bookId, err := primitive.ObjectIDFromHex(ctx.Param("bookId"))
    if err != nil {
        ctx.JSON(http.StatusBadRequest, "Invalid Book ID")
        return
    }

    var newBook BookCreateUpdateRequest

    // to bind the received JSON to BookRequest to strip the unnecessary fields.
    if err := ctx.ShouldBind(&newBook); err != nil {
        ctx.JSON(http.StatusBadRequest, "Invalid Request")
        return
    }

    //Getting the Book Data from database
    var book Book
    err = collection.Find(ctx, bson.M{"_id": bookId}).One(&book)

    // to send error response if any error occurs
    if err != nil {
        ctx.JSON(http.StatusNotFound, "Book Not Found")
        return
    }

    // set the updated value in the book
    book.Author = newBook.Author
    book.Title = newBook.Title

    // update in database
    err = collection.ReplaceOne(ctx, bson.M{"_id": bookId}, &book)
    if err != nil {
        ctx.JSON(http.StatusInternalServerError, "Something went wrong, Try again after sometime")
        return
    }

    // to send success response on completion
    ctx.JSON(http.StatusOK, GetBooksResponse(book))
}

In the above code block, collection.ReplaceOne is used to replace the existing document based on the condition the ReplaceOne method also updates the default field UpdateAT in the database.

Delete Book Handler:

func DeleteBook(ctx *gin.Context) {

    // to get and convert the received path variable to  desired type
    bookId, err := primitive.ObjectIDFromHex(ctx.Param("bookId"))
    if err != nil {
        ctx.JSON(http.StatusBadRequest, "Invalid Request")
        return
    }

    //Getting the Book Data from database
    var book Book
    err = collection.Find(ctx, bson.M{"_id": bookId}).One(&book)

    // to send error response if any error occurs
    if err != nil {
        ctx.JSON(http.StatusNotFound, "Book Not Found")
        return
    }

    // Deleting the book
    err = collection.RemoveId(ctx, bookId)

    // to send error response if any error occurs
    if err != nil {
        ctx.JSON(http.StatusInternalServerError, "Something went wrong, Try again after sometime")
        return
    }

    // to send success response on completion
    ctx.JSON(http.StatusOK, true)
}

In the above code block, collection.RemoveId is used to remove the specific data based on the ID provided.

Books List Handler

func GetBooks(ctx *gin.Context) {

    //Getting the Book Data to database
    var books []BookListResponse
    err := collection.Find(ctx, bson.M{}).All(&books)

    // to send error response if any error occurs
    if err != nil {
        fmt.Println(err)
        ctx.JSON(http.StatusInternalServerError, "Something went wrong, Try again after sometime")
        return
    }

    // to send success response on completion
    ctx.JSON(http.StatusOK, books)
}

Here in the List handler, we have used BookListResponse which is used to limit the values read from the database since the id and the name of the book would suffice in the list API. Below is the BookListResponse type.

type BookListResponse struct {
    Id    primitive.ObjectID `json:"id" bson:"_id"` // bson to map mongo _id to id
    Title string             `json:"title"`
}

Here the bson:"_id" is used to map the mongo _id to the ID attribute in the response.

Now all the Handlers have been created let's register the Handlers in the router in main.go by adding the below code block after the router declaration.

    router.GET("/books", GetBooks)
    router.GET("/books/:bookId", GetBook)
    router.PATCH("/books/:bookId", UpdateBook)
    router.DELETE("/books/:bookId", DeleteBook)

Congratulations!!! the CRUD REST API for books in a library using Mongo and Go is successfully completed.

You will be able to find the source code in the Github repo:

Deeshath / books

Books REST API using go and mongo

Books

Books REST API using go and mongo

Pre requisites:

An installation of Go 1.20 or later. For installation instructions, visit.
An installation of MongoDb 6.0 or later. For installationinstructions, visit

To start thr REST API server use the cmd go run .

View on GitHub