DEV Community: Agrima Gupta

How Caching Saves Your Website From Thousands of Requests

Agrima Gupta — Tue, 06 Oct 2026 11:18:00 +0000

How Does Netflix Load So Fast? The Secret Is Caching.

Have you ever wondered why Netflix can serve millions of users without making its database work insanely hard for every single request?

Or why YouTube thumbnails appear almost instantly?

Or why a website you've already visited sometimes loads noticeably faster the second time?

One of the biggest reasons is caching.

You might have heard this word a hundred times, but the idea behind it is actually pretty simple:

If you've already done the expensive work once, why do it again?

Let's see how this works in real applications.

So, What Exactly Is Caching?

Imagine you go to a restaurant and order coffee.

The waiter doesn't go to the farm, collect coffee beans, roast them, grind them, and make the coffee from scratch every time you ask for another cup.

The restaurant already has the ingredients ready.

Caching follows a similar idea.

Instead of repeatedly fetching or calculating the same data, we temporarily store the result somewhere that can be accessed much faster.

Without caching, a request might look like this:

User
 ↓
Server
 ↓
Database
 ↓
Get Data
 ↓
Server
 ↓
User

With caching:

User
 ↓
Server
 ↓
Cache ⚡
 ↓
Data
 ↓
User

The database doesn't have to do the same work over and over again.

Let's Take a Simple Example

Imagine your website has an API:

GET /api/products

Every time someone visits your website, the backend might query the database:

SELECT * FROM products;

Now imagine 10,000 people visit your website.

You could end up with:

10,000 requests
      ↓
10,000 database queries

But what if the product list only changes occasionally?

There's no reason to ask the database the exact same question 10,000 times.

Instead, we can cache the result.

The first request goes to the database:

Request
   ↓
Backend
   ↓
Database
   ↓
Products
   ↓
Cache

The next requests can simply use the cached result:

Request
   ↓
Backend
   ↓
Cache ⚡
   ↓
Products

That's the basic idea behind caching.

Cache Hit vs Cache Miss

You'll often hear developers use the terms cache hit and cache miss.

A cache hit means the data you're looking for is already in the cache.

Request
   ↓
Cache
   ↓
Found ✅
   ↓
Return Data

A cache miss means the cache doesn't have the data.

Request
   ↓
Cache
   ↓
Not Found ❌
   ↓
Database
   ↓
Store Result in Cache
   ↓
Return Data

The goal is generally to have a high cache hit rate for data that makes sense to cache.

Your Browser Is Already Using Caching

Caching isn't something you only encounter in backend development.

Your browser does it all the time.

When you visit a website, your browser may store things like:

Images
CSS
JavaScript
Fonts
Other static resources

Suppose you download:

logo.png

The next time you visit the same website, your browser may already have that image.

Instead of downloading it again:

Server → Browser

the browser can use:

Browser Cache → logo.png

This is one of the reasons websites can feel faster when you revisit them.

Then There Are CDNs

Now imagine your main server is located in the US, but your users are all over the world.

A user in India requesting a large image might have to communicate with a server thousands of kilometres away.

That's where a CDN — Content Delivery Network — becomes useful.

A CDN stores copies of content across servers distributed around the world.

Without a CDN:

India
  ↓
US Server
  ↓
Response

With a CDN:

India
  ↓
Nearby CDN
  ↓
Response ⚡

CDNs are commonly used for images, videos, JavaScript, CSS, static pages, and sometimes API responses.

This is one of the techniques that allows large platforms to deliver content efficiently to users around the world.

What About Backend Caching?

Let's move one level deeper.

Suppose your backend has this endpoint:

GET /api/products

Without caching:

Request
  ↓
Backend
  ↓
Database
  ↓
Response

With caching:

Request
  ↓
Backend
  ↓
Cache
 ↙     ↘
Hit    Miss
 ↓       ↓
Data   Database
         ↓
       Cache
         ↓
      Response

This is where technologies like Redis are commonly used.

Redis is an in-memory data store, which makes it extremely useful for storing frequently accessed data.

For example, we might store:

Key:
products

Value:
[
  { id: 1, name: "Laptop" },
  { id: 2, name: "Mouse" }
]

Instead of hitting the database for every request, the backend can first check Redis.

A Simple Backend Example

Imagine a Python backend.

Without caching:

@app.get("/products")
def get_products():

    products = database.get_products()

    return products

Every request goes directly to the database.

With caching:

@app.get("/products")
def get_products():

    products = cache.get("products")

    if products:
        return products

    products = database.get_products()

    cache.set("products", products)

    return products

Now the logic is basically:

Request
   ↓
Check Cache
   ↓
Is data there?
  ↙       ↘
YES       NO
 ↓         ↓
Return   Database
           ↓
         Cache
           ↓
        Return

Simple idea, but extremely powerful when an application gets large.

But Here's the Problem With Caching

Imagine your website stores:

Product price = ₹999

in the cache.

Then the actual price changes:

₹999 → ₹899

But the cache still contains:

₹999

Now your users are seeing old information.

This is called stale data.

So caches usually have an expiration time, commonly called TTL — Time To Live.

For example:

Cache Data
    ↓
TTL = 60 seconds
    ↓
60 seconds pass
    ↓
Cache expires

The next request can fetch fresh data and store it again.

Cache Invalidation Is Where Things Get Interesting

There's a famous saying in software:

"There are only two hard things in Computer Science: cache invalidation and naming things."

And cache invalidation really can become complicated.

Suppose we have:

Database:
₹899

Cache:
₹999

When the product changes, we need to make sure the cache is updated or removed.

One approach is to delete the cached value:

cache.delete("product:123")

Then the next request fetches fresh data from the database.

Another approach is to update the cache whenever the database changes.

The right strategy depends on the application.

Why Not Cache Everything?

If caching makes things faster, why not cache the entire application?

Because caching comes with trade-offs.

Caches consume memory.

Cached data can become stale.

And now you have another system to manage.

Instead of:

Backend → Database

your architecture becomes:

Backend → Cache → Database

Now you have to think about what should be cached, how long it should live, and what should happen when the cache is unavailable.

Caching is powerful, but it isn't free.

What Makes Good Data for Caching?

A good candidate is usually data that is:

Frequently requested, expensive to calculate or retrieve, and doesn't change constantly.

For example:

Product catalog
Popular posts
API responses
Configuration
Frequently accessed database queries

Data that changes constantly or must always be completely real-time may be less suitable.

The important question isn't:

"Can I cache this?"

It's:

"Does caching this actually improve my application?"

How Much Faster Can Caching Be?

Let's use a simplified example.

Imagine:

Database query = 200ms
Cache lookup   = 5ms

If 1,000 requests all hit the database, that's a lot of unnecessary work.

But if most of those requests can be served from the cache, the database can focus on requests that actually need it.

The exact numbers will vary depending on your architecture, network, database, query, and cache setup.

But the fundamental idea remains:

Avoid expensive work when you already have the answer.

Caching Exists at Multiple Levels

One interesting thing about caching is that there isn't just one cache.

A modern application might look something like:

              USER
                ↓
         Browser Cache
                ↓
               CDN
                ↓
        Application Cache
                ↓
             Redis
                ↓
            Database
                ↓
         Database Cache

Different layers solve different problems.

And together, they can dramatically reduce the amount of work your backend and database need to perform.

The Mental Model I Use for Caching

Whenever you hear the word cache, think about one simple question:

"Do I already have this?"

If yes:

Return it ⚡

If no:

Fetch / Calculate
       ↓
Store
       ↓
Return

That's caching at its core.

Everything else — Redis, CDNs, TTLs, cache invalidation, cache hit rates — is built around this basic idea.

Final Takeaway

Caching isn't some complicated trick that only companies like Netflix or YouTube use.

It's a simple idea that becomes incredibly powerful at scale:

Don't do expensive work repeatedly when you can reuse the result.

Your browser caches images.

CDNs cache content.

Backends cache API responses.

Redis stores frequently accessed data.

Databases have their own caching mechanisms.

Together, these layers help modern applications serve huge amounts of traffic without forcing the database to handle every single request.

So the next time Netflix loads your homepage almost instantly, remember:

It probably isn't asking the database to figure everything out from scratch.

Somewhere along the way, someone already did the work — and the result is waiting in a cache. ⚡

How API Calls Actually Work: From Button Click to Server Response

Agrima Gupta — Mon, 05 Oct 2026 16:12:45 +0000

If you've worked with React, Next.js, or any frontend framework, you've probably written something like:

const response = await fetch("/api/users");
const data = await response.json();

Looks simple, right?

But I used to wonder:

What actually happens after I call fetch()?

Where does the request go?

How does the backend know what I want?

Where does the database fit in?

And how does the response come back to my UI?

Let's break down the entire journey of one API call.

The Big Picture

Imagine we have a button:

<button onclick="getUsers()">
  Get Users
</button>

When the user clicks it, the flow looks roughly like this:

Button Click
     ↓
Frontend JavaScript
     ↓
HTTP Request
     ↓
Backend API
     ↓
Database
     ↓
Backend Response
     ↓
Frontend
     ↓
UI Update

That's it.

Now let's see what's actually happening at each step.

1. The Button Click

Let's say we want to fetch a list of users.

async function getUsers() {

    const response = await fetch(
        "http://localhost:8000/api/users"
    );

    const data = await response.json();

    console.log(data);
}

The important line is:

fetch("http://localhost:8000/api/users")

We're basically telling the browser:

"Hey, send a request to this API and give me whatever comes back."

2. The Browser Sends an HTTP Request

Behind that simple fetch() call, the browser creates an HTTP request.

Something like:

GET /api/users

GET is the HTTP method.

Some common methods are:

GET     → Get data
POST    → Create data
PUT     → Replace data
PATCH   → Update data
DELETE  → Delete data

So:

GET /api/users

basically means:

"Give me the users."

3. The Backend Receives It

Now imagine our backend is built using FastAPI.

from fastapi import FastAPI

app = FastAPI()

@app.get("/api/users")
def get_users():

    return [
        {"id": 1, "name": "Agrima"},
        {"id": 2, "name": "Rahul"}
    ]

This line:

@app.get("/api/users")

is basically telling the backend:

"Whenever someone sends a GET request to /api/users, run this function."

So our request:

GET /api/users

finds this function:

get_users()

4. What About the Database?

In a real application, we're obviously not going to hardcode users.

The backend would probably query a database.

For example:

SELECT * FROM users;

The database might return:

1 | Agrima
2 | Rahul
3 | Priya

The backend then converts this into a response, usually JSON:

[
  {
    "id": 1,
    "name": "Agrima"
  },
  {
    "id": 2,
    "name": "Rahul"
  }
]

So now our flow becomes:

Frontend
   ↓
API Request
   ↓
Backend
   ↓
Database
   ↓
Backend

5. The Response Comes Back

The backend sends an HTTP response.

For example:

200 OK
Content-Type: application/json

with:

[
  {
    "id": 1,
    "name": "Agrima"
  },
  {
    "id": 2,
    "name": "Rahul"
  }
]

And this is where our frontend gets the data.

Remember this?

const response = await fetch("/api/users");

Now we do:

const data = await response.json();

And data becomes:

[
    { id: 1, name: "Agrima" },
    { id: 2, name: "Rahul" }
]

We can finally use it in our UI.

6. What Does `await` Actually Do?

This was another thing that confused me when I started.

An API request takes time.

Maybe:

50ms
200ms
1 second

JavaScript can't just assume the response will instantly arrive.

That's why we use:

await fetch(...)

We're basically saying:

"Send the request and wait for the response before continuing this function."

That's why API calls are commonly written with:

async/await

7. What If the API Fails?

Things go wrong. A lot.

Maybe the server is down.

Maybe the URL is wrong.

Maybe the user isn't authenticated.

Maybe the backend crashes.

So don't write API calls assuming everything will always work.

A basic pattern is:

async function getUsers() {

    try {

        const response =
            await fetch("/api/users");

        if (!response.ok) {
            throw new Error(
                `Request failed: ${response.status}`
            );
        }

        const data =
            await response.json();

        console.log(data);

    } catch (error) {

        console.error(
            "Something went wrong:",
            error
        );

    }
}

Some status codes you'll see often:

200 → Success
201 → Created
400 → Bad Request
401 → Unauthorized
403 → Forbidden
404 → Not Found
500 → Server Error

The Complete Flow

So that tiny line:

fetch("/api/users")

actually triggers a much bigger process:

        USER
          │
          ↓
    Clicks Button
          │
          ↓
     JavaScript
          │
          ↓
     fetch()
          │
          ↓
    HTTP Request
          │
          ↓
      Backend
          │
          ↓
      Database
          │
          ↓
      Backend
          │
          ↓
    HTTP Response
          │
          ↓
     JavaScript
          │
          ↓
       UI Update

And that's basically the request-response cycle.

One More Real-World Example

Think about an AI notes app.

You type:

"Summarize these notes"

and click Summarize.

Behind the scenes:

You click "Summarize"
        ↓
Next.js frontend
        ↓
POST /api/summarize
        ↓
Backend
        ↓
AI API
        ↓
AI generates summary
        ↓
Backend sends response
        ↓
Frontend receives it
        ↓
Summary appears on screen

You only see a button and a result.

But underneath, multiple systems are talking to each other.

That's the part I find most interesting about APIs.

Final Takeaway

An API call isn't just:

fetch("/api/users");

It's a conversation between different parts of an application.

Client → Request → Server → Process → Response → Client

Once you understand this flow, things like REST APIs, Axios, FastAPI, Express, authentication, CORS, and even AI APIs start making a lot more sense.

So next time you write:

await fetch(...)

you'll know what's actually happening behind that one line. 🚀

If you're learning APIs right now, try this: build a tiny frontend + backend with just three endpoints:

GET    /users
POST   /users
DELETE /users/:id

You'll learn more by building those three endpoints than by reading ten API tutorials.

An LLM Is Not Your Backend — Here's What I Learned

Agrima Gupta — Wed, 16 Sep 2026 04:53:23 +0000

When I first started working with AI, I used to think of an LLM as something like a super-smart backend. You give it some input, it understands it, processes it, and gives you an answer. So naturally, I started thinking, "Why do I need so much backend logic? Can't I just tell the LLM what my application does and let it handle everything?"

Turns out, no.
And understanding why completely changed the way I think about building AI applications.

So, What Even Is an LLM?
LLM stands for Large Language Model. In simple terms, an LLM is a system trained on huge amounts of text that learns patterns in language and uses those patterns to generate text.

Imagine you've spent your entire life reading books, articles, conversations, documentation, stories, emails, questions, answers, and code. After seeing enough examples, you become really good at understanding how language works and predicting what kind of sentence would make sense next. If someone says, "The sun rises in the..." your brain immediately expects the word "east." If someone says, "Can you pass me the..." you might expect "salt" or "water."

An LLM does something conceptually similar, except it does this using mathematical representations and neural networks at a massive scale.This is also why LLMs can do so many different things. They can explain concepts, write code, summarize documents, translate languages, generate stories, analyze text, and have conversations. It can feel like you're talking to something that understands everything. But underneath all of that, the model is still working with patterns it learned during training and generating tokens based on those patterns.

Is an LLM Just Autocomplete?
In a way, yes. But modern LLMs are obviously much more complicated than the autocomplete on your phone.Your keyboard might see "I'll see you" and predict that "tomorrow" could come next. That's a very simple example of predicting what comes next.

An LLM takes this basic idea to a completely different scale. Instead of looking at just a few words, it can process much larger contexts and has learned incredibly complicated patterns across language and code.That's why you can give it a paragraph of context and ask it to summarize it, give it a programming problem and ask for an implementation, or give it a messy explanation and ask it to turn that into structured information.

The important thing to understand is that the model doesn't work exactly like a human brain. It processes language mathematically and predicts what should come next based on the context it has received.

How Does an LLM Learn?
This is probably the most important part to understand.

Imagine teaching a child a language. You don't simply give them a dictionary and ask them to memorize every word. They hear people speaking, read things, see patterns, make mistakes, receive feedback, and gradually become better at understanding and producing language.Training an LLM is obviously much more complicated than this, but the basic intuition is somewhat similar.

The model is exposed to enormous amounts of training data. It processes pieces of text and repeatedly tries to predict what comes next. When its prediction is different from the actual data, the model's parameters are adjusted. This process happens again and again across huge amounts of data.

Over time, the model becomes extremely good at recognizing patterns in language.

This doesn't mean it stores every sentence it has seen like a giant database. Instead, the training process changes the model's internal parameters so that it becomes better at representing and generating language.

Here's a Simple Example,
Imagine I say, "I went to the restaurant and ordered..." Your brain might expect words like "food", "pizza", "dinner", or "burger."

An LLM does something conceptually similar, but mathematically. It calculates probabilities for possible next tokens based on the context. For example, the probabilities might conceptually look something like 32% for "pizza", 21% for "food", 15% for "dinner", and so on. These numbers are only an illustration, not actual probabilities from a specific model. The important idea is that the model is constantly predicting what should come next based on the context it has.

Then Why Does It Feel Like It's Thinking?
This is where things get really interesting.

If I ask an LLM, "Explain recursion like I'm five," I get a completely different kind of response than if I ask, "Write a Java solution for this problem."

If I ask it to rewrite an email professionally, it changes its tone. If I give it a long document, it can summarize the important parts. If I give it a programming problem, it can reason through possible approaches.

It can feel like there is an actual person sitting behind the screen thinking through my question. Modern models can perform surprisingly sophisticated reasoning and problem-solving tasks. But we shouldn't confuse that capability with an LLM being a complete software system.

The model generates an output.
Your application still needs to decide what happens after that output.

And that's where the title of this article comes in.

An LLM Is Not Your Backend

Let's imagine you're building an online food delivery application with an AI assistant.

A user says, "Order me a pizza."

You send that message to an LLM, and the LLM replies, "Sure! Your pizza has been ordered."

Sounds great.
But wait.
Did it actually order anything?

No.
The LLM generated a sentence. That's it.

It didn't check which restaurants are open. It didn't check whether the pizza is available. It didn't verify your address. It didn't process your payment. It didn't create an order in the database. It didn't send the order to the restaurant.

Those are backend responsibilities.

This distinction became very clear to me when I started thinking about AI applications as actual software products instead of just chatbots.

LLM vs Backend
The LLM is great at understanding natural language, extracting intent, generating responses, summarizing information, working with context, and deciding which tool might be useful.

The backend is responsible for things like authentication, authorization, database operations, business rules, transactions, validation, payments, APIs, security, and other deterministic operations.

These aren't competing responsibilities.
They're different responsibilities.

The LLM provides flexibility and intelligence around human language, while the backend provides control and reliability.

I Built an AI Voice Sales Agent — Here’s the Architecture Behind It

Agrima Gupta — Tue, 15 Sep 2026 08:55:51 +0000

I Built an AI Voice Sales Agent — Here’s the Architecture Behind It

What if a dealer could simply call a number, tell an AI what they need, and place an order through a normal conversation?

No app.
No searching through products.
No filling out forms.

Just talk.

That was the idea behind a project I've been working on: an AI Voice Sales Agent for dealers.

For example:

"I need 50 boxes of Product X and 20 of Product Y."

The AI should understand the request, check whether the products are available, figure out the dealer's pricing, apply any applicable schemes, confirm the order, and eventually push it into the company's ERP.

This isn't a finished enterprise product yet. I'm still building and figuring things out, but I wanted to document how I'm approaching the architecture and some of the decisions I've made along the way.

The Architecture

The basic flow looks something like this:

Dealer
↓
Phone Call
↓
Twilio
↓
OpenAI Realtime API
↓
LangGraph
↓
Business Tools
↓
FastAPI Backend
↓
PostgreSQL / Redis
↓
ERP

The interesting part isn't just getting an LLM to talk.

The AI actually needs to do things.

It should be able to:

Understand what the dealer wants
Search for products
Check inventory
Get dealer-specific pricing
Check applicable schemes
Remember useful customer context
Create draft orders
Eventually sync with an ERP
Handle failures without making things up

Let's break down how I'm thinking about each part.

The Voice Layer

The first challenge is simple:

How does the dealer actually talk to the system?

I'm using Twilio as the telephony layer.

The basic flow is:

Dealer calls
↓
Twilio receives the call
↓
Audio goes to the AI system
↓
AI processes the conversation
↓
Response is generated
↓
Dealer hears the response

Initially, I thought of voice as simply:

Speech → Text → LLM → Text → Speech

But once you start thinking about an actual conversation, latency becomes a huge deal.

Imagine saying something to an AI and waiting 4–5 seconds for every response.

Technically, it works.

As a conversation?

Not great.

That's why I'm looking at real-time voice capabilities rather than treating the system like a normal chatbot with a microphone attached to it.

The AI Layer

For the conversational intelligence, I'm using the OpenAI Realtime API.

But here's something I realized pretty quickly:

The LLM shouldn't be responsible for everything.

For example, if a dealer says:

"Give me 50 of the blue ones."

The AI needs to understand what "blue ones" refers to.

That requires conversation context.

The system might have something like:

Dealer:
ABC Distributors

Current conversation:
Product: Product X
Variant: Blue
Requested quantity: 50

Dealer preferences:
Preferred warehouse: Pune
Preferred language: English

So the AI understands the conversation instead of treating every sentence as an isolated question.

LangGraph — Making the AI Actually Do Things

This is probably one of the parts I'm most interested in.

I don't want the LLM to directly interact with my database and randomly decide what to do.

Instead, I'm giving the agent specific tools.

For example:

search_product()
check_inventory()
get_dealer_price()
get_scheme()
get_customer_history()
create_draft_order()

So a conversation could look something like:

Dealer:
"I need 50 units of Product X."

AI:
"Let me check the availability."

↓

check_inventory("Product X")

↓

Inventory:
72 units available

↓

AI:
"We have 72 units available. Would you like me to create the order for 50?"

↓

Dealer:
"Yes."

↓

create_draft_order(...)

This is where LangGraph becomes useful.

Instead of having one massive prompt trying to handle the entire business workflow, the agent can move through different states and use specific tools.

Conceptually:

START
↓
Understand Request
↓
Identify Product
↓
Check Inventory
↓
Check Dealer Pricing
↓
Apply Scheme
↓
Confirm Order
↓
Create Draft Order
↓
END

This also makes debugging much easier.

If something goes wrong, I can ask:

Which step failed?

Instead of:

Why did the AI randomly do that?

PostgreSQL — Where the Real Data Lives

One thing I definitely don't want is for the AI to become the source of truth.

The LLM can understand things.

It can reason.

It can communicate.

But it shouldn't be the database.

The system needs structured data for things like:

Dealers
Dealer contacts
Dealer addresses
Products
Product variants
SKUs
Inventory
Orders
Pricing
Schemes
Customer preferences

For example:

Dealer
├── Contacts
├── Addresses
├── Credit Limit
├── Language Preference
└── Preferred Warehouse

Product
├── Product Variant
├── SKU
├── Pricing
└── Inventory

I'm using PostgreSQL for this persistent data.

I'm also using UUIDs for primary keys and keeping things like timestamps, foreign keys, indexes, and soft-delete support in the domain design.

I'm still refining the schema, but getting this foundation right is important because adding AI on top of a messy data model isn't going to magically fix it.

Redis — The Fast Stuff

Not everything needs to be stored permanently in PostgreSQL.

A voice conversation can have a lot of temporary state.

For example:

Current session
Conversation state
Temporary context
Caching
Rate limits

That's where Redis comes in.

The mental model I'm using is basically:

PostgreSQL
= Persistent business data

Redis
= Fast temporary state + caching

Inventory Is Where Things Get Real

Let's say a dealer says:

"I need 100 units."

The AI can't just say:

"Sure, I've placed the order."

It needs to check what's actually available.

Something like:

Dealer Request
↓
Identify SKU
↓
Inventory Service
↓
Available Quantity
↓
AI Response

If the system only has 60 units, the AI should say:

"We currently have 60 units available. Would you like me to create the order for 60?"

This is a principle I'm trying to stick to throughout the project:

The AI should reason about business data, not invent business data.

Dealer-Specific Pricing

This is another place where a normal chatbot approach isn't enough.

In B2B, everyone doesn't necessarily get the same price.

Different dealers might have:

Different price lists
Different discounts
Different schemes
Different credit limits
Different warehouses
Different purchasing histories

So the flow could look like:

Base Product Price
↓
Dealer-specific Price
↓
Applicable Scheme
↓
Discount
↓
Final Price

But here's an important architectural decision:

The LLM shouldn't calculate the final business-critical price itself.

The backend should do that.

The AI can explain the result to the dealer.

The actual calculation should come from deterministic business logic.

Customer Memory

This is probably one of the coolest parts of the idea.

A useful sales agent shouldn't feel like it has amnesia after every phone call.

Suppose a dealer usually orders a particular product or prefers a particular warehouse.

The system could remember useful information such as:

Preferred warehouse
Preferred products
Typical order quantity
Preferred language
Previous orders
Communication preferences

Then a future conversation could be much smoother.

Instead of asking:

"Which warehouse do you want?"

every single time, the system could already know the dealer's preferred warehouse and simply confirm it when needed.

But there's an important distinction here.

Not everything the dealer says should automatically become permanent memory.

Memory needs rules.

Some information is temporary conversation context.

Some information is actual customer data.

And some information probably shouldn't be stored at all.

That's something I want to handle carefully as the project develops.

ERP Integration

Eventually, the AI needs to connect with the systems that the business already uses.

That's where ERP integration comes in.

The architecture I'm aiming for is:

AI Agent
↓
Backend
↓
Business Logic
↓
ERP APIs
↓
Orders / Inventory / Customers

I don't want the AI agent directly modifying ERP data.

Instead:

AI
↓
Tool
↓
Backend Validation
↓
ERP

That gives us a much safer boundary between the unpredictable nature of AI and the deterministic nature of enterprise systems.

Why I'm NOT Giving the LLM Direct Database Access

This is probably one of the biggest things I've learned while designing this.

It can be tempting to just give an LLM database access and say:

"Do whatever you need."

But for a system dealing with real orders, pricing and customer information, that's a very bad idea.

I'd rather have:

❌ LLM → Database

✅ LLM → Tool → Backend → Database

For example:

LLM
↓
get_inventory("SKU123")
↓
Backend validates request
↓
Database query
↓
Structured result
↓
LLM

Now I have a proper place for:

Authentication
Authorization
Validation
Logging
Security
Error handling
Auditing

And most importantly, I know exactly what the AI is allowed to do.

The Tech Stack

Frontend: Next.js

Styling: Tailwind CSS + shadcn/ui

Backend: FastAPI

Database: PostgreSQL

Cache / State: Redis

ORM: SQLAlchemy

Migrations: Alembic

Voice: Twilio

Real-time AI: OpenAI Realtime API

Agent Orchestration: LangGraph

Vector Search: pgvector

Version Control: Git + GitHub

Monitoring: Sentry + OpenTelemetry

I'm trying to avoid choosing technologies just because they're popular.

I want every piece of the stack to have a reason for being there.

What I'm Still Figuring Out

The project is still a work in progress, so there are a lot of things I'm actively figuring out.

Voice latency

A voice agent has to feel like a conversation.

Even a technically correct answer feels bad if it takes too long.

Hallucinations

The agent absolutely cannot randomly invent:

Product availability
Prices
Discounts
Order status
Credit information

Those things need to come from actual systems.

Conversation recovery

What happens when the dealer says:

"No, not that one. The other blue one."

The agent needs enough context to understand what they're referring to.

Tool failures

What if the inventory service is down?

The AI shouldn't pretend that everything worked.

It needs to understand that the tool failed and communicate that properly.

Security

Once you're dealing with actual business transactions, security becomes a major part of the architecture.

Things like:

Authentication
Authorization
PII protection
Call verification
Audit logs
Tool permissions

can't just be an afterthought.

The Biggest Thing I've Learned

The biggest lesson from this project so far is:

Building an AI application isn't the same as putting an LLM inside an application.

The LLM is only one part of the system.

A useful AI product needs:

AI
+
Business Logic
+
Data
+
Tools
+
State
+
Security
+
Observability
+
Reliable Infrastructure

The model provides the intelligence.

But the architecture around it provides the reliability.

And I think that's an important distinction, especially when moving from AI demos to actual products.

What's Next?

Right now, I'm focusing on building the foundation properly before trying to make everything "smart."

The next things on my list are:

Building the core backend APIs
Implementing the voice pipeline
Connecting the agent to business tools
Building the order workflow
Adding customer memory
Connecting inventory
Designing ERP synchronization
Adding observability
Testing real-world conversations

The end goal is pretty simple:

A dealer should be able to pick up a phone and complete a business transaction through a natural conversation.

Something like:

Call
↓
Talk
↓
Confirm
↓
Order

That's the experience I'm trying to build.

Final Thoughts

This project has also changed the way I think about software engineering.

Earlier, I mostly thought about applications as:

Frontend
+
Backend
+
Database

Now I'm asking a lot more questions:

What should the AI be allowed to decide?

What should the backend decide?

Where does the actual source of truth live?

What happens when the AI is wrong?

What happens when a tool fails?

How do we recover from a misunderstood request?

How do we make the whole thing observable?

And honestly, I'm still figuring out many of these answers.

That's probably my favorite part of building this.

I'm not trying to pretend I have the perfect architecture figured out.

I'm building it, breaking things, learning, and improving it as I go.

Build → Break → Learn → Improve.

If you're also building AI agents, voice applications, or AI-powered SaaS products, I'd love to hear what you're working on and what problems you've run into.

Let's learn from each other. 🚀

DEV Community: Agrima Gupta

How Caching Saves Your Website From Thousands of Requests

How Does Netflix Load So Fast? The Secret Is Caching.

So, What Exactly Is Caching?

Let's Take a Simple Example

Cache Hit vs Cache Miss

Your Browser Is Already Using Caching

Then There Are CDNs

What About Backend Caching?

A Simple Backend Example

But Here's the Problem With Caching

Cache Invalidation Is Where Things Get Interesting

Why Not Cache Everything?

What Makes Good Data for Caching?

How Much Faster Can Caching Be?

Caching Exists at Multiple Levels

The Mental Model I Use for Caching

Final Takeaway

How API Calls Actually Work: From Button Click to Server Response

The Big Picture

1. The Button Click

2. The Browser Sends an HTTP Request

3. The Backend Receives It

4. What About the Database?

5. The Response Comes Back

6. What Does await Actually Do?

7. What If the API Fails?

The Complete Flow

One More Real-World Example

Final Takeaway

An LLM Is Not Your Backend — Here's What I Learned

I Built an AI Voice Sales Agent — Here’s the Architecture Behind It

6. What Does `await` Actually Do?