DEV Community: Gaurav Talesara

3 Redis Design Failures You Should Avoid Before They Become Production Incidents

Gaurav Talesara — Wed, 30 Sep 2026 17:01:48 +0000

Caching is usually introduced for one simple reason:

Take pressure off the database and make reads faster.

The architecture often looks straightforward:

Client
  |
  v
Application
  |
  +----> Redis
  |
  +----> Database

A request checks Redis first.

If the value exists, return it.

If it doesn't, query the database, put the result into Redis, and return it.

Simple.

Until the system meets real production traffic.

That's where caching stops being just a performance optimization and becomes a system-design problem.

Let's walk through how a seemingly simple cache evolves through three stages—and the failure modes that appear at each stage.

1. Stale Data

The first version of a cache is usually cache-aside.

The application owns the caching logic.

The flow is:

Request
   |
   v
Check Redis
   |
   +---- HIT ----> Return cached value
   |
   +---- MISS ---> Query database
                     |
                     v
                 Store in Redis
                     |
                     v
                  Return

For read-heavy workloads, this can dramatically reduce database traffic.

Suppose we have a user profile:

{
  "id": 42,
  "name": "Gaurav",
  "email": "gaurav@example.com"
}

The application stores it under:

user:42

Everything works.

Until the user changes their name.

The database is updated:

Gaurav
    ↓
Gaurav Talesara

But Redis still contains:

Gaurav

Now you have two versions of reality.

Database → Gaurav Talesara
Redis    → Gaurav

The database is correct.

The application can still return the wrong answer.

This is stale data.

Why this matters

Caching introduces another copy of your data.

The moment you do that, you have to answer:

When does the cached copy stop being valid?

That's the beginning of cache consistency design.

2. TTL Helps—but Doesn't Solve Everything

A common next step is adding a TTL (Time To Live).

For example:

user:42
TTL = 5 minutes

After five minutes, Redis considers the entry expired.

The next request becomes a cache miss:

Redis MISS
    |
    v
Database
    |
    v
Fresh value
    |
    v
Redis

This gives you eventual freshness without requiring every application write to explicitly update the cache.

But TTL introduces a trade-off.

Longer TTL

You get:

fewer database reads
better cache hit rate

But potentially:

older data
longer periods of inconsistency

Shorter TTL

You get:

fresher data

But potentially:

more cache misses
more database traffic
lower cache efficiency

So TTL isn't really a "freshness setting."

It's a consistency vs. performance trade-off.

And some data simply can't wait five minutes.

If a user changes their email address, waiting for TTL expiration may be unacceptable.

That's where explicit invalidation comes in.

3. Invalidation on Write

Instead of waiting for the cache to expire, the application can invalidate the relevant key when the database changes.

For example:

UPDATE users
SET name = 'Gaurav Talesara'
WHERE id = 42;

Then:

DELETE user:42 FROM Redis

The next read becomes:

Redis MISS
    |
    v
Database
    |
    v
Fresh data
    |
    v
Redis

This gives you a much fresher cache.

But now the application has to maintain the relationship between:

database writes ↔ cache invalidation

And that creates its own failure scenarios.

What if the database update succeeds but cache invalidation fails?

You can end up with:

Database → NEW value
Redis    → OLD value

What if the invalidation happens first but the database transaction fails?

Now the cache is empty even though the database still contains the old value.

There is no universal answer here.

The right approach depends on the consistency requirements of the application.

But there is another problem that has nothing to do with stale data.

And this one can take down the database.

4. Cache Stampede

Imagine you have a popular product page:

product:123

Normally it receives thousands of requests.

Because the data is cached, most of those requests never reach the database.

That's exactly what we wanted.

Now the cache entry expires.

At almost the same moment, thousands of requests arrive.

They all see:

CACHE MISS

And they all do this:

Request 1 → DB
Request 2 → DB
Request 3 → DB
Request 4 → DB
...
Request 5000 → DB

The database was protected by Redis a moment ago.

Now thousands of requests are hitting it simultaneously.

This is a cache stampede.

Another name you'll often see for this pattern is thundering herd.

The important point is that the problem isn't simply that the cache missed.

The problem is that many requests independently react to the same cache miss.

5. Defense #1: Single Flight

One way to solve this is to allow only one request to refresh a missing key.

Suppose 5,000 requests miss:

             Redis MISS
                  |
        +---------+---------+
        |                   |
    Request 1            Requests 2-5000
        |                   |
      DB query              WAIT
        |                   |
        +--------+----------+
                 |
          Fresh result
                 |
             Redis SET
                 |
        Everyone gets result

Request #1 becomes responsible for fetching the data.

The other requests wait for the result instead of independently hitting the database.

This pattern is often called single-flight or request coalescing.

The key idea is:

One cache miss should not become 5,000 database queries.

This is particularly useful when a small number of keys receive a very large amount of traffic.

6. Defense #2: TTL Jitter

There is another failure pattern that can happen even when you aren't dealing with one single hot key.

Imagine millions of cache entries are created at roughly the same time.

If every entry gets exactly:

TTL = 300 seconds

then many of them can expire around the same time.

That can create a synchronized wave of cache misses.

Instead, you can introduce jitter.

Rather than:

TTL = 300 seconds

you might use something conceptually like:

TTL = 300 + random(0, 60)

Now expiration is spread across time.

Instead of:

                    ↓
              EVERYTHING EXPIRES
                    ↓
             DATABASE SPIKE

you get something more like:

      ↓      ↓          ↓    ↓       ↓
   expire  expire     expire expire  expire

The database sees a more distributed load pattern.

TTL jitter doesn't eliminate cache misses.

It reduces the probability of synchronized expiration becoming a traffic spike.

7. Defense #3: Hot-Key Replication

Now consider a different situation.

One key becomes extremely popular.

For example:

product:123

might represent a product that suddenly goes viral.

You can have thousands or millions of requests for essentially the same piece of data.

Even if Redis is distributed, concentrating a huge amount of traffic around one key can create a bottleneck.

This is a hot key.

One approach is to replicate frequently accessed data across multiple cache nodes so requests don't all depend on the same location.

Conceptually:

                  product:123
                       |
            +----------+----------+
            |          |          |
            v          v          v
         Redis-1    Redis-2    Redis-3

Now requests can be distributed rather than concentrating entirely on one cache location.

But this introduces another design question:

How do you keep replicated hot data consistent?

Again, the optimization introduces another system-design trade-off.

8. The Pattern Behind All Three Failures

This is the part that's easy to miss.

You start with:

Database

You add Redis:

Application
     |
   Redis
     |
 Database

Now you have better read performance.

Then production reveals stale data.

So you add:

TTL + Invalidation

Then production traffic reveals cache stampedes.

So you add:

Single Flight
TTL Jitter
Request Coalescing

Then traffic concentration reveals hot keys.

So you consider:

Replication
Load Distribution
Failure Isolation

The architecture keeps evolving.

That's because caching isn't simply:

"Put Redis in front of the database."

You're introducing another distributed component with its own:

consistency behavior
expiration behavior
concurrency
failure modes
traffic patterns
recovery requirements

9. What to Design Before Adding Redis

Before introducing a cache, I'd want the team to answer at least these questions.

1. What can be stale?

Not every piece of data has the same consistency requirement.

Ask:

Can this value be 1 second old?
10 seconds?
5 minutes?
Never?

That answer should influence the caching strategy.

2. What invalidates the data?

Define the relationship between writes and cache entries.

For example:

User updated
    ↓
Database committed
    ↓
Invalidate user:42

Don't leave invalidation as an afterthought.

3. What happens when the key expires?

Ask:

1 request misses?
100?
10,000?
1,000,000?

If the answer is "they all query the database," you may have a stampede waiting to happen.

4. What happens when one key becomes extremely popular?

Measure key access patterns.

A distributed cache doesn't automatically mean distributed traffic.

You can still have:

99% of requests → 1 key

5. What happens when Redis is unavailable?

This is one of the most important questions.

If Redis disappears, does the application:

Fail closed?
      or
Fall back to database?

If everything falls back to the database simultaneously, Redis failure can become a database incident.

Your fallback path needs capacity planning too.

10. The Real Lesson

Caching is one of the easiest performance optimizations to add to an architecture.

It's also easy to underestimate what you're introducing.

The progression often looks like this:

Cache
  ↓
Stale Data
  ↓
TTL / Invalidation
  ↓
Cache Stampede
  ↓
Single Flight / Jitter
  ↓
Hot Keys
  ↓
Replication / Distribution

Each step solves a real problem.

And each step introduces another design decision.

That's the real lesson:

A cache doesn't remove complexity. It moves complexity somewhere else in the system.

So before adding Redis, don't only ask:

"How much faster will this make our application?"

Ask:

"What happens when the cache is stale?"

"What happens when the key expires?"

"What happens when 10,000 requests miss simultaneously?"

"What happens when one key receives 100× normal traffic?"

"What happens when Redis itself fails?"

Those questions are where caching stops being a feature and becomes system design.

I Tried to Break a $12 Server. It Took 5,000+ Concurrent Users.

Gaurav Talesara — Sat, 26 Sep 2026 05:15:45 +0000

There is a common assumption in backend engineering:

If you expect traffic to grow, you need to start thinking about scaling infrastructure early.

More servers.
More CPU.
A dedicated database.
Redis.
Load balancers.
Containers.
Kubernetes.

Sometimes you do.

But sometimes the application isn't actually asking for more infrastructure yet.

I wanted to see how far a very small server could go before reaching a real bottleneck.

So I rented a server for $12/month.

The machine had:

1 CPU
2 GB RAM
Nginx
Node.js
PostgreSQL
No dedicated database server
No Redis

Then I started increasing the load until something broke.

The interesting part wasn't how cheap the server was.

It was what became the bottleneck first.

The experiment

The goal was simple:

Find the point where the application starts showing meaningful performance degradation, identify the bottleneck, and improve it without upgrading the server.

I used a small API application running on a single machine.

The application had a database-backed feed/workflow-style endpoint that required reading data from PostgreSQL and generating a response for the client.

The database was running on the same machine as the application.

There was no separate cache layer.

No Redis.

No horizontally scaled application servers.

Just:

Internet
   |
 Nginx
   |
 Node.js
   |
PostgreSQL

The server cost about $12/month.

That was the entire point.

I wanted to see what the application could actually handle before adding infrastructure.

Making the test more realistic

A simple benchmark that repeatedly calls one endpoint isn't particularly interesting.

Real users don't behave like that.

They open something.

They read.

They perform an action.

They come back.

They refresh.

They request more data.

So I used k6 to create virtual users that followed a more realistic pattern.

The virtual users would:

Request the feed/workflow data.
Perform an occasional write/action.
Request the data again.
Repeat the cycle.

I also populated the database with a meaningful amount of data rather than testing against an empty database.

The load generator itself ran from a second VM so that the machine being tested wasn't also responsible for generating all the traffic.

The goal was to put pressure on the application server while keeping the test environment simple.

Starting with a small load

I didn't jump directly to thousands of users.

The load increased progressively:

This matters because performance problems aren't always obvious when you only look at average latency.

At low traffic, everything looked healthy.

As the number of virtual users increased, the system continued to respond normally.

Then something started changing around the higher loads.

2,000 users: the first warning sign

At around 2,000 virtual users, the average response time still looked reasonable.

But the tail latency started moving.

This is where metrics like p95 and p99 become much more useful than simply looking at average latency.

Imagine these two systems:

System A
Average: 100 ms
p95:     150 ms
p99:     200 ms

versus:

System B
Average: 100 ms
p95:     700 ms
p99:     2,000 ms

Both have the same average.

But the user experience is very different.

The second system has a tail-latency problem.

That was the first signal that the server was approaching a limit.

3,000 users: the test failed

I increased the load again.

At approximately 3,000 virtual users, the experiment crossed the failure criteria.

That gave me an important data point:

The server wasn't simply getting "a little slower."

There was a point where additional concurrency started pushing the system beyond the acceptable performance envelope.

So instead of continuing to increase the load blindly, I backed down.

I wanted to understand the bottleneck first.

2,500 concurrent users

At around 2,500 concurrent users, the system handled approximately:

~232 requests/sec
~288 ms p95 latency

Those numbers were interesting.

But the most interesting information came from the resource utilization.

The server wasn't running out of memory.

RAM usage was still under 1 GB.

PostgreSQL was also behaving normally.

The CPU, however, was sitting around 90%.

That changed the direction of the investigation.

The first instinct could have been:

"The server is too small. Buy a bigger one."

But the measurements were telling a different story.

Finding the actual bottleneck

The resource picture looked roughly like this:

RAM       < 1 GB
Postgres  Healthy
CPU       ~90%

The CPU was the constraint.

This is an important distinction.

If RAM had been nearly exhausted, adding memory would have been a reasonable direction.

If PostgreSQL had been saturated, database optimization would have been the next area to investigate.

But neither was the immediate problem.

The application was spending too much CPU doing work for requests.

So I asked a different question:

How much of this work actually needs to happen for every request?

That led directly to caching.

Optimization #1: Cache the response in Node.js

The feed/workflow response wasn't changing every millisecond.

Yet the application was repeatedly doing the same expensive work for requests that could safely receive a recently generated response.

So I introduced a very simple cache.

The idea was:

Request
   |
   v
Is cached response available?
   |
  Yes ----> Return cached response
   |
  No
   |
   v
Query database
   |
   v
Build response
   |
   v
Cache for 1 second
   |
   v
Return response

The important detail was the TTL:

1 second.

This wasn't an attempt to build a complicated distributed caching architecture.

It was simply:

If another request arrives immediately after this one, don't repeat the same work unnecessarily.

That small change had a significant effect.

The system moved from roughly:

2,500 concurrent users

to:

4,000 concurrent users

before reaching the same kind of limitation.

And I hadn't changed the server.

No additional CPU.

No additional RAM.

No Redis cluster.

No second application server.

The workload had simply changed.

Why did a 1-second cache help so much?

Caching works because computation has a cost.

Consider a simplified request:

Request
   ↓
Node.js
   ↓
PostgreSQL query
   ↓
Process data
   ↓
Serialize response
   ↓
Send response

If 100 requests arrive close together and they all need essentially the same data, doing the complete operation 100 times may be unnecessary.

With a short-lived cache:

Request 1
   ↓
Generate response
   ↓
Cache

Request 2 ─┐
Request 3 ─┤
Request 4 ─┤──> Cached response
Request 5 ─┘

The expensive work can be shared across multiple requests.

That reduces CPU work.

It can also reduce database work.

And because the response is already available, the application has less work to perform per request.

The important point is that caching didn't make the CPU faster.

It reduced how much CPU work was required.

Optimization #2: Move the cache closer to the edge

The next question was:

Why should Node.js be responsible for serving a response that Nginx can serve directly?

The architecture originally looked like:

Client
  |
 Nginx
  |
 Node.js
  |
PostgreSQL

After introducing application-level caching:

Client
  |
 Nginx
  |
 Node.js
  |
Cache
  |
PostgreSQL

That was already better.

But there was another opportunity.

If Nginx could cache the response, a cache hit wouldn't need to travel through the Node.js application at all.

So the architecture became conceptually:

Client
  |
 Nginx
  |
  +---- Cache hit ----> Response
  |
  +---- Cache miss ---> Node.js
                           |
                       PostgreSQL

Now the request could potentially be handled before reaching the application layer.

This is an important optimization because every request that Nginx can satisfy is a request that doesn't consume Node.js CPU.

The result

After moving the caching responsibility to Nginx, the system pushed past 5,000 concurrent users on the same $12/month server.

The progression was approximately:

~2,500 users
     |
     | 1-second application cache
     v
~4,000 users
     |
     | Move cache to Nginx
     v
5,000+ users

The infrastructure didn't become more powerful.

The application simply stopped doing unnecessary work for every request.

That distinction is the main lesson from the experiment.

Concurrent users are not total users

One important clarification:

5,000 concurrent users does not mean the server can only support 5,000 users total.

Concurrency and total user count are different measurements.

If an application has 100,000 registered users but only 500 are actively making requests at a given moment, the server is dealing with roughly 500 concurrent users, not 100,000.

That's why saying:

"This server supports 5,000 users"

would be misleading.

The accurate statement from this experiment is:

The setup was able to push past 5,000 concurrent virtual users under this particular load-test workload.

That doesn't automatically translate into a production capacity number.

Real applications have different request patterns, payload sizes, database queries, background jobs, connection behavior, and traffic distributions.

Benchmarks are useful when we understand exactly what was benchmarked.

What I would not conclude from this experiment

This experiment doesn't prove that everyone should run production systems on a $12 server.

It doesn't.

A production system may need:

High availability
Backups
Replication
Monitoring
Failover
Security controls
Multiple application instances
Database replicas
Disaster recovery
Horizontal scaling
Background workers
Rate limiting

Those requirements are independent of whether a small server can handle a particular workload.

The experiment answers a narrower question:

How much performance can I get from simple infrastructure before infrastructure itself becomes the problem?

In this case, quite a lot.

The bigger lesson: measure before scaling

One of the easiest mistakes in backend engineering is solving a capacity problem before identifying the actual constraint.

You see latency increasing.

You add a bigger server.

But what if the application is wasting CPU?

You see database queries getting slower.

You add a larger database instance.

But what if the same expensive query is being executed thousands of times unnecessarily?

You see more traffic.

You add Redis.

But what if a simple Nginx cache would have handled the workload?

Infrastructure can absolutely solve performance problems.

But infrastructure should be guided by measurements.

In this experiment, the sequence was:

Load test
   ↓
Observe latency
   ↓
Measure resources
   ↓
Find CPU bottleneck
   ↓
Reduce repeated work
   ↓
Test again
   ↓
Find the next limit

That loop is more valuable than any particular server size.

A simple scaling mindset

When an application starts struggling, I like thinking about the problem in this order:

1. What exactly is getting slower?

Don't stop at:

"The API is slow."

Look at:

Average latency
p95
p99
Requests/sec
Error rate
CPU
Memory
Database utilization
Connection counts
Network
Disk I/O

2. What resource is actually saturated?

Find the constraint.

In this experiment:

RAM     → fine
Postgres → fine
CPU     → ~90%

That pointed the investigation toward application work.

3. Can the work be eliminated?

Before making hardware bigger, ask:

Are we calculating the same thing repeatedly?
Can the result be cached?
Can a query be avoided?
Can data be precomputed?
Can work move to a cheaper layer?
Can Nginx/CDN handle something before it reaches the application?

4. Test the change

Don't assume the optimization worked.

Run the same workload again.

Compare the metrics.

5. Scale only when necessary

Eventually, the $12 server will hit another limit.

That's completely fine.

Scaling isn't the failure.

Scaling without understanding why you're scaling is the expensive part.

What surprised me most

The surprising part wasn't that a cheap server handled thousands of concurrent virtual users.

It was how quickly the result changed after removing unnecessary work.

At roughly 2,500 concurrent users, CPU was close to its limit.

A one-second cache increased the tested capacity to roughly 4,000.

Moving that cache to Nginx pushed the test past 5,000.

Same machine.

Same CPU.

Same RAM.

The biggest change was the amount of work the application had to perform.

That's a useful reminder for any backend system:

Before adding capacity, find out whether you can remove the work creating the demand.

Sometimes the cheapest performance optimization isn't a bigger server.

It's one less thing your server has to do.

Jev: AI for Decisions, Not Just Generation

Gaurav Talesara — Thu, 24 Sep 2026 18:05:32 +0000

Most AI applications today are built around generation.

Give a model some context → ask a question → get text back.

But many application workflows don't actually need generated text.

They need a decision.

Should this support ticket go to billing or technical support?

Is this lead worth prioritizing?

Should an AI agent retry a failed action?

Does this content need human review?

Which tool should an agent call next?

This is where Jev gets interesting.

Instead of treating every AI problem as a text-generation problem, Jev is designed around decision-making.

You provide the current state and typed questions, and the model can return structured decisions along with probabilities and confidence.

A simplified flow looks like:

Application State
       ↓
      Jev
       ↓
Decision + Probability + Confidence
       ↓
Application Action

For example:

{
  "route": "billing",
  "probability": 0.92,
  "confidence": "high"
}

Your application can then use that output directly.

Where this could be useful

Support automation
→ classify and route incoming tickets

Sales systems
→ score and prioritize leads

AI agents
→ decide which tool or action should happen next

Content pipelines
→ identify what requires human review

Risk workflows
→ evaluate a situation before allowing an automated action

Data processing
→ classify and triage unstructured inputs

The interesting architectural idea isn't simply adding another AI model.

It's separating generation from decision-making.

A general-purpose LLM is great when the output needs to be language.

But when the application needs something like:

route = billing
retry = false
risk = medium
next_action = create_ticket

a decision-oriented model can fit the workflow much more naturally.

I'm interested to see where this pattern goes as AI applications move from chat interfaces toward systems that continuously make decisions and take actions.

Voice AI can sound human and still fail at sales. The real problem is often conversation logic: understanding customer state, adapting questions, and knowing when to hand off to a human. I wrote about the architecture behind that.

Gaurav Talesara — Sun, 13 Sep 2026 12:01:28 +0000

Gaurav Talesara

Sep 13

Voice AI Doesn't Have a Voice Problem. It Has a Conversation Problem.

#ai #automation #architecture #performance

5 min read

Voice AI Doesn't Have a Voice Problem. It Has a Conversation Problem.

Gaurav Talesara — Sun, 13 Sep 2026 11:55:18 +0000

Voice AI can sound completely human and still be terrible at sales.

The voice can be natural, the response time can be fast, and the agent can handle thousands of calls. It can qualify leads, answer questions, schedule follow-ups, and update the CRM.

And the sales results can still be disappointing.

The problem is that sounding human and having a good sales conversation are two different engineering problems.

A customer rarely follows a predefined script. They interrupt, change direction, raise objections, ask their own questions, or reveal the most important information halfway through the conversation.

That is where many Voice AI sales systems start to struggle.

The problem with linear sales conversations

A typical sales agent follows a simple flow: introduce the company, ask qualification questions, explain the product, handle objections, and try to book a meeting.

It looks reasonable on a whiteboard.

Real customers don't behave that way.

Two customers can give the same answer but need completely different follow-ups. One may already be comparing vendors and ready to buy. Another may only be researching because a problem recently appeared.

A rigid workflow sees the same answer and moves to the next predefined question.

A good salesperson thinks about what the answer actually means.

That difference is the core problem.

Model the customer, not just the lead

Most sales systems already have plenty of customer data: lead score, company information, previous interactions, campaign source, product interest, and CRM history.

That information is useful, but the agent still needs to answer one question during the conversation:

What should happen next?

For that, the system needs conversational state.

The agent should understand what the customer is trying to solve, how urgent the problem is, whether they are actively evaluating solutions, what they already know, and which objections have appeared.

That state should change as the conversation develops.

A customer can move from unfamiliar with the product to curious, then to problem-aware and eventually ready for a sales conversation. The opposite can happen too. A customer may discover that the timing is wrong or that the product isn't a fit.

The system should respond to those changes instead of forcing everyone through the same flow.

This is why I think a Voice AI sales agent needs a state machine, not a better prompt.

Make every answer useful

The next question should depend on what the customer just said.

Suppose a customer says price is the main concern.

The agent shouldn't simply record "price objection" and continue with the script. It should understand what is behind that answer.

Maybe the customer is comparing vendors. Maybe the budget isn't approved. Maybe the value isn't clear. Maybe they are interested but not ready to commit.

Each situation requires a different response.

The same applies to positive signals.

If someone says they are interested, that doesn't automatically mean they want a demo. They may be researching the market or trying to understand whether the product solves a specific problem.

The job of the next question is to reduce that uncertainty.

This is where conversation logic becomes more important than prompt length.

Optimize for progression, not meetings

A common mistake is making "book a meeting" the main objective.

That can push the agent to ask for a meeting even when the customer isn't ready.

A better objective is progression.

The next useful action might be a meeting. It could also be sending pricing information, arranging a callback, connecting the customer with a specialist, or ending the conversation because there is no fit.

Not every cold lead should become a warm lead.

The goal is to reduce wasted conversations and increase the number of interactions that move toward a useful sales outcome.

That also changes how the system should be measured. Call volume and conversation duration are useful operational metrics, but they don't tell you whether the sales process is improving.

Meaningful conversations, qualified leads, warm-lead progression, appropriate human handoffs, meetings that turn into opportunities, and revenue influenced by the system are much more useful measures.

Human handoff is part of the product

The human handoff is another place where Voice AI systems often lose context.

Transferring a call to a salesperson is easy.

Transferring the customer's understanding is harder.

If the customer has spent five minutes explaining their situation and the salesperson starts the conversation from zero, the customer has to repeat everything.

A good handoff should carry the relevant context: the customer's problem, intent, objections, urgency, important questions, and why the AI decided that a human should take over.

The salesperson should be able to continue the conversation instead of restarting it.

This is especially important for technical products where the AI may identify an integration, security, or implementation concern that should be handled by a specialist.

The handoff is not an exception to the product. It is part of the product design.

Build the feedback loop

The call itself isn't the final outcome.

What happens after the call tells you whether the conversation logic worked.

Did the salesperson accept the lead? Was the customer qualified? Did the meeting happen? Did the opportunity progress? Was the lead rejected because there was no fit?

Those outcomes should flow back into the system.

Without that feedback, teams can end up optimizing activity instead of results.

A Voice AI agent may handle thousands of calls and produce impressive transcripts, but if qualified opportunities aren't increasing, the system still has a sales problem.

The feedback loop connects conversation behavior to actual business outcomes.

The architecture matters more than the prompt

From an engineering perspective, I would separate customer context, conversation state, decision logic, response generation, human handoff, and outcome feedback.

The prompt should not become the place where the entire sales process lives.

Once the prompt contains customer-state management, qualification rules, CRM behavior, escalation logic, and every possible objection, it becomes difficult to test and maintain.

A state-driven design gives the team clearer boundaries. You can test whether the system understood the customer's intent, whether the next action was appropriate, and whether the handoff happened at the right time.

You can then improve each part without rewriting the entire conversation.

Voice quality will continue to improve. Voices will become more natural and models will handle more complex conversations.

But the harder problem is deciding what the agent should do with everything it hears.

That requires customer context, conversational state, adaptive decision-making, clear handoff boundaries, and a feedback loop connected to real sales outcomes.

A Voice AI sales agent doesn't need a better script. It needs a better model of the conversation.

Building AI systems? I’d love to connect.

Website - gauravspace.in
Connect on LinkedIn

Gaurav Talesara

The Website Is No Longer the Center of Commerce

Gaurav Talesara — Wed, 02 Sep 2026 18:59:10 +0000

For years, digital commerce has been built around a simple assumption:

Get the user to the website.

Once the user arrives, guide them through a carefully designed journey.

Search.

Landing page.

Product page.

Cart.

Checkout.

The website was the center of the system.

But that assumption is starting to change.

AI-powered interfaces are creating a different model of interaction—one where users may discover products, evaluate options, and potentially complete transactions without following the traditional journey through a company's website.

The journey is becoming something closer to:

Intent → AI → Discovery → Decision → Transaction

The website may still exist.

It may still be important.

But it may no longer be the only interface between a business and its customers.

As an engineering leader, I find the architectural implications of this shift more interesting than the interface itself.

Because if AI becomes another layer between users and businesses, we need to ask a different question.

Instead of:

How do we build a better website?

We may need to start asking:

How does our product exist outside our website?

From page-centric products to capability-centric systems

Traditional web architecture is often built around pages.

A user visits a page.

The page loads data.

The user performs an action.

The backend processes that action.

The journey is designed around a human navigating an interface.

For example:

User
  ↓
Website
  ↓
Product Page
  ↓
Cart
  ↓
Checkout
  ↓
Transaction

The interface controls the experience.

But AI introduces another possibility.

User
  ↓
Intent
  ↓
AI Interface / Agent
  ↓
Business Systems
  ↓
Decision
  ↓
Transaction

This is a fundamentally different interaction model.

The AI does not necessarily need to understand how your website looks.

It needs to understand what your business can do.

What products do you offer?

What are the prices?

What is available?

What are the policies?

What actions can be performed?

What restrictions exist?

What systems can be accessed safely?

This moves part of the engineering problem away from pages and toward capabilities.

Your product needs to become understandable

Humans can navigate ambiguity.

They can look at a product page, interpret an image, read descriptions, compare options, and understand context.

Software systems cannot rely on that in the same way.

AI systems and agents need structured information.

If an external AI system needs to interact with your product, your underlying data becomes increasingly important.

For commerce, that could include:

Product information
Categories and attributes
Pricing
Availability
Inventory
Offers
Shipping information
Return policies
Product compatibility
Restrictions
Business rules

The quality of the user interface still matters.

But structured information behind the interface may become equally important.

A beautifully designed website is not enough if an external system cannot understand what the business actually offers.

APIs may become part of the product experience

For a long time, APIs were mostly considered technical infrastructure.

Something used by:

Mobile applications
Internal systems
Third-party integrations
Partner platforms

But as AI agents become more capable of interacting with external systems, APIs and system capabilities could increasingly become part of the product experience itself.

The question is no longer only:

Can our frontend perform this action?

It may become:

Can a trusted external system understand and safely perform this action?

For example:

Check availability
Get price
Apply offer
Create order
Calculate delivery
Process payment
Track order

These capabilities should not exist only inside tightly coupled frontend flows.

They need to be represented clearly within the system.

This does not mean every business suddenly needs a public API for everything.

Security, authentication, permissions, and business risk still matter.

But the architectural mindset is changing.

The system needs to expose capabilities in a way that can be understood and controlled.

The architecture behind AI commerce

A useful way to think about the future architecture is through four layers.

1. Interface Layer

This is where the user interacts.

It could be:

A website
A mobile app
Search
An AI assistant
A conversational interface
A voice interface
A third-party agent

The important point is that the business should not depend entirely on one interface.

Interfaces can change.

The core capabilities should remain stable.

2. Intelligence Layer

This layer understands intent.

For example:

"I need a laptop for software development under a certain budget."

The system needs to interpret:

User requirements
Budget
Product preferences
Constraints
Available options

An AI system may help translate human intent into structured actions.

But AI should not be responsible for inventing the truth.

It should retrieve information from reliable systems.

That distinction is important.

AI can interpret.

Your systems should remain the source of truth.

3. Business Capability Layer

This is where the actual business operations exist.

For example:

Product Catalog
Pricing Engine
Inventory System
Order Management
Payment System
Shipping System
Customer Management
Business Rules

This layer should contain the actual capabilities of the business.

The interface should consume these capabilities.

An AI agent may consume them too.

The website is just one consumer.

That is the architectural shift I find particularly interesting.

4. Trust and Control Layer

Once AI systems begin taking actions, trust becomes a core engineering problem.

An AI should not simply have unrestricted access to a system.

Businesses need to think about:

Authentication
Authorization
Permission boundaries
Transaction limits
Human confirmation
Audit trails
Observability
Error handling
Rollbacks
Fraud prevention

The more autonomous the system becomes, the more important these controls become.

AI agents may create a more flexible interface.

But flexibility without control creates risk.

The website becomes one interface among many

I don't think websites are disappearing.

People will continue using websites.

Brands will continue designing experiences.

Companies will continue optimize their products for humans.

But the website may gradually lose its position as the only center of the digital experience.

A product could increasingly exist across multiple interfaces.

                 Website
                    │
                    │
Search ─────── Business Systems ─────── Mobile App
                    │
                    │
               AI Agents
                    │
                    │
             Voice Interfaces

The business system becomes the foundation.

Interfaces become ways of accessing it.

This architecture is more resilient to changes in user behavior.

If a new interface becomes important, you don't need to rebuild the entire business.

You need to connect that interface to well-defined capabilities.

The engineering challenge is bigger than adding AI

This is why I don't think the right response is simply:

Let's add an AI chatbot.

The bigger opportunity is to look at the architecture underneath the product.

Can the system clearly represent the business?

Can capabilities be accessed independently?

Are APIs reliable?

Is product data structured?

Are business rules centralized?

Can actions be safely executed?

Can the system explain what happened?

Can failures be detected and recovered?

These are engineering questions.

And they will become increasingly important as AI moves from generating answers to coordinating actions.

The real competitive advantage may move deeper into the stack

For many years, companies competed heavily through the interface.

Better UX.

Better design.

Better onboarding.

Better conversion funnels.

Those things will remain important.

But AI-driven interfaces could make the underlying system architecture increasingly visible.

If an AI agent cannot understand your product, access reliable information, or interact safely with your capabilities, you may become harder to discover and transact with outside your own interface.

The competitive advantage may increasingly come from how well your business capabilities are represented inside the system.

Not just how good the website looks.

A question I think engineering teams should start asking

When building a new product, we often ask:

What screens do we need?

Maybe we should increasingly ask another question:

What capabilities does this business need to expose?

For example:

Instead of starting with:

Product Page
↓
Cart Page
↓
Checkout Page

Start by understanding:

Discover Product
Get Product Details
Check Availability
Calculate Price
Apply Offer
Create Order
Authorize Payment
Arrange Delivery
Track Order

Once the capabilities are clear, different interfaces can be built around them.

A website.

A mobile app.

An AI assistant.

A partner integration.

A voice interface.

The interfaces may change.

The business capabilities remain.

The interface is becoming more flexible. The system needs to become more structured.

That is the part of this shift that interests me most.

AI-powered commerce is not only a change in search, advertising, or checkout.

It could push engineering teams toward a different way of designing products.

One where products need to serve both:

Humans interacting through interfaces.

And increasingly:

AI systems acting on behalf of humans.

The website is not disappearing.

But it may become one interface among many.

And the businesses best positioned for that future may be the ones that have built systems with clear data, well-defined capabilities, strong controls, and reliable foundations.

The interface is becoming more flexible.

The system behind it needs to become more structured.

And I think that's going to be one of the most interesting engineering challenges of the next few years.

15 Things CA Firms Can Automate With AI — Starting With the Work Nobody Wants to Do

Gaurav Talesara — Fri, 28 Aug 2026 17:26:27 +0000

WhatsApp follow-ups, missing documents, GST workflows, loan cases, bank queries, fee collection and more.

A client sends a WhatsApp message:

“Sir, GST ka kya hua?”

Someone from the CA firm's team opens WhatsApp, searches for the client, checks an Excel sheet, looks through Google Drive, asks another team member, and finally replies.

The answer may take two minutes.

Finding the answer can take ten.

Now multiply that by hundreds of clients.

This is where I think the real opportunity for AI in CA firms begins.

Not with another chatbot.

With the repetitive work happening around the CA every single day.

AI Shouldn't Replace the CA. It Should Remove the Work Around the CA.

A CA's time is valuable.

Professional judgment, tax planning, financial advice, audits, client relationships and complex decisions require experience.

But many activities surrounding that work are repetitive.

Someone has to ask for documents.

Someone has to send reminders.

Someone has to check whether documents arrived.

Someone has to update an Excel sheet.

Someone has to follow up with a client.

Someone has to track a bank query.

Someone has to remember which loan case is waiting for what.

These are exactly the kinds of workflows where automation can help.

The question isn't:

“Where can we add AI?”

A better question is:

“What does the team repeatedly do every day that doesn't actually require professional judgment?”

Here are 15 areas worth exploring.

1. Client Document Collection

This is probably one of the easiest places to start.

CA firms constantly need documents from clients: bank statements, purchase registers, sales data, invoices, investment proofs, previous ITRs, GST information and more.

The difficult part isn't asking once.

It's keeping track of what's still missing across hundreds of clients.

Imagine:

Client: Patel Textiles

Required: 10 documents
Received: 8
Verified: 7
Missing: 2

An automation can send the request through WhatsApp or email.

When the client uploads a document, the system can identify it, attach it to the correct client, update the checklist and request whatever is still missing.

The staff no longer has to remember every outstanding document.

The system does.

2. AI Document Classification and Verification

Receiving a document is only half the problem.

The next question is:

Did the client send the right document?

AI can classify uploaded files and perform basic checks before they reach the CA team.

For example:

Uploaded: bank_statement_august.pdf

Document type: Bank Statement
Client: Patel Textiles
Period: August
Readable: Yes
Status: Verified

Or:

Expected: Bank Statement

Received: Invoice

Status: Wrong Document
Action: Request correct document

This can save staff from manually opening and sorting hundreds of files.

3. WhatsApp Client Query Automation

WhatsApp has become one of the most important communication channels for many Indian businesses.

CA firms receive questions throughout the day:

“What documents do I need?”

“When is the GST deadline?”

“Did you receive my statement?”

“What is pending from my side?”

“Has my return been filed?”

An AI assistant can handle the first level of these questions.

But there is an important difference between a generic chatbot and a useful CA assistant.

A generic chatbot knows what GST is.

A useful assistant knows that this particular client's GST filing is waiting for two documents.

Context is the real value.

4. Client Onboarding

New clients create a surprising amount of administrative work.

A new client may require:

KYC.

PAN.

GST details.

Previous returns.

Accounting data.

Bank statements.

Engagement documentation.

Service selection.

Team assignment.

Instead of managing this through spreadsheets and messages, an onboarding workflow can track everything.

Client Onboarding: 72%

Completed:
KYC
PAN
GST

Pending:
Previous ITR
Bank Statements
Accounting Data

The client sees what is pending.

The staff sees what is incomplete.

The partner sees the overall status.

5. GST Compliance Readiness

Compliance automation shouldn't stop at sending deadline reminders.

Knowing that a GST return is due in five days is useful.

Knowing why the return isn't ready is much more useful.

For example:

GSTR-3B

Sales Data        Complete
Purchase Data     Complete
GSTR-2B           Complete
Reconciliation    3 Exceptions
CA Review         Pending
Client Approval   Pending

Now the system can identify exactly what is blocking the filing.

That turns a simple reminder system into a workflow system.

6. GST Reconciliation and Exception Detection

AI doesn't need to review everything.

Sometimes its most valuable role is to find the things that deserve human attention.

For example:

Duplicate invoices
GST amount mismatches
Purchase invoices missing from 2B
Unusual ITC movements
Sales data mismatches
Unexpected changes in margins
Credit-note anomalies

Instead of asking someone to manually inspect everything, AI can create an exception list.

The CA reviews the exceptions.

This is a powerful pattern for professional services:

AI prepares and prioritizes. Humans review and approve.

7. ITR Document and Information Collection

ITR preparation involves another predictable collection process.

The firm may need Form 16, capital gains information, investment proofs, bank statements, interest certificates, previous returns and other supporting information.

The workflow can become:

ITR Client
↓
Checklist Generated
↓
Documents Requested
↓
Documents Received
↓
AI Classification
↓
Missing Information Identified
↓
Staff Review
↓
CA Review

The system isn't preparing tax advice on its own.

It's reducing the administrative work required to get the case ready.

8. Compliance Deadline Management

A calendar tells you when something is due.

A workflow tells you whether you're actually ready.

A better compliance system can automatically create tasks based on the client's services.

For example:

Client
↓
GST
↓
TDS
↓
ITR
↓
ROC
↓
Audit

Each service creates its own workflow.

Tasks can be assigned to staff.

Missing information can trigger client reminders.

Unresolved items can be escalated.

The partner gets visibility without having to ask every employee for an update.

9. Internal Task Assignment

One of the biggest problems in a growing CA firm is simply knowing:

Who is doing what?

A compliance workflow could automatically create:

Client: ABC Industries

Task: GST Reconciliation
Owner: Rahul
Due: 25 August
Status: Waiting for Client

Once the documents arrive:

Status: Ready for Review

Once the CA approves:

Status: Completed

This creates a clear operational trail.

It also makes workload visible across the team.

10. Fee and Payment Follow-ups

Another repetitive process is collecting fees.

The basic workflow is simple:

Invoice
↓
Due Date
↓
Reminder
↓
Client Response
↓
Payment
↓
Receipt

But automation can make this smarter.

If a client says:

“I'll pay on Friday.”

The system can record the commitment.

If Friday passes and payment hasn't arrived, the follow-up can happen automatically.

Staff only needs to intervene when the normal workflow doesn't work.

11. Loan Lead Qualification

This is where things get particularly interesting.

Many CA firms also have finance or loan departments.

They may help clients with business loans, working capital, project finance, machinery finance, home loans, vehicle loans and other financing requirements.

A client might simply send:

“Mare ₹2 crore ni working capital joiye che.”

Instead of immediately handing the conversation to someone, an AI assistant can collect the basic information required for an initial assessment.

For example:

Business Type
Annual Turnover
Existing Loans
Current Bank
GST Status
ITR History
Loan Requirement
Purpose
Collateral

The result becomes a structured finance lead instead of an unstructured WhatsApp conversation.

12. Loan Document Collection

Loan processing involves a significant amount of documentation.

Depending on the case, the team may need financial statements, ITRs, GST information, bank statements, existing loan details, KYC, property documents and other lender-specific information.

A Loan Desk can show:

Loan Case #1047

Required: 24
Received: 18
Verified: 14
Missing: 6

The system can automatically follow up with the client.

It can also identify which cases are blocked because of missing information.

That makes the finance team's pipeline much easier to manage.

13. Bank Query Tracking

This is one of the most interesting workflows in loan processing.

A bank might ask:

“Please explain the increase in debtor days.”

Then:

“Please provide updated CMA.”

Then:

“Please submit promoter ITR.”

These requests can arrive through email, WhatsApp or phone calls.

A structured Loan Desk could turn each request into a task.

Loan Case #1047

Bank Query:
Explain increase in debtor days

Owner:
Rahul

Status:
Waiting for Client

Due:
Tomorrow

Required:
Debtor Ageing Report

Now the team knows exactly what is pending and who owns it.

14. AI-Assisted CMA and Financial Analysis

For CA firms involved in project finance and business loans, financial analysis can become another area for automation.

A system could take historical financial information and help prepare a first-pass analysis.

Historical Financials
↓
P&L
Balance Sheet
Cash Flow
↓
Financial Ratios
↓
Projected Financials
↓
CMA / Project Analysis
↓
CA Review

AI could also help identify unusual changes or prepare explanations for financial movements.

But the final assumptions, analysis and professional output should remain under human review.

AI should accelerate the preparation.

It shouldn't blindly make the decision.

15. An AI Daily Brief for the CA Partner

Eventually, all of these workflows can come together.

But I don't think the answer is another dashboard filled with charts.

A CA partner doesn't necessarily need more information.

They need to know:

What needs my attention today?

The system could provide a simple daily summary:

7 filings blocked
5 clients haven't submitted documents
3 bank queries pending
₹12.8L overdue
8 loan cases waiting for action

That's more useful than opening five different spreadsheets and asking five different people for updates.

The goal isn't more information.

It's less cognitive load.

Don't Build Everything at Once

This is probably the most important point.

It is tempting to build a complete CA platform with:

CRM.

WhatsApp.

AI chatbot.

GST.

ITR.

Documents.

Loans.

CMA.

Payments.

Analytics.

That can quickly turn into a six-month or one-year project before anyone knows whether the product solves a real problem.

A better approach is to pick one workflow.

For example:

Document Collection
↓
Document Verification
↓
Compliance Workflow
↓
GST Exceptions
↓
Loan Desk
↓
Financial Intelligence

Each module can solve a real problem independently.

Then they can gradually connect.

What Should Actually Be Automated?

A simple rule can help.

Automate work that is:

Repetitive
High-volume
Time-consuming
Rule-based
Easy to verify

Keep humans in control of:

Professional judgment
Tax positions
Financial decisions
Client advice
Final approvals
Lending decisions

The best AI system isn't the one that makes every decision.

It's the one that handles the repetitive work and brings the important exceptions to the professional.

The Bigger Opportunity

The future of CA-firm software probably isn't another chatbot.

It's a system that understands the state of every client and every workflow.

Which documents are missing?

Which filings are blocked?

Which tasks are overdue?

Which loan cases are waiting for the bank?

Which clients haven't paid?

Which cases need the CA's attention?

And which tasks can simply be handled automatically?

The CA remains the expert.

The team remains responsible for professional execution.

AI becomes the operational layer that connects the work.

That's where I think the opportunity becomes much more interesting.

If you run a CA firm, what is the one repetitive process your team does every day that you would happily never have to manage manually again?

NOOA: What If an AI Agent Was Just a Python Object?

Gaurav Talesara — Wed, 12 Aug 2026 17:32:29 +0000

There is something interesting happening in the AI agent space.

We have spent the last couple of years building agents using prompts, tools, function calling, workflows, graphs, memory systems, orchestration layers, and increasingly complicated frameworks.

And now NVIDIA Labs has released something that made me stop and think:

What if an AI agent was just a Python object?

That is the basic idea behind NOOA — NVIDIA Object-Oriented Agents.

It is an open-source, model-agnostic Python framework that represents an agent as a Python class, where the object's fields represent state, methods represent capabilities, docstrings can define instructions, and type annotations define interfaces.

The project is still very new and explicitly described as research software. But I think the idea behind it is worth paying attention to.

NOOA on GitHub

So, what is actually different?

Let's look at the traditional way we might build an agent.

We could have:

A system prompt
A collection of tools
Tool schemas
A memory component
An orchestration loop
State management
Some workflow engine
Observability/tracing
Retry logic
Structured output handling

All of these pieces are useful.

But they also create a growing abstraction layer between the developer and the agent.

NOOA takes a different approach.

You define something like:

from nooa import Agent

class SupportAgent(Agent):
    """You are a customer support agent."""

    order_db: OrderDB

    def is_refund_eligible(self, order: Order) -> bool:
        return (
            order.delivered
            and order.days_since_delivery <= 30
        )

    async def triage(
        self,
        message: str,
        order: Order
    ) -> Ticket:
        """Create a typed support ticket."""
        ...

And suddenly the architecture feels very familiar.

The agent is an object.

Its state is on the object.

Its capabilities are methods.

Its interfaces are typed.

Its instructions can live with the methods.

And the interesting part is the ....

A method with a real Python implementation behaves like normal deterministic Python.

A method with an ... body becomes an LLM-driven method at runtime.

That is a surprisingly simple idea.

Why does this matter?

The thing that caught my attention isn't just the syntax.

It is the mental model.

We have traditionally thought about an AI agent as:

Prompt + Model + Tools + Memory + Loop

NOOA is suggesting another abstraction:

Agent = Software Object + Model

That is a meaningful shift.

If the agent is a Python object, then many things software engineers already know how to do become natural again.

Testing.

Refactoring.

Version control.

Tracing.

Dependency injection.

Type checking.

Composition.

State management.

Code review.

Instead of learning another workflow DSL or another orchestration abstraction, developers can start with something they already understand: Python.

NVIDIA's implementation goes further than simply wrapping an LLM in a class. The framework supports typed I/O, live Python objects passed by reference, model-generated Python as an action mechanism, programmable agent loops, context/event APIs, tracing, and long-term memory.

The part I find especially interesting: code as action

This is probably one of the most interesting pieces of NOOA.

Instead of every capability necessarily becoming a traditional function/tool schema that gets serialized into the model context, the model can generate Python and operate within the agent's environment.

In other words:

The model doesn't only call tools.
The model can write code to use the agent's capabilities.

That changes the interface between the LLM and the application.

Python methods and type annotations can become the interface the model works with.

For developers, this could eventually mean less time maintaining huge collections of tool definitions and more time defining clean software interfaces.

Of course, this also creates a very important security problem.

If an LLM can generate and execute Python, we should treat that code as untrusted.

NOOA's own documentation is very clear about this: its AST validation and module restrictions are defense-in-depth mechanisms, not a security boundary. The recommended containment boundary is OS-level sandboxing such as containers or NVIDIA OpenShell.

And I think that distinction is extremely important.

Another interesting idea: agent state

There is another architectural question that NOOA makes interesting.

Where should an agent's state live?

Today, a lot of agent state effectively lives inside the context window.

The longer the conversation becomes, the more we start thinking about:

summarization
context compression
retrieval
memory
token optimization

NOOA instead treats the agent as an object with state.

That opens the door to a different model:

Agent
 ├── State
 ├── Capabilities
 ├── Memory
 ├── Methods
 ├── Context
 └── Model

The LLM becomes part of the object rather than the object being constructed around an LLM conversation.

That is subtle, but I think it could become important as agents become more persistent and autonomous.

Is NOOA going to replace other agent frameworks?

I don't think we know that yet.

And I would actually argue that this is the wrong question.

NOOA is currently a 0.x research preview, and NVIDIA says its public API is not yet stable and can change between releases.

So I wouldn't recommend looking at it today and saying:

"This is the new standard for AI agents."

It isn't.

But I would definitely recommend watching it.

Because research projects like this sometimes introduce an abstraction that looks unusual initially and becomes obvious later.

Remember how strange some ideas looked before they became standard programming patterns?

The interesting question here is whether object-oriented programming becomes a useful abstraction for agent engineering.

My prediction

I don't think the future will be one giant agent framework.

I think we are going to see a convergence of several ideas.

Agents will increasingly look like software components rather than chatbot conversations.

They will have:

Persistent state
Typed interfaces
Memory
Capabilities
Deterministic code
Model-driven reasoning
Observability
Sandboxed execution
Testable behavior
Composable sub-agents

And the boundary between "AI logic" and "software logic" will become much thinner.

That's what makes NOOA interesting to me.

It isn't necessarily introducing another way to call an LLM.

It is asking a more fundamental question:

What should an AI agent look like from a software engineer's perspective?

And the answer from NVIDIA Labs is:

Maybe it should just look like a Python object.

What I want to see next

This is where things get really interesting.

I'd love to see how this approach evolves around:

1. Production reliability

How do these agents behave under real workloads, failures, retries, concurrency, and partial state?

2. Security

If agents can generate and execute code, sandboxing will become a fundamental part of the architecture, not an optional feature.

3. Multi-agent systems

What happens when Python objects representing agents start collaborating with each other?

ResearchAgent
       ↓
PlanningAgent
       ↓
CodingAgent
       ↓
ReviewAgent
       ↓
DeploymentAgent

Can these simply become composable software objects?

4. Agent testing

This could be particularly powerful.

Imagine being able to test an agent almost like any other Python component:

def test_refund_policy():
    agent = SupportAgent(...)
    assert agent.is_refund_eligible(order) is True

And then separately evaluate the probabilistic behavior of its LLM-driven methods.

That separation between deterministic logic and model-driven behavior could become a very useful engineering pattern.

5. Agent observability

NOOA already traces LLM calls, code execution, and method invocations.

I think this will become critical.

As agents become more autonomous, "the model said something weird" won't be enough for debugging.

We will need to know:

What did the agent know?
What state did it have?
What method did it invoke?
What code did it generate?
What did that code change?
What model decision happened?
What happened next?

That is software observability applied to probabilistic systems.

The bigger shift

Maybe the most interesting thing about NOOA isn't NOOA itself.

Maybe it is the direction it represents.

The first generation of AI applications taught us how to integrate models into software.

The next generation is teaching us how to make models behave like software components.

And eventually, I think we will stop drawing such a sharp line between the two.

An agent won't just be:

"an LLM with some tools."

It may become:

"a software component whose reasoning happens to be powered by a model."

NOOA is still early.

The APIs will change.

The patterns will evolve.

There will be competing approaches.

Some ideas will work. Some won't.

But that's exactly why I think it is worth experimenting with now.

Not because NOOA is already the answer.

But because it might be pointing toward a different question.

What does software engineering look like when the software itself can reason?

That is the part I'm watching.

And I have a feeling we are going to see some very interesting updates in this space over the next year.

If you're building AI agents today, I'm curious:

Would you rather build your next agent as a workflow/graph — or as a Python object?

I'd love to hear what other engineers think.

AI #AIAgents #AgenticAI #NVIDIA #Python #LLM #GenerativeAI #SoftwareEngineering #ArtificialIntelligence #DeveloperExperience

Building an AI Workforce for Insurance with n8n, OpenAI, LangGraph and Supabase

Gaurav Talesara — Tue, 16 Jun 2026 18:59:15 +0000

AI for Preparation. Humans for Judgment.

Most AI projects today are one of these:

A chatbot
A customer support bot
A voice assistant
A Q&A system

But I wanted to explore something bigger:

What if businesses could build an AI Workforce?

Instead of one AI assistant,

imagine:

Customer

↓

AI Workforce

├── Discovery Agent

├── Research Agent

├── Policy Comparison Agent

├── Recommendation Agent

├── CRM Agent

└── Follow-up Agent

↓

Human Advisor

↓

Customer

This article explains the architecture and design decisions behind such a system.

Why Insurance?

Insurance is an interesting industry for AI.

Because:

Research is repetitive.
Recommendations are data-driven.
Follow-ups are expensive.
Trust is critical.
Human judgment is still necessary.

This makes Insurance a perfect Human-in-the-Loop AI use case.

Human In The Loop

This is the core philosophy.

I don't want AI to automatically sell insurance.

I don't want AI replacing advisors.

I want:

AI prepares.

Humans decide.

The workflow becomes:

Customer

↓

AI Workforce

↓

Human Advisor Review

↓

Customer

This creates:

Faster recommendations
Better customer experience
Safer AI adoption
Human accountability

AI Workforce Architecture

Customer

↓

WhatsApp
Phone Call
Website Chat
Email

↓

AI Workforce

├── Discovery Agent

├── Research Agent

├── Comparison Agent

├── Recommendation Agent

├── CRM Agent

└── Follow-up Agent

↓

Human Advisor

↓

Customer

Discovery Agent

The Discovery Agent understands the customer.

Responsibilities:

Collect customer profile
Understand goals
Assess risk
Understand existing insurance
Identify gaps

Example Output:

{
  "risk_level":"medium",
  "family_type":"married_with_children",
  "insurance_goal":"health_and_term",
  "recommended_health_cover":"20L",
  "recommended_term_cover":"3Cr"
}

Research Agent

The Research Agent acts like an insurance analyst.

Responsibilities:

Analyze policies
Compare waiting periods
Review exclusions
Evaluate premiums
Generate recommendations

Example:

{
  "customer_profile_summary":"...",
  "top_recommendations":[
      "...",
      "...",
      "..."
  ],
  "risks":[
      "...",
      "..."
  ],
  "confidence_score":0.92
}

Comparison Agent

Creates structured comparisons:

Feature	Plan A	Plan B	Plan C
Coverage	✓	✓	✓
Premium	✓	✓	✓
Waiting Period	✓	✓	✓
Claim Process	✓	✓	✓

Output:

Best Overall
Best Budget
Best Family Plan

Recommendation Agent

Creates:

Customer Summary
Recommended Plan
Alternatives
Risk Analysis
Advisor Notes

Everything before the advisor joins.

CRM Agent

Updates:

Customer Records
Recommendations
Activities
Opportunity Status
Tasks

Follow-up Agent

Handles:

WhatsApp reminders
Renewal alerts
Email follow-ups
Call notes
Engagement tracking

Omnichannel AI

One important decision:

Customers should not install a new application.

The AI Workforce should operate through:

WhatsApp
Phone Calls
Website Chat
Email
SMS

Different channels.

Same intelligence.

Technology Stack

Frontend

Next.js
Tailwind
Lovable AI

Workflow Layer

AI Models

OpenAI
Gemini
Claude

Multi-Agent Framework

LangGraph

Database

Supabase
PostgreSQL

Memory

Pinecone

Monitoring

LangSmith
PostHog

Why n8n First?

I intentionally started with n8n.

Because:

Fast prototyping
Visual workflows
Easy OpenAI integration
Easy Supabase integration
Easy WhatsApp integration
Easy Email workflows

After validation:

n8n

↓

NestJS

↓

LangGraph

↓

Production AI Workforce

The Bigger Vision

I don't think AI will replace Insurance Advisors.

I think every advisor may eventually have:

An AI Workforce working behind the scenes.

AI provides:

Speed
Consistency
Scale

Humans provide:

Trust
Empathy
Judgment

The future is not:

Human vs AI

The future is:

Human + AI Workforce

If you're building something similar, I'd love to hear your thoughts.

The Rise of Production-Grade AI Infrastructure

Gaurav Talesara — Sat, 23 May 2026 07:01:21 +0000

Most AI products today are impressive in demos.

But the moment they hit production:

workflows break
context fails
hallucinations appear
costs explode
observability disappears

The AI industry does not really have an “intelligence” problem anymore.

It has an infrastructure problem.

For the last two years, the ecosystem focused heavily on:

chat interfaces
prompt engineering
copilots
wrappers around foundation models
“AI-powered” product features

That phase accelerated adoption.

But the market is now entering a different stage.

The hard problem is no longer:

“Can AI generate something useful?”

The hard problem is:

“Can AI systems operate reliably in real production environments?”

And that is where the next major opportunity is emerging.

The Demo Problem

Most AI demos look incredible.

They can:

generate code
summarize documents
automate workflows
answer questions
orchestrate tasks

But production environments expose a completely different reality.

Once real users, real workflows, and real operational constraints enter the system, problems begin to appear:

hallucinations
fragile context handling
inconsistent outputs
broken execution chains
runaway costs
poor observability
unsafe automation
missing governance
unpredictable agent behavior

This is why so many AI pilots never move beyond experimentation.

The market today is filled with:

AI interfaces
AI assistants
AI wrappers
AI copilots

But what enterprises actually need are:

reliable systems
operational controls
execution runtimes
observability layers
governance infrastructure
context orchestration

That is the real bottleneck now.

AI Systems Need a New Production Stack

Traditional software engineering was built around deterministic systems.

AI systems are different.

They are:

probabilistic
context-sensitive
state-fragile
operationally unpredictable

That means traditional software patterns are no longer enough.

AI requires an entirely new operational layer.

This feels very similar to earlier infrastructure shifts:

Kubernetes standardized container orchestration
Datadog transformed observability
Stripe simplified payment infrastructure
Temporal improved workflow reliability

AI is now reaching a similar stage.

The next generation of products will not just be AI applications.

They will be:

AI production infrastructure platforms.

The Real Layers of a Production-Grade AI System

Most discussions about AI still focus only on models.

But production-grade AI systems require much more than a model.

Below are the infrastructure layers that are becoming increasingly important.

1. Context Engineering

This is becoming one of the most critical areas in AI engineering.

Most AI systems fail not because the model is weak, but because the context is poor.

Production systems need to manage:

historical memory
workflow state
user intent
permissions
business logic
external data
codebase understanding
semantic relationships

This goes far beyond basic RAG.

The future belongs to systems that can dynamically assemble the right context at the right moment.

Prompt engineering is becoming commoditized.

Context engineering is becoming the moat.

2. Agent Execution Runtime

Most AI agents today are unreliable because they lack execution infrastructure.

A production runtime needs:

retries
rollback support
checkpoints
workflow state tracking
timeout handling
safe execution paths
human approval systems

Without this, AI workflows become fragile very quickly.

The market does not just need agents.

It needs:

workflow infrastructure for AI systems.

3. Observability for AI Systems

Debugging traditional software is already difficult.

Debugging AI systems is significantly harder.

Production AI requires visibility into:

prompts
memory retrieval
tool calls
reasoning chains
execution paths
token usage
latency
hallucination patterns
workflow failures

Most current systems still operate like black boxes.

This creates a massive opportunity for:

AI observability
AgentOps
runtime tracing
execution replay
quality monitoring

The industry will likely see a:

“Datadog for AI systems”

category emerge.

4. Governance and Safety

As AI systems become more autonomous, governance becomes mandatory.

Enterprises need:

approval workflows
audit trails
permission systems
policy enforcement
data isolation
secure execution environments

Without operational controls, companies will struggle to trust autonomous systems at scale.

This becomes especially important in:

healthcare
finance
enterprise automation
internal copilots
operational workflows

Governance is no longer optional infrastructure.

It is foundational infrastructure.

5. Evaluation and Reliability Testing

One of the biggest problems in AI today is silent degradation.

An AI workflow may work perfectly today and fail tomorrow because of:

model updates
prompt changes
retrieval drift
API schema changes
edge cases
workflow changes

That means AI systems need continuous evaluation.

Production-grade AI requires:

regression testing
scenario simulation
adversarial testing
replay systems
benchmark scoring
workflow validation

This category is still massively underdeveloped.

Why Infrastructure Will Matter More Than Interfaces

The first AI wave rewarded:

interfaces
demos
speed
accessibility

The next AI wave will reward:

reliability
orchestration
observability
governance
scalability
operational maturity

That changes where the real value gets created.

The winning companies may not be the ones with the best chat interface.

They may be the ones building:

context runtimes
orchestration layers
observability platforms
execution infrastructure
repo intelligence systems
AI governance tooling

The real opportunity is shifting downward into the infrastructure layer.

Repo Intelligence Might Become a Major Category

One particularly interesting opportunity is repo intelligence.

Current AI coding tools can generate code.

But they often lack:

architectural understanding
dependency awareness
service relationships
domain knowledge
operational context

That creates problems in large production codebases.

A smarter system would:

scan repositories
understand architecture
build dependency graphs
map services
infer business domains
track workflows
generate contextual intelligence for AI systems

This could dramatically improve:

AI coding reliability
automated refactoring
debugging
onboarding
workflow automation

The future of AI-assisted engineering may depend heavily on systems that deeply understand software architecture.

What This Means for Builders

If you are building in AI today, this shift matters.

The market is getting saturated with:

wrappers
chat interfaces
generic copilots
shallow automation tools

But infrastructure gaps are still massively underbuilt.

That means opportunities are emerging in:

context orchestration
observability
evaluation systems
governance tooling
repo intelligence
workflow runtimes
execution reliability

The next major AI products may come from engineering pain, not prompt creativity.

The Market Is Moving from Apps to Systems

This is the transition happening right now.

We are moving from:

AI apps → AI infrastructure
prompts → context systems
copilots → execution runtimes
experimentation → operational maturity
wrappers → production platforms

The companies that win in AI will likely be the ones that solve:

reliability
orchestration
observability
governance
context management
execution safety

Not just generation.

The biggest AI companies of the next decade may not even look like AI companies.

They may look like infrastructure companies.

Final Thoughts

AI will absolutely transform software.

But models alone are not enough.

The next major challenge is building systems that AI can operate inside reliably.

That means:

better infrastructure
better orchestration
better context systems
better observability
better governance
better operational tooling

The future of AI does not belong only to model providers.

It also belongs to the companies building the operational layer around those models.

And that may become one of the biggest infrastructure opportunities of the next decade.

Why Software Isn’t Built for AI Agents

Gaurav Talesara — Sat, 02 May 2026 12:52:03 +0000

The next users of your software won’t be humans.
They’ll be agents.

And most software today is completely unprepared for that.

Right now, AI agents are:

Browsing websites
Filling forms
Clicking buttons
Navigating dashboards

That’s not scale. That’s a workaround.

We’re forcing machines to behave like humans—because our systems were never designed for anything else.

The Core Problem

Modern software is built around a simple assumption:

A human will be sitting in front of a screen.

That assumption drives everything:

UI-heavy workflows
Step-by-step interactions
Documentation meant to be read, not executed

But agents don’t need interfaces.
They need interfaces they can reason about and execute against programmatically.

Where Current Systems Break for Agents

Let’s break this down from a systems perspective.

1. UI-First Architecture

Most SaaS products expose functionality through:

Dashboards
Forms
Buttons

Agents interacting with these:

Rely on scraping or automation layers
Break when UI changes
Lack reliability

2. Non-Deterministic Outputs

Agents need:

Structured responses
Predictable schemas

Instead, they get:

HTML pages
Inconsistent API responses
Unstructured data

3. Human-Centric Authentication

Current flows:

OAuth screens
Email verification
CAPTCHA

These are friction points for agents trying to:

Discover tools
Authenticate
Execute tasks autonomously

4. Documentation Isn’t Machine-Readable

Docs today are:

Written for humans
Scattered across pages
Hard to parse programmatically

Agents need:

Structured capability descriptions
Executable contracts
Clear input/output expectations

APIs Alone Are Not the Answer

A common assumption is:

“We already have APIs, so we’re agent-ready.”

That’s not true.

APIs are:

Too generic
Often inconsistent
Not designed for autonomous decision-making

Agents need more than endpoints.

They need:

Action schemas (what can be done, not just how)
Deterministic contracts (guaranteed outputs)
Capability discovery (what tools exist and when to use them)

What “Agent-First Software” Actually Looks Like

If we design systems for agents as first-class users, the architecture changes.

1. Machine-Readable Interfaces

Instead of UI-first:

Structured APIs with strict schemas
Tool definitions with clear contracts
Standardized input/output formats

2. Programmatic Onboarding

Instead of:

Signup → verify → explore

Agents should:

Discover → authenticate → execute

This requires:

Auto-provisioned credentials
Machine-readable pricing/limits
Capability endpoints

3. Permissioned Execution

Agents need controlled autonomy:

Scoped access tokens
Role-based permissions
Execution boundaries

4. Deterministic Execution Layer

Every action should be:

Predictable
Retry-safe
Observable

5. Observability for Agents

Traditional logs aren’t enough.

We need:

Decision tracing
Tool-call lineage
Cost per execution
Latency breakdowns

A Practical Agent-System Architecture

A simplified flow looks like this:

Agent
  ↓
Planner (decides what to do)
  ↓
Tool Registry (what tools are available)
  ↓
Execution Layer (calls APIs/tools)
  ↓
Response Validator (ensures correctness)
  ↓
Memory (stores context + learnings)

Each layer is critical:

Planner → reasoning
Tool registry → discoverability
Execution → action
Validator → reliability
Memory → continuity

This is very different from traditional request-response systems.

Where the Opportunity Is

Most people today are focused on:

“How do we build better agents?”

But the bigger opportunity is:

“How do we build better systems for agents to operate on?”

Every major category is open:

CRM → agent-native workflows
Payments → programmable financial actions
Support → autonomous resolution systems
Analytics → queryable, structured insights

Not as add-ons.
But as core design principles.

What Most People Get Wrong

❌ “APIs are enough”

They’re not.
Agents need structured, reliable, discoverable systems.

❌ “Just add AI on top”

That creates brittle layers, not scalable systems.

❌ “Agents will replace software”

No.
Agents will consume software differently.

The Shift That’s Coming

We’re moving from:

Human-first software → to
Agent-first systems

This isn’t a feature upgrade.

It’s a paradigm shift in how software is designed and consumed.

Final Thought

The companies that win won’t be the ones with the smartest agents.

They’ll be the ones:

Agents prefer to use.

👋 If You’re Building in This Space

I’m currently working on:

Agent-based systems
Automation architectures
AI-native SaaS workflows

If you’re exploring similar problems or thinking about building agent-first products, I’d be interested to exchange ideas.

Engineers Won’t Just Have Salaries - They’ll Have Token Budgets

Gaurav Talesara — Mon, 13 Apr 2026 18:40:27 +0000

Introduction

There’s a subtle shift happening in how software is being built.

It’s not loud.
It’s not fully standardized.
But it’s already visible if you look closely.

We are moving from a world where:

Engineering output was limited by human effort

To a world where:

Output is increasingly limited by how much AI you can effectively use

And that introduces a new concept most teams are not yet fully prepared for:

Token budgets.

What’s Changing Right Now

If you zoom into how modern engineering teams are working:

AI tools are no longer optional — they’re embedded in daily workflows
Engineers are generating, reviewing, and iterating faster than ever
The bottleneck is shifting from writing code to orchestrating systems

Some early signals:

Companies are beginning to track AI usage per employee
AI costs are becoming a visible line item in engineering budgets
Token consumption is growing at an unpredictable pace

This isn’t theoretical.

It’s already happening in pockets of the industry.

The Real Constraint Has Changed

Traditionally, engineering constraints looked like this:

Developer bandwidth
System architecture
Infrastructure scaling

Now there’s a new constraint emerging:

Effective AI utilization

Two engineers today are no longer equal if:

One uses AI occasionally
The other builds workflows, agents, and automation around it

The second engineer is operating with leverage.

And that leverage is powered by tokens.

Why Token Budgets Will Emerge

Right now, most companies are in an experimental phase:

Pay-as-you-go AI usage
No clear limits
Costs that are hard to predict

This doesn’t scale.

As usage increases, companies will need:

Cost control
Predictability
Fair distribution of resources

The natural evolution?

Allocated token budgets per engineer or team

Just like:

Cloud budgets
API rate limits
SaaS seat allocations

Tokens = The New Productivity Unit

We’re used to measuring productivity through:

Output (features shipped)
Velocity (story points, sprints)
Efficiency (time to deliver)

But AI introduces a different layer.

Now, productivity is increasingly tied to:

How effectively you can convert tokens into outcomes

Not all token usage is equal.

Some engineers waste tokens on low-value prompts
Others build reusable systems that compound output

This is where the real differentiation will happen.

The Rise of the “AI-Orchestrating Engineer”

The best engineers in the next phase won’t just:

Write clean code
Design scalable systems

They will:

Design agent workflows
Optimize token usage vs output
Build systems that act, not just respond

In other words:

They will orchestrate intelligence.

What This Means for Engineering Leaders

If you’re leading teams today, this shift has implications:

1. Budgeting will change

AI costs will move from “tools” to core infrastructure spend

2. Hiring signals will change

You won’t just evaluate:

Coding ability

You’ll evaluate:

AI leverage
System thinking
Automation mindset

3. Internal tooling will evolve

Teams will build:

Internal agents
Workflow automation systems
Token-efficient pipelines

Why This Isn’t Mainstream Yet

It’s important to stay grounded.

Most companies today:

Do NOT have formal token budgets
Are still figuring out pricing and limits
Are experimenting without clear governance

This is still early-stage behavior.

But the direction is clear.

The Bigger Shift: From Software to Systems

Today’s companies are built on:

APIs
Databases
Services

Tomorrow’s companies will increasingly rely on:

Agents
Workflows
Token pipelines

And that changes how value is created.

Final Thought

We’re not just adding AI to existing systems.

We’re redefining how work gets done.

The question is no longer:

“How fast can your team build?”

It’s:

“How effectively can your team deploy intelligence?”

And in that world—

Tokens become leverage.

Closing

This shift isn’t fully visible yet.

But it’s already in motion.

The teams that understand it early will have an advantage that compounds over time.

If you're building or leading engineering teams right now—

How are you thinking about AI usage?

As a tool…

Or as infrastructure?