DEV Community: Gaurav Talesara

7 Ways to Improve AI Code Reviews

Gaurav Talesara — Sun, 08 Nov 2026 04:33:59 +0000

AI coding assistants have changed how we develop software.

Tools like Claude Code, Cursor, GitHub Copilot and others can now generate features, refactor code, modify multiple files and implement entire workflows in a fraction of the time it traditionally takes.

This is a significant productivity improvement.

But there is an emerging challenge that engineering teams need to address.

AI can generate code faster than humans can review, understand and validate it.

Consider a typical development scenario.

A developer asks an AI assistant to implement a feature. Within minutes, it modifies multiple files, introduces new components, updates APIs and generates supporting tests.

The pull request is ready.

But someone still needs to understand what changed, whether the implementation follows the architecture, whether edge cases are covered and whether the changes could affect existing functionality.

Now imagine multiple developers doing this simultaneously.

Reviewers have limited time, existing commitments and delivery deadlines. Reviewing every generated change deeply becomes increasingly difficult.

This can lead to superficial reviews, missed issues and approvals without sufficient validation.

So, how can engineering teams improve their code review processes while continuing to benefit from AI-assisted development?

Here are seven practical approaches.

1. Keep Pull Requests Small and Focused

AI coding assistants can modify multiple files and generate large amounts of code in one go.

However, larger pull requests generally require more effort to understand and review.

Instead of allowing AI to implement an entire feature as one massive change, break the work into smaller, logical units.

For example, when implementing a user management module, separate the work into:

Database schema and migrations.
Backend APIs and business logic.
Authentication and authorization.
Frontend implementation.
Integration and regression tests.

What to implement:

Define clear boundaries for AI coding tasks.
Separate refactoring from feature implementation.
Avoid unrelated changes in the same PR.
Set reasonable PR size expectations.
Encourage incremental commits.

The objective is not to impose an arbitrary line-count limit. It is to make each change understandable and independently reviewable.

2. Review the Intent Before the Implementation

One mistake reviewers can make is jumping directly into the code without first understanding what the change is supposed to accomplish.

AI might produce an implementation that looks technically correct but doesn't fully address the original requirement.

Before reviewing the code, understand:

What problem is being solved?
What approach was selected?
Which modules and files were modified?
What assumptions did the AI make?
What architectural decisions were introduced?
What existing functionality could be affected?

What to implement:

Ask developers to include an AI-generated implementation summary in every pull request.

A useful PR description should contain:

Business requirement.
Implementation approach.
Important files and components changed.
Design decisions and assumptions.
Potential impact areas.
Tests performed.

This gives reviewers context before they start examining individual changes.

3. Use AI to Review AI-Generated Code

If we are using AI to generate code, why not use AI to assist with reviewing it?

AI-assisted review tools can help identify issues that might otherwise require significant manual effort.

For example:

Potential bugs and incorrect logic.
Missing error handling.
Security vulnerabilities.
Missing test scenarios.
Inconsistent coding patterns.
Possible regression areas.
Unnecessary complexity.

Some tools worth exploring:

CodeRabbit: AI-powered pull request reviews.
GitHub Copilot Code Review: AI-assisted code review within GitHub.
Qodo: AI-powered code review and quality workflows.

What to implement:

Introduce an independent AI review step before human approval.

Ideally, the reviewing agent should have access to the original requirements, relevant repository context and the actual code changes.

One important consideration: AI-generated review comments also need validation. AI review can miss defects or produce false positives.

It should assist human reviewers, not replace their accountability.

4. Prioritize Reviews Based on Risk

Not every code change deserves the same level of scrutiny.

A change to a UI component has a different risk profile from a change to authentication, payment processing or database permissions.

Treating every PR equally can waste valuable engineering review capacity.

Consider introducing risk-based reviews.

High-risk changes:

Authentication and authorization.
Payment processing.
Database migrations.
Data access and permissions.
Security-sensitive APIs.
Infrastructure and deployment configurations.
Critical business logic.

Medium-risk changes:

Third-party API integrations.
Shared components.
Business workflows.
Data transformation logic.

Lower-risk changes:

UI styling.
Static content.
Isolated presentation changes.

What to implement:

Introduce risk labels for pull requests.
Define mandatory reviewers for critical modules.
Require additional validation for sensitive changes.
Use CODEOWNERS to enforce ownership-based reviews.
Apply stricter approval policies to high-risk changes.

The goal is to direct human attention towards changes where engineering judgment matters most.

5. Generate Tests Alongside Code, Not After It

One common mistake is treating testing as a separate activity that happens after implementation.

With AI-assisted development, testing should be part of the implementation workflow itself.

For meaningful changes, consider:

Unit tests.
Integration tests.
API contract tests.
End-to-end tests.
Regression tests.
Negative and edge-case scenarios.

For example, if AI implements a payment workflow, don't test only successful payments.

Also validate:

Failed transactions.
Duplicate requests.
Payment timeouts.
Invalid amounts.
Webhook retries.
Unexpected third-party responses.

What to implement:

Ask AI to explain the test scenarios it has covered and identify scenarios it has not covered.

Review the tests against business requirements rather than simply accepting tests because they were generated alongside the implementation.

Remember: Passing tests do not automatically mean the implementation is correct. The tests themselves must be meaningful.

6. Automate Quality Checks Before Human Approval

Human reviewers shouldn't spend their limited time identifying every predictable issue.

Move those checks into the CI/CD pipeline.

A practical pipeline should include:

Automated check	Purpose
Linting and formatting	Maintain code consistency
Static analysis	Identify code-quality issues
Unit and integration tests	Validate functionality
Dependency scanning	Detect vulnerable packages
Secret detection	Prevent credential exposure
Build validation	Catch build failures
Architecture checks	Enforce engineering standards
Coverage checks	Identify potential testing gaps

Tools worth exploring:

SonarQube for code quality and static analysis.
Snyk for application and dependency security.
GitHub Actions for automated CI/CD checks.

What to implement:

Make critical checks mandatory before merging.

Use automated results to identify which changes need additional human attention.

This reduces repetitive review effort and creates a consistent quality baseline across the team.

7. Monitor Production and Close the Feedback Loop

Code review doesn't end when a pull request is merged.

Even after thorough reviews and automated testing, some issues will reach production.

This is why production monitoring and feedback are important parts of the engineering quality process.

Monitor:

Application errors.
API failures.
Performance and latency.
Database issues.
Security events.
Deployment health.
Critical user workflows.

Tools such as Sentry and Datadog can help teams identify and investigate production issues.

What to implement:

Whenever a production incident occurs, evaluate whether it was related to:

An incorrect implementation.
An overlooked edge case.
Insufficient test coverage.
An architectural decision.
An incomplete review.

Feed those findings back into coding guidelines, review checklists and automated tests.

This creates a continuous improvement cycle instead of treating every production incident as an isolated problem.

Putting It All Together: An Improved AI Code Review Workflow

These seven practices can be combined into a more structured engineering workflow.

1. AI-assisted implementation

Generate code in small, focused changes with clear requirements.

↓

2. Developer self-review

Review generated changes and provide implementation context.

↓

3. Automated validation

Run builds, tests, static analysis and security checks.

↓

4. Independent AI review

Identify potential bugs, missing tests and risk areas.

↓

5. Risk-based human review

Focus on architecture, business logic, security and critical decisions.

↓

6. Controlled deployment

Use staging, deployment checks and appropriate release safeguards.

↓

7. Production monitoring

Detect issues and feed findings back into development and testing.

What Should Engineering Teams Measure?

As AI increases code generation speed, traditional productivity metrics may become less meaningful.

Instead of measuring productivity only through lines of code, PR counts or implementation speed, consider tracking:

Metric	What it tells us
PR review time	How long changes wait for review
Review backlog	Whether review capacity is becoming a bottleneck
Change failure rate	How often deployments cause failures
Escaped defects	How many defects reach production
Rework after merging	How much additional engineering effort is required
Test effectiveness	Whether tests catch meaningful defects
Mean time to recovery	How quickly teams resolve production incidents

The goal is to understand whether AI is improving engineering outcomes, not simply increasing the volume of code being delivered.

Final Thoughts

AI is not going away, and asking developers to stop using AI coding assistants isn't the answer.

We need to evolve our engineering practices alongside these tools.

AI can accelerate implementation, but code quality, security, architecture, testing and production reliability still require deliberate engineering processes.

The objective shouldn't be to make humans review code faster. It should be to make the entire review and validation process more intelligent and effective.

Don't measure AI productivity only by how much code it generates.

Measure it by how much reliable, secure and maintainable software reaches production.

Let's Discuss

For developers, tech leads and engineering managers:

Are you seeing pull requests becoming significantly larger with AI coding assistants?
Are you using AI-powered code review tools in your development workflow?
Which of these seven practices have made the biggest difference to your team's code quality?

I'd be interested in hearing what's working in real engineering teams.

6 Automations You Can Put in Place to Strengthen Security Across Your Software Projects

Gaurav Talesara — Sat, 31 Oct 2026 18:16:41 +0000

Modern software projects depend on a growing number of frameworks, libraries, APIs, cloud services, and third-party integrations.

An engineering team might maintain several applications using different technology stacks. One project may use React and Node.js, another Python and FastAPI, while an AI application may depend on LangChain, LiteLLM, vector databases, and external model APIs.

Keeping all these technologies secure requires continuous attention.

Security advisories need to be monitored. Dependencies need to be checked. Vulnerabilities need to be mapped to affected projects. Development pipelines need security checks. Framework changes and deprecated APIs need to be reviewed.

Doing all of this manually across multiple projects can become difficult to manage.

The challenge is not simply finding security information. It is identifying which information affects your projects and making sure the right engineers know what action to take.

This is where automation can help.

By combining workflow automation platforms such as n8n, GitHub APIs, vulnerability databases, security scanners, and AI-powered analysis, engineering teams can build a system that continuously monitors their technology stack and brings relevant issues to their attention.

Here are six automations worth considering.

1. Technology and Vulnerability Intelligence

The first automation is to monitor the technologies used across your projects and identify relevant security advisories.

Most engineering teams use many different technologies. Keeping track of every security announcement from every framework, library, cloud provider, and development platform is not practical.

Instead of asking developers to monitor these sources manually, create a workflow that collects relevant updates automatically.

What should it monitor?

Depending on your technology stack, the monitoring list could include:

Category	Technologies to monitor
JavaScript	Node.js, npm, React, Next.js, Express
Python	Python, FastAPI, Django, Flask
AI frameworks	LangChain, LangGraph, LiteLLM
AI platforms	OpenAI, Anthropic, Google AI
Automation	n8n, Make, VAPI
Databases	PostgreSQL, MySQL, MongoDB, Redis
Vector databases	Pinecone, Weaviate, Milvus
Cloud	AWS, Azure, Google Cloud
Infrastructure	Docker, Kubernetes, Terraform
Authentication	Auth0, Supabase Auth, Firebase Auth
DevOps	GitHub Actions, GitLab CI/CD

Where can the information come from?

GitHub Security Advisories
OSV.dev
CISA Known Exploited Vulnerabilities (KEV)
National Vulnerability Database (NVD)
CERT-In advisories
OWASP security announcements
Official cloud and framework security advisories

How should the automation work?

Run scheduled checks against selected security sources.
Collect newly published advisories and updates.
Extract affected technologies, packages, versions, and severity.
Use AI to classify and summarize the information.
Match the affected technology against the organization's project inventory.
Notify the relevant project owner when an action is required.

For example, suppose a security advisory is published for LiteLLM.

The automation should identify the affected versions, check whether any project uses LiteLLM, and notify the responsible engineer if an affected version is present.

A useful notification should contain:

Advisory or vulnerability reference.
Affected technology and versions.
Projects potentially affected.
Severity and exploitation status, where available.
Recommended next steps.
Assigned engineering owner.

The objective is to turn a general security announcement into a project-specific action.

2. Project and Dependency Intelligence

The first automation tells us what is happening in the technology ecosystem. The second tells us whether those changes matter to our projects.

This requires a reliable inventory of the technologies and dependencies used across every repository.

Without that inventory, matching a newly announced vulnerability to an affected application becomes difficult.

What should the automation collect?

For each GitHub repository, maintain information such as:

Project and repository name.
Primary programming language.
Frameworks and libraries.
Package manager.
Direct dependencies.
Transitive dependencies.
Installed package versions.
Docker base images.
GitHub Actions and other build dependencies.
Infrastructure dependencies.
Project owner.
Production or staging environment.
Last dependency update.

Tools that can help:

Tool	Purpose
GitHub API	Discover repositories and project information
Dependabot	Identify vulnerable dependencies
Syft	Generate Software Bills of Materials
Trivy	Scan dependencies and container images
OSV-Scanner	Check open-source dependencies for known vulnerabilities
Renovate	Manage dependency updates

A Software Bill of Materials (SBOM) is a machine-readable inventory of the software components included in an application. It helps teams understand what their software contains and investigate whether a newly discovered vulnerability affects it.

How should the automation work?

Connect to the GitHub organization.
Discover repositories and their package manifests.
Extract dependency names and versions.
Identify direct and transitive dependencies.
Generate or collect SBOMs for production builds.
Store the results in a central project inventory.
Update the inventory whenever dependencies change.

Consider an organization maintaining these applications:

Project	Technology	Dependency
Real Estate CRM	Node.js	Next.js
AI Support	Python	LangChain
Voice AI	Node.js	Express
RAG Platform	Python	LiteLLM

If a vulnerability is discovered in a particular version of LiteLLM, the team can search the inventory and identify which projects need investigation.

One important detail: do not track only dependencies developers install directly. Transitive dependencies can introduce vulnerabilities too.

3. GitHub Pull Request Security Automation

Finding vulnerabilities after an application has been deployed is often more expensive than identifying them during development.

Security checks should therefore be part of the existing development workflow.

A practical starting point is to run automated security checks whenever a developer pushes code or opens a pull request.

What should be checked?

Security check	Suggested tools
Static application security testing (SAST)	Semgrep, CodeQL
Dependency vulnerability scanning (SCA)	Dependabot, OSV-Scanner
Secret detection	Gitleaks, GitHub Secret Scanning
Container scanning	Trivy
Infrastructure-as-Code scanning	Checkov
AI application security testing	Promptfoo

Example workflow:

Developer creates a pull request.

The CI pipeline runs the configured security checks.

Results are collected and evaluated against the team's security policies.

If a mandatory check fails, the pull request is blocked from merging until the issue is resolved or an authorized exception is approved.

The developer receives a notification with the finding and remediation guidance.

For example, imagine a developer accidentally commits an API key.

The automation should:

Detect the exposed credential.
Fail the relevant security check.
Notify the developer and repository owner.
Create a remediation issue where appropriate.
Guide the team through revoking or rotating the exposed credential.

Simply deleting the key in a subsequent commit is not enough. Once a credential has been exposed, it should be treated as compromised.

The goal is to make security checks part of the normal development process rather than a separate activity that happens only before release.

4. AI Security and Framework Announcement Monitoring

AI applications introduce dependencies and risks that evolve quickly.

A project may depend on an AI model provider, an agent framework, a vector database, an orchestration platform, or an external tool integration. Changes in any of these components can affect application security and reliability.

For this reason, AI-related announcements deserve dedicated monitoring.

What should be monitored?

OpenAI security and platform announcements.
Anthropic security updates.
Google AI announcements.
LangChain and LangGraph releases.
n8n security advisories.
LiteLLM updates.
Hugging Face security announcements.
Vector database advisories.
OWASP GenAI security updates.

Do not limit monitoring to vulnerabilities. Also track:

Breaking changes.
Deprecated APIs.
Model retirements.
Authentication changes.
New security features.
Changes to tool execution or agent capabilities.
Important changes in framework behaviour.

How should the automation work?

Monitor official release notes, advisories, and announcement feeds.
Identify the affected product or framework.
Classify the update as a vulnerability, breaking change, deprecation, or general announcement.
Check which projects use the affected technology.
Use AI to summarize the technical implications.
Notify the relevant engineering team if action is needed.

For example, an AI framework announces a breaking change to its tool-execution interface.

The automation identifies projects using that framework and informs their owners. The team can then review the change and plan any required updates.

An important boundary: AI can help classify announcements and recommend actions, but it should not independently update production dependencies or change live AI workflows. Those changes should follow the team's normal review and approval process.

5. Automated Project Health Monitoring

Security is not limited to newly announced vulnerabilities.

A project can become risky because its dependencies are outdated, its framework is approaching end of life, its cloud configuration has changed, or its deployment pipeline is no longer working correctly.

A scheduled project health workflow can help identify these issues before they become larger problems.

What should it monitor?

Area	What to check
Dependencies	Vulnerable or outdated packages
Frameworks	End-of-life versions and deprecated components
GitHub	Security alerts and repository configuration
CI/CD	Failed builds and deployment checks
Cloud	Configuration and access-control issues
SSL/TLS	Certificate expiry
APIs	Integration failures
AI services	Model and API deprecations
Databases	Version status and backup checks
Containers	Known image vulnerabilities
Repository hygiene	Unused dependencies and outdated workflows

How should the automation work?

Run scheduled checks for each registered project.

Collect the findings and categorize them by severity and operational impact.

Generate a project health report that identifies:

Issues requiring immediate attention.
Dependencies that need updates.
Security controls that are missing or failing.
Infrastructure or integration problems.
Recommended remediation actions.
Responsible project owners.

The report can be delivered weekly through Slack or email, with critical findings sent immediately rather than waiting for the weekly summary.

This gives engineering leads a consistent view of project health without requiring them to manually inspect every repository.

6. Security News and Industry Intelligence Digest

Not every security announcement affects a project directly. However, broader security developments can still help engineering teams understand emerging threats and adjust their practices.

A dedicated intelligence digest can collect important security news and make it easier for teams to stay informed.

Potential information sources:

The Hacker News.
BleepingComputer.
CISA.
CERT-In.
GitHub Security Advisories.
OWASP.
Microsoft Security Blog.
AWS Security Bulletins.
Google Cloud Security updates.
Official open-source release notes.

How should the automation work?

Collect new articles, advisories, and announcements.
Remove duplicate reports.
Categorize them by threat type and technology.
Identify the affected products and industries.
Use AI to create a concise technical summary.
Highlight whether the issue relates to technologies used by the organization.
Publish a daily or weekly digest.

A useful digest should not simply list headlines.

It should explain:

What happened?
Which technology or business area is affected?
Why does it matter to engineering teams?
Do any internal projects need investigation?
Is there an action the team should take?

This distinction matters. A long list of security headlines creates more reading work. A short, relevant digest helps engineers focus on what deserves attention.

Bringing These Automations Together

These six automations become more useful when they share information instead of operating independently.

For example, vulnerability monitoring becomes much more effective when it can access the dependency inventory. Similarly, project health monitoring can reuse findings from the pull request security checks.

A practical architecture could look like this:

Security Sources       GitHub Repositories
      |                        |
      v                        v
Technology &            Dependency Inventory
Vulnerability                  |
Monitoring                      |
      |                        |
      +-----------+------------+
                  |
                  v
       Central Project Inventory
                  |
                  v
       AI Classification & Analysis
                  |
                  v
        Risk and Impact Assessment
                  |
                  v
       Notifications & Action Tracking
          |          |          |
          v          v          v
        Slack      Email    GitHub Issues

n8n can coordinate scheduled workflows, API integrations, notifications, and data movement. GitHub provides repository and development-event information. Security scanners and vulnerability databases provide technical findings. AI can help classify, summarize, and explain those findings.

A central project inventory connects all of these components.

The most important part is that the system should understand which technologies each project uses. Without that context, it becomes another generic security news aggregator.

What Should You Implement First?

Trying to build all six automations at once would create unnecessary complexity.

I would approach implementation in phases.

Phase	Automation	Priority
1	GitHub repository and dependency inventory	Critical
2	Technology and vulnerability monitoring	Critical
3	GitHub pull request security checks	Critical
4	Security news and technology digest	High
5	AI and framework announcement monitoring	High
6	Automated project health monitoring	High

The first three provide the foundation.

You need to know what your projects use before you can reliably identify which vulnerabilities affect them. You also need security checks inside the development workflow to catch issues before they reach production.

Once these are in place, the remaining automations can expand monitoring and improve engineering visibility.

A Few Things to Keep in Mind

Automation can improve security visibility, but it does not replace engineering judgment.

A few principles should guide the implementation:

Validate vulnerability findings. Match package names and affected version ranges carefully. A vulnerability announcement does not automatically mean every project using that technology is affected.

Prioritize by actual risk. Consider severity, exploitability, exposure, environment, and business impact. A high-severity issue in an isolated development tool may require a different response from a remotely exploitable vulnerability in a production authentication service.

Keep humans involved in sensitive changes. AI can recommend dependency updates or remediation steps, but production changes should follow an approved engineering process.

Avoid alert fatigue. Deduplicate findings, group related alerts, and notify people based on project ownership and urgency.

Track remediation, not just detection. An alert is useful only if someone owns it, investigates it, and verifies that the issue has been resolved.

Final Thoughts

As engineering teams maintain more projects and adopt more technologies, manually tracking every vulnerability, dependency update, framework announcement, and security issue becomes increasingly difficult.

These six automations offer a practical way to improve that process.

The real value does not come from collecting more security information. It comes from connecting that information to the projects that matter, identifying the potential impact, and helping the responsible engineers take action.

Start with project visibility, dependency intelligence, and automated security checks in your development pipeline. Build the remaining workflows around those foundations.

The objective is straightforward: make security monitoring a continuous part of software engineering, rather than something teams remember to do only when an issue makes the news.

6 Security Practices Developers Should Prioritize in the Age of AI

Gaurav Talesara — Sat, 31 Oct 2026 17:43:22 +0000

Software development is changing rapidly.

AI coding assistants are helping developers write code faster. AI agents are beginning to interact with APIs, databases, cloud services, and internal tools. Modern applications are also becoming more connected through third-party integrations, open-source dependencies, and distributed infrastructure.

These changes create new opportunities, but they also expand the attack surface.

A vulnerability is no longer limited to a piece of application code. It can originate from a compromised dependency, an exposed developer token, an insecure API integration, an overprivileged AI agent, or a misconfigured cloud service.

The challenge is that many traditional security practices were designed for a different development environment.

Security can no longer be treated as a final checklist before deployment. It needs to be part of how we design, build, test, deploy, and operate software.

Here are six security practices developers should prioritize in 2026.

1. Treat AI-Assisted Development as a Security Concern

AI coding assistants can accelerate development, but generated code is not automatically secure code.

An AI assistant may suggest an implementation that works functionally while introducing security weaknesses. It might generate an unsafe database query, overlook input validation, expose sensitive information in logs, or recommend an outdated package.

The risk increases when developers accept generated code without understanding or reviewing it.

What developers should do:

Review AI-generated code with the same scrutiny as human-written code.
Validate input and encode output appropriately.
Check authorization rules, not just authentication.
Avoid sharing secrets, credentials, customer data, or proprietary information with AI tools unless the tool and its data-handling policies have been approved.
Run static analysis, dependency checks, and security tests on AI-assisted changes.
Understand the code before merging it.

AI should improve developer productivity, not replace engineering judgment.

The same principle applies to AI agents. If an agent can access internal tools, databases, APIs, or cloud resources, treat it as a security-sensitive identity. Give it narrowly scoped permissions, validate its inputs and outputs, and keep a record of the actions it performs.

2. Secure the Software Supply Chain

Modern applications depend on a large ecosystem of open-source libraries, frameworks, packages, build tools, and external services.

That ecosystem makes development faster, but it also creates supply-chain risk.

A vulnerability or compromise in a dependency can affect every application that uses it. Attackers may also target package registries, build pipelines, maintainers, or developer tooling to introduce malicious code into otherwise trusted software.

A package being popular or widely used does not automatically make it safe.

What developers should do:

Maintain an inventory of direct and transitive dependencies.
Scan dependencies for known vulnerabilities.
Remove packages that are no longer needed.
Pin dependency versions where appropriate and use lockfiles consistently.
Verify package sources and integrity.
Review dependency updates before introducing them into production.
Generate and maintain a Software Bill of Materials (SBOM) where appropriate.
Protect build pipelines and restrict who can publish or modify packages.

Tools such as dependency scanners and software composition analysis (SCA) solutions can help identify known risks.

The objective is not to avoid open source. It is to understand what your application depends on and maintain control over that dependency chain.

3. Treat Third-Party Integrations as Part of Your Attack Surface

Most modern applications do not operate in isolation.

They integrate with payment providers, authentication services, analytics platforms, messaging systems, cloud services, customer-management tools, and many other external vendors.

Every integration introduces a trust relationship.

Even if your application is secure, a compromised vendor, insecure integration, or excessive access permission can create a path into your environment.

This is particularly important for organizations that rely on multiple SaaS platforms and external service providers.

What developers and engineering teams should do:

Evaluate the security requirements of third-party services before integrating them.
Request only the permissions an integration actually needs.
Store and rotate integration credentials securely.
Validate incoming webhook signatures.
Apply authentication and authorization to service-to-service communication.
Set timeouts, rate limits, and sensible failure-handling behavior.
Review vendor access periodically and remove integrations that are no longer required.

Do not assume that a third-party service is safe simply because it is widely adopted.

Every integration should have a clear owner, a defined purpose, and an appropriate level of access.

4. Protect Credentials, Tokens, and Privileged Identities

Credentials are among the most valuable assets in a modern software environment.

An exposed cloud access key, database password, CI/CD token, developer credential, or service-account secret can give an attacker access far beyond a single application.

The problem is not limited to production credentials. Developer machines, build systems, deployment pipelines, and internal automation tools can also hold powerful access.

As applications become more distributed and AI agents gain access to tools, identity management becomes even more important.

What developers should do:

Never hardcode credentials in source code.
Use approved secret-management systems.
Avoid storing secrets in configuration files that are committed to version control.
Rotate credentials regularly and immediately after suspected exposure.
Use short-lived credentials where supported.
Apply least-privilege access to users, services, pipelines, and agents.
Enable multi-factor authentication for privileged accounts.
Audit access to sensitive resources.
Remove unused accounts, tokens, and service credentials.

A useful question to ask during a security review is:

If this credential were compromised today, what could an attacker access?

If the answer includes more systems or data than necessary, the permission model needs attention.

5. Build Application Security Into the Development Lifecycle

Application vulnerabilities remain a major source of security incidents.

Common weaknesses include broken access control, injection vulnerabilities, insecure authentication, security misconfiguration, and improper handling of sensitive data.

These problems are often preventable when security is considered during design and development rather than after an application has been released.

For example, an application may authenticate a user correctly but fail to verify whether that user is authorized to access a particular record.

That is an authorization failure, and it can expose customer data even when login functionality appears to work as expected.

What developers should do:

Perform threat modeling for important application features.
Validate and sanitize untrusted input.
Use parameterized queries instead of constructing SQL from user input.
Enforce authorization checks on the server side.
Apply secure session management.
Protect sensitive data in transit and at rest.
Configure security headers and safe defaults.
Review error handling to avoid exposing internal details.
Include security testing in code reviews and CI/CD pipelines.

The OWASP Top 10 is a useful starting point for understanding common web application security risks.

Security testing should also cover APIs, mobile applications, background jobs, and internal services—not just the user-facing web interface.

The earlier a vulnerability is identified, the easier it is generally to address.

6. Move From One-Time Security Checks to Continuous Monitoring and Response

Passing a security scan before deployment does not mean an application will remain secure.

New vulnerabilities are discovered. Dependencies change. Infrastructure configuration drifts. Credentials may be exposed. Attack patterns evolve.

Security is an ongoing operational responsibility.

Organizations need to know not only whether vulnerabilities exist, but also whether suspicious activity is happening in their environments and how quickly they can respond.

What engineering teams should do:

Continuously monitor application and infrastructure logs.
Collect security-relevant events from applications, APIs, identity systems, and cloud services.
Configure alerts for unusual authentication and access patterns.
Monitor privileged operations and sensitive data access.
Reassess vulnerabilities as new information becomes available.
Establish an incident-response process with clear ownership.
Maintain tested backups and recovery procedures.
Run incident-response exercises so teams understand their responsibilities.

Monitoring is useful only when teams know how to act on what they discover.

A security alert without an owner, escalation path, or response procedure is unlikely to provide much protection.

The goal is to reduce the time between detecting a potential compromise and taking meaningful action.

Making Security Part of Everyday Engineering

These six practices are connected.

AI-assisted development affects code quality and review. Dependencies and third-party integrations expand the trust boundary. Credentials determine what users, services, and agents can access. Application security protects data and business logic. Continuous monitoring helps identify problems that make it through preventive controls.

No single tool can address all of these risks.

A practical approach is to integrate security into the existing engineering workflow:

Engineering stage	Security practice
Architecture and design	Threat modeling and secure design reviews
Development	Secure coding and AI-generated code review
Dependency management	Vulnerability scanning and dependency verification
Code review	Static analysis and security-focused reviews
CI/CD	Automated security checks and protected pipelines
Deployment	Secure configuration and least-privilege access
Production	Continuous monitoring and incident response

Start with the risks that matter most to your application and business. Assign ownership, automate repeatable checks, and make security findings part of normal engineering discussions.

The objective is not to slow down development with unnecessary controls. It is to make secure development the natural way of working.

Final Thoughts

AI is accelerating software development, but it is also changing how software can be attacked.

More integrations, more dependencies, more automation, and more powerful identities mean that security needs to evolve alongside engineering practices.

Developers do not need to become security specialists overnight. But they do need to understand the risks introduced by the systems they build and the tools they use.

Build with security in mind. Review what you trust. Limit what systems can access. Monitor what happens in production. And make sure your team knows how to respond when something goes wrong.

The best security strategy is not a checklist completed before release. It is an engineering discipline practiced throughout the software lifecycle.

Further reading:

Tags: #cybersecurity #applicationsecurity #devsecops #ai #softwaredevelopment

Your AI Demo Works. Will Your LLM Architecture Survive 15 Seconds?

Gaurav Talesara — Sat, 10 Oct 2026 06:04:30 +0000

Imagine you are building an AI customer support assistant for an e-commerce company.

A customer opens the chat and asks:

"Where is my order? It was supposed to arrive yesterday."

Your application retrieves the order status, checks the shipping information, sends the relevant details to an LLM, and returns a helpful answer.

During development, everything works.

Then a major sale begins. Thousands of customers start using the assistant. The LLM provider becomes slower, some requests receive HTTP 429 responses, and a few requests time out.

Now customers are waiting for answers about their orders. Support tickets begin to pile up.

The model is still capable of generating an answer. Your application is struggling to deliver it reliably.

This is where production LLM architecture matters.

1. Understand the business workflow first

Let's define a simple target: the application should return a response within 15 seconds.

That is an illustrative service objective for this example, not a universal recommendation. The appropriate deadline depends on the user experience and business requirements.

A typical request might follow this path:

Customer
   |
   v
Support Chat API
   |
   v
Authentication + Order Lookup
   |
   v
LLM Gateway
   |
   +------> Primary LLM Provider
   |
   +------> Approved Fallback
   |
   v
Validate Answer
   |
   v
Return to Customer

Notice that the LLM is not responsible for looking up the order by itself.

Your backend should retrieve authoritative order and shipping information from the relevant systems. The model can explain those facts in natural language, but it should not invent an order status or guess a delivery date.

This separation is the first important design decision.

2. What can go wrong within 15 seconds?

Suppose the request takes this much time:

Operation	Time
Authenticate customer	200 ms
Retrieve order and shipment	800 ms
Build prompt and context	200 ms
Wait for LLM response	8 seconds
Validate and return answer	300 ms
Total	9.5 seconds

This request succeeds within the 15-second budget.

Now imagine the LLM takes 12 seconds and returns a rate-limit error. Your application retries, and the second request takes another 8 seconds.

The customer has now waited more than 20 seconds.

The lesson is simple: setting a timeout on each API call is not the same as setting a deadline for the entire business operation.

3. Enforce one end-to-end deadline

Start with a deadline for the complete request.

const REQUEST_BUDGET_MS = 15_000;

function createDeadline() {
  return Date.now() + REQUEST_BUDGET_MS;
}

function remainingTimeMs(deadline: number) {
  return Math.max(0, deadline - Date.now());
}

When the order lookup finishes, calculate the remaining time before calling the model.

const deadline = createDeadline();

const order = await getOrderForCustomer(
  customerId,
  orderId
);

const remaining = remainingTimeMs(deadline);

if (remaining <= 0) {
  throw new Error("Request deadline exceeded");
}

const answer = await generateSupportAnswer({
  order,
  timeoutMs: remaining
});

The generateSupportAnswer function must pass timeoutMs to the actual SDK or HTTP client. The snippet illustrates the design, not a complete implementation.

In a real application, you should also account for authentication, database timeouts, cancellation, and any other work performed before or after the model call.

Important: a timeout means your application stopped waiting. It does not always mean the provider stopped processing the request.

That distinction becomes especially important when a request can trigger a real-world action.

4. Handle rate limits without creating a retry storm

During the sale, your provider starts returning HTTP 429 responses.

A tempting implementation is to retry every failed request immediately.

That can make the situation worse. If 1,000 requests fail and each retries three times, the provider could receive up to 3,000 additional attempts from that batch.

Instead, use a bounded retry policy.

function retryDelayMs(attempt: number): number {
  const baseMs = 250;
  const maxMs = 2_000;

  const cap = Math.min(
    maxMs,
    baseMs * 2 ** attempt
  );

  return Math.random() * cap;
}

This function calculates a randomized backoff delay. It is only one part of a retry policy.

A production implementation should:

Classify the error before retrying.
Respect the provider's Retry-After guidance when available.
Limit the number of attempts.
Check the remaining deadline before sleeping or retrying.
Avoid retrying invalid requests or authentication failures.

For example, if only 700 milliseconds remain and the next retry would require a longer delay plus a model response, the application should stop trying.

It is better to return a controlled response than to keep a customer waiting for a deadline the application has already missed.

5. Design a useful fallback

What should happen when the primary LLM provider is unavailable?

For this support assistant, there are two reasonable paths.

Path A: Use an approved fallback model.

This can work when the alternative model has been evaluated for the task and meets your latency, quality, privacy, and cost requirements.

Path B: Return a deterministic response.

If no suitable model is available, the backend can return a simple message based on the verified order data:

"Your order is currently marked as in transit. We cannot generate a detailed explanation right now. You can check the latest tracking status here."

This response may be less conversational, but it is based on real data and still helps the customer.

The application should never fabricate a delivery date just to make the conversation feel complete.

Fallbacks should be designed around the business outcome, not around the assumption that another model will always work.

6. Keep order access outside the model's authority

This is a critical security boundary.

Suppose a customer asks:

"Show me the status of order #84521."

The backend must verify that the authenticated customer is authorized to access that order before including any order information in the model context.

A safe request flow is:

Authenticated Customer
         |
         v
Authorize Order Access
         |
         v
Fetch Verified Order Data
         |
         v
Send Minimum Required Context
         |
         v
Generate Answer
         |
         v
Validate and Return

Do not let the model decide whether the customer has permission to view an order. That decision belongs to your application's authorization layer.

Likewise, if the assistant can cancel orders or issue refunds, those actions should use explicit backend authorization, business rules, and appropriate confirmation steps. The LLM can help interpret the customer's intent, but it should not bypass the controls that protect the transaction.

7. Control the cost of each support interaction

During a large sale, request volume can increase sharply.

Suppose the assistant sends the customer's entire conversation history and a large amount of order documentation with every request. Token usage can grow even when the actual question is simple.

For "Where is my order?", the model probably needs only the relevant order status, tracking information, and a small amount of conversational context.

Set limits for:

Input size and conversation history.
Retrieved context.
Maximum generated tokens.
Requests per user or tenant.
Overall spending and usage alerts.

Track the cost per successfully resolved support interaction, not just the average cost of an LLM request.

A request that is cheap but repeatedly fails may still increase total support costs if customers abandon the assistant and contact a human agent.

8. Monitor the customer outcome

A dashboard showing successful HTTP responses is not enough.

Imagine that the model returns HTTP 200 for 99% of requests, but many answers are too slow or fail to address the customer's question.

The infrastructure may appear healthy while the support experience is deteriorating.

Track the full request lifecycle.

Metric	Why it matters
End-to-end latency	Measures the customer's wait
Model latency	Shows how much time the provider consumes
HTTP 429 rate	Identifies rate-limit pressure
Retry count	Helps reveal repeated recovery attempts
Fallback rate	Shows how often the primary path fails
Token usage and cost	Tracks spending per interaction
Answer validation failures	Finds malformed or unusable responses
Resolution rate	Measures whether the assistant actually helps

For answer quality, evaluate whether the response matches the verified order data. For business impact, measure whether the customer receives a useful answer or still needs a support agent.

Avoid logging sensitive order details or personal information unnecessarily. Use request IDs for tracing, and apply appropriate access controls and retention rules to logs.

9. Test failure before the sale begins

Do not test only the happy path.

Run controlled tests where:

The LLM takes 12 seconds to respond.
The provider returns HTTP 429.
The provider returns HTTP 503.
The request exceeds its deadline.
The model returns malformed JSON or unsupported claims.
The primary provider becomes unavailable.

Check that the application stops retrying when the budget is exhausted, falls back only when allowed, and never exposes another customer's order information.

If the assistant can initiate transactions, test the case where the provider times out after the backend may already have completed an action. A retry must not accidentally cancel an order twice or create duplicate refunds.

10. Production-readiness checklist

Before releasing this assistant to real customers, I would want these controls in place:

[ ] A deadline for the complete request.
[ ] Bounded retries with appropriate backoff.
[ ] Explicit handling for rate limits and provider outages.
[ ] A tested fallback or deterministic response.
[ ] Authorization checks before retrieving customer data.
[ ] Limits on context size, generated tokens, and spending.
[ ] Monitoring for latency, failures, retries, cost, and answer quality.
[ ] Tests for timeouts and duplicate side effects.
[ ] A clear response when the AI service is unavailable.

You do not need to build a complex microservice architecture immediately. A well-structured module inside your existing backend can enforce many of these controls. Extract it into a separate gateway when operational needs justify the added complexity.

Final thought

The real test of an AI support assistant is not whether it can answer a question when everything is working.

It is whether the business can still serve the customer when the model is slow, the provider is rate-limiting requests, or the request budget is nearly exhausted.

The LLM generates the language. Your architecture determines whether the feature is reliable enough to use.

If your primary LLM provider became unavailable during your busiest hour, would your application still be able to help the customer?

3 Redis Design Failures You Should Avoid Before They Become Production Incidents

Gaurav Talesara — Wed, 30 Sep 2026 17:01:48 +0000

Caching is usually introduced for one simple reason:

Take pressure off the database and make reads faster.

The architecture often looks straightforward:

Client
  |
  v
Application
  |
  +----> Redis
  |
  +----> Database

A request checks Redis first.

If the value exists, return it.

If it doesn't, query the database, put the result into Redis, and return it.

Simple.

Until the system meets real production traffic.

That's where caching stops being just a performance optimization and becomes a system-design problem.

Let's walk through how a seemingly simple cache evolves through three stages—and the failure modes that appear at each stage.

1. Stale Data

The first version of a cache is usually cache-aside.

The application owns the caching logic.

The flow is:

Request
   |
   v
Check Redis
   |
   +---- HIT ----> Return cached value
   |
   +---- MISS ---> Query database
                     |
                     v
                 Store in Redis
                     |
                     v
                  Return

For read-heavy workloads, this can dramatically reduce database traffic.

Suppose we have a user profile:

{
  "id": 42,
  "name": "Gaurav",
  "email": "gaurav@example.com"
}

The application stores it under:

user:42

Everything works.

Until the user changes their name.

The database is updated:

Gaurav
    ↓
Gaurav Talesara

But Redis still contains:

Gaurav

Now you have two versions of reality.

Database → Gaurav Talesara
Redis    → Gaurav

The database is correct.

The application can still return the wrong answer.

This is stale data.

Why this matters

Caching introduces another copy of your data.

The moment you do that, you have to answer:

When does the cached copy stop being valid?

That's the beginning of cache consistency design.

2. TTL Helps—but Doesn't Solve Everything

A common next step is adding a TTL (Time To Live).

For example:

user:42
TTL = 5 minutes

After five minutes, Redis considers the entry expired.

The next request becomes a cache miss:

Redis MISS
    |
    v
Database
    |
    v
Fresh value
    |
    v
Redis

This gives you eventual freshness without requiring every application write to explicitly update the cache.

But TTL introduces a trade-off.

Longer TTL

You get:

fewer database reads
better cache hit rate

But potentially:

older data
longer periods of inconsistency

Shorter TTL

You get:

fresher data

But potentially:

more cache misses
more database traffic
lower cache efficiency

So TTL isn't really a "freshness setting."

It's a consistency vs. performance trade-off.

And some data simply can't wait five minutes.

If a user changes their email address, waiting for TTL expiration may be unacceptable.

That's where explicit invalidation comes in.

3. Invalidation on Write

Instead of waiting for the cache to expire, the application can invalidate the relevant key when the database changes.

For example:

UPDATE users
SET name = 'Gaurav Talesara'
WHERE id = 42;

Then:

DELETE user:42 FROM Redis

The next read becomes:

Redis MISS
    |
    v
Database
    |
    v
Fresh data
    |
    v
Redis

This gives you a much fresher cache.

But now the application has to maintain the relationship between:

database writes ↔ cache invalidation

And that creates its own failure scenarios.

What if the database update succeeds but cache invalidation fails?

You can end up with:

Database → NEW value
Redis    → OLD value

What if the invalidation happens first but the database transaction fails?

Now the cache is empty even though the database still contains the old value.

There is no universal answer here.

The right approach depends on the consistency requirements of the application.

But there is another problem that has nothing to do with stale data.

And this one can take down the database.

4. Cache Stampede

Imagine you have a popular product page:

product:123

Normally it receives thousands of requests.

Because the data is cached, most of those requests never reach the database.

That's exactly what we wanted.

Now the cache entry expires.

At almost the same moment, thousands of requests arrive.

They all see:

CACHE MISS

And they all do this:

Request 1 → DB
Request 2 → DB
Request 3 → DB
Request 4 → DB
...
Request 5000 → DB

The database was protected by Redis a moment ago.

Now thousands of requests are hitting it simultaneously.

This is a cache stampede.

Another name you'll often see for this pattern is thundering herd.

The important point is that the problem isn't simply that the cache missed.

The problem is that many requests independently react to the same cache miss.

5. Defense #1: Single Flight

One way to solve this is to allow only one request to refresh a missing key.

Suppose 5,000 requests miss:

             Redis MISS
                  |
        +---------+---------+
        |                   |
    Request 1            Requests 2-5000
        |                   |
      DB query              WAIT
        |                   |
        +--------+----------+
                 |
          Fresh result
                 |
             Redis SET
                 |
        Everyone gets result

Request #1 becomes responsible for fetching the data.

The other requests wait for the result instead of independently hitting the database.

This pattern is often called single-flight or request coalescing.

The key idea is:

One cache miss should not become 5,000 database queries.

This is particularly useful when a small number of keys receive a very large amount of traffic.

6. Defense #2: TTL Jitter

There is another failure pattern that can happen even when you aren't dealing with one single hot key.

Imagine millions of cache entries are created at roughly the same time.

If every entry gets exactly:

TTL = 300 seconds

then many of them can expire around the same time.

That can create a synchronized wave of cache misses.

Instead, you can introduce jitter.

Rather than:

TTL = 300 seconds

you might use something conceptually like:

TTL = 300 + random(0, 60)

Now expiration is spread across time.

Instead of:

                    ↓
              EVERYTHING EXPIRES
                    ↓
             DATABASE SPIKE

you get something more like:

      ↓      ↓          ↓    ↓       ↓
   expire  expire     expire expire  expire

The database sees a more distributed load pattern.

TTL jitter doesn't eliminate cache misses.

It reduces the probability of synchronized expiration becoming a traffic spike.

7. Defense #3: Hot-Key Replication

Now consider a different situation.

One key becomes extremely popular.

For example:

product:123

might represent a product that suddenly goes viral.

You can have thousands or millions of requests for essentially the same piece of data.

Even if Redis is distributed, concentrating a huge amount of traffic around one key can create a bottleneck.

This is a hot key.

One approach is to replicate frequently accessed data across multiple cache nodes so requests don't all depend on the same location.

Conceptually:

                  product:123
                       |
            +----------+----------+
            |          |          |
            v          v          v
         Redis-1    Redis-2    Redis-3

Now requests can be distributed rather than concentrating entirely on one cache location.

But this introduces another design question:

How do you keep replicated hot data consistent?

Again, the optimization introduces another system-design trade-off.

8. The Pattern Behind All Three Failures

This is the part that's easy to miss.

You start with:

Database

You add Redis:

Application
     |
   Redis
     |
 Database

Now you have better read performance.

Then production reveals stale data.

So you add:

TTL + Invalidation

Then production traffic reveals cache stampedes.

So you add:

Single Flight
TTL Jitter
Request Coalescing

Then traffic concentration reveals hot keys.

So you consider:

Replication
Load Distribution
Failure Isolation

The architecture keeps evolving.

That's because caching isn't simply:

"Put Redis in front of the database."

You're introducing another distributed component with its own:

consistency behavior
expiration behavior
concurrency
failure modes
traffic patterns
recovery requirements

9. What to Design Before Adding Redis

Before introducing a cache, I'd want the team to answer at least these questions.

1. What can be stale?

Not every piece of data has the same consistency requirement.

Ask:

Can this value be 1 second old?
10 seconds?
5 minutes?
Never?

That answer should influence the caching strategy.

2. What invalidates the data?

Define the relationship between writes and cache entries.

For example:

User updated
    ↓
Database committed
    ↓
Invalidate user:42

Don't leave invalidation as an afterthought.

3. What happens when the key expires?

Ask:

1 request misses?
100?
10,000?
1,000,000?

If the answer is "they all query the database," you may have a stampede waiting to happen.

4. What happens when one key becomes extremely popular?

Measure key access patterns.

A distributed cache doesn't automatically mean distributed traffic.

You can still have:

99% of requests → 1 key

5. What happens when Redis is unavailable?

This is one of the most important questions.

If Redis disappears, does the application:

Fail closed?
      or
Fall back to database?

If everything falls back to the database simultaneously, Redis failure can become a database incident.

Your fallback path needs capacity planning too.

10. The Real Lesson

Caching is one of the easiest performance optimizations to add to an architecture.

It's also easy to underestimate what you're introducing.

The progression often looks like this:

Cache
  ↓
Stale Data
  ↓
TTL / Invalidation
  ↓
Cache Stampede
  ↓
Single Flight / Jitter
  ↓
Hot Keys
  ↓
Replication / Distribution

Each step solves a real problem.

And each step introduces another design decision.

That's the real lesson:

A cache doesn't remove complexity. It moves complexity somewhere else in the system.

So before adding Redis, don't only ask:

"How much faster will this make our application?"

Ask:

"What happens when the cache is stale?"

"What happens when the key expires?"

"What happens when 10,000 requests miss simultaneously?"

"What happens when one key receives 100× normal traffic?"

"What happens when Redis itself fails?"

Those questions are where caching stops being a feature and becomes system design.

I Tried to Break a $12 Server. It Took 5,000+ Concurrent Users.

Gaurav Talesara — Sat, 26 Sep 2026 05:15:45 +0000

There is a common assumption in backend engineering:

If you expect traffic to grow, you need to start thinking about scaling infrastructure early.

More servers.
More CPU.
A dedicated database.
Redis.
Load balancers.
Containers.
Kubernetes.

Sometimes you do.

But sometimes the application isn't actually asking for more infrastructure yet.

I wanted to see how far a very small server could go before reaching a real bottleneck.

So I rented a server for $12/month.

The machine had:

1 CPU
2 GB RAM
Nginx
Node.js
PostgreSQL
No dedicated database server
No Redis

Then I started increasing the load until something broke.

The interesting part wasn't how cheap the server was.

It was what became the bottleneck first.

The experiment

The goal was simple:

Find the point where the application starts showing meaningful performance degradation, identify the bottleneck, and improve it without upgrading the server.

I used a small API application running on a single machine.

The application had a database-backed feed/workflow-style endpoint that required reading data from PostgreSQL and generating a response for the client.

The database was running on the same machine as the application.

There was no separate cache layer.

No Redis.

No horizontally scaled application servers.

Just:

Internet
   |
 Nginx
   |
 Node.js
   |
PostgreSQL

The server cost about $12/month.

That was the entire point.

I wanted to see what the application could actually handle before adding infrastructure.

Making the test more realistic

A simple benchmark that repeatedly calls one endpoint isn't particularly interesting.

Real users don't behave like that.

They open something.

They read.

They perform an action.

They come back.

They refresh.

They request more data.

So I used k6 to create virtual users that followed a more realistic pattern.

The virtual users would:

Request the feed/workflow data.
Perform an occasional write/action.
Request the data again.
Repeat the cycle.

I also populated the database with a meaningful amount of data rather than testing against an empty database.

The load generator itself ran from a second VM so that the machine being tested wasn't also responsible for generating all the traffic.

The goal was to put pressure on the application server while keeping the test environment simple.

Starting with a small load

I didn't jump directly to thousands of users.

The load increased progressively:

This matters because performance problems aren't always obvious when you only look at average latency.

At low traffic, everything looked healthy.

As the number of virtual users increased, the system continued to respond normally.

Then something started changing around the higher loads.

2,000 users: the first warning sign

At around 2,000 virtual users, the average response time still looked reasonable.

But the tail latency started moving.

This is where metrics like p95 and p99 become much more useful than simply looking at average latency.

Imagine these two systems:

System A
Average: 100 ms
p95:     150 ms
p99:     200 ms

versus:

System B
Average: 100 ms
p95:     700 ms
p99:     2,000 ms

Both have the same average.

But the user experience is very different.

The second system has a tail-latency problem.

That was the first signal that the server was approaching a limit.

3,000 users: the test failed

I increased the load again.

At approximately 3,000 virtual users, the experiment crossed the failure criteria.

That gave me an important data point:

The server wasn't simply getting "a little slower."

There was a point where additional concurrency started pushing the system beyond the acceptable performance envelope.

So instead of continuing to increase the load blindly, I backed down.

I wanted to understand the bottleneck first.

2,500 concurrent users

At around 2,500 concurrent users, the system handled approximately:

~232 requests/sec
~288 ms p95 latency

Those numbers were interesting.

But the most interesting information came from the resource utilization.

The server wasn't running out of memory.

RAM usage was still under 1 GB.

PostgreSQL was also behaving normally.

The CPU, however, was sitting around 90%.

That changed the direction of the investigation.

The first instinct could have been:

"The server is too small. Buy a bigger one."

But the measurements were telling a different story.

Finding the actual bottleneck

The resource picture looked roughly like this:

RAM       < 1 GB
Postgres  Healthy
CPU       ~90%

The CPU was the constraint.

This is an important distinction.

If RAM had been nearly exhausted, adding memory would have been a reasonable direction.

If PostgreSQL had been saturated, database optimization would have been the next area to investigate.

But neither was the immediate problem.

The application was spending too much CPU doing work for requests.

So I asked a different question:

How much of this work actually needs to happen for every request?

That led directly to caching.

Optimization #1: Cache the response in Node.js

The feed/workflow response wasn't changing every millisecond.

Yet the application was repeatedly doing the same expensive work for requests that could safely receive a recently generated response.

So I introduced a very simple cache.

The idea was:

Request
   |
   v
Is cached response available?
   |
  Yes ----> Return cached response
   |
  No
   |
   v
Query database
   |
   v
Build response
   |
   v
Cache for 1 second
   |
   v
Return response

The important detail was the TTL:

1 second.

This wasn't an attempt to build a complicated distributed caching architecture.

It was simply:

If another request arrives immediately after this one, don't repeat the same work unnecessarily.

That small change had a significant effect.

The system moved from roughly:

2,500 concurrent users

to:

4,000 concurrent users

before reaching the same kind of limitation.

And I hadn't changed the server.

No additional CPU.

No additional RAM.

No Redis cluster.

No second application server.

The workload had simply changed.

Why did a 1-second cache help so much?

Caching works because computation has a cost.

Consider a simplified request:

Request
   ↓
Node.js
   ↓
PostgreSQL query
   ↓
Process data
   ↓
Serialize response
   ↓
Send response

If 100 requests arrive close together and they all need essentially the same data, doing the complete operation 100 times may be unnecessary.

With a short-lived cache:

Request 1
   ↓
Generate response
   ↓
Cache

Request 2 ─┐
Request 3 ─┤
Request 4 ─┤──> Cached response
Request 5 ─┘

The expensive work can be shared across multiple requests.

That reduces CPU work.

It can also reduce database work.

And because the response is already available, the application has less work to perform per request.

The important point is that caching didn't make the CPU faster.

It reduced how much CPU work was required.

Optimization #2: Move the cache closer to the edge

The next question was:

Why should Node.js be responsible for serving a response that Nginx can serve directly?

The architecture originally looked like:

Client
  |
 Nginx
  |
 Node.js
  |
PostgreSQL

After introducing application-level caching:

Client
  |
 Nginx
  |
 Node.js
  |
Cache
  |
PostgreSQL

That was already better.

But there was another opportunity.

If Nginx could cache the response, a cache hit wouldn't need to travel through the Node.js application at all.

So the architecture became conceptually:

Client
  |
 Nginx
  |
  +---- Cache hit ----> Response
  |
  +---- Cache miss ---> Node.js
                           |
                       PostgreSQL

Now the request could potentially be handled before reaching the application layer.

This is an important optimization because every request that Nginx can satisfy is a request that doesn't consume Node.js CPU.

The result

After moving the caching responsibility to Nginx, the system pushed past 5,000 concurrent users on the same $12/month server.

The progression was approximately:

~2,500 users
     |
     | 1-second application cache
     v
~4,000 users
     |
     | Move cache to Nginx
     v
5,000+ users

The infrastructure didn't become more powerful.

The application simply stopped doing unnecessary work for every request.

That distinction is the main lesson from the experiment.

Concurrent users are not total users

One important clarification:

5,000 concurrent users does not mean the server can only support 5,000 users total.

Concurrency and total user count are different measurements.

If an application has 100,000 registered users but only 500 are actively making requests at a given moment, the server is dealing with roughly 500 concurrent users, not 100,000.

That's why saying:

"This server supports 5,000 users"

would be misleading.

The accurate statement from this experiment is:

The setup was able to push past 5,000 concurrent virtual users under this particular load-test workload.

That doesn't automatically translate into a production capacity number.

Real applications have different request patterns, payload sizes, database queries, background jobs, connection behavior, and traffic distributions.

Benchmarks are useful when we understand exactly what was benchmarked.

What I would not conclude from this experiment

This experiment doesn't prove that everyone should run production systems on a $12 server.

It doesn't.

A production system may need:

High availability
Backups
Replication
Monitoring
Failover
Security controls
Multiple application instances
Database replicas
Disaster recovery
Horizontal scaling
Background workers
Rate limiting

Those requirements are independent of whether a small server can handle a particular workload.

The experiment answers a narrower question:

How much performance can I get from simple infrastructure before infrastructure itself becomes the problem?

In this case, quite a lot.

The bigger lesson: measure before scaling

One of the easiest mistakes in backend engineering is solving a capacity problem before identifying the actual constraint.

You see latency increasing.

You add a bigger server.

But what if the application is wasting CPU?

You see database queries getting slower.

You add a larger database instance.

But what if the same expensive query is being executed thousands of times unnecessarily?

You see more traffic.

You add Redis.

But what if a simple Nginx cache would have handled the workload?

Infrastructure can absolutely solve performance problems.

But infrastructure should be guided by measurements.

In this experiment, the sequence was:

Load test
   ↓
Observe latency
   ↓
Measure resources
   ↓
Find CPU bottleneck
   ↓
Reduce repeated work
   ↓
Test again
   ↓
Find the next limit

That loop is more valuable than any particular server size.

A simple scaling mindset

When an application starts struggling, I like thinking about the problem in this order:

1. What exactly is getting slower?

Don't stop at:

"The API is slow."

Look at:

Average latency
p95
p99
Requests/sec
Error rate
CPU
Memory
Database utilization
Connection counts
Network
Disk I/O

2. What resource is actually saturated?

Find the constraint.

In this experiment:

RAM     → fine
Postgres → fine
CPU     → ~90%

That pointed the investigation toward application work.

3. Can the work be eliminated?

Before making hardware bigger, ask:

Are we calculating the same thing repeatedly?
Can the result be cached?
Can a query be avoided?
Can data be precomputed?
Can work move to a cheaper layer?
Can Nginx/CDN handle something before it reaches the application?

4. Test the change

Don't assume the optimization worked.

Run the same workload again.

Compare the metrics.

5. Scale only when necessary

Eventually, the $12 server will hit another limit.

That's completely fine.

Scaling isn't the failure.

Scaling without understanding why you're scaling is the expensive part.

What surprised me most

The surprising part wasn't that a cheap server handled thousands of concurrent virtual users.

It was how quickly the result changed after removing unnecessary work.

At roughly 2,500 concurrent users, CPU was close to its limit.

A one-second cache increased the tested capacity to roughly 4,000.

Moving that cache to Nginx pushed the test past 5,000.

Same machine.

Same CPU.

Same RAM.

The biggest change was the amount of work the application had to perform.

That's a useful reminder for any backend system:

Before adding capacity, find out whether you can remove the work creating the demand.

Sometimes the cheapest performance optimization isn't a bigger server.

It's one less thing your server has to do.

Jev: AI for Decisions, Not Just Generation

Gaurav Talesara — Thu, 24 Sep 2026 18:05:32 +0000

Most AI applications today are built around generation.

Give a model some context → ask a question → get text back.

But many application workflows don't actually need generated text.

They need a decision.

Should this support ticket go to billing or technical support?

Is this lead worth prioritizing?

Should an AI agent retry a failed action?

Does this content need human review?

Which tool should an agent call next?

This is where Jev gets interesting.

Instead of treating every AI problem as a text-generation problem, Jev is designed around decision-making.

You provide the current state and typed questions, and the model can return structured decisions along with probabilities and confidence.

A simplified flow looks like:

Application State
       ↓
      Jev
       ↓
Decision + Probability + Confidence
       ↓
Application Action

For example:

{
  "route": "billing",
  "probability": 0.92,
  "confidence": "high"
}

Your application can then use that output directly.

Where this could be useful

Support automation
→ classify and route incoming tickets

Sales systems
→ score and prioritize leads

AI agents
→ decide which tool or action should happen next

Content pipelines
→ identify what requires human review

Risk workflows
→ evaluate a situation before allowing an automated action

Data processing
→ classify and triage unstructured inputs

The interesting architectural idea isn't simply adding another AI model.

It's separating generation from decision-making.

A general-purpose LLM is great when the output needs to be language.

But when the application needs something like:

route = billing
retry = false
risk = medium
next_action = create_ticket

a decision-oriented model can fit the workflow much more naturally.

I'm interested to see where this pattern goes as AI applications move from chat interfaces toward systems that continuously make decisions and take actions.

Voice AI can sound human and still fail at sales. The real problem is often conversation logic: understanding customer state, adapting questions, and knowing when to hand off to a human. I wrote about the architecture behind that.

Gaurav Talesara — Sun, 13 Sep 2026 12:01:28 +0000

Gaurav Talesara

Sep 13

Voice AI Doesn't Have a Voice Problem. It Has a Conversation Problem.

#ai #automation #architecture #performance

5 min read

Voice AI Doesn't Have a Voice Problem. It Has a Conversation Problem.

Gaurav Talesara — Sun, 13 Sep 2026 11:55:18 +0000

Voice AI can sound completely human and still be terrible at sales.

The voice can be natural, the response time can be fast, and the agent can handle thousands of calls. It can qualify leads, answer questions, schedule follow-ups, and update the CRM.

And the sales results can still be disappointing.

The problem is that sounding human and having a good sales conversation are two different engineering problems.

A customer rarely follows a predefined script. They interrupt, change direction, raise objections, ask their own questions, or reveal the most important information halfway through the conversation.

That is where many Voice AI sales systems start to struggle.

The problem with linear sales conversations

A typical sales agent follows a simple flow: introduce the company, ask qualification questions, explain the product, handle objections, and try to book a meeting.

It looks reasonable on a whiteboard.

Real customers don't behave that way.

Two customers can give the same answer but need completely different follow-ups. One may already be comparing vendors and ready to buy. Another may only be researching because a problem recently appeared.

A rigid workflow sees the same answer and moves to the next predefined question.

A good salesperson thinks about what the answer actually means.

That difference is the core problem.

Model the customer, not just the lead

Most sales systems already have plenty of customer data: lead score, company information, previous interactions, campaign source, product interest, and CRM history.

That information is useful, but the agent still needs to answer one question during the conversation:

What should happen next?

For that, the system needs conversational state.

The agent should understand what the customer is trying to solve, how urgent the problem is, whether they are actively evaluating solutions, what they already know, and which objections have appeared.

That state should change as the conversation develops.

A customer can move from unfamiliar with the product to curious, then to problem-aware and eventually ready for a sales conversation. The opposite can happen too. A customer may discover that the timing is wrong or that the product isn't a fit.

The system should respond to those changes instead of forcing everyone through the same flow.

This is why I think a Voice AI sales agent needs a state machine, not a better prompt.

Make every answer useful

The next question should depend on what the customer just said.

Suppose a customer says price is the main concern.

The agent shouldn't simply record "price objection" and continue with the script. It should understand what is behind that answer.

Maybe the customer is comparing vendors. Maybe the budget isn't approved. Maybe the value isn't clear. Maybe they are interested but not ready to commit.

Each situation requires a different response.

The same applies to positive signals.

If someone says they are interested, that doesn't automatically mean they want a demo. They may be researching the market or trying to understand whether the product solves a specific problem.

The job of the next question is to reduce that uncertainty.

This is where conversation logic becomes more important than prompt length.

Optimize for progression, not meetings

A common mistake is making "book a meeting" the main objective.

That can push the agent to ask for a meeting even when the customer isn't ready.

A better objective is progression.

The next useful action might be a meeting. It could also be sending pricing information, arranging a callback, connecting the customer with a specialist, or ending the conversation because there is no fit.

Not every cold lead should become a warm lead.

The goal is to reduce wasted conversations and increase the number of interactions that move toward a useful sales outcome.

That also changes how the system should be measured. Call volume and conversation duration are useful operational metrics, but they don't tell you whether the sales process is improving.

Meaningful conversations, qualified leads, warm-lead progression, appropriate human handoffs, meetings that turn into opportunities, and revenue influenced by the system are much more useful measures.

Human handoff is part of the product

The human handoff is another place where Voice AI systems often lose context.

Transferring a call to a salesperson is easy.

Transferring the customer's understanding is harder.

If the customer has spent five minutes explaining their situation and the salesperson starts the conversation from zero, the customer has to repeat everything.

A good handoff should carry the relevant context: the customer's problem, intent, objections, urgency, important questions, and why the AI decided that a human should take over.

The salesperson should be able to continue the conversation instead of restarting it.

This is especially important for technical products where the AI may identify an integration, security, or implementation concern that should be handled by a specialist.

The handoff is not an exception to the product. It is part of the product design.

Build the feedback loop

The call itself isn't the final outcome.

What happens after the call tells you whether the conversation logic worked.

Did the salesperson accept the lead? Was the customer qualified? Did the meeting happen? Did the opportunity progress? Was the lead rejected because there was no fit?

Those outcomes should flow back into the system.

Without that feedback, teams can end up optimizing activity instead of results.

A Voice AI agent may handle thousands of calls and produce impressive transcripts, but if qualified opportunities aren't increasing, the system still has a sales problem.

The feedback loop connects conversation behavior to actual business outcomes.

The architecture matters more than the prompt

From an engineering perspective, I would separate customer context, conversation state, decision logic, response generation, human handoff, and outcome feedback.

The prompt should not become the place where the entire sales process lives.

Once the prompt contains customer-state management, qualification rules, CRM behavior, escalation logic, and every possible objection, it becomes difficult to test and maintain.

A state-driven design gives the team clearer boundaries. You can test whether the system understood the customer's intent, whether the next action was appropriate, and whether the handoff happened at the right time.

You can then improve each part without rewriting the entire conversation.

Voice quality will continue to improve. Voices will become more natural and models will handle more complex conversations.

But the harder problem is deciding what the agent should do with everything it hears.

That requires customer context, conversational state, adaptive decision-making, clear handoff boundaries, and a feedback loop connected to real sales outcomes.

A Voice AI sales agent doesn't need a better script. It needs a better model of the conversation.

Building AI systems? I’d love to connect.

Website - gauravspace.in
Connect on LinkedIn

Gaurav Talesara

The Website Is No Longer the Center of Commerce

Gaurav Talesara — Wed, 02 Sep 2026 18:59:10 +0000

For years, digital commerce has been built around a simple assumption:

Get the user to the website.

Once the user arrives, guide them through a carefully designed journey.

Search.

Landing page.

Product page.

Cart.

Checkout.

The website was the center of the system.

But that assumption is starting to change.

AI-powered interfaces are creating a different model of interaction—one where users may discover products, evaluate options, and potentially complete transactions without following the traditional journey through a company's website.

The journey is becoming something closer to:

Intent → AI → Discovery → Decision → Transaction

The website may still exist.

It may still be important.

But it may no longer be the only interface between a business and its customers.

As an engineering leader, I find the architectural implications of this shift more interesting than the interface itself.

Because if AI becomes another layer between users and businesses, we need to ask a different question.

Instead of:

How do we build a better website?

We may need to start asking:

How does our product exist outside our website?

From page-centric products to capability-centric systems

Traditional web architecture is often built around pages.

A user visits a page.

The page loads data.

The user performs an action.

The backend processes that action.

The journey is designed around a human navigating an interface.

For example:

User
  ↓
Website
  ↓
Product Page
  ↓
Cart
  ↓
Checkout
  ↓
Transaction

The interface controls the experience.

But AI introduces another possibility.

User
  ↓
Intent
  ↓
AI Interface / Agent
  ↓
Business Systems
  ↓
Decision
  ↓
Transaction

This is a fundamentally different interaction model.

The AI does not necessarily need to understand how your website looks.

It needs to understand what your business can do.

What products do you offer?

What are the prices?

What is available?

What are the policies?

What actions can be performed?

What restrictions exist?

What systems can be accessed safely?

This moves part of the engineering problem away from pages and toward capabilities.

Your product needs to become understandable

Humans can navigate ambiguity.

They can look at a product page, interpret an image, read descriptions, compare options, and understand context.

Software systems cannot rely on that in the same way.

AI systems and agents need structured information.

If an external AI system needs to interact with your product, your underlying data becomes increasingly important.

For commerce, that could include:

Product information
Categories and attributes
Pricing
Availability
Inventory
Offers
Shipping information
Return policies
Product compatibility
Restrictions
Business rules

The quality of the user interface still matters.

But structured information behind the interface may become equally important.

A beautifully designed website is not enough if an external system cannot understand what the business actually offers.

APIs may become part of the product experience

For a long time, APIs were mostly considered technical infrastructure.

Something used by:

Mobile applications
Internal systems
Third-party integrations
Partner platforms

But as AI agents become more capable of interacting with external systems, APIs and system capabilities could increasingly become part of the product experience itself.

The question is no longer only:

Can our frontend perform this action?

It may become:

Can a trusted external system understand and safely perform this action?

For example:

Check availability
Get price
Apply offer
Create order
Calculate delivery
Process payment
Track order

These capabilities should not exist only inside tightly coupled frontend flows.

They need to be represented clearly within the system.

This does not mean every business suddenly needs a public API for everything.

Security, authentication, permissions, and business risk still matter.

But the architectural mindset is changing.

The system needs to expose capabilities in a way that can be understood and controlled.

The architecture behind AI commerce

A useful way to think about the future architecture is through four layers.

1. Interface Layer

This is where the user interacts.

It could be:

A website
A mobile app
Search
An AI assistant
A conversational interface
A voice interface
A third-party agent

The important point is that the business should not depend entirely on one interface.

Interfaces can change.

The core capabilities should remain stable.

2. Intelligence Layer

This layer understands intent.

For example:

"I need a laptop for software development under a certain budget."

The system needs to interpret:

User requirements
Budget
Product preferences
Constraints
Available options

An AI system may help translate human intent into structured actions.

But AI should not be responsible for inventing the truth.

It should retrieve information from reliable systems.

That distinction is important.

AI can interpret.

Your systems should remain the source of truth.

3. Business Capability Layer

This is where the actual business operations exist.

For example:

Product Catalog
Pricing Engine
Inventory System
Order Management
Payment System
Shipping System
Customer Management
Business Rules

This layer should contain the actual capabilities of the business.

The interface should consume these capabilities.

An AI agent may consume them too.

The website is just one consumer.

That is the architectural shift I find particularly interesting.

4. Trust and Control Layer

Once AI systems begin taking actions, trust becomes a core engineering problem.

An AI should not simply have unrestricted access to a system.

Businesses need to think about:

Authentication
Authorization
Permission boundaries
Transaction limits
Human confirmation
Audit trails
Observability
Error handling
Rollbacks
Fraud prevention

The more autonomous the system becomes, the more important these controls become.

AI agents may create a more flexible interface.

But flexibility without control creates risk.

The website becomes one interface among many

I don't think websites are disappearing.

People will continue using websites.

Brands will continue designing experiences.

Companies will continue optimize their products for humans.

But the website may gradually lose its position as the only center of the digital experience.

A product could increasingly exist across multiple interfaces.

                 Website
                    │
                    │
Search ─────── Business Systems ─────── Mobile App
                    │
                    │
               AI Agents
                    │
                    │
             Voice Interfaces

The business system becomes the foundation.

Interfaces become ways of accessing it.

This architecture is more resilient to changes in user behavior.

If a new interface becomes important, you don't need to rebuild the entire business.

You need to connect that interface to well-defined capabilities.

The engineering challenge is bigger than adding AI

This is why I don't think the right response is simply:

Let's add an AI chatbot.

The bigger opportunity is to look at the architecture underneath the product.

Can the system clearly represent the business?

Can capabilities be accessed independently?

Are APIs reliable?

Is product data structured?

Are business rules centralized?

Can actions be safely executed?

Can the system explain what happened?

Can failures be detected and recovered?

These are engineering questions.

And they will become increasingly important as AI moves from generating answers to coordinating actions.

The real competitive advantage may move deeper into the stack

For many years, companies competed heavily through the interface.

Better UX.

Better design.

Better onboarding.

Better conversion funnels.

Those things will remain important.

But AI-driven interfaces could make the underlying system architecture increasingly visible.

If an AI agent cannot understand your product, access reliable information, or interact safely with your capabilities, you may become harder to discover and transact with outside your own interface.

The competitive advantage may increasingly come from how well your business capabilities are represented inside the system.

Not just how good the website looks.

A question I think engineering teams should start asking

When building a new product, we often ask:

What screens do we need?

Maybe we should increasingly ask another question:

What capabilities does this business need to expose?

For example:

Instead of starting with:

Product Page
↓
Cart Page
↓
Checkout Page

Start by understanding:

Discover Product
Get Product Details
Check Availability
Calculate Price
Apply Offer
Create Order
Authorize Payment
Arrange Delivery
Track Order

Once the capabilities are clear, different interfaces can be built around them.

A website.

A mobile app.

An AI assistant.

A partner integration.

A voice interface.

The interfaces may change.

The business capabilities remain.

The interface is becoming more flexible. The system needs to become more structured.

That is the part of this shift that interests me most.

AI-powered commerce is not only a change in search, advertising, or checkout.

It could push engineering teams toward a different way of designing products.

One where products need to serve both:

Humans interacting through interfaces.

And increasingly:

AI systems acting on behalf of humans.

The website is not disappearing.

But it may become one interface among many.

And the businesses best positioned for that future may be the ones that have built systems with clear data, well-defined capabilities, strong controls, and reliable foundations.

The interface is becoming more flexible.

The system behind it needs to become more structured.

And I think that's going to be one of the most interesting engineering challenges of the next few years.

15 Things CA Firms Can Automate With AI — Starting With the Work Nobody Wants to Do

Gaurav Talesara — Fri, 28 Aug 2026 17:26:27 +0000

WhatsApp follow-ups, missing documents, GST workflows, loan cases, bank queries, fee collection and more.

A client sends a WhatsApp message:

“Sir, GST ka kya hua?”

Someone from the CA firm's team opens WhatsApp, searches for the client, checks an Excel sheet, looks through Google Drive, asks another team member, and finally replies.

The answer may take two minutes.

Finding the answer can take ten.

Now multiply that by hundreds of clients.

This is where I think the real opportunity for AI in CA firms begins.

Not with another chatbot.

With the repetitive work happening around the CA every single day.

AI Shouldn't Replace the CA. It Should Remove the Work Around the CA.

A CA's time is valuable.

Professional judgment, tax planning, financial advice, audits, client relationships and complex decisions require experience.

But many activities surrounding that work are repetitive.

Someone has to ask for documents.

Someone has to send reminders.

Someone has to check whether documents arrived.

Someone has to update an Excel sheet.

Someone has to follow up with a client.

Someone has to track a bank query.

Someone has to remember which loan case is waiting for what.

These are exactly the kinds of workflows where automation can help.

The question isn't:

“Where can we add AI?”

A better question is:

“What does the team repeatedly do every day that doesn't actually require professional judgment?”

Here are 15 areas worth exploring.

1. Client Document Collection

This is probably one of the easiest places to start.

CA firms constantly need documents from clients: bank statements, purchase registers, sales data, invoices, investment proofs, previous ITRs, GST information and more.

The difficult part isn't asking once.

It's keeping track of what's still missing across hundreds of clients.

Imagine:

Client: Patel Textiles

Required: 10 documents
Received: 8
Verified: 7
Missing: 2

An automation can send the request through WhatsApp or email.

When the client uploads a document, the system can identify it, attach it to the correct client, update the checklist and request whatever is still missing.

The staff no longer has to remember every outstanding document.

The system does.

2. AI Document Classification and Verification

Receiving a document is only half the problem.

The next question is:

Did the client send the right document?

AI can classify uploaded files and perform basic checks before they reach the CA team.

For example:

Uploaded: bank_statement_august.pdf

Document type: Bank Statement
Client: Patel Textiles
Period: August
Readable: Yes
Status: Verified

Or:

Expected: Bank Statement

Received: Invoice

Status: Wrong Document
Action: Request correct document

This can save staff from manually opening and sorting hundreds of files.

3. WhatsApp Client Query Automation

WhatsApp has become one of the most important communication channels for many Indian businesses.

CA firms receive questions throughout the day:

“What documents do I need?”

“When is the GST deadline?”

“Did you receive my statement?”

“What is pending from my side?”

“Has my return been filed?”

An AI assistant can handle the first level of these questions.

But there is an important difference between a generic chatbot and a useful CA assistant.

A generic chatbot knows what GST is.

A useful assistant knows that this particular client's GST filing is waiting for two documents.

Context is the real value.

4. Client Onboarding

New clients create a surprising amount of administrative work.

A new client may require:

KYC.

PAN.

GST details.

Previous returns.

Accounting data.

Bank statements.

Engagement documentation.

Service selection.

Team assignment.

Instead of managing this through spreadsheets and messages, an onboarding workflow can track everything.

Client Onboarding: 72%

Completed:
KYC
PAN
GST

Pending:
Previous ITR
Bank Statements
Accounting Data

The client sees what is pending.

The staff sees what is incomplete.

The partner sees the overall status.

5. GST Compliance Readiness

Compliance automation shouldn't stop at sending deadline reminders.

Knowing that a GST return is due in five days is useful.

Knowing why the return isn't ready is much more useful.

For example:

GSTR-3B

Sales Data        Complete
Purchase Data     Complete
GSTR-2B           Complete
Reconciliation    3 Exceptions
CA Review         Pending
Client Approval   Pending

Now the system can identify exactly what is blocking the filing.

That turns a simple reminder system into a workflow system.

6. GST Reconciliation and Exception Detection

AI doesn't need to review everything.

Sometimes its most valuable role is to find the things that deserve human attention.

For example:

Duplicate invoices
GST amount mismatches
Purchase invoices missing from 2B
Unusual ITC movements
Sales data mismatches
Unexpected changes in margins
Credit-note anomalies

Instead of asking someone to manually inspect everything, AI can create an exception list.

The CA reviews the exceptions.

This is a powerful pattern for professional services:

AI prepares and prioritizes. Humans review and approve.

7. ITR Document and Information Collection

ITR preparation involves another predictable collection process.

The firm may need Form 16, capital gains information, investment proofs, bank statements, interest certificates, previous returns and other supporting information.

The workflow can become:

ITR Client
↓
Checklist Generated
↓
Documents Requested
↓
Documents Received
↓
AI Classification
↓
Missing Information Identified
↓
Staff Review
↓
CA Review

The system isn't preparing tax advice on its own.

It's reducing the administrative work required to get the case ready.

8. Compliance Deadline Management

A calendar tells you when something is due.

A workflow tells you whether you're actually ready.

A better compliance system can automatically create tasks based on the client's services.

For example:

Client
↓
GST
↓
TDS
↓
ITR
↓
ROC
↓
Audit

Each service creates its own workflow.

Tasks can be assigned to staff.

Missing information can trigger client reminders.

Unresolved items can be escalated.

The partner gets visibility without having to ask every employee for an update.

9. Internal Task Assignment

One of the biggest problems in a growing CA firm is simply knowing:

Who is doing what?

A compliance workflow could automatically create:

Client: ABC Industries

Task: GST Reconciliation
Owner: Rahul
Due: 25 August
Status: Waiting for Client

Once the documents arrive:

Status: Ready for Review

Once the CA approves:

Status: Completed

This creates a clear operational trail.

It also makes workload visible across the team.

10. Fee and Payment Follow-ups

Another repetitive process is collecting fees.

The basic workflow is simple:

Invoice
↓
Due Date
↓
Reminder
↓
Client Response
↓
Payment
↓
Receipt

But automation can make this smarter.

If a client says:

“I'll pay on Friday.”

The system can record the commitment.

If Friday passes and payment hasn't arrived, the follow-up can happen automatically.

Staff only needs to intervene when the normal workflow doesn't work.

11. Loan Lead Qualification

This is where things get particularly interesting.

Many CA firms also have finance or loan departments.

They may help clients with business loans, working capital, project finance, machinery finance, home loans, vehicle loans and other financing requirements.

A client might simply send:

“Mare ₹2 crore ni working capital joiye che.”

Instead of immediately handing the conversation to someone, an AI assistant can collect the basic information required for an initial assessment.

For example:

Business Type
Annual Turnover
Existing Loans
Current Bank
GST Status
ITR History
Loan Requirement
Purpose
Collateral

The result becomes a structured finance lead instead of an unstructured WhatsApp conversation.

12. Loan Document Collection

Loan processing involves a significant amount of documentation.

Depending on the case, the team may need financial statements, ITRs, GST information, bank statements, existing loan details, KYC, property documents and other lender-specific information.

A Loan Desk can show:

Loan Case #1047

Required: 24
Received: 18
Verified: 14
Missing: 6

The system can automatically follow up with the client.

It can also identify which cases are blocked because of missing information.

That makes the finance team's pipeline much easier to manage.

13. Bank Query Tracking

This is one of the most interesting workflows in loan processing.

A bank might ask:

“Please explain the increase in debtor days.”

Then:

“Please provide updated CMA.”

Then:

“Please submit promoter ITR.”

These requests can arrive through email, WhatsApp or phone calls.

A structured Loan Desk could turn each request into a task.

Loan Case #1047

Bank Query:
Explain increase in debtor days

Owner:
Rahul

Status:
Waiting for Client

Due:
Tomorrow

Required:
Debtor Ageing Report

Now the team knows exactly what is pending and who owns it.

14. AI-Assisted CMA and Financial Analysis

For CA firms involved in project finance and business loans, financial analysis can become another area for automation.

A system could take historical financial information and help prepare a first-pass analysis.

Historical Financials
↓
P&L
Balance Sheet
Cash Flow
↓
Financial Ratios
↓
Projected Financials
↓
CMA / Project Analysis
↓
CA Review

AI could also help identify unusual changes or prepare explanations for financial movements.

But the final assumptions, analysis and professional output should remain under human review.

AI should accelerate the preparation.

It shouldn't blindly make the decision.

15. An AI Daily Brief for the CA Partner

Eventually, all of these workflows can come together.

But I don't think the answer is another dashboard filled with charts.

A CA partner doesn't necessarily need more information.

They need to know:

What needs my attention today?

The system could provide a simple daily summary:

7 filings blocked
5 clients haven't submitted documents
3 bank queries pending
₹12.8L overdue
8 loan cases waiting for action

That's more useful than opening five different spreadsheets and asking five different people for updates.

The goal isn't more information.

It's less cognitive load.

Don't Build Everything at Once

This is probably the most important point.

It is tempting to build a complete CA platform with:

CRM.

WhatsApp.

AI chatbot.

GST.

ITR.

Documents.

Loans.

CMA.

Payments.

Analytics.

That can quickly turn into a six-month or one-year project before anyone knows whether the product solves a real problem.

A better approach is to pick one workflow.

For example:

Document Collection
↓
Document Verification
↓
Compliance Workflow
↓
GST Exceptions
↓
Loan Desk
↓
Financial Intelligence

Each module can solve a real problem independently.

Then they can gradually connect.

What Should Actually Be Automated?

A simple rule can help.

Automate work that is:

Repetitive
High-volume
Time-consuming
Rule-based
Easy to verify

Keep humans in control of:

Professional judgment
Tax positions
Financial decisions
Client advice
Final approvals
Lending decisions

The best AI system isn't the one that makes every decision.

It's the one that handles the repetitive work and brings the important exceptions to the professional.

The Bigger Opportunity

The future of CA-firm software probably isn't another chatbot.

It's a system that understands the state of every client and every workflow.

Which documents are missing?

Which filings are blocked?

Which tasks are overdue?

Which loan cases are waiting for the bank?

Which clients haven't paid?

Which cases need the CA's attention?

And which tasks can simply be handled automatically?

The CA remains the expert.

The team remains responsible for professional execution.

AI becomes the operational layer that connects the work.

That's where I think the opportunity becomes much more interesting.

If you run a CA firm, what is the one repetitive process your team does every day that you would happily never have to manage manually again?

NOOA: What If an AI Agent Was Just a Python Object?

Gaurav Talesara — Wed, 12 Aug 2026 17:32:29 +0000

There is something interesting happening in the AI agent space.

We have spent the last couple of years building agents using prompts, tools, function calling, workflows, graphs, memory systems, orchestration layers, and increasingly complicated frameworks.

And now NVIDIA Labs has released something that made me stop and think:

What if an AI agent was just a Python object?

That is the basic idea behind NOOA — NVIDIA Object-Oriented Agents.

It is an open-source, model-agnostic Python framework that represents an agent as a Python class, where the object's fields represent state, methods represent capabilities, docstrings can define instructions, and type annotations define interfaces.

The project is still very new and explicitly described as research software. But I think the idea behind it is worth paying attention to.

NOOA on GitHub

So, what is actually different?

Let's look at the traditional way we might build an agent.

We could have:

A system prompt
A collection of tools
Tool schemas
A memory component
An orchestration loop
State management
Some workflow engine
Observability/tracing
Retry logic
Structured output handling

All of these pieces are useful.

But they also create a growing abstraction layer between the developer and the agent.

NOOA takes a different approach.

You define something like:

from nooa import Agent

class SupportAgent(Agent):
    """You are a customer support agent."""

    order_db: OrderDB

    def is_refund_eligible(self, order: Order) -> bool:
        return (
            order.delivered
            and order.days_since_delivery <= 30
        )

    async def triage(
        self,
        message: str,
        order: Order
    ) -> Ticket:
        """Create a typed support ticket."""
        ...

And suddenly the architecture feels very familiar.

The agent is an object.

Its state is on the object.

Its capabilities are methods.

Its interfaces are typed.

Its instructions can live with the methods.

And the interesting part is the ....

A method with a real Python implementation behaves like normal deterministic Python.

A method with an ... body becomes an LLM-driven method at runtime.

That is a surprisingly simple idea.

Why does this matter?

The thing that caught my attention isn't just the syntax.

It is the mental model.

We have traditionally thought about an AI agent as:

Prompt + Model + Tools + Memory + Loop

NOOA is suggesting another abstraction:

Agent = Software Object + Model

That is a meaningful shift.

If the agent is a Python object, then many things software engineers already know how to do become natural again.

Testing.

Refactoring.

Version control.

Tracing.

Dependency injection.

Type checking.

Composition.

State management.

Code review.

Instead of learning another workflow DSL or another orchestration abstraction, developers can start with something they already understand: Python.

NVIDIA's implementation goes further than simply wrapping an LLM in a class. The framework supports typed I/O, live Python objects passed by reference, model-generated Python as an action mechanism, programmable agent loops, context/event APIs, tracing, and long-term memory.

The part I find especially interesting: code as action

This is probably one of the most interesting pieces of NOOA.

Instead of every capability necessarily becoming a traditional function/tool schema that gets serialized into the model context, the model can generate Python and operate within the agent's environment.

In other words:

The model doesn't only call tools.
The model can write code to use the agent's capabilities.

That changes the interface between the LLM and the application.

Python methods and type annotations can become the interface the model works with.

For developers, this could eventually mean less time maintaining huge collections of tool definitions and more time defining clean software interfaces.

Of course, this also creates a very important security problem.

If an LLM can generate and execute Python, we should treat that code as untrusted.

NOOA's own documentation is very clear about this: its AST validation and module restrictions are defense-in-depth mechanisms, not a security boundary. The recommended containment boundary is OS-level sandboxing such as containers or NVIDIA OpenShell.

And I think that distinction is extremely important.

Another interesting idea: agent state

There is another architectural question that NOOA makes interesting.

Where should an agent's state live?

Today, a lot of agent state effectively lives inside the context window.

The longer the conversation becomes, the more we start thinking about:

summarization
context compression
retrieval
memory
token optimization

NOOA instead treats the agent as an object with state.

That opens the door to a different model:

Agent
 ├── State
 ├── Capabilities
 ├── Memory
 ├── Methods
 ├── Context
 └── Model

The LLM becomes part of the object rather than the object being constructed around an LLM conversation.

That is subtle, but I think it could become important as agents become more persistent and autonomous.

Is NOOA going to replace other agent frameworks?

I don't think we know that yet.

And I would actually argue that this is the wrong question.

NOOA is currently a 0.x research preview, and NVIDIA says its public API is not yet stable and can change between releases.

So I wouldn't recommend looking at it today and saying:

"This is the new standard for AI agents."

It isn't.

But I would definitely recommend watching it.

Because research projects like this sometimes introduce an abstraction that looks unusual initially and becomes obvious later.

Remember how strange some ideas looked before they became standard programming patterns?

The interesting question here is whether object-oriented programming becomes a useful abstraction for agent engineering.

My prediction

I don't think the future will be one giant agent framework.

I think we are going to see a convergence of several ideas.

Agents will increasingly look like software components rather than chatbot conversations.

They will have:

Persistent state
Typed interfaces
Memory
Capabilities
Deterministic code
Model-driven reasoning
Observability
Sandboxed execution
Testable behavior
Composable sub-agents

And the boundary between "AI logic" and "software logic" will become much thinner.

That's what makes NOOA interesting to me.

It isn't necessarily introducing another way to call an LLM.

It is asking a more fundamental question:

What should an AI agent look like from a software engineer's perspective?

And the answer from NVIDIA Labs is:

Maybe it should just look like a Python object.

What I want to see next

This is where things get really interesting.

I'd love to see how this approach evolves around:

1. Production reliability

How do these agents behave under real workloads, failures, retries, concurrency, and partial state?

2. Security

If agents can generate and execute code, sandboxing will become a fundamental part of the architecture, not an optional feature.

3. Multi-agent systems

What happens when Python objects representing agents start collaborating with each other?

ResearchAgent
       ↓
PlanningAgent
       ↓
CodingAgent
       ↓
ReviewAgent
       ↓
DeploymentAgent

Can these simply become composable software objects?

4. Agent testing

This could be particularly powerful.

Imagine being able to test an agent almost like any other Python component:

def test_refund_policy():
    agent = SupportAgent(...)
    assert agent.is_refund_eligible(order) is True

And then separately evaluate the probabilistic behavior of its LLM-driven methods.

That separation between deterministic logic and model-driven behavior could become a very useful engineering pattern.

5. Agent observability

NOOA already traces LLM calls, code execution, and method invocations.

I think this will become critical.

As agents become more autonomous, "the model said something weird" won't be enough for debugging.

We will need to know:

What did the agent know?
What state did it have?
What method did it invoke?
What code did it generate?
What did that code change?
What model decision happened?
What happened next?

That is software observability applied to probabilistic systems.

The bigger shift

Maybe the most interesting thing about NOOA isn't NOOA itself.

Maybe it is the direction it represents.

The first generation of AI applications taught us how to integrate models into software.

The next generation is teaching us how to make models behave like software components.

And eventually, I think we will stop drawing such a sharp line between the two.

An agent won't just be:

"an LLM with some tools."

It may become:

"a software component whose reasoning happens to be powered by a model."

NOOA is still early.

The APIs will change.

The patterns will evolve.

There will be competing approaches.

Some ideas will work. Some won't.

But that's exactly why I think it is worth experimenting with now.

Not because NOOA is already the answer.

But because it might be pointing toward a different question.

What does software engineering look like when the software itself can reason?

That is the part I'm watching.

And I have a feeling we are going to see some very interesting updates in this space over the next year.

If you're building AI agents today, I'm curious:

Would you rather build your next agent as a workflow/graph — or as a Python object?

I'd love to hear what other engineers think.

AI #AIAgents #AgenticAI #NVIDIA #Python #LLM #GenerativeAI #SoftwareEngineering #ArtificialIntelligence #DeveloperExperience