Why LLMs Should Never Touch Money Directly: Building an AI Payment Recovery Agent Solo

Subhamoy Datta — Sat, 05 Sep 2026 17:33:03 +0000

How I built REVA for Track 3 of the Razorpay AI Buildathon 2026—and why deterministic guardrails matter more than AI autonomy when money is involved.

The Problem I Wanted to Solve

Building for Track 3 of the Razorpay AI Buildathon 2026 as a solo developer came with one obvious challenge:

There were a lot of talented engineers building impressive things.

I didn't want to build another chatbot with an LLM sitting on top of an API.

I wanted to build something where AI had to make real decisions, while still operating within the constraints of a financial system.

That led me to a problem I found particularly interesting:

What should happen after a payment fails?

A failed payment doesn't always mean lost revenue.

Sometimes it's a temporary bank issue. Sometimes the card is expired. Sometimes the customer needs another payment method. And sometimes the payment may have actually succeeded even though the original request didn't return a definitive response.

The difficult part isn't simply retrying.

The difficult part is deciding when to retry, when not to retry, and when to involve a human—without accidentally charging the customer twice.

That's what led me to build REVA — Razorpay Revenue Recovery Agent.

1. The Problem: A Failed Payment Isn't Always Lost Revenue

Payment failures can leave merchants with potentially recoverable revenue.

The obvious solution seems simple:

Retry the payment.

But blindly retrying every failed transaction creates another set of problems.

A recovery system needs to understand why the payment failed before deciding what to do next.

For example:

A temporary bank failure might justify a retry after a delay.
An expired card should not be retried.
A high-value order might deserve human intervention.
A payment with uncertain settlement status should not be blindly retried.

Most basic recovery systems approach this in one of two ways.

Blind Retries

Retry failed payments according to a fixed schedule.

The system doesn't really understand the failure. It simply assumes that trying again is better than doing nothing.

That can lead to unnecessary retries and, more importantly, duplicate-charge risks when payment state is uncertain.

Static Rules

Build a large collection of if/else conditions.

This is more predictable, but eventually becomes difficult to maintain and doesn't handle contextual decisions particularly well.

I wanted something in between.

An agent that could reason about the situation, while a deterministic system made sure that the reasoning could never violate critical financial policies.

2. The Realization: AI Shouldn't Touch Money Directly

My initial idea was straightforward:

Payment fails
      |
      v
Send error to LLM
      |
      v
LLM decides what to do
      |
      v
LLM retries payment

It sounded reasonable.

Then I started thinking about failure scenarios.

What happens if the model incorrectly interprets a payment failure?

What if it decides to retry an expired card?

What if the payment actually succeeded, but the response was delayed because of a network issue?

What if the model decides to retry the same transaction twice?

That's when the core design principle became clear:

LLMs are probabilistic. Financial invariants cannot be.

A language model can produce a reasonable decision most of the time.

A payment system cannot accept "most of the time."

A single incorrect action can potentially result in a duplicate charge, an invalid retry, or a loss of customer trust.

So instead of giving the AI direct access to payment execution, I separated reasoning from execution.

3. The Brain-Gatekeeper Architecture

The architecture became:

                    Failed Payment
                          |
                          v
              +------------------------+
              |       THE BRAIN        |
              |       Gemini AI        |
              |                        |
              | Diagnose failure       |
              | Analyze context        |
              | Propose recovery plan  |
              +-----------+------------+
                          |
                   Proposed Action
                          |
                          v
              +------------------------+
              |     THE GATEKEEPER     |
              |     PolicyEngine       |
              |                        |
              | Idempotency checks     |
              | Retry limits           |
              | Card validity checks   |
              | Risk / LTV policies    |
              +-----------+------------+
                          |
                    Approved Action
                          |
                          v
              +------------------------+
              |    PAYMENT EXECUTOR    |
              |      Razorpay API      |
              +------------------------+

The separation is intentional.

The Brain

Gemini receives relevant transaction context, including:

Payment failure information
Failure codes
Customer history
Transaction value
Previous recovery attempts

It then proposes a recovery strategy.

For example:

Failure: Temporary bank failure
Attempts: 1
Order value: ₹4,999

Proposed action:
Retry after backoff

The model is responsible for reasoning about context.

But its output is only a proposal.

The Gatekeeper

The PolicyEngine is deterministic TypeScript code.

It doesn't care how convincing the AI's reasoning sounds.

It checks whether the proposed action is actually allowed.

Here's what an actual REVA transaction decision looks like:

──────────────────────────────────────────────────────────────────────────────
  TRANSACTION BREAKDOWN:
  ▸ [txn_10005]
    • Diagnosis:     Payment failed: expired_card
    • Strategy:      REQUEST_CUSTOMER_ACTION (Risk: MEDIUM)
    • Policy Guard:  APPROVED (Deterministic invariant check)
    • Settlement:    PENDING_WEBHOOK (₹0 claimed)
──────────────────────────────────────────────────────────────────────────────

This is where the architecture becomes concrete.

The AI identifies the failure as an expired_card scenario and recommends REQUEST_CUSTOMER_ACTION instead of blindly retrying the payment.

The Gatekeeper then independently evaluates that proposed action against deterministic policies and approves it.

Most importantly, the transaction remains in PENDING_WEBHOOK, so ₹0 is counted as recovered until settlement is actually verified.

This is the Brain-Gatekeeper principle in practice:

The AI reasons. The policy engine verifies. The payment system executes.

If any critical invariant fails:

         |
         v

       BLOCKED

The AI can propose an action, but it cannot override the Gatekeeper.

4. Why This Architecture Matters

This isn't simply about adding another validation layer.

It's about separating two fundamentally different responsibilities.

AI is good at:

Understanding unstructured information
Reasoning over multiple signals
Classifying failure scenarios
Generating contextual recovery strategies
Explaining why an action was proposed

Deterministic code is good at:

Enforcing hard constraints
Checking idempotency
Enforcing retry limits
Validating state transitions
Blocking unsafe actions
Guaranteeing predictable behavior

Trying to make one system do both creates unnecessary risk.

So REVA treats the LLM as a reasoning engine, not as the final authority.

5. The Engineering Battles

Building the architecture was only the beginning.

The real work started when I had to make the system behave correctly under failure conditions.

Building solo meant there was nobody to pass the hard bugs to.

Three challenges stood out.

Challenge 1: Making the Audit Ledger Resumable

Financial workflows need reliable audit trails.

I implemented an append-only audit ledger using JSONL records with a SHA-256 parent-hash chain.

Conceptually:

Record 1
   |
   +-- hash₁
         |
         v
Record 2
   |
   +-- parentHash = hash₁
   +-- hash₂
         |
         v
Record 3
   |
   +-- parentHash = hash₂
   +-- hash₃

Every record references the hash of the previous record.

That gives us a tamper-evident chain.

But during the early runs, I ran into a subtle problem.

Whenever a new process started appending to an existing audit file, the hash chain broke.

The new AuditLogger instance was resetting to the genesis hash instead of continuing from the previous record.

That meant the next entry couldn't correctly reference the existing chain.

The Fix

I updated the AuditLogger constructor to inspect the existing ledger.

If the file already exists, it:

Reads the existing JSONL file.
Locates the final JSON record.
Extracts its currentHash.
Uses that hash as the parent for the next record.

So the chain can resume correctly across process restarts.

This taught me an important lesson:

Cryptographic integrity isn't only about hashing.

You also need reliable state recovery.

Challenge 2: Eliminating "Phantom Revenue"

This became one of the most important design decisions in REVA.

Many recovery systems are tempted to treat an API success response as recovered revenue.

For example:

API returns HTTP 200
        |
        v
₹5,000 recovered

But that's not necessarily true.

Payment state can change after the initial request.

A transaction can still be pending, fail later, or require confirmation through a webhook.

So I introduced a Zero-Assumption Settlement state machine.

RECOVERY_ACTION
       |
       v
PENDING_WEBHOOK
       |
       |  ₹0 claimed
       |
       v
Verified Razorpay Webhook
       |
       v
SETTLED
       |
       v
Revenue Counted

The important rule is:

An attempted recovery is not recovered revenue.

When an action is executed, the transaction moves to:

PENDING_WEBHOOK

At this point:

Recovered Revenue = ₹0

Revenue is only counted after an inbound Razorpay webhook arrives and its HMAC-SHA256 signature is successfully verified.

Only then can the transaction move to the settlement state.

This may make the benchmark look less impressive than simply counting API successes.

But it makes the metric much more honest.

Challenge 3: Taming API Rate Limits

There was another practical problem.

I wanted to evaluate REVA against a meaningful number of payment scenarios.

But making an external LLM request for every transaction quickly runs into API rate limits.

So I designed REVA with two execution modes.

Interactive LLM Mode

--mode llm

This is designed for live operation and demonstrations.

It provides:

Real LLM reasoning
Contextual recovery decisions
Explainable AI output
Interactive transaction analysis

Deterministic Simulation Mode

--mode deterministic

This is designed for large-scale evaluation.

It provides:

Reproducible simulations
Seeded transaction generation
Large-scale benchmarking
No dependency on external LLM calls

The benchmark uses a deterministic seed so the same scenarios can be reproduced.

This distinction was important to me.

I didn't want to claim that I had made 1,000 live LLM calls when I hadn't.

The benchmark and the live AI mode are deliberately separated.

6. The Benchmark: 1,000 Failed Payment Scenarios

At this point, I had the architecture.

But I still needed to answer the most important question:

Does it actually work better?

So I evaluated REVA against an empirical benchmark of 1,000 simulated failed-payment scenarios.

The scenarios were modeled across 9 real-world Indian payment failure categories, using:

seed = 42

Together, these scenarios represented approximately:

₹24.85 Lakhs of at-risk revenue.

I compared three approaches:

A naive blind retry bot
Static if/else rules
REVA's autonomous recovery agent

The results were:

Strategy	Recovery Rate	Duplicate Charge Warnings	Human Escalations
Naive Blind Retry Bot	27.4%	19	0
Static If/Else Rules	41.2%	0	0
REVA	68.3%	0	14

The difference was significant.

Blind Retry Bot

The naive retry system recovered only:

27.4%

It also generated:

19 duplicate-charge warnings

The system was essentially following one rule:

Payment failed → try again.

That isn't enough for a financial recovery system.

Static Rules

The static rules improved recovery to:

41.2%

And importantly:

0 duplicate-charge warnings

This showed that deterministic safety rules work.

But they still lacked the contextual reasoning needed to choose better recovery strategies.

REVA

REVA reached:

68.3% recovery

with:

0 duplicate-charge warnings

and:

14 intentional human escalations

The recovery improvement was:

+40.9 percentage points over blind retries.
+27.1 percentage points over static rules.

And the 14 human escalations weren't treated as failures.

They were part of the design.

For certain high-value or high-risk transactions, the safest action isn't another autonomous attempt.

It's:

Stop and ask a human.

7. Why the Benchmark Is Important

The recovery percentage is obviously interesting.

But I think the methodology is more important.

It would have been easy to make the benchmark look better.

For example, I could have counted an HTTP success as recovered revenue.

I didn't.

I could have mixed live LLM calls with deterministic simulation and presented the entire thing as one benchmark.

I didn't.

I could have ignored duplicate-charge risks because the recovery percentage looked good.

I didn't.

Instead, REVA uses explicit states and deterministic simulation so that the results are reproducible and the definition of "recovered revenue" is clear.

That matters when the system you're evaluating is dealing with money.

8. What Building REVA Solo Taught Me

1. Guardrails Matter More Than Prompts

It's easy to spend hours optimizing prompts.

But in a financial agent, the more important question is:

What happens when the model is wrong?

The answer shouldn't be:

"Hopefully nothing."

It should be:

Deterministic code blocks the unsafe action.

The best prompt in the world can't guarantee that an LLM will never produce an incorrect output.

A policy engine can guarantee that certain outputs will never be executed.

That distinction matters.

2. Autonomy Doesn't Mean Unlimited Authority

Before building REVA, I associated agentic AI with giving an agent more tools and more freedom.

This project changed that perspective.

A useful agent doesn't necessarily need unlimited authority.

REVA can:

Diagnose failures
Reason about transaction context
Recommend strategies
Prioritize recovery actions
Decide when human intervention may be appropriate

But it cannot simply override financial policies.

That's a much more useful definition of autonomy for high-stakes systems:

The agent can reason independently without being given unlimited execution authority.

3. Honest Metrics Are More Valuable Than Impressive Metrics

The PENDING_WEBHOOK state was probably one of the most important decisions in the project.

It would have been much easier to claim revenue as soon as a recovery API returned successfully.

But that would create a misleading benchmark.

Instead:

Action executed
      |
      v
PENDING_WEBHOOK
      |
      v
Verified settlement
      |
      v
Revenue counted

That makes the numbers more conservative.

But it also makes them defensible.

I'd rather have a smaller number that I can explain than a bigger number that depends on assumptions.

4. Building Solo Forces You to Understand the Whole System

There was nobody else to hand the difficult parts to.

I had to work through:

Architecture
TypeScript
Payment workflows
Agent reasoning
Policy enforcement
Security
Audit logging
State machines
CLI design
Testing
Benchmarking
API integration
Failure handling

It was exhausting.

But it also meant that I couldn't hide behind abstractions.

When something broke, I had to understand why.

And that gave me a much deeper understanding of the system I was building.

9. Testing the System

I also wanted the safety layer to be testable independently of the LLM.

The integration suite currently has:

17/17 passing tests on Bun.

This matters because the most important safety guarantees shouldn't depend on whether the LLM happens to produce the expected output during a test run.

The deterministic components need deterministic tests.

The goal isn't to test whether the model is always right.

The goal is to test whether the system behaves safely even when the model isn't right.

That's the boundary I wanted REVA to enforce.

10. The Final Architecture

By the end, REVA became less about:

"Put an LLM on payments."

And more about:

"Build a safe boundary around an LLM that reasons about payments."

The final flow looks like this:

                    AI
                     |
                   Reason
                     |
                     v
              Proposed Action
                     |
                     v
          Deterministic Policies
                     |
              +------+------+
              |             |
           BLOCKED       APPROVED
                            |
                            v
                      Payment API
                            |
                            v
                         Webhook
                            |
                            v
                         SETTLED
                            |
                            v
                    Revenue Counted

The LLM doesn't get to decide whether money moves.

It gets to suggest what should happen next.

The deterministic layer decides whether that suggestion is safe.

And the settlement system determines whether the money was actually recovered.

11. The Bigger Lesson

The most interesting thing I learned from this project wasn't a specific Gemini technique.

It wasn't a particular prompt.

It wasn't even the recovery rate.

It was the realization that AI autonomy and system authority don't have to be the same thing.

An AI agent can be highly autonomous in its reasoning while still operating inside strict boundaries.

That gives us a useful design pattern for high-stakes AI:

LLM
 |
 | Reason
 v
Proposed Action
 |
 | Validate
 v
Deterministic Policy
 |
 +---- BLOCK ----> Stop
 |
 +---- APPROVE --> Execute
                       |
                       v
                   Verify State
                       |
                       v
                  Count Outcome

This pattern can extend beyond payments.

The same principle applies anywhere an AI system can make decisions with real-world consequences:

Financial operations
Infrastructure changes
Security systems
Healthcare workflows
Customer account actions
Automated commerce
Production deployments

The model can be the reasoning layer.

But the system around it needs to control what that reasoning is allowed to do.

12. What's Next?

REVA is currently a buildathon project, but there are several directions I'd like to explore further.

Better Failure Classification

Use richer transaction and historical signals to distinguish between temporary, recoverable, and fundamentally unrecoverable failures.

Smarter Recovery Policies

Instead of simply choosing between a small set of strategies, optimize recovery decisions based on historical outcomes.

Better Risk Modeling

Introduce stronger fraud and transaction-risk signals into the policy layer.

More Granular Human-in-the-Loop Decisions

Instead of a simple escalation path, determine exactly when human intervention creates the most value.

Production-Grade Event Processing

Move toward a more robust event-driven architecture for handling payment state changes and webhooks at scale.

Deeper Observability

Make every AI proposal, policy decision, state transition, and settlement event observable and explainable.

The interesting problem isn't simply making the AI smarter.

It's figuring out:

How much authority should an AI agent have—and how can we safely increase that authority over time?

13. Final Takeaway

Building REVA changed the way I think about AI agents.

I started with a simple idea:

"Let an LLM recover failed payments."

I ended up with a very different architecture:

Let the LLM reason about recovery, but never let it bypass deterministic financial invariants.

The Brain proposes.

The Gatekeeper verifies.

The payment system executes.

The webhook confirms.

Only then do we count the money.

And that's probably the biggest lesson I took away from building REVA:

In high-stakes AI systems, the goal shouldn't be to make the model all-powerful.

The goal should be to make the model useful while making the consequences of being wrong controllable.

Tech Stack

TypeScript · Bun · Docker · Google Gemini · Razorpay APIs

Built for Track 3 of the Razorpay AI Buildathon 2026.

If you're building AI agents for payments, finance, or other high-stakes domains, I'd love to hear how you're handling the boundary between model reasoning and deterministic execution.

DEV Community: Subhamoy Datta

Why LLMs Should Never Touch Money Directly: Building an AI Payment Recovery Agent Solo

The Problem I Wanted to Solve

1. The Problem: A Failed Payment Isn't Always Lost Revenue

Blind Retries

Static Rules

2. The Realization: AI Shouldn't Touch Money Directly

3. The Brain-Gatekeeper Architecture

The Brain

The Gatekeeper

4. Why This Architecture Matters

AI is good at:

Deterministic code is good at:

5. The Engineering Battles

Challenge 1: Making the Audit Ledger Resumable

The Fix

Challenge 2: Eliminating "Phantom Revenue"

Challenge 3: Taming API Rate Limits

Interactive LLM Mode

Deterministic Simulation Mode

6. The Benchmark: 1,000 Failed Payment Scenarios

Blind Retry Bot

Static Rules

REVA

7. Why the Benchmark Is Important

8. What Building REVA Solo Taught Me

1. Guardrails Matter More Than Prompts

2. Autonomy Doesn't Mean Unlimited Authority

3. Honest Metrics Are More Valuable Than Impressive Metrics

4. Building Solo Forces You to Understand the Whole System

9. Testing the System

10. The Final Architecture

11. The Bigger Lesson

12. What's Next?

Better Failure Classification

Smarter Recovery Policies

Better Risk Modeling

More Granular Human-in-the-Loop Decisions

Production-Grade Event Processing

Deeper Observability

13. Final Takeaway

Tech Stack