DEV Community

Cover image for When AI Remembers What Workers Are Owed
Santhosh Kumar Gudimetla
Santhosh Kumar Gudimetla

Posted on

When AI Remembers What Workers Are Owed

#devchallenge #ai #python #agents

Every morning at street-corner labor hubs across India, millions of daily-wage workers agree to jobs through a verbal handshake.

A painter agrees to ₹800 a day for six days of work. On Saturday evening, the contractor hands him ₹3,000 in cash: "Take this for now. Cement delivery was delayed, so I will settle the remaining balance next week."

Ten days later, the painter asks for his remaining money. Neither party has a timesheet, a contract, or a written log. Did the painter take Monday off? Was ₹500 given as a mid-week advance for groceries? In the absence of an audit trail, memories blur—and because the contractor holds the purse strings, the worker inevitably absorbs the loss.

Nobody is necessarily acting with malice. Forgetting is simply human nature. But in the informal economy, forgetting is economically one-sided: the person holding the cash always benefits when records fade.

We built HakDaar (Hindi for "The One Who Is Entitled") to eliminate that information asymmetry. HakDaar is an open-source, voice-first platform where workers speak or text in Telugu, Hindi, or English to log their work, record verbal promises, and track every rupee owed.

Yet, building an agent for informal laborers forced us to confront a dangerous engineering reality early on: Large Language Models must never be allowed to calculate money.

Here is the story of how a dangerous prototype failure shaped HakDaar's architecture, and how we decoupled deterministic accounting from episodic agent memory.

Video Demo


Video Walkthrough: Watch HakDaar in action on YouTube

The ₹50,000 Hallucination

In our earliest proof-of-concept, we used the pattern common to many LLM agent tutorials: we fed user conversations into an LLM context window, retrieved recent messages, and prompted the model:

"You are an accounting assistant. Review the user's conversation and state their outstanding balance."

It felt responsive and fluid—until we ran tests with real workers.

A laborer sent two voice notes across a span of twenty minutes:

  1. "I worked 5 days on Suresh's site."
  2. "Rate was 10000." (He meant a fixed lump-sum contract of ₹10,000 for completing a ten-day masonry segment).

The database had not yet parsed or confirmed the lump-sum rate. However, the LLM saw both numbers—5 days and 10,000—sitting together inside its context window. During its forward pass, the model did what probabilistic autoregressive engines do: it associated the numbers, multiplied them, and generated a confident reply:

"You worked 5 days at ₹10,000 per day. Suresh still owes you ₹50,000."

The actual balance was zero. The model had conjured a massive phantom debt out of thin air.

If a daily-wage laborer brings an AI-calculated figure to a job site and demands ₹50,000, the consequences are immediate and severe: they are accused of extortion, fired on the spot, and blacklisted across the local contractor network. In high-stakes human systems, probabilistic arithmetic is an unacceptable liability.

The Rule of Absolute Separation

That failure established HakDaar's core architectural principle: Never mix narrative memory with mathematical calculation.

NARRATIVE CONTEXT (Hindsight)       TRANSACTIONAL STATE (SQLite + Decimal)
• Verbal promises and agreements    • Immutable event stream (work days, cash)
• Contractor excuses and delays     • Arbitrary-precision Python Decimal math
• Temporal history across weeks     • Sole authority on balances and dues
• Relational context and sentiment  • Completely isolated from LLM generation
Enter fullscreen mode Exit fullscreen mode

We split the runtime into two completely decoupled layers:

  1. The Ledger (SQLite + Python Decimal): The ledger is the sole mathematical authority. It stores immutable events (RATE_PROMISED, WORK_DAY, PAYMENT, ADVANCE). It calculates earned, paid, and owed deterministically. The LLM is never allowed to perform arithmetic or mutate financial state.
  2. Episodic Agent Memory (Hindsight): Powered by self-hosted Hindsight, this layer stores the evolving human story—who made what verbal commitment, which contractor cited material delays, how many days have elapsed, and what conversational context was established in past sessions.

With this separation, if the LLM misinterprets a phrase, it might adjust the phrasing of a sentence, but it can never alter the worker's balance sheet.

Deterministic Accounting Without Speculation

The ledger engine operates on an event-sourcing model. Rather than updating a volatile balance column, the account standing is computed on demand. Crucially, the system refuses to speculate when terms are incomplete:

from decimal import Decimal
from typing import Optional, Dict, Any

def compute_ledger_standing(
    daily_rate: Optional[Decimal],
    fixed_sum: Decimal,
    shifts: int,
    disbursed: Decimal
) -> Dict[str, Any]:
    # Defensive guard: Never guess or synthesize unagreed rates
    if daily_rate is None and fixed_sum == Decimal("0"):
        return {
            "earned": None,
            "disbursed": disbursed,
            "balance_owed": Decimal("0"),
            "advance_held": disbursed,
            "status": "UNAGREED_RATE"
        }

    total_earned = (
        (daily_rate * Decimal(shifts) if daily_rate is not None else Decimal("0")) 
        + fixed_sum
    )

    net = total_earned - disbursed

    if net > Decimal("0"):
        return {
            "earned": total_earned,
            "disbursed": disbursed,
            "balance_owed": net,
            "advance_held": Decimal("0"),
            "status": "OWED"
        }
    elif net < Decimal("0"):
        return {
            "earned": total_earned,
            "disbursed": disbursed,
            "balance_owed": Decimal("0"),
            "advance_held": abs(net),
            "status": "ADVANCE"
        }
    else:
        return {
            "earned": total_earned,
            "disbursed": disbursed,
            "balance_owed": Decimal("0"),
            "advance_held": Decimal("0"),
            "status": "SETTLED"
        }
Enter fullscreen mode Exit fullscreen mode

If an agreement has not been finalized, the status becomes UNAGREED_RATE. The agent prompts the worker to clarify the agreed terms with their employer, rather than inventing numbers.

Long-Term Episodic Memory with Hindsight

A database ledger records numbers, but human work relationships rely on context. A worker returning after a two-week absence will not re-explain their employment history; they speak in shorthand:

"He gave me 2,000 today."

A stateless chat model has no way of knowing who "he" refers to. HakDaar uses Hindsight to provide persistent, multi-tenant memory across sessions.

Dual-Bank Memory Topology

  1. Private Worker Banks (worker-{id}): Each worker has an isolated memory store. This bank retains the worker's personal employment timeline, employer names, site locations, past verbal commitments, and recorded excuses (e.g., "Contractor stated the cement delivery did not arrive on Tuesday").
  2. Shared Reputation Bank (employer-reputation): A collective bank that stores anonymized wage dispute reports. To safeguard worker safety, this bank contains zero worker names, phone numbers, or personal identifying information.

Concurrent Recall and Asynchronous Retention

To keep conversational response times below one second while maintaining deep memory indexing, HakDaar executes memory operations concurrently:

async def _retain_all():
    # Asynchronously retain the worker's turn context
    await memory.retain(
        bank, turn_content, 
        context="chat message from worker", 
        timestamp=now,
        metadata={"worker_id": worker_id, "kind": "chat"}
    )
    # Retain anonymous employer performance if a payment delay occurred
    for employer, kind, summary in reports:
        await memory.retain(
            memory.REPUTATION_BANK, summary, 
            context=f"anonymous wage report ({kind})",
            timestamp=now, 
            metadata={"employer": employer, "kind": kind}
        )

# Recall historical context concurrently while retaining new information in parallel
results = await asyncio.gather(
    memory.recall(bank, recall_query),
    _recall_rep(),
    _retain_all(),
    return_exceptions=True
)
Enter fullscreen mode Exit fullscreen mode

Three Critical Engineering Details:

  • Asynchronous Ingestion: By invoking aretain with retain_async=True, Hindsight extracts entities, temporal anchors, and relational context in the background. The user sees their balance card update immediately without waiting for embedding generation.
  • Context-Enriched Recall: A bare query like "he gave me 2000" is expanded before retrieval to "he gave me 2000 (employers: Santosh Builders)", ensuring relevant historical records are fetched accurately.
  • The "Recall Before Retain" Invariant: Running memory retrieval concurrently with retention ensures that the agent never "recalls" the very message it is currently in the middle of processing, preventing circular reasoning loops.

Output Interception: Guarding Against Fabricated Numbers

Even when the prompt provides exact numbers calculated by the ledger, an LLM can still alter digits during generation (e.g., turning ₹1,800 into ₹1,880 or combining separate amounts).

To eliminate this vulnerability, HakDaar deploys an Output Interceptor Gate that inspects generated completions before they reach the user:

def invented_amounts(reply: str, *sources: str) -> set[int]:
    """
    Scans generated text for any currency-sized numbers (>= 100) 
    that do not exist in the authorized ledger values or user utterances.
    """
    allowed = set()
    for src in sources:
        allowed |= numbers_in(src)
    return {n for n in numbers_in(reply) if n >= 100 and n not in allowed}
Enter fullscreen mode Exit fullscreen mode

If the generated reply contains a single number ($\ge ₹100$) that cannot be traced back to the ledger or the worker's explicit words, the output is rejected. The agent attempts a rewrite, and if that still produces ungrounded figures, the system reverts to a deterministic, pre-compiled template response built directly from the ledger.

Collective Accountability Without Defamation

Individual laborers possess zero bargaining power when an employer withholds wages. However, allowing arbitrary public reviews invites spam, extortion, and defamation.

HakDaar implements a Multi-Witness Verification Protocol using Hindsight's reflect capability:

  1. Grounded Triggers: A reputation entry is generated only when a verified ledger records an unpaid balance exceeding 14 calendar days.
  2. Threshold Gate ($N \ge 2$): A single report never triggers a public advisory. Two or more independent, unlinked worker accounts must log payment delays with the same contractor before any advisory is shown.
  3. Reflective Neutrality: When a worker asks about a contractor, Hindsight synthesizes reports using strict neutrality guidelines: 
async def query_employer_trust(employer_name: str) -> str:
    summary = await memory.reflect(
        bank="employer-reputation",
        query=(
            f"Provide a neutral summary of payment timeliness for {employer_name}. "
            "Report exact worker counts. Acknowledge full settlements if present. "
            "Never use speculative, judgmental, or defamatory phrasing."
        )
    )
    return summary
Enter fullscreen mode Exit fullscreen mode

The resulting advisory gives the prospective laborer actionable clarity without slandering the employer:

"2 workers have recorded payment delays with Santosh Builders over the past 30 days (averaging ₹1,200). 1 worker recorded a full, on-time payment. Make sure to agree on payment milestones before starting work."

Engineering for Low-Literacy Realities

Designing for informal laborers requires rethinking conventional UI paradigms:

  • Frictionless Authentication: Migrant workers frequently swap SIM cards or lack cellular SMS balances. HakDaar uses a lightweight 4-digit PIN system tied to a primary phone number, requiring zero SMS or OTP verification.
  • Spoken Indic Numbers: Reading ₹1,800 as individual digits ("one-eight-zero-zero") confuses users. HakDaar expands currency numbers into full natural words in Telugu ("వెయ్యి ఎనిమిది వందల రూపాయలు") and Hindi ("एक हज़ार आठ सौ रुपये") prior to Text-to-Speech synthesis.
  • Conversational State Resolution: On noisy construction sites, workers reply in fragments. When the agent asks "What daily rate was promised?" and the worker says "800", HakDaar tracks active state and automatically binds the isolated figure to the pending rate agreement.

Key Takeaways for AI Builders

  1. Keep arithmetic in the database. Use language models for dialogue, intent parsing, and natural communication. Use SQLite and Decimal for financial calculations.
  2. Recall before you retain. Decoupling retrieval from storage prevents agents from citing unverified statements from the active turn.
  3. Validate outputs, not just inputs. Lexical filters that scan generated text for unauthorized numbers catch hallucinations before they reach the user.
  4. Shared memory demands governance. When one user's history informs another user's recommendations, enforce anonymity, multi-witness thresholds, and neutral synthesis.
  5. Accessibility is an engineering discipline. Voice-native workflows, spoken native currency formatting, and low-friction authentication make applications more resilient for all users.

Open Source & Resources

HakDaar is fully open source. If you are building software where a user's verbal history is the primary source of truth (wages, informal micro-loans, rent agreements, or healthcare tracking), consider implementing persistent agent memory with deterministic ledger guards.

Keep the stories in memory, and keep the numbers in a ledger the model cannot touch.

Top comments (1)