Why We Stopped Using Stateless AI Agents for Enterprise Sustainability Scoring

Janani Gunasekaran — Mon, 06 Jul 2026 15:36:36 +0000

I Built a Shopping Assistant That Refuses to Forget You Hate Fast Fashion

I've used a dozen "AI shopping assistants" that treat every conversation like the first time we've ever met. Ask about a product, get a generic answer, close the tab, ask again tomorrow, get the same generic answer. None of them ever get better. They just repeat.

I wanted to see what happens if an agent actually remembers what you tell it — and what happens to your API bill when you stop paying full price for every single question. That's EcoWorthyAI: a sustainability shopping assistant built on Hindsight for persistent agent memory and cascadeflow for runtime cost control.

What it does

You describe a product, EcoWorthyAI evaluates its sustainability — materials, supply chain, certifications — and gives you a recommendation. Nothing revolutionary so far. The interesting part is what happens across a session, not within a single question.

Here's a real exchange from testing:

Interaction 1 — a new session, user demo_123 asks about a product:
USER: "Can you tell me about the Aura Cotton Tee?"
[Hindsight: Recall] Retrieving context for user_demo_123...
-> Retrieved priorities and recent decisions.
[Cascadeflow: Routing] Query detected as MODERATE complexity.
[Cascadeflow: Audit] Model: llama-3.1-8b-instant | Cost: $0.000006 | Latency: 522ms
ECOWORTHY AI RESPONSE:
The Aura Cotton Tee is a sustainable apparel choice made of 100% Organic Cotton.
[Hindsight: Retain] Saving memory for user_demo_123...

Fine, generic, correct. Nothing has been learned yet.

Interaction 2 — same session, the user volunteers a preference and a decision:
USER: "Actually, I also saw the TrendVibe Polyester Tee for $8. But I prefer
to avoid fast fashion and prefer organic materials, so I'll pass on TrendVibe."
[Hindsight: Retain] Saving memory for user_demo_123...
-> Logged decision: Rejected TrendVibe Polyester Tee due to fast fashion
and synthetic materials.

This is the moment that matters. The agent didn't just answer a question — it retained a decision and the reasoning behind it, tied to this specific user.

Fast-forward to a later interaction, a completely different product, no mention of fast fashion in the question at all:
USER: "The Velocity Sneakers by SwiftRuns are priced at $40.00."
[Cascadeflow: Budget] Warning: Nearing session budget. Remaining: $0.000140
[Cascadeflow: Routing] Query detected as HARD complexity.
[Cascadeflow: Routing] Escalation blocked due to budget. Forcing fallback
to cheap model. Cascade requires 2+ models but got 1. Disabling cascade.
[Cascadeflow: Audit] Model: llama-3.1-8b-instant | Cost: $0.000050 | Latency: 520ms
ECOWORTHY AI RESPONSE:
The Velocity Sneakers by SwiftRuns are priced at $40.00. However, please
note: SwiftRuns is a fast fashion vendor. These sneakers are made from
synthetic leather and virgin plastics, and have a zero recyclability rating.
Since you've told me you avoid fast fashion, I'm flagging this vendor even
though it's cheaper.
[Hindsight: Retain] Saving memory for user_demo_123...

Nobody told the agent about fast fashion in this message. It pulled that constraint from a decision made several turns earlier and applied it, unprompted, to a brand-new product. That's the whole pitch of persistent memory in one interaction: the fifth question gets a materially different answer than the first, because of what happened in between.

The other half: not paying full price for every question

Every one of those calls above went through cascadeflow's routing layer before hitting a model. The logic is simple and the payoff is not: classify the query's complexity, send easy questions ("what's this made of?") to a small, cheap model, and only escalate to something stronger when the question genuinely requires it — a multi-product lifecycle comparison, a tradeoff analysis, something with real reasoning depth.

What I didn't expect was how well the budget enforcement held up under pressure. In the run above, the session was approaching its cost ceiling right as a HARD-complexity query came in — exactly the kind of query that should escalate to a stronger model. Instead of blowing the budget or crashing, cascadeflow blocked the escalation, forced a fallback to the cheap model, and logged why:
[Cascadeflow: Routing] Escalation blocked due to budget. Forcing fallback
to cheap model. Cascade requires 2+ models but got 1. Disabling cascade.

That line is the audit trail doing its job. Every decision — which model, why, at what cost, at what latency — gets logged, not just the final answer. If you're trying to run agents in anything resembling production, that log is the difference between "the AI did something" and "I can tell you exactly what the AI did and why."

Reflection: turning history into a profile

Raw interaction history is not the same as understanding a user. After 10 interactions, EcoWorthyAI runs a reflection pass over the session:
[Hindsight: Reflect] Running reflection loop for user_demo_123...
-> Analyzing 10 past interactions...
-> Reflection complete. Generated distilled preference profile.
Updated Distilled Preference Profile:
User is highly conscious of material origins and supply chain ethics.
They actively boycott fast fashion and prioritize organic, certified
materials. Recommendation strategy: Highlight certifications, origin
of materials, and long-term durability. Flag any products linked to
fast fashion or poor labor practices.

Instead of re-reading ten raw interactions every time, future sessions can recall this compressed profile directly — cheaper to retrieve, and a genuinely useful summary rather than a transcript dump.

What I'd do differently

A few things I'd change if I kept building this:

FastEmbed wasn't installed initially, and the retrieval step silently fell back to a slower path. Worth pinning dependencies explicitly rather than discovering it mid-demo (pip install fastembed).
The budget-blocked escalation needs a user-facing signal, not just a log line. Right now a HARD query silently downgrades to a cheap model — that's the correct behavior for cost control, but a real product should probably surface "I'm using a faster/cheaper model right now because we're near budget" so it's not invisible.
Reflection frequency (every 10 interactions) is a guess. I'd want to tune this against how often user preferences actually shift versus how much it costs to keep re-summarizing.
Memory needs an expiry or override path. A rejected product from six months ago shouldn't necessarily carry the same weight forever — preferences change, and the recall layer should let a user explicitly overwrite past decisions.
The routing complexity classifier is doing a lot of work and I'd want to see its false-positive rate at scale — sending a genuinely hard query to a cheap model quietly degrades answer quality in a way that's easy to miss unless you're checking the audit log.

Why this combination specifically

Memory and cost control solve different problems, but they compound. Hindsight's agent memory is what makes the agent worth returning to — it gets more useful the more you use it. Cascadeflow's routing is what makes that sustainable to actually run — you're not paying premium-model prices for "what's this made of?" every single time. Neither one alone would have made for an interesting demo. Together, the delta between interaction 1 and interaction 5 is the whole story.

If you're building an agent that's supposed to accumulate value over a relationship rather than reset every session, I'd start with Hindsight's docs for the memory layer and cascadeflow's docs for the routing and budget side. They're built to be dropped into an existing agent loop rather than requiring a rewrite, which is the only reason I had this working in an afternoon instead of a week.