DEV Community: Chichebe John

Scam Listings In, Trustworthy Price Out: The Guardrail Stack Behind an AI Pricing Engine

Chichebe John — Thu, 30 Jul 2026 14:19:16 +0000

Someone asked how I handle noise in search results when there's no clean dataset to fall back on. Here's the full answer: six layers, none of which trust the model.

After I wrote about building a car pricing engine for a market with no data, @topstar_ai asked something in the comments that deserved more than a reply:

Comment on The Next Billion Users Don't Come With Datasets, and That's the Opportunity

Luis Cruz Jul 15

I was struck by the idea that LLMs can synthesize a price at request time from live search results, which flips the traditional supervised learning approach on its head. The fact that the dataset is no longer a prerequisite, but rather the exhaust, is a game-changer for building AI products in data-poor markets. I've worked on similar projects where we had to rely on proxy data sources and clever feature engineering to overcome data scarcity, but the LLM approach seems to offer a more scalable and flexible solution. How do you handle the issue of data quality and noise in the search results, and what kinds of domain guardrails have you implemented to ensure the model's output is reliable and trustworthy?

Two questions in there, and they're the right two. If your ground truth is live search results instead of a curated dataset, then your data quality problem doesn't go away. It moves. It becomes a runtime problem you have to solve on every single request.

Here's what I actually do at AutoValue, layer by layer.

First, what the noise actually looks like

Generic advice about "filtering low-quality sources" is useless until you know the specific failure modes. Mine, when I search for Nigerian used car prices:

Asking prices, not transaction prices. Every listed price on Jiji is an opening move. Nigerian buyers negotiate 10 to 20% below the ask as a matter of routine. A model that reports the listed number is systematically overpricing the market.
Pre-devaluation listings that never got taken down. The naira went from around ₦460 to the dollar to around ₦1,650. Listings from 2022 are still indexed and still show 2022 prices. They aren't wrong, exactly. They're archaeology.
Trim contamination. Search "Toyota Land Cruiser Prado price Nigeria" and most of what comes back is the VX full option, because that's what dealers advertise. If you take that as the base price for a standard trim, every Prado prices high forever.
Currency mixing. Export and auction sites quote USD. Nigerian listing sites quote naira. Both land in the same result set.
Scam bait. A ₦2.5 million Range Rover is not a data point. It's a lure. But it's a perfectly well-formed number that any naive parser will happily average in.

Notice that only the last one is "bad data" in the usual sense. The rest are all correct data that's wrong for my purpose. That distinction drives the whole design.

Layer 1: Constrain the search, not just the parsing

The cheapest filter is the one that stops noise from being retrieved at all.

const query = `${year} ${make} ${model} price Nigeria site:jiji.ng OR site:cars45.com`;

const searchRes = await fetch("https://google.serper.dev/search", {
  method: "POST",
  headers: { "X-API-KEY": serperKey, "Content-Type": "application/json" },
  body: JSON.stringify({ q: query, gl: "ng", hl: "en", num: 10 }),
});

site: scoping to the two marketplaces that actually matter, and gl: "ng" to geo-locate the search in Nigeria. That last one matters more than it looks. Without it you get US results for the same car, in dollars, for a market with entirely different import economics.

I also prioritise Google's answer box and knowledge panel above organic results when they're present, because Google has already done a round of synthesis across sources:

const priorityLines: string[] = [];
if (answerBox?.snippet || answerBox?.answer) {
  priorityLines.push(`GOOGLE SUMMARY: ${answerBox.title ?? ""}\n${answerBox.snippet ?? ""}`);
}
const organicLines = organic.slice(0, 7).map(r => `${r.title ?? ""}\n${r.snippet ?? ""}`);
const allLines = [...priorityLines, ...organicLines];

Layer 2: Separate extraction from reasoning

This is the structural decision everything else depends on, and it's the one I'd most want someone to take away.

There are two completely different jobs here. One is read these messy snippets and tell me what this car currently sells for. The other is given a base price, adjust it for this specific car's mileage, condition, trim and location. The tempting move is one prompt that does both. Don't.

They're two separate Claude calls in my pipeline, and the reason is auditability. When a price comes out wrong in a single-call design, you cannot tell whether the model misread a snippet or misapplied an adjustment rule. Split them and every bad output localises to one stage. My debugging time dropped more from this change than from any prompt improvement.

Extraction runs on Haiku with a procedure, not a request:

Extract the REALISTIC TRANSACTION PRICE RANGE, what this car actually sells
for, not what sellers hope to get.

- Naira devalued sharply: ₦460/$ in 2022 to ₦1,650/$ in 2026. Car prices are
  now 3 to 4x higher than pre-2023.
- Jiji and Cars45 show ASKING prices. Nigerian buyers negotiate 10 to 20%
  below the listed ask.
- STEP 1: any price more than 60% below the highest-priced cluster member is a
  pre-devaluation or junk listing. SKIP IT.
- STEP 2: SKIP listings marked "5+ years on Jiji". ENTERPRISE is a dealer
  badge, do NOT skip those, dealers are valid sellers.
- STEP 3: prices in $ multiply by ₦1,650.
- STEP 4: from remaining valid prices take the MIDDLE cluster. Ignore the
  bottom 15% and anything above 2x the cluster median.
- price_low = 25th percentile. price_high = 75th percentile.
- If only 1 to 2 valid prices remain: low = price × 0.88, high = price × 1.08.

Every one of those rules is a scar. STEP 2's carve-out for ENTERPRISE exists because my first version of "skip listings with badges" quietly deleted all the dealer inventory, which is the most reliable pricing signal on the site.

Layer 3: Percentiles, never averages

Worth calling out on its own because it's a one-line change with outsized effect.

The output is the 25th and 75th percentile of the surviving cluster. Not the mean, not min and max. A single scam listing at ₦2.5M destroys a mean and it destroys a min. It barely moves a 25th percentile. Robustness to outliers is a property you get for free from the right statistic, and no amount of prompt engineering substitutes for picking it.

The narrow-data case gets handled explicitly too. With only one or two valid prices left, percentiles are meaningless, so the rule degrades to a fixed spread around the single observation. Say what should happen when the data is thin, or the model will invent something.

Layer 4: Make the model report, let code decide

Trim contamination is the subtlest failure in the list, and the fix generalises well beyond pricing.

I could ask the extraction model to "adjust the price down if the listings look like a high trim." That's asking for a silent, unauditable, unrepeatable adjustment buried inside a number. Instead the model reports what it saw as a separate field, and the arithmetic happens in code where I can read it:

const TRIM_FACTOR: Record<string, number> = {
  full_option: 1.12,
  limited:     1.10,
  sport:       1.08,
  standard:    1.00,
  unknown:     1.00,
};

const trimFactor = TRIM_FACTOR[searched.detected_trim] ?? 1.00;
const normalizedLow  = Math.round(searched.price_low  / trimFactor / 50_000) * 50_000;
const normalizedHigh = Math.round(searched.price_high / trimFactor / 50_000) * 50_000;

Now every stored anchor is a base-trim price, the trim premium gets re-added later as a visible line item the seller can see, and I can change the multipliers without touching a prompt.

Ask the model for observations. Keep the calculations in code. If a model's judgment is going to move a number, make it move the number through a mechanism you can print.

Layer 5: A deterministic invariant after the model responds

The adjustment model is given explicit percentage rules: excellent interior is +2%, mileage below expected earns +0.3% per 10,000km for a 4 to 7 year old car, and so on.

It does not always follow its own rules. Verified live: on one car it applied a +7.8% mileage bonus where its own stated rule produced 0.3%. That's not hallucination in the dramatic sense. It's a model doing approximate arithmetic and landing somewhere plausible-looking.

So there's a clamp that runs in code after the response comes back:

const CONDITION_UPLIFT_KEYS = ["condition", "interior", "engine", "mileage"] as const;

function applyAgeConditionCeiling(estimate, base, carAge) {
  // Younger car, smaller allowed uplift: "excellent" is the expected norm for a
  // 2-year-old car and already priced into the anchor, but a genuine rarity on
  // a 10-year-old one.
  const capPct = carAge <= 3 ? 0.03 : carAge <= 7 ? 0.06 : 0.10;
  const baseMedian = (base.price_low + base.price_high) / 2;

  const upliftNGN = CONDITION_UPLIFT_KEYS.reduce(
    (sum, key) => sum + Math.max(0, estimate.adjustments?.[key] ?? 0), 0
  );
  const upliftPct = upliftNGN / baseMedian;
  if (upliftPct <= capPct) return { estimate, cappedPct: 0 };

  // Subtract only the excess above the ceiling, proportionally.
  const excessPct = upliftPct - capPct;
  ...
}

The domain reasoning matters as much as the code. A search anchor for a two-year-old car already assumes near-excellent condition, because that's what two-year-old cars on the market are. Stacking excellent body, excellent interior, excellent engine and a low-mileage bonus on top of that anchor prices a clean used car like a new one. The failure was worst on thin-data models, where the only search results are new-car dealer prices and the anchor is already the ceiling.

Trim and location uplifts are deliberately exempt from the cap, because they're structural rather than condition-based. Trim was divided out in layer 4 and is legitimately being re-added.

Never trust a model to guarantee an arithmetic invariant. If a relationship must hold, enforce it in code after the fact.

Layer 6: Fail closed

This is the one I'd argue about with most people building LLM products.

When the search returns nothing usable, the endpoint does not fall back to the model's own knowledge. It returns a 503:

if (!searched) {
  return NextResponse.json({
    success: false,
    error: "pricing_unavailable",
    message: "We couldn't find live market data for this car right now. Please try again in a few minutes.",
  }, { status: 503 });
}

Most LLM products fail open. Retrieval comes back empty, and the model answers anyway from training data, and the user cannot tell the difference between a grounded answer and a guess. For a chatbot that's mildly bad. For a product whose entire value proposition is "this number is trustworthy," it's fatal, because the failure is invisible exactly when it's most damaging.

Related, and covered in more depth in the previous article: model-generated prices were once cached as anchors, then re-injected on the next request as if they were market data. The fix has since hardened from a runtime bypass into a type. The stored anchor's source can only be manual, serper_search, or google_cse, and the read query filters on those three explicitly:

.in("source", ["serper_search", "google_cse", "manual"])

A model's own output is no longer something the system is capable of treating as evidence.

What I'd still fix

Being honest about the holes, since that's the part these posts usually skip.

The exchange rate is a hardcoded constant, USD_TO_NGN = 1650. I wrote an entire article about how stale currency assumptions destroyed my prices, and then I put a stale currency assumption in my own code. It needs to be a live lookup, and it will be.

The percentile cutoffs, the 60% staleness threshold, the trim multipliers: those are calibrated from my own market knowledge, not fitted to outcome data. Once enough listings sell through the platform, actual transaction prices become the training signal that replaces my guesses. That's the point where the exhaust finally becomes the dataset.

The short version

Know your specific noise. Most of my bad data isn't false, it's correct data that's wrong for the question.
Separate extraction from reasoning. Two calls, two failure domains, debuggable.
Use robust statistics. Percentiles over means. One scam listing shouldn't move your answer.
Models report, code computes. Any adjustment that moves a number should be visible and versioned.
Enforce invariants after the model, in code. Stated rules are not guaranteed rules.
Fail closed. No data is a better answer than a confident guess, when trust is the product.

I build AutoValue in the open at autovalue.tech. If you're doing retrieval-grounded work in a market where the sources are this messy, I'd like to hear which layer you'd add.

The Next Billion Users Don't Come With Datasets, and That's the Opportunity

Chichebe John — Wed, 15 Jul 2026 12:52:33 +0000

An opinion from someone building AI where the training data runs out.

Every AI product tutorial makes the same silent assumption: that your problem domain is already measured. Want to build a price estimator? There's a Kaggle dataset. A recommendation engine? Scrape the reviews. A valuation model? Kelley Blue Book publishes the ground truth and your model just has to approximate it.

Now try building the same product in Lagos.

I build AutoValue, an AI pricing platform for used cars in Nigeria, and I can tell you exactly what the data landscape looks like: there is no Blue Book. There is no clean listings API. The largest marketplaces are full of asking prices that everyone understands to be opening bids, posted by sellers who expect to be haggled down 20%. Two identical cars sit ₦8 million apart on the same page. The "dataset" for my domain is a negotiation culture, a WhatsApp group, and a dealer's gut feeling.

The standard takeaway is that markets like this are too hard for AI products. I think that conclusion is exactly backwards, and I want to make the case for why data-poor markets are the most interesting place to build right now.

Where there's no dataset, there's no incumbent

In the US, a startup doing AI car valuation is competing against Kelley Blue Book, Carvana's pricing engine, CarGurus' deal ratings, and forty years of accumulated market data. The AI is a marginal improvement on an already-solved problem. The moat belongs to whoever collected the data decades ago.

In Nigeria, the same product is not an improvement on the incumbent. It creates the category. Whoever produces the first trusted price reference doesn't compete with the Blue Book. They become it. And this generalizes far beyond cars: think rents in Nairobi, used electronics in Dhaka, farm equipment in rural Brazil. Enormous, active markets that trade daily without a single reliable price signal. Every one of them is waiting for its reference source, and the reference source is now buildable by a small team with an LLM, a search API, and domain stubbornness.

LLMs changed what "no dataset" means

Here's the technical shift that makes this possible. The classical approach to a pricing model is supervised learning: collect a hundred thousand labeled transactions, train, deploy. No transactions, no model. Data-poor markets were locked out by definition.

LLMs flipped the order of operations. My pricing engine doesn't learn from a historical dataset; it synthesizes a price at request time from live search results, structural knowledge (depreciation curves, trim hierarchies, mileage effects), and hard domain guardrails. The model contributes judgment, the search contributes freshness, and the database of resolved prices grows as a byproduct of usage. The dataset isn't a prerequisite anymore. It's the exhaust.

That inversion is the entire unlock. You can now bootstrap a data business in a market that has no data, because the first version of the product doesn't need the data the product will eventually accumulate.

The catch: the model thinks your market doesn't exist

I won't romanticize this. Building AI for a market the training data forgot means the model's priors actively work against you. Mine believed a dollar was still ₦450 and priced every car in the country at a third of its value, because its memorized world predated a currency collapse. Data that does surface in search is polluted with scam listings and aspirational prices. Payment rails that every Stripe tutorial assumes simply don't apply.

So you build differently. Retrieval isn't an optimization here; it's load-bearing. Domain guardrails aren't nice-to-haves; they're the only thing standing between you and confidently wrong output. Every architectural decision that's optional in a data-rich market is mandatory in a data-poor one. The upside is that these constraints force you to build the robust version of the system on day one, the version that data-rich builders only discover they need after their first embarrassing failure.

Who should be building here

If you're a developer in Lagos, Nairobi, Karachi, or Jakarta, you have an unfair advantage that no one in San Francisco can replicate: you can tell when the model is wrong. You know the street price, the negotiation dance, the scam patterns, the real exchange rate. Ground truth lives in your head, and in markets without datasets, ground truth in someone's head is the scarcest resource in the entire stack.

The standard advice says to copy proven Western products for your local market. I'd argue the opposite is now more valuable: build the measurement layer your market never had. Not another marketplace on top of chaos, but the price signal, the quality score, the trust index that makes the chaos legible. Marketplaces come and go. Reference data compounds.

The next billion users aren't waiting for translated versions of solved products. They're living inside unsolved measurement problems, and for the first time, the tools to solve them don't require a data moat you don't have. They require judgment about a market you already understand better than the training data does.

That's not a handicap. That's the entire edge.

I build AutoValue in the open at autovalue.tech, and I write about the engineering behind it, including the pricing engine and the currency bug that nearly sank it. If you're building for a market the datasets forgot, I'd genuinely like to hear what you're measuring.

Build an AI Agent With Tool Calling in a Next.js Route Handler (Claude API)

Chichebe John — Tue, 14 Jul 2026 16:16:53 +0000

A step-by-step guide to the pattern behind AutoValue's listing agent: a conversational AI that calls real functions, keeps state across turns, and lives entirely in one Next.js route.

Most "build a chatbot" tutorials stop at text in, text out. Production agents are different: they call your functions, wait for real data, and decide what to do next based on the results. I run one in production at AutoValue, where a conversational agent walks Nigerian car sellers through a full listing flow: identifying the car from a photo, reading the odometer, pulling live market prices, and publishing the finished listing to the marketplace. This tutorial builds the core of that pattern from scratch: a tool-calling agent inside a single Next.js route handler.

You'll need Node 18+, a Next.js App Router project, and an Anthropic API key.

Step 1: Install the SDK and set your key

npm install @anthropic-ai/sdk

Put your key in .env.local so Next.js keeps it server-side:

ANTHROPIC_API_KEY=sk-ant-...

The SDK reads that variable automatically, so the client needs no arguments.

Step 2: Define a tool

A tool is a JSON Schema description of a function the model may call. The model never executes anything itself; it returns a structured request, and your code runs the function. Here's a simplified version of AutoValue's pricing tool:

import Anthropic from "@anthropic-ai/sdk";

const tools: Anthropic.Tool[] = [
  {
    name: "get_market_price",
    description:
      "Get the current Nigerian market price range for a specific car. " +
      "Call this whenever the user asks what a car is worth, or before " +
      "suggesting a listing price. Do not estimate prices from memory.",
    input_schema: {
      type: "object",
      properties: {
        make: { type: "string", description: "e.g. Toyota" },
        model: { type: "string", description: "e.g. Camry" },
        year: { type: "integer", description: "e.g. 2015" },
        mileage_km: {
          type: "integer",
          description: "Odometer reading in kilometres, if known",
        },
      },
      required: ["make", "model", "year"],
    },
  },
];

Two lessons from production hiding in that description. First, say when to call the tool, not just what it does. Models decide from the description, and "call this whenever the user asks what a car is worth" measurably beats a bare "gets car prices." Second, the line "do not estimate prices from memory" exists because a model's memorized prices can be years stale. In a volatile economy that's a 3x error. Make the tool the only allowed source of truth for anything that changes fast.

Step 3: The route handler with the agent loop

The Claude API is stateless: you send the full conversation every time. The agent loop is simple in shape: call the model, and if it stops because it wants a tool (stop_reason: "tool_use"), run the tool, append the result, and call again. When it stops for any other reason, you have your answer.

app/api/agent/route.ts:

import Anthropic from "@anthropic-ai/sdk";
import { NextResponse } from "next/server";

const client = new Anthropic();

const SYSTEM_PROMPT =
  "You are AutoValue's listing assistant. You help Nigerian car sellers " +
  "price and list their cars. Always base prices on the get_market_price " +
  "tool, never on memory. Be concise and friendly.";

async function runTool(name: string, input: unknown): Promise<string> {
  if (name === "get_market_price") {
    const { make, model, year } = input as {
      make: string;
      model: string;
      year: number;
    };
    // In production this hits our search pipeline (Serper → Google CSE)
    // and a Supabase anchor table. Stubbed here:
    return JSON.stringify({
      currency: "NGN",
      low: 8_500_000,
      high: 10_200_000,
      source: "live market listings",
      car: `${year} ${make} ${model}`,
    });
  }
  throw new Error(`Unknown tool: ${name}`);
}

export async function POST(req: Request) {
  const { messages } = (await req.json()) as {
    messages: Anthropic.MessageParam[];
  };

  const history: Anthropic.MessageParam[] = [...messages];

  while (true) {
    const response = await client.messages.create({
      model: "claude-opus-4-8",
      max_tokens: 16000,
      thinking: { type: "adaptive" },
      system: SYSTEM_PROMPT,
      tools,
      messages: history,
    });

    // Append the FULL content array, not just the text.
    // It contains tool_use and thinking blocks the API needs back.
    history.push({ role: "assistant", content: response.content });

    if (response.stop_reason !== "tool_use") {
      const reply = response.content
        .filter((b): b is Anthropic.TextBlock => b.type === "text")
        .map((b) => b.text)
        .join("");
      return NextResponse.json({ reply, messages: history });
    }

    // The model may request several tools in one turn. Run them all,
    // then return ALL results in a single user message.
    const toolResults: Anthropic.ToolResultBlockParam[] = [];
    for (const block of response.content) {
      if (block.type !== "tool_use") continue;
      try {
        const result = await runTool(block.name, block.input);
        toolResults.push({
          type: "tool_result",
          tool_use_id: block.id,
          content: result,
        });
      } catch (err) {
        toolResults.push({
          type: "tool_result",
          tool_use_id: block.id,
          content: err instanceof Error ? err.message : String(err),
          is_error: true,
        });
      }
    }
    history.push({ role: "user", content: toolResults });
  }
}

The frontend just POSTs { messages } and stores the returned messages array for the next turn. That's the whole state model: the client holds the transcript, the server holds nothing.

Step 4: The four bugs you will hit (because I hit them)

1. Appending only the text. If you push { role: "assistant", content: replyText } instead of the full response.content, you drop the tool_use blocks, and the API rejects the next request because your tool_result references an ID that no longer exists in history. Always append the whole content array.

2. Splitting parallel tool results. The model can request three tools in one turn. If you send each result as its own user message, the request may fail on role alternation, and worse, you quietly teach the model to stop parallelizing. All results go in one user message.

3. Swallowing tool errors. If a tool throws and you skip its result, the API errors on the dangling tool_use. Return a tool_result with is_error: true instead; the model reads the error and adapts, often by asking the user a clarifying question, which is exactly what you want.

4. Trusting the model to carry state. In AutoValue's six-phase flow we learned that data the model collected in phase one (like body type) doesn't reliably survive to a tool call in phase three if you expect the model to re-emit it as a tool argument. Accumulate authoritative state in your own code and inject it into tool executions yourself. The model drives the conversation; your code owns the facts.

Where to go from here

The SDK can run the loop for you. client.beta.messages.toolRunner() with betaZodTool handles the call-execute-append cycle automatically. I've shown the manual loop because you should understand what the runner does before you let it do it.
Streaming: swap create for client.messages.stream() and forward text deltas to the client for a typing effect; stream.finalMessage() gives you the same object the loop needs.
Cost: for high-traffic routes, a smaller model like claude-haiku-4-5 handles narrow, well-prompted tool flows at a fraction of the cost. AutoValue runs different models for different jobs: a big model for the conversational agent, a small one for the pricing calculator.

The full production version of this pattern (vision-based car identification, odometer OCR, photo phase management, publish flow) runs at autovalue.tech. I wrote about the pricing engine behind the get_market_price tool, and the currency bug that nearly sank it, in my previous article.

The LLM Thought a Dollar Was Still ₦450: Building a Car Pricing Engine for a Market With No Data

Chichebe John — Tue, 14 Jul 2026 10:56:09 +0000

How I built an AI valuation engine for Nigerian used cars, and what it taught me about why you should never let a language model price anything on its own.

The problem: a market where nobody knows the price

In the US, if you want to know what a 2018 Toyota Camry is worth, you check Kelley Blue Book. There is no Kelley Blue Book for Nigeria. Used cars here trade through an informal economy: dealer clusters, WhatsApp groups, and listing sites where the asking price is an opening move in a negotiation, not a market value. Two identical cars can be listed ₦8 million apart on the same site, on the same day.

I'm building AutoValue, a platform that gives Nigerian sellers a data-backed valuation from photos of their car. The AI parts you'd expect worked early: a vision model identifies the car from a photo, reads the odometer, and scores the condition. The part that nearly sank the product was the one that looked easiest. Asking an LLM: "what is this car worth in Nigeria?"

The bug: every car costs ₦22 million

The first version did the obvious thing. Collect the car's details, hand them to a fast model (Claude Haiku), and ask for a Nigerian market price. In testing it seemed plausible. Then a pattern emerged that no prompt tweak would kill:

Almost every car priced out between ₦20 and 25 million, regardless of what it was.

A 2020 Land Cruiser Prado, a ₦70 to 95 million car in today's market, came back at ₦21 million. Budget sedans and luxury SUVs drifted toward the same band. The model wasn't hallucinating randomly. It was being wrong in a suspiciously consistent way, and consistent wrongness always has a cause.

The root cause: LLMs freeze the economy at training time

Here's the cause, and it's the most useful thing in this article if you're building LLM products for any volatile economy.

The model's training data was full of Nigerian car prices from when the naira traded around ₦450 to the dollar. By the time I was building, the street rate was around ₦1,500. The naira had lost roughly 70% of its dollar value, and nearly every used car in Nigeria is an import whose price tracks dollars. So the model wasn't guessing badly. It was recalling accurately from a world that no longer existed, and quoting it into one where every number had tripled.

No prompt fixes this. You can tell the model "the exchange rate is now ₦1,500/$" and it will acknowledge the fact and then keep leaning on thousands of memorized price examples denominated in the old world. Training data isn't a fact the model can bracket off. It's the water the model swims in.

This generalizes beyond Nigeria and beyond cars. If your product asks an LLM for prices, salaries, rents, or costs in any economy with meaningful inflation or currency movement, the model is answering from the past, with total confidence, in fluent and convincing prose.

The fix: demote the model from oracle to calculator

The fix was not better prompting. It was changing the model's job description.

The current architecture works like this:

Get a real number first. Before the LLM sees anything, the system searches the live web for the actual asking prices of that make, model, and year in Nigeria right now. I use Serper (a Google Search API) as the primary engine, with Google Custom Search as fallback, scoped toward Nigerian listing sites. The result becomes the base price anchor.
Make the anchor mandatory, not advisory. The search result is injected into the prompt as a hard constraint: this is the base price, from live data, for a base-model car in good condition at average mileage. Not "for context." The base.
The LLM only applies adjustments. The system prompt no longer asks "what is this car worth?" It says, in effect: you are a pricing calculator. Here is the base price from live market data. Apply percentage adjustments for this car's mileage, condition, verified faults, and location, and show your work.

The model went from source of truth to arithmetic-with-judgment, which is what it's actually good at. Depreciation for high mileage, a discount for a rough interior, a premium for a clean one: those relative adjustments are stable knowledge that doesn't expire the way absolute prices do.

Accuracy went from unusable to defensible in one architectural change.

The second bug: the cache that poisoned itself

Live search costs time and API calls, so every resolved price gets saved to a car_price_anchors table in Supabase. Next time anyone prices a 2018 Lexus RX 350, the anchor is already there. The table is effectively a self-training price database: the more cars users price, the less searching the system needs.

Then prices went wrong again, and this bug was self-inflicted.

Early on, when search returned nothing, I let the model's own estimate be saved as an anchor, labeled claude_estimate. It felt harmless, a placeholder until real data arrived. But on the next request for that car, the pipeline found an existing anchor, skipped the live search, and injected the model's old guess back into the prompt as if it were market data. The model was seasoning its own hallucination and being told it was ground truth. Wrong prices weren't just persisting. They were compounding.

The fix was one rule with a big lesson inside it: provenance beats existence. Every anchor carries its source, and model-generated anchors are now bypassed at read time, which forces a fresh live search to overwrite them. Trust ranking: manually verified > live search > nothing. A model's own output never re-enters its context labeled as data.

If you're building anything where LLM outputs get cached and might be fed back in, label the provenance of every stored value and decide explicitly what the model is allowed to see again. Feedback loops in LLM systems don't announce themselves. They just slowly replace your data with the model's opinion of itself.

The last line of defense: domain floors

Search results have their own failure modes: a scam listing, a clickbait price, a snippet quoting the wrong trim. So the final guard is dumb and effective: a table of price floors by brand and year. If an anchor claims a 2018 Range Rover Velar costs less than ₦35 million, the anchor is rejected as implausible, because no real Velar in Nigeria does. It's the kind of rule a human dealer applies without thinking, encoded as the system's sanity check.

What I'd tell anyone building LLM pricing features

LLM knowledge has an expiry date, and prices expire fastest. In a stable economy the staleness hides. In a volatile one it's a 3-4x error delivered confidently.
Retrieval must be mandatory, not decorative. If live data is "context," the model can ignore it. Make it the base the model isn't allowed to override.
Give the model relative judgments, not absolute recall. "Adjust this verified base for mileage and condition" survives inflation. "What does this cost?" does not.
Track provenance on everything you cache. The moment a model's output can re-enter its own context as "data," you have a poisoning loop.
Keep one layer of dumb domain rules. Floors and plausibility checks catch the failures your clever layers create.

The next challenge is pricing cars with almost no local listing data at all: Chinese brands like Changan and Chery are entering Nigeria faster than resale data about them exists, so the anchor itself has to be synthesized from new-price and import-cost signals. Thin-data pricing is its own problem, and probably its own article.

I'm building AutoValue in the open at autovalue.tech. If you're working on LLM products for markets the training data forgot, I'd genuinely like to compare notes.