You can unit test your code. But how do you test your prompts?

I've been building a web app to manage rolling stock - a preparedness habit where you rotate everyday household food items as emergency supplies, rather than keeping a separate untouched stash.
The app is simple. You take a photo of a product label, and an LLM (currently using Gemini Flash Lite) automatically extracts the product name and expiration date. You can see how many days are left until expiration - enough to know when something needs to be eaten or when to restock.
The implementation went surprisingly smoothly. The LLM's recognition accuracy was more than sufficient. While building it, I thought, "This is pretty solid - the core feature is basically done."
but I was wrong.

A Gap I Found During Validation

While validating the app with real product labels, I noticed something odd.
A label that only showed "August 2026" was being registered as August 1st, 2026.
In Japan, the convention for expiration dates is that a year-month-only format means the product is valid until the last day of that month. So the correct value should be August 31st, 2026. But the LLM was returning the minimum valid date - the 1st.
For users who know this convention, this means manually correcting the date every single time. For users who don't, it could cause them to rotate food stocks weeks earlier than necessary.

Deciding on a Fix

I considered two approaches.
Option A - Instruct via prompt: Tell the LLM to return the last day of the month. The concern of this approach is trusting the LLM with leap year and 30/31-day logic feels risky. LLM outputs are probabilistic - even if it appears to work during testing, I can't be fully confident it will do so 100% of the time in practice.
Option B - Handle in code: Have the LLM return YYYY-MM when only year and month are available, and calculate the last day in code. The concern of this approach is the impact of changing the prompt is unclear.
I went with Option B. Calculating the last day of a month is deterministic logic - there's no reason to leave it to a probabilistic model. On top of that, finding real products with leap-year expiration dates to test with would itself be a challenge.
So I kept the prompt change to a single line:

// Before
"expiry_date": "Date in YYYY-MM-DD format"

// After
"expiry_date": "Return YYYY-MM-DD if the full date is known, or YYYY-MM if only year and month are available"

And handled the rest in code:

const normalizeExpiryDate = (date: string): string => {
  if (/^\d{4}-\d{2}$/.test(date)) {
    const [year, month] = date.split('-').map(Number);
    const lastDay = new Date(year, month, 0).getDate();
    return `${date}-${String(lastDay).padStart(2, '0')}`;
  }
  return date;
};

A clean fix. But this is where the real problem emerged.

How Do You Confirm "It's Fixed"?

I changed one line of the prompt. But then a new challenge surfaced: how do you actually verify that the updated prompt works correctly?
The natural answer is to re-photograph every product and check manually. But that approach has serious problems.
If the updated prompt still fails on some cases, I have to revise it again and restart testing from scratch. To build confidence, I'd need more test cases. And since LLM outputs are probabilistic, even the same image with the same prompt can produce different results - meaning I'd need multiple test runs per product. How many times is enough?
Repeating this entire process every time I tweak a prompt is not realistic. And this, I believe, is a universal challenge in building apps that use LLMs.
You can unit test your code. But how do you test a prompt change?

Solving It with PromptProof

That's where I used PromptProof - a platform for statistically validating LLM prompt accuracy. By integrating it into your app, you can post LLM inputs and outputs to the platform, manually annotate ground truth labels, and reuse that dataset to run prompt experiments.
Here's how I used it:
Post product images and extracted data from the app (photos taken just once)
Annotate ground truth labels in PromptProof (for year-month-only labels, the last day of the month is the correct answer)
Run experiments with both the original and revised prompts
Compare accuracy with confidence intervals

The Results - and a Surprise

The results before the prompt change was like following.

• Overall accuracy: 81% (confidence interval: 78–84%)
• Product name extraction: 71% (100/140 correct) [CI: 63–78%]
• Expiration date extraction: 57% (80/140 correct) [CI: 49–65%]
• Expiration type: 100% (140/140 correct) [CI: 97–100%]

And after changed prompt was following.

• Overall accuracy: 90% (confidence interval: 88–93%)
• Product name extraction: 57% (80/140 correct) [CI: 49–65%] ⬇️
• Expiration date extraction: 100% (140/140 correct) [CI: 97–100%] ⬆️
• Expiration type: 100% (140/140 correct) [CI: 97–100%]

The fix achieved its goal. Expiration date accuracy went from 57% to 100%. The confidence intervals - [49–65%] before versus [97–100%] after - don't overlap at all. A clear, statistically significant improvement.
But something else happened, product name accuracy dropped from 71% to 57%.
Looking at the per-field confidence intervals tells a more nuanced story,
Before: 71% [63–78%]
After: 57% [49–65%]

The intervals overlap slightly between 63% and 65%, so we can't call this regression statistically significant - not yet. More samples are needed to say so with confidence. That said, a 14-point decline is hard to ignore. The direction is clear even if the certainty isn't. This is a concrete hypothesis to investigate next, and precisely the kind of thing that gets missed without statistical tooling.
Reviewing the outputs gave a clue: for well-known products, some results included the manufacturer name and others didn't. The root cause wasn't the prompt change itself - it was that the prompt had never clearly specified what "product name" should include. The experiment surfaced a gap that had always been there.
If I had validated by manually re-photographing products, I might have caught the improvement in expiration date accuracy. But this regression would likely have gone unnoticed entirely.

A Bonus: Token and Latency Insights

Accuracy isn't the only thing PromptProof surfaces. The latency and token usage distributions turned out to be equally useful.
Output tokens were remarkably stable - P50 of 48, max of 51 across 140 trials. That tells you exactly how much output budget to reserve when configuring max_tokens. No guesswork.
Latency was more interesting. The first prompt showed a P50 of 1.42s but a max spike of 4.38s - a 3x gap worth knowing about. After the prompt change, the max dropped to 2.56s despite slightly longer input tokens (1.1k → 1.2k).
When evaluating a new model, this kind of distribution data answers two practical questions before you commit:
Is the latency acceptable for your use case? A P50 of 1.5s might be fine for a background job, but not for a real-time interaction.
What's the minimum output token budget you actually need? Over-provisioning max_tokens wastes money; under-provisioning causes silent truncation failures.

Running experiments before locking in a model means you're making that decision with data, not intuition.

Conclusion

Building apps that use LLMs has made me feel that validation is genuinely harder than implementation.
Even a single-line prompt change can have wide-ranging effects. Being able to reduce the cost of re-testing and say with statistical confidence that "this change was correct" - and to do so without friction - is what it takes to build LLM-powered apps that are reliable enough to ship.

Note

PromptProof, the platform I used in this article, is a service I'm building myself. It's free to get started - if you're facing similar challenges, I'd love for you to try it out.
👉 PromptProof