DEV Community: KNALLHART.DEV

Getting an LLM to Actually Follow Your Output Format (Without Fighting It Every Request)

KNALLHART.DEV — Fri, 26 Jun 2026 18:10:39 +0000

If you've ever asked an LLM to return output in a strict format —
valid JSON, a specific HTML structure, exactly N items — you've
probably noticed it drifts. Not constantly, but often enough that
"mostly works" isn't good enough for production code parsing the result.

I ran into this building a tool that sends a website screenshot to
Gemini and expects back a strict HTML structure: an ordered list,
each item with a specific tag layout, nothing extra.

What kept breaking

Early versions of my prompt just described the desired format in
prose: "format the response as an HTML list with this structure."
That worked maybe 80% of the time. The other 20%:

Extra commentary before or after the list ("Here's my analysis:")
The <em> tag meant only for one specific line showing up elsewhere in the output, sometimes even written out as literal visible text
Two distinct issues merged into a single list item
Occasionally a missing <li> entirely

None of these are "the model is bad." They're the model treating a
descriptive request as a soft suggestion rather than a hard constraint.

What actually fixed it

1. Validate the output programmatically, don't trust it.

const roastHtml = result.response.text().trim();

if (!roastHtml.includes("<li>")) {
  throw new Error("Unexpected format — triggering retry");
}

This alone changes the failure mode from "silently broken downstream"
to "automatically retried." A simple structural check (does the
expected tag exist) catches most drift without needing to validate
every detail.

2. Be explicit about what NOT to do, not just what to do.

Positive instructions ("format it like this") leave room for
interpretation. Adding explicit negative constraints closes the gaps:

FORMAT RULES — these are strict:
- The <em> tag is used ONLY for the Fix line at the end of each 
  point — never in the description
- Do not write tag names as visible text anywhere
- Do not add any introduction, conclusion, or commentary outside 
  the list
- Do not add extra HTML attributes, classes, or styles

The difference between "use em tags for fixes" and "never use em tags
anywhere else, and never write them as visible text" is the difference
between a suggestion and a constraint, even though both describe the
same intended behavior.

3. Retry on validation failure, with backoff.

async function withRetry(fn, label, maxAttempts = 3) {
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (attempt === maxAttempts) throw err;
      await new Promise((r) => setTimeout(r, attempt * 5000));
    }
  }
}

Format drift is usually not consistent — the same prompt against the
same input often succeeds on a second attempt. Treating a format
mismatch as a retryable error, the same way you'd treat a network
timeout, costs almost nothing and fixes most of the remaining cases.

The underlying pattern

Soft, descriptive instructions get treated as suggestions, not
requirements — even when they're logically necessary for the output
to be usable. The model needs the constraint stated as a constraint,
and your code needs to verify compliance rather than assume it.
Neither piece alone was enough; the prompt changes reduced drift,
but the validation + retry is what makes the pipeline actually reliable
end to end.

If you're building anything that parses LLM output downstream:
validate structurally, state constraints as negatives as well as
positives, and treat format mismatches as retryable failures rather
than edge cases to patch around later.

The Stripe Webhook Gotcha Nobody Warns You About: 100% Off Coupons and Missing PaymentIntents

KNALLHART.DEV — Thu, 25 Jun 2026 13:56:46 +0000

If you've ever added a 100%-off coupon to a Stripe Checkout flow
and then watched your webhook handler silently do nothing, you've
run into a gotcha that isn't well documented anywhere obvious.

The setup

I was building a small test program: give a few people free access
to a paid product via a Stripe Promotion Code, so they could go
through the real checkout flow (price, decision, everything) instead
of skipping straight to the result.

The webhook listens for checkout.session.completed, like every
Stripe tutorial tells you to. Worked perfectly for paid orders.
Free orders went through Checkout fine on the frontend... and then
nothing happened on the backend.

The actual cause

Two details that aren't obvious until you hit them:

checkout.session.completed still fires for €0 sessions — that part is fine, no surprises there.
But the session looks different. A free Checkout Session has no associated PaymentIntent (there's nothing to charge, so Stripe never creates one), and session.payment_status is "no_payment_required" instead of "paid".

If your webhook handler has any logic like:

if (session.payment_status === "paid") {
  // fulfill order
}

...free orders get silently ignored. No error, no log, nothing —
the condition is just false and the code below it never runs.

The fix

Don't gate fulfillment on payment_status === "paid". Treat
"paid" and "no_payment_required" as equally valid completion
states:

const validStatuses = ["paid", "no_payment_required"];

if (validStatuses.includes(session.payment_status)) {
  // fulfill order
}

And if you need the PaymentIntent ID for anything downstream
(refund logic, in my case), just make sure your code tolerates
it being null or undefined rather than assuming it always exists:

const paymentIntentId = session.payment_intent ?? null;

// later, somewhere that might issue a refund:
if (paymentIntentId) {
  await stripe.refunds.create({ payment_intent: paymentIntentId });
}
// if it's null, there's nothing to refund anyway — skip silently

Why this matters beyond coupons

This isn't just a coupon edge case. Anything that can legitimately
bring a Checkout Session total to zero — full discounts, free trial
codes, internal test accounts — hits this same gap. If your
fulfillment logic only recognizes "paid", every one of those
paths fails silently, and you won't notice until someone reports
"I redeemed the code but never got anything."

Costly to debug after the fact, free to avoid if you know about
it upfront.

If you're testing your own coupon flow: don't trust that webhook
silence means "nothing happened." Check payment_status directly
on a real free-session payload before assuming your handler covers it.

The scrollHeight Lie: How I Finally Got Full-Page Screenshots Right with Playwright

KNALLHART.DEV — Sun, 21 Jun 2026 16:59:14 +0000

If you've ever tried to take a full-page screenshot of a modern website
programmatically, you've probably run into the same wall I did:
document.body.scrollHeight lies. Constantly.

The problem

Lazy-loaded images, infinite scroll sections, sticky headers, scroll-triggered
animations — all of these mess with the page's reported height in ways that
make scrollHeight an unreliable signal for "have I reached the bottom of
this page yet?"

I was building knallhart.dev, a tool that takes a
full-page screenshot of any website and sends an AI-generated critique via
email. The screenshot step turned out to be the hardest part of the entire
build — harder than the AI integration, harder than the payment flow.

What didn't work

My first approach was the obvious one: read scrollHeight, calculate how
many scroll steps are needed, scroll that many times, done.

const pageHeight = await page.evaluate(() => document.body.scrollHeight);
const steps = Math.ceil(pageHeight / 600);

for (let i = 0; i < steps; i++) {
  await page.evaluate((y) => window.scrollBy(0, y), 600);
  await page.waitForTimeout(200);
}

This worked on simple static pages and broke immediately on anything modern.
Pages with lazy-loaded content report a short initial scrollHeight, then
grow as you scroll — so the calculated step count is wrong before you've
even started. Pages with infinite scroll never stop growing at all.

What actually works

Instead of trying to calculate the destination upfront, I scroll step by
step and watch whether window.scrollY is still changing. If it stops
changing for a few consecutive checks, I've actually hit the bottom —
regardless of what scrollHeight claims.

let lastScrollY = -1;
let noChangeCount = 0;
const maxAttempts = 40; // safety cap, ~8 seconds

for (let i = 0; i < maxAttempts; i++) {
  await page.evaluate(() => window.scrollBy(0, 600));
  await page.waitForTimeout(200);

  const currentScrollY = await page.evaluate(() => window.scrollY);

  if (currentScrollY === lastScrollY) {
    noChangeCount++;
    if (noChangeCount >= 3) break; // confirmed: no more progress
  } else {
    noChangeCount = 0;
    lastScrollY = currentScrollY;
  }
}

await page.evaluate(() => window.scrollTo(0, 0));

This is more robust because it doesn't trust any single height value — it
trusts observed behavior over time. A page that's still loading content
will keep moving scrollY; a page that's truly done won't, no matter how
confusing its scrollHeight is.

The bonus problem: images

Triggering the scroll isn't enough on its own — lazy-loaded images often
need a moment to actually finish loading after they enter the viewport.
I added an explicit wait for image load events before taking the final
screenshot:

\\javascript
await page.evaluate(async () => {
const images = Array.from(document.querySelectorAll("img"));
await Promise.all(
images.map((img) => {
if (img.complete) return Promise.resolve();
return new Promise((resolve) => {
img.addEventListener("load", resolve);
img.addEventListener("error", resolve);
setTimeout(resolve, 5000); // don't hang forever on one broken image
});
})
);
});
\\

What I'd tell past me

Don't trust any single DOM measurement as an endpoint signal on a page you
don't control. Modern websites are dynamic enough that almost any static
value can lie to you at some point. Behavior over time — does this keep
changing or not — is a much sturdier signal than asking the DOM "are we
there yet?"

If you're curious what this screenshot pipeline turned into:
knallhart.dev — it roasts your website with AI
and emails you three things that are actually wrong with it.

Happy to go deeper on any part of this if useful.

I built a €10 AI service that brutally roasts your website

KNALLHART.DEV — Thu, 11 Jun 2026 16:54:40 +0000

I've been building websites for clients for years. And every time
I looked at their sites, I saw the same problems — unclear CTAs,
hero sections that try to say everything at once, cookie-cutter
layouts that put the booking form three scrolls deep.

The feedback I'd give them privately was always more honest than
what anyone would write in a formal audit. So I built a tool that
does exactly that.

What it does

knallhart.dev does one thing:

You submit a URL and your email
Pay €10 via Stripe
Our service takes a full-page Playwright screenshot
Sends it to Gemini 2.5 Flash with a brutally honest prompt
You get an email with 3 specific critiques + actionable fixes

No dashboards, no subscriptions, no 47-point audit template.
Just three things that are actually wrong and how to fix them.

The tech stack

Next.js on Vercel (frontend + webhook handling)
Playwright for full-page screenshots (headless Chromium)
Gemini 2.5 Flash for the AI analysis
Stripe for payments
Resend for email delivery

The hardest part was getting Playwright to reliably handle
lazy loading on modern sites. The trick: scroll until
window.scrollY stops changing rather than calculating
scrollHeight upfront — that value is unreliable on
too many sites.

Real example output

Here's what it said about basecamp.com:

Welcome to Visual Overload — Seriously, did a screenshot
gallery throw up on your hero section? "Refreshingly
straightforward" indeed, if "straightforward" means immediately
overwhelming me with a static, unreadable dump of your entire
application UI before I even know what Basecamp is.

Fix: Replace the dense UI screenshots with a clean,
aspirational visual that highlights benefits, not features.

The Great Wall of Founder's Text — A full-page novel
from the co-founder right after being visually assaulted.
This isn't a fireside chat; it's a product page.

Fix: Break it into scannable sections. Move the full letter
to an About page.

Testimonial Tsunami — Nine testimonials in a 3x3 grid
and then a button promising a thousand more. This isn't
social proof; it's a marathon reading challenge.

Fix: 2-3 high-impact quotes max, placed near relevant features.

What I learned

Gemini 2.5 Flash is surprisingly good at visual critique
when you give it proper context. The prompt matters a lot —
telling it this is a full-page desktop screenshot, that fixed
elements may be mispositioned, and to focus on the first 900px
as the critical first impression made a huge difference in
output quality.

Bot protection is a real problem. Large e-commerce sites
(Otto, Zalando) detect headless browsers immediately and serve
a gray placeholder. I handle this by checking screenshot file
size before sending to Gemini — anything suspiciously small
triggers an automatic Stripe refund.

Serverless + async don't mix the way you think.
On Vercel, fire-and-forget async operations get killed the
moment you return a response. You have to await the service
call even if the service itself handles things asynchronously.

Try it

If you want your site roasted: knallhart.dev

If you have questions about the build, drop them in the comments.