<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Ahmed Vaseer</title>
    <description>The latest articles on DEV Community by Ahmed Vaseer (@ahmedvaseer).</description>
    <link>https://dev.to/ahmedvaseer</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3736029%2F9b646c08-c066-49f6-8094-302d17448109.jpg</url>
      <title>DEV Community: Ahmed Vaseer</title>
      <link>https://dev.to/ahmedvaseer</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/ahmedvaseer"/>
    <language>en</language>
    <item>
      <title>A landing page build checklist for developers and marketers</title>
      <dc:creator>Ahmed Vaseer</dc:creator>
      <pubDate>Thu, 05 Nov 2026 06:12:51 +0000</pubDate>
      <link>https://dev.to/ahmedvaseer/a-landing-page-build-checklist-for-developers-and-marketers-4l0e</link>
      <guid>https://dev.to/ahmedvaseer/a-landing-page-build-checklist-for-developers-and-marketers-4l0e</guid>
      <description>&lt;p&gt;Most landing page problems aren't design problems. They're handover problems.&lt;/p&gt;

&lt;p&gt;The marketer assumes the developer set up tracking. The developer assumes the marketer tested the form. The page launches, ad spend starts, and a week later someone notices the conversions report is empty, or the form has been sending leads to an inbox nobody reads.&lt;/p&gt;

&lt;p&gt;A shared checklist fixes most of this. Not a long one: four areas, each with a few checks, and a name next to each. Here's the one I'd want both teams to sign before any paid traffic goes to a page.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Speed
&lt;/h3&gt;

&lt;p&gt;A slow landing page wastes the money spent getting people to it. Paid visitors are the least patient visitors you have.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Test on a mid-range phone over mobile data&lt;/strong&gt;, not just a fast laptop on office Wi-Fi. Use Lighthouse or PageSpeed Insights, and treat the field data (real users) as more important than the lab score.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Size images for where they're shown.&lt;/strong&gt; Serve modern formats (WebP or AVIF) and set width and height so the layout doesn't jump.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Lazy-load anything below the fold&lt;/strong&gt;, but never the main hero image.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Audit third-party scripts.&lt;/strong&gt; Chat widgets, heatmaps and old pixels add up. Each one should have an owner and a reason to be there.
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="c"&gt;&amp;lt;!-- Hero image: load early, reserve space --&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;img&lt;/span&gt; &lt;span class="na"&gt;src=&lt;/span&gt;&lt;span class="s"&gt;"hero.webp"&lt;/span&gt; &lt;span class="na"&gt;width=&lt;/span&gt;&lt;span class="s"&gt;"1200"&lt;/span&gt; &lt;span class="na"&gt;height=&lt;/span&gt;&lt;span class="s"&gt;"630"&lt;/span&gt; &lt;span class="na"&gt;alt=&lt;/span&gt;&lt;span class="s"&gt;"Short description of the image"&lt;/span&gt; &lt;span class="na"&gt;fetchpriority=&lt;/span&gt;&lt;span class="s"&gt;"high"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;

&lt;span class="c"&gt;&amp;lt;!-- Below the fold: load when needed --&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;img&lt;/span&gt; &lt;span class="na"&gt;src=&lt;/span&gt;&lt;span class="s"&gt;"feature.webp"&lt;/span&gt; &lt;span class="na"&gt;width=&lt;/span&gt;&lt;span class="s"&gt;"600"&lt;/span&gt; &lt;span class="na"&gt;height=&lt;/span&gt;&lt;span class="s"&gt;"400"&lt;/span&gt; &lt;span class="na"&gt;alt=&lt;/span&gt;&lt;span class="s"&gt;"Short description"&lt;/span&gt; &lt;span class="na"&gt;loading=&lt;/span&gt;&lt;span class="s"&gt;"lazy"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Who signs:&lt;/strong&gt; developer runs the test; marketer confirms no script they need was removed.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Forms
&lt;/h3&gt;

&lt;p&gt;The form is the whole point of the page. Test it like it is.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Submit a real test entry&lt;/strong&gt; on desktop and mobile, then confirm it arrives where it should: the CRM, the right inbox, the right list. Check that someone is actually watching that destination.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Ask for as little as you need.&lt;/strong&gt; Every extra field costs completions. If sales doesn't use a field, remove it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Make errors helpful.&lt;/strong&gt; Validate inline, say what's wrong in plain words, and never clear the whole form on one mistake.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Use the right input types&lt;/strong&gt; so phones show the right keyboard, and support autofill.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Protect against spam&lt;/strong&gt; without punishing real people: a honeypot field or server-side checks before a hard captcha.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Show a clear confirmation&lt;/strong&gt;, ideally a thank-you page, which also makes conversion tracking simpler.
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;label&lt;/span&gt; &lt;span class="na"&gt;for=&lt;/span&gt;&lt;span class="s"&gt;"email"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;Work email&lt;span class="nt"&gt;&amp;lt;/label&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;input&lt;/span&gt; &lt;span class="na"&gt;id=&lt;/span&gt;&lt;span class="s"&gt;"email"&lt;/span&gt; &lt;span class="na"&gt;name=&lt;/span&gt;&lt;span class="s"&gt;"email"&lt;/span&gt; &lt;span class="na"&gt;type=&lt;/span&gt;&lt;span class="s"&gt;"email"&lt;/span&gt; &lt;span class="na"&gt;autocomplete=&lt;/span&gt;&lt;span class="s"&gt;"email"&lt;/span&gt; &lt;span class="na"&gt;required&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;

&lt;span class="nt"&gt;&amp;lt;label&lt;/span&gt; &lt;span class="na"&gt;for=&lt;/span&gt;&lt;span class="s"&gt;"phone"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;Phone (optional)&lt;span class="nt"&gt;&amp;lt;/label&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;input&lt;/span&gt; &lt;span class="na"&gt;id=&lt;/span&gt;&lt;span class="s"&gt;"phone"&lt;/span&gt; &lt;span class="na"&gt;name=&lt;/span&gt;&lt;span class="s"&gt;"phone"&lt;/span&gt; &lt;span class="na"&gt;type=&lt;/span&gt;&lt;span class="s"&gt;"tel"&lt;/span&gt; &lt;span class="na"&gt;autocomplete=&lt;/span&gt;&lt;span class="s"&gt;"tel"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Who signs:&lt;/strong&gt; marketer confirms fields and destination; developer confirms validation, spam protection and error states.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Tracking
&lt;/h3&gt;

&lt;p&gt;If you can't measure the page, you can't improve it, and you can't defend the budget behind it.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Agree the conversion event before the build&lt;/strong&gt;: what counts as a conversion, and where it fires (the thank-you page or a successful submit, not a button click).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Test every tag&lt;/strong&gt; with the platform's debug or preview mode, and confirm the conversion appears in the ad platform and analytics.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Check UTM parameters survive&lt;/strong&gt; redirects and form submissions, so leads arrive with their source attached.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Respect consent.&lt;/strong&gt; Where consent rules apply, tags should wait for it, and the banner should actually work.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Write down what's installed&lt;/strong&gt;: one short list of tags, what each one does and who owns it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Who signs:&lt;/strong&gt; marketer confirms the events and the reports; developer confirms the tags fire once, in the right place.&lt;/p&gt;

&lt;h3&gt;
  
  
  4. Accessibility
&lt;/h3&gt;

&lt;p&gt;An accessible page reaches more people, and the fixes usually make the page better for everyone.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Every image has meaningful alt text&lt;/strong&gt;, or empty alt text if it's purely decorative.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Colour contrast passes&lt;/strong&gt; for text and buttons, including text on images.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Every form field has a visible label&lt;/strong&gt;, not just placeholder text that disappears when you type.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The page works with a keyboard alone&lt;/strong&gt;: you can reach and use every link, field and button, and you can see where focus is.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Headings are in order&lt;/strong&gt; and buttons say what they do ("Get my quote", not "Submit").&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Automated checkers catch some of this. A quick keyboard-only run-through catches more.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Who signs:&lt;/strong&gt; developer.&lt;/p&gt;

&lt;h3&gt;
  
  
  The sign-off
&lt;/h3&gt;

&lt;p&gt;Put the checklist in a shared doc or ticket with three columns: the check, who owns it, and a tick. Nothing goes live and no paid traffic starts until every line is ticked.&lt;/p&gt;

&lt;p&gt;It sounds like process for the sake of process. In practice, it's ten minutes that saves you from finding out a week too late that the page was never working.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Ahmed Vaseer is Founder &amp;amp; CEO - Ingenious Concepts. 80+ projects, ISO 27001 certified via Global CB.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>marketing</category>
      <category>a11y</category>
      <category>performance</category>
    </item>
    <item>
      <title>Human in the loop by design: where the approval step goes</title>
      <dc:creator>Ahmed Vaseer</dc:creator>
      <pubDate>Wed, 04 Nov 2026 06:06:20 +0000</pubDate>
      <link>https://dev.to/ahmedvaseer/human-in-the-loop-by-design-where-the-approval-step-goes-13o4</link>
      <guid>https://dev.to/ahmedvaseer/human-in-the-loop-by-design-where-the-approval-step-goes-13o4</guid>
      <description>&lt;p&gt;"Human in the loop" is one of those phrases everyone agrees with and almost nobody designs for.&lt;/p&gt;

&lt;p&gt;In most AI projects, the human step is added at the end, usually after something goes wrong. Someone notices an odd email went out, or a wrong figure landed in a report, and the fix is "let's have someone check it first". Then that someone becomes a bottleneck, starts approving things without reading them, and the check stops being a check.&lt;/p&gt;

&lt;p&gt;The approval step works when it's designed in from the start, like any other part of the system. That means answering three questions before anything is built: where does the human sit, who are they, and what do they see?&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Put the check where a mistake becomes expensive
&lt;/h3&gt;

&lt;p&gt;Not every step needs a person. If an AI drafts a reply and a person reads it before sending, the draft step doesn't need its own approval. The sending step does.&lt;/p&gt;

&lt;p&gt;A useful rule: put the human at the point where the work leaves your control. That's usually one of these moments:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Before anything goes out to a customer&lt;/strong&gt;: emails, quotes, messages, published content.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Before money moves&lt;/strong&gt;: refunds, discounts, payments, orders.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Before records change&lt;/strong&gt;: updating a customer file, closing a ticket, deleting data.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Before access changes&lt;/strong&gt;: adding a user, sharing a file, connecting a new app.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Everything before that point can run on its own, because a wrong draft costs nothing until it's acted on. Everything after it is hard to take back.&lt;/p&gt;

&lt;p&gt;This also tells you when to remove a check. If the step only creates something internal and easy to undo, an approval there is mostly friction.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Name one owner, not "the team"
&lt;/h3&gt;

&lt;p&gt;"Someone will review it" means nobody will. Each approval point needs one named role that owns it, and a backup for when that person is away.&lt;/p&gt;

&lt;p&gt;The owner should be the person who would answer for the mistake. For customer emails, that's whoever owns the customer relationship, not the developer who built the workflow. For refunds, it's whoever owns the budget.&lt;/p&gt;

&lt;p&gt;Builders often skip this because it feels like an organisational question rather than a technical one. But it shapes the system: who gets notified, whose queue it lands in, and what happens if nobody responds. Decide what happens on silence, too. In most cases an unapproved item should wait, not go out by default.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Show the reviewer what they need, and nothing else
&lt;/h3&gt;

&lt;p&gt;A reviewer who has to open three tabs to understand a request will start approving on trust. Keep the approval screen to what a decision needs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;What will happen&lt;/strong&gt; if they say yes, in plain words ("Send this reply to the customer", "Refund 40 to this order").&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The output itself&lt;/strong&gt;, in full, as the recipient will see it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Why the system thinks this is right&lt;/strong&gt;: the source or rule it used.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Clear choices&lt;/strong&gt;: approve, edit, or reject, with a short reason for rejecting.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Editing matters more than people expect. If the only options are yes and no, reviewers approve "close enough" output to avoid redoing the work. Letting them fix a sentence and approve keeps quality up and gives the builders a record of what keeps needing correction.&lt;/p&gt;

&lt;h3&gt;
  
  
  4. Log every decision
&lt;/h3&gt;

&lt;p&gt;An approval that leaves no trace can't be trusted later. For each decision, record:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;what was proposed,&lt;/li&gt;
&lt;li&gt;who approved, edited or rejected it, and when,&lt;/li&gt;
&lt;li&gt;what was finally sent or changed,&lt;/li&gt;
&lt;li&gt;and the reason, if one was given.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This log does three jobs. It shows accountability when a customer asks what happened. It shows where the AI keeps getting things wrong, so the builders know what to fix. And it gives you the evidence you need to relax a check safely.&lt;/p&gt;

&lt;h3&gt;
  
  
  5. Earn the right to remove the human
&lt;/h3&gt;

&lt;p&gt;The goal isn't to keep a person in every loop forever. It's to remove them when the evidence says it's safe.&lt;/p&gt;

&lt;p&gt;If the log shows a reviewer has approved a certain type of output unchanged, over and over, for a long stretch, that step might be ready to run on its own, perhaps with spot checks instead. If edits are still frequent, it isn't ready, however good the demo looked.&lt;/p&gt;

&lt;p&gt;Make that a deliberate decision, made by the owner, with the log in front of them. Not a quiet change because the queue got long.&lt;/p&gt;

&lt;h3&gt;
  
  
  For owners: three questions to ask your builders
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;At what exact point does a person approve this before it reaches a customer, money or records?&lt;/li&gt;
&lt;li&gt;Who is that person, and what happens if they don't respond?&lt;/li&gt;
&lt;li&gt;Where can I see a record of what was approved, by whom?&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  For builders: three things to design in
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;Approval as a real state in the workflow (pending, approved, edited, rejected), not a comment in a chat.&lt;/li&gt;
&lt;li&gt;A review screen that shows the action, the output and the reason, with an edit option.&lt;/li&gt;
&lt;li&gt;A decision log you can query, not just application logs.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A human in the loop is only as good as the loop around them. Put them where mistakes get expensive, give them one clear decision to make, and keep the record. That's what turns "someone checks it" into a control you can rely on.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Ahmed Vaseer is Founder &amp;amp; CEO - Ingenious Concepts. 80+ projects, ISO 27001 certified via Global CB.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>business</category>
      <category>productivity</category>
      <category>architecture</category>
    </item>
    <item>
      <title>E-commerce checkout: technical fixes that reduce drop-off</title>
      <dc:creator>Ahmed Vaseer</dc:creator>
      <pubDate>Mon, 02 Nov 2026 06:06:08 +0000</pubDate>
      <link>https://dev.to/ahmedvaseer/e-commerce-checkout-technical-fixes-that-reduce-drop-off-5</link>
      <guid>https://dev.to/ahmedvaseer/e-commerce-checkout-technical-fixes-that-reduce-drop-off-5</guid>
      <description>&lt;p&gt;Some people will always leave a checkout. They were comparing prices, or the shipping cost surprised them, or they got distracted. That part is about pricing and policy.&lt;/p&gt;

&lt;p&gt;But a share of abandoned orders comes from things developers can fix: a slow page, a forced sign-up, a payment error nobody understands, a form that fights the customer. These are the cheapest sales to win back, because the customer had already decided to buy.&lt;/p&gt;

&lt;p&gt;Here are the four areas I'd check first.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Load time on the checkout itself
&lt;/h3&gt;

&lt;p&gt;Teams optimise the home page and product pages, then forget the checkout. It's often the heaviest page on the site, because it loads payment scripts, address lookups, fraud checks and every tracking tag marketing ever added.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Measure the checkout separately&lt;/strong&gt;, on a mid-range phone over mobile data.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Remove scripts that don't belong.&lt;/strong&gt; Chat widgets, heatmaps and review widgets rarely need to run on payment steps.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Load payment and address scripts when needed&lt;/strong&gt;, not on page load if the customer won't reach that step for a while.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Never block the page on a slow third party.&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="c"&gt;&amp;lt;!-- Load non-critical scripts without blocking the page --&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;script &lt;/span&gt;&lt;span class="na"&gt;src=&lt;/span&gt;&lt;span class="s"&gt;"/js/address-lookup.js"&lt;/span&gt; &lt;span class="na"&gt;defer&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&amp;lt;/script&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  2. Guest checkout
&lt;/h3&gt;

&lt;p&gt;Forcing account creation before payment is one of the most common reasons people give up. They came to buy something, not to start a relationship with a password.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Offer guest checkout by default.&lt;/strong&gt; Ask for an email for the receipt, nothing more.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Offer an account after the order&lt;/strong&gt;, on the confirmation page: "Save your details for next time?" At that point it's a convenience, not a barrier.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Let returning customers sign in, but don't make it the first thing they see.&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If your platform links guest orders to an account later by email, make sure that only happens after the customer verifies the address, so nobody can see someone else's orders.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Payment errors
&lt;/h3&gt;

&lt;p&gt;A declined card is stressful. A declined card with the message "Error 402" or "Something went wrong" loses the sale.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Map payment provider error codes to plain messages&lt;/strong&gt; that tell the customer what to do next.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep everything they typed&lt;/strong&gt; when a payment fails. Don't make them re-enter their address.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Prevent double charges.&lt;/strong&gt; Disable the pay button while the request is in flight, and use the provider's idempotency key feature so a retry doesn't create a second charge.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Log failures with the reason code&lt;/strong&gt; so you can see whether a pattern is a customer issue or a configuration issue. Log the reason code and order ID, never card numbers or the security code.
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Map provider decline codes to messages a customer can act on.&lt;/span&gt;
&lt;span class="c1"&gt;// Check the exact codes against your payment provider's documentation.&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;DECLINE_MESSAGES&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;card_declined&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Your bank declined this card. Try another card or contact your bank.&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;insufficient_funds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Your card was declined for insufficient funds. Try another card.&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;expired_card&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;This card has expired. Please check the date or use another card.&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;incorrect_cvc&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;The security code doesn't match. Please check the 3 or 4 digits on your card.&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;processing_error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;We couldn't reach your bank just now. Please try again in a moment.&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;paymentErrorMessage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;code&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;DECLINE_MESSAGES&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;code&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Your payment didn't go through. Please try again or use a different card.&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;payButton&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;querySelector&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;#pay-button&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;submitPayment&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;submitFn&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;payButton&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;disabled&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;          &lt;span class="c1"&gt;// prevent double clicks&lt;/span&gt;
  &lt;span class="nx"&gt;payButton&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;textContent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Processing…&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;submitFn&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;  &lt;span class="c1"&gt;// your provider's confirm call&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;result&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nx"&gt;result&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;error&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;throw&lt;/span&gt; &lt;span class="nx"&gt;result&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;error&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// some providers return errors instead of throwing&lt;/span&gt;
    &lt;span class="c1"&gt;// success: leave the button disabled and move to the confirmation page&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nf"&gt;showError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;paymentErrorMessage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;err&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;decline_code&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nx"&gt;err&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;code&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt; &lt;span class="c1"&gt;// showError: your own UI helper&lt;/span&gt;
    &lt;span class="nx"&gt;payButton&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;disabled&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;       &lt;span class="c1"&gt;// re-enable only after a failure&lt;/span&gt;
    &lt;span class="nx"&gt;payButton&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;textContent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Pay now&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  4. Form validation
&lt;/h3&gt;

&lt;p&gt;Checkout forms ask for a lot, so every bit of friction adds up.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Validate as people go&lt;/strong&gt;, when they leave a field, not only when they press the final button.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Accept input the way people type it.&lt;/strong&gt; Strip spaces from phone numbers instead of rejecting them, and use your payment provider's hosted card fields (they handle card formatting for you and keep card data off your servers). Don't force one postcode format for every country.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Use the right input types and autocomplete attributes&lt;/strong&gt; so phones show the right keyboard and browsers can fill saved details.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Show errors next to the field&lt;/strong&gt;, in plain words, and move focus to the first error. Put payment errors in a &lt;code&gt;role="alert"&lt;/code&gt; area, and set &lt;code&gt;aria-invalid="true"&lt;/code&gt; on fields with errors.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Don't ask twice.&lt;/strong&gt; Default the billing address to the shipping address.
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;label&lt;/span&gt; &lt;span class="na"&gt;for=&lt;/span&gt;&lt;span class="s"&gt;"ship-name"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;Full name&lt;span class="nt"&gt;&amp;lt;/label&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;input&lt;/span&gt; &lt;span class="na"&gt;id=&lt;/span&gt;&lt;span class="s"&gt;"ship-name"&lt;/span&gt; &lt;span class="na"&gt;name=&lt;/span&gt;&lt;span class="s"&gt;"name"&lt;/span&gt; &lt;span class="na"&gt;autocomplete=&lt;/span&gt;&lt;span class="s"&gt;"shipping name"&lt;/span&gt; &lt;span class="na"&gt;required&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;

&lt;span class="nt"&gt;&amp;lt;label&lt;/span&gt; &lt;span class="na"&gt;for=&lt;/span&gt;&lt;span class="s"&gt;"ship-postcode"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;Postcode&lt;span class="nt"&gt;&amp;lt;/label&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;input&lt;/span&gt; &lt;span class="na"&gt;id=&lt;/span&gt;&lt;span class="s"&gt;"ship-postcode"&lt;/span&gt; &lt;span class="na"&gt;name=&lt;/span&gt;&lt;span class="s"&gt;"postcode"&lt;/span&gt; &lt;span class="na"&gt;autocomplete=&lt;/span&gt;&lt;span class="s"&gt;"shipping postal-code"&lt;/span&gt; &lt;span class="na"&gt;required&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;

&lt;span class="nt"&gt;&amp;lt;label&lt;/span&gt; &lt;span class="na"&gt;for=&lt;/span&gt;&lt;span class="s"&gt;"ship-phone"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;Phone (for delivery updates)&lt;span class="nt"&gt;&amp;lt;/label&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;input&lt;/span&gt; &lt;span class="na"&gt;id=&lt;/span&gt;&lt;span class="s"&gt;"ship-phone"&lt;/span&gt; &lt;span class="na"&gt;name=&lt;/span&gt;&lt;span class="s"&gt;"phone"&lt;/span&gt; &lt;span class="na"&gt;type=&lt;/span&gt;&lt;span class="s"&gt;"tel"&lt;/span&gt; &lt;span class="na"&gt;autocomplete=&lt;/span&gt;&lt;span class="s"&gt;"tel"&lt;/span&gt; &lt;span class="na"&gt;aria-describedby=&lt;/span&gt;&lt;span class="s"&gt;"ship-phone-error"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;p&lt;/span&gt; &lt;span class="na"&gt;id=&lt;/span&gt;&lt;span class="s"&gt;"ship-phone-error"&lt;/span&gt; &lt;span class="na"&gt;class=&lt;/span&gt;&lt;span class="s"&gt;"field-error"&lt;/span&gt; &lt;span class="na"&gt;hidden&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&amp;lt;/p&amp;gt;&lt;/span&gt;

&lt;span class="nt"&gt;&amp;lt;div&lt;/span&gt; &lt;span class="na"&gt;id=&lt;/span&gt;&lt;span class="s"&gt;"payment-error"&lt;/span&gt; &lt;span class="na"&gt;role=&lt;/span&gt;&lt;span class="s"&gt;"alert"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&amp;lt;/div&amp;gt;&lt;/span&gt;

&lt;span class="nt"&gt;&amp;lt;label&amp;gt;&amp;lt;input&lt;/span&gt; &lt;span class="na"&gt;type=&lt;/span&gt;&lt;span class="s"&gt;"checkbox"&lt;/span&gt; &lt;span class="na"&gt;id=&lt;/span&gt;&lt;span class="s"&gt;"same-billing"&lt;/span&gt; &lt;span class="na"&gt;checked&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt; Billing address is the same&lt;span class="nt"&gt;&amp;lt;/label&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Tidy spaces, dashes and brackets when the customer leaves the field&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;phoneInput&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;querySelector&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;#ship-phone&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="nx"&gt;phoneInput&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;addEventListener&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;blur&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;phoneInput&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;phoneInput&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;value&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;[\s&lt;/span&gt;&lt;span class="sr"&gt;()-&lt;/span&gt;&lt;span class="se"&gt;]&lt;/span&gt;&lt;span class="sr"&gt;/g&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;""&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  How to know it worked
&lt;/h3&gt;

&lt;p&gt;Before changing anything, set up a simple funnel in your analytics: cart, checkout start, shipping, payment, order complete. Note where people drop off. Then ship one fix at a time and watch the step it should affect. Combined with your payment failure logs, that tells you which fixes are paying for themselves and where to look next.&lt;/p&gt;

&lt;p&gt;Checkout work isn't glamorous. But it's the one place on the site where every improvement lands on customers who'd already decided to pay.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Ahmed Vaseer is Founder &amp;amp; CEO - Ingenious Concepts. 80+ projects, ISO 27001 certified via Global CB.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>ecommerce</category>
      <category>javascript</category>
      <category>ux</category>
    </item>
    <item>
      <title>Logging AI tool usage without logging sensitive data</title>
      <dc:creator>Ahmed Vaseer</dc:creator>
      <pubDate>Fri, 30 Oct 2026 06:11:20 +0000</pubDate>
      <link>https://dev.to/ahmedvaseer/logging-ai-tool-usage-without-logging-sensitive-data-171k</link>
      <guid>https://dev.to/ahmedvaseer/logging-ai-tool-usage-without-logging-sensitive-data-171k</guid>
      <description>&lt;p&gt;When a team adds an AI feature, the first debugging instinct is to log everything: the full prompt, the full response, the user's message. It works. It also means your logs now hold a copy of every customer email, contract and support ticket that went through the feature, sitting in a system with weaker access controls than your database and a much longer memory.&lt;/p&gt;

&lt;p&gt;You need logs. You need them to debug bad answers, answer audit questions and keep costs under control. You don't need them to be a second copy of your customers' data.&lt;/p&gt;

&lt;p&gt;Here's a practical middle path.&lt;/p&gt;

&lt;h3&gt;
  
  
  Decide what the logs are for
&lt;/h3&gt;

&lt;p&gt;Before writing any code, list the questions your logs must answer. For most small teams it's these:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Debugging:&lt;/strong&gt; which prompt version and model produced this answer, and did the call fail?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Audit:&lt;/strong&gt; who (which user or service) used the AI feature, when, and on what record?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost:&lt;/strong&gt; how many tokens did we use, by feature and by customer?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Safety:&lt;/strong&gt; did a call get blocked, filtered or flagged?&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Notice that none of these needs the full text of the prompt or the response. That's the key idea: &lt;strong&gt;log metadata by default, and content only by exception.&lt;/strong&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The default log record
&lt;/h3&gt;

&lt;p&gt;This is the record to write for every call:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;hmac&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;logging&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;uuid&lt;/span&gt;

&lt;span class="n"&gt;log&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;logging&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getLogger&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ai_usage&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="n"&gt;FP_KEY&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;AI_LOG_FP_KEY&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;   &lt;span class="c1"&gt;# random secret, not in code
&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;fingerprint&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;Keyed, non-reversible fingerprint so identical inputs can be grouped.&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;hmac&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;FP_KEY&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sha256&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;hexdigest&lt;/span&gt;&lt;span class="p"&gt;()[:&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;log_ai_call&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;feature&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;prompt_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;prompt_version&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;actor_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;record_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;prompt_text&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;response_text&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;usage&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;status&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;started&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="c1"&gt;# started: value of time.monotonic() taken before the call
&lt;/span&gt;    &lt;span class="n"&gt;usage&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;usage&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;{}&lt;/span&gt;                      &lt;span class="c1"&gt;# failed calls often have no usage data
&lt;/span&gt;    &lt;span class="n"&gt;call_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;uuid&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;uuid4&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;
    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;info&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;event&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ai_call&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;call_id&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;call_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;feature&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;feature&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;                  &lt;span class="c1"&gt;# e.g. "ticket_summary"
&lt;/span&gt;        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompt&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;prompt_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompt_version&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;prompt_version&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;model&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;actor_id&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;actor_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;                &lt;span class="c1"&gt;# internal user or service id, not an email
&lt;/span&gt;        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;record_id&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;record_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;              &lt;span class="c1"&gt;# e.g. ticket id
&lt;/span&gt;        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;input_chars&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt_text&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;input_fp&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;fingerprint&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt_text&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;output_chars&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;response_text&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="sh"&gt;""&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;input_tokens&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;usage&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;input_tokens&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;output_tokens&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;usage&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;output_tokens&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;status&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;status&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;                    &lt;span class="c1"&gt;# "ok", "error", "blocked"
&lt;/span&gt;        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;latency_ms&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;int&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;monotonic&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;started&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="p"&gt;}))&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;call_id&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With this, you can trace any answer to a prompt version and a record, add up costs by feature, and see failure rates, all without storing a word of the customer's text.&lt;/p&gt;

&lt;p&gt;A note on the fingerprint: a plain hash of short or predictable text (an email address, a yes/no answer) can be reversed by guessing, so the fingerprint is keyed with a secret (&lt;code&gt;AI_LOG_FP_KEY&lt;/code&gt;), so it can't be checked by guessing without the key. The fingerprint is for grouping long, varied inputs ("were these two calls the same request?"), not for hiding identifiers. Identifiers like emails shouldn't go into it in the first place.&lt;/p&gt;

&lt;h3&gt;
  
  
  When you do need content: redact first
&lt;/h3&gt;

&lt;p&gt;Sometimes you need to see what was sent, for example while fixing a specific bug. When that happens, log a &lt;strong&gt;redacted sample&lt;/strong&gt;, not the raw text, and only when a flag is switched on.&lt;/p&gt;

&lt;p&gt;A simple redaction pass catches the most common sensitive patterns:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;

&lt;span class="n"&gt;PATTERNS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;r&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;\b(?:sk|pk|api|key|token|ghp|xox[bp])[-_][A-Za-z0-9_-]{12,}\b&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;I&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;[SECRET]&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;r&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;[\w.+-]+@[\w-]+\.[\w.-]+&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;[EMAIL]&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;r&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;\b(?:\d[ -]?){13,19}\b&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;[CARD]&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;r&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;(?&amp;lt;![\w.])\+?\d(?:[\s()-]{0,2}\d){9,14}(?![\w.])&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;[PHONE]&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;redact&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;limit&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;500&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;label&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;PATTERNS&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pattern&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sub&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;label&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;[:&lt;/span&gt;&lt;span class="n"&gt;limit&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Order matters: run the most specific patterns first. Secrets go before emails, cards and phone numbers, or a phone pattern can eat the digits in the middle of a key and leave the rest in the log.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;

&lt;span class="n"&gt;DEBUG_CONTENT&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getenv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;AI_LOG_CONTENT&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;

&lt;span class="n"&gt;call_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;log_ai_call&lt;/span&gt;&lt;span class="p"&gt;(...)&lt;/span&gt;  &lt;span class="c1"&gt;# same call as above
&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;DEBUG_CONTENT&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;info&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;event&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ai_call_content&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;call_id&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;call_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompt_sample&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;redact&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt_text&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;response_sample&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;redact&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;response_text&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="sh"&gt;""&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="p"&gt;}))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Be honest about the limits. Regular expressions catch emails, phone numbers, card-like numbers and obvious keys. They &lt;strong&gt;won't&lt;/strong&gt; catch names, many provider key formats (cloud access keys, bearer tokens), addresses, medical details or anything written in free text. That's why content logging stays off by default, is switched on for a short time for a specific problem, and is truncated. If your feature handles health, financial or legal information, don't log content at all; reproduce the problem with test data instead.&lt;/p&gt;

&lt;h3&gt;
  
  
  Keep content logs somewhere separate, for less time
&lt;/h3&gt;

&lt;p&gt;Metadata and content have different risks, so treat them differently:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Metadata log&lt;/th&gt;
&lt;th&gt;Content sample log&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Default&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Always on&lt;/td&gt;
&lt;td&gt;Off; switched on per incident&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Who can read it&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Engineering and whoever handles costs&lt;/td&gt;
&lt;td&gt;A named few engineers&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;How long it's kept&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;As long as your audit needs (agree this with whoever owns compliance)&lt;/td&gt;
&lt;td&gt;Days, not months&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Where&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Your normal log system&lt;/td&gt;
&lt;td&gt;A separate log stream or index with its own access rule and expiry&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Most log platforms let you route by a field (here, &lt;code&gt;event&lt;/code&gt;) and set a separate retention period per stream. Use that, so content samples expire automatically and nobody has to remember to delete them.&lt;/p&gt;

&lt;h3&gt;
  
  
  Don't forget the other copies
&lt;/h3&gt;

&lt;p&gt;Your own logs are only one place prompts end up:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The AI provider.&lt;/strong&gt; Check their data retention and training settings for your account, and turn off anything you don't need. Write down what you chose and why.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Error trackers.&lt;/strong&gt; Exception reports often capture local variables, including the prompt. Configure them to scrub request bodies and the variables that hold prompt text. Provider error messages sometimes echo part of the prompt too, so log the exception type (&lt;code&gt;type(exc).__name__&lt;/code&gt;) and status code, not &lt;code&gt;str(exc)&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Analytics and tracing tools.&lt;/strong&gt; If you trace requests end to end, make sure spans don't attach the full prompt as an attribute.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Chat and tickets.&lt;/strong&gt; When someone pastes a "bad answer" into a team chat to discuss it, that's a copy too. Share the &lt;code&gt;call_id&lt;/code&gt; instead, and look the details up where access is controlled.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  What auditors usually ask
&lt;/h3&gt;

&lt;p&gt;If you work with larger clients, sooner or later someone will ask how you control AI use. The metadata log answers most of it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Which features use AI, and which models? (&lt;code&gt;feature&lt;/code&gt;, &lt;code&gt;model&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Who used them, on which records? (&lt;code&gt;actor_id&lt;/code&gt;, &lt;code&gt;record_id&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Can you show a change history for prompts? (&lt;code&gt;prompt_version&lt;/code&gt;, plus your repository history)&lt;/li&gt;
&lt;li&gt;How long do you keep AI-related data, and who can see it? (your retention table above)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Being able to answer those from a dashboard, rather than by searching raw text, is a good sign you've got the balance right.&lt;/p&gt;

&lt;h3&gt;
  
  
  A short checklist
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;Write a metadata record for every AI call: feature, prompt version, model, actor, record, tokens, status, latency.&lt;/li&gt;
&lt;li&gt;Never put emails or other identifiers into &lt;code&gt;actor_id&lt;/code&gt; or the fingerprint.&lt;/li&gt;
&lt;li&gt;Keep content logging off by default; when it's on, redact and truncate.&lt;/li&gt;
&lt;li&gt;Send content samples to a separate stream with short retention and narrow access.&lt;/li&gt;
&lt;li&gt;Review provider retention settings, error-tracker scrubbing and tracing attributes.&lt;/li&gt;
&lt;li&gt;Share call IDs in chat, not prompts.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Good AI logs tell you what happened without telling everyone what your customers said.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Ahmed Vaseer is Founder &amp;amp; CEO - Ingenious Concepts, an AI-first technology and digital growth agency. 80+ projects, ISO 27001 certified via Global CB.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>ai</category>
      <category>python</category>
      <category>logging</category>
    </item>
    <item>
      <title>The handover checklist: what you should own when a project ends</title>
      <dc:creator>Ahmed Vaseer</dc:creator>
      <pubDate>Thu, 29 Oct 2026 09:22:02 +0000</pubDate>
      <link>https://dev.to/ahmedvaseer/the-handover-checklist-what-you-should-own-when-a-project-ends-2m6e</link>
      <guid>https://dev.to/ahmedvaseer/the-handover-checklist-what-you-should-own-when-a-project-ends-2m6e</guid>
      <description>&lt;p&gt;The most expensive moment in a software project is often not the build. It's the day, months later, when you want to change something and discover you don't actually control it.&lt;/p&gt;

&lt;p&gt;The domain is registered in a former developer's personal account. The hosting is billed to the agency's card. Nobody has the admin password for the analytics. The code exists, somewhere, but not anywhere you can reach.&lt;/p&gt;

&lt;p&gt;None of this usually comes from bad intent. It comes from convenience. During a build, it's faster for the team doing the work to set things up in their own accounts. If nobody plans the handover, it never happens.&lt;/p&gt;

&lt;p&gt;Here's the checklist I'd want any buyer to work through before paying the final invoice. It works whether the project was built by an agency, a freelancer or an internal team that's moving on.&lt;/p&gt;

&lt;h3&gt;
  
  
  The rule behind the checklist
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;You should own the accounts. Your partners should have access to them.&lt;/strong&gt; Not the other way round.&lt;/p&gt;

&lt;p&gt;Access can be given and taken away in minutes. Ownership, once it sits with someone else, can take weeks to recover, if you can recover it at all.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Code
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;The source code is in a repository &lt;strong&gt;owned by your organisation&lt;/strong&gt;, not a personal account. Your partners are added as members.&lt;/li&gt;
&lt;li&gt;You can see the full history, not just a zip file of the final version.&lt;/li&gt;
&lt;li&gt;Any private packages, plugins or themes the project depends on are either in your account or clearly listed with their licences.&lt;/li&gt;
&lt;li&gt;Someone on your side has successfully downloaded the code and confirmed it's complete.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  2. Domains and DNS
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;The domain is registered in your organisation's name, in an account you control, with a contact email you'll still have in five years.&lt;/li&gt;
&lt;li&gt;You know where DNS is managed and have admin access there.&lt;/li&gt;
&lt;li&gt;Auto-renew is switched on, with a card your business pays.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Losing a domain because a renewal reminder went to someone's old inbox is one of the most avoidable disasters in business.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Hosting, cloud and paid services
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Hosting and cloud accounts are in your name and billed to you.&lt;/li&gt;
&lt;li&gt;You have a list of every paid service the project uses (email sending, maps, payments, search, AI APIs, storage), who owns each account, and what it costs each month.&lt;/li&gt;
&lt;li&gt;Anything still billed through your partner is written down, with a date to move it across.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  4. Credentials and secrets
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;You have an owner-level login for every account above, stored in your organisation's password manager, not a spreadsheet or a chat thread.&lt;/li&gt;
&lt;li&gt;API keys and other secrets used by the application are documented: what each one is for and where it's stored.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;After handover, rotate them.&lt;/strong&gt; Anyone who had them during the build shouldn't still have working copies. Agree this with your partner in advance so it's routine, not an accusation.&lt;/li&gt;
&lt;li&gt;Two-factor authentication on key accounts is tied to people in your organisation, not to a developer's phone.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  5. Admin access to the product itself
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;You have the top-level admin account in the application, CMS or store.&lt;/li&gt;
&lt;li&gt;Partner and developer accounts are separate, named logins, so they can be removed individually later.&lt;/li&gt;
&lt;li&gt;You know how to add and remove users yourself.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  6. Data and analytics
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Analytics, tag manager, search console and advertising accounts are owned by your organisation, with partners added as users.&lt;/li&gt;
&lt;li&gt;You know where backups are, how often they run, and you've seen one restored at least once.&lt;/li&gt;
&lt;li&gt;If the project collects personal data, you know where it's stored and who can access it.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  7. Documentation
&lt;/h3&gt;

&lt;p&gt;Documentation doesn't have to be long. It has to answer the questions the next person will ask:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;How do I run this locally, and how is it deployed?&lt;/li&gt;
&lt;li&gt;What are the main parts, and what talks to what?&lt;/li&gt;
&lt;li&gt;What breaks most often, and how was it fixed last time?&lt;/li&gt;
&lt;li&gt;What's unfinished or known to be fragile?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One honest page beats a polished document that's out of date the week it's delivered.&lt;/p&gt;

&lt;h3&gt;
  
  
  8. Contracts and rights
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Your contract says the work, including code and design files, belongs to you once it's paid for, apart from clearly listed third-party components.&lt;/li&gt;
&lt;li&gt;Any ongoing support or maintenance terms are written down, including how quickly issues will be looked at.&lt;/li&gt;
&lt;li&gt;You know what it will cost, and what you'll need, if you decide to move to a different partner.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  How to run the handover
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Agree it at the start.&lt;/strong&gt; Put "handover checklist complete" in the contract as a condition of final payment. This protects good partners too, because it makes expectations clear.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Set up ownership on day one.&lt;/strong&gt; It's far easier to create the domain, repository and hosting in your accounts at the beginning than to migrate them at the end.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hold a short handover meeting.&lt;/strong&gt; Go through the list together. Tick each item only when someone on your side has logged in and checked.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Rotate and remove.&lt;/strong&gt; Once you're satisfied, rotate secrets and reduce partner access to what ongoing support actually needs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep the list.&lt;/strong&gt; Update it whenever a new service is added. Next time, the handover takes an hour.&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  A word for the people doing the build
&lt;/h3&gt;

&lt;p&gt;If you build for clients, offering this list unprompted is one of the easiest ways to earn trust. Clients remember the partner who handed them the keys without being asked, and they tend to come back to them.&lt;/p&gt;

&lt;h3&gt;
  
  
  The short version
&lt;/h3&gt;

&lt;p&gt;Before the final invoice, make sure you can answer "yes" to one question for every part of the project: &lt;strong&gt;if our partner disappeared tomorrow, could we keep running this?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If the answer is no, that's what the handover is for.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Ahmed Vaseer is Founder &amp;amp; CEO - Ingenious Concepts, an AI-first technology and digital growth agency. 80+ projects, ISO 27001 certified via Global CB.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>business</category>
      <category>devops</category>
      <category>security</category>
      <category>career</category>
    </item>
    <item>
      <title>Treat prompts like code: versioning, testing and review</title>
      <dc:creator>Ahmed Vaseer</dc:creator>
      <pubDate>Mon, 26 Oct 2026 06:08:40 +0000</pubDate>
      <link>https://dev.to/ahmedvaseer/treat-prompts-like-code-versioning-testing-and-review-53hj</link>
      <guid>https://dev.to/ahmedvaseer/treat-prompts-like-code-versioning-testing-and-review-53hj</guid>
      <description>&lt;p&gt;Somebody changes one sentence in a prompt on a Friday afternoon. On Monday, half the support summaries are in the wrong format and nobody knows why.&lt;/p&gt;

&lt;p&gt;This happens because many teams treat prompts as settings, not code. They live in a dashboard, a database row or a string pasted into a function. They change without review, and there's no record of what they said last week.&lt;/p&gt;

&lt;p&gt;A prompt decides what your product does. That makes it code. The fix is not a new platform. It's three habits you already use for everything else: keep it in the repo, test it, and review changes.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Keep prompts in the repo, as files
&lt;/h3&gt;

&lt;p&gt;Put each prompt in its own file, next to the code that uses it. Plain text or Markdown is fine. Give it a version in the file itself so logs can say which version produced an answer.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;app/
  prompts.py
prompts/
  summarise_ticket/
    prompt.md
    cases.jsonl
tests/
  test_summarise_ticket.py
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;prompt.md&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;summarise_ticket&lt;/span&gt;
&lt;span class="na"&gt;version&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
You summarise customer support tickets for an internal dashboard.

Return JSON with exactly these keys:
&lt;span class="p"&gt;-&lt;/span&gt; "summary": one sentence, under 30 words
&lt;span class="p"&gt;-&lt;/span&gt; "category": one of "billing", "bug", "how-to", "other"
&lt;span class="p"&gt;-&lt;/span&gt; "urgent": true or false

Ticket:
{ticket_text}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Load it with a small helper rather than copying strings around:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pathlib&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Path&lt;/span&gt;

&lt;span class="n"&gt;PROMPTS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Path&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;__file__&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="n"&gt;parent&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parent&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompts&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;   &lt;span class="c1"&gt;# repo-root prompts/
&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;load_prompt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;tuple&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
    &lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;PROMPTS&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="n"&gt;name&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompt.md&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;read_text&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;encoding&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;_&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;header&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;---&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;next&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="nf"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;:&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;header&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;splitlines&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;startswith&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;version:&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="n"&gt;version&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;render&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;tuple&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
    &lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;load_prompt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;format&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;version&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This assumes the helper lives in &lt;code&gt;app/prompts.py&lt;/code&gt; and the &lt;code&gt;prompts/&lt;/code&gt; folder sits at the repo root.&lt;/p&gt;

&lt;p&gt;Now every change to the prompt is a commit. You can see who changed it, when, and why, and you can roll it back with one revert.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;One caution:&lt;/strong&gt; &lt;code&gt;str.format&lt;/code&gt; treats curly braces as placeholders. If your prompt contains literal braces (for example a JSON example), double them (&lt;code&gt;{{&lt;/code&gt; and &lt;code&gt;}}&lt;/code&gt;) or use a different templating approach.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Test against a small, fixed set of cases
&lt;/h3&gt;

&lt;p&gt;You don't need a large evaluation suite to start. Twenty real examples, with what a good answer must contain, will catch most regressions.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;cases.jsonl&lt;/code&gt; (one case per line):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"refund-late"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"ticket_text"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"I was charged twice this month and want a refund."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"expect"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"category"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"billing"&lt;/span&gt;&lt;span class="p"&gt;}}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"login-bug"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"ticket_text"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"The login button does nothing on Safari."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"expect"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"category"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"bug"&lt;/span&gt;&lt;span class="p"&gt;}}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"site-down"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"ticket_text"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Your checkout is down and we are losing orders right now."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"expect"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"category"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"bug"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"urgent"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;}}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Test what you can check reliably: valid JSON, the right keys, allowed values, and the fields that must match. Don't test exact wording; it will change from run to run.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pathlib&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Path&lt;/span&gt;

&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;pytest&lt;/span&gt;

&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;app.llm&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;call_model&lt;/span&gt;          &lt;span class="c1"&gt;# your existing model wrapper
&lt;/span&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;app.prompts&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;render&lt;/span&gt;

&lt;span class="n"&gt;CASES_FILE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Path&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;__file__&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="n"&gt;parent&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parent&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompts/summarise_ticket/cases.jsonl&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;  &lt;span class="c1"&gt;# adjust to your tests folder
&lt;/span&gt;&lt;span class="n"&gt;CASES&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;CASES_FILE&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read_text&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;encoding&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;splitlines&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;()]&lt;/span&gt;
&lt;span class="n"&gt;ALLOWED&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;billing&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;bug&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;how-to&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;other&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;


&lt;span class="nd"&gt;@pytest.mark.llm&lt;/span&gt;
&lt;span class="nd"&gt;@pytest.mark.parametrize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;case&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;CASES&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ids&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;id&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;CASES&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;test_summarise_ticket&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;case&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;_version&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;render&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;summarise_ticket&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ticket_text&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;case&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ticket_text&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
    &lt;span class="n"&gt;raw&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;call_model&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;temperature&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;raw&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;summary&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;category&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;urgent&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;category&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;ALLOWED&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nf"&gt;isinstance&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;urgent&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="nb"&gt;bool&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;summary&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="k"&gt;case&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;expect&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;items&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
        &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;: expected &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="si"&gt;!r}&lt;/span&gt;&lt;span class="s"&gt;, got &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="si"&gt;!r}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A few practical notes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;These tests call a real model&lt;/strong&gt;, so they cost money and are slower than unit tests. They're marked with &lt;code&gt;@pytest.mark.llm&lt;/code&gt; (registered below) and skipped by default; run them with &lt;code&gt;pytest -m llm&lt;/code&gt; when a prompt or model changes, not on every commit.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Setting temperature to 0 reduces variation but doesn't remove it.&lt;/strong&gt; If a case fails now and then, run it three times and treat two out of three as a pass, or rewrite the case so the expected answer is clearer.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Add a case every time you find a bad answer in production.&lt;/strong&gt; That's how the set grows into something valuable without a big upfront effort.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;If your wrapper doesn't use the provider's JSON mode,&lt;/strong&gt; strip code fences before parsing.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Register the marker so pytest knows it, and skip these tests by default:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight toml"&gt;&lt;code&gt;&lt;span class="c"&gt;# pyproject.toml&lt;/span&gt;
&lt;span class="k"&gt;[&lt;/span&gt;&lt;span class="n"&gt;tool&lt;/span&gt;&lt;span class="k"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pytest&lt;/span&gt;&lt;span class="k"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ini_options&lt;/span&gt;&lt;span class="k"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;markers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"llm: calls a real model; run with -m llm"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;addopts&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s"&gt;"-m 'not llm'"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  3. Review prompt changes like code changes
&lt;/h3&gt;

&lt;p&gt;Prompt changes go through the same pull request as any other change. Two rules make the review useful:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Bump the version&lt;/strong&gt; in the file header whenever the wording changes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Paste the test results&lt;/strong&gt; in the pull request: which cases passed before and after.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A reviewer should be able to answer three questions in a couple of minutes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;What behaviour is this change meant to fix?&lt;/li&gt;
&lt;li&gt;Which case proves it's fixed? (If none exists, add one.)&lt;/li&gt;
&lt;li&gt;Did any other case get worse?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That's it. It sounds too simple, but most prompt incidents come from changes nobody looked at, not from changes that were reviewed and still went wrong.&lt;/p&gt;

&lt;h3&gt;
  
  
  4. Log the version with every answer
&lt;/h3&gt;

&lt;p&gt;When something goes wrong in production, the first question is "which prompt produced this?" Log the prompt name and version with each call (not the full text, and not the user's data):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;logging&lt;/span&gt;

&lt;span class="n"&gt;log&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;logging&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getLogger&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;llm&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;render&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;summarise_ticket&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ticket_text&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;ticket&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;raw&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;call_model&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;temperature&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;info&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;llm_call&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;extra&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompt&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;summarise_ticket&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompt_version&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;version&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ticket_id&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;ticket&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;id&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;extra&lt;/code&gt; fields only show up if your formatter or JSON log handler includes them. With plain &lt;code&gt;logging.basicConfig&lt;/code&gt;, put them in the message instead: &lt;code&gt;log.info("llm_call prompt=%s version=%s ticket=%s", "summarise_ticket", version, ticket.id)&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Now a bad answer leads you straight to a commit.&lt;/p&gt;

&lt;h3&gt;
  
  
  5. Change the model the same way
&lt;/h3&gt;

&lt;p&gt;Switching to a newer or cheaper model is also a behaviour change. Keep the model name in config, run the same cases against the new model before switching, and record the result in the pull request. The cases you built for prompts protect you here too.&lt;/p&gt;

&lt;h3&gt;
  
  
  What this doesn't solve
&lt;/h3&gt;

&lt;p&gt;Twenty test cases won't catch everything. They won't tell you whether summaries are &lt;em&gt;good&lt;/em&gt;, only whether they're valid and correct on the points you check. For quality, a person still needs to read a sample of real answers from time to time. The aim of this setup is narrower: no prompt change reaches production without someone seeing it, testing it and being able to undo it.&lt;/p&gt;

&lt;h3&gt;
  
  
  A one-afternoon checklist
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;Move each prompt into its own file in the repo, with a version number.&lt;/li&gt;
&lt;li&gt;Load prompts through one helper instead of copied strings.&lt;/li&gt;
&lt;li&gt;Write ten to twenty cases from real inputs.&lt;/li&gt;
&lt;li&gt;Add a marked test that checks structure and the fields that must match.&lt;/li&gt;
&lt;li&gt;Require a version bump and test results in every prompt pull request.&lt;/li&gt;
&lt;li&gt;Log prompt name and version with every model call.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Boring, and that's the point. The less exciting your prompt changes are, the fewer surprises your users get.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Ahmed Vaseer is Founder &amp;amp; CEO - Ingenious Concepts, an AI-first technology and digital growth agency. 80+ projects, ISO 27001 certified via Global CB.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>python</category>
      <category>testing</category>
      <category>programming</category>
    </item>
    <item>
      <title>Driving AI-Powered Digital Growth: Lessons from Building Tech Solutions at Ingenious Concepts</title>
      <dc:creator>Ahmed Vaseer</dc:creator>
      <pubDate>Tue, 27 Jan 2026 21:28:27 +0000</pubDate>
      <link>https://dev.to/ahmedvaseer/driving-ai-powered-digital-growth-lessons-from-building-tech-solutions-at-ingenious-concepts-4a91</link>
      <guid>https://dev.to/ahmedvaseer/driving-ai-powered-digital-growth-lessons-from-building-tech-solutions-at-ingenious-concepts-4a91</guid>
      <description>&lt;p&gt;Hello Dev.to community! I’m Ahmed Vaseer, CEO &amp;amp; Founder of Ingenious Concepts, a company dedicated to building software, web and mobile applications, games, and AI-driven digital marketing solutions. Over the years, I’ve learned that digital growth isn’t just about adopting new tools — it’s about integrating technology in ways that create real business impact.&lt;/p&gt;

&lt;p&gt;In this article, I want to share some of the key lessons we’ve learned while helping businesses leverage AI and technology for growth.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. AI is a Growth Engine, Not Just a Buzzword&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Artificial intelligence can be intimidating for many businesses, but the truth is that it’s most effective when applied practically. At Ingenious Concepts, we’ve helped companies automate repetitive tasks, analyze customer behavior, and optimize marketing campaigns using AI tools. The result? More efficient operations and smarter decision-making.&lt;/p&gt;

&lt;p&gt;Tip: Start small. Automate one process at a time and measure results before scaling.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Build Digital Products That Solve Real Problems&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Many startups focus on flashy features but neglect whether their product actually solves a real problem. We always ask: “Does this feature help our client achieve a measurable goal?”&lt;/p&gt;

&lt;p&gt;Whether it’s a SaaS platform, a mobile app, or a game, product design should always be purpose-driven. That mindset has helped our clients reduce costs, improve engagement, and increase revenue.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Integrate Marketing and Technology Early&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Digital marketing and technology often work in silos. The companies that win are those that integrate these two from day one. For example, when developing an e-commerce platform, we build automation into the sales and marketing workflows so campaigns can run efficiently while data drives decisions.&lt;/p&gt;

&lt;p&gt;Pro Tip: Analytics, automation, and AI-driven insights should be part of your product from the start — not an afterthought.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. Teamwork and Leadership Matter More Than Tools&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Even the best AI and software tools won’t succeed without a strong team. Leading cross-functional teams of developers, designers, and strategists requires clarity, vision, and an environment where experimentation is encouraged.&lt;/p&gt;

&lt;p&gt;At Ingenious Concepts, we focus on building multidisciplinary teams that can execute quickly without sacrificing quality — a critical factor in today’s fast-paced digital landscape.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. Focus on Sustainable, Scalable Growth&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Technology moves fast, but sustainable growth is a marathon, not a sprint. Our philosophy is to build systems and processes that scale, rather than chasing short-term trends. This approach allows businesses to evolve without constant overhauls, saving time and money while maximizing long-term impact.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Final Thoughts&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Digital transformation is no longer optional — it’s a necessity. But it’s not enough to adopt AI or new software for the sake of it. Real impact comes from integrating these tools thoughtfully, with a focus on measurable growth, user experience, and scalability.&lt;/p&gt;

&lt;p&gt;At Ingenious Concepts, we continue to explore innovative ways to merge technology, marketing, and strategy to empower businesses worldwide.&lt;/p&gt;

&lt;p&gt;If you’re curious about AI, digital growth, or building scalable tech products, feel free to connect or leave a comment below — I’d love to share more insights from our journey.&lt;/p&gt;

&lt;p&gt;About the Author:&lt;br&gt;
&lt;strong&gt;&lt;a href="http://www.ahmedvaseer.com" rel="noopener noreferrer"&gt;Ahmed Vaseer&lt;/a&gt;&lt;/strong&gt; is CEO &amp;amp; Founder of &lt;strong&gt;&lt;a href="https://www.ingeniousc.com" rel="noopener noreferrer"&gt;Ingenious Concepts&lt;/a&gt;&lt;/strong&gt;, a tech company specializing in software, web and mobile apps, game development, and AI-powered digital marketing solutions. He helps businesses leverage technology for growth, efficiency, and global expansion.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>leadership</category>
      <category>softwaredevelopment</category>
      <category>startup</category>
    </item>
  </channel>
</rss>
