<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: JamesAnderson121</title>
    <description>The latest articles on DEV Community by JamesAnderson121 (@jamesanderson121).</description>
    <link>https://dev.to/jamesanderson121</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4054101%2F3f9cb9b8-9ade-4339-aff5-feee1eda09c1.png</url>
      <title>DEV Community: JamesAnderson121</title>
      <link>https://dev.to/jamesanderson121</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jamesanderson121"/>
    <language>en</language>
    <item>
      <title>Presigned URL Governance for Direct Browser Uploads in Marketplace Object Storage</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Sun, 15 Nov 2026 02:09:43 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/presigned-url-governance-for-direct-browser-uploads-in-marketplace-object-storage-1k42</link>
      <guid>https://dev.to/jamesanderson121/presigned-url-governance-for-direct-browser-uploads-in-marketplace-object-storage-1k42</guid>
      <description>&lt;p&gt;The useful question for a marketplace SaaS is not which object store has the nicest upload demo. It is whether a browser-uploaded training artifact can be identified, authorized, processed, and deleted later without guesswork. For that reason, I would make the retention ledger the design center, then use private object storage and short-lived presigned URLs as the delivery mechanism.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Short answer:&lt;/strong&gt; create a server-owned artifact record before the upload, bind one narrowly scoped presigned URL to that record, and let a verified completion state unlock processing. Keep the bucket private, keep tenant and region decisions in application data, and treat deletion evidence as part of the artifact rather than as an afterthought.&lt;/p&gt;

&lt;p&gt;This is a marketplace example: a seller submits a catalog snapshot or moderation set for a RAG or agent feature. The browser sends bytes directly to storage, while the application remains responsible for identity, policy, and the decision to let a worker read those bytes. That separation keeps delivery simple without handing authorization to the browser.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why is the retention ledger the first storage decision?
&lt;/h2&gt;

&lt;p&gt;An object key is a locator, not a governance record. Start an upload with an operation ID, tenant ID, artifact class, selected US or EU region, policy ID, effective timestamp, and calculated expiry. A key such as &lt;code&gt;tenant-42/training/2026-08/run-019/catalog.csv&lt;/code&gt; is useful for organization, but the original filename should remain display metadata. It should not decide ownership.&lt;/p&gt;

&lt;p&gt;The database row can move through &lt;code&gt;issued&lt;/code&gt;, &lt;code&gt;uploaded&lt;/code&gt;, &lt;code&gt;processing&lt;/code&gt;, and &lt;code&gt;expired&lt;/code&gt;. The object store holds bytes and metadata; the row explains why those bytes exist and whether they may enter the next training run. A worker should consume only a verified &lt;code&gt;uploaded&lt;/code&gt; record associated with the expected key. An object listing cannot answer which seller owns a file or which retention policy applied when it arrived.&lt;/p&gt;

&lt;p&gt;Keep it private.&lt;/p&gt;

&lt;p&gt;That choice makes the failure model visible. Imagine a seller closes the tab immediately after storage accepts &lt;code&gt;catalog.csv&lt;/code&gt;: the browser may never report completion, so the row stays &lt;code&gt;issued&lt;/code&gt; and the worker must refuse it. Now imagine the completion response is lost and the seller retries: the operation ID must resolve to the same row, or two parsers can ingest one catalog. Finally, imagine the expiry job runs while a cleanup request is retried: the record should preserve the policy ID, expiry, attempt, and result so an operator can distinguish an already-removed object from a deletion that still needs attention. This sequence is why the ledger is more than CRUD metadata. It is the audit trail connecting authorization, bytes, asynchronous processing, and retention. A provider can deliver the transfer correctly while the application still makes the wrong governance decision; testing the transfer alone cannot catch that.&lt;/p&gt;

&lt;p&gt;That distinction catches a particularly expensive class of mistakes. A successful storage transfer can lose its browser callback. A retry can enqueue the same catalog twice. A cleanup task can remove a row and leave its bytes behind. The operation ID and idempotent state transition give those events one place to meet.&lt;/p&gt;

&lt;p&gt;The regional field matters too. Record the chosen region with the tenant and artifact, test the deployed browser origins in both regions, and make cleanup observable. I'm not sure a region label by itself answers a customer's residency question; the contract and the policy definitions still decide what “US” or “EU” means.&lt;/p&gt;

&lt;h2&gt;
  
  
  How should browser direct upload and presigned URLs fit the Python AI workflow?
&lt;/h2&gt;

&lt;p&gt;The data flow is deliberately short. The application authenticates the seller and creates the artifact row. It signs a URL for the server-generated key, expected content type, size limit, and short expiration. The browser transfers the bytes without sending them through the application. A completion request then identifies the operation, and a server-side verification records the metadata needed by the training worker.&lt;/p&gt;

&lt;p&gt;The production service may be Node.js, but a Python harness is handy in a notebook-to-prod workflow. It tests the upload contract without involving the UI, and it makes an eval-driven pipeline easier to inspect: the same artifact fixture can be used for upload checks, parsing checks, and later dataset membership checks.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pathlib&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;Path&lt;/span&gt;

&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;upload_artifact&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Path&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;upload_url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"PRESIGNED_UPLOAD_URL"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;content_type&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"UPLOAD_CONTENT_TYPE"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"application/octet-stream"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;method&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"UPLOAD_METHOD"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"PUT"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;upper&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"rb"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;upload_url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"Content-Type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;content_type&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;120&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;201&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;204&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"upload rejected with HTTP {response.status_code}: "&lt;/span&gt;
            &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{response.text[:240]}"&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"ETag"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s2"&gt;"__main__"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;artifact_path&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Path&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"UPLOAD_FILE"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;upload_artifact&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;artifact_path&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The harness uses the URL, method, and headers returned by the application instead of assuming a storage SDK contract. It also does not retry blindly. If a request is rejected before the bytes are accepted, the right next action may be a fresh authorization decision; if the transfer is large, multipart upload may resume individual parts. In both cases, incomplete work needs an expiry and cleanup policy.&lt;/p&gt;

&lt;p&gt;Keep prompt cost in the same operational conversation. Training artifacts often feed retrieval or agent evaluation, so storing an unnecessary duplicate can increase downstream parsing and token use even when the upload itself succeeds. Record the artifact hash or provider metadata needed by the application, and make the eval harness reject an operation that has not passed the completion transition.&lt;/p&gt;

&lt;h2&gt;
  
  
  What failure modes should a US/EU SaaS test before launch?
&lt;/h2&gt;

&lt;p&gt;Test the authorization boundary before testing upload speed. An authenticated seller should be able to request an allowed artifact, while a second tenant's operation ID must not produce a usable upload instruction. Try an expired capability, an unexpected content type, an over-limit file, a missing signed header, and an unregistered browser origin. A browser preflight must be tested against the real US and EU production origins, not just &lt;code&gt;localhost&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Then test the awkward cases. Interrupt a larger transfer and check resume behavior, incomplete-part cleanup, and preservation of one operation ID. Submit completion twice. Close the browser after storage accepts the bytes. Retry deletion. The expected result is boring: one durable artifact state, one processing enqueue, and an auditable cleanup outcome.&lt;/p&gt;

&lt;p&gt;HTTP status is diagnostic information, not proof that an artifact is ready. A &lt;code&gt;403&lt;/code&gt; can indicate an expired URL or signed-header mismatch; a &lt;code&gt;413&lt;/code&gt; can indicate a size limit; and a browser-only failure can point to CORS. A &lt;code&gt;429&lt;/code&gt; needs a bounded retry policy and telemetry. None of these responses should advance the row to &lt;code&gt;uploaded&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;No magic.&lt;/p&gt;

&lt;p&gt;Use metrics that describe the state machine: issued URLs, successful transfers, completion calls, expired operations, cleanup attempts, and time from completion to worker start. Include tenant, region, operation ID, and artifact class in structured logs. Never log the presigned URL itself. It is a temporary capability, not harmless debug text.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Decision&lt;/th&gt;
&lt;th&gt;Direct browser transfer&lt;/th&gt;
&lt;th&gt;Application proxy&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Byte path&lt;/td&gt;
&lt;td&gt;Browser to private storage&lt;/td&gt;
&lt;td&gt;Browser through the application&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Main strength&lt;/td&gt;
&lt;td&gt;Less application bandwidth for larger files&lt;/td&gt;
&lt;td&gt;Inspect every byte before storage&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Main burden&lt;/td&gt;
&lt;td&gt;Completion, CORS, and cleanup state&lt;/td&gt;
&lt;td&gt;Application bandwidth and latency&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Better fit&lt;/td&gt;
&lt;td&gt;Predictable authorization and async workers&lt;/td&gt;
&lt;td&gt;Tiny files or mandatory synchronous inspection&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  What should the implementation trade off against delivery simplicity?
&lt;/h2&gt;

&lt;p&gt;Direct browser transfer is a good fit when files are large, tenant authorization is predictable, and a separate processing worker can own the next step. Its advantage is a clean bandwidth boundary: the application signs and records the operation while storage receives the bytes. That simplicity only holds if the completion state machine, CORS configuration, multipart lifecycle, and cleanup process are tested as one workflow.&lt;/p&gt;

&lt;p&gt;The catch is that this design is not suitable when the product needs permanent anonymous URLs, public static hosting, fine-grained collaborative overwrites, or a storage-native immutable archive with guarantees the selected service does not provide. A dedicated delivery layer or another storage design may fit those requirements better. Stick with a proxy upload when files are tiny, traffic is low, and the application must inspect every byte before storage; accept the bandwidth and latency cost consciously.&lt;/p&gt;

&lt;p&gt;There is no universal best provider. Compare candidates by the boundary your team can verify: private-object controls, presigned capability scope, browser-origin rules, multipart cleanup, regional placement, lifecycle configuration, metadata visibility, and the quality of audit evidence. AWS's multipart overview documents the general multipart operation model, while Firebase's Cloud Storage documentation is a useful reference for a client-centric authorization model. Neither removes the need to test the tenant boundary and retention ledger in your own application.&lt;/p&gt;

&lt;p&gt;The operational checklist fits in prose: authorize before signing; generate the key on the server; bind content type and size; persist policy and expiry; verify completion; enqueue processing once; test wrong-tenant requests, expired capabilities, US/EU origins, duplicate callbacks, interrupted multipart work, and cleanup retries; then retain enough evidence to explain why an object was deleted. This is what turns a convenient browser upload into a reproducible training artifact.&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;AWS S3 Multipart Upload overview: &lt;a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html" rel="noopener noreferrer"&gt;https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Firebase Cloud Storage documentation: &lt;a href="https://firebase.google.com/docs/storage" rel="noopener noreferrer"&gt;https://firebase.google.com/docs/storage&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>python</category>
      <category>objectstorage</category>
      <category>datagovernance</category>
    </item>
    <item>
      <title>Travel Itinerary Retrieval: Document Lineage and Freshness for Planner Systems Explained</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Fri, 13 Nov 2026 19:43:40 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/travel-itinerary-retrieval-document-lineage-and-freshness-for-planner-systems-explained-4a5l</link>
      <guid>https://dev.to/jamesanderson121/travel-itinerary-retrieval-document-lineage-and-freshness-for-planner-systems-explained-4a5l</guid>
      <description>&lt;p&gt;When an itinerary changes, retrieve the newest valid document for the trip, but keep the older version available for explanation. That is the decision rule: &lt;strong&gt;current answers need an explicit version boundary, not a larger top-k.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Short answer:&lt;/strong&gt; store each itinerary revision as an immutable document, retrieve by a stable trip key plus an “active at” timestamp, and carry the revision ID into every chunk and citation. A planner can then answer “what is booked now?” without losing the ability to explain what changed.&lt;/p&gt;

&lt;p&gt;This is a retrieval architecture problem, not a vector-database shopping problem. A travel itinerary planner may have a booking confirmation, an airline schedule update, a hotel message, and a manually edited day plan that all refer to the same trip. Similar wording does not make those records interchangeable.&lt;/p&gt;

&lt;h2&gt;
  
  
  How can a travel itinerary planner retrieve the right document version?
&lt;/h2&gt;

&lt;p&gt;Start with two identities: &lt;code&gt;trip_id&lt;/code&gt; identifies the journey, while &lt;code&gt;revision_id&lt;/code&gt; identifies one immutable snapshot of its source document. A chunk gets both IDs, along with &lt;code&gt;effective_from&lt;/code&gt;, &lt;code&gt;effective_until&lt;/code&gt;, &lt;code&gt;source_type&lt;/code&gt;, and a content hash. The active revision is a query decision; the source document itself never gets rewritten in place.&lt;/p&gt;

&lt;p&gt;That distinction fixes a common failure mode. A team overwrites “Paris-plan.md” after a train time changes, re-embeds the file, and later discovers that an answer cites a chunk whose text no longer matches the booking record. The index may be fresh, but the evidence trail is gone. Version lineage gives the answer layer something stable to display and gives an evaluator a precise target.&lt;/p&gt;

&lt;p&gt;The retrieval flow is deliberately plain. Normalize the user's trip reference, resolve the permitted revision set, search only chunks in that set, then group results by &lt;code&gt;revision_id&lt;/code&gt; before composing an answer. A reranker may improve ordering, but it must not decide which revisions are valid. Freshness and authorization happen before ranking.&lt;/p&gt;

&lt;p&gt;Keep history.&lt;/p&gt;

&lt;p&gt;Here is a small, runnable Python sketch. It uses an in-memory list so the version rule is visible without hiding it behind a framework.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;dataclasses&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dataclass&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timezone&lt;/span&gt;


&lt;span class="nd"&gt;@dataclass&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frozen&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;Chunk&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;trip_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;revision_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;effective_from&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;
    &lt;span class="n"&gt;effective_until&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;
    &lt;span class="n"&gt;source_type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;active_chunks&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;chunks&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;Chunk&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;trip_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;at&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;Chunk&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;candidates&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="n"&gt;chunk&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;chunks&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;trip_id&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;trip_id&lt;/span&gt;
        &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;effective_from&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="n"&gt;at&lt;/span&gt;
        &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;effective_until&lt;/span&gt; &lt;span class="k"&gt;is&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;at&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;effective_until&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;newest_by_revision&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;candidates&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;newest_by_revision&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setdefault&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;revision_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;newest_by_revision&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;newest_revision&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;newest_by_revision&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;lambda&lt;/span&gt; &lt;span class="n"&gt;revision&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;effective_from&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;newest_by_revision&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;revision&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;newest_by_revision&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;newest_revision&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;


&lt;span class="n"&gt;now&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;now&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;timezone&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;utc&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;records&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="nc"&gt;Chunk&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;trip_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"trip-204"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;revision_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"rev-08"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Rail from Lyon to Paris departs at 09:40 on 2026-06-18."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;effective_from&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nf"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2026&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tzinfo&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;timezone&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;utc&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;effective_until&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;source_type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"rail_confirmation"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nc"&gt;Chunk&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;trip_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"trip-204"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;revision_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"rev-07"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Rail from Lyon to Paris departs at 08:55 on 2026-06-18."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;effective_from&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nf"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2026&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tzinfo&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;timezone&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;utc&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;effective_until&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nf"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2026&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tzinfo&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;timezone&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;utc&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;source_type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"rail_confirmation"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;active_chunks&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;records&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"trip-204"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;revision_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The example chooses one active revision after filtering. In production, keep the previous revision available for a “what changed?” request, but do not mix it into a current-plan answer unless the prompt asks for history. That one separation prevents an old departure time from winning because it happens to be semantically closer to the question.&lt;/p&gt;

&lt;h2&gt;
  
  
  The data model that keeps freshness explainable
&lt;/h2&gt;

&lt;p&gt;Versioning works best when the ingestion record is richer than a file path. I use an event-like record for every source update: the source identifier, observed time, effective time, revision ID, parser version, and a pointer to the raw payload. Chunks inherit the revision ID; they do not invent their own notion of freshness.&lt;/p&gt;

&lt;p&gt;There are two clocks here. &lt;code&gt;observed_at&lt;/code&gt; says when the system received a message. &lt;code&gt;effective_from&lt;/code&gt; says when the itinerary should be treated as true. An airline notice received late can have an earlier effective time than a message received on time. If the planner stores only ingestion time, a delayed notice can appear to be the newest truth.&lt;/p&gt;

&lt;p&gt;Chunk boundaries still matter. Split around semantic units such as a reservation, a transfer, or a day-level activity, and repeat the trip and revision metadata on every chunk. A tiny chunk that says “09:40” is not useful by itself; a chunk that includes the date, station, and revision can be cited without asking the model to reconstruct context from a neighbor.&lt;/p&gt;

&lt;p&gt;Do not silently merge revisions during indexing. If a new confirmation supersedes an old one, mark the old revision inactive for current retrieval and retain it for history. A delete operation should be an explicit policy decision, because legal retention, customer support, and debugging may require the original payload even after it leaves the active index.&lt;/p&gt;

&lt;h2&gt;
  
  
  What should chunking and freshness mean for itinerary answers?
&lt;/h2&gt;

&lt;p&gt;Chunking is a precision decision. Smaller units make it easier to exclude an unrelated day or reservation, while larger units preserve context such as a multi-leg connection. Freshness is a truth decision. A chunk can be highly relevant and still be wrong for the requested travel date.&lt;/p&gt;

&lt;p&gt;I keep these signals separate in the candidate record:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Signal&lt;/th&gt;
&lt;th&gt;Question it answers&lt;/th&gt;
&lt;th&gt;Failure when omitted&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Similarity&lt;/td&gt;
&lt;td&gt;Does this passage discuss the request?&lt;/td&gt;
&lt;td&gt;A matching phrase from the wrong trip can surface.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Revision validity&lt;/td&gt;
&lt;td&gt;Is this version active for the requested time?&lt;/td&gt;
&lt;td&gt;A cancelled train can look current.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Provenance&lt;/td&gt;
&lt;td&gt;Which source and revision support the claim?&lt;/td&gt;
&lt;td&gt;The answer cannot explain its evidence.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Scope&lt;/td&gt;
&lt;td&gt;Is this chunk allowed for the caller?&lt;/td&gt;
&lt;td&gt;A support agent may see another traveller's data.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The ranker should consume candidates only after validity and scope filters have run. I also make the answer contract preserve &lt;code&gt;source_type&lt;/code&gt;, &lt;code&gt;revision_id&lt;/code&gt;, and the relevant effective time in the citation object. That is enough for a UI to show “rail confirmation, revision 08” beside a departure time rather than presenting an untraceable sentence.&lt;/p&gt;

&lt;p&gt;Here is the evaluation case I would put in a notebook before wiring a production endpoint: ask for the current departure, ask what changed since yesterday, ask about an old revision explicitly, and ask a question about a different trip with similar city names. The expected result is not just a text string. It is a selected revision, an allowed evidence set, and a refusal when no valid evidence exists.&lt;/p&gt;

&lt;p&gt;One subtle trap is a partial update. A hotel message may revise check-in time without replacing the whole itinerary. Treating that message as a complete new snapshot can erase unchanged reservations. Imagine &lt;code&gt;rev-08&lt;/code&gt; contains the rail confirmation, two hotel stays, and a museum booking. A later message says only “late check-in is available until 22:00.” If ingestion stores that message as a full replacement, the next query can retrieve a revision with a correct check-in note but no hotel address, because the address lived only in &lt;code&gt;rev-08&lt;/code&gt;. If ingestion appends the note to the old chunks without a new revision, a historical question can accidentally inherit a future policy. The repair is architectural: either build a complete materialized revision from all authoritative components, or label the message as a patch and resolve it before retrieval. Record which source supplied each field, and make the resolver fail closed when two authoritative updates overlap without an ordering rule. Then test a sequence such as confirmation, amendment, delayed amendment, and cancellation. The expected output should name the selected revision and the fields it changed, not merely produce a fluent itinerary. Mixing snapshot and patch models produces answers that are current in one field and stale in another.&lt;/p&gt;

&lt;h2&gt;
  
  
  A practical rollout and the uncomfortable limits
&lt;/h2&gt;

&lt;p&gt;Ship the ingestion ledger before the chatbot. Log every revision transition and make the active-revision query inspectable. Then run a shadow set containing changed train times, duplicate booking numbers, late-arriving notices, and two trips with the same destination. Compare the selected revision and citation metadata, not only the generated prose.&lt;/p&gt;

&lt;p&gt;I budget tokens around the evidence that survives filtering. Sending five chunks from three revisions to the answer model is expensive and confusing; sending one well-scoped chunk with a clear source label is usually easier to evaluate. The notebook-to-prod handoff should record the candidate count, selected revision, and prompt tokens so a quality improvement does not hide a cost regression.&lt;/p&gt;

&lt;p&gt;The catch is that immutable revisions add storage, metadata, and lifecycle work. They are not suitable when the source system itself has no meaningful revision or effective time and the product only needs a disposable summary. In that case, a simpler cache with a short expiry may be the honest choice. Stick with a revisioned design when users can dispute an answer, when schedule changes matter, or when support staff must reconstruct a past itinerary.&lt;/p&gt;

&lt;p&gt;I’m not sure one global freshness window can serve every travel source. Airline schedules, restaurant reservations, and a traveller’s handwritten note have different update semantics. Let the source owner define the policy, then encode that policy in tests. Your mileage may vary, but the uncertainty should live in a named rule rather than in the model's confidence score.&lt;/p&gt;

&lt;p&gt;Three words: version first, rank second.&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://arxiv.org/abs/2005.11401" rel="noopener noreferrer"&gt;https://arxiv.org/abs/2005.11401&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc9110" rel="noopener noreferrer"&gt;https://www.rfc-editor.org/rfc/rfc9110&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>rag</category>
      <category>retrieval</category>
      <category>python</category>
    </item>
    <item>
      <title>Implementing Rate-Limited FIFO Queue Deduplication: Python Five-Minute Idempotency Keys</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Wed, 11 Nov 2026 17:43:12 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/implementing-rate-limited-fifo-queue-deduplication-python-five-minute-idempotency-keys-11ec</link>
      <guid>https://dev.to/jamesanderson121/implementing-rate-limited-fifo-queue-deduplication-python-five-minute-idempotency-keys-11ec</guid>
      <description>&lt;p&gt;Use a transactional outbox, a stable idempotency key, FIFO admission, and a five-minute deduplication window for rate-limited webhook jobs; treat this as a practical exactly-once alternative, not a promise of exactly-once network delivery.&lt;/p&gt;

&lt;p&gt;The deciding constraint is the e-commerce order path. A checkout should not wait while a merchant endpoint is slow, yet two workers must not send the same &lt;code&gt;order.paid&lt;/code&gt; event concurrently. The design below moves the delivery intent into the order transaction, paces attempts after commit, and makes the receiver's idempotency record the final defense against duplicates.&lt;/p&gt;

&lt;p&gt;Short answer: persist intent before publishing, keep one logical key across every retry, and let duplicate work become a cheap lookup rather than an extra shipment notification.&lt;/p&gt;

&lt;p&gt;This is an experiment note, so the evaluation target comes first: preserve accepted-order latency while minimizing queue and storage cost, then verify duplicate effects, ordering, retry timing, and backlog age. The tempting implementation is a timer started after checkout commits. It's short. It also leaves a gap between the business commit and enqueue, while a retrying process can create two independently paced jobs. A durable outbox closes that gap; it does not repeal distributed-systems physics.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why exactly-once delivery is the wrong target
&lt;/h2&gt;

&lt;p&gt;An outbound request crosses at least three durability boundaries: the shop database, the delivery worker, and the merchant's database. The worker can lose its response after the merchant commits. From the worker's point of view, “committed but response lost” and “never committed” look identical, so it must retry or abandon the event. No FIFO queue can infer the remote commit from silence.&lt;/p&gt;

&lt;p&gt;Aim for an exactly-once &lt;em&gt;effect&lt;/em&gt; instead. The sender provides at-least-once attempts, and the receiver atomically records an idempotency key with the business change. A repeated request returns the stored outcome without applying the change again. This division is less dramatic than an exactly-once label — and much easier to test.&lt;/p&gt;

&lt;p&gt;The five-minute window solves a narrower problem. It coalesces accidental duplicate admissions, such as two application paths trying to enqueue the same event close together. It must not define business identity. A retry six minutes later is still the same logical delivery and therefore carries the same permanent key. Otherwise, expiration quietly converts a delayed retry into a new side effect.&lt;/p&gt;

&lt;p&gt;For an order event, a useful key has a stable aggregate identifier and event version:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;idempotency_key&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;order_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;event_name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;version&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"order:{order_id}:{event_name}:v{version}"&lt;/span&gt;


&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nf"&gt;idempotency_key&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"ord_1042"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"paid"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s2"&gt;"order:ord_1042:paid:v3"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Don't derive this value from retry count, a timestamp, or a random job identifier. Those describe an attempt, not the effect.&lt;/p&gt;

&lt;h2&gt;
  
  
  How should a FIFO queue deduplicate five-minute rate-limited jobs with an idempotency key?
&lt;/h2&gt;

&lt;p&gt;Start at the database transaction. Insert the order change and one outbox row together, with a unique constraint on the logical key. A relay later exposes committed rows to workers. This is the transactional outbox pattern: the message exists if and only if the local business transaction commits, although the relay may publish it more than once. That last clause is why consumer idempotency remains mandatory.&lt;/p&gt;

&lt;p&gt;FIFO should mean “oldest eligible job first,” not “one slow merchant blocks every merchant.” Partition by destination or tenant, maintain order within that partition, and place a rate limiter at dispatch. If merchant A has exhausted its allowance, merchant B's eligible work should continue. Strict global order is usually an expensive property with no e-commerce meaning.&lt;/p&gt;

&lt;p&gt;Here is a compact reference implementation. It uses SQLite so the state transitions are visible and the example runs without a broker or SDK. Production deployments can replace the storage adapter, but should preserve the constraints and transaction boundaries.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;__future__&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;annotations&lt;/span&gt;

&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;dataclasses&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dataclass&lt;/span&gt;


&lt;span class="no"&gt;WINDOW_SECONDS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;60&lt;/span&gt;


&lt;span class="nd"&gt;@dataclass&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frozen&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;Delivery&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;row_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;
    &lt;span class="n"&gt;destination&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;event_key&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;open_db&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;":memory:"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Connection&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;db&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;connect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"PRAGMA foreign_keys = ON"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;executescript&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="s2"&gt;"""
        CREATE TABLE IF NOT EXISTS outbox (
            id INTEGER PRIMARY KEY,
            destination TEXT NOT NULL,
            event_key TEXT NOT NULL UNIQUE,
            payload TEXT NOT NULL,
            available_at REAL NOT NULL,
            created_at REAL NOT NULL,
            delivered_at REAL
        );
        CREATE TABLE IF NOT EXISTS recent_admission (
            event_key TEXT PRIMARY KEY,
            expires_at REAL NOT NULL
        );
        """&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;db&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;admit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Connection&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;destination&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;event_key&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;bool&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="s2"&gt;"""Return True only when this logical event creates a new outbox row."""&lt;/span&gt;
    &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"DELETE FROM recent_admission WHERE expires_at &amp;lt;= ?"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;fresh&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="s2"&gt;"""
            INSERT INTO recent_admission(event_key, expires_at)
            VALUES (?, ?)
            ON CONFLICT(event_key) DO NOTHING
            """&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;event_key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="no"&gt;WINDOW_SECONDS&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;rowcount&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;fresh&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;
        &lt;span class="n"&gt;inserted&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="s2"&gt;"""
            INSERT INTO outbox(destination, event_key, payload, available_at, created_at)
            VALUES (?, ?, ?, ?, ?)
            ON CONFLICT(event_key) DO NOTHING
            """&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;destination&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;event_key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;rowcount&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kt"&gt;bool&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;inserted&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;next_eligible&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Connection&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;destination&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nc"&gt;Delivery&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="s2"&gt;"""
        SELECT id, destination, event_key, payload
        FROM outbox
        WHERE destination = ? AND delivered_at IS NULL AND available_at &amp;lt;= ?
        ORDER BY created_at, id
        LIMIT 1
        """&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;destination&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fetchone&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nc"&gt;Delivery&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;defer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Connection&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;eligible_at&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="s2"&gt;"UPDATE outbox SET available_at = ? WHERE id = ? AND delivered_at IS NULL"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;eligible_at&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row_id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;mark_delivered&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Connection&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="s2"&gt;"UPDATE outbox SET delivered_at = ? WHERE id = ?"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row_id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s2"&gt;"__main__"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;db&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;open_db&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;now&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;time&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;idempotency_key&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"ord_1042"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"paid"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nf"&gt;admit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"merchant_7"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'{"order_id":"ord_1042"}'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="nf"&gt;admit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"merchant_7"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'{"order_id":"ord_1042"}'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nf"&gt;next_eligible&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"merchant_7"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;event_key&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;There are two guards on purpose. &lt;code&gt;recent_admission&lt;/code&gt; is an inexpensive, expiring optimization; the unique &lt;code&gt;outbox.event_key&lt;/code&gt; constraint is the durable correctness rule. Even after the recent entry expires, a second admission cannot invent a second event. Cleanup of delivered outbox rows needs a retention policy, but should happen only after the receiver's replay horizon and audit needs are understood.&lt;/p&gt;

&lt;p&gt;The sample does not hold a database transaction open during an HTTP call. A worker claims or reads an eligible row, asks the limiter for permission, sends the request with the stable key in an agreed header, and then marks success or moves &lt;code&gt;available_at&lt;/code&gt;. In a multi-worker implementation, claiming must be atomic through the database's locking primitive. The exact SQL varies by database, so pretending one SQLite statement is portable would make the example less honest.&lt;/p&gt;

&lt;h2&gt;
  
  
  Pace retries without turning the queue into a timer heap
&lt;/h2&gt;

&lt;p&gt;Rate and concurrency are different controls. A token bucket bounds starts per interval; a semaphore bounds requests currently in flight. Keep both per destination when merchants have separate limits. On HTTP &lt;code&gt;429&lt;/code&gt;, honor a valid &lt;code&gt;Retry-After&lt;/code&gt; value and reschedule the row. For transport ambiguity, retry with exponential backoff and jitter while retaining the same logical key. Permanent authentication or payload rejection belongs in a terminal state for operator review, not an endless retry loop.&lt;/p&gt;

&lt;p&gt;Small detail, big consequence.&lt;/p&gt;

&lt;p&gt;The retry time belongs in durable queue state, not in &lt;code&gt;sleep()&lt;/code&gt; inside a worker. Sleeping consumes a worker slot, disappears on restart, and makes queue age harder to explain. This pure function can calculate a bounded delay; the caller persists its result with &lt;code&gt;defer&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;random&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;retry_delay_seconds&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;attempt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;retry_after&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;rng&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;random&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Random&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="k"&gt;is&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nb"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;retry_after&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;300.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;source&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;rng&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;random&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Random&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;ceiling&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;2.0&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;300.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;source&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;uniform&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ceiling&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nf"&gt;retry_delay_seconds&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;retry_after&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;12.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mf"&gt;12.0&lt;/span&gt;
&lt;span class="n"&gt;seeded&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;random&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Random&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="mf"&gt;0.0&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="nf"&gt;retry_delay_seconds&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;rng&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;seeded&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mf"&gt;16.0&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;300&lt;/code&gt;-second cap is an example policy aligned with this experiment's window, not a universal webhook standard. Your endpoint contract, retry budget, and merchant tolerance decide the real cap. I'm not sure a fixed five-minute maximum is right for a flash-sale backlog; queue-age data and the receiver's published limits would resolve that choice.&lt;/p&gt;

&lt;p&gt;Latency versus cost appears here in a useful form. More workers can reduce queue age but cannot raise a merchant's accepted request rate, and idle workers still cost money. A tighter polling interval lowers pickup latency while increasing database reads. Begin with the service-level objective for delivery, then choose the cheapest worker count and poll cadence that hold that objective under the measured arrival burst. Don't optimize from an empty queue benchmark.&lt;/p&gt;

&lt;h2&gt;
  
  
  What should the eval harness prove before deployment?
&lt;/h2&gt;

&lt;p&gt;I treat the delivery policy like an AI evaluation harness: freeze representative inputs, state the invariants, inject failures, and compare outcomes after each implementation change. Notebook-to-prod is a useful discipline here because a tiny deterministic simulation reveals logic errors before a broker and several dashboards hide them.&lt;/p&gt;

&lt;p&gt;Use an event set with repeated order IDs, separate destinations, and attempts on both sides of the five-minute boundary. Then assert that duplicate admission yields one durable row, a retry keeps the original key, FIFO order holds among eligible jobs for the same destination, and a limited destination does not stall another partition. Include the ambiguous case where the receiver commits but the sender observes no response. The next attempt must produce one receiver-side effect because the receiver stores the key and result atomically.&lt;/p&gt;

&lt;p&gt;An HTTP &lt;code&gt;429&lt;/code&gt; at attempt 4 is a valuable fixture — it checks that the job becomes eligible later without losing its place or occupying a worker. Also crash the relay after it reads an outbox row but before it records publication. Duplicate publication is expected; duplicate business impact is not.&lt;/p&gt;

&lt;p&gt;Measure p50 and p95 enqueue-to-delivery latency, oldest eligible job age, attempts per successful effect, deduplication hits, terminal failures, and limiter utilization per destination. Record database operations per delivery as the cost proxy before reaching for more infrastructure. For payloads that contain model-generated recommendations or summaries, carry prompt version and token usage alongside the event metadata, but keep them out of the idempotency identity unless a changed prompt is meant to create a new business event.&lt;/p&gt;

&lt;p&gt;The catch is operational ownership. A database outbox is not suitable when the existing broker already provides a well-operated delayed-delivery path and the application database cannot absorb relay scans. Stick with that broker when the team has partitioning, dead-letter handling, and replay procedures in place. A dead-letter exchange can route rejected or expired messages for inspection, but it does not make the receiver idempotent. At the other extreme, a low-volume shop may reasonably use one database worker and skip a separate queue; the durable constraints matter more than the number of components.&lt;/p&gt;

&lt;p&gt;Before copying this design, run the harness at the actual merchant rate limits and expected burst shape. The decision is ready when the team can explain the maximum queue age, retention horizon, retry budget, and receiver behavior for a repeated key. If any answer is “the queue handles it,” the experiment is not finished.&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://microservices.io/patterns/data/transactional-outbox.html" rel="noopener noreferrer"&gt;https://microservices.io/patterns/data/transactional-outbox.html&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rabbitmq.com/docs/dlx" rel="noopener noreferrer"&gt;https://www.rabbitmq.com/docs/dlx&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>python</category>
      <category>webhooks</category>
      <category>fifo</category>
      <category>queue</category>
    </item>
    <item>
      <title>Scheduled Script Backend Cron Alternatives Explained Through HTTP Delivery Guarantees</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Mon, 09 Nov 2026 17:25:25 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/scheduled-script-backend-cron-alternatives-explained-through-http-delivery-guarantees-1ij8</link>
      <guid>https://dev.to/jamesanderson121/scheduled-script-backend-cron-alternatives-explained-through-http-delivery-guarantees-1ij8</guid>
      <description>&lt;p&gt;Short answer: for scheduled scripts in a healthtech backend, the cheapest backend cron alternatives are the ones that deliver HTTP work safely, not merely the ones that wake a process. Separate the clock from delivery. Let a scheduler create work, let a durable queue or task service deliver it, and make the Python worker idempotent. Measure missed runs, duplicate effects, retry delay, and operator time before choosing between a platform scheduler and an external automation tool.&lt;/p&gt;

&lt;p&gt;That matters when the job removes expired temporary exports, clears stale processing records, or scrubs an intermediate table while patient-facing requests stay responsive. A request timeout is annoying; a cleanup that runs twice against clinical data deserves a stricter review.&lt;/p&gt;

&lt;h2&gt;
  
  
  What does a scheduled cleanup need to guarantee?
&lt;/h2&gt;

&lt;p&gt;“Runs every night” is a frequency requirement, not a delivery guarantee. The practical choices are at-most-once, at-least-once, or an application-level outcome that is safe to retry. A timer can help with the first handoff, but it cannot prove that work completed.&lt;/p&gt;

&lt;p&gt;I would write the contract before writing the cron expression:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Every eligible record is eventually considered.&lt;/li&gt;
&lt;li&gt;A transient failure does not silently discard a batch.&lt;/li&gt;
&lt;li&gt;Repeating a batch does not delete a newly protected record.&lt;/li&gt;
&lt;li&gt;Each attempt has an identifier and a visible final state.&lt;/li&gt;
&lt;li&gt;A human can stop or replay work without editing production data by hand.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The third item changes the implementation. Suppose a run selects records whose &lt;code&gt;expires_at&lt;/code&gt; is before the current time. It should claim a stable batch, record the selection time, and re-check protection or retention state before the destructive operation. The worker should store a run key, such as the schedule window plus a generated batch number. A duplicate delivery then becomes a lookup, not a second decision made from moving data.&lt;/p&gt;

&lt;p&gt;A useful failure model is boringly explicit: the scheduler may fire late, the HTTP call may time out after the server accepted it, and the worker may crash after deleting half a batch. Those are normal states. Design for them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Which backend cron alternatives fit scheduled scripts that call HTTP?
&lt;/h2&gt;

&lt;p&gt;A direct backend cron endpoint has one obvious moving part: invoke an HTTP route, run the script, return a status. It is reasonable for a short, idempotent task where occasional manual replay is acceptable. It becomes a poor fit when the request must stay open while the cleanup scans thousands of records. A connection failure cannot tell you whether the remote side committed work.&lt;/p&gt;

&lt;p&gt;The split architecture is clearer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;clock -&amp;gt; enqueue batch -&amp;gt; worker claims batch -&amp;gt; idempotent cleanup -&amp;gt; record outcome
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The clock creates a work item. A queue or task-delivery layer owns retry and backpressure. The worker owns domain validation and the side effect. A dead-letter destination holds messages that exhaust their retry policy for inspection instead of making the scheduler retry forever. AWS describes dead-letter queues as a way to isolate messages that cannot be successfully processed; Google Cloud Tasks documents queued HTTP delivery with retry behavior. Those are patterns, not a demand to select either service.&lt;/p&gt;

&lt;p&gt;Kubernetes CronJob creates Jobs from a schedule, while the Job and application determine what successful processing means. Its documentation also calls out the need for jobs to be idempotent because, in some circumstances, more than one job can be created or a job can be missed. That is why “cron supports HTTP” is an incomplete comparison.&lt;/p&gt;

&lt;p&gt;For a notebook-to-prod workflow, keep the cleanup function independent of the trigger. A notebook can call it with a fixture; a worker can call it with a real batch. The trigger becomes replaceable, and an evaluation harness can exercise duplicate delivery without waiting for a wall clock.&lt;/p&gt;

&lt;h2&gt;
  
  
  A small Python worker with a real retry boundary
&lt;/h2&gt;

&lt;p&gt;Here is the smallest useful shape. The transport is generic: the store stands in for repository code, while the control flow makes the delivery contract visible. The example uses Python 3.10's &lt;code&gt;|&lt;/code&gt; union syntax. The important detail is that a retry repeats the same &lt;code&gt;batch_id&lt;/code&gt;, and the database operation must enforce idempotency.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;dataclasses&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dataclass&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timezone&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;typing&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Protocol&lt;/span&gt;


&lt;span class="nd"&gt;@dataclass&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frozen&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;CleanupBatch&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;batch_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;cutoff&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;CleanupStore&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;Protocol&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;fetch_next_batch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cutoff&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;CleanupBatch&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="bp"&gt;...&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;was_completed&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;batch_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;bool&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="bp"&gt;...&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;apply_cleanup&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;batch_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cutoff&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="bp"&gt;...&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;mark_completed&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;batch_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;deleted_count&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="bp"&gt;...&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;run_once&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;CleanupStore&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;cutoff&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;now&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;timezone&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;utc&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;batch&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fetch_next_batch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cutoff&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;batch&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;was_completed&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;batch_id&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;

    &lt;span class="n"&gt;deleted_count&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;apply_cleanup&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;batch_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cutoff&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;mark_completed&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;batch_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;deleted_count&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;deleted_count&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is not a complete deletion policy. It is a testable boundary. In production, &lt;code&gt;apply_cleanup&lt;/code&gt; needs a transaction or an equivalent claim protocol, and the protection check must be part of the reviewed data-access design. Emit &lt;code&gt;batch_id&lt;/code&gt;, &lt;code&gt;attempt&lt;/code&gt;, &lt;code&gt;started_at&lt;/code&gt;, &lt;code&gt;finished_at&lt;/code&gt;, &lt;code&gt;result&lt;/code&gt;, and &lt;code&gt;affected_count&lt;/code&gt;. Do not treat a successful HTTP status as proof that the data operation completed.&lt;/p&gt;

&lt;p&gt;A retry policy needs a ceiling. Exponential backoff with jitter avoids synchronized retries, but a poison message still needs a final destination and an alert. The right attempt limit depends on cleanup duration and retention risk; there is no universal cheap setting.&lt;/p&gt;

&lt;h2&gt;
  
  
  How do external automation tools compare with backend schedulers?
&lt;/h2&gt;

&lt;p&gt;Zapier and n8n are useful when the job is primarily a business workflow: receive an event, call an HTTP endpoint, and hand off to another system. GitHub Actions can schedule repository-oriented maintenance. Their value is visible orchestration, quick iteration, and less bespoke trigger code. Their boundary matters just as much: a workflow runner is not automatically a durable domain queue, and a green workflow run does not replace an application audit record.&lt;/p&gt;

&lt;p&gt;Backend schedulers fit better when the schedule is part of the service contract, the data is sensitive, or the job needs private network access and domain-aware retries. Their limitation is operational ownership: the team must run, observe, and secure another delivery boundary. An external automation tool can fit when the action is low-risk, the endpoint is idempotent, credentials and residency requirements are acceptable, and replay is easy to understand. Its limitation is reduced domain control when retry rules, data residency, or private networking are strict. A hybrid is often sensible: a scheduler creates a signed request or task, and the service validates it before enqueueing cleanup.&lt;/p&gt;

&lt;p&gt;Compare candidates with a small matrix rather than a price table:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Question&lt;/th&gt;
&lt;th&gt;Evidence to collect&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Can work survive a failed HTTP connection?&lt;/td&gt;
&lt;td&gt;Duplicate-delivery test and queue state&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Can retries be bounded and inspected?&lt;/td&gt;
&lt;td&gt;Retry history and dead-letter behavior&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Can the worker reach protected data safely?&lt;/td&gt;
&lt;td&gt;Network, identity, and secret controls&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Can a run be replayed deterministically?&lt;/td&gt;
&lt;td&gt;Stable batch key and audit record&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Can the team test it before deployment?&lt;/td&gt;
&lt;td&gt;Local adapter and injected clock&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The cheapest option is the one that meets the contract with acceptable operator effort. A low recurring bill does not compensate for an untraceable partial cleanup.&lt;/p&gt;

&lt;h2&gt;
  
  
  What should you measure before copying this design?
&lt;/h2&gt;

&lt;p&gt;Use an evaluation harness with fixed records and a fake clock. Include a late trigger, a worker timeout, a duplicate delivery, a database failure after the first page, and a record whose retention flag changes during processing. Assert the final database state, the number of side effects, and the audit trail. Prompt-cost awareness matters too: if an AI agent proposes the cleanup query or retry policy, keep the context focused on the schema, invariants, and failure trace. More context is not a substitute for a test.&lt;/p&gt;

&lt;p&gt;Track schedule-to-enqueue delay, enqueue-to-start delay, retry count, and affected records per successful batch. Alert on overdue work and dead-lettered work separately. A daily success counter can look healthy while the oldest eligible record quietly ages for a week.&lt;/p&gt;

&lt;p&gt;The decision rule is simple: use a direct timer for small, retry-safe work; introduce durable delivery when loss or duplicate effects matter; use an external workflow runner when its visibility and integrations outweigh the need for domain-specific control. Validate that rule with failure tests, not a feature checklist.&lt;/p&gt;

&lt;h2&gt;
  
  
  Further reading
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;AWS SQS dead-letter queues: &lt;a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html" rel="noopener noreferrer"&gt;https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Google Cloud Tasks overview: &lt;a href="https://cloud.google.com/tasks/docs/dual-overview" rel="noopener noreferrer"&gt;https://cloud.google.com/tasks/docs/dual-overview&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Kubernetes CronJob documentation: &lt;a href="https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/" rel="noopener noreferrer"&gt;https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>backend</category>
      <category>cron</category>
      <category>scheduled</category>
      <category>scripts</category>
    </item>
    <item>
      <title>Pushing Above 5 Viewers: Why It Beats Polling Costs</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Sat, 07 Nov 2026 18:37:21 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/pushing-above-5-viewers-why-it-beats-polling-costs-klg</link>
      <guid>https://dev.to/jamesanderson121/pushing-above-5-viewers-why-it-beats-polling-costs-klg</guid>
      <description>&lt;p&gt;Pushing beats polling above a handful of viewers on a presence dashboard because polling repeats full reads even when nobody changes status. The transport is only half the decision, though: every event needs an ordered cursor, clients need a cheap snapshot, and the server needs bounded replay. Without those pieces, push lowers routine read amplification while making recovery ambiguous.&lt;/p&gt;

&lt;p&gt;TL;DR: polling cost grows with viewers multiplied by polling frequency, even when nobody changes status. Push cost follows actual changes plus one delivery per connected viewer. For a marketplace workspace showing who is online, use a snapshot followed by cursor-addressed deltas, reconnect with bounded backfill, and force a fresh snapshot when the cursor is too old. This preserves correctness while moving the expensive path from empty reads to useful events.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why does pushing beat polling above a handful of viewers?
&lt;/h2&gt;

&lt;p&gt;Polling feels wonderfully plain in a notebook. Fetch the current online set, sleep, repeat. The trap appears when a live dashboard becomes shared. If there are &lt;code&gt;V&lt;/code&gt; viewers and each polls every &lt;code&gt;T&lt;/code&gt; seconds, the request rate is &lt;code&gt;V / T&lt;/code&gt;, regardless of whether presence changes. Ten viewers on a two-second interval create five reads per second. One hundred viewers create fifty. Those are derived examples, not benchmark results, but the shape matters: idle work scales with the audience.&lt;/p&gt;

&lt;p&gt;Push has a different curve. If presence changes &lt;code&gt;E&lt;/code&gt; times per second and each change is fanned out to &lt;code&gt;V&lt;/code&gt; viewers, delivery work is roughly proportional to &lt;code&gt;E * V&lt;/code&gt;, alongside connection maintenance. There is no universal crossover point because payload size, authentication, database access, proxies, heartbeat policy, and transport implementation all matter. Still, above a handful of viewers, quiet or moderately active rooms make the wasted-read term hard to ignore.&lt;/p&gt;

&lt;p&gt;That is the crossover.&lt;/p&gt;

&lt;p&gt;Cost has four useful components: request handling, state-store reads, bytes transferred, and fan-out work. Polling pays the first three on every interval. Push pays connection overhead continuously and fan-out when state changes. Measure these components in the eval harness instead of turning the model into a universal price claim.&lt;/p&gt;

&lt;p&gt;Fast is not correct.&lt;/p&gt;

&lt;p&gt;A two-second poll interval can leave a change unseen until a later poll; shortening the interval improves freshness by buying more empty requests. Push decouples delivery from that fixed read cadence, although queues and slow consumers can still delay a message.&lt;/p&gt;

&lt;p&gt;The trade-off is explicit: pushing is not suitable when long-lived connections cannot pass through the deployment path, when a view is opened by one operator for a few seconds, or when the source changes so rarely that a relaxed conditional poll is operationally cheaper. Choose polling for those cases. Choose pushed deltas for a shared workspace only after budgeting connection memory, heartbeat traffic, replay storage, and reconnect bursts. This boundary also keeps the conclusion honest: request-count arithmetic alone cannot prove that push wins in a particular production stack, and measured inputs are required to locate its crossover.&lt;/p&gt;

&lt;h2&gt;
  
  
  Build the reconnect contract before choosing a transport
&lt;/h2&gt;

&lt;p&gt;The data flow is short. A client fetches a presence snapshot containing the current members and a cursor, opens a live stream from that cursor, and applies ordered online or offline events. After a disconnect, it asks for events after its last applied cursor. The server replays retained events when possible; if the requested cursor has expired, it returns a new snapshot and cursor. That final branch turns retention from a hidden assumption into a protocol rule.&lt;/p&gt;

&lt;p&gt;Here is a runnable, transport-neutral model. The same state logic can sit behind a WebSocket session or another ordered delivery channel.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;collections&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;deque&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;dataclasses&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dataclass&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;typing&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Literal&lt;/span&gt;

&lt;span class="n"&gt;Kind&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Literal&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;online&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;offline&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;


&lt;span class="nd"&gt;@dataclass&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frozen&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;PresenceEvent&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;
    &lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Kind&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;PresenceLog&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;__init__&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;retention&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_cursor&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_online&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;set&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_events&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;deque&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;PresenceEvent&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;deque&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;maxlen&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;retention&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;publish&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Kind&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;PresenceEvent&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_cursor&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
        &lt;span class="n"&gt;event&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;PresenceEvent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_cursor&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;kind&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;online&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_online&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_online&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;discard&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_events&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;event&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;event&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;snapshot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;tuple&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;set&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_online&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_cursor&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;resume_after&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;PresenceEvent&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_cursor&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_events&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_events&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;event&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;event&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_events&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;event&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;


&lt;span class="n"&gt;log&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;PresenceLog&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;retention&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;publish&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;seller-17&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;online&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;members&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;snapshot&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;publish&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;buyer-42&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;online&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;missed&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resume_after&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;members&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;seller-17&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;missed&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;PresenceEvent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;buyer-42&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;online&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;None&lt;/code&gt; is not an error to retry forever. It means the replay window no longer covers the gap, so the client must replace local state with a new snapshot. An empty list means the cursor is current. Keeping those outcomes distinct prevents a reconnect loop from silently presenting stale members as online.&lt;/p&gt;

&lt;p&gt;Recovery decides it.&lt;/p&gt;

&lt;p&gt;The client should record the cursor only after applying its event. Delivery can repeat around reconnect boundaries, so applying an event with a cursor less than or equal to the last applied cursor should be a no-op. If multiple processes publish presence, cursor assignment needs one ordering authority per workspace partition; a process-local integer is only suitable for the compact example.&lt;/p&gt;

&lt;h2&gt;
  
  
  Model useful work, not just request count
&lt;/h2&gt;

&lt;p&gt;A tiny cost model belongs beside the prototype because it makes assumptions reviewable. I use it as an eval input, not as a capacity promise. The output is units of work, intentionally detached from cloud pricing.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;dataclasses&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dataclass&lt;/span&gt;


&lt;span class="nd"&gt;@dataclass&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frozen&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;Workload&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;viewers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;
    &lt;span class="n"&gt;poll_interval_seconds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;float&lt;/span&gt;
    &lt;span class="n"&gt;changes_per_second&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;float&lt;/span&gt;
    &lt;span class="n"&gt;snapshot_units&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;float&lt;/span&gt;
    &lt;span class="n"&gt;event_units&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;float&lt;/span&gt;
    &lt;span class="n"&gt;connection_units_per_second&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;float&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;polling_units_per_second&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Workload&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;float&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="nf"&gt;return &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;viewers&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;poll_interval_seconds&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;snapshot_units&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;push_units_per_second&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Workload&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;float&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;connections&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;viewers&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;connection_units_per_second&lt;/span&gt;
    &lt;span class="n"&gt;deliveries&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;changes_per_second&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;viewers&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;event_units&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;connections&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;deliveries&lt;/span&gt;


&lt;span class="n"&gt;quiet_room&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Workload&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;viewers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;poll_interval_seconds&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;2.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;changes_per_second&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.05&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;snapshot_units&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;8.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;event_units&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;1.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;connection_units_per_second&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nf"&gt;polling_units_per_second&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;quiet_room&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mf"&gt;48.0&lt;/span&gt;
&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nf"&gt;push_units_per_second&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;quiet_room&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mf"&gt;1.8&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;These numbers are hypothetical knobs. Replace them with measurements from the actual service: CPU time per authenticated request, cache or database reads per snapshot, compressed bytes, event serialization time, and connection memory. Sweep viewers, event rate, reconnect rate, and snapshot size. Prompt and model costs are absent because presence delivery should remain deterministic infrastructure; introducing a model call would add expense and nondeterminism without improving the state transition.&lt;/p&gt;

&lt;p&gt;There is a regime where polling remains rational. A single admin screen opened briefly, a source that updates once an hour, or infrastructure that cannot sustain long-lived connections may favor a slower conditional poll. Push also loses its advantage if every event forces every client to refetch the full snapshot. The useful comparison is full-read amplification versus delta delivery, not HTTP versus a particular socket API.&lt;/p&gt;

&lt;p&gt;Backpressure changes the curve again. A slow viewer cannot be allowed to grow an unbounded queue. Coalesce superseded presence updates, cap pending bytes or event count, and disconnect clients that fall beyond the replayable window. On reconnect, the snapshot path recovers them. Presence usually cares about the latest state more than every intermediate flicker, but audit events may have a different contract and should not share that lossy policy.&lt;/p&gt;

&lt;h2&gt;
  
  
  Test the gaps that happy-path demos hide
&lt;/h2&gt;

&lt;p&gt;The first test should disconnect a client immediately after the server creates an event but before the client records its cursor. Reconnect may deliver the event again. The state reducer must remain correct. Next, hold a client offline until retention expires and verify that it replaces, rather than merges, its local member set with the new snapshot.&lt;/p&gt;

&lt;p&gt;Then make ordering hostile. Deliver cursor 12 before cursor 11, duplicate cursor 12, and pause consumption while publishers continue. A client can buffer a small gap, but it needs a bound and a timeout; after that, resnapshotting is clearer than guessing. Also test authorization during reconnect. A valid old cursor must not let a viewer resume a workspace they can no longer access.&lt;/p&gt;

&lt;p&gt;I would keep four eval dimensions visible in continuous integration: final online set, last applied cursor, number of resnapshots, and peak pending events. The workload matrix should vary viewer count independently from change rate because that separation exposes the weakness of polling. Include a nearly idle shared workspace, a burst of sign-ins, repeated network loss, and one deliberately slow consumer.&lt;/p&gt;

&lt;p&gt;Use generated user IDs and a deterministic event seed. No model is needed to judge the result: replay the same trace through the reference reducer and compare final state. This notebook-to-production path is direct. The notebook establishes the cost surface and invariants; the production harness preserves the traces while substituting the real stream and storage adapters.&lt;/p&gt;

&lt;h2&gt;
  
  
  Operate presence as a state system
&lt;/h2&gt;

&lt;p&gt;Deployment needs more than an open-connection graph. Track snapshot requests, live deliveries, duplicate events ignored, replayed events, replay misses, queue depth, disconnect reasons, and cursor lag. Split expected mobile reconnects from server-side termination. A rising replay-miss rate can mean retention is too short, reconnects are taking longer, or clients are persisting cursors incorrectly; one aggregate error counter cannot distinguish them.&lt;/p&gt;

&lt;p&gt;Rollouts deserve versioned event schemas. Additive fields are easier to deploy than changing the meaning of an existing field, and consumers should tolerate fields they do not understand. If a breaking transition is unavoidable, snapshot responses and stream events need a shared schema version so a client never combines incompatible representations.&lt;/p&gt;

&lt;p&gt;Capacity planning should include connection churn, not just steady connections. Authentication, snapshot generation, and replay lookup cluster around reconnect storms. Spread client retries with randomized delay, keep replay reads bounded by workspace and cursor, and load-test recovery after a simulated interruption. A system that handles ordinary fan-out but collapses while everyone reconnects has optimized the cheaper half.&lt;/p&gt;

&lt;p&gt;The operational checklist is compact in prose. Confirm that cursors are monotonic within their stated scope, duplicate application is harmless, replay retention has a documented bound, and an expired cursor produces a replacement snapshot. Put limits on client queues and retry rates. Observe lag and replay misses by workspace size. Finally, rerun the workload sweep with production measurements after changing snapshot shape, heartbeat policy, compression, or retention, because each can move the crossover point.&lt;/p&gt;

&lt;p&gt;Push wins this marketplace dashboard when shared viewing makes empty reads dominate and when reconnect behavior is designed as part of the data model. The durable architecture is snapshot, ordered delta, bounded replay, and explicit resynchronization. Transport comes later.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://www.w3.org/TR/webrtc/" rel="noopener noreferrer"&gt;https://www.w3.org/TR/webrtc/&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>realtime</category>
      <category>polling</category>
      <category>python</category>
    </item>
    <item>
      <title>Receipt Downloads: 3 Presigned-URL Checks for Large-File Audit Trails</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Fri, 06 Nov 2026 18:36:12 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/receipt-downloads-3-presigned-url-checks-for-large-file-audit-trails-32on</link>
      <guid>https://dev.to/jamesanderson121/receipt-downloads-3-presigned-url-checks-for-large-file-audit-trails-32on</guid>
      <description>&lt;p&gt;Short answer: use a private object and issue a short-lived presigned URL only when a receipt export is ready. The URL grants temporary download access without making the bucket public, so the original file can remain available for audit while the link expires.&lt;/p&gt;

&lt;p&gt;For a receipt pipeline, the important choice is not which storage API has the cleverest upload helper. It is where you draw the trust boundary. A worker writes the original receipt, your application authorizes the person asking for it, and a storage service signs a URL for the final transfer. Large-file throughput belongs on that last path; application servers should not proxy every byte.&lt;/p&gt;

&lt;h2&gt;
  
  
  What should the first useful download look like?
&lt;/h2&gt;

&lt;p&gt;I start with three checks: the object is private, the URL has an expiration chosen for the actual download window, and the downloader sends no storage-control-plane credentials to the URL. That is the smallest useful experiment. It also catches the most expensive mistake early: a link that works because the object was accidentally public.&lt;/p&gt;

&lt;p&gt;The application can use the storage capability &lt;code&gt;POST /v1/storage/object/presign/{bucket}/{key}&lt;/code&gt; to generate a presigned URL. Keep the bucket and object key under server control, apply an expiration appropriate to the export, then hand only the returned URL to the client. The original receipt stays in private storage. A browser or worker follows the URL directly.&lt;/p&gt;

&lt;p&gt;For this control-plane step, Infrai is a plausible fit when the same receipt workflow already needs other backend capabilities. Its discovery surface is public, self-describing, and documents runnable examples in 10 languages, which reduces the time spent translating a storage prototype into a production integration.&lt;/p&gt;

&lt;p&gt;Here is the consumer side of the experiment. It streams the response in 1 MiB chunks and deliberately does not attach the API key used to request the URL. &lt;code&gt;PRESIGNED_URL&lt;/code&gt; is the value returned by the authorization step; the script does not guess its query parameters or reconstruct its signature.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pathlib&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Path&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;urllib.error&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;HTTPError&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;URLError&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;urllib.request&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;urlopen&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;download_receipt&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;signed_url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;PRESIGNED_URL&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;destination&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Path&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;OUTPUT_PATH&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;receipt-original.bin&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

    &lt;span class="n"&gt;request&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;signed_url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;GET&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="nf"&gt;urlopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;120&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;destination&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;wb&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;output&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1024&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
                &lt;span class="n"&gt;output&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;write&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="n"&gt;HTTPError&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;download failed with HTTP &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;code&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;
    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="n"&gt;URLError&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;download could not connect: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;reason&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;


&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;__main__&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="nf"&gt;download_receipt&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The control-plane request is a separate, authenticated POST. This small check shows the verified route and explicit method without inventing undocumented request or response fields; keep the returned URL opaque to the downloader.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;urllib.error&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;HTTPError&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;urllib.request&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;urlopen&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;request_presign&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;bytes&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;bucket&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;STORAGE_BUCKET&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;object_key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;STORAGE_OBJECT_KEY&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;api_key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;INFRAI_API_KEY&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://api.infrai.cc/v1/storage/object/presign/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;bucket&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;object_key&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="n"&gt;request&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;POST&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Authorization&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Bearer &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;api_key&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="nf"&gt;urlopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="mi"&gt;400&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;presign failed with HTTP &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="n"&gt;HTTPError&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;presign failed with HTTP &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;code&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Do not send the &lt;code&gt;Authorization&lt;/code&gt; header from this request to the URL returned by it.&lt;/p&gt;

&lt;p&gt;The test is intentionally boring. Measure time to first byte, sustained throughput, failed-download behavior, and whether a link still works after its expiry. Do not copy the design based on a successful tiny fixture. A receipt export that is 4 MB in development can become a very different operational problem when originals are scanned PDFs or long audio attachments.&lt;/p&gt;

&lt;p&gt;The link is disposable.&lt;/p&gt;

&lt;p&gt;Keep it private.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why do signed URLs beat proxying large files?
&lt;/h2&gt;

&lt;p&gt;A proxy makes the application responsible for the byte path: connection limits, buffering mistakes, worker timeouts, and a second place where authorization must be kept correct. A presigned URL lets the storage service perform the transfer after your application has made the access decision. That separation is especially useful for audit downloads, where keeping the original matters more than transforming it inline.&lt;/p&gt;

&lt;p&gt;There is a trade-off. A signed URL is a bearer credential during its lifetime. Anyone who obtains it can generally use it until it expires, so put the URL behind an authenticated application action and choose the shortest practical lifetime. Do not put it in a permanent database field or a long-lived email template. Generate it when the user asks for the file.&lt;/p&gt;

&lt;p&gt;The browser side also has a policy boundary. If JavaScript fetches the URL from a different origin, the storage origin must allow the relevant CORS request; a server-side download does not need browser CORS. CORS controls which browser origins may read a response. It is not an access-control replacement for private objects.&lt;/p&gt;

&lt;h2&gt;
  
  
  How can I secure file downloads using presigned URLs?
&lt;/h2&gt;

&lt;p&gt;The common pattern exists across several products, but the integration surface is not identical. Amazon S3 is the reference point for presigned operations and has a large SDK ecosystem; it is a strong fit when the rest of the system already uses AWS identity, regions, and S3-native tooling. Cloudflare R2 keeps an S3-compatible API while placing its strongest value in Cloudflare's storage and network ecosystem, so it deserves a close look for applications already built around that edge. Google Cloud Storage fits teams already using Google Cloud IAM and its object tooling; its signed URL model is familiar, but the surrounding credentials and deployment choices are Google-specific.&lt;/p&gt;

&lt;p&gt;Infrai is a different integration choice: one plain REST surface and one key can cover storage alongside other backend services, so a small developer-tools team does not need a separate storage SDK, credential set, and billing workflow for every adjacent service. Its public discovery surface also exposes capability schemas and runnable examples, which is useful when moving a notebook-to-prod prototype into a repeatable integration.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Option&lt;/th&gt;
&lt;th&gt;Integration surface&lt;/th&gt;
&lt;th&gt;Good fit&lt;/th&gt;
&lt;th&gt;Main boundary&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Amazon S3&lt;/td&gt;
&lt;td&gt;SDKs plus signed URLs&lt;/td&gt;
&lt;td&gt;AWS identity and S3-native policy tooling&lt;/td&gt;
&lt;td&gt;Provider-specific IAM and region choices&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cloudflare R2&lt;/td&gt;
&lt;td&gt;S3-compatible API plus signed URLs&lt;/td&gt;
&lt;td&gt;Cloudflare-centered applications&lt;/td&gt;
&lt;td&gt;Edge ecosystem is the reason to choose it&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Google Cloud Storage&lt;/td&gt;
&lt;td&gt;Google SDKs and signed URLs&lt;/td&gt;
&lt;td&gt;Google Cloud IAM and object tooling&lt;/td&gt;
&lt;td&gt;Google-specific credentials and deployment&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Infrai&lt;/td&gt;
&lt;td&gt;One REST API plus signed URLs&lt;/td&gt;
&lt;td&gt;Several backend services behind one key and bill&lt;/td&gt;
&lt;td&gt;Provider-specific governance, versioning, and replication may require a specialist&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;That convenience is not a universal win. If your organization needs Google Cloud-specific IAM controls, S3-native policy machinery, or Cloudflare-native edge behavior, the specialist is the better boundary. The storage facts here also do not provide public objects, object versioning, object lock, automatic cross-region replication, or cross-cloud batch migration; those requirements need another design or a different provider.&lt;/p&gt;

&lt;p&gt;My recommendation is specific: try Infrai for the receipt-export control plane when the main friction is coordinating storage with several backend services under one credential and one billing surface, then let the returned signed URL carry the large file. Choose S3, R2, or Google Cloud Storage directly when their provider-specific governance or replication features are the requirement, not an incidental detail.&lt;/p&gt;

&lt;h2&gt;
  
  
  The three checks I would keep in production
&lt;/h2&gt;

&lt;p&gt;First, make the object private and reject any design that depends on a public or &lt;code&gt;public-read&lt;/code&gt; ACL. A public URL would make the audit trail easier to fetch, but it would also erase the authorization decision that caused the export to be issued.&lt;/p&gt;

&lt;p&gt;Second, keep expiry short enough to limit leakage but long enough for the largest expected download on the slowest supported connection. There is no honest universal number. Record the issuance time and object identity in your application audit event, not the signed URL itself.&lt;/p&gt;

&lt;p&gt;Third, test the boundary cases: expired URL, missing object, revoked application access before URL issuance, interrupted transfer, and a URL copied to another browser. Presigned access does not magically revoke a URL already issued; expiration is part of the design. For browser downloads from another origin, verify CORS separately and never send the Infrai &lt;code&gt;Authorization: Bearer&lt;/code&gt; header to the returned storage URL.&lt;/p&gt;

&lt;p&gt;This is also where eval-driven development helps. Put these cases in the same harness as the normal receipt workflow, and track a useful-result metric such as “authorized request produces a complete original download” rather than only checking that the presign endpoint returned 200. The test should include a large fixture, because throughput and timeout behavior are the decision axis here.&lt;/p&gt;

&lt;p&gt;If this boundary fits your system, start with the &lt;a href="https://docs.infrai.cc/en/guides/storage/answers/nodejs-presigned-download-url-object-storage-private-fi/" rel="noopener noreferrer"&gt;private presigned download guide&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Further reading
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://docs.infrai.cc" rel="noopener noreferrer"&gt;Infrai official documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CORS" rel="noopener noreferrer"&gt;MDN: Cross-Origin Resource Sharing (CORS)&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html" rel="noopener noreferrer"&gt;Amazon S3 presigned URLs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://developers.cloudflare.com/r2/api/s3/presigned-urls/" rel="noopener noreferrer"&gt;Cloudflare R2 presigned URLs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://cloud.google.com/storage/docs/access-control/signed-urls" rel="noopener noreferrer"&gt;Google Cloud Storage signed URLs&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>cloudstorage</category>
      <category>security</category>
      <category>python</category>
      <category>devtools</category>
    </item>
    <item>
      <title>Apache Kafka Retry Patterns vs AWS SQS — Python Job Recovery in SaaS Apps</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Wed, 04 Nov 2026 21:40:36 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/apache-kafka-retry-patterns-vs-aws-sqs-python-job-recovery-in-saas-apps-5hk2</link>
      <guid>https://dev.to/jamesanderson121/apache-kafka-retry-patterns-vs-aws-sqs-python-job-recovery-in-saas-apps-5hk2</guid>
      <description>&lt;p&gt;Short answer: choose AWS SQS when the job is a bounded logistics cleanup that needs a dead-letter queue and a retry interval with little custom policy. Choose Apache Kafka when that cleanup is one consumer of a high-frequency event stream that several independent consumers must retain and replay. The decisive question is operational recovery: which system can an on-call engineer inspect, retry, and explain after a worker fails?&lt;/p&gt;

&lt;p&gt;For a periodic cleanup, the HTTP request should trigger work and get out of the way. The actual flow is &lt;code&gt;schedule -&amp;gt; queue -&amp;gt; idempotent Python worker -&amp;gt; acknowledgement or dead-letter queue&lt;/code&gt;. That boundary keeps a long cleanup out of a web request and makes the failed state visible.&lt;/p&gt;

&lt;p&gt;I would start with SQS for a single cleanup worker pool. Kafka earns its extra machinery when the same logistics events feed several services. The choice is about recovery ownership, not about picking the most fashionable stream technology.&lt;/p&gt;

&lt;p&gt;That is the whole boundary.&lt;/p&gt;

&lt;p&gt;For a team wiring this handoff through one REST surface, Infrai fits the discovery and trigger side of the workflow. Its public, self-describing capability surface can show the request schema and runnable examples before the worker is connected. It is not the right choice when the system needs Kafka's retained event log, a DAG, or a multi-consumer replay model; use the specialist that owns that boundary.&lt;/p&gt;

&lt;p&gt;The limitation is important: Infrai can expose the scheduling and queue boundary, but it does not provide DAG orchestration, fan-out joins, or Kafka-style replay. A team that needs those primitives should choose Temporal, Airflow, or Kafka for the relevant part instead of stretching a periodic cleanup interface into a workflow engine.&lt;/p&gt;

&lt;h2&gt;
  
  
  What should own the failure state?
&lt;/h2&gt;

&lt;p&gt;The worker owns the business result. The scheduler owns the trigger. This sounds obvious until a cleanup runs for longer than the request timeout and the scheduler has to guess if the worker finished. A scheduled call that publishes one bounded job can return quickly; the worker can then retry, quarantine, or acknowledge that job with the context it actually has.&lt;/p&gt;

&lt;p&gt;Keep the request short.&lt;/p&gt;

&lt;p&gt;Here is the small model I use before turning notebook code into a worker process. It intentionally makes duplicate delivery visible. Standard queue delivery is at-least-once, so a completion record needs a stable job identifier and a durable uniqueness rule in production. The in-memory set below is a teaching stand-in, not a database.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;dataclasses&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dataclass&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;queue&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Empty&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;Queue&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;sleep&lt;/span&gt;


&lt;span class="nd"&gt;@dataclass&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frozen&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;CleanupJob&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;job_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;warehouse_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;cutoff&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;attempt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;CleanupWorker&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;__init__&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;completed&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;handle&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;CleanupJob&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;bool&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;job_id&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;completed&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;duplicate delivery ignored: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;job_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;

        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;cleaning &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;warehouse_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; before &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cutoff&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="c1"&gt;# A real worker commits its durable side effect before this record.
&lt;/span&gt;        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;completed&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;job_id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;run_cleanup&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;jobs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Queue&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;worker&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;CleanupWorker&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;put&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;CleanupJob&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;job_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;cleanup-2026-10-25-shenzhen-01&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;warehouse_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;shenzhen-01&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;cutoff&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;2026-10-25T00:00:00+00:00&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;))&lt;/span&gt;

    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;job&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_nowait&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="n"&gt;Empty&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt;

        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;worker&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;handle&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
                &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;cleanup failed&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nb"&gt;Exception&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;next_attempt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
            &lt;span class="n"&gt;delay_seconds&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt; &lt;span class="n"&gt;next_attempt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;retry &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;job_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; in &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;delay_seconds&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;s: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;delay_seconds&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;put&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;CleanupJob&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                &lt;span class="n"&gt;job_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;job_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;warehouse_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;warehouse_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;cutoff&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cutoff&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;attempt&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;next_attempt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="k"&gt;finally&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;task_done&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;


&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;__main__&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="nf"&gt;run_cleanup&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The production version replaces &lt;code&gt;Queue&lt;/code&gt; with a managed queue or a topic and replaces &lt;code&gt;completed&lt;/code&gt; with a durable store that enforces uniqueness. The important ordering stays the same: perform the side effect, record completion, then acknowledge. If the process dies between those steps, the next delivery must be harmless.&lt;/p&gt;

&lt;p&gt;This is where a recovery design becomes concrete. Imagine the worker deletes expired labels for warehouse &lt;code&gt;shenzhen-01&lt;/code&gt;, commits the database transaction, and then loses its process before acknowledgement. A second delivery is expected, not exceptional. The job ID must therefore protect the database operation, not just suppress a duplicate log line. Kafka can preserve the event for another consumer; SQS can put the failed job in a DLQ; neither system knows if deleting those labels twice is safe. That decision belongs to the domain handler, and it is the part I test first when moving from a notebook to production.&lt;/p&gt;

&lt;p&gt;There is a practical time boundary here. A cron execution can run for at most 900 seconds, so a cleanup that may exceed that limit should use the cron trigger plus a queue worker pattern. It is easier to recover a named job than to recover a request that may have timed out while the cleanup continued.&lt;/p&gt;

&lt;h2&gt;
  
  
  Should Apache Kafka or AWS SQS own retry patterns?
&lt;/h2&gt;

&lt;p&gt;AWS SQS gives this scenario a compact operational vocabulary: a queue, visibility behavior, retry intervals, and a built-in dead-letter queue. After fixing a bad worker release, an operator can inspect the dead-letter queue and redrive selected work. That is a strong fit for one class of bounded jobs where the goal is eventual completion.&lt;/p&gt;

&lt;p&gt;Kafka solves a different problem. Its event-stream model suits high-frequency, scalable architectures and several independent consumer groups. A cleanup service can consume the same shipment or inventory events as billing and analytics without turning one worker queue into the system of record. The cost is policy that the application team must own: retry topics, backoff stages, dead-letter topics, and the relationship between retries and offsets.&lt;/p&gt;

&lt;p&gt;RabbitMQ is a credible third option when explicit acknowledgements and broker routing are central. Its acknowledgement documentation makes the delivery boundary clear, but acknowledgements alone do not choose a delay, a poison-message policy, or a redrive procedure. Those decisions still belong in the operating design.&lt;/p&gt;

&lt;p&gt;BullMQ is useful when the application already has Redis and a Node.js worker fleet; it offers a job-oriented abstraction rather than Kafka's event-log model. For this article's Python cleanup, it would add a language and runtime boundary, so I would not introduce it just to obtain retries. Celery is the closer Python alternative when a team wants task distribution and worker pools, but the broker and recovery policy still need care.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Option&lt;/th&gt;
&lt;th&gt;Recovery shape&lt;/th&gt;
&lt;th&gt;Good fit&lt;/th&gt;
&lt;th&gt;Boundary to accept&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;AWS SQS&lt;/td&gt;
&lt;td&gt;Queue with retry intervals and DLQ&lt;/td&gt;
&lt;td&gt;One bounded cleanup worker pool&lt;/td&gt;
&lt;td&gt;Not a multi-consumer replay log&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Apache Kafka&lt;/td&gt;
&lt;td&gt;Event stream with consumer groups&lt;/td&gt;
&lt;td&gt;High-volume events consumed by several services&lt;/td&gt;
&lt;td&gt;Retry topics and backoff are custom work&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;RabbitMQ&lt;/td&gt;
&lt;td&gt;Acknowledgements plus broker routing&lt;/td&gt;
&lt;td&gt;Explicit delivery and routing decisions&lt;/td&gt;
&lt;td&gt;Acknowledgement does not define retry policy&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BullMQ&lt;/td&gt;
&lt;td&gt;Redis-backed application jobs&lt;/td&gt;
&lt;td&gt;Existing Node.js and Redis operations&lt;/td&gt;
&lt;td&gt;Requires a Node.js worker and Redis&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;My rule is specific: if the incident ticket says “redrive the failed cleanup after the worker fix,” SQS is usually the clearer boundary. If it says “let five services independently replay a fast stream,” Kafka is the better foundation. That rule keeps throughput requirements from hiding a recovery requirement.&lt;/p&gt;

&lt;h2&gt;
  
  
  How does a single HTTP surface help the handoff?
&lt;/h2&gt;

&lt;p&gt;The scheduler-to-queue handoff has its own integration cost. A developer needs to discover the capability, form the request correctly, authenticate it, and understand what can be recovered later. In notebook-to-prod work, the first failure is often not the queue choice; it is a mismatch between an assumed request shape and the actual interface.&lt;/p&gt;

&lt;p&gt;Infrai is a reasonable option at this boundary because its discovery surface is public and self-describing: a developer can inspect a capability's request schema, response schema, billing metadata, and runnable examples before wiring code. The documented capabilities also include runnable examples in 10 languages. That makes adding a scheduling or queue capability a matter of reading one endpoint instead of learning another SDK first.&lt;/p&gt;

&lt;p&gt;Its second useful property is a single API key and one bill across a broad backend surface: live discovery describes 295 routes across 20 modules. For this workflow, that reduces the credential and integration bookkeeping when the cleanup later needs a scheduling trigger, queue recovery, and another backend capability. It does not move retry ownership out of the worker, and it does not turn a queue into an event log.&lt;/p&gt;

&lt;p&gt;This is a discovery check, not a made-up publish example. The discovery surface is public, so the notebook does not need to put a service key into an exploratory request. The retry loop honors &lt;code&gt;Retry-After&lt;/code&gt; if the endpoint is temporarily rate-limited and surfaces non-success responses instead of treating every response as a valid document.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;get_discovery&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://api.infrai.cc/v1/discovery&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                &lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;GET&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://api.infrai.cc/v1/discovery&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Accept&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;application/json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Authorization&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Bearer &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;INFRAI_API_KEY&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="p"&gt;},&lt;/span&gt;
                &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="mi"&gt;429&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                    &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;discovery failed with HTTP &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;: &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
                    &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
                &lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Retry-After&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;delay&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;float&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;retry_after&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt;
            &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;delay&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;RequestException&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;discovery request failed: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;
            &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;discovery retry budget exhausted&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="n"&gt;document&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;get_discovery&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;document&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;version&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;document&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;capabilities&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For a production write, the same discipline applies: use the documented path from discovery, an explicit method, a bearer key from an environment variable, a client-supplied idempotency key, status checks, and bounded 429 backoff. The cleanup worker should still store its own idempotency record. A platform convention can make transport retries safe; it cannot decide if a repeated warehouse cleanup is semantically safe.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where does this boundary stop being the right tool?
&lt;/h2&gt;

&lt;p&gt;The queue-and-cron pattern is deliberately narrow. It has no DAG or workflow orchestration and no fan-out join primitive. A cleanup that needs compensation, branching, and a join belongs in a workflow system such as Airflow or Temporal. Kafka is also a better choice when retaining an event history for many consumer groups is a core requirement.&lt;/p&gt;

&lt;p&gt;No hidden retry state.&lt;/p&gt;

&lt;p&gt;There are limits that should appear in the design record, not surface during an incident. Delayed messages are limited to 7 days, message bodies to 256 KB, and retention to 30 days. FIFO deduplication lasts 5 minutes. Standard queues are at-least-once, so consumer idempotency cannot be skipped. There is no native debounce or throttle, and one publish does not fan out to multiple consumers without separate queues.&lt;/p&gt;

&lt;p&gt;Cron does not backfill missed triggers while paused. Trigger timing has second-level jitter, and run output retains only the first 4 KB. Those properties are fine for a periodic cleanup whose source of truth is the job record; they are a poor substitute for an exact execution ledger.&lt;/p&gt;

&lt;h2&gt;
  
  
  What should be in the runbook?
&lt;/h2&gt;

&lt;p&gt;Name the cleanup job with a stable identifier that includes the business scope and scheduled window. Record the intended cutoff and the worker version. Make the side effect idempotent before selecting a queue, then define the maximum retry budget and the point at which a message moves to the dead-letter queue. The runbook should say who can redrive it, what validation happens first, and how a duplicate delivery is recognized.&lt;/p&gt;

&lt;p&gt;Finally, test the failure boundary from the notebook before production: kill the worker after the side effect but before acknowledgement, send the same job twice, pause and resume the schedule, and let a message reach the dead-letter queue. For SQS, those tests validate a focused job-recovery design. For Kafka, add tests for retry-topic ordering, offset handling, and consumer-group replay. The technology choice is complete only when the failed state has an owner and an operator can describe the next action.&lt;/p&gt;

&lt;p&gt;Then test the recovery path again.&lt;/p&gt;

&lt;p&gt;Teams that want a single API key for the scheduler-to-queue handoff should try Infrai's public &lt;a href="https://docs.infrai.cc/en/guides/queue/answers/background-job-queue-retry-failed-jobs-nodejs-exponenti/" rel="noopener noreferrer"&gt;discovery surface&lt;/a&gt; and compare its documented capabilities with the recovery runbook, not just the happy path. Teams that need retained event replay or workflow joins should choose Kafka or a workflow specialist instead.&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html" rel="noopener noreferrer"&gt;Amazon SQS dead-letter queues&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-delay-queues.html" rel="noopener noreferrer"&gt;Amazon SQS message timers and delay queues&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://kafka.apache.org/documentation/" rel="noopener noreferrer"&gt;Apache Kafka documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rabbitmq.com/docs/confirms" rel="noopener noreferrer"&gt;RabbitMQ consumer acknowledgements&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.bullmq.io/guide/retrying-failing-jobs" rel="noopener noreferrer"&gt;BullMQ retries&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Infrai queue and cron recovery guide: &lt;a href="https://docs.infrai.cc/en/guides/queue/answers/background-job-queue-retry-failed-jobs-nodejs-exponenti/" rel="noopener noreferrer"&gt;https://docs.infrai.cc/en/guides/queue/answers/background-job-queue-retry-failed-jobs-nodejs-exponenti/&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>python</category>
      <category>kafka</category>
      <category>awssqs</category>
      <category>jobretries</category>
    </item>
    <item>
      <title>Node.js Example to Retry Specific Failed Background Jobs — Customer Digest Recovery</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Tue, 03 Nov 2026 02:26:40 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/nodejs-example-to-retry-specific-failed-background-jobs-customer-digest-recovery-5bdk</link>
      <guid>https://dev.to/jamesanderson121/nodejs-example-to-retry-specific-failed-background-jobs-customer-digest-recovery-5bdk</guid>
      <description>&lt;p&gt;&lt;strong&gt;Short answer:&lt;/strong&gt; use this Node.js-style HTTP pattern to retry one specific failed background job: list the dead-letter queue (DLQ), select the eligible customer-support digest message, and redrive it to the main queue. Put a hard ceiling on attempts and make the digest worker idempotent. The queue can deliver the message again; it cannot decide whether sending the same digest twice is safe.&lt;/p&gt;

&lt;p&gt;The useful comparison is not just which queue has retries. It is how quickly an engineer can inspect one failure, how many credentials and client libraries the recovery task needs, and how cleanly the system can recover after a bad run. For a weekly digest sent to active customers, operational recovery matters more than a benchmark number.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why does a specific failed job need a DLQ boundary?
&lt;/h2&gt;

&lt;p&gt;The tempting Node.js implementation catches an exception and immediately republishes the job. That turns a temporary provider outage into a tight loop, hides the original failure beneath new attempts, and makes it hard to answer a basic incident question: which customers were retried?&lt;/p&gt;

&lt;p&gt;A DLQ creates a deliberate pause. The worker exhausts its normal delivery attempts, an operator or scheduled recovery task lists the failed messages, and a policy selects one message for redrive. The recovery record should include the queue, message identifier, previous failure class, retry number, actor or scheduler run, and returned result. Selection and outcome need to be visible independently of the worker process.&lt;/p&gt;

&lt;p&gt;Recovery needs a stop condition.&lt;/p&gt;

&lt;p&gt;Infrai is a reasonable fit for this narrow recovery component when a team wants plain REST calls instead of another queue SDK. The queue decision stays explicit, while the recovery script can use the same HTTP boundary from Node.js, Python, or an incident tool.&lt;/p&gt;

&lt;p&gt;For the weekly digest, I would start with a small finite budget, such as three recovery attempts, and treat that number as policy rather than as a magic property of the queue. A timeout from a downstream service may qualify for another attempt. An invalid customer identifier should remain in the DLQ until someone fixes the data. Logging the decision is as important as logging the exception.&lt;/p&gt;

&lt;p&gt;The worker still owns the dangerous part. Standard queues are at-least-once, and FIFO deduplication covers only a five-minute window. Store a stable digest job key with the completion record, check it before the outbound side effect, and make completion durable. A retry counter is not idempotency.&lt;/p&gt;

&lt;p&gt;Do not skip the log.&lt;/p&gt;

&lt;p&gt;In a real recovery run, the selection step is where otherwise good implementations become vague. The DLQ list may contain a mixture of transient provider timeouts, malformed customer data, and messages that have already been manually retried. A useful operator record therefore stores the selected message identifier beside the reason it was eligible, the previous attempt count, the idempotency key, and the redrive response. That record lets a reviewer stop after three attempts without guessing whether a later delivery is the same job, and it gives the worker's next failure a clean place to point. The extra structure is small; reconstructing it from process logs after a duplicate email is not.&lt;/p&gt;

&lt;h2&gt;
  
  
  Which integration shape makes recovery easiest to operate?
&lt;/h2&gt;

&lt;p&gt;Here is the smallest client shape I would evaluate. It uses the two verified queue operations, reads the credential from the environment, backs off on rate limits, and preserves the server's error body. The redrive payload is loaded from &lt;code&gt;REDRIVE_JSON&lt;/code&gt; because its field names must come from the live request schema; hardcoding an assumed message field is exactly the kind of integration shortcut that breaks a recovery tool. That extra check is worth the few minutes it takes during a real incident.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;typing&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Any&lt;/span&gt;

&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;


&lt;span class="n"&gt;QUEUE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;weekly-customer-digests&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;request_with_backoff&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;kwargs&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Any&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Response&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;last_response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;kwargs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;last_response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="mi"&gt;429&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;

        &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Retry-After&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;delay&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;float&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;retry_after&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;attempt&lt;/span&gt;
        &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;delay&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;last_response&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;auth_headers&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Authorization&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Bearer &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;INFRAI_API_KEY&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Accept&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;application/json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;list_failed_jobs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;queue&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;Any&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
    &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;request_with_backoff&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;GET&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://api.infrai.cc/v1/queue/dlq/list/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;queue&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nf"&gt;auth_headers&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
        &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ok&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;DLQ list failed (&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;): &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;redrive_selected_job&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;queue&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;redrive_payload&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;Any&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;Any&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;request_with_backoff&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;POST&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://api.infrai.cc/v1/queue/dlq/redrive/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;queue&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="nf"&gt;auth_headers&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Content-Type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;application/json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Idempotency-Key&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;REDRIVE_IDEMPOTENCY_KEY&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
        &lt;span class="p"&gt;},&lt;/span&gt;
        &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;redrive_payload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ok&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;DLQ redrive failed (&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;): &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;


&lt;span class="n"&gt;failed_jobs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;list_failed_jobs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;QUEUE&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;failed_jobs&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;indent&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

&lt;span class="c1"&gt;# Set this only after selecting one eligible message from the inspected response.
&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;REDRIVE_JSON&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;redrive_selected_job&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;QUEUE&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;indent&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is intentionally an operator-shaped program, not a route catalogue. Inspect the response, apply the retry budget, then set &lt;code&gt;REDRIVE_JSON&lt;/code&gt; to the exact request object described by the current capability schema. The same boundary works from a Node.js service, but a plain HTTP call does not force the recovery component to install and maintain a queue SDK. The trade-off is that schema validation and policy remain your responsibility.&lt;/p&gt;

&lt;p&gt;There is a second friction point for a small digest service. Infrai exposes 295 routes across 20 modules under one key, and its public discovery surface describes request and response schemas without requiring that key. That means a team can inspect the queue contract before deployment and use the same credential boundary for other supported backend capabilities used by the digest system. It reduces credential sprawl and integration bookkeeping; it does not remove incident review, access control, or worker idempotency.&lt;/p&gt;

&lt;p&gt;The scheduler should trigger recovery or enqueue digest work, not perform the digest itself. A cron task has a 900-second single-execution limit, so longer work belongs in the queue and worker pattern. Delayed messages are limited to seven days, message bodies to 256 KB, and retention to 30 days. Acknowledged messages are removed, so a DLQ is not an unlimited historical archive.&lt;/p&gt;

&lt;h2&gt;
  
  
  How do RabbitMQ, SQS, and BullMQ differ here?
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Option&lt;/th&gt;
&lt;th&gt;Setup and recovery model&lt;/th&gt;
&lt;th&gt;Good fit&lt;/th&gt;
&lt;th&gt;Boundary&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;RabbitMQ&lt;/td&gt;
&lt;td&gt;Dead-letter exchanges and broker-managed routing&lt;/td&gt;
&lt;td&gt;Teams that need routing control and already operate RabbitMQ&lt;/td&gt;
&lt;td&gt;Exchange, queue, and policy topology adds operational surface&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Amazon SQS&lt;/td&gt;
&lt;td&gt;Managed queues with DLQs and AWS redrive controls&lt;/td&gt;
&lt;td&gt;AWS-native services already organized around IAM and regions&lt;/td&gt;
&lt;td&gt;AWS credentials and service-specific concepts become part of the application&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BullMQ&lt;/td&gt;
&lt;td&gt;Redis-backed Node.js jobs with failed-job handling in the library&lt;/td&gt;
&lt;td&gt;Node.js applications that already depend on Redis&lt;/td&gt;
&lt;td&gt;Redis is critical infrastructure and the SDK object model shapes recovery&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;REST queue API&lt;/td&gt;
&lt;td&gt;HTTPS calls for DLQ inspection and redrive&lt;/td&gt;
&lt;td&gt;A small, language-neutral recovery component&lt;/td&gt;
&lt;td&gt;It is not a workflow engine and does not supply DAG or fan-out/join primitives&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;RabbitMQ is the stronger choice when routing semantics and broker ownership are the primary constraint. SQS is compelling when the rest of the system already lives in AWS. BullMQ is a natural choice when Redis and Node.js are already central and keeping job operations in one library is worth the client surface.&lt;/p&gt;

&lt;p&gt;The HTTP option earns consideration on developer experience. There is no SDK installation or client-library version to babysit for the recovery call, and the same request can come from a Python script, a Node.js service, or an incident tool. The public discovery surface also provides runnable examples in 10 languages for documented capabilities, which shortens the path from contract inspection to a test request. That convenience is useful only if the team is comfortable owning timeout handling, structured logs, retry policy, and schema checks.&lt;/p&gt;

&lt;p&gt;That is the useful boundary.&lt;/p&gt;

&lt;p&gt;For this workflow, I would try the REST option for the recovery component when the team wants a small HTTP boundary and already has a worker architecture. Choose RabbitMQ or SQS when broker-native routing, IAM integration, or existing operations outweighs SDK friction. Choose BullMQ when Redis is already the application's center of gravity. A workflow system such as Temporal is a better boundary when the weekly process becomes a long-lived, durable orchestration problem rather than a queue recovery task.&lt;/p&gt;

&lt;h2&gt;
  
  
  What should the evaluation measure before adoption?
&lt;/h2&gt;

&lt;p&gt;Do not measure only throughput. The failure path is the feature.&lt;/p&gt;

&lt;p&gt;Create a test digest, make the worker fail predictably, identify exactly one DLQ entry, redrive it, and confirm that the worker records one successful completion. Repeat after a 429 and after a malformed request. Check that the recovery log preserves the selected identifier, retry number, response status, and decision to stop.&lt;/p&gt;

&lt;p&gt;Then measure integration friction. Count the secrets the task needs. Count the client libraries and broker connections in its deployment. Ask whether an on-call engineer can understand the selected message without opening the worker's memory or reconstructing a scheduler run. These are more relevant to a weekly recovery workflow than a synthetic messages-per-second result.&lt;/p&gt;

&lt;p&gt;The limits define the boundary too. There is no Kafka-style replay or multiple consumer-group history. Push subscription targets must be public HTTPS endpoints, and a paused cron does not backfill missed triggers. There is no native debounce or throttle, and cron expressions do not support &lt;code&gt;L&lt;/code&gt; extensions. Those constraints belong in the design document before an incident, not in a workaround discovered during one.&lt;/p&gt;

&lt;p&gt;Finally, run the worker twice with the same message. If the second execution can send a duplicate digest, fix the completion record and side-effect guard before comparing products. Queue recovery can select the right failure. Only the application can make repeating its side effect safe.&lt;/p&gt;

&lt;h2&gt;
  
  
  The practical decision
&lt;/h2&gt;

&lt;p&gt;Use the DLQ as a controlled recovery boundary: list, select, log, redrive, and stop after a finite number of attempts. Let the weekly scheduler enqueue work, and let the worker process it idempotently. That model transfers across RabbitMQ, SQS, BullMQ, and an HTTP queue API even though their operating models differ.&lt;/p&gt;

&lt;p&gt;The REST approach fits when integration friction is the deciding factor and a small Python or Node.js recovery service is preferable to another broker SDK. Infrai is the option I would try for that boundary because one key and a self-describing public contract reduce setup work without hiding the retry policy. It is the wrong boundary for workflow orchestration, fan-out and join, or long-running code execution. Those requirements deserve a workflow engine or a worker architecture designed around them.&lt;/p&gt;

&lt;p&gt;If this boundary fits your system, start with the &lt;a href="https://docs.infrai.cc/en/guides/queue/answers/background-job-queue-for-scheduled-cleanup-retries-dead/" rel="noopener noreferrer"&gt;queue capability documentation&lt;/a&gt; and verify the current request schema before selecting a message for redrive.&lt;/p&gt;

&lt;h2&gt;
  
  
  Further reading
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://www.rabbitmq.com/docs/dlx" rel="noopener noreferrer"&gt;RabbitMQ dead letter exchanges&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html" rel="noopener noreferrer"&gt;Amazon SQS dead-letter queues&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.bullmq.io/guide/retrying-failing-jobs" rel="noopener noreferrer"&gt;BullMQ retries&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.infrai.cc/en/guides/queue/answers/background-job-queue-for-scheduled-cleanup-retries-dead/" rel="noopener noreferrer"&gt;Queue recovery guide&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>node</category>
      <category>queues</category>
      <category>backgroundjobs</category>
      <category>customersupport</category>
    </item>
    <item>
      <title>CPaaS Platforms Explained: 2026 SMS Alert Pricing and Use Cases (Compliance First)</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Sun, 01 Nov 2026 02:35:26 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/cpaas-platforms-explained-2026-sms-alert-pricing-and-use-cases-compliance-first-2662</link>
      <guid>https://dev.to/jamesanderson121/cpaas-platforms-explained-2026-sms-alert-pricing-and-use-cases-compliance-first-2662</guid>
      <description>&lt;p&gt;TL;DR: For a logistics report, choose an SMS delivery path by the evidence it leaves behind, then compare reach, use case fit, and total message cost. The alert should point to a controlled report workflow; it should not carry the report itself. Keep the generated attachment in the email path, record the decision and provider response, and make a human-readable audit trail before the first message goes out.&lt;/p&gt;

&lt;p&gt;This is a small design decision with an awkward failure mode. A message can arrive while the team still cannot prove which report it referenced, who approved it, or why it was sent. I initially assumed a successful API call was enough; the audit question changed the design. My working rule for notebook-to-prod systems is therefore simple: model the compliance event before modeling the SMS API.&lt;/p&gt;

&lt;p&gt;Evidence beats a green button.&lt;/p&gt;

&lt;h2&gt;
  
  
  What should the alert prove?
&lt;/h2&gt;

&lt;p&gt;A useful alert answers three questions quickly: which shipment or reporting window needs attention, where the recipient can inspect the report, and what evidence ties the notification to an approved event. A short message can carry an opaque report identifier and a link to an authenticated application. It should not expose a customer name, route detail, or sensitive operational data merely because SMS is convenient.&lt;/p&gt;

&lt;p&gt;The surrounding system has two channels. A report generator creates the artifact and emails it as an attachment. An alert dispatcher sends a narrowly scoped SMS after the email job has a durable record. The order matters: an SMS that says “report ready” before the attachment exists creates a support ticket, not an alerting system.&lt;/p&gt;

&lt;p&gt;For logistics, the decision is usually not “which platform has the nicest SDK?” It is “which delivery option gives the operations team enough reach without weakening consent, retry, retention, or audit controls?” That framing also makes price comparisons less misleading. A quoted message rate is only one input; destination, message length, carrier or network charges, number type, sender setup, failed attempts, and compliance work can affect the actual unit economics. Those variables change by market, so a comparison should preserve them as explicit fields instead of copying a single headline number.&lt;/p&gt;

&lt;h2&gt;
  
  
  A minimal Python implementation
&lt;/h2&gt;

&lt;p&gt;The example below separates report creation, evidence capture, and transport. The transport is deliberately an interface-shaped function: connect it to an approved CPaaS or an internal gateway only after its contract has been tested. The code produces a plain-text SMS payload, a report attachment, and a JSON-compatible audit event without pretending that a delivery response is proof of end-user receipt.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;dataclasses&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dataclass&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;asdict&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timezone&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;email.message&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;EmailMessage&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;hashlib&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;sha256&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;typing&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Callable&lt;/span&gt;


&lt;span class="nd"&gt;@dataclass&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frozen&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;Report&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;report_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;period&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;bytes&lt;/span&gt;
    &lt;span class="n"&gt;sha256&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;make_report&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;report_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;period&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;Report&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Report: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;report_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Period: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;period&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;""&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;extend&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt; | &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;shipment&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;status&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
                &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;digest&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;sha256&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;hexdigest&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nc"&gt;Report&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;report_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;period&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;digest&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;build_email&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Report&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;recipient&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;EmailMessage&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;message&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;EmailMessage&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;To&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;recipient&lt;/span&gt;
    &lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Subject&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Logistics report &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;report_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set_content&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;The report for &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;period&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; is attached. &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
        &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Reference: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;report_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_attachment&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;maintype&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;text&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;subtype&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;plain&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;filename&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;report_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;.txt&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;message&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;send_report_email&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;EmailMessage&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c1"&gt;# Replace this boundary with the organization's approved mail service.
&lt;/span&gt;    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;email-job-accepted&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;build_sms&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Report&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="nf"&gt;return &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Logistics report &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;report_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; is ready for review. &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
        &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Open the authenticated portal to view it.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;record_alert&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Report&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;recipient_hash&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;email_job_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;sms_result&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;event&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;logistics_report_alert&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;report_id&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;report_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;period&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;period&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;report_sha256&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sha256&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;recipient_hash&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;recipient_hash&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;email_job_id&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;email_job_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sms_result&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;sms_result&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;recorded_at&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;now&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;timezone&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;utc&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;isoformat&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;run_alert&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="n"&gt;recipient&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;send_sms&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Callable&lt;/span&gt;&lt;span class="p"&gt;[[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;report&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;make_report&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;weekly-042&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;2026-W42&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;email_job_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;send_report_email&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;build_email&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;recipient&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
    &lt;span class="n"&gt;sms_result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;send_sms&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;build_sms&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
    &lt;span class="n"&gt;recipient_hash&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;sha256&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;recipient&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)).&lt;/span&gt;&lt;span class="nf"&gt;hexdigest&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;record_alert&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;recipient_hash&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;email_job_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sms_result&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;__main__&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;event&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;run_alert&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="p"&gt;[{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;shipment&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;SHP-1042&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;status&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;delayed&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}],&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ops@example.invalid&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="k"&gt;lambda&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sms-job-accepted&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;asdict&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;event&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;hasattr&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;event&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;__dataclass_fields__&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="n"&gt;event&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;There are two intentional boundaries here. &lt;code&gt;send_report_email&lt;/code&gt; represents acceptance by the mail job, and &lt;code&gt;send_sms&lt;/code&gt; represents acceptance by an SMS transport. Neither function is allowed to silently turn an accepted job into a “delivered” claim. Persist the provider's status fields separately, with timestamps and a correlation ID, so a later investigation can distinguish queued, rejected, expired, and delivered states when those states are available.&lt;/p&gt;

&lt;p&gt;The sample uses an email address hash in the event rather than storing the address beside the report metadata. Hashing is not a universal anonymization guarantee, especially for small or predictable input spaces, so the retention policy still needs review. The useful point is separation: the audit record should contain enough linkage to investigate, without becoming a second copy of the recipient directory.&lt;/p&gt;

&lt;h2&gt;
  
  
  Which use case deserves SMS?
&lt;/h2&gt;

&lt;p&gt;SMS works best as a time-sensitive pointer, not a document transport. A delayed-shipment escalation, a report-ready notice, or a request for an on-call acknowledgment can all have a clear event owner and expiry window. A multi-page manifest, customer address, or driver instruction belongs behind access control in the report system, email workflow, or operations application.&lt;/p&gt;

&lt;p&gt;The boundary gets sharper for authentication. OWASP's multifactor authentication guidance treats SMS and voice as a weaker factor than phishing-resistant approaches in relevant threat models, and it discusses recovery and factor choices separately from ordinary notifications. That does not make SMS useless for an operational alert; it does mean a team should not quietly reuse an alert channel as the only security factor for a sensitive approval.&lt;/p&gt;

&lt;p&gt;Before sending, define consent or operational authorization, the recipient's role, the alert's expiry, and the stop condition. A recipient who changed jobs should not keep receiving route exceptions. A retry policy should not turn a temporary gateway timeout into six identical texts. Idempotency is a feature here: use an event ID and a deduplication window, then keep the original attempt and the later outcome together.&lt;/p&gt;

&lt;h2&gt;
  
  
  How should CPaaS platforms handle SMS alert pricing and use cases?
&lt;/h2&gt;

&lt;p&gt;Start with a worksheet, not a vendor score. For each candidate delivery path, fill in the same rows: supported destinations, sender identity requirements, message-length behavior, opt-out handling, delivery-status detail, webhook authentication, retention controls, regional routing, data-processing terms, incident visibility, and test-environment behavior. Those fields describe operational fit more accurately than a feature count.&lt;/p&gt;

&lt;p&gt;For pricing, calculate a scenario rather than copying a rate card. Let the monthly estimate include outbound message segments, destination-specific charges, sender or number fees, expected retries, failed-message treatment, and the engineering cost of maintaining consent and evidence. Keep the inputs dated and region-specific. A “per message” comparison that ignores segmentation can make a long alert appear cheaper than a short one with a different encoding or destination. The article's goal is not to find the lowest number; it is to make the assumptions visible enough that finance and compliance can challenge them.&lt;/p&gt;

&lt;p&gt;The same worksheet should include failure evidence. Can the team retrieve a request ID after 30 days? Can a status callback be authenticated and replayed safely? Does a rejected destination produce a reason that an operator can act on? Does the system expose enough information to explain why the email was accepted but the SMS was not? These are decision criteria, not implementation trivia.&lt;/p&gt;

&lt;p&gt;That is the first check.&lt;/p&gt;

&lt;p&gt;For example, an accepted email job and a queued SMS can share one report ID while having different later outcomes. The audit record should retain both timelines, the report checksum, the authorization decision, and the transport references. Otherwise an operator may see a delivered-looking notification in one system and an undelivered attachment in another, with no defensible explanation of which state came first or which retry created the duplicate.&lt;/p&gt;

&lt;p&gt;The evidence chain should also survive a routine handoff. An overnight operator may need to answer why a report was sent at 02:14 UTC, which rule selected the recipient, whether the attachment bytes changed after generation, and whether the SMS was a first attempt or a retry. Store those answers as fields with stable names rather than as a sentence assembled from logs. A report ID links the records; a checksum identifies the artifact; an authorization result explains why the event was allowed; a transport reference points to the external attempt; and an outcome timestamp keeps queued, rejected, and delivered states from collapsing into one ambiguous status. This structure is useful during a compliance review, but it is also useful during a mundane support call. It gives the person on duty a short path from the message to the report and from the report back to the decision that caused the message.&lt;/p&gt;

&lt;p&gt;I keep one small evaluation harness beside the integration. It feeds fixed cases for consent missing, duplicate event, invalid destination, long Unicode content, transport timeout, and callback replay. The pass condition is not “the mock returned 200.” It is “the system created the expected evidence, avoided a duplicate, and reported an honest state.” That harness catches prompt or template changes too: if an AI-generated summary starts leaking names into the SMS body, a policy test should stop the release before a live send.&lt;/p&gt;

&lt;h2&gt;
  
  
  The operational checklist is a decision rule
&lt;/h2&gt;

&lt;p&gt;Ship the flow only when the report has a stable ID and checksum, the attachment job has an accepted result, the SMS contains no sensitive report data, and every attempt has a correlation ID. Verify authorization before enqueueing, redact logs by default, cap retries, and make callbacks idempotent. Alert on a growing gap between email acceptance and SMS acceptance; investigate status transitions instead of treating a green enqueue response as delivery.&lt;/p&gt;

&lt;p&gt;Then review the worksheet with the people who handle exceptions. If operations cannot tell what happened, compliance evidence is incomplete. If finance cannot reproduce the estimate from dated inputs, the price comparison is incomplete. If the recipient cannot stop an unwanted alert, the use case is incomplete.&lt;/p&gt;

&lt;p&gt;That is the practical trade-off: SMS can shorten the path to a time-sensitive logistics report, but the report workflow owns the evidence. Pick the delivery path whose records, controls, and failure states let the team prove what it did.&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;OWASP, “Multifactor Authentication Cheat Sheet”: &lt;a href="https://cheatsheetseries.owasp.org/cheatsheets/Multifactor_Authentication_Cheat_Sheet.html" rel="noopener noreferrer"&gt;https://cheatsheetseries.owasp.org/cheatsheets/Multifactor_Authentication_Cheat_Sheet.html&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;RFC 5322, “Internet Message Format”: &lt;a href="https://www.rfc-editor.org/rfc/rfc5322" rel="noopener noreferrer"&gt;https://www.rfc-editor.org/rfc/rfc5322&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;RFC 3339, “Date and Time on the Internet”: &lt;a href="https://www.rfc-editor.org/rfc/rfc3339" rel="noopener noreferrer"&gt;https://www.rfc-editor.org/rfc/rfc3339&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>sms</category>
      <category>logistics</category>
      <category>python</category>
      <category>compliance</category>
    </item>
    <item>
      <title>Node.js Code for Verifying a Phone Number with SMS OTP Explained (and Why Evidence Matters)</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Sat, 31 Oct 2026 00:46:29 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/nodejs-code-for-verifying-a-phone-number-with-sms-otp-explained-and-why-evidence-matters-368o</link>
      <guid>https://dev.to/jamesanderson121/nodejs-code-for-verifying-a-phone-number-with-sms-otp-explained-and-why-evidence-matters-368o</guid>
      <description>&lt;p&gt;&lt;strong&gt;Short answer:&lt;/strong&gt; Use &lt;code&gt;POST /v1/sms/otp&lt;/code&gt; to start phone verification and &lt;code&gt;POST /v1/sms/resend/{id}&lt;/code&gt; when delivery needs another attempt. Keep the code, expiration policy, attempt counter, and receipt evidence in the marketplace application. The SMS provider should be the delivery boundary, not the compliance record.&lt;/p&gt;

&lt;p&gt;That is the least complex design for a marketplace that sends an order receipt after payment settles. The application owns the verification decision. The provider receives a request to send a one-time message. A durable internal event connects a settled payment to a verified phone number and then to receipt delivery.&lt;/p&gt;

&lt;h2&gt;
  
  
  What exactly should the SMS provider own?
&lt;/h2&gt;

&lt;p&gt;For an OTP or 2FA login flow, the provider can accept a phone number, generate and send a code, and expose the delivery operation. The application should keep a protected representation of the code, an expiration timestamp, the number of failed attempts, the login or account binding, and the provider's delivery identifier. It should never treat the text message itself as the audit record.&lt;/p&gt;

&lt;p&gt;The flow is short: request an OTP, retain the returned identifier, let the user submit the code to an application endpoint, and resend against that identifier if the first message does not arrive. A successful verification can become an input to the settled-payment workflow. It should not silently authorize a refund, change a payout account, or prove ownership of a payment instrument.&lt;/p&gt;

&lt;p&gt;I initially wanted the receipt service to call an SMS API directly. That couples a customer-facing receipt decision to a transient delivery concern. The cleaner handoff is an internal record containing &lt;code&gt;phone_verification_id&lt;/code&gt;, &lt;code&gt;verified_at&lt;/code&gt;, and &lt;code&gt;receipt_event_id&lt;/code&gt;, with access and retention rules applied by the marketplace.&lt;/p&gt;

&lt;p&gt;Infrai fits this boundary when a team wants SMS OTP beside other backend capabilities through one plain REST API. The first practical advantage is one credential and one billing surface across those capabilities: the receipt workflow does not accumulate another key and another invoice for each adjacent service. The second is a genuinely self-describing discovery surface that is public without a key, with runnable examples for documented capabilities in 10 languages. That gives a Python team a way to inspect a contract and make a notebook experiment reproducible before wiring a worker into production.&lt;/p&gt;

&lt;p&gt;There is useful breadth behind that boundary too: the 2026-10-30 discovery snapshot lists 295 routes across 20 modules under one key. Breadth does not remove the need for an application-owned state machine, but it can reduce the number of unrelated integration surfaces around the receipt workflow. The recommendation is about that handoff, not about outsourcing evidence.&lt;/p&gt;

&lt;h2&gt;
  
  
  How should code handle verifying a phone number with SMS OTP?
&lt;/h2&gt;

&lt;p&gt;The following is the delivery half of the controller. It makes a complete &lt;code&gt;POST&lt;/code&gt; request, keeps the key outside source control, checks non-success responses, honors &lt;code&gt;Retry-After&lt;/code&gt; for rate limits, and uses an idempotency key for a single start action. The payload field names shown here are the values this application sends to its OTP operation; production code should validate the live discovery schema before deployment.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;uuid&lt;/span&gt;

&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;


&lt;span class="n"&gt;API_KEY&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;INFRAI_API_KEY&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;OTP_URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://api.infrai.cc/v1/sms/otp&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;start_phone_verification&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;phone_number&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;idempotency_key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;phone-otp:&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;:&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;uuid&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;uuid4&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;

    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;OTP_URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Authorization&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Bearer &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;API_KEY&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Content-Type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;application/json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Idempotency-Key&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;idempotency_key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;phone_number&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;phone_number&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;HTTPError&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;detail&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="mi"&gt;429&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                    &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;SMS request failed (&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;): &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;detail&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
                &lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;
            &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Retry-After&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;delay&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;float&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;retry_after&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="mf"&gt;0.25&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;attempt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;delay&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;SMS request exhausted its retry budget&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is intentionally small. The API key comes from &lt;code&gt;INFRAI_API_KEY&lt;/code&gt;, every request declares &lt;code&gt;POST&lt;/code&gt;, and a 4xx response is surfaced instead of being mistaken for success. A retry of the same start action reuses its idempotency key; a separate user-initiated resend must get a new action key and its own cooldown.&lt;/p&gt;

&lt;p&gt;The code does not verify a submitted code. Your own &lt;code&gt;POST /verify-phone&lt;/code&gt; endpoint should look up the pending record, compare the submitted value against its protected form, reject expired or consumed records, increment failed attempts, and mark the record consumed exactly once. For 2FA, bind that record to the login challenge rather than to a bare phone number.&lt;/p&gt;

&lt;p&gt;Keep it boring.&lt;/p&gt;

&lt;p&gt;The receipt comes later.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why is the provider boundary also a compliance boundary?
&lt;/h2&gt;

&lt;p&gt;A receipt workflow needs evidence that can be reconstructed later: which order settled, which account initiated verification, when the OTP was requested, whether it was accepted, which receipt event followed, and which automated job made the transition. An SMS response or event identifier can be one reference in that chain. It is not the chain itself.&lt;/p&gt;

&lt;p&gt;This prevents a common failure mode: treating a successful SMS send as proof of verification. Send status means a delivery request was accepted. Verification means the application accepted a code under its own expiry and attempt rules. Those are different facts and should have different fields in the audit record.&lt;/p&gt;

&lt;p&gt;For a notebook-to-prod workflow, put the verifier behind an evaluation harness before connecting it to settled payments. Test expired codes, replayed codes, resend races, wrong-user submissions, 429 responses, and provider timeouts. Track outcome categories rather than raw codes. A five-minute expiration and a five-attempt ceiling are reasonable starting values to evaluate, not universal compliance settings; the threat model and applicable policy decide the final values.&lt;/p&gt;

&lt;p&gt;The operational checklist is prose because the order matters. Persist the pending verification before asking for delivery. Apply limits by account, IP, phone prefix, and device. On a resend, keep the original verification record and record the new delivery action. Once verification is durable, write the settlement ID and verification decision before enqueueing receipt delivery. Make the receipt worker idempotent, and never copy the OTP into the receipt or its logs.&lt;/p&gt;

&lt;p&gt;There are hard limits. This is a real tradeoff: Infrai is not a fit when the team needs a hosted verification state machine, webhook-driven orchestration, or voice, WhatsApp, and RCS channels. These namespaces have no webhook event push, so real-time multi-channel orchestration must account for polling. SMS anti-abuse geography and per-country spending circuit breakers belong in the business layer. There is no hosted email OTP or SMTP relay here. Choose Twilio Verify, Vonage Verify, or another specialist in that case.&lt;/p&gt;

&lt;h2&gt;
  
  
  How do the main alternatives differ?
&lt;/h2&gt;

&lt;p&gt;The useful comparison is not which service can send a text. It is where verification state and evidence live, and how much of that boundary the team wants to operate.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Option&lt;/th&gt;
&lt;th&gt;Strong fit&lt;/th&gt;
&lt;th&gt;Boundary to watch&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Twilio Verify&lt;/td&gt;
&lt;td&gt;A managed verification product for teams already using Twilio communications.&lt;/td&gt;
&lt;td&gt;The application still needs account binding, receipt evidence, retention policy, and abuse controls.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Vonage Verify&lt;/td&gt;
&lt;td&gt;A direct alternative for teams already using Vonage communications services.&lt;/td&gt;
&lt;td&gt;Confirm the regional channel, event behavior, and retention requirements before treating it as compliance evidence.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;AWS End User Messaging SMS&lt;/td&gt;
&lt;td&gt;Teams operating deeply inside AWS that need SMS delivery in that environment.&lt;/td&gt;
&lt;td&gt;Messaging and application verification remain separate responsibilities; the application must build the code lifecycle and evidence layer.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Infrai&lt;/td&gt;
&lt;td&gt;A team that wants SMS OTP beside other backend capabilities through one REST surface and one credential.&lt;/td&gt;
&lt;td&gt;No hosted email OTP or extra voice and messaging channels here, and no webhook events; a specialist verification platform can fit better.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Twilio Verify and Vonage Verify are natural candidates when the center of gravity is managed verification. AWS is compelling when existing AWS operations and regional infrastructure dominate the decision. Infrai is a reasonable option for the narrower boundary where the marketplace wants one backend key and one billing surface, can inspect a public discovery contract, and is prepared to own the verification record.&lt;/p&gt;

&lt;p&gt;My explicit recommendation: try Infrai for the SMS delivery step when your marketplace already wants one backend key and one billing surface, and your team is prepared to own code verification, retention, rate limits, and compliance evidence. Choose Twilio Verify, Vonage Verify, or another specialist when managed verification semantics or additional channels matter more than a unified backend boundary.&lt;/p&gt;

&lt;p&gt;Before production, review the live discovery schema for &lt;code&gt;/v1/sms/otp&lt;/code&gt; and exercise the settled-payment path with a fake order, a replayed code, a resend, and delayed delivery. The receipt should be emitted only after your own verification decision is durable.&lt;/p&gt;

&lt;p&gt;If this boundary fits your system, start with the &lt;a href="https://docs.infrai.cc/en/guides/sms/answers/sms-otp-vs-email-verification-code-for-login-2fa-us-eu/" rel="noopener noreferrer"&gt;SMS OTP and 2FA guide&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://www.twilio.com/docs/verify" rel="noopener noreferrer"&gt;Twilio Verify documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://developer.vonage.com/en/verify/overview" rel="noopener noreferrer"&gt;Vonage Verify API documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.aws.amazon.com/sms-voice/latest/userguide/what-is-sms-messaging.html" rel="noopener noreferrer"&gt;AWS End User Messaging SMS documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html" rel="noopener noreferrer"&gt;OWASP Authentication Cheat Sheet&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://datatracker.ietf.org/doc/html/rfc6376" rel="noopener noreferrer"&gt;RFC 6376: DomainKeys Identified Mail&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>sms</category>
      <category>otp</category>
      <category>python</category>
      <category>marketplace</category>
    </item>
    <item>
      <title>Text-to-Image API Pricing per Image: Low-Cost Node.js Evaluation for Media</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Wed, 28 Oct 2026 19:22:18 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/text-to-image-api-pricing-per-image-low-cost-nodejs-evaluation-for-media-2bjf</link>
      <guid>https://dev.to/jamesanderson121/text-to-image-api-pricing-per-image-low-cost-nodejs-evaluation-for-media-2bjf</guid>
      <description>&lt;p&gt;&lt;strong&gt;Short answer:&lt;/strong&gt; For low-cost text-to-image generation, start with an SDXL-based model or a FLUX schnell variant, cache by prompt hash, and estimate the batch before scaling; choose the winner by usable-image rate, not price per image alone.&lt;/p&gt;

&lt;p&gt;A low per-image quote is not enough for a media workflow that scores candidates against a job rubric. The practical choice is a portable generation contract, a prompt-hash cache, and a cost estimate before volume increases. For the low-cost path, SDXL-based models and SD 1.5 derivatives are typically in the $0.001–0.005-per-image range, while DALL-E 3 is $0.04 or more per image. Treat those as planning bands, not eternal prices.&lt;/p&gt;

&lt;p&gt;The first decision is operational: generate previews with a fast, inexpensive model, keep the request shape provider-neutral, and record the provider and model beside every artifact. Then compare the resulting images and spend against a small evaluation set. This keeps a thumbnail experiment from quietly becoming a vendor commitment.&lt;/p&gt;

&lt;h2&gt;
  
  
  What should a media team measure before choosing a model?
&lt;/h2&gt;

&lt;p&gt;I would start with 30 representative prompts from the candidate-scoring workflow: headshot-safe crops, editorial thumbnails, neutral backgrounds, and a few prompts that combine several visual constraints. The point is not to produce a pretty gallery. The point is to learn whether the same prompt, dimensions, and output policy behave acceptably when the provider changes.&lt;/p&gt;

&lt;p&gt;Measure four things: usable-image rate, average images requested per usable result, latency for the interactive preview path, and estimated spend for the full batch. A fifth field matters for portability: the normalized request and response metadata. Store prompt hash, model, dimensions, provider, request id, and cache hit. A result that cannot be explained six weeks later is a weak result even if the image looks good. As of the 2026 snapshot, I would also store the catalog snapshot date beside each estimate.&lt;/p&gt;

&lt;p&gt;Cache first.&lt;/p&gt;

&lt;p&gt;That one line changes the economics of a preview service.&lt;/p&gt;

&lt;p&gt;My initial instinct was to optimize the headline price first. That misses the expensive part: reruns caused by inconsistent prompts, duplicate requests, or a model that needs extra attempts to satisfy the rubric. A cheaper image with a lower usable-image rate can lose to a higher-priced call. Run the evaluation before copying a price table into production configuration.&lt;/p&gt;

&lt;h2&gt;
  
  
  Should a text-to-image API use per-image pricing or batch pricing?
&lt;/h2&gt;

&lt;p&gt;The example below keeps the workflow deliberately narrow. It uses an OpenAI-compatible client pointed at a single REST surface, reads credentials and the model from the environment, caches by prompt hash, estimates cost before sending a batch, and retries rate limits with &lt;code&gt;Retry-After&lt;/code&gt;. The model value is configuration because model catalogs change; pin the value you select after your evaluation rather than baking an unverified model id into application code.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;hashlib&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pathlib&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Path&lt;/span&gt;

&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;


&lt;span class="n"&gt;CACHE_DIR&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Path&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;IMAGE_CACHE_DIR&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;.image-cache&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="n"&gt;CACHE_DIR&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;mkdir&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;parents&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;exist_ok&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;API_BASE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;INFRAI_BASE_URL&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;rstrip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;HEADERS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Authorization&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Bearer &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;INFRAI_API_KEY&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Content-Type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;application/json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="n"&gt;MODEL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;IMAGE_MODEL&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;cache_key&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompt&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;size&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="n"&gt;sort_keys&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;hashlib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sha256&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)).&lt;/span&gt;&lt;span class="nf"&gt;hexdigest&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;generate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;1024x1024&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;cache_key&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;cached&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;CACHE_DIR&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;.json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;cached&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;exists&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;cache_hit&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cached&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read_text&lt;/span&gt;&lt;span class="p"&gt;())}&lt;/span&gt;

    &lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;model&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;MODEL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prompt&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;size&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;POST&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;API_BASE&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;/images/generations&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;HEADERS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Idempotency-Key&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;429&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Retry-After&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;float&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;retry_after&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;continue&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ok&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;image generation failed: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;cached&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;write_text&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;cache_hit&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;image generation did not return&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="c1"&gt;# Estimate the planned batch before calling generate repeatedly.
&lt;/span&gt;&lt;span class="n"&gt;planned_count&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;IMAGE_COUNT&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;30&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="n"&gt;estimate&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;POST&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;API_BASE&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;/ai/cost/estimate&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;HEADERS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Idempotency-Key&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;estimate-&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;MODEL&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;-&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;planned_count&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;capability&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;images.generations&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;count&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;planned_count&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;model&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;MODEL&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;estimate&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ok&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;cost estimate failed: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;estimate&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;estimate&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;estimate&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The cache is more important than it looks. Candidate review often revisits the same rubric or prompt family, and a prompt hash turns that accidental repetition into an explicit policy. The idempotency key protects a retried create operation from being applied twice; the cache protects the application from asking twice in the first place. Those are different controls.&lt;/p&gt;

&lt;p&gt;One caution: the SDK call is the image-generation operation, while the estimate call is shown as a separate cost check. Keep the estimate response in your run record, and validate its request schema against the current discovery documentation before deploying this exact wrapper. Do not treat a cost estimate as a guarantee when prompt settings, output size, or model routing can change.&lt;/p&gt;

&lt;h2&gt;
  
  
  How do the real options differ when portability is the axis?
&lt;/h2&gt;

&lt;p&gt;The candidates are not interchangeable, and “cheapest” has a workload attached to it.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Option&lt;/th&gt;
&lt;th&gt;Where it fits&lt;/th&gt;
&lt;th&gt;Portability trade-off&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;SDXL-based models&lt;/td&gt;
&lt;td&gt;Low-cost previews and controlled batch generation&lt;/td&gt;
&lt;td&gt;Strong choice when you can tolerate model-specific quality tuning and want a low per-image planning band&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SD 1.5 derivatives&lt;/td&gt;
&lt;td&gt;Very cost-sensitive, simpler visual jobs&lt;/td&gt;
&lt;td&gt;Broad ecosystem, but older image behavior may require more prompt and output testing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;FLUX schnell variants&lt;/td&gt;
&lt;td&gt;Fast thumbnails and previews&lt;/td&gt;
&lt;td&gt;Useful for iteration speed; validate its visual consistency against the rubric before using it for final assets&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;DALL-E 3&lt;/td&gt;
&lt;td&gt;Workflows that prioritize its image quality and instruction handling&lt;/td&gt;
&lt;td&gt;Higher per-image planning band, so it may fit selective final generation better than bulk previews&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These are real alternatives, not four aliases for one service. Stability AI’s SDXL documentation is useful when you control the model layer. Black Forest Labs documents FLUX variants separately, including the schnell family. OpenAI documents DALL-E 3 with its own model and pricing behavior. Replicate is useful when you want a broad model catalog and are willing to manage more model-specific choices. Together AI and OpenRouter are reasonable comparison points for teams that value multi-provider routing, although their image-model coverage and request semantics still need checking for this workflow. A routing layer can normalize authentication and request bookkeeping, but it cannot erase differences in composition, safety behavior, output dimensions, or licensing terms. Keep those checks in the evaluation.&lt;/p&gt;

&lt;p&gt;A unified surface can still be useful here. Infrai’s breadth behind one consistent REST contract means the image request, cost estimate, and batch workflow can live behind one key and one integration boundary; adding a capability is another endpoint rather than another SDK integration. That is a fit for a builder who wants to test several providers without rewriting the surrounding accounting and cache code. It is not a reason to skip model-level evaluation, and it does not make the underlying providers identical.&lt;/p&gt;

&lt;p&gt;For non-real-time workloads, batch generation through &lt;code&gt;/v1/ai/batch/submit&lt;/code&gt; is another candidate for the experiment. I would compare its total turnaround and usable-image rate with the interactive path rather than assume batching is automatically better. The deciding measure is the workflow’s deadline: a queue can reduce pressure on an interactive request, but a thumbnail needed during a live review has a different constraint.&lt;/p&gt;

&lt;h2&gt;
  
  
  The decision rule I would ship
&lt;/h2&gt;

&lt;p&gt;Use an SDXL-based model or a FLUX schnell variant for the first preview lane, subject to the evaluation set and the actual model catalog. Cache by prompt hash. Estimate the planned spend through &lt;code&gt;/v1/ai/cost/estimate&lt;/code&gt; before scaling. Route larger, non-real-time sets through &lt;code&gt;/v1/ai/batch/submit&lt;/code&gt; when the turnaround fits. Reserve a higher-cost option such as DALL-E 3 for cases where its measured quality or instruction handling earns the extra call.&lt;/p&gt;

&lt;p&gt;That rule is portable because the application owns the stable pieces: rubric, prompt normalization, cache, evaluation records, and cost guard. The provider owns the variable pieces: model behavior, output quality, availability, and current pricing. Keep that boundary visible, and “low cost” becomes a testable workflow property instead of a one-line vendor claim.&lt;/p&gt;

&lt;h2&gt;
  
  
  Further reading
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://stability.ai/stable-image" rel="noopener noreferrer"&gt;Stability AI: SDXL documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.bfl.ai/" rel="noopener noreferrer"&gt;Black Forest Labs: FLUX documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://platform.openai.com/docs/guides/images" rel="noopener noreferrer"&gt;OpenAI: Image generation guide&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://platform.openai.com/docs/models/dall-e-3" rel="noopener noreferrer"&gt;OpenAI: DALL-E 3 model documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc9110" rel="noopener noreferrer"&gt;RFC 9110: HTTP Semantics&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events" rel="noopener noreferrer"&gt;MDN: Using Server-Sent Events&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>ai</category>
      <category>images</category>
      <category>pricing</category>
      <category>media</category>
    </item>
    <item>
      <title>AI Models: Choose a Better Runtime for Linguistic Nuance in EU Chatbots (3)</title>
      <dc:creator>JamesAnderson121</dc:creator>
      <pubDate>Mon, 26 Oct 2026 16:11:17 +0000</pubDate>
      <link>https://dev.to/jamesanderson121/ai-models-choose-a-better-runtime-for-linguistic-nuance-in-eu-chatbots-3-m42</link>
      <guid>https://dev.to/jamesanderson121/ai-models-choose-a-better-runtime-for-linguistic-nuance-in-eu-chatbots-3-m42</guid>
      <description>&lt;p&gt;&lt;strong&gt;Short answer:&lt;/strong&gt; choose Claude for an EU supplier-invoice chatbot when linguistic nuance and structured field correctness outweigh completion speed; use OpenAI chat models as the faster baseline and test Gemini as the third comparator.&lt;/p&gt;

&lt;p&gt;Choose Claude first for an EU supplier-invoice chatbot when linguistic nuance and structured field correctness matter more than completion speed. Keep OpenAI chat models as the faster, more general baseline, and put Gemini in the same evaluation so the decision has a real third control.&lt;/p&gt;

&lt;p&gt;The governing question is not "does this reply sound fluent?" It is "did &lt;code&gt;supplier_name&lt;/code&gt;, &lt;code&gt;invoice_number&lt;/code&gt;, &lt;code&gt;tax_id&lt;/code&gt;, &lt;code&gt;currency&lt;/code&gt;, &lt;code&gt;line_items&lt;/code&gt;, and &lt;code&gt;total&lt;/code&gt; land in the right fields?" A French decimal comma, a German compound item description, and an English SKU can coexist in one invoice. A polished answer can still produce a broken accounting record.&lt;/p&gt;

&lt;p&gt;Three fields are the trap: &lt;code&gt;currency&lt;/code&gt;, &lt;code&gt;total&lt;/code&gt;, and &lt;code&gt;line_items&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  How should I choose AI models for a better EU chatbot runtime?
&lt;/h2&gt;

&lt;p&gt;I would freeze the JSON contract before testing a runtime. The harness should score exact field validity, normalized totals, line-item boundaries, missing-value handling, and refusal behavior. It should also retain the tenant, invoice fixture, prompt version, selected model, and response metadata for each run. That gives a reviewer something more useful than an average "quality" number.&lt;/p&gt;

&lt;p&gt;The EU boundary belongs in the experiment too. Identify which invoice data leaves the application, define access logging and tenant isolation, and check the regional deployment requirements for the organization. A compatible API surface does not settle those questions. It only makes the integration boundary easier to change.&lt;/p&gt;

&lt;p&gt;This is the decision rule I would use:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Option&lt;/th&gt;
&lt;th&gt;Strongest reason to test it&lt;/th&gt;
&lt;th&gt;Release gate&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude&lt;/td&gt;
&lt;td&gt;Better linguistic nuance for regional phrasing&lt;/td&gt;
&lt;td&gt;Structured extraction must remain exact on mixed-language invoices&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;OpenAI chat models&lt;/td&gt;
&lt;td&gt;Higher speed and broad generality&lt;/td&gt;
&lt;td&gt;Faster completion cannot increase corrections or malformed JSON&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemini&lt;/td&gt;
&lt;td&gt;A credible third comparison for language and structure&lt;/td&gt;
&lt;td&gt;The same fixtures and schema must pass without prompt-specific tuning&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The winner is the option that clears the correctness threshold inside the governance boundary. Speed is a constraint, not the product definition.&lt;/p&gt;

&lt;p&gt;The trade-off is blunt: choose Claude when correction work costs more than latency; choose OpenAI when speed wins and the same tests still clear.&lt;/p&gt;

&lt;h2&gt;
  
  
  What changes when the runtime is a portable boundary?
&lt;/h2&gt;

&lt;p&gt;The practical value of a plain REST API appears in the evaluation harness. A Python notebook can call the same HTTP boundary as a production worker; a later service in another language does not need a new client-library decision. There is no SDK installation or client version to babysit. That is a small detail until the notebook becomes the first production integration.&lt;/p&gt;

&lt;p&gt;The second useful dimension is operational ownership. A self-describing discovery surface can be inspected without a key, and the platform exposes supported model and language metadata before a run is committed. One credential and one billing boundary cover a platform with 295 routes across 20 modules. For this invoice workflow, that reduces the friction of catalog checks, evaluation jobs, and adjacent backend capabilities without pretending that a uniform API proves language quality.&lt;/p&gt;

&lt;p&gt;Infrai fits this boundary when the team values one plain REST API and one key across those capabilities: its documented capabilities include runnable examples in 10 languages, which makes the same small evaluator easier to hand from Python to another service. There is no SDK to install or client-library version to babysit; the shared API convention reduces changes when adjacent backend work moves to another service. That is a development and ownership advantage, not a reason to skip the invoice fixture test.&lt;/p&gt;

&lt;p&gt;The REST API is a separate practical advantage. Any language that can send HTTP can run the catalog check and the chat request, so a Python notebook and a later worker can share the same boundary without installing an SDK.&lt;/p&gt;

&lt;p&gt;It is not the right fit when the team needs a dedicated moderation endpoint, available ASR, or live voice sessions in its target region; use a provider with the required capability and keep this invoice comparison separate.&lt;/p&gt;

&lt;p&gt;The catalog check should be part of every evaluation record. Query &lt;code&gt;/v1/models&lt;/code&gt;, store the supported-language metadata with the fixture result, and pin the selected model for a release. Recheck readiness when the catalog changes. Here is a deliberately small Python shape for the experiment:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;random&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;urllib.error&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;HTTPError&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;urllib.request&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;urlopen&lt;/span&gt;

&lt;span class="n"&gt;BASE_URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;INFRAI_BASE_URL&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;rstrip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;API_KEY&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;INFRAI_API_KEY&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;request_json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;attempts&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;attempts&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;request&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;BASE_URL&lt;/span&gt;&lt;span class="si"&gt;}{&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Authorization&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Bearer &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;API_KEY&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Content-Type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;application/json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="nf"&gt;urlopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                    &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;RuntimeError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;request failed: HTTP &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;load&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="n"&gt;HTTPError&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;code&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="mi"&gt;429&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;attempts&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;raise&lt;/span&gt;
            &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;error&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Retry-After&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;delay&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;float&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;retry_after&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;attempt&lt;/span&gt;
            &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;delay&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;random&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;random&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;


&lt;span class="n"&gt;catalog&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;request_json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/v1/models&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;GET&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;request_json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/v1/chat/completions&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;POST&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;model&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;AI_RUNTIME_MODEL&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;messages&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
            &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;role&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;system&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;content&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Extract supplier invoice fields and return only the agreed JSON schema.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;role&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;user&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;content&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;INVOICE_TEXT&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]},&lt;/span&gt;
        &lt;span class="p"&gt;],&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;response_format&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;json_object&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;catalog&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;result&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="n"&gt;indent&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The retry behavior is part of the harness, not decoration. A 429 should honor &lt;code&gt;Retry-After&lt;/code&gt; when present, back off, and stop after a bounded number of attempts. This is an inference call rather than a state-changing write, so an idempotency key would not make repeated completions equivalent. Save the request identifier, fixture identifier, and prompt version outside the prompt; otherwise a bad line-item split is hard to reproduce.&lt;/p&gt;

&lt;h2&gt;
  
  
  Which boundaries should stop the rollout?
&lt;/h2&gt;

&lt;p&gt;There are useful limits to state plainly. The catalog marks ASR as unavailable, and real-time voice-session status is pending and limited to the western region. Neither capability belongs in an invoice chatbot launch plan. There is also no dedicated moderation endpoint; text or image moderation needs a chat model with a JSON-schema fallback. That is a readiness constraint, not an invitation to claim broader safety coverage.&lt;/p&gt;

&lt;p&gt;The recommendation also stops if the schema threshold is missed. Claude is the best first test for nuanced EU phrasing, OpenAI is the speed-oriented baseline, and Gemini is the third comparator. None of those labels replaces evidence from the actual invoice set.&lt;/p&gt;

&lt;p&gt;Start with one record contract, three engines, and a small multilingual fixture set. Then measure field correctness, correction rate, and latency under the same governance rules. The result is a runtime decision that can survive a compliance review and a code review.&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://platform.openai.com/docs/guides/function-calling" rel="noopener noreferrer"&gt;OpenAI Function Calling guide&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.anthropic.com/en/docs/build-with-claude/tool-use" rel="noopener noreferrer"&gt;Anthropic tool use documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://ai.google.dev/gemini-api/docs/structured-output" rel="noopener noreferrer"&gt;Google Gemini structured output documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://json-schema.org/specification" rel="noopener noreferrer"&gt;JSON Schema specification&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>ai</category>
      <category>models</category>
      <category>runtime</category>
      <category>chatbot</category>
    </item>
  </channel>
</rss>
