<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: rohan9024</title>
    <description>The latest articles on DEV Community by rohan9024 (@rohan9024).</description>
    <link>https://dev.to/rohan9024</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F944870%2F096884f9-af7a-4b5d-adf4-025a7f0b8164.jpeg</url>
      <title>DEV Community: rohan9024</title>
      <link>https://dev.to/rohan9024</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/rohan9024"/>
    <language>en</language>
    <item>
      <title>Why AI Apps Crash Under Real Traffic (And How to Fix It in 2 Lines)</title>
      <dc:creator>rohan9024</dc:creator>
      <pubDate>Sat, 21 Nov 2026 21:35:06 +0000</pubDate>
      <link>https://dev.to/rohan9024/why-ai-apps-crash-under-real-traffic-and-how-to-fix-it-in-2-lines-43a1</link>
      <guid>https://dev.to/rohan9024/why-ai-apps-crash-under-real-traffic-and-how-to-fix-it-in-2-lines-43a1</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; Generic retry wrappers like &lt;code&gt;p-retry&lt;/code&gt; or &lt;code&gt;while&lt;/code&gt; loops weren't built for LLM APIs. They multiply SDK retries into &lt;strong&gt;9x–12x traffic storms&lt;/strong&gt;, sleep on hard billing errors (&lt;code&gt;insufficient_quota&lt;/code&gt;), corrupt streamed UI text on mid-stream disconnects, and break tool-call message histories. &lt;a href="https://github.com/rohan9024/retrykit" rel="noopener noreferrer"&gt;&lt;code&gt;@rohan9024/retrykit&lt;/code&gt;&lt;/a&gt; (&lt;code&gt;v2.0.0&lt;/code&gt;) is a &lt;strong&gt;zero-dependency TypeScript reliability engine&lt;/strong&gt; for AI apps that solves rate limits, multi-key rotation, circuit breaking, stream recovery, and atomic context compaction in one package.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;If you have ever shipped an AI feature with OpenAI, Anthropic, or Gemini, you have probably seen this story play out in production:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Users send a burst of requests.&lt;/li&gt;
&lt;li&gt;Your provider returns &lt;code&gt;429 Too Many Requests&lt;/code&gt; (or Anthropic returns &lt;code&gt;529 Overloaded&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;Your retry loop waits a second and retries every single request at the exact same time.&lt;/li&gt;
&lt;li&gt;Worse: the OpenAI or Anthropic SDK &lt;em&gt;already&lt;/em&gt; has built-in retries (&lt;code&gt;maxRetries: 2&lt;/code&gt; by default). Wrapping your own 3-retry loop around it means &lt;strong&gt;one failed call can turn into 12 requests (&lt;/strong&gt;&lt;code&gt;3 × 4&lt;/code&gt;&lt;strong&gt;)&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Everything slows to a crawl, users see duplicated streamed text, and your bill spikes.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This guide explains &lt;strong&gt;why standard HTTP retry logic fails on LLM APIs&lt;/strong&gt; and how to fix it cleanly in under a minute using &lt;strong&gt;RetryKit (&lt;/strong&gt;&lt;code&gt;@rohan9024/retrykit&lt;/code&gt;&lt;strong&gt;)&lt;/strong&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  5 Ways Generic Retries Break AI Applications
&lt;/h2&gt;

&lt;p&gt;Before writing code, it helps to understand what actually goes wrong when you use a generic retry helper for LLMs:&lt;/p&gt;

&lt;h3&gt;
  
  
  1. The Hidden Double-Retry Storm (&lt;code&gt;maxRetries&lt;/code&gt; Multiplication)
&lt;/h3&gt;

&lt;p&gt;Most official AI SDKs already retry failed requests twice internally. When developers add a custom &lt;code&gt;retry()&lt;/code&gt; wrapper or queue around the SDK without setting &lt;code&gt;maxRetries: 0&lt;/code&gt;, retries multiply instead of adding up—flooding your own API quota during an outage.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Sleeping on Hard Billing Errors (&lt;code&gt;insufficient_quota&lt;/code&gt;)
&lt;/h3&gt;

&lt;p&gt;Not every &lt;code&gt;429&lt;/code&gt; error means &lt;em&gt;"slow down and try again."&lt;/em&gt; OpenAI returns &lt;code&gt;429&lt;/code&gt; for &lt;code&gt;insufficient_quota&lt;/code&gt; or spend limits, and Anthropic returns &lt;code&gt;enforced_spend_limit_reached&lt;/code&gt;. Waiting 1s → 2s → 4s → 8s on a dead key just freezes your user's request for 15+ seconds for no reason. You should &lt;strong&gt;quarantine that key immediately and rotate to a backup key&lt;/strong&gt; with zero sleep.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Ignoring Provider-Specific Reset Headers
&lt;/h3&gt;

&lt;p&gt;Providers return rate-limit resets in wildly different formats:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;retry-after-ms&lt;/code&gt; (milliseconds)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;retry-after&lt;/code&gt; (seconds or HTTP date strings)&lt;/li&gt;
&lt;li&gt;OpenAI Go-style durations (&lt;code&gt;"464ms"&lt;/code&gt;, &lt;code&gt;"1m30s"&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Anthropic RFC 3339 timestamps (&lt;code&gt;anthropic-ratelimit-requests-reset&lt;/code&gt;)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Generic backoff ignores these headers—retrying either too early (hitting another 429) or much too late.&lt;/p&gt;

&lt;h3&gt;
  
  
  4. Replaying Streams Duplicates Text in the UI
&lt;/h3&gt;

&lt;p&gt;Suppose your LLM streams 40 tokens to the user's chat screen and then stalls on token 41. If a naive wrapper restarts the HTTP call from scratch and yields tokens from the beginning, &lt;strong&gt;your user sees the first paragraph pasted twice&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Safe streaming recovery must treat &lt;strong&gt;pre-first-token failures&lt;/strong&gt; differently from &lt;strong&gt;mid-stream failures&lt;/strong&gt; (where resuming requires an explicit checkpoint mechanism like &lt;code&gt;getCheckpoint&lt;/code&gt;).&lt;/p&gt;

&lt;h3&gt;
  
  
  5. Naive Context Truncation Crashes Tool Calling
&lt;/h3&gt;

&lt;p&gt;When an agent hits a context limit and you trim older chat messages blindly, you can accidentally delete an &lt;code&gt;assistant&lt;/code&gt; message containing &lt;code&gt;tool_calls&lt;/code&gt; while leaving its &lt;code&gt;tool&lt;/code&gt; response behind (or vice versa). Both OpenAI and Anthropic reject orphaned tool pairs with a hard &lt;code&gt;400 Bad Request&lt;/code&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Fix: &lt;code&gt;@rohan9024/retrykit&lt;/code&gt; (Zero Dependencies, AI-Native)
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://www.npmjs.com/package/@rohan9024/retrykit" rel="noopener noreferrer"&gt;&lt;code&gt;@rohan9024/retrykit&lt;/code&gt;&lt;/a&gt; is a lightweight, zero-dependency TypeScript reliability library built specifically for AI apps, Node.js backends, and Edge runtimes (&lt;code&gt;Node &amp;gt;= 20&lt;/code&gt;, Bun, Deno, Cloudflare Workers).&lt;/p&gt;

&lt;p&gt;Install it with your package manager:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npm install @rohan9024/retrykit
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;em&gt;(Also works with&lt;/em&gt; &lt;code&gt;pnpm add @rohan9024/retrykit&lt;/code&gt;&lt;em&gt;,&lt;/em&gt; &lt;code&gt;yarn add @rohan9024/retrykit&lt;/code&gt;&lt;em&gt;, or&lt;/em&gt; &lt;code&gt;bun add @rohan9024/retrykit&lt;/code&gt;&lt;em&gt;.)&lt;/em&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  Quick Start: Wrap OpenAI / Anthropic in 2 Lines
&lt;/h2&gt;

&lt;p&gt;The fastest way to protect an existing codebase is &lt;code&gt;@rohan9024/retrykit/auto&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;OpenAI&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="s2"&gt;"openai"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;wrapAI&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="s2"&gt;"@rohan9024/retrykit/auto"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;openai&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;wrapAI&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;OpenAI&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;apiKey&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;OPENAI_API_KEY&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;provider&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"openai"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;// Use the SDK exactly as before:&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;openai&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;chat&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;completions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"gpt-4o"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;messages&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;role&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"user"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Explain circuit breakers simply."&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  What &lt;code&gt;wrapAI()&lt;/code&gt; handles automatically under the hood:
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Stops retry storms:&lt;/strong&gt; Automatically normalizes request-level &lt;code&gt;maxRetries: 0&lt;/code&gt; so the SDK and RetryKit never stack retries.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Auto-discovers backup API keys:&lt;/strong&gt; Reads &lt;code&gt;OPENAI_API_KEYS&lt;/code&gt; (comma-separated) or &lt;code&gt;OPENAI_API_KEY_1&lt;/code&gt;, &lt;code&gt;OPENAI_API_KEY_2&lt;/code&gt;, etc. (plus &lt;code&gt;ANTHROPIC_API_KEYS&lt;/code&gt; and &lt;code&gt;GEMINI_API_KEYS&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Concurrency-safe per-request key injection:&lt;/strong&gt; Injects rotated keys cleanly via per-request headers (&lt;code&gt;Authorization&lt;/code&gt;, &lt;code&gt;x-api-key&lt;/code&gt;, &lt;code&gt;x-goog-api-key&lt;/code&gt;) instead of mutating shared global client state.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fast-fails hard billing errors:&lt;/strong&gt; Detects &lt;code&gt;insufficient_quota&lt;/code&gt;, &lt;code&gt;credit_balance_exhausted&lt;/code&gt;, &lt;code&gt;organization_spend_limit_exceeded&lt;/code&gt;, &lt;code&gt;project_spend_limit_exceeded&lt;/code&gt;, and &lt;code&gt;enforced_spend_limit_reached&lt;/code&gt; and rotates immediately instead of wasting sleep time.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;True timeout &amp;amp; cancellation:&lt;/strong&gt; Propagates linked &lt;code&gt;AbortController&lt;/code&gt; signals (&lt;code&gt;attemptTimeoutMs&lt;/code&gt;, &lt;code&gt;totalDeadlineMs&lt;/code&gt;) so timed-out network calls actually abort instead of leaking sockets.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Audit Your Existing Codebase in 5 Seconds
&lt;/h2&gt;

&lt;p&gt;Already have a big TypeScript project? Run the built-in static doctor CLI to find unguarded AI clients and hidden double-retry loops:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx @rohan9024/retrykit doctor
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It scans your source files and points to the exact file and line where:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;new OpenAI(...)&lt;/code&gt; or &lt;code&gt;new Anthropic(...)&lt;/code&gt; is called without protection&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;maxRetries &amp;gt; 0&lt;/code&gt; could trigger double-retry amplification&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;You can also scaffold a project starter config safely with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx @rohan9024/retrykit init --dry-run
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Modular Subpaths (Import Only What You Need)
&lt;/h2&gt;

&lt;p&gt;RetryKit v2.0.0 ships clean ESM + CommonJS + TypeScript declarations across focused subpaths with &lt;code&gt;"sideEffects": false&lt;/code&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Import Path&lt;/th&gt;
&lt;th&gt;What It Solves&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;@rohan9024/retrykit/auto&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Drop-in &lt;code&gt;wrapAI()&lt;/code&gt; proxy, multi-key discovery (&lt;code&gt;discoverApiKeys&lt;/code&gt;), SDK retry neutralization&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;@rohan9024/retrykit/core&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Core &lt;code&gt;retry()&lt;/code&gt;, &lt;code&gt;CircuitBreaker&lt;/code&gt;, &lt;code&gt;KeyPool&lt;/code&gt;, backoff + jitter, linked timeouts&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;@rohan9024/retrykit/fetch&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Composable &lt;code&gt;createRetryFetch()&lt;/code&gt; with safe body cloning &amp;amp; idempotency checks&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;@rohan9024/retrykit/stream&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Streaming watchdog &amp;amp; explicit &lt;code&gt;getCheckpoint&lt;/code&gt;-based safe recovery&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;@rohan9024/retrykit/compaction&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Tool-call-safe atomic context compaction (never orphans &lt;code&gt;tool_calls&lt;/code&gt; / &lt;code&gt;tool&lt;/code&gt; results)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;@rohan9024/retrykit/budget&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;RetryBudget&lt;/code&gt; and &lt;code&gt;TokenBucketRetryBudget&lt;/code&gt; to cap retry amplification&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;@rohan9024/retrykit/ai-sdk&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Zero-dependency structural middleware (&lt;code&gt;transformParams&lt;/code&gt;, &lt;code&gt;wrapGenerate&lt;/code&gt;, &lt;code&gt;wrapStream&lt;/code&gt;) for Vercel AI SDK&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;@rohan9024/retrykit/mcp&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Safe Model Context Protocol (&lt;code&gt;MCP&lt;/code&gt;) retry wrapper (read-only allowlists; never retries unsafe &lt;code&gt;tools/call&lt;/code&gt; by default)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;@rohan9024/retrykit/validate&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Retry-on-validation-failure utilities&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  How RetryKit Compares to Alternatives
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Capability&lt;/th&gt;
&lt;th&gt;
&lt;code&gt;@rohan9024/retrykit&lt;/code&gt; v2.0&lt;/th&gt;
&lt;th&gt;
&lt;code&gt;p-retry&lt;/code&gt; / &lt;code&gt;async-retry&lt;/code&gt;
&lt;/th&gt;
&lt;th&gt;&lt;code&gt;cockatiel&lt;/code&gt;&lt;/th&gt;
&lt;th&gt;Raw OpenAI / Anthropic SDK&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Zero runtime dependencies&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes (&lt;code&gt;0&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Full SDK&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;OpenAI / Anthropic reset header parsing&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes (&lt;code&gt;464ms&lt;/code&gt;, &lt;code&gt;1m30s&lt;/code&gt;, RFC 3339, &lt;code&gt;529&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;Basic&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Hard billing error fast-rotation&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;No (sleeps &amp;amp; fails)&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Per-request multi-key rotation&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Circuit breakers + token-bucket retry budgets&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;Partial&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Checkpoint-safe stream handling&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes (&lt;code&gt;getCheckpoint&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Atomic tool-call context compaction&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Vercel AI SDK &amp;amp; MCP adapters included&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;strong&gt;AI Coding Agent Skill + CLI&lt;/strong&gt; &lt;code&gt;doctor&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  Using RetryKit with AI Coding Agents (Cursor, Claude Code, Codex, Copilot)
&lt;/h2&gt;

&lt;p&gt;Building with an AI coding assistant? RetryKit includes an official &lt;strong&gt;Agent Skill&lt;/strong&gt; (&lt;code&gt;skills/retrykit/SKILL.md&lt;/code&gt; following the &lt;code&gt;agentskills.io&lt;/code&gt; standard), &lt;code&gt;llms.txt&lt;/code&gt;, &lt;code&gt;llms-full.txt&lt;/code&gt;, &lt;code&gt;AGENTS.md&lt;/code&gt;, and &lt;code&gt;context7.json&lt;/code&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Option 1: Install the Reusable Agent Skill
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx skills add rohan9024/retrykit --skill retrykit
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Option 2: Paste This Prompt Into Cursor / Claude Code / Copilot
&lt;/h3&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;"Inspect my project for AI SDK calls (&lt;/em&gt;&lt;code&gt;openai&lt;/code&gt;&lt;em&gt;,&lt;/em&gt; &lt;code&gt;@anthropic-ai/sdk&lt;/code&gt;&lt;em&gt;, Vercel AI SDK, or&lt;/em&gt; &lt;code&gt;fetch&lt;/code&gt; &lt;em&gt;LLM requests). Run&lt;/em&gt; &lt;code&gt;npx @rohan9024/retrykit doctor&lt;/code&gt;&lt;em&gt;, install&lt;/em&gt; &lt;code&gt;@rohan9024/retrykit&lt;/code&gt; &lt;em&gt;with my project's package manager, wrap the clients using&lt;/em&gt; &lt;code&gt;@rohan9024/retrykit/auto&lt;/code&gt;&lt;em&gt;, ensure&lt;/em&gt; &lt;code&gt;maxRetries: 0&lt;/code&gt; &lt;em&gt;is set to prevent double-retry storms, and verify with tests."&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;You can also run RetryKit's built-in stdio MCP diagnostic server in any MCP-compatible editor:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx @rohan9024/retrykit mcp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Frequently Asked Questions (FAQ)
&lt;/h2&gt;

&lt;h3&gt;
  
  
  How do I handle OpenAI 429 rate limits in TypeScript without causing a retry storm?
&lt;/h3&gt;

&lt;p&gt;Use &lt;code&gt;@rohan9024/retrykit/auto&lt;/code&gt; (&lt;code&gt;wrapAI&lt;/code&gt;) or &lt;code&gt;@rohan9024/retrykit/core&lt;/code&gt; (&lt;code&gt;retry&lt;/code&gt; + &lt;code&gt;TokenBucketRetryBudget&lt;/code&gt;). RetryKit automatically sets request-level &lt;code&gt;maxRetries: 0&lt;/code&gt; on wrapped OpenAI/Anthropic SDK calls so SDK retries and custom retries never multiply, while honoring &lt;code&gt;retry-after-ms&lt;/code&gt;, &lt;code&gt;x-ratelimit-reset-*&lt;/code&gt;, and jittered backoff.&lt;/p&gt;

&lt;h3&gt;
  
  
  Should I retry &lt;code&gt;429 insufficient_quota&lt;/code&gt; errors?
&lt;/h3&gt;

&lt;p&gt;No. Sleeping and retrying an exhausted key wastes latency. RetryKit detects &lt;code&gt;insufficient_quota&lt;/code&gt;, &lt;code&gt;credit_balance_exhausted&lt;/code&gt;, &lt;code&gt;organization_spend_limit_exceeded&lt;/code&gt;, &lt;code&gt;project_spend_limit_exceeded&lt;/code&gt;, and &lt;code&gt;enforced_spend_limit_reached&lt;/code&gt;, failing fast on that key so a backup key can take over immediately.&lt;/p&gt;

&lt;h3&gt;
  
  
  How does RetryKit prevent duplicate output when an LLM stream disconnects?
&lt;/h3&gt;

&lt;p&gt;If a stream fails mid-way after chunks have already been emitted, silently replaying from the beginning would duplicate text. RetryKit refuses silent replay after output begins unless you supply an explicit &lt;code&gt;getCheckpoint&lt;/code&gt; recovery mechanism.&lt;/p&gt;

&lt;h3&gt;
  
  
  Does &lt;code&gt;@rohan9024/retrykit&lt;/code&gt; add bloat or third-party supply-chain dependencies?
&lt;/h3&gt;

&lt;p&gt;No. &lt;code&gt;@rohan9024/retrykit&lt;/code&gt; has &lt;code&gt;"dependencies": {}&lt;/code&gt; (zero runtime dependencies) and exposes tree-shakeable ESM/CJS subpath exports with &lt;code&gt;"sideEffects": false&lt;/code&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  Get Started
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;GitHub:&lt;/strong&gt; &lt;a href="https://github.com/rohan9024/retrykit" rel="noopener noreferrer"&gt;https://github.com/rohan9024/retrykit&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;npm:&lt;/strong&gt; &lt;a href="https://www.npmjs.com/package/@rohan9024/retrykit" rel="noopener noreferrer"&gt;https://www.npmjs.com/package/@rohan9024/retrykit&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npm install @rohan9024/retrykit
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If RetryKit saves you from a production &lt;code&gt;429&lt;/code&gt; incident, leave a star on GitHub or open an issue telling us which AI SDK or framework adapter you want next!&lt;/p&gt;

</description>
      <category>ai</category>
      <category>node</category>
      <category>openai</category>
      <category>llm</category>
    </item>
  </channel>
</rss>
