DEV Community

Cover image for VernLLM - lightweight resilience layer for OpenAI SDK
LakBud
LakBud

Posted on

VernLLM - lightweight resilience layer for OpenAI SDK

#ai #webdev #opensource #showdev

Introducing vernLLM: A Resilience Layer for LLM Applications

Building production-ready LLM applications is not just about sending prompts and receiving responses. Real-world AI systems need to handle timeouts, provider failures, rate limits, inconsistent outputs, and reliability issues.

That is where vernLLM comes in.

vernLLM is a lightweight resilience layer for OpenAI-compatible chat completion APIs, providing a single interface with built-in retries, timeouts, circuit breaking, caching, structured output, and usage tracking.

Instead of rebuilding the same reliability features for every LLM project, vernLLM gives you the tools needed to make your AI integrations more robust from the start.

Features

Automatic retries with backoff

Transient failures happen. vernLLM automatically retries recoverable errors while failing fast on validation errors and non-retryable responses.

Timeouts & cancellation

Prevent hanging requests with configurable timeouts and cancellation support.

Circuit breaker protection

Automatically stop sending requests to failing providers and recover when the service becomes healthy again.

Structured output with type safety

Pass a Zod schema and receive validated, typed results back.

const result = await llm.call({
  systemPrompt: 'Return JSON: { "skills": string[] }',
  userContent: 'Extract skills from: ...',
  schema: SkillsSchema // zod schema
});
Enter fullscreen mode Exit fullscreen mode

Provider-native JSON Schema support

Constrain model generation itself instead of only validating responses afterward.

Built-in caching support

Cache LLM responses using your own cache adapter with cachedCall and cachedLLMCall.

One interface across providers

Use the same API across multiple providers:

  • OpenAI
  • Groq
  • Mistral
  • DeepSeek
  • Cerebras
  • Together AI
  • Fireworks AI
  • Ollama
  • Anthropic
  • Gemini
  • AWS Bedrock
  • Any HTTP-compatible provider through fromFetch

Why vernLLM?

Many LLM applications end up creating their own wrappers around provider SDKs to handle:

  • retry logic
  • API failures
  • provider switching
  • response validation
  • caching
  • monitoring

vernLLM packages these patterns into a reusable, lightweight library so developers can focus on building AI features instead of maintaining infrastructure.

Quick Start

Install:

pnpm add vern-llm openai
Enter fullscreen mode Exit fullscreen mode

Create your client:

import OpenAI from 'openai';
import { VernLLM } from 'vern-llm';

export const llm = new VernLLM({
  client: new OpenAI({ apiKey: process.env.OPENAI_API_KEY }),
  model: 'gpt-4o',

  maxRetries: 3,
  timeoutMs: 10_000,
  circuitBreaker: true,

  onUsage: ({ totalTokens }) => {
    console.log(`Used ${totalTokens} tokens`);
  },
});


// now do something with it!
export const summary = await llm.cachedLLMCall({
  cacheKey: `resume:${resumeId}`,
  ttl: 3600,
  call: {
    systemPrompt: 'Analyze this resume and return structured hiring insights.',
    userContent: resumeText,
    schema: HiringSummarySchema, // Zod schema
  }
});
Enter fullscreen mode Exit fullscreen mode

Built for Production

vernLLM is designed with production usage in mind:

  • Zero bundled dependencies
  • TypeScript-first API
  • Provider adapters
  • Extensive test coverage
  • MIT licensed

Whether you are building AI agents, document processing pipelines, chat applications, or LLM-powered tools, vernLLM provides the reliability layer needed to ship with confidence.

Learn More

Documentation: https://vernllm.vercel.app/

GitHub: https://github.com/LakBud/vernLLM

Contributions, feedback, and ideas are welcome!

Top comments (3)

Collapse
 
topstar_ai profile image
Luis Cruz •

I particularly appreciate how vernLLM integrates circuit breaker protection, which can be a game-changer in preventing cascading failures when dealing with external providers like OpenAI. The example code snippet showcasing the cachedLLMCall method with a Zod schema for structured output also highlights the library's ability to simplify complex workflows. Have you considered adding support for more advanced caching strategies, such as cache invalidation based on provider-side updates, to further enhance the library's resilience features?

Collapse
 
lakbud profile image
LakBud •

Thanks for the thoughtful feedback! Glad you liked it :)

For cache invalidation, I agree that freshness is an important problem, but I intentionally kept invalidation application-specific. A library layer usually cannot reliably know when a cached LLM response has become stale, since the right strategy depends heavily on the use case ya know. For example a resume analysis, a support chatbot, and a real-time information query all have very different freshness requirements.

vernLLM currently exposes the primitives for developers to control this, including custom cache adapters, TTL-based expiration, manual cache deletion, and cache keys that can include things like model/prompt/schema versions.

That said, more advanced cache strategies are definitely interesting. Things like stale-while-revalidate patterns, smarter cache key helpers, or optional revalidation hooks could be useful additions while still keeping control in the hands of the application.

Appreciate the suggestion, it's great to think about as the library evolves!

Collapse
 
topstar_ai profile image
Luis Cruz •

Thanks and I would be happy, too.
Please check my profile and portfolio.