DEV Community: LakBud

[Boost]

LakBud — Wed, 19 Aug 2026 17:28:42 +0000

LakBud

Aug 17

14 Core Features you need for your LLM calls which VernLLM covers

#ai #programming #llm #productivity

4 min read

14 Core Features you need for your LLM calls which VernLLM covers

LakBud — Mon, 17 Aug 2026 15:13:20 +0000

The first version of any LLM app always works. Send a prompt, get a response, ship it.

Then real traffic shows up. Providers go down. Rate limits kick in. Requests hang and time out. Models return broken JSON. Users double click submit and you pay twice. Nobody can say how much you spent last month.

VernLLM exists so you don't relearn all of this the hard way. The core call stays simple, and the reliability and control features are ready to switch on when you need them. Everything is opt-in.

Here are the 14 features that make that possible.

1. Provider Fallback

Every provider has bad days. Fallback lets you define backup targets and a policy for when to switch to them. It's not "try everything until something works." It's a deliberate policy for which errors trigger a switch and in what order. Your app stays up even when one provider doesn't.

2. Circuit Breaker

Retrying a dead provider just adds load to something already struggling. A circuit breaker tracks failure patterns and, once a provider is clearly unhealthy, stops sending it traffic for a while. This works especially well alongside fallback: the circuit breaker keeps a bad provider from eating retry attempts while a healthy one picks up the slack.

3. Retries

Some failures deserve another attempt: a network blip, a temporary 500. VernLLM's retries include exponential backoff, jitter (so failures don't all retry in sync), and Retry-After awareness. The goal is to retry what's worth retrying and let permanent errors fail fast instead of burning attempts on something that was never going to work.

4. Rate Limiting

Providers cap requests, tokens, concurrency, or time based quotas. Most teams discover these limits by hitting them. VernLLM's rate limiter lets you define those limits ahead of time and coordinate traffic before the provider has to say no.

5. Usage Metering

Metering can reserve budget before a call and refund it automatically if the call fails. That refund step matters: without it, a failed request still looks like spent budget, and your internal usage numbers quietly drift from reality.

6. Caching

A lot of LLM traffic is repeats: the same prompt twice, a page refresh, an unnecessary retry. cachedCall wraps requests so identical work doesn't get redone, which means fewer provider calls, lower cost, and faster responses for anything stable enough to reuse.

7. Observability

Reliability features you can't see are hard to trust. VernLLM emits one unified event stream for retries, circuit breaker transitions, rate limit waits, and fallback events, so you can actually answer questions like why a request was slow or why the provider switched, instead of guessing.

8. Usage Tracking

Metering controls usage in the moment; tracking tells you what was actually consumed afterward. VernLLM captures token usage from provider responses, which is the raw material for dashboards, cost reports, per user or per model accounting, quotas, and billing.

9. Error Handling

Not all failures are the same, but a single generic catch block treats them that way. VernLLM's structured LLMError types let you tell a rate limit from a timeout from a broken request, so you can decide what's worth retrying and what should fail immediately.

10. Cancellation & Timeouts

Requests take time, and users don't always want to wait. VernLLM supports AbortSignal, per-attempt timeouts, and retry-aware cancellation, so a user backing out mid-request actually stops the flow instead of letting it keep retrying in the background.

11. Structured Output

A model that "usually" returns JSON isn't the same as data you can trust. VernLLM supports structured output through client-side Zod validation and provider-native JSON Schema modes, so every response gets checked against a real schema before your app touches it.

12. Tool Calling

VernLLM supports tool calling but never executes tools itself. The model can request a tool call; your application decides which tools exist, what arguments are valid, whether the call is authorized, how it runs, and what result goes back. The model proposes, your app disposes.

13. Streaming

Streaming pushes chunks to the user as they're generated instead of making everyone wait for the full response. VernLLM streams without losing the reliability layer around it, retries and caching still apply, so it's not a separate, lesser API.

14. Pluggable Logger

Every team logs differently: console output, structured JSON, an observability platform, an internal abstraction. VernLLM lets you plug in your own logger instead of forcing you to adopt its own.

The Bigger Picture

Each feature solves a narrow problem. Together, they answer a bigger one: how do you turn a raw LLM call into a production-ready capability?

                    Your Application
                           │
                           ▼
                    ┌─────────────┐
                    │   VernLLM   │
                    └─────────────┘
                           │
          ┌────────────────┼────────────────┐
          │                │                │
          ▼                ▼                ▼
     Rate Limit        Cache          Usage Meter
          │                │                │
          └────────────────┼────────────────┘
                           ▼
                    Provider Request
                           │
                 ┌─────────┴─────────┐
                 │                   │
              Success              Failure
                 │                   │
                 ▼                   ▼
          Usage Tracking         Retry /
          + Streaming           Circuit Breaker
                                     │
                                     ▼
                                  Fallback
                                     │
                                     ▼
                              Backup Provider

You don't need all 14 at once. A small project might just want structured output and streaming. A production platform might use all of it together. You choose what your app needs.

Why This Matters

Networks fail, dependencies go down, users cancel, providers have limits, and responses need validation. That was true before LLMs and it's still true now. Getting a model to generate a good answer was never the hard part. Building a system that can rely on it at scale is.

Use one feature. Use all fourteen. The architecture stays yours.

Docs: https://vernllm.vercel.app

npm install vern-llm

Thanks for 1,000+ downloads! VernLLM v1.0.0 is here

LakBud — Fri, 24 Jul 2026 14:04:11 +0000

Thanks for 1,000+ downloads! VernLLM just hit v1.0.0, and it comes with one breaking change worth knowing about if you're using a custom cache adapter, plus a handful of smaller improvements that landed alongside it.

What is VernLLM?

VernLLM is a lightweight resilience layer for LLM chat completions: retries, timeouts, caching, circuit breaking, structured output, and usage tracking. All behind one typed interface, with adapters for OpenAI-compatible providers (OpenAI, Groq, Mistral, DeepSeek, Cerebras, Together AI, Fireworks, Ollama), plus Anthropic, Gemini, and AWS Bedrock.

import OpenAI from 'openai';
import { VernLLM } from 'vern-llm';

const llm = new VernLLM({
  client: new OpenAI({ apiKey: process.env.OPENAI_API_KEY }),
  model: 'gpt-4o',
  maxRetries: 3,
  timeoutMs: 10_000,
  circuitBreaker: true,
});

const result = await llm.call({
  systemPrompt: 'Return JSON: { "skills": string[] }',
  userContent: 'Extract skills from: ...',
});

The breaking change: `CacheAdapter.get()`

Previously, CacheAdapter.get() returned Promise<T | null>. The problem: there was no way to tell the difference between "nothing is cached for this key" and "the cached value legitimately is null." Both looked identical, so a real null result got treated as a cache miss and silently re-triggered another LLM call every time.

In v1.0.0, get() now returns:

Promise<{ hit: boolean; value: T | null }>

hit reflects whether the key existed in the underlying store, independent of what the value was. { hit: true, value: null } means "we have a cached result, and it's null." { hit: false, value: null } means "nothing is cached."

If you're using the built-in InMemoryCacheAdapter, you don't need to do anything, it's already updated.

If you've written a custom adapter (Redis, Upstash, etc.), you'll need to update get():

// Before
async get(key: string): Promise<MyValue | null> {
  const raw = await redis.get(key);
  return raw ? JSON.parse(raw) : null;
}

// After
async get(key: string): Promise<{ hit: boolean; value: MyValue | null }> {
  const raw = await redis.get(key);

  if (raw === null) {
    return { hit: false, value: null };
  }

  return { hit: true, value: JSON.parse(raw) };
}

Most stores already give you this signal for free — Redis's GET returning null for "key doesn't exist" is distinguishable from a null you stored yourself, so the fix is usually just an existence check rather than a value !== null check.

If you'd rather not bother with the distinction and are fine with null results never being served from cache, there's also a one-line shim:

return { hit: value !== null, value };

Also in this release

CallParams.systemPrompt is now optional — omit it and no system message is sent at all, instead of forcing an empty string through.
AnthropicClient, GeminiClient, and BedrockConverseClient are now exported as public types, so you can type your own client instances against them without reaching into internals.
New adapters barrel export — import { fromAnthropic, fromGemini, fromBedrock, fromFetch, fromOpenAICompatible } from 'vern-llm/adapters' instead of digging through individual files.
In-memory cache now has a size limit to prevent unbounded growth, evicting the oldest entries once the limit is hit.
Internal types got refactored into focused modules, and test coverage was extended for optional system prompts, adapter payloads, and cache adapter edge cases (custom adapters, size bounds, failure handling).

Upgrading

npm install vern-llm@latest

If you're only using the built-in InMemoryCacheAdapter (the default), this is a drop-in upgrade. If you've implemented a custom CacheAdapter, update its get() method per the migration guide above before upgrading in production.

Full changelog and docs: vernllm.vercel.app · GitHub

Thanks for using VernLLM, and for the 1,000+ downloads. If you hit any issues with the migration, open an issue on GitHub.

[Boost]

LakBud — Tue, 21 Jul 2026 00:05:36 +0000

LakBud

Jul 20

VernLLM - lightweight resilience layer for LLM Applications

#showdev #opensource #ai #webdev

2 min read

VernLLM - lightweight resilience layer for LLM Applications

LakBud — Mon, 20 Jul 2026 21:20:29 +0000

Introducing vernLLM: A Resilience Layer for LLM Applications

Building production-ready LLM applications is not just about sending prompts and receiving responses. Real-world AI systems need to handle timeouts, provider failures, rate limits, inconsistent outputs, and reliability issues.

That is where vernLLM comes in.

vernLLM is a lightweight resilience layer for OpenAI-compatible chat completion APIs, providing a single interface with built-in retries, timeouts, circuit breaking, caching, structured output, and usage tracking.

Instead of rebuilding the same reliability features for every LLM project, vernLLM gives you the tools needed to make your AI integrations more robust from the start.

Features

Automatic retries with backoff

Transient failures happen. vernLLM automatically retries recoverable errors while failing fast on validation errors and non-retryable responses.

Timeouts & cancellation

Prevent hanging requests with configurable timeouts and cancellation support.

Circuit breaker protection

Automatically stop sending requests to failing providers and recover when the service becomes healthy again.

Structured output with type safety

Pass a Zod schema and receive validated, typed results back.

const result = await llm.call({
  systemPrompt: 'Return JSON: { "skills": string[] }',
  userContent: 'Extract skills from: ...',
  schema: SkillsSchema // zod schema
});

Provider-native JSON Schema support

Constrain model generation itself instead of only validating responses afterward.

Built-in caching support

Cache LLM responses using your own cache adapter with cachedCall and cachedLLMCall.

One interface across providers

Use the same API across multiple providers:

OpenAI
Groq
Mistral
DeepSeek
Cerebras
Together AI
Fireworks AI
Ollama
Anthropic
Gemini
AWS Bedrock
Any HTTP-compatible provider through fromFetch

Why vernLLM?

Many LLM applications end up creating their own wrappers around provider SDKs to handle:

retry logic
API failures
provider switching
response validation
caching
monitoring

vernLLM packages these patterns into a reusable, lightweight library so developers can focus on building AI features instead of maintaining infrastructure.

Quick Start

Install:

pnpm add vern-llm openai

Create your client:

import OpenAI from 'openai';
import { VernLLM } from 'vern-llm';

export const llm = new VernLLM({
  client: new OpenAI({ apiKey: process.env.OPENAI_API_KEY }),
  model: 'gpt-4o',

  maxRetries: 3,
  timeoutMs: 10_000,
  circuitBreaker: true,

  onUsage: ({ totalTokens }) => {
    console.log(`Used ${totalTokens} tokens`);
  },
});


// now do something with it!
export const summary = await llm.cachedLLMCall({
  cacheKey: `resume:${resumeId}`,
  ttl: 3600,
  call: {
    systemPrompt: 'Analyze this resume and return structured hiring insights.',
    userContent: resumeText,
    schema: HiringSummarySchema, // Zod schema
  }
});

Built for Production

vernLLM is designed with production usage in mind:

Zero bundled dependencies
TypeScript-first API
Provider adapters
Extensive test coverage
MIT licensed

Whether you are building AI agents, document processing pipelines, chat applications, or LLM-powered tools, vernLLM provides the reliability layer needed to ship with confidence.

Learn More

Documentation: https://vernllm.vercel.app/

GitHub: https://github.com/LakBud/vernLLM

Contributions, feedback, and ideas are welcome!

DEV Community: LakBud

[Boost]

14 Core Features you need for your LLM calls which VernLLM covers

14 Core Features you need for your LLM calls which VernLLM covers

1. Provider Fallback

2. Circuit Breaker

3. Retries

4. Rate Limiting

5. Usage Metering

6. Caching

7. Observability

8. Usage Tracking

9. Error Handling

10. Cancellation & Timeouts

11. Structured Output

12. Tool Calling

13. Streaming

14. Pluggable Logger

The Bigger Picture

Why This Matters

Thanks for 1,000+ downloads! VernLLM v1.0.0 is here

What is VernLLM?

The breaking change: CacheAdapter.get()

Also in this release

Upgrading

[Boost]

VernLLM - lightweight resilience layer for LLM Applications

VernLLM - lightweight resilience layer for LLM Applications

Introducing vernLLM: A Resilience Layer for LLM Applications

Features

Automatic retries with backoff

Timeouts & cancellation

Circuit breaker protection

Structured output with type safety

Provider-native JSON Schema support

Built-in caching support

One interface across providers

Why vernLLM?

Quick Start

Built for Production

Learn More

The breaking change: `CacheAdapter.get()`