DEV Community

Cover image for I threw 1000+ LLM calls. Here is how VernLLM handled it.
LakBud
LakBud

Posted on

I threw 1000+ LLM calls. Here is how VernLLM handled it.

#programming #javascript #api #ai

VernLLM is the LLM call framework for TypeScript. This video tests out its features, which includes adaptive AIMD rate-limiting, advanced circuit-breakers, multi provider fallbacks, developed retries with budget and observability. Open source with MIT license.

Website: https://vernllm.dev
GitHub: https://github.com/LakBud/vernLLM
npm: https://www.npmjs.com/package/vern-llm

Top comments (1)

Collapse
 
lakbud profile image
LakBud •

This test is using a mock AI provider with a 70% guaranteed failure rate. This is because using 1000+ LLM calls with an actual AI provider isnt needed to actually test out its features and its also expensive.