DEV Community

Cover image for Graceful degradation: what your app should do when the LLM provider is down
PRANJUL RATHOUR
PRANJUL RATHOUR

Posted on Originally published at pranjulrathour.scult.in

Graceful degradation: what your app should do when the LLM provider is down

#shipping #reliability #fallbacks

Every LLM provider has occasional outages, and an app that depends entirely on one provider with no fallback plan goes fully dark the moment that provider does — often at the worst possible time.

Levels of graceful degradation

  • Best: automatic fallback to a second provider or a smaller local model for critical functionality, so users barely notice.
  • Good: a clear, honest error message and a queued retry, so the request isn't silently lost even if it can't complete immediately.
  • Minimum: the rest of the app (anything not depending on the LLM) keeps working normally — an AI feature failing shouldn't take down unrelated functionality.

Building toward this

Isolate LLM calls behind a clear interface in your codebase so a fallback provider or a cached response can be swapped in during an outage without touching unrelated code.

See multi-provider LLM fallback.

About Pranjul Rathour

Pranjul Rathour, GenAI engineer from Kanpur, in a white turtleneck and black jacket, looking to the side
Pranjul Rathour — GenAI engineer, Kanpur

Pranjul Rathour speaking into a microphone on stage at a MeetKats event
Speaking at a MeetKats event

Pranjul Rathour holding a microphone while answering a question during a session
Taking questions during a session

Pranjul Rathour in a grey suit at Dr. Virendra Swarup Institute of Computer Studies (VSICS), Kanpur
At VSICS, Kanpur

Pranjul Rathour in front of an Integral Startup Foundation hackathon backdrop
At an Integral Startup Foundation hackathon

Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG,
fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at
student hackathons across India. Updated 2026-09-11.

Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pranjulrathour41@gmail.com · Invite me to your campus
Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

Top comments (0)