Selixes - I Engineered a Sub-15ms Failover Proxy to Stop AI Agent Crashes

Kunal Kaushik — Fri, 10 Jul 2026 07:40:02 +0000

This is a submission for Weekend Challenge: Passion Edition

What I Built

The modern AI ecosystem relies entirely on upstream API stability. Yet, building autonomous agents right now feels like building on quicksand. OpenAI, Anthropic, and other major providers routinely suffer from latency spikes, unexpected 502 Bad Gateway errors, and aggressive 429 Too Many Requests rate limits.

For consumer apps, a dropped connection is an annoyance. For enterprise autonomous agents executing multi-step workflows, a single dropped request mid-stream can crash an entire loop—often resulting in runaway retry logic that burns hundreds of dollars in API credits before you even receive a PagerDuty alert.

I am obsessed with system architecture and high availability. I became so passionate about solving this specific edge case that I spent the last few months engineering Selixes — a high-throughput edge proxy gateway designed to sit between your application and your LLM providers.

Selixes intercepts the data stream in real-time. If the primary provider's latency spikes beyond an acceptable threshold or drops the connection mid-generation, Selixes instantly fails over to a backup model in under 15ms, completely invisibly to the end-user. It also utilizes atomic Redis operations to enforce hard session spend caps, acting as an absolute circuit breaker against runaway agent spend.

Demo

You can explore the live architecture, documentation, and the developer platform here: https://selixes.com

(Note: Because Selixes operates entirely at the infrastructure routing layer, the magic happens invisibly on the backend. However, you can experience the failover mechanism by swapping out your base URL using the SDK integration below.)

Code

The core philosophy behind Selixes was zero-friction adoption. Because the gateway exposes a fully standard, OpenAI-compatible endpoint, engineering teams do not need to rewrite their agent logic or learn a new SDK.

You can view the full source code for the proxy and deployment instructions on our GitHub repository:
🔗 GitHub Repository: selixes/selixes-gateway

You simply point your existing client to the gateway:

import OpenAI from 'openai';

// Before: Direct connection (Highly vulnerable to outages and rate limits)
// const client = new OpenAI({ apiKey: 'sk-...' });

// After: Proxied through Selixes with Google AI as the failover engine
const client = new OpenAI({
  baseURL: 'https://gateway.selixes.com/v1', 
  apiKey: 'slx_...' 
});

How I Built It

The fundamental engineering challenge was achieving mid-stream failover without introducing massive latency. If an upstream provider accepts a connection but drops it mid-generation, catching that error and spinning up a brand-new TLS handshake to a backup model typically causes a massive lag spike that breaks the UX.

Here is a technical teardown of the architecture I built to solve this:

1. Atomic Spend Caps via Redis

Every incoming request performs a lightning-fast lookup using a Redis INCRBYFLOAT operation. It calculates the exact token cost and checks if the session has exceeded its pre-defined budget cap in under 1ms. If the budget is blown, the proxy intercepts the request and returns a standard 429 error instantly, killing the runaway loop at the edge.

2. Optimistic TCP Pre-warming

To solve the failover latency problem, I implemented a rolling health window for providers using a Redis sliding window algorithm. When the primary provider's latency telemetry spikes above the 95th percentile, the Selixes gateway preemptively initiates a parallel, background TCP connection to the fallback provider. If the primary request ultimately fails, the fallback connection is already TLS-handshaked and ready to pipe the stream.

3. The Google AI Integration (Gemini 3.1 Pro)

The failover architecture is only as good as the fallback engine. I integrated Google Gemini 3.1 Pro via the Google AI ecosystem as the default safety net. To achieve true sub-15ms failover, the fallback model must be globally distributed, incredibly fast, and highly available.

Because Google's infrastructure is arguably the most robust in the world, it serves as the perfect enterprise failover. When OpenAI drops the ball, my gateway pipes the conversational context straight to Gemini 3.1 Pro. The autonomous loop stays alive, the user experience remains uninterrupted, and the engineering team doesn't get woken up at 3 AM.

Prize Categories

Best Use of Google AI
I am submitting this project for the Google AI category. Instead of building a standard chatbot wrapper, I utilized Google AI (specifically Gemini 3.1 Pro) as the backbone for an enterprise-grade failover infrastructure. This project demonstrates how Google's speed and reliability can be leveraged to keep competing systems online.

What's Next & Closing Thoughts

Building Selixes started out of pure frustration with upstream API instability, but it's evolved into a robust piece of infrastructure that I rely on daily. My next goal is to implement predictive latency routing—where the gateway dynamically routes traffic before a primary provider fully spikes, based on global telemetry trends.

If you are an engineer dealing with autonomous agent crashes, rate limits, or spiraling LLM costs, I'd love for you to try out the gateway.

Any feedback, code reviews, or pull requests on the GitHub repository are highly appreciated. Let's make AI infrastructure bulletproof! 🚀

DEV Community: Kunal Kaushik