DEV Community

Sonam
Sonam

Posted on

Switch AI Models at Runtime on Telnyx Edge Compute

#ai #edge #typescript #telnyx

Most AI examples hardcode the model name.

That is fine until you actually want to compare models.

If every model change requires a code edit and redeploy, experimenting gets annoying fast. The multi-model-inference-switcher example turns model choice into runtime configuration instead.

Code: https://github.com/team-telnyx/telnyx-code-examples/tree/main/multi-model-inference-switcher

What it builds

This is a TypeScript app running on Telnyx Edge Compute with the Agent SDK.

It gives you:

  • an admin UI
  • a model dropdown
  • a chat panel
  • durable conversation history
  • usage stats by model
  • a KV-backed active-model flag

The active model is read from Telnyx KV Storage every time /chat is called. When you switch the model from the UI or API, the next message uses the new model immediately.

No redeploy.

The flow 

GET /
  -> admin UI

POST /model
  -> validate model
  -> write active-model to KV

POST /chat
  -> read active-model from KV
  -> SwitcherAgent.process(text, model)
  -> Telnyx AI Inference
  -> return reply + model
Enter fullscreen mode Exit fullscreen mode

The sample includes these models:

  • moonshotai/Kimi-K2.6
  • zai-org/GLM-5.2
  • meta-llama/Llama-3.3-70B-Instruct

Why this is useful

Model choice is product behavior.

Changing the model can affect:

  • latency
  • cost
  • output quality
  • reasoning depth
  • tone
  • reliability

So it helps to make the active model observable and switchable without mixing that decision into application deploys.

API examples

Switch the active model:

curl -X POST https://multi-model-inference-switcher-<id>.telnyxcompute.com/model \
  -H "Content-Type: application/json" \
  -d '{"model":"zai-org/GLM-5.2"}'
Enter fullscreen mode Exit fullscreen mode

Send a chat message:

curl -X POST https://multi-model-inference-switcher-<id>.telnyxcompute.com/chat \
  -H "Content-Type: application/json" \
  -d '{"text":"Explain feature flags for AI models."}'
Enter fullscreen mode Exit fullscreen mode

Example response:

{
  "reply": "Feature flags let you change behavior at runtime...",
  "model": "zai-org/GLM-5.2"
}
Enter fullscreen mode Exit fullscreen mode

Inspect history and usage:

curl https://multi-model-inference-switcher-<id>.telnyxcompute.com/history
Enter fullscreen mode Exit fullscreen mode

Agent SDK primitives used

The SwitcherAgent uses:

  • durable message history for chat context
  • actor state for total requests and model usage
  • the Telnyx binding for zero-credential AI Inference
  • KV Storage for the global model flag

The inference call looks like:

this.env.TELNYX.ai.openai.chat.createCompletion({
  model,
  messages,
  max_tokens: 2000,
  temperature: 0.7,
});
Enter fullscreen mode Exit fullscreen mode

The key part is that model comes from KV, not a hardcoded constant.

Run it 

git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/multi-model-inference-switcher
npm install
Enter fullscreen mode Exit fullscreen mode

Create and seed KV:

telnyx-edge storage kv create --name "switcher-flag"
telnyx-edge storage kv key put <kv-id> active-model moonshotai/Kimi-K2.6
Enter fullscreen mode Exit fullscreen mode

Set your namespace ID in telnyx.toml, add your secret, and deploy:

telnyx-edge secrets add TELNYX_API_KEY <YOUR_API_KEY>
telnyx-edge ship
Enter fullscreen mode Exit fullscreen mode

Production notes

Before exposing this publicly, add:

  • auth on the admin UI and /model
  • audit logging for model changes
  • a reviewed model allowlist
  • fallback behavior if the active model fails
  • latency and cost tracking per model
  • scoped flags by environment, tenant, or cohort

The small idea here is powerful: keep your app deployed, but make model selection something you can operate.

Resources:

Top comments (0)