DEV Community

Cover image for How to Deploy DeepSeek-R1 & Qwen on Azure with Scale-to-Zero ($0 Idle Bill)
Faizal Bagwan
Faizal Bagwan

Posted on

How to Deploy DeepSeek-R1 & Qwen on Azure with Scale-to-Zero ($0 Idle Bill)

How to Deploy DeepSeek-R1 & Qwen on Azure with Scale-to-Zero ($0 Idle Bill)

If you are building AI applications, you quickly hit a fork in the road:

  1. Proprietary APIs (OpenAI/Anthropic): Great to start, but per-token costs ramp up quickly with volume, and enterprise clients often restrict sending sensitive data to external third-party endpoints.
  2. Self-Hosting Open Models (DeepSeek-R1, Qwen 2.5, Llama 3.2): Total data privacy, but traditional cloud deployments on Azure require dedicated GPU virtual machines running 24/7. That can easily cost $600 to $1,800 every month—even when nobody is using the app overnight.

In this guide, I'll walk through how we solved this by building a lightweight FastAPI gateway deployed to Azure Container Apps (ACA) configured with automated scale-to-zero.


The Architecture at a Glance

The gateway sits between client applications and the underlying Hugging Face Transformers model:

  • 100% Drop-in OpenAI Compatibility: Implements /v1/chat/completions and /v1/models. Existing LangChain, LlamaIndex, or frontend chat UIs only need their base_url pointed to the Azure endpoint.
  • Scale-to-Zero Rule: Azure Container Apps HTTP scaler monitors traffic. If 5 minutes pass with zero incoming requests, the container replicas terminate completely. Compute cost while idle is $0.00.
  • On-Demand Spin-up: The moment an HTTP request hits the gateway, Azure provisions the container replica. Cold start takes ~25–35 seconds on CPU/GPU, but once warm, responses return in milliseconds.
  • Real-Time Token Streaming: Implemented Server-Sent Events (SSE) with Hugging Face's TextIteratorStreamer for full typewriter streaming in client UIs.

1. The FastAPI Gateway Core

Here is the simplified implementation of the OpenAI-compatible streaming endpoint:

import json
import asyncio
from fastapi import FastAPI, Depends, HTTPException, Security
from fastapi.responses import StreamingResponse
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from pydantic import BaseModel
from typing import List, Optional

app = FastAPI(title="Private Azure AI Gateway")
security = HTTPBearer()

class ChatMessage(BaseModel):
    role: str
    content: str

class ChatCompletionRequest(BaseModel):
    model: str
    messages: List[ChatMessage]
    stream: Optional[bool] = False
    temperature: Optional[float] = 0.7
    max_tokens: Optional[int] = 512

@app.post("/v1/chat/completions")
async def chat_completions(
    req: ChatCompletionRequest,
    auth: HTTPAuthorizationCredentials = Security(security)
):
    if req.stream:
        async def event_generator():
            # Stream tokens using Server-Sent Events (SSE)
            tokens = ["Hello", " from", " self-hosted", " DeepSeek!"]
            for token in tokens:
                chunk = {
                    "id": "chatcmpl-123",
                    "object": "chat.completion.chunk",
                    "choices": [{"delta": {"content": token}, "finish_reason": None}]
                }
                yield f"data: {json.dumps(chunk)}\n\n"
                await asyncio.sleep(0.05)
            yield "data: [DONE]\n\n"

        return StreamingResponse(event_generator(), media_type="text/event-stream")

    return {
        "id": "chatcmpl-123",
        "object": "chat.completion",
        "choices": [{
            "message": {"role": "assistant", "content": "Sample response"},
            "finish_reason": "stop"
        }]
    }
Enter fullscreen mode Exit fullscreen mode

2. Configuring Azure Container Apps Scale-to-Zero

In the deployment configuration, the secret sauce is setting min_replicas = 0:

{
  "min_replicas": 0,
  "max_replicas": 3,
  "scale_to_zero": true,
  "idle_timeout_seconds": 300
}
Enter fullscreen mode Exit fullscreen mode

When deployed via Azure CLI:

az containerapp create \
  --name deepseek-gateway \
  --resource-group my-ai-rg \
  --environment my-aca-env \
  --image myacr.azurecr.io/deepseek-gateway:latest \
  --target-port 8080 \
  --ingress external \
  --min-replicas 0 \
  --max-replicas 3 \
  --scale-rule-name http-rule \
  --scale-rule-type http \
  --scale-rule-http-concurrency 10
Enter fullscreen mode Exit fullscreen mode

3. Drop-in OpenAI Client Integration

In your existing Python application, you do not need to install custom Hugging Face libraries. Just point the official OpenAI SDK to your Azure endpoint:

from openai import OpenAI

client = OpenAI(
    base_url="https://deepseek-gateway.azurecontainerapps.io/v1",
    api_key="your-secure-bearer-key"
)

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
    messages=[{"role": "user", "content": "Write a Python script to sort a list of dicts."}],
    stream=True
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")
Enter fullscreen mode Exit fullscreen mode

Open Source Repo & Production Boilerplate

The complete open-source repository is available on GitHub:
👉 GitHub: faizalbagwan786/azure-model-deployer

If you want the production-ready boilerplate with pre-configured NVIDIA CUDA 12.1 GPU Dockerfiles, vision support (SmolVLM), test suite, and 1-command Azure CLI automated deployment scripts, check out the Gumroad Release ($10).


If you are working on deploying custom private models on Azure or optimizing cloud AI costs, feel free to connect or ask questions in the comments below!

Top comments (0)