DEV Community: skedaddle

How to Self-Host an AI API Gateway With Sub2API

skedaddle — Sun, 26 Jul 2026 14:16:15 +0000

Sub2API can put one Base URL in front of several AI providers and accounts. Your clients use keys that you issue, while the gateway handles authentication, routing, usage accounting, and concurrency limits.

Getting the containers online is the easy part. Once the gateway is running, you also own its database, Redis instance, TLS endpoint, credentials, logs, backups, and upgrades.

This guide builds the smallest useful private setup and shows what to verify before giving anyone else access.

What Sub2API adds

The request path looks like this:

Sub2API currently supports multiple upstream account types, platform API keys, token-level usage accounting, sticky-session scheduling, per-user and per-account concurrency controls, rate limits, composite groups, and an admin dashboard.

This does not make you independent of the upstream provider. If an upstream account is suspended, overloaded, or incompatible with a request, the gateway cannot manufacture capacity or model access.

The project README also warns that distributing quota from AI subscriptions may violate upstream terms of service. Keep the first deployment private, use accounts you are authorized to use, and review the provider terms before sharing or selling access.

Should you self-host it?

Sub2API is useful when you need to:

keep upstream credentials off developer machines;
issue separate, revocable keys to users or projects;
set budgets, rate limits, and concurrency limits centrally;
switch an upstream without reconfiguring every client;
see usage and gateway failures in your own dashboard.

If one person calls one official API, adding a gateway usually makes the system more fragile. Self-hosting starts to pay off when access management and routing are already becoming a coordination problem.

Deploy the smallest useful setup

For the Docker path, you need a Linux server, Docker 20.10 or newer, Docker Compose v2, a domain, and at least one authorized upstream account or API key. PostgreSQL and Redis are included in the official Compose stack.

The project provides a deployment preparation script. Download it first so you can inspect what it will run:

mkdir -p sub2api-deploy
cd sub2api-deploy

curl -fsSLo docker-deploy.sh \
  https://raw.githubusercontent.com/Wei-Shaw/sub2api/main/deploy/docker-deploy.sh

less docker-deploy.sh
chmod +x docker-deploy.sh
./docker-deploy.sh

The script saves the Compose definition as docker-compose.yml, creates .env, generates the PostgreSQL password and application secrets, and prepares local data directories. It prints the generated credentials to the terminal, so do not paste that output into an issue or chat.

Before starting the stack, edit .env. If Nginx or Caddy will run on the same server, bind Sub2API to loopback so port 8080 is not exposed directly:

BIND_HOST=127.0.0.1
SERVER_PORT=8080
ADMIN_EMAIL=you@example.com
ADMIN_PASSWORD=replace-with-a-generated-password
TZ=UTC

Keep the generated JWT_SECRET, TOTP_ENCRYPTION_KEY, and POSTGRES_PASSWORD stable. Changing them later can invalidate sessions, break existing 2FA configuration, or disconnect the database.

Start the services and check their status:

docker compose up -d
docker compose ps
docker compose logs -f sub2api

In another shell, check the local health endpoint:

curl -i http://127.0.0.1:8080/health

Confirm that the Sub2API, PostgreSQL, and Redis containers are healthy. Seeing the login page is not enough to prove that the dependencies are working.

Put HTTPS in front of it

Use a domain and HTTPS for the public Base URL. Sub2API handles long-lived SSE and WebSocket traffic, so the reverse proxy must avoid buffering those responses or closing them too early.

The relevant part of an Nginx configuration is:

# Place these directives in the http block.
underscores_in_headers on;
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    listen 443 ssl http2;
    server_name api.example.com;

    # Add your TLS certificate configuration here.

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_http_version 1.1;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;

        proxy_buffering off;
        proxy_request_buffering off;
        proxy_read_timeout 1800s;
        proxy_send_timeout 1800s;
    }
}

underscores_in_headers on; matters because Nginx otherwise drops headers such as session_id, which Sub2API uses for sticky-session routing.

If you put a CDN in front of the server, restrict origin access and configure trusted proxy ranges instead of accepting forwarded IP headers from any client. The project's edge security guide covers that setup in more detail.

Verify the public route separately:

curl -i https://api.example.com/health

Add one upstream and one client key

Start with one route you can understand end to end. The dashboard labels may change between releases, but the setup is roughly:

Add one upstream account or API key in Account Management.
Create a group and attach the account to it.
Expose one model alias through that group.
Create a non-admin test user with a small balance or quota.
Create a Sub2API key for that user and restrict it to the group.

There are two different credentials in this setup:

Upstream credential  -> Sub2API uses it to call the provider
Sub2API key          -> your client uses it to call the gateway

Give clients the second key. They should not need the upstream credential.

Verify more than a 200 response

For an OpenAI Responses-compatible route, make a small request with the test user's key:

export OPENAI_BASE_URL="https://api.example.com/v1"
export OPENAI_API_KEY="your-sub2api-key"

curl "$OPENAI_BASE_URL/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-alias",
    "input": "Return exactly: gateway-ok"
  }'

Claude Code commonly uses:

export ANTHROPIC_BASE_URL="https://api.example.com"
export ANTHROPIC_AUTH_TOKEN="your-sub2api-key"
claude

For Codex, start with a clean shell:

export OPENAI_BASE_URL="https://api.example.com/v1"
export OPENAI_API_KEY="your-sub2api-key"
codex

After the first request, open the dashboard and check the record. Confirm that the expected user, key, group, account, model, token count, and charge were used.

Then test the behavior your real client depends on:

streaming and long responses;
tool calls;
model alias mapping;
usage and cache accounting;
concurrency limits;
upstream error propagation.

Also try an invalid key, a user with no remaining budget, and an unavailable upstream. These tests tell you whether access stops cleanly and whether a failure can silently reach the wrong model.

Keep it maintainable

Before using the gateway for regular work:

Pin the Sub2API image tag you tested instead of upgrading latest unattended.
Keep the admin account separate from API use and enable 2FA.
Rotate a user's Sub2API key when that user or project no longer needs access.
Monitor /health, container restarts, disk space, upstream errors, latency, and certificate expiry.
Back up .env, the Compose file, application data, and PostgreSQL.

Do not copy a live PostgreSQL data directory and assume it is consistent. Use pg_dump, a database-aware storage snapshot, or stop the stack before taking a filesystem archive. Encrypt backups and test a restore on another machine.

Self-host or let someone else run it

Sub2API is a reasonable choice when you want to own the gateway policy and are willing to operate the supporting infrastructure.

If you would rather not maintain PostgreSQL, Redis, TLS, backups, security, and upstream accounts, do not self-host merely because the code is available. You can compare existing AI API relay services, including services built with Sub2API, on CCNavX.

Sources

I Built an AI API Directory Because OpenAI-Compatible Is Not Enough

skedaddle — Thu, 14 May 2026 03:19:13 +0000

Developers do not need another “best AI API provider” list.

Most lists collapse into the same problem: a few affiliate links, some vague pricing claims, and no clear way to verify whether a provider actually supports the model, Base URL pattern, payment method, or billing behavior a real project needs.

So I built a more boring thing: an AI API directory.

Not a leaderboard.

Not a recommendation engine.

A structured directory of observed facts.

The idea is simple: before wiring a third-party AI API provider into Codex, Cursor, Claude Code, or your own app, you should be able to compare a few concrete fields:

supported providers and models
OpenAI-compatible or Anthropic-compatible API behavior
custom Base URL support
pricing notes
payment methods
invoice support
referral or recharge rules
public verification sources
traffic and domain signals when available

The important shift is this:

“Compatible with OpenAI” describes an API shape. It does not describe trust, uptime, pricing, ownership, or support quality.

That distinction matters.

A provider can expose an OpenAI-style endpoint and still differ wildly in model names, streaming behavior, rate limits, credit expiration, recharge rules, error formats, and whether pricing is even visible before login.

For small experiments, that might be fine.

For developer tools, internal agents, or production workflows, those details are the product.

Why a Directory Instead of a Ranking?

Ranking sounds useful, but it hides too much judgment.

If I say “Provider A is better than Provider B,” that may be true for one person who needs Claude access, Alipay recharge, and a cheap test balance. It may be wrong for someone who needs invoices, stable OpenAI-compatible endpoints, or multi-model routing.

So the directory takes a different approach:

Collect providers.
Normalize the fields.
Show what can be verified.
Let people filter by their actual use case.

This is closer to documentation than marketing.

The page currently groups providers around search intents such as:

AI API directory
OpenAI API proxy
Claude API proxy
cheap OpenAI API
OpenRouter alternatives

Each intent becomes a landing page, but the underlying data stays structured.

That means the same provider can appear in different contexts without duplicating the source of truth.

The Engineering Lesson

The surprisingly hard part was not building the UI.

The hard part was deciding what counts as a useful fact.

“Supports GPT-4” is less useful than:

what exact model names are exposed?
is the endpoint OpenAI-compatible?
does it require a custom Base URL?
does streaming work?
is pricing public or login-gated?
what payment methods are listed?
when was the information last checked?

Once those fields exist, the frontend becomes straightforward: search, filter, compare, and link to deeper provider profiles.

The stack is intentionally simple:

Astro
TypeScript
Content Collections
React only for interactive islands
structured JSON content for provider profiles
static output deployed to Cloudflare Workers

This works well because the data is mostly content, but the user experience needs interactivity.

Astro handles the content pages.

React handles the searchable directory UI.

The content collection schema keeps the provider data from drifting into chaos.

The Bigger Point

AI infrastructure is moving fast, but developer decisions still need boring verification.

A nice homepage is not enough.

A low price is not enough.

“OpenAI-compatible” is not enough.

The real question is:

Can this provider be understood, compared, tested, and replaced without turning the whole project into glue code?

That is what the directory is trying to answer.

Here is the page: https://ccnavx.com/directory/ai-api-directory/

Next, I am thinking about adding more explicit test result fields: streaming behavior, error format, model alias mapping, and minimum viable curl examples for each provider.

Because at the end of the day, the best AI API provider is not the one with the loudest claim.

It is the one whose behavior can be verified before it touches production.