DEV Community: amy5193111-svg

Running One Gateway for Multiple Model Providers: Lessons From Production

amy5193111-svg — Mon, 21 Sep 2026 13:47:55 +0000

Every team eventually ends up with three API keys, four SDKs, and no consistent way to answer "what did we spend on this feature last month?"

A gateway fixes the accounting. It does not automatically fix the architecture. Here is what actually matters.

What a gateway buys you

One schema. Providers disagree on field names, streaming formats, and error shapes. Normalizing that in one place beats normalizing it in five services.

One bill. Per-token attribution by feature, team, or model. This is the underrated win — you cannot optimize what you cannot see.

One failover path. When a provider degrades, rerouting should be a config change, not a deploy.

What it costs you

A gateway is another hop. That means another place for latency and another component that can be down. If your gateway is a single container on a single host, you have built a SPOF in exchange for convenience.

The mitigations that matter: weight-based routing rather than strict primary/backup (a degraded primary should shed load gradually), and health checks that actually exercise the model endpoint rather than just the HTTP port.

The measurement trap

Aggregate spend looks informative and is not. What you want is per-model, per-feature token counts, split by input/output/cache. Without that split, "costs went up" is unactionable.

I wrote up the full set of trade-offs, including the weighting formula and why equal-priority retries behave the way they do, at multi-model API gateway and the LLM API pricing guide.

The lesson: adopt a gateway for the accounting and the schema normalization. Do not adopt it thinking it makes you highly available for free.

Why Developers in China Route Through API Relays (And When It's a Bad Idea)

amy5193111-svg — Mon, 21 Sep 2026 13:42:03 +0000

If you write code outside mainland China and deploy inside it, you already know the shape of this problem: the API works from your laptop and times out from your server.

The actual failure modes

Three things break, in this order:

TLS handshakes to some endpoints get reset intermittently, not consistently. Hard to reproduce, easy to blame on your code.
Latency spikes that look like model slowness but are network-side.
Payment rails — even when the network is fine, billing is where it stops.

What a relay actually does

A relay terminates your request domestically and forwards it over a stable egress path. Done well, it is transparent: same request shape, same response shape, one base URL change.

Done badly, it is a single point of failure with worse uptime than what you started with.

When it is the wrong answer

You need provider-specific beta features that the relay has not implemented
You need a data residency guarantee the relay cannot give you in writing
Your traffic is low enough that retry logic on your side is genuinely simpler

The practical checklist

Before picking one, verify: does it expose the same request schema, does it document its egress path, and can you get per-token cost attribution rather than a monthly total?

I wrote a longer comparison of the trade-offs, including how to test a relay before committing, at API relay station worth it and a setup guide for Claude-compatible clients.

Rule of thumb: if you cannot articulate what breaks without the relay, you are adding a dependency for reasons you cannot measure.

Cutting Claude Code API Costs: Where the Money Actually Goes

amy5193111-svg — Mon, 21 Sep 2026 13:41:56 +0000

Most developers hit the same wall a week into using Claude Code: the token bill grows faster than the output. Here is where the money actually goes, and the config changes that move the needle.

Where the tokens go

Three buckets dominate:

Bucket	Typical share	Why
Cache writes	20-35%	Every new file in context re-billed
Cache reads	30-50%	Cheap per token, but constant
Fresh output	15-25%	The only bucket you actually want

The counterintuitive part: context management beats model choice. Switching from a frontier model to a mid-tier one saves less than fixing context bloat.

Three changes that work

1. Set an explicit context ceiling. Auto-compaction helps, but a hard ceiling forces the agent to summarize early rather than discover the limit mid-task.

2. Separate read-heavy from write-heavy sessions. Exploration and implementation have different cost shapes; mixing them means paying cache-write prices for work that never needed repo context.

3. Watch the base URL. If you are routing through a gateway rather than hitting the provider directly, the per-token price may differ from the published rate card.

Measuring it properly

The only honest way to compare is to log input/output/cache tokens separately per session. Aggregate "total spend" hides the structure you need to fix.

I wrote up the full breakdown with the actual numbers, including the config file changes, here: Claude Code monthly cost breakdown and a base URL configuration walkthrough.

The short version: measure the three buckets separately, cap context, and stop switching models hoping it fixes an architecture problem.

Site Is Not Indexed? Debug It in Search Console, Not With Backlinks

amy5193111-svg — Sat, 19 Sep 2026 07:55:36 +0000

Your New Site Is Not Indexed? Debug It in Search Console, Not With Backlinks

I run a few small sites. Every time one of them goes live, someone eventually asks the same question: "Google still hasn't indexed it - should I buy backlinks?"

Usually, no. Most of the time the problem is visible in Google Search Console within ten minutes, and it is not about authority at all. This is the checklist I actually run, in this order.

1. Confirm the sitemap is a sitemap

Submitting a sitemap in GSC feels like a finish line. It is a starting line.

Check two things before you submit:

The URL returns real XML, not an HTML page. A very common failure: a reverse proxy or SPA fallback serves 200 OK with your homepage HTML at /sitemap.xml. GSC records this as "couldn't fetch" and you get nothing.
robots.txt lists the sitemap and doesn't block the paths inside it.

Quick check from a terminal:

curl -sI https://example.com/sitemap.xml | head -5
curl -s https://example.com/sitemap.xml | head -3

If the first line of the body is <!doctype html>, you have your answer.

2. "Discovered - currently not indexed" is an internal-linking problem

This is the single most common status for new sites, and people misread it as a penalty. It means: Google knows the URL exists, but has no reason to spend crawl budget on it.

Crawl priority is mostly internal PageRank. If your important pages are only reachable from the sitemap - or from an archive page buried five clicks deep - they are effectively orphaned.

The fix is unglamorous and works:

Put your core pages on the homepage, in real <a href> links
Link between related articles with descriptive anchor text
Make sure your homepage isn't linking to URLs that redirect elsewhere

A quick audit: fetch your homepage and count how many links point at pages you actually want indexed.

curl -s https://example.com/ | grep -o 'href="https://example.com/[^"]*"' | sort | uniq -c

3. "Crawled - currently not indexed" is usually a quality or duplication signal

Open URL Inspection -> Test Live URL -> View Crawled Page and look at what Googlebot actually received:

Garbled characters (encoding issues)
A blank shell (content rendered client-side only)
A 404 or a redirect chain
A server so slow that the crawler gave up

And the sneaky one: near-duplicate pages. If you have 50 articles that all say roughly the same thing, Google will index one and skip the rest. In that case consolidation is the answer - merge, then 301 the duplicates into the survivor.

4. Check for redirect chains and stale links

Internal links pointing at redirected URLs burn crawl budget on every pass. Crawl your homepage and archives, and update any link whose target returns a 301.

If you consolidated pages, also remove the retired URLs from your sitemap. Advertising URLs that only redirect is a signal you should not send.

5. No impressions? Check whether the query space exists

If pages are indexed but the Performance report is empty, you probably aren't ranking - or nobody searches the way you wrote. Look at Search Console -> Performance -> Queries. If you see impressions with a position around 10-20, that is a page worth improving rather than replacing. Add depth to what already half-ranks instead of publishing another article.

6. What actually moves the needle when everything technical is clean

Once the technical side is green and nothing is indexed, the remaining bottleneck is crawl demand - which comes from off-site signals and content uniqueness:

Get real links from pages Google already crawls
Publish something that changes (live data, pricing, benchmarks)
Cut the number of near-duplicate URLs on the site instead of adding more

Publishing five more articles a week while twenty URLs sit unindexed makes the ratio worse, not better. That was the mistake I made the first time.

7. IndexNow is free - use it

Google has no public instant-indexing API outside the (quota-limited) "Request Indexing" button, but Bing and Yandex support IndexNow: host a key file, POST your URLs, and you get crawled quickly. It won't help Google directly, but it's ten minutes of work.

I wrote up the longer, ordered version of this checklist - including how to read the "Discovered" drilldown and what to do per status - here: Google Search Console indexing troubleshooting

If you're debugging this while also wiring up an API integration, the config side of things lives at api.yushou.xyz/guide.