A decision proxy in front of SearXNG: speculative execution, per-engine circuit breakers, and keyless fallback

TL;DR

searxng-gateway is a Go HTTP proxy that sits in front of SearXNG and adds multi-provider fallback (Brave, Exa, Jina, Tavily), a per-engine circuit breaker, an in-memory LRU cache, Prometheus metrics, and a zero-key mode. It preserves SearXNG's JSON response shape, so clients do not change. The interesting parts are the execution model, the breaker semantics, and the trade-offs — including a couple that are easy to get wrong.

What it adds, in one list:

• Speculative execution — SearXNG and a configurable number of premium providers start together; premium calls are serial within that pass.
• Bounded fallback — if the merged result count is below a threshold, remaining providers are tried round-robin until the time budget expires.
• Per-engine circuit breakers — one 4xx-class error opens that engine for five minutes, then a single probe decides recovery.
• Observability — Prometheus metrics plus an importable Grafana dashboard and example alert rules.
• Keyless mode — runs on SearXNG's free engines with no API keys at all.

The rest of this post covers why the pieces are shaped this way, and where the design still costs you.

Why a proxy in front of SearXNG

Self-hosted search looks simple from the outside: run SearXNG, ask it for JSON, get results. In practice the free engines behind it do not fail in a coordinated way. One engine starts returning 403, another gets rate-limited, a third serves a captcha challenge, and a fourth simply gets slower until it times out. SearXNG does report which engines were unresponsive, but it does not make a per-request decision about what to do next. The aggregate answer degrades quietly: fewer results, more latency, and no clear owner for the fix.

searxng-gateway is a small Go service that sits in front of a SearXNG instance and makes those decisions. It is open source under MIT, currently at v2.6.3, maintained at github.com/Ghilteras/searxng-gateway. It was originally derived from sx and adds an HTTP server, a per-engine circuit breaker, Prometheus metrics, an in-memory LRU cache, and Docker packaging.

A SearXNG-shape-compatible layer

The gateway is not a replacement for SearXNG; it is a layer in front of it. Clients keep talking to a single endpoint and keep receiving the same JSON shape SearXNG produces, with fields such as title, url, content, engine, and engines. Premium provider responses are normalised into that shape at merge time, so a caller does not need to know whether a result came from SearXNG or from the Brave API. The client never talks to SearXNG directly.

Three endpoints are exposed: GET /search?q=<query>&format=json for queries, GET /healthz for liveness, and GET /metrics for Prometheus exposition, with the metrics path configurable through METRICS_PATH.

The execution model, precisely

This is the part most write-ups get wrong, so it is worth stating carefully. The pipeline runs in stages:

1. The query is normalised (lowercased, whitespace collapsed) and checked against the LRU cache. A cache hit returns immediately without calling any backend.
2. SearXNG's cooldown state is checked. If SearXNG has hit its consecutive-failure threshold, it is skipped for the whole request.
3. SearXNG starts concurrently with the premium pass. The gateway picks T1_PREMIUM_COUNT providers by atomic round-robin and invokes them serially within that pass. Premium providers are not called in parallel with each other; the concurrency is between SearXNG and the premium pass as a whole.
4. Results from SearXNG and the premium pass are merged and deduplicated by URL.
5. If the merged count is below SUFFICIENT_MIN_RESULTS, a bounded fallback loop tries the remaining providers by round-robin, one at a time, until the threshold is met, the providers are exhausted, or FALLBACK_TIMEOUT_SECONDS expires.

The outcome of each request is recorded under a fixed set of labels on searxng_gateway_requests_total: cache_hit (served from cache, no backend called), searxng_ok (only SearXNG contributed results), premium_ok (only premium providers contributed, meaning SearXNG was skipped, errored, or returned nothing), searxng_plus_premium_ok (both contributed), and fallback_fail (everything failed with no results). There is also a timeout label, incremented when SearXNG returns a deadline-exceeded error; unlike the others it is recorded in addition to the final outcome rather than replacing it, so a single request can increment both.

One subtlety worth knowing before you build dashboards on these labels: "contributed" means a provider's results actually survived URL dedup into the merged set. The rule is symmetric across providers, but it is not order-free. If two providers return an identical URL set, whichever one is merged first supplies the URLs and the other is recorded as contributing nothing.

Per-engine circuit breakers

Each premium provider and each SearXNG engine gets its own circuit breaker, built on sony/gobreaker. The trip rule is deliberately aggressive: a single 4xx-class client error opens the circuit immediately. Here a "4xx-class" error is broader than the HTTP status code. It includes 403, 429, access-denied, "too many requests", "blocked by", and captcha messages. The reasoning is that these signals usually mean the server is telling you to stop, so retrying makes things worse.

Once open, the engine is excluded from subsequent requests. After a five-minute timeout the circuit goes half-open and sends a single probe; success closes it and increments the recovery counter, while failure reopens it for another five minutes. SearXNG itself is handled separately with a binary cooldown counter rather than gobreaker: after SEARXNG_FAIL_THRESHOLD consecutive failures it is skipped entirely for SEARXNG_FAIL_COOLDOWN_SECONDS.

SearXNG calls also get retries with exponential backoff, up to three attempts. Notably, that retry path makes no 4xx/5xx distinction: all error classes are retried, and the per-attempt metrics (searxng_gateway_retry_attempts_total, searxng_gateway_retry_exhausted_total) exist so the retry path stays visible even when the first attempt succeeds.

Cache, metrics, and a Grafana dashboard

The cache is an in-memory LRU with 1000 entries and a one-hour TTL by default, both configurable. It only helps repeated queries, and it is per-process, so it does not survive a restart and is not shared across replicas.

All metrics are prefixed searxng_gateway_. Beyond request outcomes, the interesting ones are per-engine result counts (searxng_gateway_engine_results_total), unresponsive reasons reported by SearXNG (searxng_gateway_engine_unresponsive_total), last-seen engine status (searxng_gateway_engine_status), retry counters, cache size, and the circuit-breaker family: state (0 closed, 1 half-open, 2 open), trips by reason, rejections, and recovery events. There is also a set of request-window quota gauges. For Brave specifically these are searxng_gateway_brave_rate_limit_remaining, searxng_gateway_brave_rate_limit_limit, and searxng_gateway_brave_rate_limit_reset_seconds; they are parsed from the X-RateLimit-Remaining, X-RateLimit-Limit, and X-RateLimit-Reset response headers, so they describe the request window the API reports, not an account credit or dollar balance.

A reference Grafana dashboard ships in the repository at examples/grafana/searxng-gateway-dashboard.json, with panels for circuit-breaker state per engine, cumulative trips coloured by reason, recoveries, cache hit rate, and cache size. The README also includes example alert rules, for instance firing when a circuit stays open and when fallback API usage spikes. One caveat on the bundled screenshots: they date from July 2026 and predate an outcome-label rename, so a few legends show older label names. The dashboard JSON is the current source of truth.

Keyless mode

The gateway runs with no API keys at all. In keyless mode it uses SearXNG's free engines, including Bing, Wikipedia, Wikidata, GitHub, StackOverflow, ArXiv, PyPI, Docker Hub, Mwmbl, and Marginalia, and still applies the circuit breaker, retry, and caching. What you lose is Google results, since Serper needs a key, and the Brave fallback. Adding keys later is just a matter of setting environment variables.

Quickstart

The repository ships a reference stack that runs SearXNG and the gateway together. The gateway image tag is pinned, so upgrade it deliberately rather than tracking latest:

services:
  searxng:
    image: searxng/searxng:latest
    container_name: searxng
    restart: unless-stopped
    volumes:
      - ./examples/searxng/settings.example.yml:/etc/searxng/settings.yml:ro
      - ./examples/searxng-engines/serper.py:/usr/local/searxng/searx/engines/serper.py:ro
      - ./examples/searxng-engines/mojeek_api.py:/usr/local/searxng/searx/engines/mojeek_api.py:ro
    environment:
      - SERPER_API_KEY=${SERPER_API_KEY:-}
    ports:
      - "127.0.0.1:8081:8080"

  searxng-gateway:
    image: ghcr.io/ghilteras/searxng-gateway:v2.6.3
    container_name: searxng-gateway
    restart: unless-stopped
    ports:
      - "8080:8080"
    environment:
      - SEARXNG_BACKEND_URL=http://searxng:8080

Then query it:

curl 'http://localhost:8080/search?q=hello+world&format=json'
curl 'http://localhost:8080/metrics'

Note that the SearXNG image above tracks an upstream tag and is not digest-pinned, so a quickstart run is not byte-for-byte reproducible — the gateway image is the pinned half.

Configuration

Everything is configured through environment variables. The most consequential ones:

• SEARXNG_BACKEND_URL: where the SearXNG instance lives.
• FALLBACK_PROVIDERS: comma-separated premium provider names (default brave).
• T1_PREMIUM_COUNT: how many providers run in the hot path alongside SearXNG; zero means none.
• SUFFICIENT_MIN_RESULTS: the merged-result target that stops the fallback loop.
• FALLBACK_TIMEOUT_SECONDS: the overall budget for the speculative pass plus the fallback loop.
• SEARXNG_FAIL_THRESHOLD and SEARXNG_FAIL_COOLDOWN_SECONDS: the SearXNG cooldown.
• CACHE_SIZE and CACHE_TTL_SECONDS: cache tuning.
• LOG_LEVEL and METRICS_PATH: operational knobs.

Premium providers need their own keys (BRAVE_API_KEY, EXA_API_KEY, JINA_API_KEY, TAVILY_API_KEY), and each consumes that provider's quota.

Trade-offs and limitations

This is a decision layer, not a free lunch, and some of the costs are structural.

• Premium providers require their own API keys and consume real quota. The fallback billing alert exists precisely because a loop over providers can spend money.
• The premium pass is serial, and the fallback loop is serial too. Concurrency exists only between SearXNG and the whole premium pass, so enabling more providers in the hot path adds latency rather than hiding it.
• The circuit-breaker thresholds are heuristics. Tripping on the first 4xx is aggressive and will occasionally sideline an engine that returned a transient error, and the five-minute cooldown is a fixed guess. These values are tunable, not auto-tuned.
• The timeout outcome is not mutually exclusive with the terminal outcome labels, which can be surprising when building dashboards.
• Attribution between SearXNG and premium contributors is not order-free: when both return an identical URL set, the label reflects whichever was merged first.
• The cache is in-memory and per-process. It does not persist and does not coordinate across instances.
• Keyless mode is genuinely usable but drops Google and Brave, which changes result quality noticeably for general web queries.

Nothing here solves engine-level blocking at the source. If every free engine is blocked from your IP, the gateway can only fail over to a provider you hold a key for, or fail cleanly.

AI-assistance disclosure

Most of this project's implementation was written by AI, working from the author's specifications. The author reviewed the code, ran the tests, and deployed and operated the gateway; the design decisions, the trade-offs above, and the choice of what to ship are the author's. The project has been AI-heavy in its implementation history, and this article is part of that same practice. No benchmarks or performance numbers are claimed beyond what the repository's code and documentation state.