DEV Community: Angelo Pantano

A decision proxy in front of SearXNG: speculative execution, per-engine circuit breakers, and keyless fallback

Angelo Pantano — Fri, 18 Sep 2026 19:03:50 +0000

TL;DR

searxng-gateway is a Go HTTP proxy that sits in front of SearXNG and adds multi-provider fallback (Brave, Exa, Jina, Tavily), a per-engine circuit breaker, an in-memory LRU cache, Prometheus metrics, and a zero-key mode. It preserves SearXNG's JSON response shape, so clients do not change. The interesting parts are the execution model, the breaker semantics, and the trade-offs — including a couple that are easy to get wrong.

What it adds, in one list:

Speculative execution — SearXNG and a configurable number of premium providers start together; premium calls are serial within that pass.
Bounded fallback — if the merged result count is below a threshold, remaining providers are tried round-robin until the time budget expires.
Per-engine circuit breakers — one 4xx-class error opens that engine for five minutes, then a single probe decides recovery.
Observability — Prometheus metrics plus an importable Grafana dashboard and example alert rules.
Keyless mode — runs on SearXNG's free engines with no API keys at all.

The rest of this post covers why the pieces are shaped this way, and where the design still costs you.

Why a proxy in front of SearXNG

Self-hosted search looks simple from the outside: run SearXNG, ask it for JSON, get results. In practice the free engines behind it do not fail in a coordinated way. One engine starts returning 403, another gets rate-limited, a third serves a captcha challenge, and a fourth simply gets slower until it times out. SearXNG does report which engines were unresponsive, but it does not make a per-request decision about what to do next. The aggregate answer degrades quietly: fewer results, more latency, and no clear owner for the fix.

searxng-gateway is a small Go service that sits in front of a SearXNG instance and makes those decisions. It is open source under MIT, currently at v2.6.3, maintained at github.com/Ghilteras/searxng-gateway. It was originally derived from sx and adds an HTTP server, a per-engine circuit breaker, Prometheus metrics, an in-memory LRU cache, and Docker packaging.

A SearXNG-shape-compatible layer

The gateway is not a replacement for SearXNG; it is a layer in front of it. Clients keep talking to a single endpoint and keep receiving the same JSON shape SearXNG produces, with fields such as title, url, content, engine, and engines. Premium provider responses are normalised into that shape at merge time, so a caller does not need to know whether a result came from SearXNG or from the Brave API. The client never talks to SearXNG directly.

Three endpoints are exposed: GET /search?q=<query>&format=json for queries, GET /healthz for liveness, and GET /metrics for Prometheus exposition, with the metrics path configurable through METRICS_PATH.

The execution model, precisely

This is the part most write-ups get wrong, so it is worth stating carefully. The pipeline runs in stages:

The query is normalised (lowercased, whitespace collapsed) and checked against the LRU cache. A cache hit returns immediately without calling any backend.
SearXNG's cooldown state is checked. If SearXNG has hit its consecutive-failure threshold, it is skipped for the whole request.
SearXNG starts concurrently with the premium pass. The gateway picks T1_PREMIUM_COUNT providers by atomic round-robin and invokes them serially within that pass. Premium providers are not called in parallel with each other; the concurrency is between SearXNG and the premium pass as a whole.
Results from SearXNG and the premium pass are merged and deduplicated by URL.
If the merged count is below SUFFICIENT_MIN_RESULTS, a bounded fallback loop tries the remaining providers by round-robin, one at a time, until the threshold is met, the providers are exhausted, or FALLBACK_TIMEOUT_SECONDS expires.

The outcome of each request is recorded under a fixed set of labels on searxng_gateway_requests_total: cache_hit (served from cache, no backend called), searxng_ok (only SearXNG contributed results), premium_ok (only premium providers contributed, meaning SearXNG was skipped, errored, or returned nothing), searxng_plus_premium_ok (both contributed), and fallback_fail (everything failed with no results). There is also a timeout label, incremented when SearXNG returns a deadline-exceeded error; unlike the others it is recorded in addition to the final outcome rather than replacing it, so a single request can increment both.

One subtlety worth knowing before you build dashboards on these labels: "contributed" means a provider's results actually survived URL dedup into the merged set. The rule is symmetric across providers, but it is not order-free. If two providers return an identical URL set, whichever one is merged first supplies the URLs and the other is recorded as contributing nothing.

Per-engine circuit breakers

Each premium provider and each SearXNG engine gets its own circuit breaker, built on sony/gobreaker. The trip rule is deliberately aggressive: a single 4xx-class client error opens the circuit immediately. Here a "4xx-class" error is broader than the HTTP status code. It includes 403, 429, access-denied, "too many requests", "blocked by", and captcha messages. The reasoning is that these signals usually mean the server is telling you to stop, so retrying makes things worse.

Once open, the engine is excluded from subsequent requests. After a five-minute timeout the circuit goes half-open and sends a single probe; success closes it and increments the recovery counter, while failure reopens it for another five minutes. SearXNG itself is handled separately with a binary cooldown counter rather than gobreaker: after SEARXNG_FAIL_THRESHOLD consecutive failures it is skipped entirely for SEARXNG_FAIL_COOLDOWN_SECONDS.

SearXNG calls also get retries with exponential backoff, up to three attempts. Notably, that retry path makes no 4xx/5xx distinction: all error classes are retried, and the per-attempt metrics (searxng_gateway_retry_attempts_total, searxng_gateway_retry_exhausted_total) exist so the retry path stays visible even when the first attempt succeeds.

Cache, metrics, and a Grafana dashboard

The cache is an in-memory LRU with 1000 entries and a one-hour TTL by default, both configurable. It only helps repeated queries, and it is per-process, so it does not survive a restart and is not shared across replicas.

All metrics are prefixed searxng_gateway_. Beyond request outcomes, the interesting ones are per-engine result counts (searxng_gateway_engine_results_total), unresponsive reasons reported by SearXNG (searxng_gateway_engine_unresponsive_total), last-seen engine status (searxng_gateway_engine_status), retry counters, cache size, and the circuit-breaker family: state (0 closed, 1 half-open, 2 open), trips by reason, rejections, and recovery events. There is also a set of request-window quota gauges. For Brave specifically these are searxng_gateway_brave_rate_limit_remaining, searxng_gateway_brave_rate_limit_limit, and searxng_gateway_brave_rate_limit_reset_seconds; they are parsed from the X-RateLimit-Remaining, X-RateLimit-Limit, and X-RateLimit-Reset response headers, so they describe the request window the API reports, not an account credit or dollar balance.

A reference Grafana dashboard ships in the repository at examples/grafana/searxng-gateway-dashboard.json, with panels for circuit-breaker state per engine, cumulative trips coloured by reason, recoveries, cache hit rate, and cache size. The README also includes example alert rules, for instance firing when a circuit stays open and when fallback API usage spikes. One caveat on the bundled screenshots: they date from July 2026 and predate an outcome-label rename, so a few legends show older label names. The dashboard JSON is the current source of truth.

Keyless mode

The gateway runs with no API keys at all. In keyless mode it uses SearXNG's free engines, including Bing, Wikipedia, Wikidata, GitHub, StackOverflow, ArXiv, PyPI, Docker Hub, Mwmbl, and Marginalia, and still applies the circuit breaker, retry, and caching. What you lose is Google results, since Serper needs a key, and the Brave fallback. Adding keys later is just a matter of setting environment variables.

Quickstart

The repository ships a reference stack that runs SearXNG and the gateway together. The gateway image tag is pinned, so upgrade it deliberately rather than tracking latest:

services:
  searxng:
    image: searxng/searxng:latest
    container_name: searxng
    restart: unless-stopped
    volumes:
      - ./examples/searxng/settings.example.yml:/etc/searxng/settings.yml:ro
      - ./examples/searxng-engines/serper.py:/usr/local/searxng/searx/engines/serper.py:ro
      - ./examples/searxng-engines/mojeek_api.py:/usr/local/searxng/searx/engines/mojeek_api.py:ro
    environment:
      - SERPER_API_KEY=${SERPER_API_KEY:-}
    ports:
      - "127.0.0.1:8081:8080"

  searxng-gateway:
    image: ghcr.io/ghilteras/searxng-gateway:v2.6.3
    container_name: searxng-gateway
    restart: unless-stopped
    ports:
      - "8080:8080"
    environment:
      - SEARXNG_BACKEND_URL=http://searxng:8080

Then query it:

curl 'http://localhost:8080/search?q=hello+world&format=json'
curl 'http://localhost:8080/metrics'

Note that the SearXNG image above tracks an upstream tag and is not digest-pinned, so a quickstart run is not byte-for-byte reproducible — the gateway image is the pinned half.

Configuration

Everything is configured through environment variables. The most consequential ones:

SEARXNG_BACKEND_URL: where the SearXNG instance lives.
FALLBACK_PROVIDERS: comma-separated premium provider names (default brave).
T1_PREMIUM_COUNT: how many providers run in the hot path alongside SearXNG; zero means none.
SUFFICIENT_MIN_RESULTS: the merged-result target that stops the fallback loop.
FALLBACK_TIMEOUT_SECONDS: the overall budget for the speculative pass plus the fallback loop.
SEARXNG_FAIL_THRESHOLD and SEARXNG_FAIL_COOLDOWN_SECONDS: the SearXNG cooldown.
CACHE_SIZE and CACHE_TTL_SECONDS: cache tuning.
LOG_LEVEL and METRICS_PATH: operational knobs.

Premium providers need their own keys (BRAVE_API_KEY, EXA_API_KEY, JINA_API_KEY, TAVILY_API_KEY), and each consumes that provider's quota.

Trade-offs and limitations

This is a decision layer, not a free lunch, and some of the costs are structural.

Premium providers require their own API keys and consume real quota. The fallback billing alert exists precisely because a loop over providers can spend money.
The premium pass is serial, and the fallback loop is serial too. Concurrency exists only between SearXNG and the whole premium pass, so enabling more providers in the hot path adds latency rather than hiding it.
The circuit-breaker thresholds are heuristics. Tripping on the first 4xx is aggressive and will occasionally sideline an engine that returned a transient error, and the five-minute cooldown is a fixed guess. These values are tunable, not auto-tuned.
The timeout outcome is not mutually exclusive with the terminal outcome labels, which can be surprising when building dashboards.
Attribution between SearXNG and premium contributors is not order-free: when both return an identical URL set, the label reflects whichever was merged first.
The cache is in-memory and per-process. It does not persist and does not coordinate across instances.
Keyless mode is genuinely usable but drops Google and Brave, which changes result quality noticeably for general web queries.

Nothing here solves engine-level blocking at the source. If every free engine is blocked from your IP, the gateway can only fail over to a provider you hold a key for, or fail cleanly.

AI-assistance disclosure

Most of this project's implementation was written by AI, working from the author's specifications. The author reviewed the code, ran the tests, and deployed and operated the gateway; the design decisions, the trade-offs above, and the choice of what to ship are the author's. The project has been AI-heavy in its implementation history, and this article is part of that same practice. No benchmarks or performance numbers are claimed beyond what the repository's code and documentation state.

Giving an OpenCode Coding Agent Persistent, Editable Memory

Angelo Pantano — Thu, 17 Sep 2026 22:24:01 +0000

Giving an OpenCode Coding Agent Persistent, Editable Memory

AI coding agents are good at using the context currently in front of them. They are much less reliable at carrying useful context from one session to the next.

Project conventions, decisions made during a debugging session, and preferences that were obvious yesterday often need to be re-explained after a restart or after context compaction.

I maintain @ghilteras/opencode-agent-memory, an experimental plugin for OpenCode that explores an alternative: treating agent memory as editable, scoped Markdown state on disk rather than as an opaque remote service.

The problem with always-in-context instructions

AGENTS.md and custom instruction files are a good start. I rely on them heavily. But they are a single flat document with no notion of scope, no size enforcement, and no dedicated operations for the agent to maintain it.

What I actually wanted was structured state:

some facts belong to me globally, across every project
some facts belong only to one codebase
each piece should have a description telling the agent how to use it
each piece should have a size limit so it cannot silently grow without bound
the agent should be able to read and rewrite that state with explicit tools

Memory blocks

The plugin gives the agent three tools: memory_list, memory_set, and memory_replace. Blocks are plain Markdown files with YAML frontmatter.

Global blocks live in ~/.config/opencode/memory/*.md and are shared across projects. Project blocks live in .opencode/memory/*.md and are shared across sessions in that codebase, and are gitignored automatically.

Each block has:

Field	Type	Default	Purpose
`label`	string	filename	unique identifier
`description`	string	generic fallback	tells the agent how to use the block
`limit`	integer	5000	maximum characters
`read_only`	boolean	false	prevents agent edits

The description field carries more weight than it looks like it should. Without it, the agent gets a generic fallback and does not know when the block is relevant. This mirrors the emphasis Letta puts on describing memory blocks well.

Three blocks are seeded on first run: persona and human globally, project for the current codebase.

The journal

Memory blocks are curated state. Some things are not: observations, dead ends, discoveries, decisions, and the reasoning behind them.

For that the plugin adds an optional append-only journal with journal_write, journal_search, and journal_read. Entries are Markdown files with YAML frontmatter stored under ~/.config/opencode/journal/, and each entry records which project, model, provider, agent, and session it came from.

The journal is deliberately opt-in. Enabling it is a line in ~/.config/opencode/agent-memory.json:

{
  "journal": {
    "enabled": true
  }
}

Append-only plus retrieval is what makes it useful. I do not want the agent rewriting history; I want it to be able to find what happened last month.

Local semantic search

journal_search uses local embeddings rather than a hosted API. Entries are embedded with paraphrase-multilingual-MiniLM-L12-v2 (384 dimensions, multilingual) through Transformers.js, and the model is cached locally. Journal content does not leave the machine for search.

Two implementation details were worth the effort:

Embedding files are versioned. Each .embedding sidecar stores { v, model, dimension, vector }. If the stored dimension does not match the current model, the entry degrades to text matching instead of failing the search. Legacy bare-array embeddings from earlier versions remain readable.

The in-memory index is fingerprinted. journal_search keeps an index per store instance and re-reads an entry only when the entry .md or its .embedding sidecar changed (mtime plus size). Regenerating or deleting a sidecar is picked up on the next search without a restart, and the embedding model is warmed up in the background at plugin init so the first search after a restart does not pay the cold model-load cost.

There is also a deliberate retrieval floor: since v0.4.2, a query whose text matches an entry title (in either direction, case-insensitive) is guaranteed a high score, so title-based pointers stay retrievable even for entries with long bodies.

Operational trade-offs

I want to be clear about the failure modes, because memory systems fail quietly.

Stale memory is worse than no memory. A block that says something true six weeks ago will be confidently applied today. The description field and size limits help the agent judge relevance, but they do not make the content true. Having the state as editable Markdown on disk means you can inspect and fix it directly; that is a feature, not a workaround.

Automatic persistence is not the same as truth. The journal records what the agent observed, including wrong conclusions. Treat it as a log, not a knowledge base.

The journal is opt-in because silent collection is worse than explicit collection.

Installation

Requires OpenCode v1.0.115 or later.

{
  "plugin": ["@ghilteras/opencode-agent-memory@0.4.3"]
}

OpenCode fetches unpinned plugins from npm on each startup; pinned versions are cached and need a manual bump. Restart OpenCode after changing plugin configuration; editing the config file alone is not enough to load a new plugin version.

A cacheDir key at the top level of agent-memory.json relocates the Transformers.js model cache if you prefer not to use the default Hugging Face cache location.

What I would like feedback on

What belongs in global memory versus project memory in your workflow?
How should stale memories be surfaced rather than silently applied?
How useful is semantic retrieval over the journal compared with plain text search, in practice?
What should survive context compaction, and what should be allowed to decay?

Status

This is experimental software, MIT licensed. It is maintained at https://github.com/Ghilteras/opencode-agent-memory and published as @ghilteras/opencode-agent-memory. It is not built by or affiliated with the OpenCode team.

If you try it, issues and concrete workflow reports are welcome.