Hindsight for Hermes Agent: Self-Hosted Memory with Local LLMs

A self-hosted AI agent needs more than a long context window. It needs a way to retain useful facts, retrieve earlier decisions, and revise what it thinks it knows as projects evolve.

Hermes Agent ships with built-in memory files, but an external provider can provide much richer retrieval across sessions. Hindsight is one of the more interesting options in that space. This guide installs it for Hermes with no cloud inference, first as an all-in-one Docker container and then against an existing PostgreSQL server with separate local model endpoints.

The guide covers deployment, choosing and wiring local embedding, reranking, and generative models, controlling what auto-retain writes into the bank, connecting Hermes, and the operational layer: backup, migration, and day-to-day bank hygiene. The examples use a deliberate port scheme — 8080 for the main llama.cpp server, 8081 for embeddings, 8082 for reranking, 11434 for Ollama — so change them to the ports you already use and keep the small servers off the main model's port.

What Hindsight Does

Hindsight is a persistent agent memory engine with three principal operations: retain, recall, and reflect. Retain extracts and stores information; recall combines semantic, keyword, graph, and temporal retrieval; reflect synthesizes an answer from accumulated memories. Background consolidation can turn overlapping observations into a more useful knowledge layer.

It is not just an embeddings database. A vector store can locate similar passages, but a memory system also has to preserve chronology, project boundaries, contradictory evidence, and changes in decisions. Those failure modes are explored in self-reinforcing memory loops.

For the wider market, see agent memory providers compared. Hermes' built-in MEMORY.md, USER.md, and memory modes are covered in the Hermes Agent memory system guide, so this article assumes you know how the built-in layer works.

Why Move Beyond Mnemosyne

Mnemosyne for Hermes offers appealing local-first ideas, including SQLite storage and hybrid retrieval. In practice, installation constraints, version compatibility, write-approval behaviour, and rapid changes around its 4.0 prereleases can make it difficult to operate predictably. Those are operational concerns, not evidence that Mnemosyne's memory model is fundamentally flawed.

Hindsight is not dependency-free either: its Docker image and local models can consume substantial disk space. Running its API as an independent service, with inference elsewhere, keeps upgrades and troubleshooting isolated from the agent itself.

Choose a Deployment

Component	A: All-in-one container	B: Existing PostgreSQL server
Hindsight API and UI	Docker container	Docker container
PostgreSQL	Embedded pg0 in container	Existing PostgreSQL machine
Generative LLM	External llama.cpp	External llama.cpp
Embeddings	Default local or external	External Ollama or llama.cpp
Reranking	Default local or external	External rerank server
Best for	Quick evaluation	Long-running, isolated installation

Here, "all-in-one" means the Hindsight API, UI, and database are in one container; the inference LLM is still hosted elsewhere. This is intentional: Docker's official Hindsight image does not bundle a ready-to-use local generative llama.cpp server.

flowchart LR
    H[Hermes Agent] -->|Memory API| A[Hindsight API]
    A --> P[(PostgreSQL)]
    A --> E[Embedding endpoint]
    A --> R[Reranking endpoint]
    A --> L[Generative LLM endpoint]

Use LAN-reachable addresses, not localhost, for services running on a different host. Restrict these ports to trusted clients; the example dummy keys are compatibility placeholders, not authentication.

Option A: All-in-One Hindsight Container

This is the simplest way to evaluate Hindsight without filling the Hermes VM with Python, Torch, and local model dependencies. The container runs Hindsight and embedded PostgreSQL, while your main llama.cpp LLM handles memory extraction and reflection.

Two image tags matter here. ghcr.io/vectorize-io/hindsight:latest is the large image — roughly 9 GB on AMD64 — because it bundles a BGE embedder, a MiniLM reranker, and embedded PostgreSQL. latest-slim is roughly 500 MB and expects those two models to live elsewhere, which is what you want once you point Hindsight at the embedding and reranking setup in Serve and Wire Local Models. Once the health check passes, pin a specific tag: a moving tag is a poor match for a database you intend to keep.

Create a directory and a Compose file on the Hindsight host:

mkdir -p ~/services/hindsight
cd ~/services/hindsight
cat > compose.yaml <<'YAML'
services:
  hindsight:
    image: ghcr.io/vectorize-io/hindsight:latest
    container_name: hindsight
    restart: unless-stopped
    shm_size: 1gb
    extra_hosts:
      - "host.docker.internal:host-gateway"
    ports:
      - "8888:8888"  # Memory API
      - "9999:9999"  # Control-plane UI
    environment:
      HINDSIGHT_API_WORKER_ID: hindsight-standalone
      HINDSIGHT_API_LLM_PROVIDER: openai
      HINDSIGHT_API_LLM_BASE_URL: http://WORKSTATION_IP:8080/v1
      HINDSIGHT_API_LLM_MODEL: LocalMegaLLM
      HINDSIGHT_API_LLM_API_KEY: dummy
      HINDSIGHT_API_LLM_MAX_CONCURRENT: "2"
    volumes:
      - hindsight-pg0:/home/hindsight/.pg0
volumes:
  hindsight-pg0:
YAML

Replace WORKSTATION_IP with the host running llama.cpp. On Linux, host.docker.internal resolves only if the container declares the host-gateway mapping shown above; without it, a base URL of http://host.docker.internal:8080/v1 points nowhere. The 1 GB shared-memory reservation is particularly useful for embedded PostgreSQL indexing; Docker's small default /dev/shm can cause failures. A named volume preserves the database when the container is recreated, and a bind mount alternative has to be owned by uid 1000, the user inside the image.

HINDSIGHT_API_WORKER_ID should stay stable across recreations — Docker's default is the container hostname, and a task that was mid-flight when a container died stays stuck under the old worker id.

docker compose up -d
docker compose logs -f --tail=100
curl -f http://localhost:8888/health

If your installed Hindsight version exposes health at a different path, inspect http://localhost:8888/docs and the container logs. The control plane is normally available at http://HINDSIGHT_HOST:9999. Once the basic deployment works, you can also point its embedding and reranking providers at the configuration in Serve and Wire Local Models rather than loading the default models inside the container.

Do not expose ports 8888 or 9999 directly to the internet. Use a private network or an authenticated reverse proxy. Embedded PostgreSQL is convenient for testing, but an externally managed database is easier to back up and maintain.

Option B: Hindsight Against an Existing PostgreSQL Server

For a long-running installation, I prefer keeping the Hindsight API, database, and inference endpoints separate. The Hindsight VM can remain small because it runs orchestration and HTTP services rather than heavy models.

flowchart LR
    H[Hermes VM] -->|port 8888| A[Hindsight VM]
    A -->|port 5432| P[(PostgreSQL host)]
    A -->|port 8080| L[Generative LLM]
    A -->|port 8081 or 11434| E[Embedding endpoint]
    A -->|port 8082| R[Reranking endpoint]

Prepare PostgreSQL and pgvector

Hindsight needs PostgreSQL 14 or newer and a vector extension in the hindsight database. The extension is per database: installing the package on the server does nothing until CREATE EXTENSION runs there. On a Debian or Ubuntu PostgreSQL server, install the pgvector package matching the actual PostgreSQL major version; for example, on PostgreSQL 16:

sudo apt update
sudo apt install postgresql-16-pgvector

The package name and availability depend on your configured apt repositories. If it is not available, follow the pgvector installation instructions, or install a compatible package from the PostgreSQL PGDG repository. Do not blindly change the PostgreSQL major version on a shared server.

Create a dedicated role and database as a PostgreSQL administrator:

CREATE ROLE hindsight LOGIN PASSWORD 'REPLACE_WITH_LONG_PASSWORD';
CREATE DATABASE hindsight OWNER hindsight;

Connect to the new database and enable the extension:

sudo -u postgres psql -d hindsight -c \
  'CREATE EXTENSION IF NOT EXISTS vector;'
sudo -u postgres psql -d hindsight -c \
  "SELECT extname, extversion FROM pg_extension WHERE extname = 'vector';"

The extension normally needs elevated privileges to install; the application role does not need to be a superuser. Hindsight uses pgvector by default and wants 0.8.0 or newer for iterative ANN scans; older servers reject that setting. HINDSIGHT_API_VECTOR_EXTENSION can alternatively select pgvectorscale (a DiskANN option Hindsight documents for very large banks) or vchord, both of which need extra packaging on this exact PostgreSQL build. Switching algorithms after memories exist stops startup and asks for a migration, so choose before the first retain. For a personal Hermes bank, pgvector is the one to install and then stop.

Allow only the Hindsight host through pg_hba.conf, for example:

host  hindsight  hindsight  192.168.1.50/32  scram-sha-256

Set PostgreSQL listen_addresses appropriately and use TLS or a trusted private network for cross-host connections. Back up this database independently of the Hindsight Docker container. If the server is shared with other applications, keep Hindsight in its own database, or set HINDSIGHT_API_DATABASE_SCHEMA to a schema you own; the default schema is public.

Run Hindsight Against PostgreSQL

Create compose.yaml on the Hindsight VM. This version deliberately does not start PostgreSQL locally:

services:
  hindsight:
    image: ghcr.io/vectorize-io/hindsight:latest-slim
    restart: unless-stopped
    ports:
      - "8888:8888"
      - "9999:9999"
    env_file: .env
    environment:
      HINDSIGHT_API_LLM_PROVIDER: openai
      HINDSIGHT_API_LLM_BASE_URL: http://WORKSTATION_IP:8080/v1
      HINDSIGHT_API_LLM_MODEL: LocalMegaLLM
      HINDSIGHT_API_LLM_API_KEY: dummy
      HINDSIGHT_API_LLM_MAX_CONCURRENT: "2"
      HINDSIGHT_API_DB_POOL_MIN_SIZE: "2"
      HINDSIGHT_API_DB_POOL_MAX_SIZE: "10"
      HINDSIGHT_API_WORKER_ID: hindsight-standalone

Store the external database URL in a file called .env:

HINDSIGHT_API_DATABASE_URL=postgresql://hindsight:YOUR_PASSWORD@POSTGRES_IP:5432/hindsight

chmod 600 .env
docker compose up -d
docker compose logs -f --tail=100

If your password contains @, :, /, or other reserved URL characters, URL-encode it before putting it in the connection string. Test that the Hindsight VM can reach PostgreSQL port 5432, and that pg_hba.conf allows the Hindsight host — a server bound only to 127.0.0.1 refuses the connection even when the URL looks right. Allow the first startup to run database migrations; migrations run on startup unless you set HINDSIGHT_API_RUN_MIGRATIONS_ON_STARTUP=false. With the API and database up, continue with Choose Models for Hindsight and Serve and Wire Local Models before connecting Hermes.

Choose Models for Hindsight

Deploy puts the API and database on the network. The next decision is which models fill the three inference roles: embeddings for recall candidates, a reranker to re-order them, and a generative LLM for retain, reflect, and consolidation. A practical default for a CPU Hindsight host whose GPU is already busy with conversation is in-process Qwen/Qwen3-Embedding-0.6B, in-process BAAI/bge-reranker-v2-m3, and a shared llama.cpp chat model for generative work.

Embedding models

Hindsight reads the embedding dimension on startup and sizes the bank to match, so the model you pick at first retain is the model the bank keeps until you rebuild it. Dimensionality and instruction behaviour matter more than leaderboard rank: a stronger 8B embedder is not worth a migration if the bank is already a 384- or 1024-dimensional one. Rows below are sorted from lightest to heaviest; RAM figures are approximate loaded footprint on CPU (FP16 / typical local load), not GGUF file size.

Model	Params	Dim	Context	Loaded	Runs on	Notes
BAAI/bge-small-en-v1.5	33M	384	512	~100–130 MB	CPU	Hindsight default; ships in hindsight:latest. Fastest local path. Score distribution still calibrates many recall_min_scores defaults after you leave it.
BAAI/bge-base-en-v1.5	109M	768	512	~400 MB	CPU	Mid-tier English BGE; locks a 768-dim bank.
sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2	~118M	384	512	~200 MB	CPU	Hindsight-documented multilingual MiniLM; 50+ languages at the same 384-dim bank width as BGE-small.
intfloat/multilingual-e5-small	~118M	384	512	~200 MB	CPU	Default for the onnx provider; light multilingual CPU path when you have an ONNX export.
nomic-ai/nomic-embed-text-v1.5	~137M	768	8k	~500 MB	CPU	Common Ollama embedder; longer context than classic BGE, still English-first.
google/embeddinggemma-2	270M	768	8k	~540 MB	CPU	EmbeddingGemma 2 text-only (disable vision/audio towers). 100+ languages + code; MRL truncates to 512/256/128; instruction prefixes for search vs document. Full multimodal load is 740M.
BAAI/bge-m3	568M	1024	8k	~1.1 GB	CPU or GPU	Strong multilingual dual-encoder; same width as Qwen3-0.6B, so a same-dim swap still needs a full re-embed.
Qwen/Qwen3-Embedding-0.6B	0.6B	1024	32k	~1.2 GB	CPU	This guide's default. Instruction-aware (search instruction on the query side); comfortable on CPU. Local provider auto-applies the instruction; OpenAI-compatible servers need HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX.
Qwen/Qwen3-Embedding-4B	4B	2560	32k	~8 GB FP16 / ~2.5 GB quant	GPU	Qwen3 quality tier; locks a 2560-dim bank. Not viable when the conversational GPU is already saturated.

How the model is run is the other half of the choice. Hindsight's embedding providers, from simplest to most moving parts:

Provider	How it runs	Best when	Notes
local	In-process SentenceTransformers; no sidecar	Single Hindsight host, CPU embedder	Default path. Set HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL. No llama.cpp, Ollama, or TEI.
onnx	In-process ONNX Runtime	Lighter CPU than Torch; you already have an ONNX export	Needs hindsight-api-slim[local-onnx] and HINDSIGHT_API_EMBEDDINGS_ONNX_*.
openai / tei	External OpenAI-compatible or HF TEI endpoint	Embeddings off the API process; shared GPU/CPU host	What Serve and Wire Local Models configures for Ollama / llama.cpp / TEI. Only worth the sidecar if you want isolation or a different machine.

Qwen3-Embedding-0.6B does not need an external embeddings service. On a CPU Hindsight host it loads inside the API process with the local provider — Hugging Face downloads the weights on first start, and the provider applies the model's search instruction itself:

# Compose environment (or equivalent .env keys)
HINDSIGHT_API_EMBEDDINGS_PROVIDER: local
HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL: Qwen/Qwen3-Embedding-0.6B

Use an external OpenAI-compatible endpoint only when you want the embedder on another machine or off the API process. Ollama is one such host:

# Embeddings: Qwen3-Embedding-0.6B on Ollama
HINDSIGHT_API_EMBEDDINGS_PROVIDER: openai
HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL: http://192.168.1.123:11434/v1
HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL: qwen3-embedding:0.6b-q8_0
HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY: dummy

Replace the host, port, and tag with whatever ollama list shows on your inference machine.

HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX depends on the provider:

Path	HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX	HINDSIGHT_API_EMBEDDINGS_PASSAGE_PREFIX
local (Qwen/Qwen3-Embedding-0.6B)	Leave unset / empty — the in-process provider applies Qwen's search instruction itself	Leave unset / empty
External Ollama / llama.cpp / OpenAI-compatible	Official Qwen3 retrieval instruction (real newline before Query:)	Leave empty — stored memories stay raw text
Server already wraps inputs in Qwen's template	Leave both empty — otherwise the instruction is applied twice	Leave empty

For the external path, set:

# Real newline before Query:. Bash $'...' is the reliable way to pass it.
HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX=$'Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:'
HINDSIGHT_API_EMBEDDINGS_PASSAGE_PREFIX=

In Compose YAML the same string needs an explicit \n (or a literal multiline value). Set the prefixes before the first retain: changing them later means old vectors and new queries no longer sit in the same space.

Two properties decide more than the leaderboard position. Instruction sensitivity: Qwen3 embeddings apply a search instruction on the query side; without the matching prefix on an OpenAI-compatible server, the stored side and the query side stop sitting in the same space. Dimension lock: because the width is read once at startup, moving from 0.6B to 4B later is a bank migration, not a config change — and even a same-dimension swap (BGE-M3 ↔ Qwen3-0.6B, or EmbeddingGemma 2 ↔ nomic at 768) changes the vector space, so bank vectors are not recomputed automatically. Plan a fresh bank or a manual re-embed/reindex; see Export, import, and migrating the memory bank. After leaving BGE-small, probe recall_min_scores against the new score distribution — thresholds calibrated on BGE-small can be too tight for Qwen3-0.6B and starve the reranker of candidates. For benchmarks and GGUF/Ollama background on the Qwen3 family, see Qwen3 Embedding and Reranker models on Ollama.

Reranking models

Embeddings recall a candidate set; the reranker scores each query/passage pair and re-orders it before Hindsight injects it. With the local provider, Hindsight loads a SentenceTransformers cross-encoder (or a generative reranker where supported). With an external Cohere-compatible endpoint, it posts the full Cohere-style body to HINDSIGHT_API_RERANKER_COHERE_BASE_URL. The reranker is optional, but once the embedder is fixed it is usually the largest recall-quality lever — and, on CPU, often most of the recall wall-clock. Rows are sorted from lightest to heaviest.

Model	Params	Type	Context	Runs on	Notes
cross-encoder/ms-marco-MiniLM-L-6-v2	23M	Cross-encoder	128	CPU	Hindsight default; ships in hindsight:latest. Sub-second for ~20 candidates. Keep as named fallback if a heavier BGE model is too slow.
cross-encoder/ms-marco-MiniLM-L-12-v2	~33M	Cross-encoder	512	CPU	Same family, higher accuracy; still English-leaning MS MARCO.
cross-encoder/mmarco-mMiniLMv2-L12-H384-v1	~33M	Cross-encoder	512	CPU	Hindsight-documented multilingual MiniLM reranker.
BAAI/bge-reranker-base	278M	Cross-encoder	512	CPU	Moderate upgrade over MiniLM; en/zh. Discussed often; skipped when v2-m3 already fits the latency budget.
BAAI/bge-reranker-v2-m3	568M	Cross-encoder	8k	CPU or GPU	Practical multilingual default on CPU. On a ~500-memory bank, ~2.6–2.8 s per recall at a 20-candidate floor (~140 ms/candidate; ~93–99% of recall time). Candidate caps 20/40/80; lowering the floor from 20 → 10 roughly halves rerank cost.
Qwen/Qwen3-Reranker-0.6B	0.6B	Generative (yes-token)	32k	GPU preferred	Instruction-aware, long context. Not a drop-in cross-encoder: needs recent SentenceTransformers generative-reranker support, and llama.cpp /rerank scoring has known wrong-score bugs (hindsight-benchmarks#11). Validate before adopting.
BAAI/bge-reranker-v2-gemma	2.5B	Cross-encoder	8k	GPU preferred	Strong multilingual quality; excessive for a few-hundred-memory personal bank when the GPU is already full.
Qwen/Qwen3-Reranker-4B	4B	Generative	32k	GPU	Higher-quality Qwen candidate; not viable without dedicated VRAM.
Qwen/Qwen3-Reranker-8B	8B	Generative	32k	GPU	Family top tier; same generative-scoring caveats, larger still.

Cloud API rerankers (Cohere, ZeroEntropy, Alibaba qwen3-rerank, TypeSafe, and similar) work in Hindsight but send memory text off-box. For a local-only cross-domain layout they are out of scope.

Two operational constraints dominate the pick. VRAM: if the conversational model already saturates the GPU (for example a Qwen3.8-27B Q5 on a 16 GB card with no headroom), every "GPU preferred" row is off the table and the live choice collapses to MiniLM or BGE-v2-m3 on CPU. Candidate count: HINDSIGHT_API_RERANKER_MAX_CANDIDATES (30 in the external-endpoint example below) bounds how many passages the model re-scores; each candidate is a full forward pass, so cut the floor before you jump a model tier. For Qwen3 via llama.cpp, the GGUF must keep the ranking head — an old convert_hf_to_gguf.py that drops it scores every passage near zero — and even a correct conversion still needs a known-good/known-bad passage test until the generative scoring path is confirmed.

Generative LLM tiers

Embedding and reranking models are not substitutes for the generative LLM used to extract and reason over memories. Hindsight supports a default LLM and operation-specific overrides, which can target the same llama.cpp server or independent endpoints.

Hindsight operation	Purpose	Practical model tier	Qwen examples
Retain	Extract facts, entities, decisions, and chronology	4B minimum for tests; 9B preferred	Qwen3.5-4B / Qwen3.5-9B
Recall	Semantic, keyword, graph and temporal retrieval	No generative LLM required	Qwen3-Embedding-0.6B + reranker
Reflect	Answer questions by synthesizing retrieved memories	9B to 27B+	Qwen3.5-9B / Qwen3.8-27B
Consolidation	Reconcile and compress related facts	9B preferred	Qwen3.5-9B / Qwen3.8-27B
Mental model refresh	Update longer-lived synthesized understanding	9B to 35B	Qwen3.5-9B / Qwen3.6-35B-A3B

These are operational recommendations, not official Hindsight minimum parameter counts. A small model may extract a port number correctly but miss that an earlier architectural decision has been superseded. Structured-output reliability and faithful handling of negation matter more than nominal parameter count alone. Hindsight's own temperature defaults are worth leaving alone — 0.1 for retain, 0.0 for consolidation, 0.9 for reflect — and for retain and consolidation, the non-thinking alias is worth serving if your llama.cpp build has one, because the small Qwen3.5 checkpoints spend many tokens in thinking mode.

For a deployment with an existing Qwen3.8-27B llama.cpp endpoint, start by sharing it across generative Hindsight tasks with low concurrency:

HINDSIGHT_API_LLM_PROVIDER=openai
HINDSIGHT_API_LLM_BASE_URL=http://WORKSTATION_IP:8080/v1
HINDSIGHT_API_LLM_MODEL=LocalMegaLLM
HINDSIGHT_API_LLM_API_KEY=dummy
HINDSIGHT_API_LLM_MAX_CONCURRENT=2
HINDSIGHT_API_RETAIN_LLM_MAX_CONCURRENT=1
HINDSIGHT_API_CONSOLIDATION_LLM_MAX_CONCURRENT=1

Later, route routine extraction to an independent 9B model and reserve the 27B for difficult reflection:

HINDSIGHT_API_RETAIN_LLM_PROVIDER=openai
HINDSIGHT_API_RETAIN_LLM_BASE_URL=http://MEMORY_LLM_HOST:8083/v1
HINDSIGHT_API_RETAIN_LLM_MODEL=MemoryLLM
HINDSIGHT_API_RETAIN_LLM_API_KEY=dummy

HINDSIGHT_API_REFLECT_LLM_PROVIDER=openai
HINDSIGHT_API_REFLECT_LLM_BASE_URL=http://WORKSTATION_IP:8080/v1
HINDSIGHT_API_REFLECT_LLM_MODEL=LocalMegaLLM
HINDSIGHT_API_REFLECT_LLM_API_KEY=dummy

The smaller 4B model is economical but needs testing; 9B is a sensible quality/performance compromise. If retain and consolidation share a small model, lower HINDSIGHT_API_CONSOLIDATION_LLM_BATCH_SIZE to 2 — the default batch of 8 is a long prompt for a small context window. Qwen3.6-35B-A3B is attractive for a separate well-provisioned inference server (it is a mixture-of-experts model with about 3 billion active parameters, so budget it as a small active model), while a 27B dense model is usually unnecessary for every routine retain event. Limiting background concurrency protects the Hermes conversational model from memory-maintenance bursts.

Serve and Wire Local Models

If the in-process local embedding and reranker settings in Choose Models are enough, you can skip the external Ollama and llama.cpp subsections and jump to wiring — or leave Compose on the local defaults shown there. External endpoints matter when the embedder or reranker should live on another host.

External embeddings: Ollama or llama.cpp

External embeddings are for when you want Qwen3-Embedding-0.6B (or another model) on a separate host. Two practical options are Ollama, which handles model acquisition and serving with little configuration, and llama.cpp, which gives precise control over GGUF quantization, CPU threads, device placement, and batching. Model background and benchmarks for both Qwen3 models are in Qwen3 Embedding and Reranker models on Ollama.

Qwen3-Embedding-0.6B returns 1024-dimensional vectors, and Hindsight reads that width on startup and sizes the database to match. A bank created with the bundled 384-dimensional BGE model cannot take this model — start from an empty database or volume, and decide the embedding model before the first retain. An embeddings provider switch later requires re-embedding stored records even if both models use the same dimensionality.

Option 1: Ollama for Qwen3-Embedding-0.6B

Install Ollama on the inference host using its official Linux installation documentation, then start the embedding model:

ollama pull qwen3-embedding:0.6b
ollama serve

For remote access, configure Ollama's OLLAMA_HOST=0.0.0.0:11434 in its service environment, restart the service, and firewall it to the Hindsight VM — the procedure is covered in Ollama remote access. A daemon bound only to 127.0.0.1 is invisible to the Hindsight container. First check the model name with ollama list, and test the standard embedding endpoint:

curl http://EMBED_HOST:11434/api/embed \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3-embedding:0.6b","input":"A test memory"}'

Hindsight's OpenAI embeddings provider instead calls Ollama's OpenAI-compatible /v1/embeddings endpoint. Verify that endpoint works with your Ollama version before configuring Hindsight. Ollama is straightforward for embeddings; do not assume its API offers a compatible reranking endpoint. For reranking, prefer the in-process local BGE path below, or a separate llama.cpp / TEI endpoint after you have validated scores.

Option 2: llama.cpp for Qwen3-Embedding-0.6B

Download a verified Qwen3-Embedding-0.6B GGUF conversion. Use the GGUF filename you actually downloaded:

#!/bin/sh
# start-embedding.sh - CPU-only embedding server
set -eu
cd "$HOME/prj"
exec ./llama.cpp/llama-server \
  --host 0.0.0.0 --port 8081 \
  --model "$HOME/ggufs/Qwen3-Embedding-0.6B-Q8_0.gguf" \
  --alias qwen3-embedding-0.6b \
  --embedding --pooling last \
  --ctx-size 4096 --parallel 1 \
  --threads 4 --threads-batch 4 \
  --batch-size 512 --ubatch-size 256 \
  --n-gpu-layers 0

Qwen3 embeddings use last-token pooling, unlike the CLS pooling used by some BGE models. A CPU-only llama.cpp build is preferable if you want absolutely no CUDA initialization on a workstation whose GPU is occupied by a large conversational model. Do not enable --reranking on this embedding server: that combination is a known way to get a vector of zeros.

curl -s http://EMBED_HOST:8081/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3-embedding-0.6b","input":"A test memory"}' \
  | jq '.data[0].embedding | length'

A full-dimension Qwen3-Embedding-0.6B output should contain 1024 floats. Check that the GGUF conversion and your Hindsight query/document instruction handling are compatible; instruction-sensitive embedding models can lose accuracy when prompted incorrectly.

Reranker: local BGE or Qwen3 via llama.cpp

Reranking is different from embeddings: the model scores each query/document pair and returns a relevance score. On a CPU-only Hindsight host — or whenever the conversational GPU has no VRAM headroom — the practical path is Hindsight's local provider with BAAI/bge-reranker-v2-m3 (or the lighter MiniLM default). No sidecar is required:

HINDSIGHT_API_RERANKER_PROVIDER=local
HINDSIGHT_API_RERANKER_LOCAL_MODEL=BAAI/bge-reranker-v2-m3
HINDSIGHT_API_RERANKER_MAX_CANDIDATES=20

Qwen3-Reranker-0.6B is a generative yes-token scorer, not a classic cross-encoder. It can be served through llama.cpp with a GGUF conversion designed for ranking, but treat that path as experimental until you confirm scores on a known-good/known-bad pair: an old convert_hf_to_gguf.py that drops the ranking head scores every passage near zero, and llama.cpp's native /rerank path has known wrong-score reports for this family.

#!/bin/sh
# start-reranker.sh - GPU or CPU as selected by --n-gpu-layers
set -eu
cd "$HOME/prj"
exec ./llama.cpp/llama-server \
  --host 0.0.0.0 --port 8082 \
  --model "$HOME/ggufs/Qwen3-Reranker-0.6B-Q8_0.gguf" \
  --alias qwen3-reranker-0.6b \
  --embedding --reranking --pooling rank \
  --ctx-size 4096 --parallel 1 \
  --batch-size 512 --ubatch-size 256 \
  --n-gpu-layers 99

Set --n-gpu-layers 0 to run it on CPU. The GPU version needs additional VRAM for compute buffers and context, not just model weights; avoid assuming that the file size is the total GPU allocation.

curl -s http://RERANK_HOST:8082/v1/rerank \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3-reranker-0.6b", "query":"PostgreSQL storage", "documents":["Hindsight stores memories in PostgreSQL.","A recipe for bread."], "top_n":2}' \
  | jq

The PostgreSQL document should rank above the unrelated document; all-zero relevance scores mean the GGUF is the wrong conversion, and recall will look random even when the facts are stored. Check the exact results[].index and results[].relevance_score shape: Hindsight expects a Cohere-compatible rerank response, and llama.cpp compatibility can vary across builds.

Wire endpoints into Hindsight

Add the following environment settings to the Hindsight Compose service, or store them in its .env file. Pick one embeddings path — in-process local, Ollama, or llama.cpp — never two at once.

# Option A: in-process Qwen3 on the Hindsight host (CPU; no sidecar)
HINDSIGHT_API_EMBEDDINGS_PROVIDER=local
HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=Qwen/Qwen3-Embedding-0.6B

# Option B: llama.cpp embedding service
# HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
# HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=http://EMBED_HOST:8081/v1
# HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=qwen3-embedding-0.6b
# HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=dummy
# HINDSIGHT_API_EMBEDDINGS_OPENAI_BATCH_SIZE=8
# HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX=$'Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:'
# HINDSIGHT_API_EMBEDDINGS_PASSAGE_PREFIX=

# Option C: Ollama on another host
# HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
# HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=http://192.168.1.40:11434/v1
# HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=qwen3-embedding:0.6b-q8_0
# HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=dummy
# HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX=$'Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:'
# HINDSIGHT_API_EMBEDDINGS_PASSAGE_PREFIX=

# Option: local BGE reranker (no sidecar; preferred on CPU)
HINDSIGHT_API_RERANKER_PROVIDER=local
HINDSIGHT_API_RERANKER_LOCAL_MODEL=BAAI/bge-reranker-v2-m3
HINDSIGHT_API_RERANKER_MAX_CANDIDATES=20

# Option: llama.cpp Cohere-compatible rerank endpoint (validate Qwen3 scores first)
# HINDSIGHT_API_RERANKER_PROVIDER=cohere
# HINDSIGHT_API_RERANKER_COHERE_BASE_URL=http://RERANK_HOST:8082/v1/rerank
# HINDSIGHT_API_RERANKER_COHERE_MODEL=qwen3-reranker-0.6b
# HINDSIGHT_API_RERANKER_COHERE_API_KEY=dummy
# HINDSIGHT_API_RERANKER_MAX_CANDIDATES=30

The variable names are provider-specific, and a near-miss is silent: HINDSIGHT_API_EMBEDDINGS_BASE_URL is the wrong key, and Hindsight quietly aims at OpenAI's own embedding endpoint instead of your local server. Use HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL. With the Cohere-compatible reranker path, Hindsight posts the full Cohere-style body to HINDSIGHT_API_RERANKER_COHERE_BASE_URL; if the container log shows a 404 for /v1/rerank/rerank, your build appends /rerank on its own and the value should stop at /v1.

For Options B and C, use the HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX value from Embedding models (Option A leaves it unset). Test query-to-passage retrieval on your actual memories after the first retain.

Control What Gets Remembered

With the stack running, the next failure mode is usually the write path: Hermes defaults to auto_retain: true, so every conversation turn can be extracted into the bank. Incorrect guesses, discarded plans, and exploratory statements become searchable facts, and later recall treats them as evidence — the loop described in self-reinforcing memory loops. Capture policy for Hindsight among other providers is summarized in agent memory providers compared.

Capture modes that exist today

Mode	Hermes / Hindsight settings	What you get
Continuous	auto_retain: true, retain_every_n_turns: 1	Every turn is retained and extracted
Batched auto	auto_retain: true, raise retain_every_n_turns	Fewer, larger retain payloads — still automatic, not a review queue
Explicit-only	auto_retain: false + tool hindsight_retain	Storage only when the agent calls the tool (for example after you say "remember this")
Session synthesis	End-of-session hindsight_retain of a summary, or bank reflect / consolidation	A curated write if you or the agent discipline the call — not a built-in flush-for-approval

Setting auto_retain to false disables automatic storage. It does not add a human approval dialog before each write. The agent can still call hindsight_retain whenever its tools allow; treat explicit retain as deliberate capture, not a pending queue.

The approval-queue gap

A common wish is a pending queue of candidate memories that you review and approve before they enter the bank. Hindsight under Hermes does not provide that contract today. Mnemosyne for Hermes implements provider-specific staged writes in some versions; that path is not a shared Hermes standard and does not apply to the Hindsight plugin. Until an upstream approval flow exists, the workable pattern is auto_retain: false, deliberate hindsight_retain, and inspection or deletion in the Hindsight control-plane UI before the next session relies on a bad fact. There is no missing write_approval key for Hindsight to hunt for in config.json.

Bank steering (missions, labels, directives, mental models)

People often call the next layer "finetuning." In Hindsight it is mostly bank configuration, not Hermes config.json alone: missions and extraction settings steer what retain keeps; labels structure entities; directives and mental models shape later synthesis.

Control	Where it lives	How it helps
Retain mission (retain_mission / Hermes bank_retain_mission)	Bank config	Plain-language "pay attention to X, ignore Y" injected into extraction
Extraction mode (concise, verbose, custom, …)	Bank config	concise (default) keeps long-term-worthy facts; custom replaces built-in rules via retain_custom_instructions
Entity labels	Bank config	Controlled key:value vocabulary at retain time → graph links and optional tags; improves retrieval structure, does not by itself reject wrong statements
Directives ("hard rules")	Bank directives API	Explicit instructions enforced during reflect (tag-scoped); constrain synthesis, not a write-approval gate
Mental models	Bank mental-model API	Standing answers to recurring questions, refreshed via reflect from matching memories; derived, so bad source facts still poison them
Reflect / observations mission	Bank config	Steers synthesis and consolidation, not the raw retain firehose

Missions, concise extraction, and labels reduce noise and improve retrieval. They do not replace turning off auto-retain when accuracy matters more than completeness. Prefer a short retain mission that names durable project decisions and ignores chat logistics; add entity labels only when you have a real taxonomy; add mental models for two or three recurring questions after the bank already holds clean facts.

Recommended hygiene profile

A concrete starting point for a personal Hermes bank:

{
  "mode": "local_external",
  "api_url": "http://HINDSIGHT_VM_IP:8888",
  "bank_id": "hermes",
  "memory_mode": "hybrid",
  "auto_recall": true,
  "recall_sync": false,
  "auto_retain": false,
  "retain_async": true,
  "recall_prefetch_method": "recall"
}

On the bank, set a retain mission and leave extraction in concise mode (via the control plane or Banks API). After an explicit retain, open the UI, confirm the observation, and delete mistakes before consolidation or mental-model refresh runs. Leave recall_prefetch_method at recall — reflect spends a chat-model call before every turn and injects synthesis rather than underlying facts.

Connect Hermes Agent to Hindsight

On the Hermes VM, install the catalog plugin, then run the setup wizard. Select Hindsight and the local_external mode; the plugin does not need to start embedded PostgreSQL or run the Hindsight model stack locally, and local_embedded would start a second Hindsight daemon beside the container you just ran.

hermes plugins install hindsight
hermes memory setup hindsight

There will be Interesting set of options of agent role using hindsight

hermes memory status

The external Hindsight connection lives in ~/.hermes/hindsight/config.json. An illustrative configuration that keeps continuous capture (Hermes defaults) is:

{
  "mode": "local_external",
  "api_url": "http://HINDSIGHT_VM_IP:8888",
  "bank_id": "hermes",
  "memory_mode": "hybrid",
  "auto_recall": true,
  "recall_sync": false,
  "auto_retain": true,
  "retain_async": true,
  "retain_every_n_turns": 1
}

For explicit-only retain and bank steering, use the hygiene profile in Control What Gets Remembered instead of leaving auto_retain on. recall_sync: false prioritizes interactive responsiveness but may inject recalled context on the following turn. Use synchronous recall when the current request must incorporate the latest memories before the model starts answering. retain_async: true avoids blocking ordinary responses while memories are processed.

The plugin's recall default is observations only; raw world and experience facts stay in the bank and out of the prompt until consolidation has written an observation, or until you widen recall_types to observation,world,experience. Widening it for the first week and watching the control plane, then narrowing, is a reasonable order.

Built-in Hermes memory stays active next to the provider; keeping MEMORY.md and USER.md on while you learn what Hindsight adds is fine, and the memory-system article is the place to decide whether the flat files should stay.

Restart the gateway if it is already running:

hermes gateway restart
hermes memory status

The status should identify Hindsight as the active provider. For a wider command reference, including Hermes' TUI and plugin commands, see the Hermes Agent CLI cheatsheet.

Validate Memory, Not Just Installation

Ask Hermes to remember a disposable fact about a test project, start another session, and ask for that fact again. Retention is asynchronous by default and extraction waits on the chat model, so give the main llama.cpp server a moment before judging. Then test a harder case: tell Hermes that an earlier decision has changed and ask which decision is current and why. Check Hindsight's UI and stored observations rather than assuming memory status proves ingestion and recall are correct.

If the answer comes back empty, inspect the control plane before changing models. A retain can succeed and still be invisible to recall while recall_types is observations only and consolidation has not run — widening recall_types is the first check. A connection refused in the embedding server log means the container cannot reach the embedding port, and all-zero reranker scores mean the GGUF is the wrong conversion.

Write-path choices (auto_retain, missions, approval gap) are covered in Control What Gets Remembered.

Troubleshooting and Operational Notes

Symptom	Likely cause	First check
Hermes says provider unavailable	Wrong local_external URL or network rule	curl http://HINDSIGHT_VM_IP:8888/health
Docker starts, PostgreSQL errors	Missing extension, wrong credentials, or insufficient shared memory	Container logs and pg_extension
Embeddings return 401 or 404	Incorrect OpenAI-compatible URL, wrong env key name, or model alias mismatch	Direct /v1/embeddings test and variable names
Reranking fails	Endpoint or response format mismatch, including doubled /rerank path	Direct /v1/rerank test and container log
Recall returns nothing usable	recall_types limited to observations, or zero reranker scores	Control plane, then recall_types, then reranker conversion
Existing bank reports dimension mismatch	Embedding model changed	Rebuild vectors/reimport with a consistent model
Hermes responses slow down	Synchronous recall or shared LLM contention	Turn off sync recall; cap retain concurrency
Irrelevant memories repeat	Weak extraction, noisy retrieval, or insufficient consolidation	Inspect observations and relevance scores

A practical small deployment is 2-4 vCPUs and 4-6 GB RAM for the Hindsight API VM when PostgreSQL and all inference are external. Disk allocation still depends on the chosen Docker image and retained image layers; allocate roughly 30-40 GB initially with room to expand. Size the PostgreSQL host based on actual memory volume, storage performance, and the other databases it serves.

Back up and restore the Hindsight bank

The memory bank is the only state that matters, and it lives in PostgreSQL. In Option B that is a database you administer; in Option A it is the embedded pg0 named volume inside the container. Back it up independently of the Hindsight container — recreating the container without first capturing the volume or the database discards the bank, and hermes backup does not reach a bank running in a separate service.

Option B — external PostgreSQL. Dump the hindsight database in custom format, then restore it into a scratch database to verify the archive before you trust it:

pg_dump -h POSTGRES_IP -U hindsight -Fc hindsight > hindsight-bank.dump

# restore into a scratch database to prove the dump is usable
createdb -h POSTGRES_IP -U postgres hindsight-verify
pg_restore -h POSTGRES_IP -U postgres -d hindsight-verify hindsight-bank.dump

Confirm the pgvector extension and a sample recall exist in the restored copy before you delete the original dump target. Keep the dump on a different host than the Hindsight VM, and run this as a PostgreSQL admin (a role that can create databases) rather than the application role.

Option A — all-in-one container. The embedded database is a named volume, so stop the container and tar the volume through a throwaway image:

docker stop hindsight
docker run --rm -v hindsight-pg0:/data -v "$PWD":/backup alpine \
  tar czf /backup/hindsight-bank.tgz -C /data .
docker start hindsight

The mount point inside the tar container (/data) is arbitrary; it simply aliases the named volume so tar can read it. To restore, stop the container, clear the volume, and untar the archive back into it, or point the container at the external PostgreSQL restore from Option B.

Take a snapshot at three moments rather than on a calendar schedule: before changing the embedding model, before a major Hindsight or PostgreSQL upgrade, and before a reflect or consolidation pass that rewrites the bank. A backup you have restored into a scratch bank is worth more than one you have only written.

Export, import, and migrating the memory bank

Backup and restore protect the bank in place. Export/import moves it between Hindsight instances, and that is what a model change actually forces you to do.

Changing the embedding model is not a configuration edit. The vectors in the bank were computed with the old model, and a different model — for example the bundled 384-dimensional BGE embedder versus 1024-dimensional Qwen3-Embedding-0.6B — writes into a different space and usually a different width. You cannot re-point the existing instance at the new embedder and keep the old vectors; the dimensions must be recalculated by re-embedding every memory with the new embedding model.

Because the embedder and its configuration are set per Hindsight instance, the clean path is to stand up a fresh instance with the new embedding configuration and migrate the bank into it, rather than rebuilding vectors inside the one that is already serving recall. The Hindsight API can move the bank too, but for a one-off change export/import is the easier and more inspectable route:

# export the bank from the old instance
docker exec hindsight \
  hindsight-admin export-bank \
    --bank hermes2 \
    --output /tmp/hermes2.zip

docker cp hindsight:/tmp/hermes2.zip ./hermes2.zip

ls -lh hermes2.zip
unzip -t hermes2.zip

# import the bank into the new instance
docker cp ./hermes2.zip hindsight-q06:/tmp/hermes2.zip

docker exec hindsight-q06 \
  hindsight-admin import-bank \
    --archive /tmp/hermes2.zip

unzip -t verifies the archive before you ship it to the new instance. After the import, re-point Hermes at the new instance's api_url in ~/.hermes/hindsight/config.json, restart the gateway, and run the memory validation from earlier: a retained fact should recall through the new embedder. Leave the old bank and its vectors in the old instance until the new one behaves, so you still have a working fallback.

Monitor, prune, and keep the bank stable

With the bank running, the day-to-day job is watching what the extraction model has decided to store and keeping derived state in check. Use the control plane UI (hindsight-embed -p hermes ui start) to inspect observations, relevance scores, and consolidation output rather than assuming memory status proves ingestion and recall are correct. Duplicated or stale memories are usually a weak extraction or an under-run consolidation, so fix the extraction tier or consolidation cadence before widening retrieval.

Two maintenance traps are worth testing before you rely on the bank. First, deleting an observation does not guarantee every derived representation disappears with it — a reflect fact or graph edge built from it can survive the delete, so verify deletion end to end, the same provenance concern covered in self-reinforcing memory loops. Second, keep HINDSIGHT_API_WORKER_ID stable across recreations and pin a specific image tag: a task that was mid-flight when the container died stays stuck under the old worker id, and a moving tag is a poor match for a database you intend to keep.

References

• Hindsight source and documentation
• Official Hindsight installation guide
• Hindsight configuration reference
• Hindsight Hermes integration
• Hindsight inference models and reranking providers
• pgvector
• llama.cpp server
• Ollama