DEV Community: MihaiBuilds

The Brain v1.0 — building open-source workflow orchestration the boring way

MihaiBuilds — Tue, 16 Jun 2026 10:59:25 +0000

Originally published on mihaibuilds.com. Cross-posting here because dev.to is where I find a lot of this kind of work myself.

Memory Vault shipped five weeks ago — the substrate for the planned compounding stack. Memory is one half of "your AI assistant should know what you've done." The other half is "your tools should run when they should, against the right state, with the history you can audit." Cron lines on a server you half-remember and a Python file glued to a webhook glued to an LLM is not that. The Brain is the runtime underneath: Python-defined workflows, every run persisted, owned by you.

The Brain is open-source, self-hosted, MIT-licensed. A Python file defines the workflow. Postgres holds every run. Four trigger types fire it (manual, cron, webhook, file). Four step types execute inside it (shell, LLM, Memory Vault REST, MCP). The whole thing is one multi-arch Docker image. Today it crosses the line from build-in-public project to v1.0 stable release.

What The Brain is

A workflow orchestrator with persistent run history. You define a workflow as a Python file — a sequence of named steps. The Brain runs it on whichever of the four triggers you registered (a CLI invocation, a cron expression, an HMAC-signed webhook, or a filesystem change), persists every step's result to a single Postgres database, and stops on the first failure with a clear error. brain history shows what ran. brain show <run-id> shows the per-step detail. brain diagnose produces a redacted zip ready to attach to a bug report.

It runs entirely on your machine. No SaaS account. No telemetry. No cloud lock-in. docker compose --profile api up -d and it's running. The CLI alone works against a Dockerized Postgres if you don't need the API or the watcher daemon.

The biggest thing about The Brain is what it deliberately doesn't do: it doesn't decide anything on its own. Workflows are scripted Python. The LLM step just transforms text. If a workflow wants "the LLM picks the next tool to call," you write that workflow — but the workflow file is doing the picking, not The Brain. The Brain is a workflow orchestrator, not an agent runtime.

What v1.0 actually does

Four step types — ShellStep runs subprocess commands with stdout capture and exit-code semantics; LLMStep calls an OpenAI-compatible chat API with per-step provider/model/key/timeout overrides; MemoryVaultStep calls Memory Vault's REST API for the friction-free case; McpToolStep calls any MCP tool over stdio with per-step spawn lifecycle.
Four trigger types — manual (CLI), cron (scheduler daemon), webhook (HTTP POST with HMAC signature, verified constant-time), file (filesystem watcher with debounced events).
Substitution between steps — {previous.step-name} references prior step output; {trigger.body} / {trigger.path} / {trigger.headers.X} reference trigger payload data. Textual substitution, not eval'd Python — str.format shape, strict-by-default errors when a reference doesn't resolve.
Three processes, one database — the CLI for one-shot runs, brain serve for the HTTP API + webhook intake, brain watch for cron and file triggers. Compose profiles let you run any subset.
Structured logging with run_id binding — bind_run_id() attaches the run UUID to every log line emitted during a run. LOG_FORMAT=keyvalue for human-readable console output, LOG_FORMAT=json for log aggregation pipelines.
Redacted diagnostic bundle — brain diagnose writes a timestamped zip with recent logs, environment, OS info, Brain version, and Docker container state. The redaction model is allow-list, not blocklist: nine env vars are recorded with values; four (DB_PASSWORD, LLM_API_KEY, MEMORY_VAULT_TOKEN, THE_BRAIN_API_TOKEN) are presence-only — name appears, value never appears in the bundle.
One-command Docker, multi-arch — linux/amd64 and linux/arm64 images published to ghcr.io/mihaibuilds/the-brain. Compose profiles for api and watcher. Derive your own image when you need to add MCP servers or extra binaries.
MIT-licensed, self-hosted — your workflows, your data, your hardware. The whole thing is yours.
363 tests passing — pytest with a real Postgres service container, no mocks at integration boundaries.

Architectural decisions worth naming

Postgres is the only state store. No Redis. No queue. No second datastore. Run history, webhook secrets, and scheduler state all live in the same Postgres database, with LISTEN/NOTIFY for cross-process wakeup and plain SQL polling for the rest. You already know how to back up Postgres. You already know how to monitor it. Adding a queue or a Redis would double the backup story and the operational mental model for no win at the scale a self-hosted workflow runner actually runs at. When that stops being true, the migration path is sane. Until then, one database is the right answer.

Three processes, not one. The runner is synchronous per-run, the HTTP API is async per-request, the watcher is a long-running event loop. Splitting them means a crashing scheduler doesn't take the API down, and operators can run only the parts they need via Compose profiles. CLI-only users skip the daemons entirely. API-only users skip the watcher. The full stack runs all three. Process boundaries are cheaper than recovery procedures.

Per-step subprocess spawn for shell and MCP. A crashing shell command or a misbehaving MCP server should not take the runner down. ShellStep always spawns a fresh subprocess per step. McpToolStep spawns a fresh MCP server, runs the protocol handshake, makes one tool call, and tears it down at step end. No shared client. No pooling. The fork/exec cost is in the noise for the workflow shapes The Brain targets, and the isolation is real — if an MCP server crashes mid-call, only that one step fails and the next step gets a brand-new subprocess.

Workflow files are trusted Python, not a YAML DSL. Every YAML workflow language ends up needing escape hatches that turn back into code. Workflow-as-Python means full editor support, full type checking, zero new syntax to learn — at the cost of treating workflow files as part of your trust boundary, same as any other file in your repo. The threat model in SECURITY.md says so up front rather than hiding it behind a "do not run untrusted workflows" footnote.

The first time the ecosystem composed end-to-end

Memory Vault has been live since early May. The Brain has been under construction since mid-May. I've been calling them "the ecosystem" the whole time, but they were two completely separate products living in two completely separate places — separate Postgres databases, separate codebases, separate Docker images. They had never actually worked together end-to-end.

v1.0 of The Brain is the first time the two compose in production shape. The pattern is what I call horizontal substrate, vertical products: Memory Vault is the substrate that stores and retrieves; The Brain is the engine that runs workflows; both run independently, both compose together. A workflow on The Brain can ask Memory Vault for memories over MCP, pipe those memories into an LLM step that summarizes them, write the summary to a file. Real Postgres on both sides. Real MCP stdio transport. Real LLM. Real file written.

The integration path is the derive-pattern: you write a Dockerfile that derives FROM ghcr.io/mihaibuilds/the-brain:1.0, installs Memory Vault into the image, and the workflow step's server_command spawns Memory Vault's MCP server inside the same container as a per-step subprocess. The Brain ships zero MCP servers in the stock image — that's deliberate. The Brain stays small. Memory Vault stays independent. The composition is opt-in and explicit. The same shape works for any MCP server: GitHub's, Sentry's, your own.

This is the moment "the ecosystem" stops being something I write on a roadmap and becomes a system that genuinely exists. Two products. Two databases. One Docker network. Composing.

The version-string story

I tagged v1.0.0 from main after the security audit pass. The release workflow fired, built the multi-arch image, pushed it to ghcr.io, and auto-published the GitHub Release from my annotated tag message. Two minutes twenty-nine seconds, end to end. Clean.

Then I ran the smoke test. docker pull ghcr.io/mihaibuilds/the-brain:1.0.0 succeeded. Multi-arch manifest verified — linux/amd64 and linux/arm64 both present. I ran brain --version inside the published image to confirm the version string matched the tag.

It returned brain, version 0.0.1.

I never bumped pyproject.toml from 0.0.1. The image tag said 1.0.0. The CLI inside the image said 0.0.1. The release was published, immutable, and visibly inconsistent on every fresh install.

The fix was four lines. Three options were on the table: hotfix to v1.0.1, delete the published tag and re-tag (which would break anyone who'd already pulled :1.0.0), or leave it for the natural course of a v1.0.x. The right call was the cheap one — hotfix. Bump pyproject to 1.0.1 (skipping 1.0.0 because that tag is published and immutable), open a one-file PR, watch CI go green, merge, tag v1.0.1, push. The release workflow re-fired and republished. Five minutes later, brain --version inside :1.0.1 and :latest both returned brain, version 1.0.1. The digest is sha256:41d0becdf7d036….

The lesson isn't about version strings. It's that automated release infrastructure makes hotfix cycles cheap, and "cheap to do correctly the second time" is what gives you permission to do v1.0.0 with confidence rather than paralysis. The pre-tag checklist now has a pyproject.version line. Future ecosystem product launches will hit this gate before the tag is pushed. The hotfix lesson costs five minutes of CI; the lesson is permanent.

The discipline that built the v1.0 ship survived its own first encounter with reality — by treating a real release-engineering bug like a real bug, not by quietly editing history.

What v1.0 doesn't do, on purpose

No retries on step failure. First step failure halts the workflow. The persisted run row shows exactly where it stopped and why. Retry-on-failure with backoff is a v1.x candidate when real users ask for it, not a v1.0 default that hides bugs.

No catch-up on missed cron windows. If The Brain is down when a cron expression matches, that window is skipped — not deferred and replayed. Distributed time-shifted scheduling is its own product; The Brain assumes the host is online when the cron fires.

No tools/list discovery for MCP servers. Workflow authors know the tool name and argument shape in advance, like they know which shell commands they're calling. Discovery is an interactive-IDE problem, not a workflow-orchestrator problem.

No MCP over HTTP transport. Stdio only in v1.0. HTTP transport is a v1.x candidate if real demand surfaces.

No LLM-driven tool orchestration. LLMStep is chat-completion only. If a workflow wants "the LLM picks the MCP tool to call," that's a two-step pattern: the LLM step produces a tool name, the substitution pipes it into a downstream McpToolStep. The workflow is the agent. The Brain is the runtime that runs the workflow.

No bundled MCP servers in the stock image. The base image is intentionally minimal. Derive-your-own-image is the documented composition path.

No multi-user. v1.0 is single-tenant. The single THE_BRAIN_API_TOKEN env var is the auth model. Multi-user with workspace isolation is part of the planned PRO tier.

No visual workflow builder. Workflows are Python.

These are deliberate trade-offs. Honest gaps documented up front build more trust than feature bullets that fall apart when someone actually tries them.

The open-core model

The Brain is and will always be MIT-licensed. The whole thing — the runner, the four step types, the four trigger types, the HTTP API, the CLI, the scheduler daemon, the watcher daemon, the structured logging, the diagnose bundle, the multi-arch Docker images. You can run it on your machine. You can fork it. You can use it inside a commercial product. The free tier is the full runtime, not a crippled demo of a paid tier.

A paid PRO tier is planned: multi-user with workspace isolation, a hosted scheduler for people who don't want to run their own watcher daemon, a secrets vault, conflict resolution for shared workflow repos, additional integrations. The PRO tier is genuinely paid operational features — what teams running shared workflow infrastructure actually need, not what a solo developer on a laptop strictly needs. v1.x stays free forever.

Try it

docker pull ghcr.io/mihaibuilds/the-brain:latest

git clone https://github.com/MihaiBuilds/the-brain
cd the-brain
cp .env.example .env
docker compose --profile api up -d

Open http://localhost:8001 and the HTTP API is running. brain run examples/hello.py from the CLI.

GitHub release page
README and quick start
ARCHITECTURE.md — the design overview
SECURITY.md — the threat model and the trust posture for workflow inputs
Questions and bug reports: GitHub Issues

Follow along

Twitter / X: @mihaibuilds
Blog: mihaibuilds.com
GitHub: github.com/MihaiBuilds/the-brain

The Brain talks to everything now

MihaiBuilds — Fri, 12 Jun 2026 10:41:24 +0000

Originally published on mihaibuilds.com. Cross-posting here because dev.to is where I read a lot of work like this myself.

A few days ago I shipped the third milestone of The Brain — webhook triggers with HMAC auth, file watchers in their own container, the {trigger.X} placeholder family for inbound payloads. That was M3. The Brain had the four classical trigger types: manual, scheduled, webhook, file.

Today M4 is done. The Brain now talks to other tools — natively, over MCP — and the LLM step picks its own model per call.

Why this matters

M1 was the runner. M2 made the runner work unattended. M3 made the runner reactive. M4 makes the runner ecosystem-aware.

Before M4, The Brain was a workflow orchestrator that knew how to do three things on its own: run shell commands, call a local LLM through a fixed configured endpoint, and call Memory Vault over its REST API. Useful, but every integration with anything new required writing a custom adapter.

After M4, The Brain can call any MCP server as a workflow step. Memory Vault's MCP server, GitHub's, Sentry's, your own. The stdio transport is the v1.0 commitment; the workflow file says "spawn this MCP server, call this tool, here are the arguments" and The Brain handles the lifecycle.

The LLM step also got per-step overrides. Before M4, every workflow used one configured model server at one URL. Now each step can name its own provider URL, its own model, its own API key, its own timeout, its own max tokens. Mix a fast local model and a slow careful one in the same workflow.

What M4 ships

Per-step LLM overrides. Each LLMStep can override the global LLM_BASE_URL / LLM_API_KEY / LLM_MODEL env vars per call:

LLMStep(
    name="fast_summary",
    prompt="Two sentences: {previous.recall}",
    model="mistralai/ministral-3-3b",
    timeout_seconds=60,
    max_tokens=400,
)

LLMStep(
    name="careful_analysis",
    prompt="Detailed breakdown of: {fast_summary}",
    provider_url="http://other-host:1234/v1",
    api_key="sk-...",
    model="anthropic/claude-3-5-sonnet",
    timeout_seconds=600,
    max_tokens=4000,
)

Each field falls back to the corresponding env var when set to None. Tested against LM Studio only — other OpenAI-compatible providers (Ollama, vLLM, llama.cpp server, OpenAI proper) may work via the same wire format but are not promised in v1.0.

MCP tool calling as a step type. A new McpToolStep peer to the existing step types:

McpToolStep(
    name="recall",
    server_command="python -m memory_vault.mcp",
    tool="recall",
    args={"query": "{previous.search_term}", "limit": 10},
    timeout_seconds=30,
)

The server_command and string values in args accept {previous.X} and {trigger.X} placeholders the same way ShellStep.command does. The tool name and args keys are never substituted — protocol-level identifiers, not user data. Non-string args values (ints, bools, nested dicts) pass through unchanged.

stdio transport only in v1.0. initialize + tools/call only — no tools/list, no resources, no prompts, no server-initiated notifications. Each step spawns the MCP server fresh, runs the handshake, calls one tool, and tears the subprocess down. No shared state. No pooling.

The derive-your-own-image pattern. The stock mihaibuilds/the-brain image bundles zero MCP servers. The Brain is a workflow orchestrator; MCP servers are independent products. Coupling them would force users into installing things they don't need.

If your workflow calls an MCP server, install that server in a derived image:

FROM mihaibuilds/the-brain:latest
RUN <install-command-per-the-mcp-server-s-readme>

examples/brain-with-mv-mcp/ ships a complete worked composition with Memory Vault — Dockerfile, docker-compose.yml, a verify workflow, and a runbook README.

Architectural decisions worth naming

Per-step spawn lifecycle. Every McpToolStep spawns its MCP server subprocess at step start, runs the MCP initialize handshake, calls one tools/call, and kills the subprocess at step end. No shared client. No connection pool. Cold start cost per step is ~200-500ms for a server like MV's that loads sentence-transformers + spaCy + a pgvector connection on every spawn. The trade-off: isolation per call. A crashed MCP server kills only one step. A leaked file descriptor in the MCP server is cleaned up by the OS when we kill it. The next step gets a fresh subprocess. Per-run pooling is a future consideration if real latency complaints surface; v1.0 takes the isolation.

stdio transport, newline-delimited JSON, no Content-Length framing. The MCP spec defines stdio framing as newline-delimited JSON — one JSON message per line, terminated by \n on both stdin and stdout. The Content-Length framing is the streamable-HTTP transport, which is a separate protocol surface with its own auth concerns (Bearer / mTLS / OAuth). For v1.0, stdio is the deeper and more universal transport — Memory Vault's MCP server uses it, Claude Desktop uses it, and every reference MCP implementation uses it. HTTP transport may come in a future version.

Single-flight via asyncio.Lock. A single StdioMcpClient instance serializes call_tool invocations internally. The per-step-spawn lifecycle means concurrent calls per client never happen in normal use, but the lock removes a real foot-gun if someone hand-shares a client. Cheap insurance.

Eager handshake on connect. The MCP initialize handshake runs in __aenter__ / connect, not lazily on first call_tool. The per-call timeout covers handshake + tool call together from the caller's POV. If initialize hasn't run yet when call_tool fires, the caller's 30-second budget would silently include some unknown amount of handshake time. Eager handshake makes the budget actually mean what it says.

Background stderr reader for pipe-fill resilience. A continuous background task drains the subprocess's stderr pipe to a rolling ~1 KB tail. Without it, a chatty MCP server writing lots of stderr (say, a debug-build that logs everything) would fill the OS pipe buffer (~64 KB on macOS) and the subprocess would block waiting for someone to read stderr. Meanwhile The Brain would be waiting for stdout, deadlocking the whole call. The background reader prevents that. The captured tail is exposed via the stderr_tail property for debug logging at step boundary — and never returned in StepResult.output. Workflow data and debug data are different surfaces. A workflow author querying {previous.recall} must never see stderr noise mixed into their workflow values.

Substitution boundaries are sharp. The runner's _resolve_step function gains a new branch for McpToolStep.args (a dict). It iterates dict values, substitutes string-typed values via {previous.X} + {trigger.X} resolvers, leaves non-strings and keys untouched. The tool name is never substituted. Nested-dict args (args={"filter": {"query": "{previous.X}"}}) are not recursively substituted — consistent with the {trigger.body.foo} no-nesting rule from M3. Pinned by five separate substitution-boundary tests plus cross-PR pins in the audit-pass test file.

isError: true becomes step failure. When an MCP server returns a successful JSON-RPC response containing isError: true, The Brain treats it as step failure — same shape as a non-zero shell exit code. The first text content block in the response becomes the step's error message. MCP-side tool errors flow through the same workflow-halt semantics as every other failure path, so workflow authors don't have to check isError in every downstream step.

MemoryVaultStep ↔ McpToolStep coexistence. Both ship in v1.0. Neither is deprecated. MemoryVaultStep calls MV over its REST API with no extra setup — easy default for "I just want hybrid search from MV." McpToolStep is the generic any-MCP-server mechanism — works for MV's MCP server (via the derive-pattern), GitHub's, Sentry's, your own. The deprecation question was considered and rejected — forcing users into the harder setup path right at v1.0 is the wrong direction.

The moment for the ecosystem

I want to call this out separately because it matters more than either feature individually.

Memory Vault went live two months ago. The Brain has been under construction since May. I've been calling them "the ecosystem" the whole time, but they were two completely separate projects living in two completely separate repositories. They had never actually worked together end-to-end.

For M4's verify pass, I built a derived image with both projects installed, separate Postgres instances (Brain's tables + MV's pgvector tables), three containers in one Docker network. The verify workflow asks Memory Vault — over MCP — for memories matching a query. Memory Vault searches its pgvector index and returns chunks with similarity scores. The Brain pipes the chunks into a local LLM step. The LLM writes a digest. A shell step saves it.

It worked. Real database, real hybrid search, real LLM call, real file written.

I ran it twice. Once with Ministral-3B-Instruct loaded in LM Studio — about 4 seconds end-to-end. Once with Qwen3.5-9B, a reasoning-style model — about 2 minutes 13 seconds. Same workflow file. The only difference was three fields on the LLM step: model, timeout_seconds, max_tokens.

Both summaries were real. The fast model wrote a tight two-sentence digest. The reasoning model produced a longer, more comprehensive summary that captured more of the original context — at thirty times the wall-clock cost. Same per-step override mechanism made the swap trivial.

This is the first time The Brain and Memory Vault have actually composed in production shape. The moment where "the ecosystem" stops being a roadmap word and starts being a system that exists.

What v1.0 won't do, on purpose

The LLM step does not drive tool calling. LLMStep is chat-completion only — it produces text. If a workflow wants "LLM picks an MCP tool to call," it wires that explicitly: LLMStep produces a tool name, {previous.X} substitution puts that name into the next step's args (the tool field itself is locked NOT-substituted, so the workflow author chains through args or uses separate branches). The workflow file is the orchestrator. The LLM transforms text. It does not decide. This is by design.

No tools/list discovery. Workflow authors know the tool name and the args shape in advance, the same way they know what shell commands they're calling. If you want introspection, build it in a separate workflow step.

MCP HTTP transport is not in v1.0. Stdio only. HTTP transport (the streamable-HTTP MCP variant) brings its own auth surface. For v1.0, stdio is the deeper transport.

The stock image bundles zero MCP servers. Per the ecosystem rule. Derive-your-own-image is the documented path.

Per-run MCP server pooling is not implemented. Per-step spawn is the v1.0 lifecycle. Two McpToolStep calls to the same server in one workflow run produce two distinct subprocess PIDs. The cold-start cost is real; v1.0 takes the isolation guarantee.

No custom LLM auth schemes. Bearer-only when an api_key is set, no header when it isn't. If your provider needs something else, bake it into your derived image.

No bundled MCP servers. Stock image stays lean. Each MCP server is a separate install in your derived Dockerfile.

No Docker-socket-mount for The Brain container. Considered and rejected. A leaked webhook secret + a malicious payload substituted into server_command would become a host escape. The derive-your-own-image pattern is the secure alternative — you control the contents of your derived image, not a runtime Docker socket.

Reasoning models need bigger budgets. Reasoning-style LLMs (qwen 3.x+, o1-style, R1-style, QwQ) consume token budget on internal reasoning before producing visible content. If you point a per-step LLM call at a reasoning model with default budgets, you may get empty visible output. The fix is bigger budgets — timeout_seconds=600 and max_tokens=8000+ is a reasonable starting point for a 9B reasoning model. Instruct models (Ministral, Mistral Instruct, Llama Instruct) don't have this behavior.

These are deliberate trade-offs. M4 is the smallest correct ecosystem-aware surface, not the most ambitious one.

Who this is for

Same audience as M1 + M2 + M3, with one addition: anyone building self-hosted workflow automation that needs to reach multiple specialized tools without writing a custom adapter for each one. The MCP ecosystem in 2026 has dozens of servers — for memory (Memory Vault), for code review (GitHub MCP), for observability (Sentry MCP), for filesystems, for databases, for browser control. M4 makes any of them callable from a Brain workflow step with the same shape.

If you've ever wanted to wire an LLM workflow into multiple specialized backends without committing to LangChain — this is for you.

What's next

Milestone 5 is the v1.0 launch milestone. It's not new features — continuous integration, a security audit, full docs, the public README polish, and the launch ritual. After M5 ships, The Brain is publicly v1.0 — open-source, MIT, single-tenant, self-hosted, same shape Memory Vault took at its own v1.0.

There's no M5 dev-log post on this dev.to series. The next post will be the v1.0 launch post itself.

Try it

git clone https://github.com/MihaiBuilds/the-brain
cd the-brain
THE_BRAIN_API_TOKEN=any-value docker compose up -d

# call any MCP server from a workflow (build your own derived image first
# with the MCP server installed — see examples/brain-with-mv-mcp/)
docker compose exec brain brain run examples/mcp_recall_memory.py

# or use per-step LLM overrides without any MCP setup
docker compose exec brain brain run examples/daily_digest.py

The repo has the full README, the derive-pattern example with a complete runbook for composing The Brain with Memory Vault, and reference workflows for both LLMStep and McpToolStep.

Follow along

Twitter / X: @mihaibuilds
Blog: mihaibuilds.com
GitHub: github.com/MihaiBuilds/the-brain

The Brain reacts now

MihaiBuilds — Wed, 10 Jun 2026 09:49:18 +0000

Originally published on mihaibuilds.com. Cross-posting here because dev.to is where I read a lot of work like this myself.

Three weeks ago I shipped the second milestone of The Brain — the scheduler daemon, cron triggers, workflows that read their previous run, an opt-in HTTP endpoint. That was M2. The Brain could run unattended on a clock.

Today M3 is done. The Brain now reacts — to HTTP requests, and to filesystem changes.

Why this matters

M2 made The Brain worth running unattended on a schedule. M3 makes it react to things that happen. The hardest, most useful workflow automations are the reactive ones — the workflow that fires when a customer signs up, the workflow that processes a file the moment it lands on disk, the workflow that wakes up because another system has news.

M1 was the runner. M2 made the runner unattended. M3 makes the runner reactive. M1 + M2 + M3 together is the trigger surface most people actually need.

What M3 ships

Webhook triggers. Register any workflow as a webhook endpoint. The Brain prints a secret once, you save it, and from that moment on, any HTTP caller with the secret can fire the workflow over the network.

docker compose exec brain brain register-webhook examples/webhook_handler.py

The CLI prints the HMAC secret exactly once — same caller-side-storage discipline as a GitHub personal access token. There's no brain show-webhook-secret command by design; if you lose the secret, you unregister and re-register to issue a fresh one.

Fire it from anywhere that can compute an HMAC signature:

SECRET=<your-saved-secret>
BODY='{"hello":"world"}'
SIG="sha256=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$SECRET" | awk '{print $2}')"

curl -X POST http://localhost:8001/webhook/webhook-handler \
    -H "X-Brain-Signature: $SIG" \
    -H "Content-Type: application/json" \
    -d "$BODY"

The X-Brain-Signature: sha256=<hex> header convention is identical to GitHub's X-Hub-Signature-256 — so existing webhook senders work without translation. The endpoint runs the workflow synchronously and returns the run metadata. Wrong signature is 401. Unknown workflow name is 404, same shape as a disabled webhook, so existence is not leaked through the response code.

File watcher triggers. Register a workflow to fire when something changes on disk. The Brain runs a separate watcher daemon that observes the directory and fires the workflow on filesystem events.

docker compose exec brain-watcher brain register-watcher examples/markdown_watcher.py \
    --path /data/watched --events modified

--events accepts any combination of created, modified, deleted. The watcher daemon picks up the new registration on its next 10-second sync. A 500ms debounce per (workflow, path) coalesces multiple filesystem events from a single editor save into one workflow run.

The watcher runs in its own container behind the watcher compose profile. If the watcher crashes, the scheduler from M2 keeps running. If the scheduler crashes, the watcher keeps watching. Isolation by container, not by retry loops.

A trigger placeholder family. Workflows triggered by a webhook or file event can read the inbound payload via a new placeholder family:

ShellStep(
    name="received",
    command="echo got event={trigger.event} body={trigger.body}",
)

Four placeholders are available wherever string substitution works: {trigger.event} (the trigger mechanism), {trigger.body} (the inbound body — parsed JSON stringified deterministically, or raw string fallback), {trigger.headers.X} (case-insensitive HTTP header lookup, allowlist gated), {trigger.path} (the file path for file-triggered runs). Referencing {trigger.X} on a workflow you ran manually fails the step with a clear error — same strict-failure shape as M2's {previous.X} placeholder.

The four classical trigger types — manual, cron, webhook, file — are now all there. Same workflow model. Same persistence model. Same brain history and brain show view of every run.

Architectural decisions worth naming

HMAC verification is constant-time at every failure path. Wrong prefix, wrong algorithm, malformed hex, length mismatch, non-string input — every failure shape runs through the same hmac.compare_digest call against a placeholder digest. There is no early len(a) != len(b) branch. The verifier is pinned by an end-to-end timing-attack regression test: it measures wall-clock variance between "wrong-length" and "right-length-wrong-value" failures across 2000 iterations and asserts the ratio stays under 10x. If a future refactor introduces a length-check shortcut, the test breaks loudly.

404 on unknown webhook is a locked v1.0 behavior, not a bug. A probe CAN distinguish "unknown webhook" from "known but wrong signature" via response code. The webhook name is not a secret in this threat model — single-token-server-to-server with known callers, and if you can list webhooks you already have privileged access. Pinning the lock as a regression test: any future refactor that adds a constant-time-equal-lookup must break the test and surface as an explicit architectural decision, not a silent change. Same lock applies to the 404-for-disabled case.

Watcher and scheduler heartbeats coexist via daemon_id suffix. Both daemons UPSERT into the same daemon_heartbeats table. The scheduler uses the container hostname as its daemon_id. The watcher appends :watcher. The crash-recovery sweeps are mutually disjoint via the trigger_context->>'event' JSONB filter — the scheduler clears running rows broadly, the watcher clears only file-triggered ones, and the two queries never overlap. Both daemons can be in the table at once without collision.

The 500ms debounce is in-memory. A dict[tuple[workflow_name, path], float] keyed by monotonic time. Lost on daemon restart, which is fine because crash recovery re-fires from current FS state and any in-flight transient state is by definition stale. The boundary is exact and pinned at three layers: the unit _should_fire function, the module constant DEBOUNCE_SECONDS == 0.5, and an audit-pass test that pins 499ms blocks, 500ms fires, 501ms fires.

Sequential within process, concurrent across processes. Two webhook calls to the same workflow queue inside the API process. Two file events to the same watcher queue inside the watcher process. But the API, scheduler, and watcher daemons all run workflows in parallel because they're separate processes against the same database. No global work queue. The database absorbs concurrent INSERTs at the workflow_runs row level.

The {trigger.body} resolver stringifies JSON deterministically. json.dumps(body, sort_keys=True, separators=(",", ":")). So the same parsed payload produces the same substituted command every time — deterministic for cache invariants, deterministic for diff workflows. Raw string bodies pass through unchanged. Nested JSON access ({trigger.body.foo}) is NOT supported in v1.0 — the body is a string after serialization; body.foo is treated as an unknown trigger field. Pinned with a locked-behavior test.

Header allowlist is hardcoded, not configurable. The four placeholders the workflow can read from {trigger.headers.X} are bounded by the allowlist: content-type, user-agent, x-github-event, x-github-delivery, x-stripe-event, x-event-key. Authorization, X-Brain-Signature, cookies, infrastructure headers — never exposed to the workflow. If a step needs another header, that's a workflow-step concern that should be visible in the workflow source, not a configuration knob.

What v1.0 won't do, on purpose

The watcher daemon is not highly available. One watcher per host, same single-daemon-per-host invariant as the scheduler. Two watchers running in parallel would clobber each other's crash-recovery logic.

No nested JSON access via {trigger.body.foo}. Locked. The body is a string after serialization; body.foo is treated as an unknown trigger field. If you need to pluck a field, do it in the workflow step (e.g. echo {trigger.body} | jq -r .foo).

No recursive directory watching. Single directory per watcher row, no globs. If you want to watch a tree, run multiple watchers. The hardest part of watcher correctness is bounding the work; bounding it explicitly via one-dir-per-row is the v1.0 choice.

No webhook API docs in production. /docs, /redoc, /openapi.json all 404 by design. The threat model is known-callers — anyone enumerating the endpoint shape is in scope.

No replay protection on webhooks. Idempotency is the workflow's concern, not the transport's. If your workflow can't be replayed safely, build the idempotency key check into the workflow itself.

No catching up on missed file events. Filesystem events are not persistent. The watcher daemon sees current FS state at boot; events that happened during downtime are missed. Don't use file watchers for anything where missing events is unacceptable — use a cron schedule that reconciles state instead.

Workflows still execute one at a time per process. Cross-process concurrency exists (API + scheduler + watcher in three containers can all run workflows simultaneously). Within-process concurrency is sequential by design for v1.0.

These are deliberate trade-offs. M3 is the smallest correct reactive trigger surface, not the most ambitious one.

Who this is for

Same audience as M1 + M2, with one addition: anyone building self-hosted automation against webhook senders (GitHub, Stripe, your own dashboards) who's tired of either rolling their own webhook server with no run history, or paying for a managed orchestrator that owns their auth.

If you've ever wired a webhook to a tiny Flask app that calls a script and then forgotten about it for six months until something breaks — this is for you.

What's next

Milestone 4 adds MCP tool calling as a step type, plus a pluggable LLM provider abstraction. That's the milestone where The Brain becomes ecosystem-aware — any MCP server in your environment becomes a callable step, not just Memory Vault. Each milestone gets a dev-log post here as it ships — one of four dev.to posts across the build period.

Try it

git clone https://github.com/MihaiBuilds/the-brain
cd the-brain
THE_BRAIN_API_TOKEN=any-value docker compose --profile api --profile watcher up -d

# register a webhook (saves the secret to stdout — copy it now)
docker compose exec brain brain register-webhook examples/webhook_handler.py

# register a file watcher (must run from inside the watcher container)
docker compose exec brain-watcher brain register-watcher examples/markdown_watcher.py \
    --path /data/watched --events modified

# see all your triggers in one place
docker compose exec brain brain list-triggers

Drop a file in ./watched, sign and POST to http://localhost:8001/webhook/webhook-handler, and the runs land in brain history alongside any manual or scheduled runs from M1 and M2. The repo has the longer version with the full HMAC signing recipe, the trigger-placeholder reference, and the lifecycle commands for both trigger types.

Follow along

Twitter / X: @mihaibuilds
Blog: mihaibuilds.com
GitHub: github.com/MihaiBuilds/the-brain

The Brain runs on a schedule now

MihaiBuilds — Fri, 05 Jun 2026 12:52:02 +0000

Originally published on mihaibuilds.com. Cross-posting here because dev.to is where I read a lot of work like this myself.

Two weeks ago I shipped the first milestone of The Brain — the bare runner. A Python file with a sequence of steps, brain run path/to/workflow.py, the run lands in Postgres, you inspect it from the CLI. That was M1. It works, and you can run it on demand whenever you want.

Today M2 is done. The Brain now runs on a schedule, on its own, without you in the loop.

Why this matters

The most useful workflow automation only kicks in when you stop having to babysit it — daily digests, scheduled exports, nightly summaries, anything that compounds. M1 proved the runner works. M2 is the milestone where leaving it alone is a reasonable thing to do.

That's the whole point of M2 in one sentence. The rest of this post is what that looks like in practice — and what it deliberately doesn't try to do.

What M2 ships

Cron schedules. Register a workflow on a standard 5-field cron expression. The Brain writes the schedule to Postgres next to your run history.

docker compose exec brain brain register examples/daily_digest.py --cron "0 9 * * 1-5"

The schedule validates the cron expression and the workflow file before it lands in the database. Duplicate schedule names are rejected — no silent overwrite. You can list everything that's registered, see when each one ran last and when it'll fire next, pause and resume schedules (idempotent), and unregister them when you're done. Same CLI you used in M1, against the same database.

A scheduler daemon. The container now runs a long-running process — a daemon — that polls the schedule table every 10 seconds and fires whatever is due. SIGTERM finishes the currently-running workflow before exiting cleanly. On a crash, any run that was in flight gets recovered as a failed run with a clear error, so the run history never lies about what's running and what isn't.

The daemon and the CLI are separate processes against the same database. You don't have to "stop the daemon to run a workflow" — docker compose exec brain brain run ... still works exactly as it did in M1, in parallel with whatever the daemon is doing.

A new brain daemon-status command tells you whether the daemon is alive (exit 0 if it ticked within the last 30 seconds). Docker uses the same command as its container healthcheck.

Workflows that read their previous run. A step can write {previous.<step_name>} in its prompt or command, and The Brain substitutes the same step's output from the last successful run of the same workflow.

LLMStep(
    name="summary",
    prompt=(
        "Yesterday's summary:\n{previous.summary}\n\n"
        "Today's memories:\n{recent}\n\n"
        "Write today's summary."
    ),
)

On the very first run, when there is no previous successful run, the step fails with a clear error rather than silently substituting empty string. Same strict-failure shape as M1's intra-run {step_name} placeholder — better to halt loudly than to leak unresolved braces into a shell command. Once one run has succeeded, every subsequent run sees its output via the placeholder.

An opt-in HTTP endpoint. POST /run accepts a workflow path, runs the workflow, and returns the run's metadata as JSON. Bearer token from an environment variable; without the token in the environment, the service refuses to start. Designed for server-to-server, not browsers — no CORS, no public docs, single token. Opt in by bringing up the api compose profile.

THE_BRAIN_API_TOKEN=your-secret docker compose --profile api up -d

If you want to fire workflows from another machine, this is the surface. If you don't, ignore the profile and nothing in M1 changes.

Architectural decisions worth naming

daemon_tick(now) is the unit of behavior, not the polling loop. The daemon does one thing well: a single async function takes a wall-clock moment and runs one poll cycle (heartbeat, look up due schedules, fire each sequentially, advance next_run_at). A separate run_daemon wraps it in a 10-second loop with signal handlers. Tests drive the cycle function directly with a frozen clock instead of spawning a real long-running process — the wrapper is dumb on purpose. Two hours of test-design savings every time a future scheduler concern needs a regression test.

Skip, don't catch up. If a workflow takes longer than its cron interval — say a 1-minute cron whose last run took 5 minutes — the daemon does not queue up four backlog fires for the boundaries it missed. It fires once, advances next_run_at to the next cron boundary after right-now, and moves on. A schedule that fell six hours behind because the container was off fires once and continues on its current cadence. Catching up across a long outage is almost always wrong; it floods the system with stale work the moment it comes back.

Sequential within a poll cycle. No concurrent workflow execution. A long-running workflow blocks the daemon from picking up other due workflows until it finishes. This is by design for v1.0 — parallel execution and a real work queue carry concurrency-control complexity that needs to wait until I have a real workload to optimize against, not a hypothetical one. v1.1 concern, called out in the explainer notes.

Crash recovery on boot, not in-flight. When run_daemon starts, it sweeps workflow_runs WHERE status='running' and marks them all failed with a locked error message. Under the single-daemon-per-host invariant these are by definition orphans from a previous crash. No heartbeat liveness check, no leader election, no consensus protocol. Single daemon means single source of truth for what "in flight" means.

A planned_steps JSONB snapshot on every run. Each workflow_runs row now has the full step list at run-creation time — [{"name": ..., "type": ...}, ...]. Lets postmortem disambiguate "step absent from output because the run halted before reaching it" from "step never existed in this workflow version." One extra json.dumps per run, no extra query. The cost is rounding error; the postmortem clarity is worth it. Suggested by a comment under the M1 dev.to post — pinned to the schema before the analyzer existed.

{previous.X} is a single indexed lookup. A partial index on workflow_runs (workflow_name, started_at DESC) WHERE status = 'success' makes the previous-run lookup an index-only scan. The previous run's output JSONB is decomposed into a step-name → output map at lookup time, which is what {previous.X} resolves against. Strict on failure: no prior successful run, or step name missing from the previous run, both fail THAT step with a clear distinct error. Two messages, two tests pinning them.

HTTPBearer with auto_error=False. FastAPI's default HTTPBearer returns 403 on a missing Authorization header. That's wrong — RFC 7235 says missing auth is 401, forbidden is 403. The explicit auto_error=False + manual 401 raise corrects this. Small bug, but it's the kind of small bug that wastes a peer's afternoon when they're integrating against the endpoint and can't figure out why curl gets 403 from a missing-header request that should be 401. Pinned by three auth-branch tests.

What v1.0 won't do, on purpose

The daemon is not highly available. One daemon per host. Two running in parallel would clobber each other's crash-recovery logic. The single-daemon invariant is what lets the recovery sweep be a simple UPDATE WHERE status='running'. Adding HA means leader election or run-level ownership — both v1.1+ concerns.

There's no instant pickup. New registrations and cron-boundary fires land within ten seconds of being due. Postgres LISTEN/NOTIFY would close that gap but adds complexity that 10s polling makes unnecessary for v1.0. Most workflows run on minute-or-coarser cron expressions; 10s is rounding error.

There's no queue for missed fires. Skip-don't-catch-up is the locked behavior. If you genuinely need every fire to land, write a workflow that does its own backfill — The Brain won't second-guess your cron expression.

The HTTP endpoint isn't a public API. Single token, no CORS, opt-in, designed for known callers on the same network. Path allowlisting and per-caller scoping are v1.1+. The threat model is single-token-server-to-server; anyone with the token can execute arbitrary server-side Python by pointing the endpoint at any file on the host. The token is the only gate. Treat it like a database password.

Workflows still execute one at a time per host. Sequential within a tick. Concurrent execution is v1.1 territory and brings concurrency-control problems that need to wait for a real workload to design against.

These are deliberate trade-offs. M2 is the smallest correct unattended-runner, not the most ambitious one.

Who this is for

Same audience as M1, with one addition: anyone who needs a scheduled workflow runner they can self-host and inspect end-to-end — and who's tired of either rolling their own cron-in-a-container with no run history, or paying for a managed orchestrator that owns their data.

If you've ever written a Python script, wired it to a system cron entry, then realized a week later you have no record of which days it failed and why — this is for you.

What's next

Milestone 3 is the reactive layer — webhook triggers and file-watcher triggers. That's when The Brain stops only firing on the clock and starts firing in response to things that happen.

The full roadmap and milestone progress table live in the repo's README. Each milestone gets a dev-log post here as it ships — one of four dev.to posts across the build period.

Try it

git clone https://github.com/MihaiBuilds/the-brain
cd the-brain
docker compose up -d
docker compose exec brain brain daemon-status
docker compose exec brain brain register examples/hello.py --cron "*/1 * * * *"
docker compose exec brain brain list

Wait a minute, run brain history, and you'll see the daemon-fired run sitting in there alongside any brain run invocations from the M1 quickstart — same row shape, same inspection commands, same database. The repo has the longer version with state-across-runs and the HTTP endpoint walkthrough.

Follow along

Twitter / X: @mihaibuilds
Blog: mihaibuilds.com
GitHub: github.com/MihaiBuilds/the-brain

I built the memory, now I'm building the brain

MihaiBuilds — Thu, 28 May 2026 08:10:21 +0000

Originally published on mihaibuilds.com. Cross-posting here because dev.to is where I read a lot of work like this myself.

Three weeks ago I shipped Memory Vault v1.0 — an open-source, self-hosted AI memory layer you run yourself. Postgres + pgvector under the hood, hybrid search on top, an MCP server so Claude can read and write to it directly. The first product in a planned compounding stack.

Today the second product in that stack exists too. It's called The Brain.

I'll get to what it is in a second. First, the honest part: I didn't announce it the day I started. I built the first milestone in private, on my own, with no audience watching. Three days of focused work, ten merged PRs, then a clean stop. Build-in-public is the long-term plan for this project the same way it was for Memory Vault. But the first week was head-down, because the riskiest part of a new product isn't the announcement — it's whether the thing actually works. Now that it does, I can tell you about it without hedging.

What The Brain is

The Brain is a workflow orchestrator, not an AI agent. It runs Python-defined workflows you author, with full visibility into every step. The intelligence is in the workflow you write; The Brain is the runtime that makes it repeatable and observable. It calls LLMs as steps when needed; it doesn't replace them.

Concretely: you write a Python file that describes a sequence of steps. Each step is a shell command, a Memory Vault query, or a local LLM call. The Brain runs them top to bottom, passes output forward between them with named placeholders, and persists every run to Postgres. You inspect runs from the CLI. Successful runs exit 0; failed runs exit 1. It drops straight into cron jobs or CI pipelines.

That's the whole pitch. There's no autonomous decision-making, no agent loop, no self-direction. It runs what you tell it to run, and it records what happened.

Why this is a workflow orchestrator, not an agent

The orchestration layer is too load-bearing to depend on someone else's framework. When the framework changes, your workflows break — and these frameworks change constantly. LangChain, LangGraph, CrewAI, AutoGen: they're all moving targets, and "agent autonomy" is a moving definition. Owned runtime, owned database, owned LLM client, owned everything. Five years from now this still runs.

The other reason: build-in-public projects have an honesty constraint that pure-agent products don't. If The Brain claims to "decide" or "reason," I'd have to explain in every blog post what that means, what model it uses, and why the decision quality is what it is. Calling it a workflow orchestrator collapses that ambiguity. The user writes the logic. The Brain runs it. The output is reproducible. The behavior is auditable. The audience this is for — solo developers who use AI seriously and want their tools to be transparent — is allergic to the alternative.

What M1 ships, today

M1 is called "Bare Runner." The name is the honest scope: it's the smallest thing that proves The Brain works end-to-end.

Run Python-defined workflows from the CLI with brain run path/to/workflow.py. A workflow is a plain Python file exposing a module-level workflow = Workflow(...). Loaded with importlib and validated at load time via Pydantic.
Three step types: ShellStep (subprocess + timeout), MemoryVaultStep (Memory Vault REST), LLMStep (OpenAI-compatible HTTP against LM Studio). Each lives in its own executor class; the runner dispatches by step type with no isinstance chains.
Placeholder substitution — steps pass output forward with {step_name} tokens in any string field (prompt, command, query). Strict: a placeholder that names no prior completed step fails THAT step with a clear error. Fail fast; never pass literal braces downstream.
Persistent run history in Postgres — every run, every step, every output, every error. One workflow_runs table; the run's full step-by-step output is stored as a JSONB array (not an object — JSONB doesn't preserve key order, and execution order is part of the data).
CLI introspection — brain history lists past runs with --limit/--workflow/--status filters; brain show <run_id> shows full step-by-step detail for one run. Run IDs match by prefix (Memory Vault's token revoke precedent).
Strict failure semantics — a workflow halts on the first failed step; the run row always lands in Postgres with a terminal status, even if an executor raises unexpectedly. The runner catches every executor exception and persists. A run that started always ends with a terminal DB row; no exception escapes unpersisted.
One-command Docker — docker compose up -d brings up Postgres and The Brain together, migrations run on boot via a hand-rolled migration runner in src/db.py.
46 hermetic tests — pytest with a real Postgres test container, MV and LLM HTTP faked via httpx.MockTransport (built-in, no respx dependency). The suite is fast, deterministic, and runs anywhere with no external services.

A run looks like this:

$ docker compose exec brain brain run examples/hello.py
Running workflow 'hello' (2 steps)
  ✓ greeting
  ✓ echo_it_back
Run c609f5e0 — success

And inspecting it after:

$ docker compose exec brain brain show c609f5e0
Run:      c609f5e0-a8d6-4221-84c0-58c0b5d0460d
Workflow: hello
Status:   success
Started:  2026-05-22 19:54:58
Duration: 0.0s

Steps:
  ✓ greeting
      Hello from The Brain
  ✓ echo_it_back
      The previous step said: Hello from The Brain

Architectural decisions worth naming

Functional/declarative workflow files, not class + decorator. A workflow is a data structure: workflow = Workflow(name=..., steps=[Step(...), Step(...)]). Easiest to introspect, easiest to serialize, easiest to register for cron in the next milestone. Class-with-decorators looks ergonomic at first and gets in the way the moment you try to load workflows dynamically. The declarative form is what every workflow tool I respect converges on for a reason.

Single workflow_runs table for M1, per-step granularity deferred to M2. The whole run's step-by-step output goes in one JSONB column. Yes, a per-step table is the "right" long-term schema. But M2 is where state-between-runs lands, and that's the milestone where it actually pays for itself. Shipping the right table in M1 would be carrying schema complexity for a feature M1 doesn't have. Defer it; revisit when the use case lands.

Thin in-repo Memory Vault REST client (~30 LOC), no shared library. The Brain talks to Memory Vault over HTTP. I could extract a shared mihaibuilds-clients library now. I'd be over-engineering for a future I haven't reached. The right time to extract a client library is when there are three or more callers — not when there's one. Right now the entire client is httpx.post(...). When The Brain plus two or three addons all talk to Memory Vault, the duplication will tell me it's time to extract.

LM Studio only in v1.0, not LM Studio + Ollama. This is the explicit lesson I'm carrying from Memory Vault. Memory Vault's marketing claimed both LM Studio and Ollama support; only LM Studio was end-to-end tested. The Brain ships LM Studio only in v1.0. Ollama probably works through the same OpenAI-compatible client shape, but "probably works" isn't a release guarantee. Only claim providers you've actually tested. This rule survives every product I build.

Owned runtime, not LangChain/LangGraph/CrewAI wrapper. Already covered above — but worth re-stating in the architecture section because it's the decision the rest of the codebase shape derives from. The Brain is ~1,500 lines of Python. A LangChain wrapper would be more code, more dependencies, and a runtime that breaks every time the upstream framework changes its API. Owned runtime is the simpler answer, not the more ambitious one.

What v1.0 won't do, on purpose

No autonomous decision-making. The Brain runs the workflow you defined. It doesn't pick a different step at runtime. If you want branching, you write a workflow that branches. Rich conditional logic is in the v1.0-out section deliberately.

No multi-user / team workflows. Single-tenant by design. Multi-user activation lives behind a PRO tier later.

No managed cloud. Self-hosted, MIT-licensed, runs on your laptop or your VPS. Always.

No visual workflow builder. The workflow file is the source of truth. You read it like Python, you diff it like Python, you grep it like Python. Visual builders are a PRO concern, not a v1.0 concern.

These are deliberate trade-offs. The Brain v1.0 is the smallest correct version, not the most ambitious one.

Who this is for

Developers who run real workflows on their own machines and want LLMs as a step inside those workflows — not as the thing in charge. Solo builders stitching together memory, models, and shell tools who are tired of agent frameworks that change their API every quarter. Anyone who wants every run to be inspectable, every output persisted, and every decision their own to make.

If you've ever written a Python script that calls an LLM, then bolted on a cron entry, then realized you have no record of what it did yesterday — this is for you.

What's next

Milestone 2 is triggers and state — cron schedules, a long-running scheduler daemon, and workflows that read the previous run's output. M2 is the milestone where The Brain becomes worth running unattended.

The full roadmap and milestone progress table live in the repo's README. Each milestone gets a dev-log post here as it ships — one of four dev.to posts across the build period.

Try it

git clone https://github.com/MihaiBuilds/the-brain
cd the-brain
docker compose up -d
docker compose exec brain brain run examples/hello.py

The repo has the full quickstart with configuration, Memory Vault wiring, and the real-world digest example (recent memories → local LLM summary → markdown file, all in one Python file).

Follow along

Twitter / X: @mihaibuilds
Blog: mihaibuilds.com
GitHub: github.com/MihaiBuilds/the-brain

Memory Vault v1.0 — building open-source AI memory the boring way

MihaiBuilds — Sat, 09 May 2026 13:15:56 +0000

Originally published on mihaibuilds.com. Cross-posting here because dev.to is where I find a lot of this kind of work myself.

For the past year I kept hitting the same wall. I'd have a real conversation with Claude — work through a database design, debug something gnarly, agree on a convention I wanted to keep — and the next morning it was gone. Not summarized. Not searchable. Just gone. ChatGPT was the same. Every assistant I used had the long-term memory of a goldfish, and the workaround the industry settled on was "paste the relevant context back in every time." That's not memory. That's me being the memory.

So I built one. Memory Vault is an open-source, self-hosted AI memory system you run yourself: Postgres with pgvector underneath, hybrid search on top, an MCP server so Claude can read and write to it directly, a knowledge graph that extracts entities without an LLM bill, a local LLM chat with retrieved-source citations, and a one-command Docker setup. Two days ago it crossed the line from "build-in-public project" to "v1.0 stable release." (v1.0.2 yesterday closed two security findings I caught after enabling branch protection — path-traversal + info-exposure on an internal stream handler.)

What Memory Vault is

A long-term memory layer for AI assistants and the apps you build on top of them. You ingest text — markdown notes, conversation logs, anything plain — and it gets chunked, embedded, full-text indexed, and stored in a single Postgres database. Hybrid search (vector similarity + keyword tsvector + Reciprocal Rank Fusion) returns the right chunks back when you query. An MCP server exposes four tools (recall, remember, forget, status) that Claude Desktop or Claude Code can call directly, which means Claude can read and write to your memory inside any conversation without you copy-pasting context. A REST API exposes the same operations for any app you build. A dashboard gives you a Search, Browse, Graph, Ingest, Stats, and Chat page. A local LLM chat (LM Studio in v1.0) lets you talk to your memories with full source citations — every response shows which chunks it pulled from, clickable.

It runs entirely on your machine. No API keys. No cloud. No telemetry. Postgres on port 5432, the API on port 8000, dashboard on the same port. docker compose up and it's running.

What v1.0 actually does

Hybrid search — pgvector HNSW for semantic + tsvector GIN for keyword + Reciprocal Rank Fusion to merge them. Vector-only search misses exact terms; keyword-only misses paraphrases. RRF gets both.
MCP server — four tools (recall, remember, forget, status) callable from Claude Desktop, Claude Code, or any MCP client. Claude reads and writes your memory in-conversation.
Knowledge graph — spaCy NER plus co-occurrence extracts entities (Person, Project, Tool, Concept) and related_to relationships from every ingested chunk. No LLM, no per-token cost, rendered as an interactive Cytoscape force-directed graph.
Memory spaces — namespacing for different contexts (work, personal, projects). Per-space dedup; cross-space isolation by default.
Local LLM chat — LM Studio native API with sources panel showing retrieved chunks for every answer. Every response is grounded and the grounding is visible.
REST API — bearer-auth-protected, OpenAPI-documented at /docs, every operation the dashboard does is also a documented endpoint.
One-command Docker — docker compose up. Postgres, the app, and the spaCy model bundled into a single image at build time, no first-run download.
Self-hosted, MIT-licensed — your data stays on your machine. The whole thing is yours.
170 tests passing — pytest with a real Postgres + pgvector service container, no mocks of the database.

Architectural decisions worth naming

Postgres + pgvector instead of a dedicated vector database. I run one database, not two. Operationally this matters more than the marginal performance of a purpose-built vector store at small scale. You already know how to back up Postgres. You already know how to monitor it. HNSW indexes plus tuned maintenance_work_mem and ef_search get you to "fast enough for hundreds of thousands of chunks on a laptop." When that stops being true, the migration path is sane. Until then, one database is the right answer for a self-hosted personal-memory tool.

Hybrid search instead of vector-only. Pure vector search is great at paraphrase and concept. It's bad at exact terms — model names, error codes, file paths, anything where the literal string is the signal. Memory Vault stores both an embedding and a tsvector for every chunk and merges the two ranked result sets with Reciprocal Rank Fusion. RRF is parameter-free, doesn't require score normalization, and consistently beats either approach alone on the kind of mixed queries real users actually type.

spaCy + co-occurrence for the knowledge graph, not an LLM. The default move in this space is to feed every chunk through an LLM and ask it for entities and relationships. It works. It also costs money on every ingest, couples your graph quality to whichever model you happened to pick, and requires API keys for a tool whose entire pitch is no API keys. spaCy's en_core_web_sm model plus a co-occurrence rule (two entities in the same chunk = a related_to edge, weighted by frequency) gets you a useful graph for zero per-ingest cost. The honest limits — English only, context-dependent NER, no fuzzy matching — are documented up front rather than masked.

MCP-first, not REST-first. Memory Vault was designed around the assumption that the primary user of this database is going to be Claude, not me. The MCP server isn't a wrapper around a REST API — it's a direct path into the same code that the REST API uses. Both are first-class. But the design starting point was "what does Claude need to call to make memory feel native," and then the REST API was the same operations exposed for human-driven apps. That ordering changes which tradeoffs are interesting.

The PoolClosed story

About a week before tag day, I added a CLI command called memory-vault diagnose. It bundles app logs, database logs, status output, OS info, and redacted environment into a zip file users can attach to bug reports. Foundation work. Paid for once. The kind of thing that makes every future bug report ten times higher signal-to-noise.

I shipped it. Then I ran the test suite. 163 passed, 52 errored. Every error was psycopg_pool.PoolClosed.

First instinct: probably an httpx lifespan thing. Modern httpx has changed how it handles ASGI lifespan events between minor versions. The test suite uses httpx.ASGITransport to drive the FastAPI app in-process, sharing a session-wide connection pool fixture. If the transport was firing shutdown events between tests, the pool would close mid-suite. There's a kwarg for this. I added lifespan="off" to the transport. TypeError: ASGITransport.__init__() got an unexpected keyword argument 'lifespan'. The kwarg doesn't exist in 0.28.x. Reverted.

Second instinct: walk the call graph. memory-vault diagnose calls into the CLI's _run_status helper to capture status output for the bundle. _run_status was implemented as asyncio.run(_cmd_status()) — directly calling the CLI's status function in-process. _cmd_status initializes a connection pool at the top of the function and closes it via a finally block at the end. Which is correct behavior for the CLI. It's also exactly what you don't want when something else in the same process — like a session-wide test fixture — already owns a pool that's mid-flight.

The fix was four lines. Replace the in-process asyncio.run with subprocess.run(["memory-vault", "status"]). The subprocess gets its own pool, lives its own lifecycle, exits cleanly, and the parent process's pool is never touched. 163 passed, 0 errored.

The lesson isn't about pools or fixtures specifically. It's that "obvious" fixes (changing the test transport config) and root causes (one function quietly tearing down state owned by a different function) live in different parts of the code. The lifespan="off" move would have masked the symptom in the tests and left the actual bug in the CLI, where users would have hit it. Almost the entire week's gap between "all my sub-steps look done" and "v1.0 is actually shippable" was the discipline of not bypassing this kind of thing when bypassing was easy.

What v1.0 doesn't do, on purpose

English-only NER. The bundled spaCy model is en_core_web_sm. Non-English content gets little to no useful entity extraction. Multilingual models exist; they're heavier and slower; they're a question driven by real user demand, not a v1.0 must-have.

No fuzzy entity matching. "PostgreSQL" and "Postgres" are separate entities in the graph. No alias merging in v1.0.

No re-extraction on edit. If you re-ingest a corrected version of a chunk, the new entities are added but the old ones aren't cleaned up.

Single-user. v1.0 has bearer auth and one user behind it. The schema has owner_id and access_level columns from day one, but multi-user activation is part of the PRO tier.

LM Studio only for chat. Ollama and llama.cpp use the same OpenAI-compatible client architecture under the hood, but the only end-to-end-tested path in v1.0 is LM Studio. Ollama support is not in v1.0.

No multi-conversation history in chat. Single-thread chat. Driven by whether real users ask for it.

These are deliberate trade-offs. Honest gaps documented up front build more trust than feature bullets that fall apart when someone actually tries them.

The open-core model

Memory Vault is and will always be MIT-licensed. The whole thing — search, MCP, graph, REST API, dashboard, local LLM chat, ingestion pipeline, the database schema, the Docker setup. You can run it on your machine. You can fork it. You can use it inside a commercial product. The free tier is genuinely useful — not a crippled demo of the paid tier.

A paid PRO tier is on the roadmap for teams: dedup with importance decay, conflict resolution and supersede chains, multi-user activation, additional adapters (PDF, web pages), automated encrypted backups, and a fuller dashboard with analytics. The PRO tier is genuinely paid features — operational tools that solo users on a laptop don't strictly need, and teams running shared knowledge bases really do. The split is honest by design.

What this took to build

Seven weeks of evenings and weekends across nine locked milestones, scope frozen on March 27. M1 was the announcement. M2 the core hybrid search. M3 the one-command Docker. M4 the MCP server. M5 the REST API. M6 the dashboard. M7 the knowledge graph. M8 — this one — was local LLM chat plus the polish, CI/CD, security review, and release engineering that turn a build-in-public project into something other people can actually use.

Two of those weeks were the kind of work nobody sees: structured JSON logging with request ID propagation, a diagnostic CLI that produces a redacted bundle for bug reports, GitHub Actions for lint and test and multi-arch Docker release, security audit (bandit, npm audit, Dependabot, CodeQL, plus a 15-test pentest pass with curl), Contributor Covenant Code of Conduct, threat model in SECURITY.md, branch protection rules, and the discipline to fix the actual root cause of a test failure instead of bypassing it. Unglamorous. Also the difference between v0.7 and v1.0.

What's next

Beyond. Memory Vault is the first product in a planned compounding stack — The Brain is the next layer, building agents on top of this memory infrastructure. The memory layer is the one that has to be solid first. Today it is.

Try it

git clone https://github.com/MihaiBuilds/memory-vault
cd memory-vault
cp .env.example .env
docker compose up -d

Open http://localhost:8000 and you're running.

GitHub — latest release
README and quick start
MCP setup for Claude Desktop / Claude Code
Questions and bug reports: GitHub Issues
General discussion: GitHub Discussions

Credits

Three Postgres tuning tips landed during M6 and M7 that materially improved Memory Vault: @rivestack on maintenance_work_mem, ef_search as a runtime knob, and post-deploy cache warmup for HNSW indexes. The first ships in v1.0; we'll use the others when we get to them. Public credit, fair credit. Build-in-public works because builders with deeper expertise see what you're shipping and tell you what's wrong before production does.

Beta tester Inevitable-Way-3916 ran the dashboard early, asked the architecture questions that forced the ARCHITECTURE.md doc to exist, and put bulk ingest on the list. Thanks.

Follow along

Twitter / X: @mihaibuilds
Blog: mihaibuilds.com
GitHub: github.com/MihaiBuilds/memory-vault