When I started building omadia — an open-source (MIT), self-hostable runtime for composing AI agents out of plugins — I assumed the hard part would be the model: prompting, tool-calling, getting reliable output.
It wasn't. The LLM bits were mostly solved problems. The parts that ate my time were isolation, data flow, and UX. Here are four lessons I wish I'd internalised earlier.
The project is open source (MIT) and self-hostable — code and docs here: https://github.com/byte5ai/omadia
1. Memory isolation is a design problem, not a setting
With a single agent you never notice. With several agents sharing a memory store, context bleeds: agent A starts "remembering" facts only agent B was ever told. It's subtle, it's intermittent, and it erodes trust fast — especially once different agents serve different users or departments.
What worked: give each agent (and each orchestrator) its own memory namespace and its own slice of the knowledge graph, and route every inbound message to the correct agent's namespace at the point where the channel binding resolves — not later, deep in the agent logic.
// simplified: scope every read/write to the agent that owns the turn
const mem = memoryFor(agent.id); // isolated namespace + KG slice
await mem.write(turn);
const context = await mem.recall(query);
The tradeoff: deliberate cross-agent sharing becomes the awkward case you have to design explicitly. That's the right default — isolation by construction, sharing by intent.
2. The data leaks through tool results, not the prompt
Everyone talks about prompt injection. The leak that actually bit me was on the way out: tool results.
A tool that returns a "summary + details" blob can quietly carry fields — including PII — into a context that has no business seeing them. The model then happily surfaces them. No injection required; the data just rode along in a tool response.
The fix was a privacy guard that expands and scopes tool output per record before it ever reaches the model, so each field is gated rather than passed through wholesale. The mental shift: treat tool outputs as an untrusted boundary, the same way you treat user input.
3. Most non-engineers bounce off a powerful builder
I built a full-featured agent builder — every knob exposed. The people it was meant for, the non-engineers, found it intimidating and left.
Shipping a stripped-down default — describe the agent in plain language, wire up a couple of tools, preview it side-by-side — moved adoption more than any single feature did. The power-user builder is still there for those who want it, but it's no longer the front door.
Lesson: for a builder product, the default surface is a product decision, not a settings decision. Optimise it for the least technical user you actually want.
4. The plugin seams are the whole game
omadia splits the world into:
- channels — where an agent talks (Teams, Telegram, web chat)
- integrations — what it can act on (Microsoft 365, Odoo, Confluence, …)
- capability providers — memory, embeddings, image generation, web search
- reference agents you can fork as a starting point
Getting those boundaries right is what makes the system composable instead of a monolith. Getting them wrong early was my single biggest source of rework. Spend the time on the seams before you have ten plugins fighting them.
If you want to see how the seams are drawn in practice, the code is here: https://github.com/byte5ai/omadia
Honest status
omadia is pre-1.0 / public preview. It works and we run it in production, but DB schemas can still break between minor versions and the upgrade path is hand-rolled today. It's single-tenant and self-hostable: run it on your own infrastructure — in containers (Docker) or directly on the Node stack, bring your own LLM API key, and keep all the data on your side. Stack is TypeScript/Node + a Next.js builder UI.
Two questions I'm still chewing on
I'd genuinely like your take on the two tensions at the heart of this:
- How would you scope memory between orchestrators? Strict per-orchestrator namespaces are clean, but they make deliberate cross-agent sharing the awkward case.
- Do you sanitise tool output before or after record expansion? I currently expand the "summary + details" blob first and then gate per field — but the ordering has real tradeoffs.
If you're building self-hosted agents, I'd love your feedback in GitHub Discussions: https://github.com/byte5ai/omadia/discussions/216 — and star the repo if you want to follow the public preview.
Repo and docs: https://github.com/byte5ai/omadia
Top comments (0)