Building an AI agent is easy. Building the system around it is where things get complicated.
You can create an agent in a few lines of Python. Then you need an API, persistence, authentication, tool integrations, RAG, scheduling, observability, a UI, deployment, and a way to operate the whole thing. That's the gap I wanted to explore when I started digging into apowerb, an open-source AI agent framework from thaink².
Rather than treating it as another agent SDK, I wanted to answer a more specific question:
What does an AI agent runtime actually need to provide once the prototype becomes a real system?
This article walks through apowerb's architecture, its runtime model, RAG engine, Text-to-SQL support, event-driven execution, deployment model, and how it compares with other popular approaches.
apowerb
/
apowerb
The open-source agentic framework to build, orchestrate, and operate production AI agents.
Build, run and govern AI agents that work while you don't.
The open-source agentic framework to build, orchestrate, and operate production AI agents.
Documentation • Quickstart • API Reference • Deployment • thaink2
apowerb is an open-source platform to build, run and govern AI agents in production. It covers multi-agent orchestration, RAG and Text-to-SQL, MCP servers and built-in business tools, email- and schedule-triggered runs, token quotas and session audit. You can self-host it with Docker Compose or Helm. The core is Apache-2.0; a few enterprise modules (billing, identity-provider sign-in, MFA agent evaluation, the supervision screen) are sold separately by thaink².
- Build agents from the web interface or the REST API — instructions, model tools, knowledge bases — and compose them: sequential, parallel, loop, sub-agents.
- Connect them to your systems: any MCP server, plus built-in tools for Google Workspace, Microsoft 365, GitHub, HubSpot, Pipedrive, Odoo, SQL databases Oracle, MongoDB, S3,…
What is apowerb, exactly?
Most agent frameworks start from a library. You import the framework, define an agent, give it tools, connect an LLM, and build your application. That model is useful, but it leaves a lot of infrastructure outside the framework: API layer, persistence, authentication, UI, deployment, scheduling, event handling, observability, RAG, integrations.
apowerb takes a different approach. Instead of treating the agent as an object embedded inside an application, it treats the agent as something that can be defined, stored, materialized and operated by a runtime. The stack:
- FastAPI : REST APIs and Server-Sent Events
-
Typer : the CLI (
apowerb serve,apowerb agents list, …) - Next.js : the web UI (build agents, give them tools/knowledge, run and inspect them)
- PostgreSQL : agent and configuration state
- Google ADK : the execution engine
- LiteLLM : the model routing layer
- Docker Compose / Helm : deployment
The result is closer to an application runtime than to a standalone Python library.
The architecture in one picture
┌────────────────────────────────────────────────────────┐
│ Clients / UI │
│ Next.js UI • Typer CLI • REST API • SSE │
└──────────────────────────┬─────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────┐
│ FastAPI Orchestrator │
│ Agent materialization • Webhooks (Pub/Sub) │
│ Cron / event scheduling • SSE output stream │
└──────────────────────────┬─────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────┐
│ Execution Engine Google ADK │
│ Base • Sequential • Parallel • Loop │
│ Hierarchical sub-agents • Human-in-the-loop │
└─────────────┬───────────────────────────┬──────────────┘
│ │
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ LiteLLM Model Router │ │ Tools & Services │
│ OpenAI • Anthropic │ │ M365 • Google • SQL │
│ Gemini • Mistral • local│ │ RAG • Webhooks │
└─────────────────────────┘ └─────────────────────────┘
Four orchestration patterns are built into the runtime:
- base : single-turn conversational agents with tool execution
- sequential : deterministic step-by-step pipelines passing context downstream
- parallel : concurrent branches aggregated for synthesis
- loop : evaluator/optimizer cycles that iterate until a target condition or stop token
Hierarchical sub-agents let a coordinator delegate work to specialized agents. The interesting part, though, isn't the list of patterns, it's what happens before execution.
The interesting choice: agents as runtime objects
In a typical application, an agent is defined directly in Python and the application owns that definition:
agent = SomeAgent(
model="...",
tools=[...],
instructions="...",
)
apowerb takes another route. Agent definitions, tools, skills and their configuration live in PostgreSQL. At startup, the backend uses those definitions to materialize executable Python modules.
Agent definition ─► PostgreSQL ─► Materialization ─► Python module ─► Google ADK
├─► Tools
├─► RAG
└─► LLM
That changes the operational model. The application isn't rebuilt every time the agent catalogue changes; the runtime loads definitions from the database and turns them into executable agents. For an organisation managing many agents, that's a meaningful distinction, the agent becomes closer to a runtime-managed resource than a static piece of application code. It's the detail I found most interesting in the project.
Model access through LiteLLM
The execution layer is deliberately separated from model access. apowerb uses LiteLLM as the routing layer, so the runtime can work with multiple providers without coupling an agent's tools to one vendor. The documented stack includes Anthropic, OpenAI, Mistral, Gemini, OVHcloud, Groq, and local Ollama/vLLM deployments.
Agent ─► Google ADK ─► LiteLLM ─┬─► OpenAI
├─► Anthropic
└─► Local (Ollama / vLLM)
The tools and orchestration logic don't need to know which provider ultimately executes the model call, useful for teams that want to swap or benchmark models.
RAG is part of the stack
RAG isn't treated as an external SaaS dependency. apowerb includes th2rag, a standalone FastAPI service for ingestion and retrieval:
Document ─► Conversion (Docling) ─► Chunking ─► Embeddings ─► LanceDB ─► Retrieval ─► Generation
The current implementation uses Docling for document conversion, its HybridChunker for semantic chunking, sentence-transformers/gtr-t5-large for embeddings, LanceDB as the vector store, a LanceDB retriever for similarity search, and Mistral AI for generation. Each stage sits behind a base class (BaseEmbedding, BaseGenerator, …), so swapping the embedding model or the LLM is a matter of subclassing, not surgery.
The service can index local files, URLs, database query results and S3 objects, into knowledge bases associated with individual agents rather than one global index. That makes RAG a runtime capability instead of something each application assembles separately.
Native Text-to-SQL
An agent can connect to PostgreSQL or MySQL, introspect the schema, generate SQL from a natural-language question, execute it and return the result:
"Which customers generated the most revenue last quarter?"
│
▼
Agent / LLM ─► Schema context ─► SQL generation ─► Query ─► Results ─► NL response
The interesting part isn't that an LLM can write SQL, it's that the connection, schema introspection, generation and execution all happen inside the agent runtime. Text-to-SQL becomes another runtime capability rather than an integration every application builds from scratch.
Event-driven agents
Most agent demos are request/response:
HTTP request ─► Agent ─► Response
apowerb can also work in the opposite direction:
External event ─► Webhook ─► Agent ─► Tools / RAG / LLM ─► Action / Result
The current implementation includes email-triggered agents. Gmail uses Google Cloud Pub/Sub push notifications; Outlook uses Microsoft Graph change subscriptions with clientState validation.
Email arrives ─► Provider notification ─► apowerb webhook ─► Fetch message ─► Run agent ─► Log result
Subscription renewal is handled in the background: Gmail watches expire after seven days, Outlook after three, and the runtime periodically renews anything approaching expiration. Combined with cron-style scheduled runs and SSE streaming (token-by-token output, RAG ingestion progress, notifications), this moves the runtime beyond chatbots, the agent becomes a component in an event-driven system.
The open-source boundary
The core stack is Apache 2.0 ; the framework plus supporting services like the RAG engine, ETL components, observability and web UI. The project follows an open-core model: some commercial extensions (billing, MFA, identity-provider SSO, agent evaluation, supervision, organisation management) are maintained separately.
A TH2_EXTENSIONS environment variable declares which commercial bricks to load; if a brick isn't installed or enabled, the core doesn't depend on it. Routes belonging to an absent brick return a clean 404. The useful distinction: the open-source runtime remains fully usable without the commercial extensions, the core isn't an artificially limited demo.
It's more than one repository
A common pattern in AI tooling is "open-source framework + several proprietary services around it." Here, much more of the surrounding infrastructure is exposed. The thaink² GitHub org holds 10 repositories, all Apache 2.0:
| Repository | Role | Technology |
|---|---|---|
apowerb |
FastAPI backend, CLI and ADK runtime | Python |
apowerb-ui |
Visual agent builder and monitoring | Next.js |
th2rag |
RAG, parsing, embeddings, vector search | LanceDB / Docling / Mistral |
th2etl |
ETL pipelines | Python |
th2pulse |
Observability and logs | OpenTelemetry |
th2forecast |
Time-series forecasting | R |
apowerb-hosting |
Docker Compose / Kubernetes / Helm | — |
apowerb-docs |
Documentation | Mintlify / MDX |
agent-hub |
Community agent templates | — |
You can read and run the RAG engine, the scheduler and the telemetry pipeline, there's no hidden proprietary service doing the heavy lifting behind an API.
Run the full stack
The fastest way to understand the project is to run it. The hosting repository provides a Docker Compose setup:
git clone https://github.com/apowerb/apowerb-hosting.git
cd apowerb-hosting
cp .env.example .env
./scripts/generate-secrets.sh
docker compose \
-f docker-compose/docker-compose.yml \
--env-file .env \
up -d
The UI comes up on http://localhost:3000, the API on :8000, and PostgreSQL runs inside the stack. You still provide an LLM API key, through the UI, or via DEFAULT_LLM_MODEL / DEFAULT_LLM_API_KEY. If you only want the Python package:
uv add apowerb # or: pip install apowerb
Two entry points, two purposes:
apowerb package ─► embed the runtime in your application
apowerb-hosting ─► run the complete platform
How it compares
Comparisons between agent frameworks are easy to get wrong because they solve different problems. The useful question isn't "which framework is best?" but "which layer does each project expect you to own?" Below, "built-in" means the capability is part of the platform rather than something you assemble around it.
| Capability | apowerb | LangGraph | CrewAI | n8n + LLM |
|---|---|---|---|---|
| Multi-model | LiteLLM | Yes | Yes | Partial |
| Built-in RAG | th2rag | Yes | Partial | Partial |
| Email webhooks | Gmail + Outlook | No | No | Yes |
| Native Text-to-SQL | Yes | Via tools | No | Partial |
| Included UI | Next.js | No | No | Yes |
| Helm deployment | Yes | DIY | DIY | Yes |
| Main approach | Runtime / platform | Framework | Framework | Workflow automation |
The distinction isn't one checkbox, it's the architecture. LangGraph and CrewAI are frameworks you embed into an application. n8n comes from workflow automation, where an LLM is one node in a larger graph. apowerb provides the runtime around the agent: API, persistence, UI, integrations, scheduling, RAG and deployment. Different design choices, not just different implementations of the same product.
What I found interesting
1. Agents are treated as managed runtime resources. The database-backed definition and materialization model is the most distinctive architectural choice.
2. RAG isn't an external dependency. The project ships its own RAG service instead of requiring every deployment to assemble one.
3. Event-driven execution is first-class. Gmail, Outlook, schedules and SSE make "proactive" agents implementable, not just request/response chatbots.
Trade-offs
Not everything is settled. The release cadence is high, so the API isn't frozen, expect to track changes if you build on it now. Schema is created by ensure_* functions at startup rather than Alembic, which is fine today but means no reversible migrations. And the external community is still small; most contribution looks internal so far, so you'd be an early adopter.
None of that is a dealbreaker, it's the normal shape of a young project that's visibly running in production.
Worth a clone if…
- you want RAG, tool auth, an API and a UI to already exist so you can build the actual agent;
- you're a data team and Text-to-SQL over your warehouse plus scheduled runs maps to your use case;
- you need everything self-hosted and auditable, your models, your data, Apache 2.0;
- you're benchmarking agent frameworks and want to see where the "runtime, not library" line falls.
Three experiments that take minutes each: point an agent at a Postgres and query it in English; drop a folder of PDFs into a knowledge base and ask questions; set a Gmail trigger that summarizes and routes incoming mail.
Links
- Core: github.com/apowerb/apowerb Apache-2.0
- Web site: thaink2.com/opensource
- Deployment: apowerb/apowerb-hosting
- Docs: docs.apowerb.com quickstart, API reference
If you try it, I'd be curious what you build drop a note in the comments.
Top comments (10)
The runtime-centric choice is interesting, most frameworks I've seen stop at the library level. One question on th2rag: when Docling ingests a PDF, do you chunk on the recovered heading hierarchy or on fixed token windows? I spend most of my time on PDF structure and the difference is not subtle. On an IPCC report today the font names were plain wrong (the body font called itself Bold) and every converter I tried flattened the eighteen headline statements into the body text, so any chunker keyed on headings would have missed the document's whole skeleton. Curious how much of that you see on the documents your users actually upload.
Great question, and you're pointing right where it hurts. th2rag uses Docling's HybridChunker: structure-aware first (it keys on the recovered heading hierarchy), then token-refined. Not fixed windows.
We've run it in production on 50+ documents with more 2500+ pages with mixed content (text, tables, images). That was mostly a test of ingestion robustness (Docling conversion plus embedding at that size without falling over), and on well-structured docs the recovered hierarchy holds up well. But your IPCC example is the real failure mode: a structure-aware chunker is only as good as the structure Docling recovers upstream. If the parser flattens those headline statements into the body, the skeleton is gone before chunking even starts. That's a parsing-layer failure, not a chunking one, and lying font metadata is exactly the kind of input we don't handle gracefully yet.
You clearly live in this: on documents with wrong font metadata, are you correcting at the parser level, or reconstructing hierarchy from layout and font-size heuristics?
Neither, in the end. I stopped reading the declared weight because on that file it lies in both directions: the body font is named "FrutigerLTPro-Bold" and is a regular condensed face, the eighteen statements are named "Light" or "Regular" and are the bold ones. pdf.js exposes no weight independent of the name, so "bold means heading" is dead on arrival.
So I measure instead: average advance per glyph for each font id on the rendered page (body ~0.39 em per character, statements ~0.50), then decide on font difference rather than weight. Per page, the dominant font is the body; a family of three or more numbered multi-line paragraphs set in one uniform non-body font at body size gets promoted. Size alone never worked there, the statements are at body size, which is exactly why everyone flattened them.
It still fails on an italic emphasis family, and earlier this month my own glyph estimate bit me on US legislative texts (assumed 0.5 em, capitals sit at 0.75, headings became table cells). If Docling exposes per-page font clusters before chunking, that is where I'd plug in.
Really useful, thank you for spelling it out.
The advance-width-per-glyph approach is clever because it sidesteps the metadata entirely. Deciding on font difference rather than absolute size is the key insight, the statements sitting at body size is exactly why size-based heuristics flatten them.
On your Docling question: no, it doesn't expose per-page font clusters. You get a DoclingDocument where each text item already has a label (title, section_header, text, ...) from its layout model, plus prov with bounding boxes and page number. The heading-vs-body decision is already made upstream by the time we see it, so if the layout model misreads your eighteen statements as body, the chunker just inherits the wrong label. The skeleton is lost before we can touch it.
So the right hook for your heuristic is between glyph extraction and layout labeling, before the structure is committed, not the chunker.
Honestly, you clearly know this space far better than we do. It's all open source (Apache 2.0): the parsing and structure recovery lives in th2rag, and the runtime that consumes it in the main repo. This is exactly the kind of contribution that would make it meaningfully better. Would you want to open an issue or a PR? I'd be glad to point you to the right place to plug in across both:
RAG / parsing: github.com/apowerb/th2rag
Core runtime: github.com/apowerb/apowerb
Thanks, that clears it up. The label being committed before the chunker sees anything is what I suspected, so the chunker is the wrong place and I'll stop looking there.
I had a look at pdf_converter.py and at Docling's pipeline options before answering, because I did not want to propose something that fights the framework. Two things I noticed. Docling on main has a heading_hierarchy_options block, off by default, that infers heading levels from bookmarks, numbering and font style. It only touches items the layout model already labeled section_header, so the eighteen statements never get there. The other thing is generate_parsed_pages: set it to True and the parsed cells survive assembly, each with a bounding box and a font. That is the raw material my heuristic needs, and it sits one level below the DoclingDocument without patching Docling itself.
So the shape I have in mind is a post-pass in th2rag, right after convert_with_docling returns. Per page, take the dominant body font from the parsed cells. Find multi-line runs at body size set entirely in a different face. Match them by bbox to TextItems labeled "text" and relabel those as section_header before the HybridChunker runs. Per page matters: in that PDF the font ids change from one page to the next for the same glyphs, and the names lie anyway (the body face is called "Bold", the statements come out as "LightCn", "MyriadPro-Regular" or "Light" depending on the page). The difference is the signal, never the name.
I'll open an issue on th2rag with that outline and the public PDF that breaks it, the IPCC AR6 Synthesis Report SPM, the case I fixed on my side this week, so you can reproduce it before anyone writes code. On the PR I'd rather be straight with you: my implementation is TypeScript inside a commercial codebase, so porting it is real work and I can't promise a date. If the issue reads well to you and the post-pass location makes sense, I'd like to do it properly rather than half-way.
Good catch. You've found the right seam, and I just checked our side against it. Two concrete things:
First, generate_parsed_pages is trivial for us to turn on. Our get_pipeline_options sets do_table_structure, do_ocr and image options but not that flag, so today the parsed cells with their bbox and font don't survive assembly. Flipping it on in that function is a safe, isolated change.
Second, and this matters for where your post-pass lives: we currently return result.document, not the full result. So the parsed pages get dropped the moment convert_with_docling returns. Your relabeling pass has to sit inside that function, right after converter.convert(...) returns and before we reduce to .document, because that's the only point where both the raw font signal and the assembled TextItems exist at the same time. That's exactly the reconcile-by-bbox step you described: per page, dominant body font from the parsed cells, find the multi-line runs at body size in a different face, match by bbox to items labeled text, relabel to section_header before the HybridChunker runs. The per-page point is well taken, we'd read the font per page rather than globally.
Please open the issue with that outline and the AR6 SPM PDF. A reproducible failing case on a public document is worth more than any discussion, and it lets anyone verify the fix independently.
On the PR, don't worry about the timing and thank you for being straight. Porting TypeScript into a Python codebase is real work and I'm not holding you to a date for something you're offering for free. The issue itself, the outline plus the reproducer, is already a serious contribution: it gives us a documented approach and a test case even if neither of us writes the code this month. If you later want to do the port properly, great; if it stays a well-specified issue someone else picks up, also great. Either way, done right beats done fast.
One practical note for when you open it: external contributions go through a CLA, I'll drop the link on the issue so it's not in your way now.
Issue is up: github.com/apowerb/th2rag/issues/6
Two things changed while writing it, because I ran the released Docling (2.69.1) on the PDF before posting instead of trusting my notes. The eighteen statements do not come out as text items, they come out as list_item, same as the A.1.1 paragraphs under them, so the relabeling pass has to accept ListItem and swap in a SectionHeaderItem rather than flip a label. And docling-parse reads this file's font names correctly: the statement lines are FrutigerLTPro-Bold and the body is FrutigerLTPro-Condensed, at the same 8.4 cell height. The lying names I described were pdf.js's reading of the same file. The rule survives either way, since it keys on "not the page's dominant face" and never on the name, but I did not want the issue to carry a claim that anyone running your stack would fail to reproduce.
I also ran the heading-hierarchy stage on Docling main against the same file: it only promotes a list_item on a bookmark match, this PDF has none, and the eighteen statements come out untouched, so the post-pass in convert_with_docling is still the place. I opened the upstream side too, as a feature request for that stage, with the same reproducer: github.com/docling-project/docling.... Details, numbers and the script are in both issues.
Awesome
Spot on. Storing agent definitions in Postgres to materialize them dynamically is brilliant. Def testing the Docker setup. Thanks for sharing!
Some comments may only be visible to logged-in visitors. Sign in to view all comments.