Anis Meziani

Posted on Sep 24

Your agent framework stops at the demo. This open-source stack ships to production.

#ai #agents #python #opensource

Building an AI agent is easy. Building the system around it is where things get complicated.

You can create an agent in a few lines of Python. Then you need an API, persistence, authentication, tool integrations, RAG, scheduling, observability, a UI, deployment, and a way to operate the whole thing. That's the gap I wanted to explore when I started digging into apowerb, an open-source AI agent framework from thaink².

Rather than treating it as another agent SDK, I wanted to answer a more specific question:

What does an AI agent runtime actually need to provide once the prototype becomes a real system?

This article walks through apowerb's architecture, its runtime model, RAG engine, Text-to-SQL support, event-driven execution, deployment model, and how it compares with other popular approaches.

apowerb / apowerb

The open-source agentic framework to build, orchestrate, and operate production AI agents.

apowerb

The open-source agentic framework to build, orchestrate, and operate production AI agents.

Documentation • Quickstart • API Reference • Deployment • thaink2

This repository is the open-source core. Some capabilities named in the product — billing, the consumption analysis screen, prospection, identity-provider sign-in multi-factor authentication, agent evaluation, the supervision screen, organisation management — ship as separate commercial bricks and are absent here. Where the core holds a hook for one, it is documented as such. A 404 on those routes means "not in this edition", not "object not found".

The administration panel is part of this edition: users, groups, permissions, MFA enforcement. Only the management of organisations is sold separately — deciding which tenant a person belongs to governs other people's reach, rather than serving whoever runs the install.

Full documentation: docs.apowerb.com.

Quick start

Three commands, a database included, and nothing to fill in:

…

View on GitHub

What is apowerb, exactly?

Most agent frameworks start from a library. You import the framework, define an agent, give it tools, connect an LLM, and build your application. That model is useful, but it leaves a lot of infrastructure outside the framework: API layer, persistence, authentication, UI, deployment, scheduling, event handling, observability, RAG, integrations.

apowerb takes a different approach. Instead of treating the agent as an object embedded inside an application, it treats the agent as something that can be defined, stored, materialized and operated by a runtime. The stack:

FastAPI : REST APIs and Server-Sent Events
Typer : the CLI (apowerb serve, apowerb agents list, …)
Next.js : the web UI (build agents, give them tools/knowledge, run and inspect them)
PostgreSQL : agent and configuration state
Google ADK : the execution engine
LiteLLM : the model routing layer
Docker Compose / Helm : deployment

The result is closer to an application runtime than to a standalone Python library.

The architecture in one picture

┌────────────────────────────────────────────────────────┐
│                     Clients / UI                        │
│    Next.js UI  •  Typer CLI  •  REST API  •  SSE         │
└──────────────────────────┬─────────────────────────────┘
                           │
                           ▼
┌────────────────────────────────────────────────────────┐
│                 FastAPI Orchestrator                    │
│   Agent materialization   •   Webhooks (Pub/Sub)        │
│   Cron / event scheduling •   SSE output stream         │
└──────────────────────────┬─────────────────────────────┘
                           │
                           ▼
┌────────────────────────────────────────────────────────┐
│              Execution Engine Google ADK              │
│   Base • Sequential • Parallel • Loop                   │
│   Hierarchical sub-agents  •  Human-in-the-loop         │
└─────────────┬───────────────────────────┬──────────────┘
              │                           │
              ▼                           ▼
┌─────────────────────────┐    ┌─────────────────────────┐
│   LiteLLM Model Router  │    │     Tools & Services     │
│ OpenAI • Anthropic      │    │ M365 • Google • SQL      │
│ Gemini • Mistral • local│    │ RAG • Webhooks           │
└─────────────────────────┘    └─────────────────────────┘

Four orchestration patterns are built into the runtime:

base : single-turn conversational agents with tool execution
sequential : deterministic step-by-step pipelines passing context downstream
parallel : concurrent branches aggregated for synthesis
loop : evaluator/optimizer cycles that iterate until a target condition or stop token

Hierarchical sub-agents let a coordinator delegate work to specialized agents. The interesting part, though, isn't the list of patterns, it's what happens before execution.

The interesting choice: agents as runtime objects

In a typical application, an agent is defined directly in Python and the application owns that definition:

agent = SomeAgent(
    model="...",
    tools=[...],
    instructions="...",
)

apowerb takes another route. Agent definitions, tools, skills and their configuration live in PostgreSQL. At startup, the backend uses those definitions to materialize executable Python modules.

Agent definition ─► PostgreSQL ─► Materialization ─► Python module ─► Google ADK
                                                                        ├─► Tools
                                                                        ├─► RAG
                                                                        └─► LLM

That changes the operational model. The application isn't rebuilt every time the agent catalogue changes; the runtime loads definitions from the database and turns them into executable agents. For an organisation managing many agents, that's a meaningful distinction, the agent becomes closer to a runtime-managed resource than a static piece of application code. It's the detail I found most interesting in the project.

Model access through LiteLLM

The execution layer is deliberately separated from model access. apowerb uses LiteLLM as the routing layer, so the runtime can work with multiple providers without coupling an agent's tools to one vendor. The documented stack includes Anthropic, OpenAI, Mistral, Gemini, OVHcloud, Groq, and local Ollama/vLLM deployments.

Agent ─► Google ADK ─► LiteLLM ─┬─► OpenAI
                                ├─► Anthropic
                                └─► Local (Ollama / vLLM)

The tools and orchestration logic don't need to know which provider ultimately executes the model call, useful for teams that want to swap or benchmark models.

RAG is part of the stack

RAG isn't treated as an external SaaS dependency. apowerb includes th2rag, a standalone FastAPI service for ingestion and retrieval:

Document ─► Conversion (Docling) ─► Chunking ─► Embeddings ─► LanceDB ─► Retrieval ─► Generation

The current implementation uses Docling for document conversion, its HybridChunker for semantic chunking, sentence-transformers/gtr-t5-large for embeddings, LanceDB as the vector store, a LanceDB retriever for similarity search, and Mistral AI for generation. Each stage sits behind a base class (BaseEmbedding, BaseGenerator, …), so swapping the embedding model or the LLM is a matter of subclassing, not surgery.

The service can index local files, URLs, database query results and S3 objects, into knowledge bases associated with individual agents rather than one global index. That makes RAG a runtime capability instead of something each application assembles separately.

Native Text-to-SQL

An agent can connect to PostgreSQL or MySQL, introspect the schema, generate SQL from a natural-language question, execute it and return the result:

"Which customers generated the most revenue last quarter?"
        │
        ▼
   Agent / LLM ─► Schema context ─► SQL generation ─► Query ─► Results ─► NL response

The interesting part isn't that an LLM can write SQL, it's that the connection, schema introspection, generation and execution all happen inside the agent runtime. Text-to-SQL becomes another runtime capability rather than an integration every application builds from scratch.

Event-driven agents

Most agent demos are request/response:

HTTP request ─► Agent ─► Response

apowerb can also work in the opposite direction:

External event ─► Webhook ─► Agent ─► Tools / RAG / LLM ─► Action / Result

The current implementation includes email-triggered agents. Gmail uses Google Cloud Pub/Sub push notifications; Outlook uses Microsoft Graph change subscriptions with clientState validation.

Email arrives ─► Provider notification ─► apowerb webhook ─► Fetch message ─► Run agent ─► Log result

Subscription renewal is handled in the background: Gmail watches expire after seven days, Outlook after three, and the runtime periodically renews anything approaching expiration. Combined with cron-style scheduled runs and SSE streaming (token-by-token output, RAG ingestion progress, notifications), this moves the runtime beyond chatbots, the agent becomes a component in an event-driven system.

The open-source boundary

The core stack is Apache 2.0 ; the framework plus supporting services like the RAG engine, ETL components, observability and web UI. The project follows an open-core model: some commercial extensions (billing, MFA, identity-provider SSO, agent evaluation, supervision, organisation management) are maintained separately.

A TH2_EXTENSIONS environment variable declares which commercial bricks to load; if a brick isn't installed or enabled, the core doesn't depend on it. Routes belonging to an absent brick return a clean 404. The useful distinction: the open-source runtime remains fully usable without the commercial extensions, the core isn't an artificially limited demo.

It's more than one repository

A common pattern in AI tooling is "open-source framework + several proprietary services around it." Here, much more of the surrounding infrastructure is exposed. The thaink² GitHub org holds 10 repositories, all Apache 2.0:

Repository	Role	Technology
`apowerb`	FastAPI backend, CLI and ADK runtime	Python
`apowerb-ui`	Visual agent builder and monitoring	Next.js
`th2rag`	RAG, parsing, embeddings, vector search	LanceDB / Docling / Mistral
`th2etl`	ETL pipelines	Python
`th2pulse`	Observability and logs	OpenTelemetry
`th2forecast`	Time-series forecasting	R
`apowerb-hosting`	Docker Compose / Kubernetes / Helm	—
`apowerb-docs`	Documentation	Mintlify / MDX
`agent-hub`	Community agent templates	—

You can read and run the RAG engine, the scheduler and the telemetry pipeline, there's no hidden proprietary service doing the heavy lifting behind an API.

Run the full stack

The fastest way to understand the project is to run it. The hosting repository provides a Docker Compose setup:

git clone https://github.com/apowerb/apowerb-hosting.git
cd apowerb-hosting

cp .env.example .env
./scripts/generate-secrets.sh

docker compose \
  -f docker-compose/docker-compose.yml \
  --env-file .env \
  up -d

The UI comes up on http://localhost:3000, the API on :8000, and PostgreSQL runs inside the stack. You still provide an LLM API key, through the UI, or via DEFAULT_LLM_MODEL / DEFAULT_LLM_API_KEY. If you only want the Python package:

uv add apowerb   # or: pip install apowerb

Two entry points, two purposes:

apowerb package   ─► embed the runtime in your application
apowerb-hosting   ─► run the complete platform

How it compares

Comparisons between agent frameworks are easy to get wrong because they solve different problems. The useful question isn't "which framework is best?" but "which layer does each project expect you to own?" Below, "built-in" means the capability is part of the platform rather than something you assemble around it.

Capability	apowerb	LangGraph	CrewAI	n8n + LLM
Multi-model	LiteLLM	Yes	Yes	Partial
Built-in RAG	th2rag	Yes	Partial	Partial
Email webhooks	Gmail + Outlook	No	No	Yes
Native Text-to-SQL	Yes	Via tools	No	Partial
Included UI	Next.js	No	No	Yes
Helm deployment	Yes	DIY	DIY	Yes
Main approach	Runtime / platform	Framework	Framework	Workflow automation

The distinction isn't one checkbox, it's the architecture. LangGraph and CrewAI are frameworks you embed into an application. n8n comes from workflow automation, where an LLM is one node in a larger graph. apowerb provides the runtime around the agent: API, persistence, UI, integrations, scheduling, RAG and deployment. Different design choices, not just different implementations of the same product.

What I found interesting

1. Agents are treated as managed runtime resources. The database-backed definition and materialization model is the most distinctive architectural choice.

2. RAG isn't an external dependency. The project ships its own RAG service instead of requiring every deployment to assemble one.

3. Event-driven execution is first-class. Gmail, Outlook, schedules and SSE make "proactive" agents implementable, not just request/response chatbots.

Trade-offs

Not everything is settled. The release cadence is high, so the API isn't frozen, expect to track changes if you build on it now. Schema is created by ensure_* functions at startup rather than Alembic, which is fine today but means no reversible migrations. And the external community is still small; most contribution looks internal so far, so you'd be an early adopter.

None of that is a dealbreaker, it's the normal shape of a young project that's visibly running in production.

Worth a clone if…

you want RAG, tool auth, an API and a UI to already exist so you can build the actual agent;
you're a data team and Text-to-SQL over your warehouse plus scheduled runs maps to your use case;
you need everything self-hosted and auditable, your models, your data, Apache 2.0;
you're benchmarking agent frameworks and want to see where the "runtime, not library" line falls.

Three experiments that take minutes each: point an agent at a Postgres and query it in English; drop a folder of PDFs into a knowledge base and ask questions; set a Gmail trigger that summarizes and routes incoming mail.