DEV Community

CommerceFrame
CommerceFrame

Posted on Originally published at kortix.com

How to evaluate an open source AI agent platform you own end to end

An open source AI agent platform is the software layer a company runs its AI agents on and owns outright. Kortix is the open-source AI Management System built for that job: your agents, the skills they share, your company memory and every connector live in one git repository you own, and any model runs with your own keys on infrastructure you choose.

The label covers a wide range of products, and only some of them let a company keep what it builds. A platform that runs agents also accumulates the context around them, so the choice decides more than which runtime you use this quarter. Four questions, plus a look at the categories, will tell you which kind of platform you are actually holding.

What an open source AI agent platform is

An open source AI agent platform gives a team somewhere for agents to run, a set of tools those agents can reach, and a record of what they did. Around those sit the parts that decide whether the company or the vendor controls the work: where agent definitions live, how the model is chosen, who holds the credentials, and how a finished change reaches production.

The word "platform" hides three different products. A developer framework is a library an engineer imports into an application. An application platform is a visual builder for workflows and retrieval pipelines. An AI Management System is a system a company operates, where agents, skills, memory, connectors, triggers and permissions are configuration the company owns. Each shape answers a different question, and the name on the box rarely says which one you bought.

Four questions that separate ownership from rental

The fastest way to judge a candidate is to ask where the configuration lives, whose model runs the work, where the agent's runtime sits, and who approves a change. A platform that answers all four in the company's favour is one you can own. One that answers them in the vendor's favour is a rental with an open-source label, however much code it publishes.

Where the configuration lives

Ask to see the files. On Kortix, agents are markdown, memory is a folder of files, and the connector config, triggers and machine image are declared in kortix.yaml inside one git repo. A team can grep the whole company, diff any change and roll part of it back. On a closed platform the same configuration lives inside the vendor's product, so you can export output but not the system that produced it. The test is blunt: can you clone the thing that defines your agents, read it, and move it to another machine?

Models and keys the company controls

Model choice on Kortix is configuration you change when a better model lands. Kortix runs Anthropic, OpenAI, Google, Groq, xAI, DeepSeek, Mistral, Bedrock and OpenRouter models, plus any OpenAI-compatible endpoint, and the model can change per agent, per session or per message. You bring your own API key, or use the ChatGPT subscription you already pay for. The closed alternatives bind the model to the vendor: Claude Cowork runs Claude models, ChatGPT Work runs OpenAI's models, and each stays inside its maker's cloud. Switching a model should never force a rewrite of your agents.

Where the agent runs

An agent that edits files and calls APIs needs a computer. Kortix gives every session its own isolated Linux machine on its own branch, and a team can run thousands of them in parallel on one configuration. The agent can install packages, run code and break things inside that machine, and only committed work survives. Running agents on a shared server or a developer laptop removes that boundary, so one bad run can reach anything the host can reach.

The human gate on every change

Finished work on Kortix lands on the default branch as a change request a human reads as a diff. Merge is deny-by-default for an agent, and permission for each tool call can be set to allow, ask or block, down to a single shell command. A platform without that gate asks you to trust an agent's output. A change request a person can read and merge turns that trust into a decision.

The three shapes of platform a team actually gets

Shape What you own Best for
Developer framework A runtime library inside your application Engineering teams building custom agent behaviour
Application platform A visual workflow and retrieval builder A product or ops team shipping one workflow
AI Management System Agents, skills, memory, connectors and permissions in one repo A company running agents as part of operations

A framework and an application platform fit when the deliverable is one application and engineering owns the runtime. They hand you primitives and leave the surrounding environment, the review path and the company context to you. An AI Management System takes the wider job: it runs the agents, holds the context they share, connects the tools the company already uses, and puts a person at the gate. Kortix belongs to that third group, and it is the pick when ownership is the priority.

What Kortix puts in the repo

Kortix keeps the whole company in one git repo, and the pieces are exactly what most platforms keep behind an API. Agents are markdown files. Skills are reusable know-how written once and shared into every session. Memory is plain files that accumulate what the company learns. Connectors, triggers and the machine image are declared in kortix.yaml, so a model change is a one-line diff and a new scheduled job is a reviewed commit.

The connector layer reaches 3,000+ apps plus any MCP, OpenAPI, Postman, GraphQL or raw HTTP endpoint, and credentials are brokered server-side, so a raw key never enters the agent's machine. Each tool call is ruled allow, ask or block, down to the arguments it was given. That matters once agents act on billing, tickets and code, because the platform then has to answer which agent may call which tool and what a person sees before it happens.

Any model can run the loop with your own keys, and self-hosting starts from one Docker Compose stack. Install the CLI with curl -fsSL https://kortix.com/install | bash, scaffold a project with kortix init, then bring it live with kortix ship. A project created in the web app needs nothing installed. The same configuration runs on a laptop, a VPS, a VPC or on-prem hardware, or on managed cloud. The code is open source under the Elastic License 2.0, which lets a team self-host it, read it and modify it; the repository is Kortix on GitHub.

Start with one job and one reviewer

Pick a task the team already does by hand: an error triage sweep, a weekly report, a reconciliation pass. Define the agent in markdown, wire the one tool it needs, and let the session run on its own machine. When the change request arrives, read the diff and decide. That single loop teaches more than a feature matrix, because it shows whether the platform can hold the company's context, run the model you chose, and stop the work for a person before it lands.

The wider field is worth reading once: the open source AI agent platform guide compares the ownership, hosting and self-hosting choices in more depth. When you are ready to run the loop yourself, Get started with open-source Kortix.

Top comments (0)