kagent AI agents Kubernetes: the framework in detail

kagent AI agents Kubernetes: the kagent-dev/kagent repository describes a « Kubernetes native » framework to build, deploy and manage AI agents. An agent becomes a Kubernetes resource, declared in YAML, tracked in Git and reconciled by a controller. The repository is licensed under Apache 2.0, and its README presents kagent as a Cloud Native Computing Foundation project.

Platform teams know the problem. Each team runs its agents its own way, with its own model keys and tools, and nobody has a shared view. kagent goes through the Kubernetes API instead: the model, the MCP tools and the agent’s behaviour are declarative objects. What does the repository contain, how do you install it, and above all how do you monitor agents that call tools on your cluster?

A word on honesty: span.observer has not installed or run kagent. What follows comes from the README, the repository’s docs/ folder (read on a clone on 8 November 2026) and the documentation pages on kagent.dev. The screenshots come from the repository or reproduce its files.

kagent AI agents Kubernetes: what the repository says

• Licence: the LICENSE file contains the text of the Apache License 2.0.
• Who is behind it: the repository belongs to the GitHub organisation kagent-dev. The README ends with « kagent is a Cloud Native Computing Foundation project » and names no commercial vendor.
• Languages: the repository holds a Go module (go/, with go.mod), Python packages (python/), a web interface (ui/) and Helm charts. The languages panel on the GitHub page was empty when read, so we give no percentage.
• Stars: the repository page showed about 4.2k stars on 8 November 2026 (a rounded value). The GitHub API was refused, so no exact figure could be read back.
• Version: the repository page showed no release when read, and we do not quote an official « latest version ». The 1.x installation documentation uses v1.0.0-rc3 for the CLI: that is a release candidate for 1.0, not a final release.
• Activity: the last commit in the clone dates from 7 November 2026. The README calls the project « in active development », with a roadmap on a GitHub board.

Concepts: agents, models and MCP tools

The README lists the building blocks. An Agent is a Kubernetes custom resource. An AgentTemplate carries the behaviour: system prompt, tools, subagents, model. A Harness decides how the agent runs: kagent’s own Go or Python runtimes, Codex, Claude or a custom runtime.

Models are set in a ModelConfig resource. The README names OpenAI, Azure OpenAI, Anthropic, Google Vertex AI and Ollama, plus custom providers behind a gateway. What you can choose depends on the Harness.

Tools go through MCP. According to the README, kagent ships an MCP server with tools for Kubernetes, Istio, Helm, Argo, Prometheus, Grafana or Cilium. Agents connect through RemoteMCPServer resources, which agents can share. For the risk of this model, read MCP tool poisoning.

The first screenshot shows the agent list of the web interface, taken from the README GIF. It comes from an earlier version of the interface: the 1.x interface may differ.

Source : github.com

Architecture: controller, PostgreSQL and Substrate

The architecture documentation describes a control plane. Kubernetes stores the desired configuration (Agent, Harness, AgentTemplate, EgressPolicy and others). PostgreSQL stores runtime identity, A2A history and lifecycle state. Substrate « Actors » run the agent processes, with suspend, resume and snapshots.

The controller exposes gRPC, A2A and MCP APIs. A session keeps the history, and the documentation mentions checkpoints to fork a conversation. The same documentation lists what is not implemented yet: dedicated agent bindings, policy-enforced cross-session delegation, checkpoint sharing and multi-replica gateway coordination.

Installing kagent, according to the documentation

Prerequisites on the 1.x installation page: Kubernetes 1.37 or later, with the PodCertificateRequest and ClusterTrustBundle APIs; helm v3 and kubectl; a model API key. For a local cluster, the page cites kind v0.32.0 or later. kagent 1.0 does not upgrade in place from 0.10.x.

Evaluation path: the CLI

The CLI deploys a development PostgreSQL database, then installs the charts in order. Commands from the page (dummy key):

kind create cluster --name kagent --image kindest/node:v1.37.0
export OPENAI_API_KEY="your-api-key-here"

curl https://raw.githubusercontent.com/kagent-dev/kagent/refs/heads/main/scripts/get-kagent | bash -s -- --version v1.0.0-rc3
kagent version

cat > kagent-values.yaml <<EOF
config:
  server:
    grpcReflection: true
EOF
cat > kagent-config.yaml <<EOF
install:
  kagent:
    valuesFiles: [kagent-values.yaml]
EOF

kagent --config kagent-config.yaml install --namespace kagent
kubectl rollout status deployment/kagent-controller -n kagent --timeout=300s

The repository documentation states that the development database is not meant for production, and that kagent install creates certificate authorities valid for one year, which nothing renews.

Production path: Helm

Here you provide PostgreSQL 15 or later: the charts do not deploy one. The page chains the kagent-crds and substrate-crds charts, certificate-authority and JWT pools (kubectl-ate plugin), substrate-podcert, substrate-atelet, substrate, then kagent from oci://ghcr.io/kagent-dev/kagent/helm/kagent. The full values files stay in the official documentation: we do not copy them.

Checking

kubectl get pods -n kagent
kubectl get pods -n ate-system
kubectl get workerpools -n kagent
kubectl port-forward -n kagent svc/kagent-ui 8082:8080

The page expects Running pods and a ready kagent-default WorkerPool, then an interface on local port 8082. The « Your first agent » guide then creates a Harness, an AgentTemplate and an Agent, and follows the state with kagent agent get.

An agent declared in YAML

The docs/architecture/ folder holds a complete example, reproduced in the screenshot: a ModelConfig and a RemoteMCPServer in a shared namespace, an AgentTemplate and an Agent in team-a. The Harness named runner is assumed to exist already. The API is api.kagent.dev/v1alpha3, so still an alpha Kubernetes API version.

Source : github.com

Points to note: apiKeySecret names a Kubernetes Secret, and allowedNamespaces controls who may use the model or the tool server. With from: Selector, only namespaces whose labels match get access. The model-auth Secret must be created separately: its credentials are not copied into team-a.

Limits and open questions

• Maturity: the 1.x series appears as a release candidate (rc3 in the documentation we read), the API is v1alpha3 and the migration path from 0.10.x is manual. An upgrade can recreate snapshots or databases (the documentation flags non-backward-compatible migrations on the Substrate side).
• Complexity: you need PostgreSQL, Substrate, an internal PKI and a very recent Kubernetes. This is not a chart you simply drop in.
• RBAC rights: without a namespace restriction, the chart creates ClusterRoles and ClusterRoleBindings. The Helm README separates a reconciliation role from a management role that writes CRDs, Secrets and ConfigMaps; it states that no mode disables those writes. Restrict with rbac.namespaces or global.watchNamespaces.
• Secrets: according to the documentation, model keys stay out of the compiled environment: the Substrate gateway injects them into the outgoing HTTP header, with a cache that can take up to five minutes to refresh after a rotation.
• Tools: an agent wired to the Kubernetes or Helm tool server acts on the cluster. Limit its rights and namespaces, as in guardrails for AI agents in production.
• Not verified: performance, model costs and the real behaviour of agents. We measured nothing.

Observing kagent: OpenTelemetry traces

The documentation describes an export of OpenTelemetry traces from the controller and the agent runtimes. Nothing is on by default: you declare an OTLP exporter in the chart. The configuration also allows metrics and logs exporters, each with endpoint, protocol (grpc or http/protobuf) and timeout.

Source : github.com

The tracing page on kagent.dev insists: the same block goes in config.telemetry (the controller) and config.runtimeDefaults.telemetry (the agents). With only one, part of the request path stays untraced. After a helm upgrade, create a new session: an existing Actor keeps the configuration it started with.

The telemetry contract uses the GenAI conventions: gen_ai.agent.name, gen_ai.conversation.id, gen_ai.request.model, plus kagent.* and a2a.* attributes. Spans carry operations such as invoke_agent and execute_tool. Messages and tool arguments are recorded only if you enable content capture: keep captureContent: false until you have assessed the risk for your data.

Two documented tips apply. First, keep SDK sampling parent-based AlwaysOn and sample in the Collector, otherwise distributed traces can be incomplete. Second, enable baggage propagation only knowingly: caller members can travel to the model API and to MCP servers.

On the SRE side, a few SLIs follow from these spans: invoke_agent duration, execute_tool error rate, model calls per invocation. For token cost, see the cost of an AI agent in OpenTelemetry tokens. Controller logs stay the usual kubectl logs deployment/kagent-controller -n kagent. The documentation we read gives no list of metrics to watch: we do not invent one.

What about Dynatrace?

kagent documents no Dynatrace integration, and we assume none. Since kagent exports OTLP, an OpenTelemetry Collector can receive these traces. The rest belongs to Dynatrace documentation: see the OpenTelemetry Collector to Dynatrace over OTLP/HTTP. We have not tested this chain.

Best practices

• Evaluate kagent on an isolated kind cluster, with a model key that has a spending cap.
• Version Agent, AgentTemplate and ModelConfig in Git and deploy them as in Argo CD for Kubernetes.
• Keep rbac.namespaces and allowedNamespaces to the strict minimum.
• Enable traces on the controller and on the runtimes, with content capture off.
• Read the release notes before each upgrade: the API is still alpha.

Frequently asked questions

Is kagent ready for production?

The documentation offers a « production » Helm path, but the 1.x series appears as a release candidate and the API as v1alpha3. Test first, read the list of unimplemented features and plan the migration from 0.10.x.

Which models does kagent accept?

According to the README: OpenAI, Azure OpenAI, Anthropic, Google Vertex AI, Ollama, and custom providers behind a gateway, through ModelConfig. Exact support depends on the chosen Harness.

How do I follow what an agent does?

Enable OTLP trace export in the chart, on the controller and on runtimeDefaults, then read the invoke_agent and execute_tool spans in your backend. Sessions and task history are also kept in PostgreSQL.

Conclusion

kagent gives AI agents a Kubernetes frame: declarative resources, MCP tools, a choice of models and OpenTelemetry traces. It needs heavy infrastructure and a recent cluster, and its API is still alpha. If your platform team wants to manage agents like the rest of its workloads, it deserves a trial on a test cluster.

span.observer is an independent community blog, not affiliated with the kagent project, the Cloud Native Computing Foundation or Dynatrace LLC. Trademarks belong to their owners. Tools change quickly: check commands and options in their official documentation.

---

Originally published on span.observer. French version: https://span.observer/fr/kagent-agents-ia-kubernetes/.