I let an AI agent set up my entire Kafka platform. Here's what actually happened.

Your AI coding assistant can explain consumer groups, rebalancing, and exactly-once semantics. Ask it to actually set up a Kafka platform with governance, though, and it won't be able to do that on its own.

Between hallucinations, misunderstanding, production impact (I really saw Claude messing up a rolling upgrade of Kafka brokers), and the lack of knowledge of the products your Kafka infra is relying on, there's a lot working against it

The models, besides their training, have zero context about your infra. They've never seen your cluster, don't know your policies (technical, governance), and often have no way to check anything against your actual environment.

You can give it the missing context using Conduktor.

The thing that was missing

There is an open-source Conduktor skill you install into your AI assistant. It works with Claude Code, Cursor, VS Code Copilot, Gemini CLI:

npx skills add conduktor/skills

This is teaching the agent the whole platform and how to run process against it: Console, Gateway, and the CLI, so it can be efficient and not hallucinate.

After the install, the agent discovers your environment (Kafka clusters, Schema Registry, policies, etc.), asks questions based on what it finds, generates configs with real values and best practices, and runs everything with dry-run validation before it touches anything.

The CLI are really its "hands" as more deep than just MCP. The skill is the playbook where all the experience and practices from years of usage are written. This does a big difference VS "generate some YAML and cross fingers"

Starting from absolutely nothing

You can start from scratch with just Docker running and nothing else. No Kafka, no Conduktor, no config. When I just ask this (with the Conduktor skill setup):

install Conduktor and set it up so I can login

It checked my environment, asked what I was trying to do, wrote a docker-compose.yml, spun up the containers, hit one error along the way, self-corrected, and handed me a working platform, Kafka & Console perfectly configured.

I could ask the same but on my production Kubernetes. It would follow best practices too, use Helm, discover my environment, etc., and in minutes everything would be wired perfectly, with policies already in place.

This is much more powerful than a "human" quickstart, as the range of applications it covers is just wider and more production-ready already. The agent knows the Kafka domain, and with the skill it knows Conduktor, so the combination of both makes it ask me the right questions.

Governance, without becoming a Kafka lawyer

Running Kafka isn't the hard part anymore. Making it safe for a team to share is the hard part: naming conventions, ownership boundaries, policies. This is what prevent a Kafka cluster from turning into a wasteland of test-topic-final-v2.

The beautiful thing is to be able to ask large prompts like this now:

set up governance for two teams, Payments and Analytics, with topic policies and cross-team permissions

It worked in stages and figured out the dependency ordering itself. When the API rejected something, it read the rejection, restructured the YAML, and retried, with minimal hand-holding from me (just asking what policies I want based on what's possible). It ended up creating the following:

• TopicPolicy objects: locking down naming per team, enforcing safe defaults (retention, replication, required labels) across every topic.
• Application objects with non-overlapping resource boundaries to define ownership of resources and teams.
• Topics with descriptions and labels in the catalog.
• Cross-team permission giving Analytics read access to payments.orders.*.

This is federated ownership in practice: the platform team sets the boundaries, developers move freely inside them. Normally that knowledge takes months to accumulate and lives spreadsheet or Jira tickets. Here it lives in a skill file that every agent on the team can read.

Now flip to the developer side

Once those guardrails exist, a developer on the Payments team installs the same skill and never has to know any of it happened. No ApplicationInstance, no TopicPolicy, no YAML. They just talk.

"What topics do we have?"

The agent runs conduktor get Topic and shows the catalog — descriptions, owners, labels, visibility.

"I need a topic for my service."

The agent checks their ApplicationInstance, reads the policy constraints (naming prefix payments.*, retention one-to-seven days, a required data-criticality label), asks what the topic is for, generates compliant YAML, dry-runs it, and applies:

Topic/payments.fulfillment.shipped: Created

The developer just got a topic that's compliant by default. Without the skill, that's a JIRA ticket most likely, and asking platform team what's the right shape and what to put.

"How do I produce to my topic?"

It reads the cluster config, grabs the real bootstrap server, and hands back working code:

from confluent_kafka import Producer

producer = Producer({
    "bootstrap.servers": "localhost:19092",
})

producer.produce(
    "payments.fulfillment.shipped",
    key="ord-123",
    value='{"orderId": "ord-123", "status": "shipped"}',
)
producer.flush()

Copy, paste, run.

"I need to read the Analytics team's clickstream."

The agent finds that analytics.clickstream.pageviews belongs to the Analytics team, then writes a read-only permission scoped to exactly that topic, at both the Kafka and Console layers. The developer doesn't know what an ACL is or what patternType: LITERAL means. They asked in English and got access.

What I actually take away from this

This walkthrough only touched governance and onboarding. The skill also covers Gateway (Kafka proxy) encryption, data quality rules, Terraform export, and CI/CD scaffolding.

Try it, it's one command:

npx skills add conduktor/skills

It's open source, so if you hit a workflow it handles badly, open a PR. And if you're new to Conduktor, the Community Edition is free and self-hosted, the skill will do the install for you.

This post was adapted from the original on the Conduktor blog.