DEV Community

Cover image for IBM Owns Confluent. Kafka Users Still Have a Runtime Problem.
Andrew Tan
Andrew Tan

Posted on Originally published at layline.io

IBM Owns Confluent. Kafka Users Still Have a Runtime Problem.

The Kafka protocol is still open. The commercial center of gravity is not. Seven months after IBM closed Confluent, the questions that matter are dual-run, replay, and who owns batch when the streaming story is watsonx.

Two facts that keep getting mashed together

Apache Kafka is still an open protocol. You can run it on MSK, on a cluster you operate, on Redpanda, on Confluent Platform. The wire format did not get acquired.

Confluent, the company that turned that protocol into the default commercial Kafka stack, has been an IBM company since March 17, 2026. IBM paid about $11 billion, $31 a share. Day one, the story was not "we remain the independent Kafka company." Day one was watsonx.data, IBM MQ, webMethods, and IBM Z. Streaming as fuel for an AI data platform and hybrid automation.

Treat those as the same event and you will sleep through the next 18 months of product decisions. Separate them and the next conversation with your vendor gets a lot more useful.

What actually closed

I already wrote the technical Kafka comparison. This is the concentration piece.

On close day, IBM put the thesis in the press release title: real-time data as the engine of enterprise AI and agents. Rob Thomas talked about models and agents acting on what is happening now. Jay Kreps, still CEO in that quote, talked about setting the world's data in motion at IBM scale.

That is a different product bet than "best Kafka."

Best Kafka means partition rebalances, connector quality, Flink packaging, cloud SKUs, and whether your consumer lag page still works on a Tuesday. IBM's bet means Confluent Cloud and Platform get pulled toward watsonx, MQ, and the mainframe. The Kafka log is the on-ramp. The destination is IBM's AI and integration suite.

You can like that bet. Plenty of enterprises already live in that suite. You should not pretend it is the same roadmap Confluent sold as a public company.

The founder left. The protocol stayed.

On August 13, Jay Kreps stepped back after nearly twelve years. Founder transitions after an $11 billion close are normal. They also tell you where the writing happens.

Roadmaps used to get argued in Mountain View, on a Kafka-company clock. They now get argued inside IBM Software, on a watsonx and mainframe clock. Connector cadence, cloud packaging, and what "supported" means will follow that clock.

A concrete signal showed up in late October. IBM announced watsonx.data integration for Apache Flink, with a visual canvas and a Python SDK that submit jobs onto Confluent Platform for Apache Flink. That is not a Kafka killer. It is IBM wrapping stream processing in the watsonx.data product line. If you bought Confluent because you wanted an independent streaming company, read that sentence again.

I am not claiming the product got worse. I do not have a telemetry feed into Confluent Cloud. I am claiming the center of gravity moved, and you can see it in public.

Kafka the language, IBM the publisher

Think of Kafka as a language. Lots of people speak it. The grammar is in the open. IBM bought the largest school that taught the commercial dialect, printed the textbooks, and ran the certification booth at the conference.

You can still speak the language. You can hire people who speak it. You can run a cluster that never sends IBM a dollar.

What you cannot assume is that the commercial default path (Confluent Cloud, Confluent Platform, the connector catalog you already standardized on) will fight for Kafka independence. It will follow IBM's data and AI platform. Dual-running a topic into watsonx.data will get more love than making it trivial to leave.

Vendors follow their thesis. Your job is to notice the thesis changed.

Three questions your next QBR should include

Skip the "are you still committed to Kafka?" slide. Of course they are. Kafka is the asset.

Ask the operational questions the press release does not cover.

Dual-run for the next two years

If IBM folds SKUs, rebrands consoles, or moves you onto a watsonx control plane, you need a period where the old path and the new path produce the same outputs. Dual-run is boring. Dual-run is how you survive a vendor merge without a weekend cutover.

Replay when MQ, Z, and a warehouse are in the same incident

Most Kafka teams already know how to rewind a consumer group. The hard case is mixed: a stream, a mainframe transaction, a nightly ERP file, and an Iceberg table someone already queried. IBM's day-one diagram puts those in one family. Your incident on a Tuesday night still needs a person who can rewind one without corrupting the others.

Whether batch still needs a second orchestrator

If the answer is "use Airflow for the files and Confluent for the topics," you did not buy a platform. You bought a streaming suite and kept the split you already had. Ask them to show one workflow that replays a late file and a bad topic with the same ownership.

If your AE cannot answer those without a solution architect and a week of delay, you have your signal.

The problem that did not get acquired

IBM bought a streaming company. It did not buy your coordination problem.

You still have to catch up a batch job that missed the window while the stream kept going. You still have to replay three hours of poison messages without double-writing the warehouse. You still have to join a slowly changing dimension to a live topic. You still have to tell a new hire which system is the source of truth on Friday afternoon.

None of that lives in the Kafka protocol. The IBM logo on the Confluent console does not make it easier. Some of it gets harder if the commercial stack is being re-aimed at agents and watsonx while your actual workload is still "make the payments file and the fraud topic agree."

The teams I talk to who are calm about this did not post "Kafka is open source, we are fine" in Slack. They already treat the broker as a transport and keep the runtime (retries, dependencies, replay, approvals) in a place they control.

Where we fit

At layline.io we work with teams who speak Kafka and still have to land files at 6pm. We are not a Kafka replacement and we are not an IBM alternative. We are the runtime that sits across batch and streaming so the coordination work does not live inside the streaming vendor's suite.

If IBM's platform is the right home for you, use it. Just do not confuse "we still produce Kafka records" with "our operational model is unchanged." The records can look the same. The gravity around them cannot.


If you are on Confluent Cloud or Platform and trying to separate protocol comfort from vendor gravity, get in touch. We have helped teams map dual-run and replay across batch and stream without waiting for the next rebrand.


Top comments (0)