confluent-kafka-go in 2026: What Still Works and What to Watch

confluent-kafka-go is Confluent's Kafka client for Go. It wraps librdkafka, Confluent's C client, through cgo, and it is still actively developed. v2.16.0 shipped on 7 October 2026. If you run a Go service on it, five things deserve a check: how it builds, what its Schema Registry packages pull in, where keyed records land, when offsets get committed, and which consumer group protocol it uses.

TL;DR: confluent-kafka-go covers transactions, admin operations, KIP-848 consumer groups, and Confluent's own Schema Registry serializers. It needs cgo, and Go turns cgo off when you cross-compile. The Schema Registry serializers need cgo too, even next to a pure Go client. The default partitioner does not match the Java client, but murmur2_random does. By default, offsets can be committed before your code finishes processing a message. The consumer group protocol is expected to default to KIP-848 in a later release, so set it explicitly.

For how confluent-kafka-go compares with franz-go, kafka-go, and Sarama on cgo, consumer groups, and Schema Registry, see Kafka Go Clients: franz-go vs kafka-go vs Sarama.

What still works

• librdkafka underneath. Most protocol behavior comes from the bundled librdkafka, the same C library behind Confluent's Python, .NET, and JavaScript clients. v2.16.0 bundles librdkafka 2.16.0, with idempotent and transactional producers and an AdminClient for topics, configs, ACLs, and consumer groups.
• No separate install on common platforms. The module ships prebuilt librdkafka for macOS (x64 and arm64), glibc Linux (x64 and arm64), musl Linux such as Alpine (x64 and arm64), and Windows x64. On those platforms, you do not install librdkafka on the build or target machine.
• KIP-848 consumer groups. The next-generation consumer group protocol, in which the broker computes partition assignments, has been production-ready since v2.12.0 on Kafka 4.0+ brokers. You turn it on with group.protocol=consumer.
• Confluent's Schema Registry serializers. The schemaregistry packages include serializers for Avro, Protobuf, and JSON Schema that register and cache schemas and write Confluent's wire format.
• Active development. In the 90 days to 25 September 2026, the repository had 55 commits and 29 merged pull requests. Over the preceding year, merges were spread across 17 accounts, and the most active one handled 39% of them. The open backlog is older, with 74 open pull requests at a median age of 2.4 years. These figures come from the GitHub snapshot in the full comparison.

What to watch

cgo is required, and cross-compiling turns it off

The client calls librdkafka through cgo, so it builds only with CGO_ENABLED=1 and a C compiler. Go turns cgo off by default when you cross-compile, for example GOOS=linux go build on a Mac, and when it cannot find a C compiler, as in the golang:alpine image. With cgo off, a v2.16.0 build fails inside the library:

# github.com/confluentinc/confluent-kafka-go/v2/kafka
.../kafka/deserialized_message.go:28:17: undefined: TopicPartition
.../kafka/deserialized_message.go:34:17: undefined: TimestampType
.../kafka/deserialized_message.go:36:19: undefined: Header

Up to v2.15.1, the same build fails in your own code instead, with errors such as undefined: kafka.NewProducer and undefined: kafka.ConfigMap. To build for another platform, set CGO_ENABLED=1 and give cgo a C compiler for that platform, or build on the target platform itself, for example in a container for that architecture.

A few more build details matter once the service ships in a container:

• Alpine needs -tags musl. The project README requires it for go get, go build, and similar commands on musl systems.
• The binary is not static. It links the system C library dynamically, so the runtime image needs one. scratch and distroless/static have none, and a binary built against a newer glibc fails on an older one. For a fully static binary, the project README describes a musl build with external-linker flags.
• Kerberos needs a system librdkafka. The bundled builds exclude GSSAPI. Install a librdkafka with GSSAPI on both the build and target systems, and build with -tags dynamic. Without librdkafka and pkg-config on the build machine, that build stops with exec: "pkg-config": executable file not found in $PATH.
• v2.16.0 needs Go 1.26. The module's go directive moved from 1.25.0 to 1.26.0. With the default GOTOOLCHAIN=auto, Go 1.25 downloads a newer toolchain by itself. With GOTOOLCHAIN=local, you need to upgrade Go first.

Schema Registry serializers need cgo too

The Schema Registry client and serializers do not call librdkafka, so it seems natural to pair them with a pure Go client such as franz-go and build with CGO_ENABLED=0. For the serializers, that stopped working in v2.10.1 (June 2025). In that release, the serializer interface gained methods that can carry the schema ID in Kafka headers. Those methods use the kafka.Header type, and the kafka package is the one that needs cgo.

We built a small program that uses the Avro serializer (avrov2) with franz-go, and a second one that uses only the Registry client, with CGO_ENABLED=0 and Go 1.27.1 against each release:

Release	Registry client only	Avro serializer
v2.5.4 to v2.10.0	Builds	Builds
v2.10.1 to v2.13.0	Builds	undefined: kafka.Header
v2.13.2 to v2.15.1	undefined: kafka.LibraryVersion	Same error
v2.16.0	undefined: TopicPartition	Same error

The v2.16.0 release notes say this dependency was removed from the Registry client, and that change did ship. The same release, however, added an experimental SerializingProducer and DeserializingConsumer API. It brought a new import of the kafka package into the Registry client, and new files in the kafka package that do not compile without cgo.

Both problems are tracked in issue #1635, with a proposed fix in pull request #1634. The release notes also mark that new API as subject to change or removal, so pin the module version if you adopt it.

The default partitioner does not match Java

The default partitioner, consistent_random, hashes keys with CRC32, while the Java client and franz-go use Murmur2. If a Go service and a Java service write the same keys to one topic, a key can land on two partitions, which breaks per-key ordering. Setting partitioner to murmur2_random gives Java-compatible placement:

p, err := kafka.NewProducer(&kafka.ConfigMap{
    "bootstrap.servers": "localhost:9092",
    "partitioner":       "murmur2_random",
})

We checked it by producing with v2.16.0 to a local Kafka 4.3.1 broker and comparing the partition in each delivery report with the Java client 4.3.1 (BuiltInPartitioner.partitionForKey). We used the keys customer_42, order-1001, user-7, 21, foobar, a-little-bit-long-string, café, and an empty key, each at 3, 6, and 12 partitions.

murmur2_random matched Java in all 24 cases. The default matched in only 1 of the 21 cases with a non-empty key. For example, on a three-partition topic customer_42 goes to partition 1 by default and to partition 0 with murmur2_random, as in Java.

Empty keys behave differently too. The default sends an empty key to a random partition. Five sends to a 12-partition topic landed on partitions 4, 0, 9, 7, and 0. murmur2_random hashes it as Java does and sent it to partition 9 every time. Records with a null key go to a random partition under both settings.

After a switch, new records for a key can go to a different partition than its earlier records. Switch when other producers write the same keys to the same topic, and compare placements with your own keys first.

Offsets can be committed before processing finishes

By default, the consumer stores a message's offset as soon as Poll or ReadMessage hands the message to your code (enable.auto.offset.store=true). It then commits the stored offsets in the background every 5 seconds (enable.auto.commit=true, auto.commit.interval.ms=5000). If the process crashes after a commit but before your code finishes a message, that message is not delivered again.

For at-least-once processing, turn off the automatic store and store each offset yourself after the work is done. Auto-commit stays on and commits only what you stored:

c, err := kafka.NewConsumer(&kafka.ConfigMap{
    "bootstrap.servers":        "localhost:9092",
    "group.id":                 "orders",
    "enable.auto.offset.store": false,
})

// After processing msg:
if _, err := c.StoreMessage(msg); err != nil {
    // handle the error
}

This works when you process each partition's messages in order. If you process one partition's messages in parallel, store an offset only after every earlier message in that partition has succeeded. After a crash, messages that finished but were not yet committed are delivered again, so processing must tolerate duplicates.

KIP-848 is opt-in for now

The default group.protocol is still classic, but librdkafka's configuration reference says the default "will change to consumer in next releases."

The new protocol changes a few contracts. The client no longer uses partition.assignment.strategy, session.timeout.ms, or heartbeat.interval.ms. Rebalance callbacks receive only the partitions being added or revoked. If your callbacks manage assignments themselves, use IncrementalAssign and IncrementalUnassign instead of Assign and Unassign.

Set group.protocol explicitly, so that a future upgrade does not switch protocols for you. Before moving a group to consumer, read Confluent's KIP-848 migration guide.

If you need what it lacks

A service that builds and runs well on confluent-kafka-go can keep running it. If cgo is the problem, franz-go is a pure Go client that supports KIP-848. Its own Schema Registry package, pkg/sr, writes the same wire format, but you supply the Avro, Protobuf, or JSON Schema codec. The full comparison linked above covers what to test before switching, including how to keep key placement.

Check what your service actually does

Whether you change the partitioner, the offset store, or the build image, check the records themselves. Kafma, a Kafka GUI client, shows each record's key, partition, headers, and raw bytes, and a consumer group's committed offsets and lag. That lets you confirm that keys land where you expect. To check whether commits run ahead of processing, compare the committed offsets with your application logs. Rebalances, retries, and crash recovery still need tests in your application.

Written by the team behind Kafma.