Learning Golang (some rough notes) - S02E01 - My First Kafka Go Producer

To install the Confluent Go client you need to run

go get gopkg.in/confluentinc/confluent-kafka-go.v1/kafka

With that done, we can run our first Kafka Go code! Make sure you’ve got a local Kafka broker running first, and then give this a spin:

package main

import (
    "gopkg.in/confluentinc/confluent-kafka-go.v1/kafka"
)

func main() {

    // --
    // The topic is passed as a pointer to the Producer, so we can't
    // use a hard-coded literal. And a variable is a nicer way to do
    // it anyway ;-)
    topic := "test_topic"

    // --
    // Create Producer instance
    // https://docs.confluent.io/current/clients/confluent-kafka-go/index.html#NewProducer
    //
    // Variable p holds the new Producer instance. There's a bunch of config that we _could_
    // specify here but the only essential one is the bootstrap server.
    //
    // Note that we ignore any error that might happen, by passing _ as the second variable
    // on the left-hand side. This is, obviously, a terrible idea.

    p, _ := kafka.NewProducer(&kafka.ConfigMap{
        "bootstrap.servers": "localhost:9092"})

    // --
    // Send a message using Produce()
    // https://docs.confluent.io/current/clients/confluent-kafka-go/index.html#Producer.Produce
    //
    // Only essential values are specified here - the topic, partition, and value
    //
    // There is NO handling of errors, timeouts, etc - we just fire & forget this message.
    // Did it work? ¯\_(ツ)_/¯
    p.Produce(&kafka.Message{
        TopicPartition: kafka.TopicPartition{Topic: &topic,
            Partition: 0},
        Value: []byte("Hello world")}, nil)

    // --
    // Close the producer
    p.Close()
}

When you run this, you get a message written to a Kafka topic!

Don’t try this at home!

Well, try it at home. But don’t take it anywhere near actual code that you need to write. As the comments in the code above show, there is NO error handling, whatsoever. I’m going to explore that in subsequent posts, but want to start at the very simplest example to start with here.

IDE FTW

(WTF? IDE FTW)

The VS Code environment is nice and helpful for writing Go and brings in function definitions etc

You can press Ctrl and click on a function or type and it will take you through to the definition, which is often quicker than looking up the API documentation