DEV Community

Davi Orlandi
Davi Orlandi

Posted on Edited on

Event-Driven Architecture and Messaging in Go with Watermill

#go #webdev #eventdriven #architecture

What is Watermill?

Watermill is an open-source Go library for event- and message-based applications. It abstracts the main brokers on the market (Kafka, RabbitMQ, NATS, Google Pub/Sub, Redis Streams, and others) behind a unified, extensible API — so you can build messaging pipelines without wrestling with each system's details.

Why use Watermill?

Event-driven systems bring decoupling, horizontal scalability, and easier CQRS / Event Sourcing. Watermill addresses common pain points in that architecture:

  • Broker abstraction: switch Kafka for RabbitMQ (or vice versa) without rewriting your logic.
  • Pluggable middleware: logging, retries, metrics, and more.
  • Performance: processes hundreds of thousands of messages per second, taking advantage of Go's concurrency model.
  • Testability: in-memory brokers for fast local tests.

Main Watermill Components

  • Publisher: publishes messages to a topic.
  • Subscriber: consumes messages from a topic.
  • Router: routes messages between publishers and subscribers, applying middlewares and handlers.
  • Middleware: cross-cutting features such as logging, retries, rate limiting, etc.

These components connect flexibly — from simple flows to complex event-processing pipelines.

Supported Protocols and Brokers

  • Kafka
  • RabbitMQ (AMQP)
  • Google Cloud Pub/Sub
  • AWS SNS/SQS
  • Redis Streams
  • SQL (PostgreSQL, MySQL)
  • GoChannel (in-memory, great for tests)
  • HTTP

Start with a simple broker and migrate to something more robust as the app grows, without major code changes.

Practical Example: Publishing and Consuming Messages with Kafka

Basic publish/consume with Kafka. The same pattern applies to other brokers — only the configuration changes.

Prerequisites: 

  go get github.com/ThreeDotsLabs/watermill
  go get github.com/ThreeDotsLabs/watermill-kafka/v3
Enter fullscreen mode Exit fullscreen mode

Publisher:

package main

import (
    "github.com/ThreeDotsLabs/watermill"
    "github.com/ThreeDotsLabs/watermill-kafka/v3/pkg/kafka"
    "github.com/ThreeDotsLabs/watermill/message"
    "log"
)

func main() {
    logger := watermill.NewStdLogger(false, false)
    publisher, err := kafka.NewPublisher(
        kafka.PublisherConfig{
            Brokers:   []string{"localhost:9092"},
            Marshaler: kafka.DefaultMarshaler{},
        },
        logger,
    )
    if err != nil {
        log.Fatal(err)
    }

    msg := message.NewMessage(watermill.NewUUID(), []byte("Olá, Watermill!"))
    if err := publisher.Publish("exemplo.topico", msg); err != nil {
        log.Fatal(err)
    }
    log.Println("Mensagem publicada!")
}
Enter fullscreen mode Exit fullscreen mode

Subscriber:

package main

import (
    "context"
    "github.com/ThreeDotsLabs/watermill"
    "github.com/ThreeDotsLabs/watermill-kafka/v3/pkg/kafka"
    "github.com/ThreeDotsLabs/watermill/message"
    "log"
)

func main() {
    logger := watermill.NewStdLogger(false, false)
    subscriber, err := kafka.NewSubscriber(
        kafka.SubscriberConfig{
            Brokers:       []string{"localhost:9092"},
            Unmarshaler:   kafka.DefaultMarshaler{},
            ConsumerGroup: "exemplo-grupo",
        },
        logger,
    )
    if err != nil {
        log.Fatal(err)
    }

    messages, err := subscriber.Subscribe(context.Background(), "exemplo.topico")
    if err != nil {
        log.Fatal(err)
    }

    for msg := range messages {
        log.Printf("Mensagem recebida: %s", string(msg.Payload))
        msg.Ack()
    }
}
Enter fullscreen mode Exit fullscreen mode

Tip:

To test locally, run Kafka with Docker, start the subscriber, then the publisher. You should see messages flowing between them.

Use Cases and Advantages

Watermill fits well for:

  • Microservices that need async, decoupled communication.
  • Real-time event processing (monitoring, analytics, notifications).
  • CQRS and Event Sourcing.
  • Projects that need horizontal scalability and resilience.

You get multi-broker support, solid performance, a straightforward API, and easy middleware / metrics integration (Prometheus).

Conclusion

Watermill abstracts broker complexity and gives you a flexible API for event-driven systems in Go. If you're building messaging-heavy services and want to stay broker-agnostic, it's a strong option.

Useful links:

Top comments (0)