DEV Community

William Rodriguez
William Rodriguez

Posted on

Stop writing raw Kafka boilerplate in Python: A decorator is enough

#opensource

Code is read twice as often as it is written — and the Kafka consumer loop is the code nobody enjoys re-reading. WKafka turns the entire consumer skeleton into a single, clean decorator.

This is Day 01 of the WKafka Open-Source Engineering Series.

The Pain Points We Faced

  • Hand-rolling while True consumer loops in every single microservice.
  • Scattered JSON deserialization try/except blocks swallowing exceptions silently.
  • Ugly shutdown handling leaving dangling consumer group partition assignments.

The Implementation 

from wkafka import WKafka

kafka = WKafka(dynamic_group_id=True)

@kafka.consumer(topic="orders", format="json")
def on_order(msg):
    print(f"New order processed: {msg.value}")
Enter fullscreen mode Exit fullscreen mode

Why This Architecture Wins

  • Zero Boilerplate: Single @kafka.consumer decorator defines the entire loop.
  • Safe Deserialization: format='json' automatically parses payloads cleanly.
  • Auto Config: Resolves KAFKA_SERVER environment variable with localhost:9092 fallback.

Verification & Status

Tested and verified with Apache Kafka against real broker clusters. Fully compatible with Python 3.9 through 3.14 with strict typing.

Author: William Steve Rodríguez Villamizar (Wisrovi)

Top comments (1)

Collapse
 
william_rodriguez_65a5898 profile image
William Rodriguez •

In event-driven Python microservices, maintaining explicit control over consumer rebalance timeouts while executing decoupled handlers is essential. Declarative subscription decorators streamline message routing while keeping offset commit contracts strictly deterministic.

Do your architectures favor centralized orchestrator sinks or localized idempotent consumer handlers for event pipelines?