DEV Community

William Rodriguez
William Rodriguez

Posted on

Sending a frame (not a byte) is a Kafka API choice. WKafka makes it format="image"

#streaming

Kafka delivers bytes. It does not care if they are a contract, a cat photo or a satellite frame. For a team that streams vision data, "raw bytes" means every consumer re-implements decoding, shape-guessing and channel-order handling. WKafka's answer is a contract the library enforces: images are a first-class message.

Day 03 of the WKafka open-source series — decorator-based, MIT, reproducible.

What "image" means as a format

Producer sends a NumPy array, not a blob:

import numpy as np
import cv2

frame = np.zeros((480, 640, 3), dtype=np.uint8)   # BGR canvas
cv2.circle(frame, (320, 240), 100, (0, 0, 255), -1)

with kafka.producer() as p:
    p.send(
        topic="stream_images",
        value=frame,
        format="image",        # serializer: array → JPEG (quality) → bytes
        quality=90,
    )
Enter fullscreen mode Exit fullscreen mode

Consumer flips bytes back to a working array — same tuple, same channels:

@kafka.consumer(topic="stream_images", format="image")
def on_frame(image):
    print(f"shape={image.shape}")      # (480, 640, 3) — came back intact
Enter fullscreen mode Exit fullscreen mode

Why format="image" is a contract, not a convenience

  • Deterministic round-trip. The same serializer that encoded is the one that decodes. Consumers never re-detect shape/dtype/channels by heuristics.
  • Frame order = product order. In vision pipelines the meaningful unit is the frame, not the byte. WKafka preserves per-entity order on top of a first-class image type.
  • OpenCV/NumPy/PIL interop out of the box — the array you send is the array consumers receive, no glue.

The hook

🖼️ Sending an image through Kafka shouldn't mean "sending bytes and hoping the other side knows what they were." It means format="image" — a typed message that rides on the same decorator, the same ordering and the same clean shutdown as your JSON events.

Try it

Author: William Steve Rodríguez Villamizar (Wisrovi)

Top comments (1)

Collapse
 
william_rodriguez_65a5898 profile image
William Rodriguez •

In event-driven Python microservices, maintaining explicit control over consumer rebalance timeouts while executing decoupled handlers is essential. Declarative subscription decorators streamline message routing while keeping offset commit contracts strictly deterministic.

Do your architectures favor centralized orchestrator sinks or localized idempotent consumer handlers for event pipelines?