DEV Community

Alex Spinov
Alex Spinov

Posted on

Redpanda Has a Free API — Kafka-Compatible Streaming Without the JVM

#redpanda #kafka #streaming #devops

TL;DR

Redpanda is a Kafka-compatible streaming platform written in C++ — no JVM, no ZooKeeper, no garbage collection pauses. It's 10x faster to start, uses less memory, and is wire-compatible with Kafka clients.

What Is Redpanda?

Redpanda is a modern streaming data platform:

  • Kafka API compatible — use existing Kafka clients, no code changes
  • No JVM — written in C++ with Seastar framework
  • No ZooKeeper — built-in Raft consensus
  • 10x faster startup — seconds, not minutes
  • Lower latency — P99 under 10ms vs Kafka's 50-200ms
  • Free — BSL license, or Redpanda Cloud free tier

Quick Start 

# Docker (single node)
docker run -d --name redpanda \
  -p 9092:9092 -p 8081:8081 -p 8082:8082 -p 9644:9644 \
  docker.redpanda.com/redpandadata/redpanda:latest \
  redpanda start --smp 1 --memory 1G --overprovisioned

# Create a topic
docker exec -it redpanda rpk topic create events --partitions 3

# List topics
docker exec -it redpanda rpk topic list
Enter fullscreen mode Exit fullscreen mode

Using with Node.js (KafkaJS) 

import { Kafka } from "kafkajs";

const kafka = new Kafka({
  clientId: "my-app",
  brokers: ["localhost:9092"],
});

// Producer
const producer = kafka.producer();
await producer.connect();

await producer.send({
  topic: "events",
  messages: [
    { key: "user-123", value: JSON.stringify({ action: "click", page: "/home" }) },
    { key: "user-456", value: JSON.stringify({ action: "signup", email: "test@example.com" }) },
  ],
});

// Consumer
const consumer = kafka.consumer({ groupId: "analytics" });
await consumer.connect();
await consumer.subscribe({ topic: "events", fromBeginning: true });

await consumer.run({
  eachMessage: async ({ topic, partition, message }) => {
    const event = JSON.parse(message.value.toString());
    console.log(`Event: ${event.action} from partition ${partition}`);
  },
});
Enter fullscreen mode Exit fullscreen mode

Schema Registry (Built-in) 

# Redpanda includes a schema registry on port 8081
curl -X POST http://localhost:8081/subjects/events-value/versions \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  -d '{
    "schema": "{\"type\":\"record\",\"name\":\"Event\",\"fields\":[{\"name\":\"action\",\"type\":\"string\"},{\"name\":\"timestamp\",\"type\":\"long\"}]}"
  }'
Enter fullscreen mode Exit fullscreen mode

HTTP Proxy (REST API) 

# Produce via HTTP (no Kafka client needed!)
curl -X POST http://localhost:8082/topics/events \
  -H "Content-Type: application/vnd.kafka.json.v2+json" \
  -d '{
    "records": [
      {"key": "user-1", "value": {"action": "purchase", "amount": 99.99}}
    ]
  }'
Enter fullscreen mode Exit fullscreen mode

Redpanda Console (Built-in UI) 

# docker-compose.yml — add Console
services:
  redpanda-console:
    image: docker.redpanda.com/redpandadata/console:latest
    ports: ["8080:8080"]
    environment:
      KAFKA_BROKERS: redpanda:9092
      KAFKA_SCHEMAREGISTRY_URLS: http://redpanda:8081
Enter fullscreen mode Exit fullscreen mode

Redpanda vs Kafka vs Pulsar

Feature Redpanda Kafka Pulsar
Language C++ Java (JVM) Java (JVM)
ZooKeeper Not needed KRaft (new) ZK or Oxia
Startup time Seconds Minutes Minutes
P99 latency <10ms 50-200ms 20-100ms
Memory usage 1 GB min 6+ GB 4+ GB
Kafka compatible 100% N/A Adapter
Schema Registry Built-in Confluent add-on Built-in
Free tier Yes (Cloud) Confluent free StreamNative

Use Cases

  1. Event streaming — clickstream, IoT, logs
  2. Change Data Capture — Debezium connectors work out of the box
  3. Real-time analytics — process events as they arrive
  4. Microservice messaging — async communication between services
  5. ML feature pipelines — real-time feature computation

Resources

Need to stream web data in real-time? My Apify scraping tools extract data from any website — pipe it into Redpanda for real-time event processing. Questions? Email spinov001@gmail.com

Top comments (0)