DEV Community

Alex Spinov
Alex Spinov

Posted on

Redpanda Has a Free API: Stream Data 10x Faster Than Kafka Without JVM

#redpanda #kafka #streaming #api

Redpanda is a Kafka-compatible streaming platform written in C++ that eliminates the JVM dependency entirely. It ships with a built-in REST API (HTTP Proxy) and Admin API — both free, no registration required.

Why Redpanda Over Kafka?

  • No JVM — runs as a single binary, 10x lower latency on tail percentiles
  • Kafka API compatible — drop-in replacement, use existing Kafka clients
  • Built-in Schema Registry — no separate Confluent service needed
  • Free REST API — produce/consume messages over HTTP

Quick Start: Redpanda REST API 

# Start Redpanda with Docker
docker run -d --name redpanda \
  -p 9092:9092 -p 8082:8082 -p 9644:9644 \
  redpandadata/redpanda:latest \
  redpanda start --smp 1 --memory 1G

# Create a topic via Admin API
curl -X POST http://localhost:9644/v1/topics \
  -H 'Content-Type: application/json' \
  -d '{"topic": "events", "partition_count": 3}'

# Produce a message via HTTP Proxy
curl -X POST http://localhost:8082/topics/events \
  -H 'Content-Type: application/vnd.kafka.json.v2+json' \
  -d '{"records": [{"value": {"event": "signup", "user_id": 42}}]}'

# Consume messages
curl -X POST http://localhost:8082/consumers/my-group \
  -H 'Content-Type: application/vnd.kafka.v2+json' \
  -d '{"name": "reader", "format": "json", "auto.offset.reset": "earliest"}'

curl http://localhost:8082/consumers/my-group/instances/reader/records \
  -H 'Accept: application/vnd.kafka.json.v2+json'
Enter fullscreen mode Exit fullscreen mode

Admin API: Monitor Everything 

# Cluster health
curl http://localhost:9644/v1/cluster/health_overview

# Broker config
curl http://localhost:9644/v1/node_config

# Topic details
curl http://localhost:9644/v1/topics/events

# Partition balancer status
curl http://localhost:9644/v1/cluster/partition_balancer/status
Enter fullscreen mode Exit fullscreen mode

Redpanda Console (Free UI) 

docker run -d --name redpanda-console \
  -p 8080:8080 \
  -e KAFKA_BROKERS=host.docker.internal:9092 \
  redpandadata/console:latest
Enter fullscreen mode Exit fullscreen mode

Browse topics, messages, consumer groups, and schema registry at http://localhost:8080.

Python Client (Kafka-Compatible) 

from kafka import KafkaProducer, KafkaConsumer
import json

# Works with standard Kafka clients!
producer = KafkaProducer(
    bootstrap_servers='localhost:9092',
    value_serializer=lambda v: json.dumps(v).encode()
)

producer.send('events', {'action': 'purchase', 'amount': 99.99})
producer.flush()

consumer = KafkaConsumer(
    'events',
    bootstrap_servers='localhost:9092',
    auto_offset_reset='earliest',
    value_deserializer=lambda m: json.loads(m.decode())
)

for msg in consumer:
    print(f"Offset {msg.offset}: {msg.value}")
Enter fullscreen mode Exit fullscreen mode

When to Use Redpanda

Use Case Why Redpanda
Real-time analytics Sub-ms p99 latency
Event sourcing Kafka-compatible + simpler ops
IoT data ingestion Single binary, low resource usage
Log aggregation Built-in transforms, no separate stream processor

Key Limits

  • REST API has higher latency than native Kafka protocol (~2-5ms overhead)
  • Redpanda Cloud free tier: 1 GB storage, shared cluster
  • Self-hosted: unlimited, but you manage infrastructure

Resources

Need to process millions of events in real-time? I build custom streaming data pipelines and scrapers. Check out my web scraping toolkit on Apify or email me at spinov001@gmail.com for custom solutions.

Top comments (0)