DEV Community: Максим Кремнев

Stop Polling Your Database: WebSocket + Redis Pub/Sub for Real-Time Data

Максим Кремнев — Wed, 04 Feb 2026 18:42:55 +0000

Article

Polling your database every second isn't real-time. It's an expensive illusion.

I was building a Telegram channel monitoring system and needed instant updates in the browser. My first instinct was the "simple" approach: poll the database every second and check for new messages.

It worked. But it also created problems that got worse as the system grew.

The Polling Problem

Here's what happens when you poll:

Constant load with no payoff. Your database handles queries even when there's nothing new. 10 clients polling every second = 10 queries per second. 1000 clients = 1000 queries. Most return empty results.

Delays that users notice. A 1-second polling interval means up to 1 second of latency. Users feel this, especially in chat-like interfaces. And if you poll less frequently to reduce load, the delays get worse.

Scaling becomes painful. More clients means linearly more database queries. Your "simple" solution is now your bottleneck.

Code complexity creeps in. You need timers, deduplication logic, error handling for failed polls, backoff strategies. The "simple" solution isn't simple anymore.

The Solution: Let Data Push Itself

What if, instead of constantly asking "is there new data?", the data just showed up when it existed?

That's the mental shift. Stop pulling. Start pushing.

The pattern I use:

Event Source → Redis Pub/Sub → WebSocket → Browser

Three components, each with a clear job:

Redis Pub/Sub acts as a message bus. It broadcasts events to all interested subscribers.
WebSocket maintains a persistent connection between server and browser. No repeated handshakes.
The database stays out of the delivery path entirely. It stores data. Redis broadcasts it.

How It Works in Practice

Here's the concrete flow from my Telegram monitoring project:

Telegram API → Telethon Handler → Redis Pub/Sub → FastAPI WebSocket → React

When a new message appears in a monitored Telegram channel:

Telethon (Python Telegram client) receives the event
Handler publishes the message to a Redis channel
WebSocket server, subscribed to that Redis channel, receives it
Server pushes it to all connected browsers
React updates the UI

Time from Telegram event to browser: 50-100ms.

Database queries for delivery: zero.

The Code

Let me show you the key pieces. This isn't a complete tutorial, but enough to understand the pattern.

Publishing to Redis

When new data arrives, publish it:

from redis.asyncio import Redis

class BroadcastService:
    def __init__(self, redis: Redis):
        self.redis = redis

    async def broadcast(self, channel: str, data: dict):
        await self.redis.publish(
            channel,
            json.dumps(data)
        )

Usage is straightforward:

# In your event handler
await broadcast_service.broadcast(
    f"channel:{channel_id}",
    {"type": "new_message", "data": message_data}
)

WebSocket Manager with Redis Subscription

The WebSocket server subscribes to Redis and forwards messages to connected clients:

from fastapi import WebSocket
from redis.asyncio import Redis

class ConnectionManager:
    def __init__(self, redis: Redis):
        self.redis = redis
        self.connections: dict[str, list[WebSocket]] = {}

    async def subscribe(self, websocket: WebSocket, channel: str):
        # Track connection
        if channel not in self.connections:
            self.connections[channel] = []
        self.connections[channel].append(websocket)

        # Subscribe to Redis channel
        pubsub = self.redis.pubsub()
        await pubsub.subscribe(channel)

        try:
            async for message in pubsub.listen():
                if message["type"] == "message":
                    await websocket.send_text(message["data"])
        finally:
            await pubsub.unsubscribe(channel)
            self.connections[channel].remove(websocket)

The FastAPI endpoint:

@app.websocket("/ws/{channel_id}")
async def websocket_endpoint(websocket: WebSocket, channel_id: str):
    await websocket.accept()
    await manager.subscribe(websocket, f"channel:{channel_id}")

Frontend: useWebSocket Hook

On the client side, a custom hook handles the connection with automatic reconnection:

import { useEffect, useRef, useState, useCallback } from 'react';

type WebSocketStatus = 'connecting' | 'connected' | 'disconnected';

interface UseWebSocketOptions {
  reconnect?: boolean;
  reconnectInterval?: number;
  maxRetries?: number;
}

export function useWebSocket<T>(
  url: string,
  options: UseWebSocketOptions = {}
) {
  const {
    reconnect = true,
    reconnectInterval = 3000,
    maxRetries = 5,
  } = options;

  const [data, setData] = useState<T | null>(null);
  const [status, setStatus] = useState<WebSocketStatus>('disconnected');
  const wsRef = useRef<WebSocket | null>(null);
  const retriesRef = useRef(0);

  const connect = useCallback(() => {
    setStatus('connecting');
    const ws = new WebSocket(url);

    ws.onopen = () => {
      setStatus('connected');
      retriesRef.current = 0;
    };

    ws.onmessage = (event) => {
      const parsed = JSON.parse(event.data) as T;
      setData(parsed);
    };

    ws.onclose = () => {
      setStatus('disconnected');
      if (reconnect && retriesRef.current < maxRetries) {
        retriesRef.current++;
        setTimeout(connect, reconnectInterval);
      }
    };

    wsRef.current = ws;
  }, [url, reconnect, reconnectInterval, maxRetries]);

  useEffect(() => {
    connect();
    return () => wsRef.current?.close();
  }, [connect]);

  return { data, status };
}

Usage:

function ChannelMonitor({ channelId }: { channelId: string }) {
  const { data, status } = useWebSocket<Message>(
    `wss://api.example.com/ws/${channelId}`
  );

  if (status === 'connecting') return <Spinner />;
  if (!data) return <Empty />;

  return <MessageList messages={data} />;
}

Gotchas I Learned the Hard Way

This pattern works great, but there are a few things that will bite you if you're not careful:

Connection Management

When a client disconnects (closes tab, loses network), clean up properly. Orphaned subscriptions leak memory and can cause issues with Redis.

# Always use try/finally for cleanup
try:
    async for message in pubsub.listen():
        await websocket.send_text(message["data"])
finally:
    await pubsub.unsubscribe(channel)
    # Remove from connection tracking

Graceful Shutdown

When you deploy a new version, existing WebSocket connections need to close gracefully. Don't just kill the process. Signal clients to reconnect, drain connections, then shut down.

Reconnection Strategy

Clients will disconnect. Network blips happen. Your frontend needs to reconnect automatically, but with backoff. Don't hammer the server with immediate reconnection attempts.

// Exponential backoff
const delay = Math.min(
  reconnectInterval * Math.pow(2, retriesRef.current),
  30000 // cap at 30 seconds
);
setTimeout(connect, delay);

Keep Connections Alive

WebSocket connections can go stale. Implement ping/pong heartbeats to detect dead connections and clean them up.

# Server-side ping
async def heartbeat(websocket: WebSocket):
    while True:
        await asyncio.sleep(30)
        try:
            await websocket.send_text('{"type": "ping"}')
        except:
            break  # Connection dead, cleanup will happen

The Results

After switching from polling to WebSocket + Redis Pub/Sub:

Metric	Polling	WebSocket + Redis
Latency	1-5 seconds	<100ms
DB load for delivery	High	Zero
Scales with clients	Poorly	Well
Code complexity	Growing	Contained

The architecture is cleaner too. Database handles persistence. Redis handles broadcasting. WebSocket handles delivery. Each component does one thing well.

When to Use This Pattern

This pattern shines for:

Real-time dashboards and monitoring
Chat and messaging features
Live notifications
Collaborative editing
Any "instant update" requirement

It might be overkill for:

Simple apps with few users
Updates that can wait minutes
Systems where eventual consistency is fine

Wrapping Up

Polling was the right solution for a different era. If you're building something that needs real-time updates today, WebSocket + Redis Pub/Sub is a battle-tested pattern that scales.

The key insight: separate data persistence from data delivery. Your database stores data. Redis broadcasts it. WebSocket delivers it.

Each component does what it's good at. That's good architecture.

What's your approach to real-time in your projects? Have you tried this pattern? I'd love to hear about your experiences in the comments.

# How I Automated Content Creation with CLI and AI

Максим Кремнев — Tue, 03 Feb 2026 12:00:50 +0000

Every time I sat down to write a post, I started from scratch. After six months, I had 50 posts — and not a single one with a consistent style. One read like API documentation, another like a tweet after three cups of coffee.

Then I built a CLI tool. And this post is a live demo of how it works.

The Problem: Why Content Creation Turns Into Chaos

Sound familiar? You sit down to write a post. Open a blank document. Stare at the cursor. The cursor stares back.

After 20 minutes fighting writer's block, you finally write something. Publish it. Then look at your previous post and realize: these could be from two different authors.

I noticed several patterns in my "creative process":

Every post = reinventing the wheel. No system. No template. Every time I'm deciding from scratch: what structure? what tone of voice? who am I writing for?

No consistent voice. One post is formal, another is packed with memes and emojis. Readers don't know what to expect.

AI used chaotically. "Write me a post about microservices" — and you get generic text with no context, no personal style, no understanding of the audience.

No reproducibility. Which post performed well? Why did it work? What did I do differently? Black box.

As a developer, I thought: what if I applied the same approach we use for code to content?

The Idea: Infrastructure as Code for Content

When you start a new React project, you don't create files manually. You run create-react-app or npm create vite — and get a ready structure. Configs, folders, base files — everything in place.

What if we did the same for content?

That's how creator-kit was born — a CLI tool that scaffolds the structure for AI-assisted content creation.

The concept is simple:

Code scaffolding	Content scaffolding
`create-react-app`	`creator init`
`package.json`	`constitution.md`
`src/components/`	`templates/content/`
npm scripts	workflow commands

Constitution — your "source of truth". A document that describes your voice, tone, and audience. Set it up once — all AI commands use this context.

Templates — templates for different content types. LinkedIn post, Twitter thread, Telegram — each has its own structure.

Workflow commands — slash commands that guide you from idea to finished post. Step by step.

Let's see how it works in practice.

Practice: Installation and First Run

Step 1: Installation

npm install -g creator-kit

Takes about 10 seconds. Package weighs less than 500KB — only what you need.

Step 2: Initialize the Project

Navigate to the directory where you want to manage content and run:

creator init --ai claude

The --ai claude flag indicates you're using Claude Code (currently the only supported AI, but Qwen and Codex are planned).

Step 3: See What Appeared

.claude/commands/              # Slash commands for Claude
├── creator.brief.md
├── creator.constitution.md
├── creator.outline.md
├── creator.refine.md
├── creator.review.md
├── creator.todo.md
├── creator.verify.md
└── creator.write.md

.contents/                     # Templates and memory
├── memory/
│   └── constitution.md        # ← Your voice lives here
├── templates/
│   ├── content/               # Post templates
│   │   ├── linkedin-post-template.md
│   │   ├── telegram-post-template.md
│   │   └── twitter-thread-template.md
│   └── workflow/              # Process templates
│       ├── content-brief-template.md
│       ├── content-plan-template.md
│       └── content-tasks-template.md
└── config.json

Files are in place. Now for the interesting part — the workflow.

Workflow: From Idea to Finished Post

Here's what the content creation process looks like with creator-kit:

/creator.constitution → /creator.brief → /creator.refine → /creator.outline → /creator.todo → /creator.write → /creator.review → /creator.verify

Let's break down each step.

1. Define Your Voice (done once)

/creator.constitution

AI asks you questions:

Who is your audience?
What tone: formal, casual, somewhere in between?
What topics do you cover?
What do you avoid?

Answers are saved to constitution.md. This is your "brand guide" — all subsequent commands will take it into account.

2. Create a Brief for Your Idea

/creator.brief "How I automated content creation with CLI"

AI takes your raw idea and turns it into a structured brief:

Working title
Target audience
Key message
Publishing platform
Main points

If the idea is vague, AI will ask clarifying questions.

3. Refine (optional)

/creator.refine

Socratic method: AI asks questions, you answer. The brief becomes more precise.

"What specific outcome do you want the reader to get?"
"Do you have examples from your practice?"
"What makes your approach different from existing ones?"

4. Create the Structure

/creator.outline

AI generates an article plan:

Hook (how to grab the reader)
Sections with key points
Transitions between sections
Call to action

5. Break Down into Tasks

/creator.todo

The outline turns into specific tasks:

Write the hook
Develop section 1
Add code examples
Formulate CTA

Convenient if you write in multiple sessions.

6. Write

/creator.write

AI takes the brief, outline, constitution — and generates a draft. Not generic text, but content in your style, for your audience, following your structure.

7. Check Quality

/creator.review

AI analyzes the draft:

Does it match the constitution?
Is there actionable value?
Is the text clear to the target audience?
Does the hook work?

/creator.verify

Final checklist before publishing.

Meta: This Post as an Example

By the way, this post was created exactly this way.

/creator.brief — generated a brief with the key message: "scaffolding for content"
/creator.outline — got the section structure
/creator.write — AI wrote the draft
I edited, added personal examples

The brief for this post lives in .contents/briefs/006-creator-kit-practical-guide.md.

The constitution defined:

Casual tone (like explaining to a friend)
Focus on practice (code, commands, examples)
Audience — developers of all levels

This isn't magic. It's a system.

Under the Hood (for those curious)

Why CLI, not a web app?

Locality. Your data stays on your machine. Constitution, briefs, drafts — all in your project.

Integration. Works in the same terminal where you code. No need to switch between windows.

Version control. Files are text. You can commit to git, view change history, roll back.

Tech Stack

TypeScript 5.x — strict typing
Commander.js — CLI parsing
fs-extra — file operations
Chalk — colored output

Three production dependencies. That's it.

Smart Update

creator update

Updates templates to the new version without touching files you've modified. Compares hashes.

If you need to force update — --force. Old files are saved as .bak.

Conclusion

Creator-kit is not a "magic AI writer". AI still writes generic text if you don't give it context.

But:

Constitution = your voice, captured in a file
Templates = your processes, reproducible every time
Workflow = system instead of chaos

AI becomes an assistant that works by your rules.

Try It

npm install -g creator-kit
cd your-project
creator init --ai claude

Create your first brief:

/creator.brief "Your idea for a post"

And see what you get.

What content workflow do you use? Share in the comments — curious to compare approaches.

Links:

npm: creator-kit
License: MIT
Source code: open source