Why modern real-time apps require 5 different systems — and why I built one instead

Jamal Saad — Mon, 09 Mar 2026 10:29:09 +0000

Modern apps are expected to feel live.

Chat apps, AI products, collaborative tools, notifications, activity feeds — they all need fast writes, real-time updates, and storage that does not become expensive as data grows.

But the usual backend stack gets complicated fast.

You store data in one system.
You push live updates with another.
You archive old data somewhere cheaper.
You add extra glue code to keep everything in sync.

That complexity is exactly what pushed me to build KalamDB.

KalamDB is an open-source, SQL-first real-time database designed around a simple idea: keep recent data fast, move older data to cheaper storage, and let developers subscribe to changes directly from the database. The project is explicitly aimed at speed, efficiency, and minimal resource use, with a goal of reducing CPU, memory, storage, and infrastructure cost.

At the core of KalamDB is a two-tier storage model.

New writes first land in RocksDB hot storage, which gives very fast acknowledgments and keeps recent data close for quick reads. Later, that data is flushed into columnar Parquet files in filesystem or object storage, where it becomes much cheaper to keep for the long term. Queries can read across both tiers, so applications get fresh data from hot storage and older history from cold storage without needing separate systems.

KalamDB also supports three table types, and that matters a lot for cost and design.

USER tables isolate data per user and flush each user’s partition into separate Parquet files. That makes them a strong fit for chat history, tenant data, and privacy-sensitive workloads.

SHARED tables store global data once for everyone, which works well for configuration data, reference datasets, and cross-user analytics.

STREAM tables are designed for short‑lived real-time events like typing indicators, presence, AI thinking signals, or notifications. They stay in memory or RocksDB hot storage, use TTL-based eviction, and are never flushed to Parquet.

Another big part of the design is the table-per-user architecture.

Instead of putting all users into one giant shared table and filtering constantly, KalamDB stores each user’s data in isolated partitions. That makes real-time subscriptions simpler and helps the system scale more naturally for workloads like conversations, notifications, and AI history. The idea is to reduce complexity while keeping performance predictable as the number of users grows.

What it feels like to use KalamDB

Instead of constantly polling the database for updates, applications can subscribe to changes directly from SQL.

For example, a chat application can subscribe to new messages in a conversation:

SUBSCRIBE TO chat.messages
WHERE conversation_id = 1
OPTIONS (last_rows = 20);

Whenever a new row is inserted into the table, the client instantly receives the update through a WebSocket connection.

The same idea can be used from application code using the JavaScript SDK:

import { createClient, Auth, SeqId } from 'kalam-link';

const client = createClient({
  url: 'http://localhost:8080',
  authProvider: async () => Auth.jwt('<ACCESS_TOKEN>'),
});

const stopMessages = await client.live(
  `SELECT id, room, role, content, created_at
    FROM chat.messages`,
  (rows) => {
    // `chat.messages` can be a USER table.
    // The same query runs for every signed-in customer, but KalamDB only
    // returns that caller's rows.
    renderRows(rows);
  },
  {
    subscriptionOptions: {
      last_rows: 200,
      from: SeqId.from('7262216745594062848'),
    },
  },
);

await client.query(
  'INSERT INTO chat.messages (room, role, content) VALUES ($1, $2, $3)',
  ['main', 'user', 'hello from the browser']
);

This is the type of pattern KalamDB was designed for: real-time apps where data storage and live updates happen in the same system.

That is the real reason I built KalamDB.

Not to add another database to the world, but to cut down the amount of infrastructure developers have to stitch together just to build something real-time.

Less glue code.
Less storage waste.
Less operational overhead.
And a simpler path from fast writes to cheaper long-term storage.

KalamDB is still in development, and I would describe file storage/BLOB support and high availability as part of the direction, not current production-ready features yet. But the core idea is already clear: a database built to make real-time apps simpler and cheaper to run.

If you want to explore KalamDB:

🌐 Website: kalamdb.org
⭐ GitHub: KalamDB on GitHub
🚀 Quick Start: Run KalamDB locally
🧠 Docs: Learn the SQL syntax

DEV Community: Jamal Saad

Why modern real-time apps require 5 different systems — and why I built one instead

What it feels like to use KalamDB