Manu Shukla

Posted on Jul 27 • Originally published at ecorpit.com

Real-time analytics in 2026: Lakehouse//RT vs ClickHouse, Druid and Pinot

#realtimeanalytics #olap #databricks #clickhouse

Real-time analytics in 2026: Lakehouse//RT vs ClickHouse, Druid and Pinot

Summary. On 16 June 2026 Databricks launched Lakehouse//RT, a real-time query engine (code-named Reyden) that Databricks says delivers sub-100 millisecond latency at 12,000 queries per second directly on governed Delta Lake and Apache Iceberg tables, with response times as low as 10ms on smaller datasets. The pitch is blunt: stop running a separate real-time serving database next to your lakehouse. For the last decade the answer to low latency at high concurrency has been a dedicated engine, usually ClickHouse, Apache Druid or Apache Pinot, and those engines are fast and proven. But a second store means data copies, change-data-capture pipelines, and split governance. This guide compares all four on latency, concurrency, ingestion, governance, cost and operational burden, and gives a decision framework for when a dedicated serving tier still wins and when consolidating on the lakehouse makes sense. Entry pricing already spans a wide band, from ClickHouse Cloud near $66 a month on Basic and $499 a month on Scale, up to StarTree Cloud tiers near $999 and $3,999 a month.

What Lakehouse//RT actually changes

Real-time online analytical processing (OLAP) means answering aggregation and filter queries over large tables in milliseconds, while fresh rows keep landing. The classic pattern serves those queries from a purpose-built store. You run your history and transformations in a warehouse or lakehouse, then copy a slice into ClickHouse, Druid or Pinot to serve user-facing dashboards and, increasingly, AI agents that poll data in loops.

Databricks wants to remove the copy. Lakehouse//RT queries Delta and Iceberg tables in place, inside Unity Catalog governance, with no proprietary format and no synchronization pipeline. Reyden uses a fully asynchronous execution model so that latency holds as concurrency climbs into the tens of thousands of users and agents. On Databricks' own benchmarks the engine reaches sub-100ms at 12,000 QPS, and the company reports response times from 10ms on smaller datasets to about 100ms on larger ones.

The performance headline is a comparison, and it is a vendor claim: Databricks says customers have seen up to 16x better performance than the specialized real-time serving stacks they used to run alongside the lakehouse. Treat that number as marketing until you test your own workload. Two named customers put softer figures on the record. Chris Kopek, Head of Data Platforms at Cisco, described "millisecond performance on live data with 5x improvement in response time" for threat-lookup queries. Kayvon Raphael, Senior Director of Engineering at Magnite, reported "sub-200 millisecond performance on our core dashboard queries, consistently." Lakehouse//RT is in Beta as of the June 2026 announcement, which matters for anyone weighing it against battle-tested engines.

The strategic quote came from the top. "Lakehouse//RT completes the engine spectrum, providing the millisecond speed layer that people want and agents require," said Ali Ghodsi, Co-founder and CEO of Databricks. "Just as we proved that the best data warehouse is a lakehouse, now, the best real-time analytics engine is the lakehouse, too."

The three incumbents: ClickHouse, Druid and Pinot

The dedicated engines earned their place, and each was built for a slightly different job.

ClickHouse is the strongest general-purpose real-time OLAP database in 2026 on the widely cited ClickBench suite, which ClickHouse itself maintains. It handles flexible, ad-hoc analytical queries at very high speed, and it has the simplest operations of the three. ClickHouse dropped its ZooKeeper dependency in favour of ClickHouse Keeper, a Raft-based consensus component, which cuts the number of moving parts. Where it strains is very high concurrency: latency rises as concurrent query load grows, so serving tens of thousands of simultaneous user-facing requests takes careful sizing, admission control and replicas, as ClickHouse's own concurrency sizing guide sets out.

Apache Pinot is the specialist. LinkedIn built and open-sourced it to power features like "who viewed your profile," and it is engineered for one extreme: very high concurrency on user-facing surfaces, with predictable low latency even as data volume grows. Independent write-ups and vendor benchmarks put Pinot in the hundreds of thousands of queries per second with consistent sub-100ms, and in some point-lookup patterns sub-10ms, response. The cost is complexity. Pinot is a multi-component cluster with several node types plus ZooKeeper, which makes small deployments and ad-hoc exploration a poor fit. StarTree Cloud is the managed Pinot service for teams that want the performance without running the cluster.

Apache Druid is the elder of the group, tuned for time-series and streaming ingestion from Apache Kafka. It shares Pinot's architectural weight: five or six node types and ZooKeeper, a high operational burden. In head-to-head latency tests Druid has trailed the newer engines. The StarTree comparison A tale of three real-time OLAP databases reports a Druid median hot-cache latency near 1.77 seconds with tail latencies far higher on queries where ClickHouse returned in a fraction of a second, though that benchmark comes from a Pinot vendor and should be read with the same skepticism as any vendor number. Imply Polaris is the managed Druid option.

Here is how the four line up on the dimensions that decide a real-time serving choice.

Dimension	Lakehouse//RT	ClickHouse	Apache Druid	Apache Pinot
Primary design goal	Real-time queries in the governed lakehouse, no copy	General-purpose fast OLAP, flexible queries	Time-series and streaming analytics	User-facing analytics at extreme concurrency
Storage and format	Delta Lake, Apache Iceberg (in place)	Native columnar (MergeTree)	Segment-based columnar	Segment-based columnar
Latency (reported)	10-100ms; sub-100ms at 12,000 QPS	Sub-second on most queries	Higher; strong on time-series	Sub-100ms, sub-10ms on point lookups
Concurrency sweet spot	Tens of thousands of users and agents	High, but latency rises with load	Moderate	Hundreds of thousands of QPS
Ingestion	Direct on lakehouse tables, no CDC	Batch and streaming inserts	Kafka streaming, batch	Kafka streaming, batch
Governance	Native Unity Catalog	External or app-managed	External or app-managed	External or app-managed
Ops burden	Managed inside Databricks	Low (ClickHouse Keeper)	High (many node types plus ZooKeeper)	High (many node types plus ZooKeeper)
Managed option and entry price	Billed within Databricks compute	ClickHouse Cloud from ~$66/mo	Imply Polaris, usage-based	StarTree Cloud, Standard ~$999/mo

The numbers that matter, and how to read them

Every headline figure here is either a vendor benchmark or a vendor-adjacent comparison, so the useful skill is knowing which number applies to your workload.

Concurrency is the sharpest divide. Databricks positions Lakehouse//RT for "tens of thousands" of concurrent users and agents. Pinot's design target is an order of magnitude higher, in the hundreds of thousands of QPS, which is why LinkedIn-scale, per-user analytics has been Pinot territory since LinkedIn open-sourced the engine. If you are serving a public dashboard to millions of end users where every page load fires several queries, Pinot's architecture still has headroom that the others do not advertise. If your concurrency is internal analysts, operational dashboards and a growing fleet of AI agents in the tens of thousands, all four are in range and the decision moves to governance and cost.

Latency numbers need their context attached. Databricks' 10-100ms is measured on standard analytical benchmarks with Reyden's asynchronous engine. ClickHouse's sub-second results come from ClickBench, which favours the query shapes ClickHouse optimises. Pinot's sub-10ms applies to denormalized point lookups, not arbitrary joins. Comparisons such as Tinybird's ClickHouse-versus-Pinot analysis and the JusDB three-way write-up agree on the shape of the field even where the exact milliseconds differ: ClickHouse for flexible speed, Pinot for concurrency, Druid for streaming time-series. A benchmark that does not match your query pattern, data size and freshness requirement tells you very little.

The honest reading in mid-2026 is that all four can hit interactive latency for well-designed workloads. What separates them is what happens at the edges: Pinot holds latency at concurrency the others cannot, ClickHouse gives the most flexibility per operational rupee, Druid remains a streaming-time-series tool, and Lakehouse//RT trades some of that specialisation for a single governed copy of the data.

Cost: what you actually pay

Sticker price is only the visible layer. The real cost of a dedicated serving engine includes the second copy of the data, the CDC or streaming pipeline that keeps it fresh, the cluster operators, and the governance work to keep permissions consistent across two systems.

On published entry pricing, ClickHouse is the cheapest way in. ClickHouse Cloud starts near $66 a month on its Basic tier and moves to roughly $499 a month on Scale, with usage-based compute and storage on top, according to 2026 ClickHouse pricing teardowns; the open-source engine is free to self-host if you can run it. StarTree Cloud lists a Developer tier near $199 a month and a Standard tier near $999 a month for five compute units and 1 TB of storage, with a Premium tier near $3,999 a month. Imply Polaris, the managed Druid service, uses usage-based pricing and asks larger workloads to contact sales.

Lakehouse//RT does not carry a separate line item in the same way. It is billed through Databricks compute, and its cost argument is subtraction: no second cluster, no CDC pipeline, no duplicate storage, no separate governance tooling. For a team already standardised on Databricks, that removed overhead can outweigh the raw per-query efficiency of a tuned ClickHouse node. For a team not on Databricks, standing up the whole platform to get the real-time engine is the wrong order of operations, and a dedicated engine is almost always cheaper. The same total-cost logic runs through our guides to cloud FinOps for Indian teams and to broader AI infrastructure planning: the migration and the operating model usually cost more than the software.

Governance and data freshness

The strongest part of the Lakehouse//RT argument is not speed, it is governance. Every query runs inside Unity Catalog, so policies, permissions and auditing are defined once and apply to real-time serving the same way they apply to batch analytics. A separate serving engine means a second permission model to keep in sync, and a copied dataset that is, by definition, slightly stale and separately secured.

That data-copy problem is also a compliance problem. Under India's Digital Personal Data Protection Act 2023, every additional copy of personal data is another store to secure, map and account for in a breach scenario, and RBI-regulated payment data carries localization expectations on top. Fewer copies is a smaller attack surface and a shorter audit. The trade-off decision echoes the one we walk through in Databricks Lakebase versus self-managed Postgres and in managed knowledge base versus self-managed RAG: a managed, consolidated store reduces governance surface, while a self-managed specialist gives you more control and, often, lower unit cost at scale.

Decision framework: drop the serving tier, or keep it

The question is not which engine is fastest in the abstract. It is which one fits your concurrency, your existing stack, and your tolerance for operational work.

Your situation	Best fit in 2026	Why
Already all-in on Databricks; governance sprawl is the pain	Lakehouse//RT (once past Beta)	One governed copy, no CDC, no second cluster
Public, user-facing analytics at hundreds of thousands of QPS	Apache Pinot (StarTree if managed)	Built for extreme concurrency with stable latency
Flexible, ad-hoc analytics; small team; cost-sensitive	ClickHouse	Fastest general-purpose engine, simplest to run
Heavy Kafka time-series streaming, existing investment	Apache Druid (Imply if managed)	Purpose-built for streaming time-series
Not on a lakehouse and do not plan to be	ClickHouse or Pinot	Adding a whole platform for one engine is the wrong trade

A dedicated serving tier still wins in three clear cases: when you need concurrency above the tens of thousands that Lakehouse//RT targets, when you are not on Databricks, or when a tuned open-source engine on your own hardware beats managed pricing at your scale. Consolidating on the lakehouse wins when you already run Databricks, when fragmented governance and stale copies are your actual bottleneck, and when your concurrency sits inside Reyden's stated range. One caveat outranks the rest: Lakehouse//RT is Beta. For a production system with a hard latency budget, a proof of concept on your real query mix, not a vendor benchmark, is the only number worth acting on.

India-specific considerations

For Indian engineering teams the calculus tilts slightly toward consolidation, for two reasons. First, cost discipline: running a separate ClickHouse, Druid or Pinot cluster with its own operators is real spend that a lakehouse-native engine can absorb, and cloud budgets in Indian product teams are watched closely. Second, data residency: the DPDP Act 2023 and RBI's data-localization expectations both punish uncontrolled data sprawl, and every copied serving store is a new residency and security obligation. Against that, the practical reality is that many Indian teams are not on Databricks and run open-source ClickHouse precisely because it is cheap and simple to operate. For them the right move is usually to master ClickHouse, not to adopt a platform to get its real-time layer.

FAQ

What is Lakehouse//RT and when did it launch?

Lakehouse//RT is Databricks' real-time query engine, code-named Reyden, announced on 16 June 2026 at the Data + AI Summit and available in Beta. It runs analytical queries directly on governed Delta Lake and Apache Iceberg tables, reaching sub-100 millisecond latency at 12,000 queries per second on Databricks' own benchmarks.

Is Lakehouse//RT faster than ClickHouse, Druid or Pinot?

It depends on the workload. Databricks reports up to 16x better performance than existing serving stacks, but that is a vendor claim on unspecified comparisons. ClickHouse leads general-purpose benchmarks, Pinot leads at extreme concurrency, and Druid suits streaming time-series. Test your own query mix before trusting any single number.

When should I keep a dedicated real-time serving database?

Keep a dedicated engine when you serve user-facing analytics at hundreds of thousands of queries per second, when you are not on Databricks, or when a tuned open-source engine on your own hardware costs less at your scale. Apache Pinot in particular targets concurrency levels above Lakehouse//RT's stated tens of thousands.

How much do these engines cost?

ClickHouse Cloud starts near $66 a month on Basic and around $499 a month on Scale, with the open-source engine free to self-host. StarTree Cloud lists a Standard tier near $999 a month, and Imply Polaris uses usage-based pricing. Lakehouse//RT is billed within Databricks compute rather than as a separate product.

Which engine is best for AI agents querying data?

Agents poll data in loops, so they need consistent low latency at high concurrency. Lakehouse//RT was designed explicitly for agents that need fresh, governed data without copies. Apache Pinot suits agents that generate very high query volumes. The deciding factors are whether your data already lives in a lakehouse and how strict your governance needs are.

Does consolidating on the lakehouse help with DPDP compliance?

Indirectly, yes. Every separate serving store is another copy of data to secure, map and account for under India's Digital Personal Data Protection Act 2023, and RBI localization rules add pressure on payment data. Querying one governed copy inside Unity Catalog reduces the number of stores you must protect and audit, which shrinks compliance surface.

What are the operational trade-offs between these engines?

ClickHouse has the lightest operations of the open-source three after replacing ZooKeeper with ClickHouse Keeper. Apache Druid and Apache Pinot run several node types plus ZooKeeper, a heavier burden that managed services like Imply and StarTree exist to absorb. Lakehouse//RT shifts operations to Databricks entirely, at the cost of platform lock-in.

Is Lakehouse//RT production-ready?

Not yet in the strictest sense. As of the June 2026 announcement it is in Beta, while ClickHouse, Druid and Pinot are proven in production at large scale. For a system with a hard latency budget, run a proof of concept on your real workload before committing, and keep a dedicated engine as the fallback until Lakehouse//RT reaches general availability.

How eCorpIT can help

eCorpIT is a Gurugram-based, CMMI Level 5 and ISO 27001:2022 certified engineering organisation that builds and runs real-time data platforms for product and enterprise teams. Our senior engineers benchmark ClickHouse, Apache Druid, Apache Pinot and Lakehouse//RT against your actual query mix, concurrency and freshness targets, then design the serving tier and governance model that fits your stack and your DPDP obligations. Whether you consolidate on a lakehouse or run a tuned dedicated engine, we handle the migration, the ingestion pipelines and the cost model. Tell us your workload at /contact-us/ and we will size the right architecture.