<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Coreframe Labs</title>
    <description>The latest articles on DEV Community by Coreframe Labs (@coreframe-labs).</description>
    <link>https://dev.to/coreframe-labs</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4010825%2F7d89e542-205e-490b-91a7-350d364e11a0.png</url>
      <title>DEV Community: Coreframe Labs</title>
      <link>https://dev.to/coreframe-labs</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/coreframe-labs"/>
    <language>en</language>
    <item>
      <title>Securing the Model Context Protocol (MCP): Building a Production-Ready, OAuth 2.1 Authenticated Server over HTTP</title>
      <dc:creator>Coreframe Labs</dc:creator>
      <pubDate>Thu, 23 Jul 2026 12:27:34 +0000</pubDate>
      <link>https://dev.to/coreframelabs/securing-the-model-context-protocol-mcp-building-a-production-ready-oauth-21-authenticated-1imd</link>
      <guid>https://dev.to/coreframelabs/securing-the-model-context-protocol-mcp-building-a-production-ready-oauth-21-authenticated-1imd</guid>
      <description>&lt;h1&gt;
  
  
  Securing the Model Context Protocol (MCP): Building a Production-Ready, OAuth 2.1 Authenticated Server over HTTP
&lt;/h1&gt;

&lt;p&gt;As engineering organizations connect Large Language Models (LLMs) to sensitive business databases and internal developer systems, secure communication boundaries are no longer optional.&lt;/p&gt;

&lt;p&gt;While running Model Context Protocol (MCP) servers locally using standard input/output (stdio) depends on local process boundaries, deploying remote MCP services over HTTP transport exposes deep security risks. Relying on static, long-lived API keys creates a single point of failure and does not support multi-user data scoping.&lt;/p&gt;

&lt;p&gt;To address this challenge, we built and open-sourced &lt;strong&gt;Coreframe Labs' Secure MCP Template&lt;/strong&gt;, implementing a production-ready, spec-compliant OAuth 2.1 authorization layer on top of a Node.js/TypeScript server over Server-Sent Events (SSE).&lt;/p&gt;

&lt;h2&gt;
  
  
  The Core Problem: Why API Keys Fail in AI Agent Environments
&lt;/h2&gt;

&lt;p&gt;In typical application architectures, an API key is granted to an external system, which then has full access to the target API. However, AI agents sit at a unique intersection: they serve multiple clients (such as Claude Desktop or Cursor), act on behalf of diverse enterprise users, and execute operations dynamically.&lt;/p&gt;

&lt;p&gt;Passing static API keys through locally installed desktop agents creates significant risk:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;No User-Level Auditing:&lt;/strong&gt; The server cannot distinguish between different users executing commands through a shared client.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Over-Permissioned Access:&lt;/strong&gt; If a token is compromised, the attacker gains full access to all exposed analytical tools.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No Expiration Rules:&lt;/strong&gt; Static keys lack automatic revocation or dynamic rotation mechanisms.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Standardizing authorization via OAuth 2.1 solves these issues natively. It enforces Proof Key for Code Exchange (PKCE) for all clients, eliminates insecure implicit flows, and requires exact matching on redirect URIs.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architectural Breakdown: The Secure Remote Handshake
&lt;/h2&gt;

&lt;p&gt;Our open-source implementation follows the standard OAuth 2.1 authorization lifecycle for remote, HTTP-based MCP servers:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[ MCP Client / Host ] ───────── (1) GET /api/tools (No Token) ────────► [ Remote MCP Server ]
                      ◄──────── (2) 401 Unauthorized (PRM Pointer) ─────

[ MCP Client / Host ] ─── (3) Fetch Protected Resource Metadata ───────► [ OAuth Provider ]
                      ◄──────────── (4) Auth Endpoint &amp;amp; Scopes ──────────

[ MCP Client / Host ] ────────── (5) Authorize with PKCE (S256) ───────► [ OAuth Provider ]
                      ◄──────────────── (6) Scoped Bearer Token ─────────

[ MCP Client / Host ] ────────── (7) GET /api/tools (With Token) ───────► [ Remote MCP Server ]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  1. The Initial Handshake &amp;amp; Discovery
&lt;/h3&gt;

&lt;p&gt;When an unauthorized client attempts to access our exposed tools or resources, the server rejects the request with an HTTP &lt;code&gt;401 Unauthorized&lt;/code&gt; status. It attaches a &lt;code&gt;WWW-Authenticate&lt;/code&gt; header containing a secure pointer to our Protected Resource Metadata (PRM) document:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight http"&gt;&lt;code&gt;&lt;span class="k"&gt;HTTP&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="m"&gt;1.1&lt;/span&gt; &lt;span class="m"&gt;401&lt;/span&gt; &lt;span class="ne"&gt;Unauthorized&lt;/span&gt;
&lt;span class="na"&gt;WWW-Authenticate&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Bearer realm="mcp", resource_metadata="https://mcp-auth-demo-production-d421.up.railway.app/.well-known/oauth-protected-resource"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The client fetches this document to learn the authorization server's URL, supported token endpoints, and the specific permissions (scopes) required.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Client ID Metadata Documents (CIMD)
&lt;/h3&gt;

&lt;p&gt;To prevent unauthorized clients from dynamically registering with our enterprise server, our implementation validates the client's identity using Client ID Metadata Documents (CIMD). This configuration replaces fragile, manually configured client secrets with dynamic, cryptographically verifiable documents hosted securely on the client's domain.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Constant-Time Cryptographic Verification
&lt;/h3&gt;

&lt;p&gt;A common vulnerability in custom token validation logic is the use of non-constant-time string comparisons. Standard string operators (&lt;code&gt;==&lt;/code&gt;) return false immediately upon detecting a character mismatch, exposing the system to timing-based side-channel attacks.&lt;/p&gt;

&lt;p&gt;Our template mitigates this risk by enforcing constant-time comparisons across all token verification routes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;timingSafeEqual&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;crypto&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;verifyAccessToken&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;suppliedToken&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;expectedToken&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt; &lt;span class="nx"&gt;boolean&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;suppliedBuf&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;Buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;suppliedToken&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;expectedBuf&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;Buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;expectedToken&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;suppliedBuf&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="nx"&gt;expectedBuf&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;timingSafeEqual&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;suppliedBuf&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;expectedBuf&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Try It Yourself: Deployment &amp;amp; Live Demo
&lt;/h2&gt;

&lt;p&gt;We have deployed the live interactive demo on &lt;strong&gt;Railway&lt;/strong&gt; to demonstrate this authorization handshake in real-time.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;🎮 &lt;strong&gt;Live Interactive Demo:&lt;/strong&gt; &lt;a href="https://mcp-auth-demo-production-d421.up.railway.app/demo/" rel="noopener noreferrer"&gt;https://mcp-auth-demo-production-d421.up.railway.app/demo/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;📦 &lt;strong&gt;GitHub Repository:&lt;/strong&gt; &lt;a href="https://github.com/CoreframeLabs/mcp-auth-template" rel="noopener noreferrer"&gt;https://github.com/CoreframeLabs/mcp-auth-template&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;To spin up the template locally, clone our repository and configure your environment variables:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/CoreframeLabs/mcp-auth-template.git
&lt;span class="nb"&gt;cd &lt;/span&gt;mcp-auth-template
npm &lt;span class="nb"&gt;install
&lt;/span&gt;npm run build
npm run &lt;span class="nb"&gt;test&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Join the Discussion
&lt;/h2&gt;

&lt;p&gt;Implementing security layers for remote AI agents is a rapidly evolving space.&lt;/p&gt;

&lt;p&gt;How is your engineering team currently securing external integrations? Are you moving toward OAuth 2.1-compliant architectures, or relying on network-isolated VPC environments?&lt;/p&gt;

&lt;p&gt;Share your architectural designs, feedback, or any security bottlenecks you've encountered in the comments below!&lt;/p&gt;

</description>
      <category>javascript</category>
      <category>security</category>
      <category>ai</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Why I chose ClickHouse over PostgreSQL for a billion-row analytics platform</title>
      <dc:creator>Coreframe Labs</dc:creator>
      <pubDate>Wed, 01 Jul 2026 11:30:17 +0000</pubDate>
      <link>https://dev.to/coreframe-labs/why-i-chose-clickhouse-over-postgresql-for-a-billion-row-analytics-platform-599f</link>
      <guid>https://dev.to/coreframe-labs/why-i-chose-clickhouse-over-postgresql-for-a-billion-row-analytics-platform-599f</guid>
      <description>&lt;p&gt;Last year I was handed a problem: an enterprise IoT monitoring dashboard that took 18–22 seconds to load. The client — a large industrial operator — had engineers staring at a spinner every time they opened it. The underlying data was billions of sensor readings, stored in PostgreSQL.&lt;/p&gt;

&lt;p&gt;My job was to fix it. The solution ended up being ClickHouse. Here's exactly why, including the benchmarks and the tradeoffs I accepted to get there.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Query That Broke Postgres
&lt;/h2&gt;

&lt;p&gt;The dashboard had a "fleet overview" view that aggregated data across all sensor racks simultaneously. In simplified terms:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt;
    &lt;span class="n"&gt;rack_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;quantile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;95&lt;/span&gt;&lt;span class="p"&gt;)(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;p95_reading&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;quantile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;50&lt;/span&gt;&lt;span class="p"&gt;)(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;median_reading&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="k"&gt;count&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;               &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;sample_count&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;sensor_readings&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;ts&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;INTERVAL&lt;/span&gt; &lt;span class="mi"&gt;7&lt;/span&gt; &lt;span class="k"&gt;DAY&lt;/span&gt;
&lt;span class="k"&gt;GROUP&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;rack_id&lt;/span&gt;
&lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;p95_reading&lt;/span&gt; &lt;span class="k"&gt;DESC&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;On PostgreSQL 16 with BRIN indexes on &lt;code&gt;ts&lt;/code&gt;: &lt;strong&gt;42,000ms&lt;/strong&gt;. Query was being killed at 60 seconds in production. Users saw a broken dashboard and assumed the system was down.&lt;/p&gt;

&lt;p&gt;On ClickHouse with MergeTree partitioned by month: &lt;strong&gt;380ms&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;That's not a configuration problem. That's a fundamentally different storage model.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why Columnar Storage Wins for This Workload
&lt;/h2&gt;

&lt;p&gt;PostgreSQL stores data row by row. To aggregate a column across millions of rows, it has to read every row — even the columns you don't need.&lt;/p&gt;

&lt;p&gt;ClickHouse stores data column by column. An aggregation query reads only the columns it touches. For a table with 12 columns, that's roughly a 12x reduction in I/O before any other optimization kicks in.&lt;/p&gt;

&lt;p&gt;On top of that, columnar storage compresses repetitively structured data extremely well. IoT sensor data is highly repetitive — the same sensor IDs, the same rack IDs, the same status codes, repeated billions of times. Our 1-billion-row dataset that occupied 847GB in PostgreSQL occupied &lt;strong&gt;94GB in ClickHouse&lt;/strong&gt;. A 9x compression ratio, on the same hardware.&lt;/p&gt;

&lt;p&gt;Less data on disk means faster reads. Faster reads means faster queries. The math is straightforward.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Benchmark Results
&lt;/h2&gt;

&lt;p&gt;I ran three representative queries against the same 1-billion-row dataset on the same hardware (8-core CPU, 32GB RAM, NVMe SSD). Here's what came back:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Query 1: Count rows in a 24-hour window&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Engine&lt;/th&gt;
&lt;th&gt;Latency (p50)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;PostgreSQL 16 (BRIN index)&lt;/td&gt;
&lt;td&gt;1,240ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;TimescaleDB&lt;/td&gt;
&lt;td&gt;310ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ClickHouse&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;18ms&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Query 2: 1-hour candle aggregation over 90 days&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Engine&lt;/th&gt;
&lt;th&gt;Latency (p50)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;PostgreSQL 16&lt;/td&gt;
&lt;td&gt;8,200ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;TimescaleDB (continuous aggregate)&lt;/td&gt;
&lt;td&gt;95ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ClickHouse (AggregatingMergeTree view)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;8ms&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Query 3: Cross-rack percentile across 7 days (the killer query)&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Engine&lt;/th&gt;
&lt;th&gt;Latency (p50)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;PostgreSQL 16&lt;/td&gt;
&lt;td&gt;42,000ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;TimescaleDB&lt;/td&gt;
&lt;td&gt;18,000ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ClickHouse&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;380ms&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The third query is the one that mattered most. TimescaleDB at 18 seconds is still too slow for an interactive dashboard. ClickHouse at 380ms feels instant.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Made the Difference: Materialised Views at Insert Time
&lt;/h2&gt;

&lt;p&gt;The 8ms result on Query 2 wasn't just columnar storage. It was ClickHouse's &lt;code&gt;AggregatingMergeTree&lt;/code&gt; materialised views — which compute aggregations &lt;strong&gt;at insert time&lt;/strong&gt;, not at query time.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="n"&gt;MATERIALIZED&lt;/span&gt; &lt;span class="k"&gt;VIEW&lt;/span&gt; &lt;span class="n"&gt;candles_1h&lt;/span&gt;
&lt;span class="n"&gt;ENGINE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;AggregatingMergeTree&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;asset&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ts_bucket&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;AS&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt;
    &lt;span class="n"&gt;asset&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;toStartOfHour&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ts&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;         &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;ts_bucket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;argMinState&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ts&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;    &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="k"&gt;open&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;maxState&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;           &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;high&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;minState&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;           &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;low&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;argMaxState&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ts&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;    &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="k"&gt;close&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;sensor_readings&lt;/span&gt;
&lt;span class="k"&gt;GROUP&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;asset&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ts_bucket&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every time data is inserted, ClickHouse updates the materialised view in the background. By the time a dashboard query arrives, the 1-hour candles are pre-computed. The query scans 2,160 rows (90 days × 24 hours) instead of 200 million raw readings.&lt;/p&gt;

&lt;p&gt;TimescaleDB has something similar — continuous aggregates. The difference: TimescaleDB continuous aggregates break when the query uses dynamic intervals (users choosing arbitrary time windows). ClickHouse materialised views handle this correctly.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Tradeoffs I Accepted
&lt;/h2&gt;

&lt;p&gt;ClickHouse is not PostgreSQL. There are real things it can't do:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;No row-level updates.&lt;/strong&gt; &lt;code&gt;UPDATE sensor_readings SET value = X WHERE id = Y&lt;/code&gt; is an async mutation in ClickHouse — expensive and non-atomic. For IoT data that's append-only, this doesn't matter. For data that needs corrections, you need a strategy (we used a separate corrections log in Postgres).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Joins are slow.&lt;/strong&gt; ClickHouse joins are hash-based and don't benefit from indexes the way PostgreSQL does. We kept all relational data — users, device registry, permissions — in PostgreSQL and only put time-series data in ClickHouse.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Local development is heavy.&lt;/strong&gt; ClickHouse in Docker takes 2–3 minutes to start and consumes significant RAM. We eventually moved to a shared dev instance rather than having every engineer run it locally.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;No transactions.&lt;/strong&gt; If you need &lt;code&gt;BEGIN / COMMIT / ROLLBACK&lt;/code&gt;, ClickHouse can't help you.&lt;/p&gt;

&lt;p&gt;The decision was straightforward for our workload: append-only time-series, aggregation-heavy queries, no relational joins on the hot path. If any of those conditions were different, the answer might have been different too.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Result
&lt;/h2&gt;

&lt;p&gt;Dashboard load time went from 18–22 seconds to under 1 second. Not through clever caching alone (though we added multi-tier Redis caching on top), but because the underlying queries went from multi-second to sub-100ms.&lt;/p&gt;

&lt;p&gt;The client's engineers went from avoiding the fleet overview to opening it as their first action every morning.&lt;/p&gt;




&lt;h2&gt;
  
  
  When ClickHouse is the Wrong Answer
&lt;/h2&gt;

&lt;p&gt;Don't use ClickHouse if:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Your queries are primarily point lookups by primary key&lt;/li&gt;
&lt;li&gt;You need multi-table relational joins on the hot path&lt;/li&gt;
&lt;li&gt;Your dataset is under ~100M rows (Postgres handles it fine)&lt;/li&gt;
&lt;li&gt;You need ACID transactions&lt;/li&gt;
&lt;li&gt;Your team has no bandwidth to learn a new query engine&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;ClickHouse solves one problem extremely well: aggregating over large amounts of time-ordered data. If that's your problem, it's the right tool. If it isn't, PostgreSQL is probably fine and you're adding operational complexity for no gain.&lt;/p&gt;




&lt;p&gt;If you want to see the full schema design, materialised view definitions, and benchmark methodology: I wrote it up in detail in my &lt;a href="https://github.com/Umar-Ali1/iot-architecture-playbook" rel="noopener noreferrer"&gt;IoT Architecture Playbook&lt;/a&gt; — specifically the &lt;a href="https://github.com/Umar-Ali1/iot-architecture-playbook/blob/main/docs/03-storage-strategy.md" rel="noopener noreferrer"&gt;storage strategy doc&lt;/a&gt; and the &lt;a href="https://github.com/Umar-Ali1/iot-architecture-playbook/blob/main/adr/001-clickhouse-over-timescaledb.md" rel="noopener noreferrer"&gt;ClickHouse vs TimescaleDB ADR&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>clickhouse</category>
      <category>postgressql</category>
      <category>backenddevelopment</category>
      <category>database</category>
    </item>
  </channel>
</rss>
