DEV Community: Tiger Data (Creators of TimescaleDB)

The Real Cost of a Single Insert in Postgres

Team Tiger Data — Fri, 11 Sep 2026 20:23:58 +0000

A 40-byte sensor reading does not cost 40 bytes to store. On a table with three indexes, Postgres writes four separate WAL records for that one row, and the log grows by roughly 345 bytes before the commit returns.

That gap is called write amplification: the ratio of physical bytes the database writes to logical bytes your application sent. Every index you add pushes it higher.

If you have ever added an index to fix a slow dashboard and watched p95 insert latency climb two weeks later, you have already paid this tax without measuring it. This post counts the bytes, gives you a probe that measures the real ratio on your own tables, and covers four ways to reduce it. Every number below was measured on PostgreSQL 16 with default settings.

What You Will Learn

Why one logical insert triggers far more physical I/O than the row's raw size.
The three components of write amplification: heap tuple, index entries, and WAL.
How to measure your own write amplification with pg_current_wal_lsn() and pg_stat_wal.
Four ways to reduce it, ordered by risk.

Why It Matters

Write amplification is the mechanism underneath most ingest ceilings. It is why a workload that looks trivial on paper saturates disk throughput in production, and why read optimizations quietly tax your writes.

It also compounds. Understanding Postgres Performance Limits for Analytics on Live Data traces how MVCC overhead, index maintenance, and WAL volume feed each other on high-frequency time-series workloads: more indexes produce more WAL, more WAL causes replica lag, and bloat pressure drives you toward still more indexes. Amplification is the number that ties those loops together.

Set Up the Test Table

Everything below runs against this table. Create it in a scratch database:

CREATE TABLE device_metrics (
    ts timestamptz NOT NULL,
    device_id bigint NOT NULL,
    metric text NOT NULL,
    value double precision
);

CREATE INDEX idx_device_metrics_device_ts ON device_metrics (device_id, ts DESC);
CREATE INDEX idx_device_metrics_ts ON device_metrics (ts DESC);
CREATE INDEX idx_device_metrics_metric ON device_metrics (metric);

Three indexes, which is modest for a time-series table. The payload is about 40 bytes per row. Postgres never writes just the payload.

Start With the Row Itself

Every heap tuple carries a fixed 23-byte header holding transaction visibility fields: t_xmin, t_xmax, and five more. Those fields exist so Multi-Version Concurrency Control can decide which transactions can see the row. The header is padded to 24 bytes so data starts on an 8-byte boundary, and the page adds a 4-byte line pointer so the tuple can be located within its 8 KB page.

You can see the header in the stored size. pg_column_size(t.*) reports the row as a composite datum, which includes that 24-byte header:

SELECT pg_column_size(t.*) AS stored_bytes,
       pg_column_size(t.*) - 24 AS payload_bytes
FROM device_metrics t
LIMIT 1;

stored_bytes | payload_bytes
--------------+---------------
           64 | 40

The table must already contain at least one row or this returns nothing. Before any index or log entry, the 40-byte reading occupies 64 bytes as a tuple, and 68 once the page's line pointer is counted. That is 1.7x, and it buys visibility bookkeeping for a row that will never be updated. Append-only workloads pay the full MVCC price on data they never touch again.

Every Index Is a Second Insert

Postgres has no partial index maintenance on write. Every index is updated by every insert, synchronously, inside the same transaction.

A two-column (device_id, ts) B-tree entry costs roughly 32 bytes: an 8-byte index tuple header, 16 bytes of key, a 4-byte line pointer, and alignment. Three indexes means three B-tree descents from root to leaf and three leaf page modifications per row.

The traversal matters as much as the bytes. On a monotonically increasing timestamp key, every insert lands on the same rightmost leaf page, which fills, splits, and can cascade toward the root. A high-cardinality index scatters those same inserts across the tree, so the pages you need are less likely to be in shared_buffers.

Index count is a permanent write tax, charged on every row, forever. An index serving one weekly report costs you 32 bytes and a tree descent on every insert for the life of the table.

The WAL Record You Did Not Ask For

Postgres writes durability before it writes data. Every heap insertion and every index insertion generates its own write-ahead log record, and those records hit disk before the commit returns. On the three-index table, that is four records per row, which the measurement below confirms exactly.

Then there is full_page_writes. The PostgreSQL documentation explains the mechanic plainly: "the first modification of a data page after each checkpoint results in logging the entire page content." After every checkpoint, the first touch of each heap page and each index leaf page writes a full page image into the log. WAL volume does not sit at a steady rate. It spikes after each checkpoint and settles between them.

Try This Now: Measure Your Write Amplification

Postgres exposes both the WAL position and a running record count, so you can measure exactly what a batch of inserts costs. Run this whole block in one psql session:

CHECKPOINT;

DROP TABLE IF EXISTS wal_probe;
CREATE TEMP TABLE wal_probe AS
SELECT pg_current_wal_lsn() AS start_lsn,
       wal_records, wal_fpi
FROM pg_stat_wal;

INSERT INTO device_metrics (ts, device_id, metric, value)
SELECT now() - (s || ' seconds')::interval,
       s % 500,
       'temperature',
       random() * 100
FROM generate_series(1, 100000) AS s;

SELECT pg_sleep(1);

SELECT
    w.wal_records - p.wal_records AS wal_records,
    w.wal_fpi - p.wal_fpi AS full_page_images,
    pg_size_pretty(
        pg_wal_lsn_diff(pg_current_wal_lsn(), p.start_lsn)
    ) AS wal_written,
    pg_size_pretty((100000 * r.row_bytes)::numeric) AS logical_written,
    round(
        pg_wal_lsn_diff(pg_current_wal_lsn(), p.start_lsn)
        / (100000 * r.row_bytes), 2
    ) AS write_amplification
FROM pg_stat_wal w, wal_probe p,
     LATERAL (SELECT pg_column_size(t.*) - 24 AS row_bytes
              FROM device_metrics t LIMIT 1) r;

The row width is computed for you, so there is nothing to substitute. Three notes before you run it: CHECKPOINT requires superuser or the pg_checkpoint role, pg_stat_wal requires PostgreSQL 14 or later, and both counters are cluster-wide, so run this during a quiet window or the numbers will include everyone else's writes.

On a 500,000-row table with three indexes, that 100,000-row batch produces:

-------------+------------------+-------------+-----------------+---------------------
      402099 | 1448 | 41 MB | 3906 kB | 10.74

402,099 records for 100,000 rows. That is four per row: one heap insert plus one for each of the three indexes, exactly as described above. 41 MB of log for 3.8 MB of readings.

Now run the same block again without the leading CHECKPOINT:

wal_records | full_page_images | wal_written | logical_written | write_amplification
-------------+------------------+-------------+-----------------+---------------------
      402286 | 50 | 33 MB | 3906 kB | 8.73

Same record count, 8 MB less log. The full_page_images column explains the entire difference: 1,448 full pages in the first run against 50 in the second. Your production workload lives between those two numbers, and the spread tells you how much of your I/O budget checkpoint tuning can recover.

Now drop all three indexes and run it once more:

wal_records | full_page_images | wal_written | logical_written | write_amplification
-------------+------------------+-------------+-----------------+---------------------
      100045 | 20 | 9516 kB | 3906 kB | 2.44

One record per row, and amplification falls from 8.73 to 2.44. Roughly 70% of the WAL on this table was index maintenance, not data. That is the price of three indexes, quantified on your own hardware.

Multiply by Ingest Rate

Steady state on the three-index table is roughly 345 bytes of WAL per 40-byte row. At 50,000 inserts per second that is 17 MB/s of log for 2 MB/s of readings, before autovacuum and before a single query runs. Each additional index adds roughly one more record and 80 more bytes per row, so a five-index table on the same workload lands near 25 MB/s.

Provisioning more IOPS raises the ceiling but leaves the ratio untouched, which is why hardware upgrades on this workload tend to buy about six months.

Four Ways to Bring It Down

These are ordered by risk. Start at the top.

1. Batch your writes with COPY . Single-row INSERT statements generate one WAL record per tuple plus a commit record per statement. COPY packs many tuples into a single multi-insert WAL record, cutting record count; multi-row INSERT shares one commit, cutting per-commit fsync overhead. No schema change, no durability tradeoff.

2. Drop indexes that do not earn their keep. The measurement above puts a number on this. Cross-reference your index list againstpg_stat_user_indexes: any index with idx_scan near zero is charging a record and a tree descent on every row for no read benefit. Confirm the counter has been accumulating since your last pg_stat_reset() before you act on it.

3. Turn on wal_compression . This compresses full-page images rather than the whole log, which targets exactly the 1,448-image spike measured in the first run. The tradeoff is CPU on the write path, so re-run the probe after enabling it.

4. Raise max_wal_size and checkpoint_timeout. Fewer checkpoints means fewer first-touch full-page writes. The tradeoff is real: longer crash recovery and a larger pg_wal directory. Move one setting at a time and re-run the probe.

All four reduce the volume of writes. None removes the irreducible per-row cost: the 24-byte header and the row's own WAL bytes are properties of the storage engine. When amplification stays high after all four, you are looking at architecture rather than configuration. Batched columnar ingestion amortizes those headers across up to 1,000 rows per compressed segment and logs WAL at the segment level, which is how Tiger Data's hybrid row-columnar engine drives write amplification toward 1:1 on sustained append workloads. The architecture whitepaper covers the write path in detail.

Conclusion

An insert is not a write. It is a heap tuple, a header, one B-tree descent per index, and a WAL record for each of them. Measure that ratio and you stop guessing about which optimization is worth making.

Run the probe against your largest table during your next quiet window, then drop one index and run it again. If the number stays above 10x, try it on Tiger Cloud and measure the same workload against columnar ingestion.

Is It Safe to Give an AI Agent Access to Your Production Database?

Team Tiger Data — Tue, 08 Sep 2026 13:19:27 +0000

The direct answer

No, giving an AI agent unrestricted access to a production database is not safe. Your AI agent has full production database access. Is it compromised yet, or just waiting to be? Most teams solve this with client-side restrictions: "We'll limit what the agent can request." But that's security theater. A determined (or accidentally malicious) agent bypasses it in minutes.

The real answer is harder to implement but actually works: enforce access at the database layer. Read-only access. Scoped queries. Role-based controls. But only when the database itself enforces them; not your client.

This article explores what that distinction means in practice. We use Tiger MCP and Tiger CLI as examples (full disclosure, we built these tools), but the architecture applies across any database or MCP server, whether or not you ever touch a Tiger Cloud service. Nothing below should be read as a claim that any specific setup is fully secure or satisfies a particular compliance framework. What follows describes what specific controls do and where they're enforced, which is a more useful question than "is this safe" answered as a yes or no.

Why this question is suddenly everywhere

Until recently, connecting an AI coding agent to your stack meant giving it access to source code. Worst case, it wrote a bad function or introduced a bug that showed up in code review before it shipped. Now, through MCP servers, agents connect directly to live databases: reading production tables, running exploratory queries, sometimes writing back. The blast radius changed shape. A bad edit to a file gets caught in a pull request. A bad query against a production table can run immediately, against real data, with no review step unless someone built one in.

As we see these shifts in AI coding agents, the developer conversation has shifted with it. A year ago, the dominant question was whether you could trust an agent's judgment: would it write reasonable code? Would it follow instructions correctly? Now there's a second, separate question layered on top: can you trust the MCP server itself, independent of how well-behaved the agent connecting to it happens to be? Those are two different failure modes. One is about agent behavior, and the other is about whether the tool sitting between the agent and your database enforces what it claims to enforce.

That second question is harder to answer than it should be right now. A separate review that graded MCP servers on usability found that roughly a third scored a D or an F, meaning that a large share of MCP servers fail basic security testing when they run in isolated environments. Neither finding is about any single vendor, but together these findings say something simpler and more useful: implementation quality varies a lot across the current MCP ecosystem. "Is MCP safe" isn't a yes-or-no question, it rather depends on what a specific server's controls actually enforce, and where. Tiger Data's own engineers have talked through why that's true in general, separately from any single product's answer to it. The general consensus is that widespread adoption of MCP servers is inevitable, but there are still security concerns and implementation gaps to address.

The real risk isn't the agent: it's unscoped access

Most incident writeups involving an AI agent and a database don't describe a rogue or malicious agent. They describe an agent that was handed too much access in the first place, and eventually did something with it that nobody wanted.

The pattern repeats because it's the path of least resistance. Someone connects an agent to a database for the first time, hits an authentication prompt, and reaches for whatever credentials are already sitting in an environment variable or a password manager entry. Often that's an admin account, or the same connection string the application itself uses. It works immediately. Nobody scopes it down later, because there's no obvious moment where that becomes urgent, until there is.

Prompt injection happens when an agent reads untrusted text as part of its normal work: a support ticket, a value stored in a table, the contents of a webpage it fetched. That text contains instructions the agent wasn't supposed to follow, and the agent can't reliably distinguish "instructions from the person who set me up" from "text that happens to look like instructions." Unscoped access makes this worse: once injected, the agent runs queries against the whole database with broad permissions, far beyond what the injection would otherwise reach. The damage scales with the access you gave it.

The subtlety here goes further than a suspicious sentence sitting in a support ticket. You can watch the whole session scroll by and see nothing unusual, because there's nothing for a human eye to see. It's a concrete example of why "I'll just watch what it does" or “human-in-the-loop” isn't a complete safeguard by itself. Manual review is the safeguard most teams reach for by default, and it has a real, measurable failure rate. A study of roughly 40,000 runs of a review exercise where humans were asked to approve or reject AI agent actions found that reviewers missed about one in three risky actions they were supposed to catch. Human review helps. It is not sufficient on its own, and treating it as the primary safety net for a database-connected agent means building on a foundation already measured to fail a third of the time. Structural, database-level controls that don't depend on a human noticing something in the moment matter more.

Safety requires a framework

None of what follows is specific to Tiger Data. It applies to any developer connecting any AI agent to any database, on any vendor's infrastructure. Think of the six practices below as general AI agent guardrails: not a single switch, but a set of independent controls that hold even if one of them fails.

Never hand an agent the same raw admin or superuser credentials a human or the application already uses: This is the single most common mistake, and it's the one that turns every other risk in this article from contained to unbounded. If the agent's connection is compromised, tricked, or simply makes a mistake, the ceiling on what it can do should not be "everything."
Create a dedicated, purpose-scoped role for the agent: Least privilege means a role that can do exactly what the agent's task requires and nothing else, not "whatever's easiest to set up today." Scoping access to specific schemas, tables, and operations only takes a few extra minutes the first time and removes an entire category of risk permanently.
Default to read-only for anything exploratory: If the agent's job is answering questions, debugging, or analyzing data, it doesn't need write access to do that job. Treat write access as something you escalate to deliberately, for a specific reason, rather than something that's granted by default because it might be useful someday.
Use a fork or a read replica for anything destructive or experimental: Testing a migration, trying a risky query, or letting an agent iterate on a schema change is a different job than answering questions about production data. Point that work at a disposable copy, not the system you can't afford to lose.
Keep a human in the loop for anything that writes or changes schema : AI-generated reviews catch some mistakes, but not all. The 40,000-run study above is a reasonable data point to keep in mind when deciding how much weight to put on it. As a best practice, introduce a human into the review cycle when the project involves using AI to write or change schema.
Log what the agent actually ran: Visibility after the fact won't prevent an incident, but it's the difference between knowing exactly what happened and reconstructing it from memory.

The common thread running through all six of these: enforcement matters more than intent. Any client-side setting, whether a toggle, a configuration, or a restriction, only works if the agent follows it. Database-level enforcement is different: it applies no matter how the agent connects, which client it uses, or what it's trying to do. You don't have to trust the agent; the database enforces the restriction.

How we implement safeguards with Tiger MCP and Tiger CLI

Tiger MCP and Tiger CLI are an active implementation of the framework above. Here's what each layer does, and specifically where it's enforced.

Read-only mode is a client-side switch with a server-side backstop. Turning it on, with tiger config set read_only true, or TIGER_READ_ONLY=true for a single session, disables mutating CLI commands: service create, fork, start, stop, resize, update-password, and delete all return an error. The matching Tiger MCP tools for those actions aren't just blocked, they aren't registered at all, so an agent can't see them or call them in the first place.

That’s the client-side half that answers the "what if the client doesn't cooperate" problem. Database sessions opened through Tiger MCP or Tiger CLI, whether via the db_execute_query tool, tiger db connect, or tiger db connection-string, run inside Tiger Cloud's immutable read-only mode at the server level. Writes and schema changes are rejected by the database itself, instead of relying on client-side restrictions alone. That's a meaningfully different restriction than a tool simply choosing not to offer a mutating action. The tool not offering an action means the tool has to keep behaving correctly. The database rejecting the write means it doesn't matter whether the tool behaves correctly, because the write will never land, regardless of how it was attempted.

A dedicated read-only database role is a second, independent layer. Tiger Cloud's default tsdbadmin role has full read and write permissions (it's not a superuser, but it can modify data). For agent access, you create a separate Postgres role scoped to read-only access, save its credentials with tiger db save-password <service-id> --role <role> --password=<password>, and connect the agent as that role rather than as tsdbadmin. Grant that role SELECT on the tables the agent needs. A role created with tiger db create role <service-id> --name <role> --read-only can log in but holds no table privileges, so an agent connecting as it gets permission denied on every query until you grant access. Because Postgres itself enforces what that role can do, the restriction holds regardless of what tool or client connects as that role, including tools you haven't configured yet or don't control directly. This is the layer that matters most if you're not fully confident every client touching your database will always respect read-only mode as intended.

Forks and read replicas remove production from the blast radius entirely for anything destructive or exploratory. An agent testing a schema migration, trying an index change, or running a query it isn't confident about can work against a fork or a read replica instead of the primary service. Nothing that happens within a fork touches production data, no matter what the agent does. If you need an agent to actually write or restructure something, give it a disposable copy to do that on.

The documented best practice is to default production services to read-only and escalate to write access deliberately, for the specific change you intend to make, rather than leaving write access on as a standing default because it's more convenient at the moment.

Data minimization is a smaller but real practice worth adopting alongside the layers above: ask an agent for a single computed answer rather than a raw export. The database does the filtering, and less data leaves it in the first place. If an agent only needs to know whether a threshold was crossed, there's no reason for it to pull the underlying rows to figure that out.

One more thing read-only mode catches that isn't about data risk at all: cost. Service creation, forking, and resizing are billable actions, and read-only mode disables the Tiger MCP tools for all three. An agent that can't create or resize services can't run up a bill by mistake, which turns out to be a meaningful side benefit of the same restriction that protects data.

For teams evaluating whether Tiger MCP's authentication model fits their setup, the tool also supports client credentials for programmatic, non-interactive access, a public and secret key pair generated separately from your personal login, so an automated agent doesn't need to share credentials with a human account. That's a different layer from the role-based authorization described above: client credentials answer "can this tool connect at all," while the role it connects as answers "what can it do once it's connected." Both matter, and they aren't substitutes for each other.

What this doesn't solve

The above doesn't eliminate risk. It's worth being direct about where the limits are, rather than treating this as a solved problem.

Read-only mode and a scoped role reduce the odds and the severity of a bad outcome. They don't remove the underlying issue that an agent with legitimate read access to sensitive data can still be manipulated, through prompt injection, into surfacing or summarizing data it shouldn't. Nothing about read-only access prevents an agent from reading a table it's allowed to read and then describing its contents to someone who shouldn't see them.

That's not a gap specific to Tiger MCP, or to any one vendor's implementation. That's a property of how current AI agents work. Agents follow instructions Agents follow instructions they find in the content they process, and there's no database-level setting that changes that behavior. Any vendor's controls that claim to fully solve prompt injection at the database layer should be read skeptically, because the problem lives upstream of where database permissions apply.

The ecosystem-wide quality variance mentioned earlier, roughly a third of scanned MCP servers graded D or F., applies here too. Read-only mode and a dedicated role are strong defaults. They're not a substitute for knowing what a specific MCP server's controls actually enforce, and where. Verify what a specific server's controls enforce, and check your own role permissions, before connecting it to anything sensitive.

To sum things up: these controls are a strong foundation, not a finish line. Pair them with ongoing judgment about what data the agent actually needs to see for its task, not just what it's technically capable of reaching.

Decision framework: choosing an access model for your AI agent

Different jobs call for different access models. Use this as a sort of checklist to self-select your access model based on what the agent is actually doing, not which vendor's tools you're using.

Choose read-only mode if the agent's job is exploration, analysis, debugging, or answering questions about the data. It never needs to change anything, so there's no reason to give it the ability to.

Choose a dedicated read-only database role if you want the restriction enforced no matter what tool or client the agent uses to connect, not only when it happens to go through one particular CLI or MCP server. This matters more the less certain you are about every future integration point.

Choose a fork or a read replica if the agent needs to test a migration, try a risky query, or do anything destructive. You want a disposable copy in the blast radius, not production.

Choose scoped, deliberate write access if you've made a specific decision that the agent should make an intended change, and you're escalating access for that purpose rather than leaving write access on by default.

One honest caveat applies across all four: no combination of these controls replaces knowing what data the agent actually needs to see, and how they’ll access it. Scoping access is necessary. It isn't sufficient by itself, and treating it as a complete answer is how teams end up over-trusting a setup that's only solved part of the problem.

If you're still comparing MCP servers before deciding which one to connect, that comparison, not this framework, is where the choice of vendor actually gets made.

Moving from ad hoc access to a scoped setup

Most teams are in one of three starting points right now.

An agent is already connected with the same admin credentials a human developer uses. The fix here is a role swap, not a rewrite: create a scoped role, point the agent's connection at it instead of the admin account, and turn on read-only mode for anything that doesn't need to write. SQL still works the same way underneath. Existing prompts and workflows carry over once the connection is scoped correctly; the agent just has less room to make an expensive mistake.

An agent has no access controls configured beyond having the connection string. This is the most common starting point and the easiest to fix, because there's no existing setup to unwind. Create the scoped role first, before doing anything else, then decide whether read-only is the right default (it usually is) or whether the agent genuinely needs write access for its task.

A team hasn't connected an agent to a database yet and wants to start correctly. This is the easiest position to be in. Set up the scoped role and read-only mode before the first connection, rather than after. There's no habit to break and no existing integration to migrate.

For the exact commands behind each of these moves, best practices for AI agents covers read-only mode and forks in detail, and the Tiger CLI and Tiger MCP overview covers how the two tools fit together. If you're setting up an AI coding agent's database connection for the first time, check out our walkthrough covering the step-by-step commands.

FAQ: AI agents and database access safety

Is it safe to give an AI agent access to a production database?

Unrestricted access is not safe. Scoped, read-only, or role-limited access, enforced at the database level rather than only by a client-side setting, can be done safely. See the framework and Tiger MCP sections above for the specifics.

What happens when an AI coding agent can see the database, not just the code?

The blast radius changes. An agent that can only edit files gets reviewed before anything ships, typically through a pull request. An agent with live database access can read, and potentially change, real data immediately, with no equivalent review step unless one is built in. That's why scoping access matters more than reviewing individual actions after the fact.

Can I trust an MCP server the same way I trust the agent using it?

No. They're separate trust boundaries. The agent's behavior is one risk. The MCP server's own implementation, how it enforces read-only mode, what it logs, how actively it's maintained, is a separate one. Ecosystem-wide quality varies enough right now that both deserve scrutiny before you connect anything sensitive.

How do I authorize an AI agent that calls an MCP server?

There are two independent layers: authenticating the tool itself, so only your intended agent can connect at all, and authorizing what it can do once connected, which is the role or permissions it connects as. Tiger CLI and Tiger MCP support client-credential-based authentication, a public and secret key pair, as one concrete example of the first layer, separate from the database role that governs the second.

What's the difference between an agent's read-only mode and a read-only database role?

Read-only mode is a setting on the tool (Tiger CLI or Tiger MCP) that disables mutating commands and tools on the client side and puts database sessions into a server-enforced read-only state. A read-only database role is a separate, Postgres-level restriction that applies no matter what tool or client connects as that role. They work together. Neither is a substitute for the other.

How do I stop an AI agent from making a destructive write?

Layer the controls: read-only mode for anything exploratory, a dedicated read-only role so the restriction holds regardless of connection method, and a fork or read replica for anything that needs to test writes without touching production.

What is the principle of least privilege for an AI agent, in practice?

A role scoped to exactly the schemas, tables, and operations the agent's task requires, not the same account a human or the application uses, and not broader access "just in case" it's needed later.

Does giving an AI agent read-only access fully protect my data?

No. It substantially reduces risk but doesn't eliminate it. An agent with legitimate read access can still be manipulated, through prompt injection, into surfacing data it shouldn't, even without writing anything.

Should I ever give an AI agent write access to a production database?

Yes, conditionally. Scoped, deliberate write access for a specific intended change is different from leaving write access on by default. Default production to read-only and escalate deliberately, for the change you actually intend to make.

What is prompt injection, and why does it matter for database-connected agents?

Prompt injection is untrusted text, a table value, a document, a webpage, that contains hidden instructions capable of redirecting what an agent does next. It matters more for database-connected agents because the "next action" can be a real query against real data, not just a suggestion a human reviews before acting on it.

How do forks or read replicas help when working with AI agents?

They give an agent a disposable or isolated copy of the data to work against, so destructive testing, migrations, or exploratory queries never touch production, regardless of what the agent does or gets tricked into doing.

What should I do before connecting any AI coding agent to a production database for the first time?

Create a scoped role rather than reusing existing credentials, default to read-only, point genuinely risky or exploratory work at a fork or replica, and confirm the restrictions are enforced by the database itself, not only by the client you happen to be using today.

Tiger Cloud on Azure: Why Fully-Managed TimescaleDB Beats Self-Hosting And Other Managed Options

Team Tiger Data — Thu, 03 Sep 2026 17:50:14 +0000

Running TimescaleDB on Azure means choosing between three paths: self-hosting the Apache 2 licensed edition, using the TimescaleDB on Azure managed service, or running Tiger Cloud (fully-managed TimescaleDB) on Azure. When evaluating these three options, the question isn't just managed vs. self-hosted; it's whether you're looking to operate TimescaleDB yourself or build on a platform engineered to run it for you.

If you're already running TimescaleDB as an Apache 2 extension on Azure PostgreSQL Flexible Server, you've already hit the constraint: Microsoft doesn't ship the Community Edition, which locks you out of compression, continuous aggregates, Toolkit, and skip-scan. These aren't nice-to-have features; they're the difference between a workload that scales and one that requires architectural workarounds.

Over the past six months, we shipped capabilities for Tiger Cloud on Azure that shift how teams evaluate this choice.

Self-hosting TimescaleDB on Azure incurs a hidden tax

You already know the operations tax is real. Self-hosting gives you control over infrastructure, backup strategy, and security posture. For regulated industries, that control matters. But TimescaleDB adds a layer of operational complexity that compounds as you scale.

Unlike vanilla Postgres, TimescaleDB demands expertise in hypertable design, continuous aggregate materialization, compression policies, and skip-scan optimization. These aren't set-it-and-forget-it features. They require understanding time-series query patterns, retention policies, and workload behavior to tune correctly. As your data grows across sensor streams, financial ticks, event logs, and metrics, the tax doesn't just grow linearly. It scales with the sophistication of the system itself.

On Azure, whether you choose self-hosted Apache 2 or work through a third-party managed service that doesn't specialize in time-series, you're still making these decisions yourself: When do you partition? What compression codec? Which data should tier to cold storage? How aggressive should you be with continuous aggregates? Get it wrong, and your queries slow down. Get it right, and you've spent months of engineering time tuning the database instead of building the product it supports.

This is what is referred to as the operations tax: engineering time keeping TimescaleDB running instead of building on top of it.

Managed TimescaleDB should handle time-series complexity, not delegate it.

The right managed platform doesn't take control away. It automates decisions with clear, right answers for time-series workloads and gives back the engineering time you'd otherwise spend tuning. Tiger Cloud on Azure is built on this principle. We built TimescaleDB, and we run it for a living. The platform makes the tuning calls your team would otherwise make by hand. This is the difference between managed Postgres (which any vendor can offer) and fully-managed TimescaleDB (which only Tiger Data can deliver).

How Tiger Cloud removes the time-series tax on Azure

Cost: tiered storage optimized for time-series retention

Tiered storage moves rarely accessed data to Azure Blob Storage while keeping it queryable through the same SQL interface. It solves a specific problem for time-series workloads: you accumulate months or years of history that has to stay available for compliance, analytics, and model training, but you rarely scan it at production speeds.

In self-hosted or third-party managed environments, a 5 TB table on premium SSD costs the same every month, forever. Tuning doesn't change that cost structure. With Tiger Cloud, you enable tiered storage on a service (available on Scale and Enterprise plans), then set a tiering policy that moves chunks older than the age you choose to Blob Storage on a schedule. You can also tier specific chunks by hand when you want tighter control. Hot data stays on fast SSD, and a single hypertable can span both tiers. To read tiered data, you turn on tiered reads for a query, a session, or all future sessions.

For time-series workloads, this typically reduces storage costs 2-5x (depending on compression rates). More importantly, it removes the false choice between keeping historical data for analytics and keeping your bill manageable.

Network security: Private endpoints for TimescaleDB on Azure VNets

Self-hosted forces a binary choice: public endpoints (fast but risky) or private endpoints (secure but complex). Tiger Cloud's private endpoint support, now generally available on Azure, connects your service directly to your Azure virtual network without exposing any public endpoint. No bastion hosts, no networking expertise required, and provisioning is available directly from the Tiger Console.

Other competitors like HorizonDB offer this too. The difference is what sits behind the endpoint. You're connecting to Tiger Cloud, where compression, continuous aggregates, and skip-scan are already configured and maintained for you. You’ll find that vanilla Postgres doesn’t have access to these features and other generalist managed services don’t specialize in them.

Achieving operational resilience through additional supported regions, cross-region backups, and Azure Monitor

Self-hosting means you own disaster recovery, monitoring, and geographic expansion as separate, ongoing projects. These aren’t one-and-done projects, it’s constant operational work. However, with Tiger Cloud, we’re committed to providing operational resilience to users, and our newest updates do just that.

Recovery and RPO/RTO: We now support Germany West Central (Frankfurt) and Southeast Asia (Singapore), enabling GDPR-compliant residency for EU customers and local latency for Asia-Pacific. But the real capability is cross-region backup. Enterprise customers can store an additional backup copy in a different region from where the service runs.That changes the two numbers you have to answer for after an outage: how fast you recover, and how much data you lose. Point-in-time recovery and failover happen automatically. In self-hosted environments, testing a cross-region restore is non-trivial. On Tiger Cloud, it's built in.
Monitoring and observability: Tiger Cloud integrates directly with Azure Monitor, the observability service already built into Azure. Database metrics and query performance flow into your existing dashboard instead of requiring a separate stack. That removes one of the more tedious parts of self-hosting: standing up and maintaining your own monitoring. No second stack to build, no vendor sprawl, no context switching between platforms.

What your team gets back

Self-hosting gives you control of TimescaleDB, but you also own everything underneath it: provisioning, patching, backups, failover, monitoring, and capacity planning. That's a standing infrastructure and DevOps commitment, not a one-time setup cost. Third-party managed services remove some friction but still leave you owning database management. Tiger Cloud removes the operations burden entirely. Your team stops tuning indexes and starts shipping features. Your operations team stops making capacity decisions and starts working on what the business needs. Your database handles growth without degradation. You gain reliability without the tax, and speed without sacrificing control.

The choice

Self-hosting TimescaleDB on Azure still works. It's a legitimate choice for teams that want architectural control and have the engineering bandwidth to own TimescaleDB's operational complexity. Managed third-party services like Aiven reduce friction but still require you to own database management. Tiger Cloud removes this operational burden. When your primary data is time-series and you still need transactional integrity, SQL semantics, high cardinality, and high throughput, you need a database that holds up under all four, plus a team to operate it. The question is simple: do you want to run that database yourself, or build on one that's already managed?

Next steps

Start by signing up for a 30-day free trial, where you will have $1,000 in trial credits. Then move a test workload and start building. You’ll notice pretty quickly how much time you’ll save when certain architectural decisions are automated for you, such as the ones outlined above. To learn more about Tiger Cloud on Azure, check out these resources:

Why a 40-Year-Old Database Is Still Winning in the AI Era

Ajay Kulkarni — Tue, 01 Sep 2026 17:00:25 +0000

There are still some fundamental truths that won't change. LLMs are making it dramatically easier to build software, so yes, we're going to have more applications, more agents, more workloads, and more data. But here's one thing that won't change: developers want to build on trusted foundations. They don't want to stitch together fragile, Rube Goldberg architectures. They want infrastructure they know, they trust, and they can bet their business on, for the next decade. That's why Postgres continues to win, even as AI reshapes the world.

Despite turning 40 years old this year, and having a name only a parent could love, Postgres is still the database developers love most. It's reliable. It's extensible. It's open. And over four decades it has grown into an incredible platform, spanning time-series, geospatial, vector, analytics, and much more, all backed by the largest developer ecosystem in databases.

The irony is that developers don't want innovation in their database. They want innovation everywhere else. They want their database to be dependable. Predictable. Boring.

So we asked the people who know Postgres best: with agents and LLMs reshaping the stack at breakneck speed, why is a 40-year-old technology like Postgres still dominating?

Mike Freedman, Co-Founder and CTO of Tiger Data

Postgres scales because it was built on first principles: transactional correctness, query planning that actually works, semantics you can depend on, and a robust ecosystem of open source extensions. The teams seeing 200x performance improvements aren't jumping to new systems. They're understanding their workloads deeply and letting Postgres do what it was built to do. That doesn't change with LLMs. If anything, it matters more.

Postgres actually made a bet 15 years ago for growth and experimentation with its extension model. This fits massively into the concept of open-source development and experimentation: Let a 1000 flowers bloom. Allow people to customize for use cases, and allow open-source ecosystems thrive (including with "coopetition") rather than restricting decisions through a foundation/core developers. The core database which moves a bit slower, more deliberatively, safety and correctness above all. And then the extension framework, which allows rapid experimentation and innovation.

Chris Preimesberger, Freelance Journalist for The New Stack and SDxCentral

PostgreSQL has always been a keeper. This is largely because its architecture has proven remarkably adaptable to each new wave of computing, from client-server applications, to cloud-native platforms, and now AI-driven workloads. While newer databases often optimize for a single use case, PostgreSQL is astute: it has evolved into a highly extensible platform that supports transactional processing, analytics, geospatial data, vector search, and AI applications without forcing enterprises to abandon existing investments. Its open source governance model has encouraged a vast ecosystem of contributors, tools, and cloud services, giving enterprises confidence that they won't be locked into a single vendor. As agents and LLMs accelerate software development, PostgreSQL benefits from being the database developers already know, trust, and can integrate into modern architectures with minimal friction. While so much else is changing around it, PostgreSQL's combination of stability, flexibility, and trustiness has made it the default foundation upon which much of the new AI-powered stack is being built.

Stephen O'Grady, Principal Analyst & Cofounder of RedMonk

No single project characteristic is responsible for it. But PostgreSQL advocates and vendors typically point to the advantages of its broader community, which again is enabled by its liberal license. Multi-entity open source means more commercial options, which large enterprises favor because competition limits vendors' leverage with respect to pricing and it limits their risk to vendor behavior and changes. It can also mean broader project support and faster innovation, as evidenced by the speed at which PostgreSQL has been able to adapt to emerging market demands by adding the ability to handle new workloads from JSON to vector.

Carl Olofson, Independent Analyst (former Research Vice President at IDC)

Relational database systems like PostgreSQL are not competing with agents and LLMs, but rather cooperating. LLMs by themselves are great if all the data you want to access in your AI system is unstructured (text files, email, harvested collaborative system content, etc.), but if you want useful answers to questions involving the structured data that drive applications, and by extension, the enterprise, you must include databases that deliver current and consistent deterministic response with guaranteed 100% accuracy. SQL has been the preferred access language for decades, and the most commonly encountered SQL database system is PostgreSQL, largely because it is fairly easy to manage and most importantly, it's open source. Old does not necessarily mean outdated, and new does not mean better.

Paul Gillin, Enterprise Editor at SiliconANGLE

In an industry that considers "old" a dirty word, Postgres demonstrates that age can also mean reliability, familiarity and adaptability. In fact, AI has amplified the need for databases that adapt to multiple data types and can function as an operational store, metadata store, document store, search substrate and support vector embeddings. Postgres has adapted well to changing needs thanks to its flexible architecture and large, enthusiastic developer community. While it still lags Oracle, MySQL and Microsoft SQL Server in overall popularity, it is a true open-source alternative that appears to have the momentum to be a safe and reliable choice for many years to come.

Conclusion

Different vantage points, but the same conclusion keeps surfacing: Postgres isn't winning the AI era in spite of its age, it's winning because of what that age represents. Decades of battle-tested correctness, a governance model that spreads trust across a whole ecosystem rather than one vendor, and an extension architecture that lets new capabilities (vector search, real-time analytics, AI workloads) bolt onto a stable core instead of requiring a new database altogether.

Agents and LLMs may be changing how fast software gets built, but they haven't changed what developers need it to run on.

How Companion.energy Reduced Query Latency 25x and Compressed Terabytes to Gigabytes with Tiger Cloud

Team Tiger Data — Mon, 31 Aug 2026 19:04:26 +0000

Migrating real-time sensor telemetry from Azure PostgreSQL to managed TimescaleDB, we unlocked skip-scan optimizations and compression that turned a scaling crisis into an architecture that handles petabytes.

About the Company & Team

Most energy infrastructure today operates in isolation. A grocery store has solar panels and a battery, but they're not optimized together. An office building has multiple power draw points: HVAC, EV charging stations, lighting, and more. Each one consumes energy independently. A telecom operator managing thousands of antennas pays fixed rates for peak demand even when the grid has cheap power available during off-peak hours.

Companion.energy solves this by gathering continuous data from our customers' infrastructure. We forecast demand and pricing using machine learning, then automatically optimize what can be controlled: charging or discharging batteries, enabling or disabling solar panels, shifting loads, and more. We built Companion.energy to handle data arriving every minute from hundreds of customers' distributed systems. We operate across Belgium and beyond, optimizing everything from large grocery chains to telecommunications infrastructure. We’re a small team, but very knowledgeable about energy infrastructure.

The Challenge

When we started, our entire platform ran on a single Azure PostgreSQL instance with flexible server options. It worked initially, but as data volumes grew—continuous sensor readings from more customers, longer historical records—performance degraded quickly. We tried adding more compute and memory, but the database simply couldn't keep up with the continuous inflow of data. Queries that had been fast became slow. Our customer-facing dashboards began to time out. Customers complained about page load times of two to three minutes.

We quickly discovered that our most frequently-used query (DISTINCT ON meter_uuid for pulling latest meter readings) would never finish on large customer accounts. It timed out consistently. We tried rewriting it, but the new version broke on small sites with only old data. The frontend's automatic 3x retry eventually finished the job. We put a 30-second statement timeout in place as a guard, but that was a band-aid. The proper fix required a database feature we didn't have: skip-scan optimizations on the hypertable physical order. Azure PostgreSQL's Apache 2 licensed TimescaleDB extension didn't include that.

Two months later, in October, Azure patched our production server. There was no advance notice and no way to prevent the outage. We went down 100% for 25 minutes with zero warning. Meanwhile, storage was becoming unsustainable. We were pricing out ways to move raw meter data out of the database just to keep it manageable. We'd written "avoid future database problems (size, performance)" as a line item in our roadmap.

The real problem was deeper. We'd hit a constraint that wasn't just about throwing hardware at it. We were running TimescaleDB on Azure Flexible Server, which gave us the extension but not the full product. We had no compression policies. We had no continuous aggregates. We couldn't access the features designed to solve exactly this problem.

Why Tiger Cloud: The Architecture-First Decision

We spent six months evaluating alternatives. My first instinct was MongoDB, which had just released time-series features. After thousands of engineering hours, we realized their time-series model didn't support the relational joins we needed for cost analytics. We abandoned it. Next we looked at InfluxDB and ClickHouse. Both are optimized for time-series and analytics, but both require significant learning curves. ClickHouse especially demands manual indexing and continuous tuning. That was overhead we couldn't afford as a small team. Neither offered meaningful improvements over what we already had.

We didn't need to migrate databases at all. We needed the full product version of TimescaleDB. We'd already bet on Postgres as a company. We knew it. Every engineer on our team could write SQL. We'd already been running TimescaleDB on Azure, so the question wasn't whether to use TimescaleDB—it was whether to get the full feature set.

Tiger Cloud changed the equation. It gave us the features Azure's Apache 2 edition held back: compression policies, continuous aggregates, and skip-scan. Ultimately, we made the decision to commit our entire platform to Tiger Cloud instead of trying to band-aid Azure.

The Stack: Real-Time Optimization Architecture

All our services connect through Tiger Cloud as the central source of truth. Continuous sensor data arrives from customer equipment and flows through our platform via message queues and pipelines. Multiple internal services consume this stream, processing and transforming data before writing results back to Tiger Cloud. We maintain separate Tiger Cloud instances for operational isolation—one for customer-facing features and one for market data analysis. This separation ensures that high-volume market data ingestion doesn't impact real-time customer features. All services run within a secure Azure V-Net with Tailscale for access control. We query Tiger Cloud via the CLI and MCP interface rather than direct database connections, which feels safer and makes it easier to audit what queries are running.

Companion.energy’s real-time energy optimization stack centers on Tiger Cloud as the source of truth. Sensor data flows left to right through 30-40 microservices, with separate instances isolating market data from customer-facing operations.

Results: From Skip-Scan Timeouts to Microseconds

1. 25x faster queries with skip-scan: Our best result is that meterdata latest/earliest lookups went from 4,950 milliseconds to 202 milliseconds. That's a 25x improvement. Here's what happened: on Azure, we were running queries to pull the latest reading for each monitoring point. The query was slow because it had to scan every chunk looking for the most recent value. When a customer had millions of data points, it timed out. When we tried rewriting it for performance, we broke the small customers with only old data. We needed skip-scan: a feature that uses the hypertable's physical ordering to jump directly to the chunk we need. Azure's Apache 2 edition didn't have it.

On Tiger Cloud, we leveraged TimescaleDB's native skip-scan functions, which use the hypertable's physical ordering to jump directly to the data we need. We also deleted workarounds that only existed to dodge statement timeouts. Those loops added latency and complexity for nothing. Just moving our database there, the performance improvements were immediate and dramatic.

2. 98% compression rate, turning 1.7 terabytes into 39 gigabytes: On Azure, we were continuously adding storage as data accumulated. Our largest data tables were growing unsustainably, together holding approximately 915 GB of raw data. We projected compression would get us to about 92 GB. When we turned on the Hypercore columnstore on Tiger Cloud, those tables compressed to 28 GB. We beat our projection by 69%. The full picture across our largest compressed tables tells the real story:

Table 1: 1,084 GB logical to 22 GB physical (98.0% saved, 49.8x compression)
Table 2: 427 GB to 11 GB (37.2x)
Table 3: 197 GB to 6.2 GB (32.6x)
Table 4: 2.5 GB to 81 MB (31.5x)

In total: 1.7 TB of logical data now sits in 39 GB of physical storage. That's a 43.5x ratio. Compression directly reduced our infrastructure costs and eliminated the monthly ritual of scaling up storage. It made it feasible to retain years of historical data for analytics and model training without the bill ballooning. When we enabled it on Tiger Cloud, it was like a surprise that the few terabytes of data were only like 300 gigabytes now. Which is a huge cost reduction.

3. Leveraging Continuous Aggregates to materialize 283 million rows: We routed our energy-cost dashboard reads through continuous aggregates that materialize pre-rolled hourly and daily data. Together, these aggregates materialize 283 million rows, replacing scans of the raw 807-million-row table. These aggregates couldn't exist on Azure. The compute and storage overhead would have been prohibitive. On Tiger Cloud, with compression and efficient hypertable storage, they're free. The dashboard is now snappier because we're reading pre-rolled data instead of raw measurements.

4. Scanning billions of rows in seconds: Right now in production, here's what we're seeing:

Counting 262.7 million rows across the last 30 days of sensor data takes 10.5 seconds. That's roughly 25 million rows per second against compressed chunks.
A 90-day range scan for a single sensor against our 4.64-billion-row sensordata hypertable takes 1.5 seconds.

For contrast, here's where we started on Azure: a 6% buffer cache hit ratio, disk reads at 2.8 milliseconds per page against 0.01 milliseconds when cached. Queries were reading 1,515 pages to return only 17,072 rows because vacuum was blocked and the planner kept choosing the wrong index off stale statistics.

5. Managed observability and infrastructure reliability: One of the biggest surprises during migration was the operational simplicity. Tiger Cloud provides managed observability for all our databases—logs, query metrics, and performance insights—that would have been painful to set up and maintain on Azure. We went from manually piecing together monitoring across tools to having a cohesive view of database health. One team member highlighted how valuable this is: "We now have managed observability for all our databases, which was quite painful to maintain on Azure."

Beyond that, the support experience has been exceptional. When we've had questions or feature requests, the responsiveness from the Tiger Data team has been outstanding. They understand our use case and move quickly to help or implement what we need. That level of partnership is rare and makes a real difference when you're running mission-critical infrastructure.

6. Improving developer velocity to achieve forking in 5 minutes: Before Tiger Cloud, testing new features against production-like data meant downloading terabytes to local machines. That was prohibitive in cost and time. Tiger Cloud's forking feature lets us spin up a fresh database with production data in minutes. Everyone on the team uses it now. It removed a major friction point in the development cycle. You just need to work on a feature, spin up a fork, wait five minutes, and you can just continue working. Everyone uses it here.

7. Migrating in eight weeks, maintaining a steady replication at 50 MB: We ran the migration using Tiger's LiveSync inside our own Kubernetes cluster, replicating from Azure PostgreSQL into Tiger Cloud across our table set. Evaluation took five months, while the migration itself took roughly eight weeks end-to-end. We validated our staging environment and achieved exact row-count parity on every table.

INSERT, UPDATE, and DELETE all replicated in under a second, with typical lag of 15 to 90 milliseconds. Replication held steady at around 50 MB of lag the whole way through, so we were ingesting new data as fast as it arrived on the old database.

The cutover was completed by one engineer in about 16 hours. The long pole surprised us: it wasn't data volume and it wasn't replication. What ate the night was converting one large table into a hypertable in place. The conversion timed out repeatedly through the GUI client, then through psql. What worked in the end: rename the table to a backup, create an empty hypertable in its place, then copy the data across. Once backfill finished, we flipped every connection string in one prepared change. GitOps apps then confirmed the migration was successful.

Within days of cutover, the platform was noticeably snappier across the board. Our team noticed it immediately in dashboard responsiveness and query performance. Ultimately, replication held steady at 50 MB the whole way through—we were ingesting new data as fast as it arrived on the old database.

Looking Ahead: Scale Without Rearchitecture

We're targeting significant scale in the coming month. Larger customers are joining the platform, bringing 10x and 100x the data volumes. New product lines are in development that will add analytics and machine learning capabilities. Our pipeline now includes customers whose data requirements span from gigabytes to petabytes.

The architectural decision to run everything on Tiger Cloud means we can scale without fundamental rearchitecture. We're not splitting between a time-series database and an OLAP warehouse. We're not managing separate hot and cold storage tiers. The same database handles real-time sensor ingestion, customer queries, analytics, and model training. As our dataset grows from terabytes to petabytes, the data model doesn't change. It just gets bigger.

That's the compounding payoff of the skip-scan and compression wins. They're not just faster and cheaper today, but will continue to be tomorrow. They're architecture decisions that eliminate the rearchitecture work that comes later.

Time-Series Cardinality: Why One More Indexed Column Costs More Than a Million More Rows

Team Tiger Data — Fri, 28 Aug 2026 16:29:32 +0000

Cardinality is a consequence of how many dimensions you index your readings by. Here is where the curve bends, measured, and what you can do about it without leaving Postgres.

Someone adds a firmware_ver column to the sensor table. It is one line of DDL, and it clears review in a minute. Two weeks later, the ingest job is missing its window, a dashboard query that used to return in milliseconds takes seconds, and the on-call engineer is digging through EXPLAIN output to work out when the planner stopped using the index. The ingest rate barely moved. What moved is the index shape: one more indexed dimension, so the number of distinct series the database has to track just multiplied.

You did not hit an engine ceiling. You changed a schema decision you own, which is why the fix is a schema change and not a new database. This piece puts a number on what that one-line change costs, measured against a million more rows from the same baseline, and shows where that cost actually lives: in index width and planner estimates, both of which follow from the schema.

"Tags Are Free": The Default That Works Until It Doesn't

Every mainstream time-series onboarding teaches the same model. Attach every attribute you might want to filter on directly to the reading. In InfluxDB line protocol, they are tags; in Prometheus, they are labels; and in Postgres, they are columns on a wide table. Device, sensor, unit, site, line, firmware. The engine then treats every unique combination of those values as its own series.

At small scale this is the right call. There is no metadata table to design, no join on the read path, and no surrogate key to mint and maintain. A query scoped to a tag is fast because the tag is right there in the index. You get all of that without designing anything up front.

It rests on one assumption: the set of attributes you index by is fixed and known in advance. For a product with a defined metric set, that assumption holds, and the wide table stays the correct choice for its whole life.

An industrial fleet violates the assumption by design. Every integration brings a descriptor someone wants to filter on. A new vendor adds a firmware field. Compliance wants a shift code, and the maintenance team wants a line variant. None of those is a mistake, and each one is a new indexed dimension.

What Cardinality Actually Means, and the Two Ways It Grows

The term is used loosely, usually as a synonym for "how many tags we have," so define it strictly before leaning on it. Cardinality is the number of distinct values one dimension can take. The number of distinct series the database tracks is the product of those counts across every dimension it indexes. Ten thousand assets, a hundred sensors each, ten firmware revisions, and a hundred sites is roughly a billion combinations.

That definition splits growth into two axes that get conflated constantly.

The first is linear. Add values to a dimension you already index, and the series count rises in proportion. A hundred more devices add a hundred devices' worth of series. A sensor that starts reporting 0 to 100 instead of 0 or 1 raises that dimension's cardinality from two to a hundred, and the series count with it. The limiting case is a continuously valued tag, such as a GPS coordinate, where the dimension is effectively unbounded.

The second is multiplicative. Add a new indexed dimension, and the series count is multiplied by that dimension's distinct count. One column, one review comment, and the number of things the database has to track jump by a factor. The schema change that does this looks trivial on the diff, which is the entire reason the failure surprises people.

In index-shape notation, the contrast is one line. (tag_id, value, timestamp) identifies a series by one dimension. (tag_id, device, location, unit, firmware, value, timestamp) identifies it by five.

In Postgres, that multiplication lands somewhere specific, and it is not where the tag-indexed intuition says. A composite B-tree holds one entry per row , not per distinct combination, so index size is roughly rows × entry_width, where entry width is the sum of the indexed column widths plus per-entry overhead.

Change	What it costs in Postgres
One million more rows	`1,000,000 × entry_width`
One more indexed column	`existing_row_count × new_column_width`

That second row is why the article title can be true: its cost scales with every row you already have. Whether it is true depends on the baseline, because on a small enough table, the million new rows outweigh the new column. The benchmark below fixes a baseline and measures both changes from it.

The second cost is the planner's, and it is the sharper one. Postgres assumes column independence when it combines selectivities, and in a production fleet, the dimensions are heavily correlated: a device sits at one site, on one line, running one firmware. Multiply correlated selectivities as though they were independent, and the row estimate collapses, with the error compounding on every correlated predicate you stack. That is how an index scan flips to a sequential scan with no change to the SQL. Extended statistics can patch the estimate without a migration, and "What You Are Trading" below weighs what that buys and what it leaves in place.

The same root cause presents differently depending on the engine, which is why two engineers can describe incompatible symptoms and both be looking at cardinality. A tag-indexed engine holds a series index mapping every unique series key, so memory, startup time, and OOM risk scale with series count, and the only levers are removing a tag dimension or deleting data. A wide Postgres schema fails through ordinary Postgres mechanics instead: indexes bloat on high-cardinality columns, autovacuum runs longer and competes with write I/O, planner statistics drift until an index scan silently becomes a sequential scan, and per-chunk planning overhead grows as wide rows shrink the rows per chunk.

From the inside, both look like the engine hit a wall. That is why the reflex is to re-platform instead of remodel.

The Cliff, Measured: A Cardinality Benchmark

Almost nothing published shows the actual curve, so we measured it. One baseline of a hundred million rows of correlated fleet data on Postgres 17.10 with TimescaleDB 2.29.1, and from it each arm changes exactly one thing. Arm A adds a million rows. Arm B adds one indexed column, firmware_ver, the same column the opening anecdote adds. Arm C is the control, sweeping distinct tag count at fixed total volume: if cardinality by itself degrades Postgres, this is where it shows. Every arm was rerun at three baseline scales, so what follows is the shape of the curve rather than a single point on it.

Arm	Index size delta	Estimate-vs-actual ratio	p95 latency	Insert throughput
Baseline	2.9 GB	9.17x	point 0.2ms / range 0.7ms / rollup 313.5ms	164,956 rows/s
A. +1,000,000 rows	+30.1 MB	9.68x	point 0.3ms / range 0.5ms / rollup 304.6ms	168,113 rows/s
B. +1 indexed column	+864.9 MB	10.42x	point 0.3ms / range 0.6ms / rollup 311.6ms	137,941 rows/s
C. 4k distinct tags	+1.5 MB	10.44x	point 0.5ms / range 0.5ms / rollup 46.9ms	148,073 rows/s
C. 40k distinct tags	+16.0 KB	10.31x	point 0.2ms / range 0.6ms / rollup 251.8ms	160,808 rows/s
C. 200k distinct tags	-128.0 KB	9.44x	point 0.1ms / range 0.4ms / rollup 1,008.3ms	180,675 rows/s

Deltas are against the baseline. Arm B ran a second time with a near-unique independent column instead of firmware_ver, and cost roughly double for the same one-line change, because the two extremes of "add a column" should not be expected to report the same.

One more indexed column cost 28.73 times what a million more rows did. The write path agrees: the column costs about a sixth of insert throughput; the million rows cost none. That multiple belongs to this baseline, though, exactly as the arithmetic above says it must. Run the same two changes on a small enough table, and the inequality inverts.

The orange line is a million more rows: constant near 30 MB at every baseline. The blue line is a one-indexed column: its cost scales with every row the table already holds. They cross near four million rows, below which the column is the cheaper change.

The planner numbers are the ones to bring to a design review. Filtering on two correlated dimensions, site and line, the estimate came out roughly nine times low. Extending that filter along the correlated chain to four collapsed it by three orders of magnitude, to 2,880 times low, on predicates the wide schema invited. The compounding comes from stacking correlated columns rather than from data volume: the two-predicate figure held at every scale we ran.

And the control came back flat. We went looking for the cliff along the cardinality axis, and it is not there. Multiplying the distinct tag count fifty times over left the index where it was, and the estimate ratio with it. The cliff belongs to the tag-indexed engine's in-memory series index, which scales with series count by design and could not produce that flat line. Postgres pays somewhere else, in index width and planner estimates, and both are properties of the schema.

Note the y-axis unit: megabytes. The whole fifty-fold sweep barely moved a 2.9 GB index, and the largest tag count came out slightly below baseline. One added column moved that same index by nearly a gigabyte.

Two honest boundaries. The flat line is measured to 200,000 tags, and industrial fleets reach millions of series, so nothing here licenses extrapolating it indefinitely. And the control's latency and throughput cells track the generator's fleet shape rather than cardinality, which is why the control is read on index size and estimate ratio.

The mechanism was documented in Postgres's behavior before this run. What the run adds is the number on the inequality, the location of the crossover, and a flat line where the folklore promised a cliff.

The Reframe: Cardinality Is a Schema Decision, Not an Engine Ceiling

The wide schema indexes the fact table by the dimensions that describe the series, not by the series itself. It carries every descriptor on every row, forever, and pays for it on every insert and in every planner estimate.

Normalize it. Give each distinct measurement point one surrogate identifier, store its descriptive attributes once in a metadata table, and key the narrow reading table on that identifier alone.

CREATE TABLE tag_metadata (
    tag_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    tag_name TEXT NOT NULL UNIQUE, -- 'Line3/Press2/Temp'
    device_id TEXT NOT NULL,
    site TEXT NOT NULL,
    line TEXT NOT NULL,
    firmware_ver TEXT NOT NULL,
    unit TEXT NOT NULL,
    ts_start TIMESTAMPTZ,
    ts_end TIMESTAMPTZ,
    ts_last_seen TIMESTAMPTZ
);

CREATE TABLE sensor_readings_narrow (
    recorded_at TIMESTAMPTZ NOT NULL,
    tag_id BIGINT NOT NULL,
    value DOUBLE PRECISION NOT NULL
) WITH (
    tsdb.hypertable,
    tsdb.partition_column = 'recorded_at'
);

CREATE INDEX ON sensor_readings_narrow (tag_id, recorded_at DESC);

The index collapses from (tag_id, device_id, site, line, firmware_ver, recorded_at) to (tag_id, recorded_at). Nothing was added to fix this schema; four columns were removed because tag_id was the series identity the whole time. The descriptors still exist. They live in tag_metadata, one row per tag, off the write path and out of the fact table's index.

The claim that adding a descriptor cannot multiply the series count follows arithmetically from the strict definition above. If the series count is the product of the counts across indexed dimensions, and there is exactly one indexed dimension, then adding a descriptor multiplies nothing. The series count grows only when you genuinely add tags.

There is a competing strategy, and pretending otherwise would be dishonest. InfluxDB 3 was rebuilt on a columnar, object-storage architecture and is pitched as offering unlimited cardinality. That is a real and different answer to the same problem, and it is a vendor claim rather than an independently benchmarked result. The difference that matters is where the work lands: one path asks you to adopt a new storage engine; the other asks you to run a normalization you already know how to write.

What You Are Trading

Relational metadata is not free. You pay upfront for schema-design work, a one-time migration with a maintenance window in it, and a join on the read path. Here is the other side of that ledger, measured: the same hundred million readings in both shapes, same box, same settings, with tag_metadata counted in full on the narrow side.

Index and total storage for the same hundred million readings, before and after. The narrow bars include tag_metadata in full, heap, and indexes both, which comes to 9.4 MB.

An index two-thirds smaller, and total storage cut roughly in half. Reads move the same way that the wide schema was hurting: the rollup on two correlated dimensions got meaningfully faster, and the four-predicate version got about thirty times faster. Point lookups and short-range scans are a wash, sub-millisecond in both shapes, which is what you would expect from queries that were always driven by the leading key. Writes gained somewhere between a quarter and a third, depending on whether the ingest path resolves tag ids on every insert or works from a cached map. The ceiling becomes ordinary Postgres row-count limits and index behavior rather than an in-memory series index.

One caveat on that ledger. Compression was off on both sides for comparability, so read those numbers as an uncompressed comparison. Turning it on changes both, and the wide table has more to gain in ratio terms, since a descriptor repeated on every row is the easiest thing a columnar format will ever compress. The metadata table needs no such caveat. It holds one row per tag while readings accumulate per tag over time, so the two scale together and the rounding error stays a rounding error.

The tempting claim is that normalizing cures the planner's independence assumption. It does not. The descriptor columns do not disappear. They move into tag_metadata, where Postgres misestimates them by much the same margin as before. What changes is which table pays for the error. Descriptor filters now resolve against a fifty-thousand-row metadata table, where a bad estimate costs close to nothing, while the fact table is reached by resolved tag_ids and estimated almost exactly right. On the wide schema, that same misestimate landed on the hundred-million-row fact table and reached three orders of magnitude, which is what the thirty-fold gap above is made of. The blind spot survives the migration. It stops being expensive.

If misestimation is your only symptom, CREATE STATISTICS will fix it with no migration at all and is the right first move; it just leaves index width, write amplification, and vacuum cost exactly where they were.

The query-rewrite cost is the one you will feel first, and a compatibility view is what bounds it: a view carrying the old wide shape, built by joining the narrow table back to tag_metadata. Without it, normalizing means rewriting every dashboard query on cutover day. With it, existing reads keep working, and the rewrite becomes incremental. The view is a join on the read path and we expected to charge you for it, but measured against querying the narrow table directly it lands within two percent on the rollups and below measurement noise on the sub-millisecond queries. The view is not what will force the schedule.

One constraint this piece does not solve. Industrial tags are not uniformly float. Booleans, integers, strings, and state codes all show up in the same historian, and a single value DOUBLE PRECISION column forces a decision you will hit on day one: multiple typed columns, one table per value type, or a wider union type.

Where to Start

The slowdown you are seeing as the fleet grows is a property of your index shape, and index shape is yours to change. The way across is a migration inside Postgres. You already know how to write it.

Before you plan anything, check one property of your own data: whether each tag has always carried the same descriptors, or whether some of them moved lines and changed firmware along the way. A single scan over your wide table answers it, about a minute against a hundred million rows.

Stable descriptors mean the metadata table falls straight out of the data you already have. Tags that change over their lifetime are still migratable; they just take longer. The reshape itself is a batched, resumable migration that leaves every historical row where it is; we ran it end to end, failure drills included, and it is the subject of a follow-up piece, along with the scan that tells you which case you are in. If you would rather have the hypertable, compression, and policies managed while you do it, that is what Tiger Cloud is for.

14,000x Faster Planning for LIMIT Queries on Hypertables

Team Tiger Data — Thu, 27 Aug 2026 14:18:10 +0000

A very common query against a time-series table is fetching the latest row for a device:

SELECT * FROM metrics WHERE device = 'D001' ORDER BY time DESC LIMIT 1;

This should be instant. The row lives in the newest chunk, that chunk has an index on device, time, and reading it is one index scan. But on a hypertable with a few thousand chunks the query can take tens or hundreds of milliseconds before it returns anything, and nearly all of that time is spent planning rather than executing. It also gets worse as the table grows: planning time scales linearly with the number of chunks, even though the query still only reads a single row.

Across the Tiger Cloud fleet, we spend over 1,000 CPU hours every day planning ORDER BY time LIMIT-style queries.

Where the planning time goes

A hypertable is a PostgreSQL table, and so is each of its chunks. The chunks are child tables of the hypertable, connected through PostgreSQL's table inheritance. As part of planning a query TimescaleDB expands the hypertable into its chunks and builds an append node over their scans.

TimescaleDB prunes chunks before it expands them. It uses the dimension metadata in its catalog to determine which chunks the query's WHERE clause can touch, and only those chunks get expanded. A query like WHERE time > now() - interval '1 hour' only expands the chunks matching the constraint.

The cost appears when nothing prunes, which is whenever the query has no constraint on a dimension column. For every chunk that survives pruning, TimescaleDB opens and locks it, reads its statistics, and plans how to scan it. This is paid once per surviving chunk, so at 10,000 surviving chunks it dominates planning.

Our example query, WHERE device = 'D001' ORDER BY time DESC LIMIT 1, restricts nothing on the time dimension, so nothing prunes and every chunk survives. The LIMIT only bounds how many rows come back.

Why every chunk ends up in the plan

The LIMIT 1 does not let the planner keep only the newest chunk and drop the rest. The planner cannot prove the others are unnecessary.

A chunk can only be left out of the plan if the query's constraints show it can't contribute a row. A LIMIT only bounds how many rows the query returns. Nothing about it guarantees the newest chunk holds the answer. That chunk might be empty, or every row in it might be filtered out by a constraint on a non-dimension column, in which case the matching row is in an older chunk. A correct plan has to be able to fall through to those chunks.

The best available plan is an ordered append over the chunks, producing rows in time order. TimescaleDB's ChunkAppend does this: it visits chunks newest-first and stops as soon as the LIMIT is satisfied. For ORDER BY time DESC LIMIT 1 it scans only one chunk, and the others show up as never executed in EXPLAIN ANALYZE.

The plan contains every chunk. Only plan-time pruning removes chunks from the plan, and it needs a dimension constraint the planner can fold to a constant while planning. The LIMIT's early stop and ChunkAppend's run-time exclusion run at execution, so they skip scanning a chunk but do not remove it from the plan. Run-time exclusion covers constraints whose value is known only at execution, such as a query parameter or a stable expression.

When the query starts, the executor sets up each chunk's part of the plan, including those that are never executed. The chunk count is paid twice, once during planning and once during executor startup; the LIMIT avoids only the scanning.

Leaving the hypertable unexpanded

DeferredChunkAppend is a new custom scan node that skips expansion during planning. It leaves the hypertable as a single relation instead of turning it into an append over chunks. The plan for the query above is just:

Limit
  -> Custom Scan (DeferredChunkAppend) on metrics
       Order: "time" DESC

No append, no per-chunk paths, nothing in the plan that grows with the number of chunks. Planning is constant.

The chunk work moves to execution instead, where the Limit can stop it early.

What happens at execution time

The node fetches chunks one at a time, in the order the query requires. With ORDER BY on the time dimension it takes them in time order, ascending or descending, so a small LIMIT is usually satisfied by the first chunk. Without ORDER BY it takes chunks newest-first. Because it fetches lazily and stops once the LIMIT is satisfied, it touches only the first few chunks. EXPLAIN (ANALYZE) reports a Chunks Visited counter with the number of chunks it opened.

For each chunk the node builds a plain SQL query against the chunk's table and runs it. With a pushed-down limit it looks like this:

SELECT time, device, value FROM _timescaledb_internal._hyper_1_42_chunk
WHERE <pushed-down filter>
ORDER BY time
LIMIT 5;

Each chunk is scanned by a normal planned query, so the node needs no logic of its own for the different chunk states, such as uncompressed, compressed, partially compressed, or ordered; the per-chunk query handles each.

When it applies

DeferredChunkAppend works by stopping once the LIMIT is satisfied, so it is used only when the LIMIT sits directly on top of the hypertable scan. An aggregate, GROUP BY, DISTINCT, a window function, HAVING, a set operation, or a join would sit between the LIMIT and the chunk scans, so the LIMIT would no longer bound how many rows are read from the chunks; those queries keep the append plan. The query must also read a single hypertable and must not have row-level-security policies, since scanning chunks directly would bypass them.

ORDER BY is only supported on the primary dimension in its natural order, with trailing keys allowed, because that's the order the chunk walk can produce without sorting. Ordered mode also requires a hypertable with a single (time) dimension; on a space-partitioned hypertable an ORDER BY query keeps the append plan, while a plain LIMIT with no ordering still uses the node.

The query can optionally have a WHERE clause, as long as it doesn't constrain a dimension column. Constraints on the dimension columns are what ordinary chunk exclusion is for, so those queries keep the append plan. Filters on regular columns are supported and pushed into each per-chunk query.

Numbers

The tables below show how planning time grows with the chunk count. The same data set is built two ways, as a native PostgreSQL declarative-partitioned table and as a TimescaleDB hypertable, at 1, 10, 100, 1,000, and 10,000 chunks, and SELECT * FROM t ORDER BY time LIMIT 1 is run on PostgreSQL 17.7 and 18.3, using release builds of TimescaleDB. Each number is the median planning time over 15 warm runs. The hypertable is measured with DeferredChunkAppend off and on, so the two hypertable rows differ only in the feature.

PostgreSQL 18 planning time (ms)

Configuration / Chunks

100

1,000

10,000

Declarative partitioning

0.042

0.170

1.310

14.3

183

|
|

Hypertable (ChunkAppend)

0.054

0.190

1.300

16.3

202

|
|

Hypertable (DeferredChunkAppend)

0.018

0.017

0.014

0.013

PostgreSQL 17 planning time (ms)

Configuration / Chunks

100

1,000

10,000

Declarative partitioning

0.038

0.140

1.380

24.2

3583

|
|

Hypertable (ChunkAppend)

0.048

0.210

1.270

25.5

3660

|
|

Hypertable (DeferredChunkAppend)

0.016

0.018

0.013

Both plans that expand chunks, native partitioning and the hypertable with the feature off, grow with the chunk count. At 10,000 chunks on PG18 that is 183 ms for native partitioning and 202 ms for the hypertable; on PG17 native partitioning and the hypertable reach 3.6 seconds. DeferredChunkAppend stays around 0.014 ms across the whole range, on both versions.

Execution scales too, because the executor sets up each chunk before the append runs. The append plan's execution grows from 0.009 ms at one chunk to 27 ms at 10,000 on PG18, while DeferredChunkAppend stays around 0.12 ms. Execution is the smaller cost: at 10,000 chunks the append plan spends about 27 ms executing against 200 ms planning on PG18.

This continues work we described in Optimizing queries on TimescaleDB hypertables with thousands of partitions, which cut planning time 15x by speeding up chunk expansion itself. DeferredChunkAppend takes the next step for this query shape: instead of expanding faster, it skips expansion entirely.

Trying it

DeferredChunkAppend ships in TimescaleDB 2.30. It's on by default, behind a GUC:

SET timescaledb.enable_deferred_chunk_append TO on; – default

Run a qualifying LIMIT query and look at the plan:

EXPLAIN (COSTS OFF) SELECT * FROM metrics ORDER BY time DESC LIMIT 1;
EXPLAIN (COSTS OFF) SELECT * FROM metrics WHERE device='D0001' ORDER BY time DESC LIMIT 1;

The plan should show a Custom Scan (DeferredChunkAppend) node in place of an append over the chunks.

Summary

Expanding a hypertable into all its chunks makes sense when a query reads most of the data. A LIMIT query that returns only a few rows doesn't; it was paying that same per-chunk cost for every chunk without needing it. A LIMIT bounds what comes back, not what gets planned; those are separate costs, and mixing them up is what made this query slow in the first place. DeferredChunkAppend is what happens when you stop conflating them: it defers the chunk work to execution, where the LIMIT ends it early, so fetching the latest row takes constant planning time regardless of the number of chunks.

DeferredChunkAppend is already running across the Tiger Cloud fleet, on by default. If you want to see it on your own hypertables, create a Tiger Cloud account and run EXPLAIN on your own ORDER BY time LIMIT queries.

Why We Aren't Ready for C-3PO

Team Tiger Data — Tue, 25 Aug 2026 12:59:27 +0000

When people picture a robot, they picture C-3PO (or if you're a Trekkie, it's Lieutenant Commander Data and his brother Lore): a humanoid that walks into any room, understands what you say, picks up anything, and improvises when things go sideways. Decades of science fiction trained us to treat that as the target, the thing we are all building toward and almost have.

For years the working assumption was that we were waiting on smarter robot brains. The evidence points the other way. The brain is now the fastest-moving part of the stack. The hard problems have moved into the body: dexterity, reliability, and a data problem with no internet-scale shortcut.

The same reordering shows up well beyond humanoids, in drones, autonomous vehicles, and factory automation, anywhere AI leaves the screen and has to act. And the machines that look nothing like C-3PO are already doing real, valuable work at scale, precisely because they refuse to be general.

The companies racing hardest toward the generalist dream are the best evidence for this. Follow their revenue, not their demos.

The brain is moving faster than the body

The reasoning layer has moved further than most people outside the field realize. Modern vision-language-action models (VLAs), which map what a robot sees and is told into what it does, borrow their backbones from the same vision-language models that power frontier multimodal LLMs. Consider how Google DeepMind shipped Gemini Robotics 1.5 in late 2025. They split it into two models. Gemini Robotics-ER 1.5 is the embodied-reasoning model: spatial grounding, multi-step planning, success and progress estimation, native tool calls. It reports state-of-the-art results across 15 academic embodied-reasoning benchmarks, and DeepMind shipped it to developers through the Gemini API. The action model, Gemini Robotics 1.5, the part that issues motor commands, was gated to select partners behind a waitlist.

That split is the tell. A lab ships the part it trusts and holds back the part it doesn't. Here the thinking ships while the doing stays behind a waitlist. The reasoning isn't finished, but it's no longer the part holding robots back.

The frontier of cognitive research is now aimed at the physical-world gap. Yann LeCun left Meta in late 2025 and raised about $1.03B for AMI Labs (announced March 2026) on the thesis that today's language models are the wrong substrate for physical intelligence, and that systems should instead learn how the world behaves from video and interaction. His group's V-JEPA 2 is one instance: trained on over a million hours of internet video plus under 62 hours of robot interaction data, it ran zero-shot on robot arms in two different labs for pick-and-place, with no data collected from either robot. The specific architecture is beside the point. Even the researchers most bullish on cognition are spending their money on the embodiment-and-data gap, not the reasoning one.

You can see the speed gap in the release cadence too. In about a year, embodied reasoning moved fast:

Gemini Robotics 1.5 to ER 1.6, each posting new state-of-the-art on embodied-reasoning benchmarks
π-series VLAs, GR00T, and V-JEPA 2
a new billion-dollar lab founded specifically to push it further

Over the same window, the things that actually gate deployment barely moved:

production robots still grip with two-fingered parallel jaws
they still run behind human supervision
they still earn reliability the slow way, through engineering rather than a model update

When one layer ships a new SOTA every few months and the layer beneath it advances on the timescale of hardware revisions, "the brain is moving faster" stops being a metaphor.

What is actually slowing things down

If intelligence is no longer the binding constraint, something else is. Three things, all of them unglamorous: dexterity, reliability, and data.

Dexterity

Human hands have more than 20 degrees of freedom, and coordinating contact-rich manipulation across them is still unsolved at production reliability. The clearest evidence is what actually ships: despite years of dexterous-hand research, most real-world robotic applications still run on simple parallel-jaw grippers, and in-hand manipulation (repositioning an object within a grasp) remains a seminal open challenge. The benchmarks show how sharply performance falls off: the same learned policies that pick objects reliably, around 80% and up, drop toward zero on contact-rich tasks like standing a cup upright or stacking, where the object has to be shifted within the hand mid-task, and the best policies on real multi-finger hands land near 63% average success across a handful of everyday tasks, a genuine research result and a non-starter for unattended deployment. The bottleneck is not only the control policy but the hardware itself: tendon-driven multi-finger hands break, drift, and are hard to calibrate.

Reliability

Even where the hands are good enough, the bar that gates deployment is not capability but reliability. A manipulation policy that succeeds 80% of the time is a great demo video. A warehouse or an operating room needs to be measured in nines, across millions of cycles, including the long tail of weird edge cases. The gap between "works in the demo" and "works unattended on the night shift" is most of the actual engineering.

Autonomous driving is the clearest illustration outside the lab. A car that follows roads competently has existed for years. What took a decade was the reliability, and proving it. Waymo had to accumulate more than 170 million rider-only miles by the end of 2025 before the safety case was statistically airtight: 92% fewer crashes causing serious or fatal injury than human drivers in the same conditions, measured with a comparison methodology published in peer review at 56.7 million miles. The car could drive years ago. Making it safe across the edge cases took a decade.

Data

Dexterity and reliability are limited by physics and engineering. Data is a different kind of limit, and unlike the first two it has no internet-scale shortcut. Language models had the internet. There is no equivalent corpus of robot actions. Skild AI, one of the most aggressive generalist players, puts it plainly: there is no "internet of robotics."

The workaround is the "data pyramid": a small amount of real robot teleoperation at the top (a human driving the robot by hand to record demonstrations), a large layer of simulated and synthetically generated data in the middle, and web-scale human video at the base. Look at the numbers each lab publishes:

NVIDIA GR00T N1 generated 780,000 synthetic trajectories, the equivalent of roughly 6,500 hours or nine continuous months of human demonstration, in about 11 hours of compute, because collecting that much real robot data was infeasible.

Physical Intelligence's π0.5 draws 97.6% of its training data from sources other than the target mobile manipulator it is trying to control. The robot you actually care about contributes under 3% of what teaches it.

But that is only the data problem you have before a machine ships. A second one starts the moment it does, and it draws far less attention because it is not about training at all. Everyone talks about training data. Far fewer people talk about operational data, the exhaust a machine throws off while it runs: joint states, video, lidar, odometry, and the record of what it tried and how it failed. Training data is scarce and has to be manufactured; operational data is the opposite problem, a firehose that never stops. The internet trains the brain; reality trains the body. And this is not robotics-only: any machine that senses and acts (a drone, an autonomous vehicle, a piece of grid hardware) generates the same flood of data, and the learning loop everyone is counting on runs entirely on it. You collect the exhaust, replay the failures, annotate them, and feed the lessons back into the next model.

That loop is where deployment turns into improvement, and its infrastructure is far less mature than the models it feeds. The live view that flags a robot misbehaving now, the replay you scrub after it fails, and the training set you assemble months later all want the same telemetry, and most stacks keep three copies fighting to stay in sync, which is how the annotation explaining a failure so often never reaches the next model's training set. The exchange formats the field reaches for (LeRobot for training, MCAP for logging) are inputs to that layer, not the layer itself. It is unglamorous, and it decides whether a deployed machine gets better over time or keeps repeating the same failures. Most teams have not hit it yet because most are not running large fleets; the ones who treat telemetry as core infrastructure early are the ones not rebuilding it under load later.

Generalization in the wild

The state of the art in generalist embodiment is π0.5. It can clean a kitchen or tidy a bedroom in homes it never saw in training, running multi-stage tasks of 10 to 15 minutes. That is a real milestone. But the authors are blunt about its limits: it does not always succeed on the first try, and it errs both in high-level semantic deductions and in low-level motor commands. The wider field shows the same gap at scale: Stanford research reported in Fortune found robots scoring nearly 90% success in controlled simulation succeeded at just 12% of real household tasks. Impressive, real, and several reliability orders of magnitude short of what "C-3PO" implies.

Meanwhile, the narrow robots are already working

Constrain the task and every bottleneck above gets smaller. A defined job means a defined environment, a bounded object set, a tractable data problem, and a reliability bar you can actually clear.

Delivery. Zipline has flown more than 125 million autonomous commercial miles and completed over 2 million deliveries, with over 20 million items delivered and zero serious injuries, at a $7.6B valuation as of January 2026. One job, executed at a scale and safety record no humanoid is remotely near: launch, navigate, drop, return. No humanoid has left the demo stage. Zipline left it years ago.

Warehouse pick-and-place. The lesson of 2025 was that reliability beats novelty. Fully autonomous picking across the entire SKU range (a bag of chips, then a bottle of shampoo, then a pair of shoes) is still unsolved at the level that fully replaces a human picker. But constrained, high-volume, low-variability operations (grocery distribution, pallet handling, sortation) scaled faster than almost anyone predicted. Symbotic runs systems for Walmart, Target, and Albertsons against a multibillion-dollar backlog; Covariant's models handle items they have not seen across apparel, pharma, and 3PL. The winners narrowed the problem until it was reliable. At the far end of that strategy sit "dark factories" like Xiaomi's Changping plant, which runs lights-out at roughly 81% line automation by redesigning the environment around the machines: fixed stations, known part positions, engineered tolerances.

Surgery. Surgery isolates the variable better than any other example. Intuitive's da Vinci performed roughly 2.6 million procedures in 2024 and is the gold standard for minimally invasive surgery. It is also Level 0 autonomy: every motion is driven by a human surgeon. Set that against a Johns Hopkins and Stanford result: a VLM trained on about 20 hours of surgical video had a da Vinci autonomously suture, lift tissue, and manipulate a needle on animal tissue, recovering a dropped needle zero-shot. The intelligence for complex surgical subtasks can be learned from roughly twenty hours of demonstration. Lack of intelligence is therefore a weak explanation for why autonomous surgery is not deployed. The real one is that failure is unacceptable, which is why the field pursues "augmented dexterity," automating narrow subtasks under a surgeon ready to take over at any instant. Reliability gates deployment, not intelligence.

Inspection. Boston Dynamics has several thousand Spot units patrolling oil platforms, substations, nuclear sites, and factories, where a single unit replaces hundreds of static sensors walking predetermined routes. Boston Dynamics puts the annual value to a typical manufacturing customer at roughly $252,000, and the breakdown is instructive: about $182,300 of it is averted equipment breakdowns and $32,500 is energy saved by catching compressed-air leaks, while the labor freed up by automating the walkthroughs themselves accounts for only $30,200. These are the vendor's own figures, drawn from what it calls a representative average of real manufacturing customers. Even in the clearest narrow-robot success story on the board, most of the money comes from catching failures early, not from replacing the person who used to walk the route. In April 2026 they shipped the next iteration of that same reasoning line, Gemini Robotics-ER 1.6, into Spot's inspection product, whose instrument-reading capability (reading analog gauges, measuring sight glass fullness) was built through the Google partnership and did not exist in the prior generation. The discipline shows up on the research side too. In a hackathon demo running the previous model in a residential home, the engineers were explicit that the reasoning layer "can't invent new capabilities or control Spot beyond what is available through the API," which "keeps Spot's behavior predictable." The same boundary holds in the field: when conditions degrade and steam obscures a gauge, Spot stops, documents the obstruction, and pings a human. The system knows what it does not know. A frontier reasoning system, a constrained body, strict guardrails, and uncertainty escalated to a human instead of guessed at.

Even the billion-dollar robot brains sell narrow

Tesla is building Optimus, Figure is putting a humanoid on commercial timelines, Physical Intelligence is training cross-embodiment foundation models, and a wave of capital is betting that the split between narrow and general is just a phase that scale will erase. So test the thesis against the strongest member of that camp. If anyone should invalidate it, it is the company pursuing the most aggressive general-purpose vision with the most money behind it. Right now that is Skild.

Skild AI raised about $1.4B at a $14B+ valuation in January 2026 to build the Skild Brain, one foundation model that claims to control quadrupeds, humanoids, arms, and mobile manipulators, with backers from SoftBank to NVIDIA to Samsung. That is the omni-bodied dream, fully funded, by serious people, as a direct rejection of the narrow-versus-general split. Then you follow the revenue. Skild went from zero to roughly $30M in 2025, and the deployments generating it are narrow: security, inspection, delivery, warehouses, data centers, construction. The household and humanoid generalist is explicitly the eventual goal, with enterprise tasks as the first application. Even the cost story points the same direction. NVIDIA's Skild case study sells the Brain running on $4,000 to $15,000 hardware versus $250,000-plus custom systems, which is a pitch about making narrow deployments cheaper, not about a humanoid that does everything in your kitchen.

So the most ambitious "any robot, any task" company on the board, the one that raised $1.4B precisely to build a general robot brain, still earns its near-term revenue from purpose-built machines doing constrained jobs. If any company should have invalidated this thesis, it was this one, and instead its income statement confirms it. For now, the vision is omni-bodied; the income is single-purpose.

The Spot-plus-Gemini pattern, a frontier mind riding a narrow dependable body, is what the next wave looks like, with those bodies widening as dexterity, reliability, and the data loop catch up. The likely path was never "humanoid arrives fully formed." It is increasingly general intelligence on increasingly capable bodies, converging slowly, intelligence leading the way.

C-3PO is still coming. Just not first, and not soon. The intelligence got there first. The body, the reliability, and the data come next. And this was never really about a gold humanoid from a movie. It is about what AI does once it leaves the screen, and for now the answer is a capable mind riding a narrow, dependable body. The robots we are ready for are already flying over our neighborhoods, running our distribution centers, and reading gauges in places no human should have to stand. That is not a lesser future. It is a remarkable one, and it's here.

Get started

This post explains why purpose-built robots are winning first. The next question is architectural: what data layer lets deployed machines become better machines? Schema design, fleet telemetry, replay, retention, operational analytics. That is where the loop either closes or doesn't.

Tiger Data (creators of TimescaleDB) works with teams building production systems on operational time-series data. For managed deployments, explore Tiger Cloud. For regulated, air-gapped, or on-prem environments, contact us about TimescaleDB Enterprise.

How to Tell Whether Your Slow Query Is a Planning or an Execution Problem

Team Tiger Data — Fri, 21 Aug 2026 15:47:50 +0000

Every EXPLAIN ANALYZE prints two numbers at the bottom, and most engineers read only one of them. Planning Time is how long Postgres spent deciding how to answer your query, while Execution Time is how long it spent actually answering it. When latency climbs, those two numbers point to opposite fixes: plan caching and fewer partitions on one side, memory and storage layout on the other.

Getting this backwards is expensive. On a table with 500 daily partitions, planning a simple time-range aggregate takes nine times longer than running it, and the reflex to add another index makes the problem worse. Every new index adds a path the planner has to consider, plus write amplification on a table that was already write-bound. You’ll end up spending weeks optimizing the phase that wasn't slow.

The good news: Postgres tells you which phase is slow, for free, in one command.

Every plan below is real output from PostgreSQL 16.13 against this table, partitioned into 500 daily chunks named device_metrics_YYYYMMDD and holding 2.1 billion rows, with work_mem at 4 MB and shared_buffers at 256 MB:

CREATE TABLE device_metrics (
    ts timestamptz NOT NULL,
    device_id bigint NOT NULL,
    metric text NOT NULL,
    value double precision,
    rack int, -- one rack per 100 devices
    region int -- one region per 500 racks
) PARTITION BY RANGE (ts);

rack and region demonstrate correlated columns in Step 3. Substitute your own table and partition names.

What you will learn

How planning time and execution time differ, and why each has a different fix.
How to read EXPLAIN (ANALYZE,BUFFERS) to attribute wall-clock time to a phase.
The signature of a planning problem: high plan time, hundreds of pruned partitions, a nearly idle scan.
The signature of an execution problem: spills to disk, heavy buffer reads, filters that discard most of what they touch.
The exact commands to run once you know which one you have.

Two clocks, two fixes

The planner runs a full optimization pass on every query it hasn't cached a plan for. It enumerates access paths, pulls cardinality estimates out of pg_statistic, evaluates join orders, and prices each candidate. For a warehouse query with eight joins, that work pays for itself many times over.

Partitioned tables change the math. Before the planner can exclude a partition, it locks the relation, loads its relcache entry, and builds planner state for it. That setup runs for every partition, whether or not pruning later discards it, so the cost scales with how many partitions exist rather than how many survive. The PostgreSQL documentation is direct about it: "Planning times become longer and memory consumption becomes higher when more partitions remain after the planner performs partition pruning."

Execution is a different animal. That time goes to reading heap pages, hashing, sorting, and aggregating. It grows with how much data the plan touches, not with how many plans were considered.

One clock measures the decision. The other measures the work.

Step 1: Read the split

Run the query below with both flags. ANALYZE executes it and reports real timings. BUFFERS reports how many 8 KB blocks each phase touched, including the planner.

EXPLAIN (ANALYZE, BUFFERS)
SELECT device_id, avg(value)
FROM device_metrics
WHERE ts > now() - interval '5 minutes'
GROUP BY device_id;

You may see output like the following:

HashAggregate (cost=5510.70..5632.72 rows=9762 width=16)
                (actual time=6.311..6.696 rows=2202 loops=1)
   Group Key: device_metrics.device_id
   Batches: 1 Memory Usage: 913kB
   Buffers: shared hit=932
   -> Append (cost=0.28..5461.89 rows=9762 width=16)
               (actual time=0.775..4.570 rows=9498 loops=1)
         Subplans Removed: 499
         -> Bitmap Heap Scan on device_metrics_20260729
                       (cost=212.21..1272.31 rows=9263 width=16)
                       (actual time=0.774..3.796 rows=9498 loops=1)
               Recheck Cond: (ts > (now() - '00:05:00'::interval))
               Heap Blocks: exact=898
               Buffers: shared hit=932
 Planning:
   Buffers: shared hit=20063
 Planning Time: 62.913 ms
 Execution Time: 7.202 ms

Read the last two lines first. Execution took 7.2 ms. Planning took 62.9 ms. Postgres spent nine times longer choosing a plan than running one, and the plan it chose was excellent: pruning left one partition out of 500, the bitmap scan hit cache on every block, and the row estimate was within three percent.

Now read the Buffers lines against each other. This is the part people miss. Execution touched 932 blocks. Planning touched 20,063. The planner read 21 times more pages than the query did, and it read them to open and price 500 partitions before discarding 499 of them.

Nothing here is fixable with an index. This is a planning problem.

One note on how pruning reports itself. Subplans Removed appears when Postgres prunes at run time, which is what happens with a cached generic plan or a stable expression like now(). When pruning happens at plan time, excluded partitions never appear in the output at all, so you count the ones that survived.

Step 2: Learn the other signature

Now the contrast: a different query against a single partition, where every number moves the other way:

EXPLAIN (ANALYZE, BUFFERS)
SELECT device_id, avg(value) AS avg_value
FROM device_metrics_20260715
WHERE value > 0.5
GROUP BY device_id
ORDER BY avg_value DESC
LIMIT 20;

After running the query, you may see:

Limit (actual time=1658.814..1658.820 rows=20 loops=1)
   -> Sort (cost=257079.27..258179.28 rows=440004 width=16)
             (actual time=1649.815..1649.818 rows=20 loops=1)
         Sort Key: (avg(value)) DESC
         Sort Method: top-N heapsort Memory: 26kB
         -> HashAggregate (cost=219176.70..245370.92 rows=440004 width=16)
                            (actual time=1189.615..1593.002 rows=492515 loops=1)
               Group Key: device_id
               Planned Partitions: 16 Batches: 17
               Memory Usage: 8337kB Disk Usage: 78856kB
               Buffers: shared hit=13212 read=17686,
                        temp read=8165 written=9857
               -> Seq Scan on device_metrics_20260715
                             (cost=0.00..83422.95 rows=2119083 width=16)
                             (actual time=0.098..410.231 rows=2102555 loops=1)
                     Filter: (value > '0.5'::double precision)
                     Rows Removed by Filter: 2099445
                     Buffers: shared hit=13212 read=17686
 Planning:
   Buffers: shared hit=103
 Planning Time: 0.505 ms
 Execution Time: 1682.995 ms

Planning is a rounding error: 0.5 ms against 1,683 ms of execution. All the time sits in the nodes, and three lines say where.

Buffers: shared read=17686 means 17,686 blocks came off disk, roughly 138 MB, against 13,212 cache hits. Batches: 17 with Disk Usage: 78856kB means the hash aggregate outgrew work_mem and spilled 77 MB to temporary files. Rows Removed by Filter: 2099445 means the scan read 4.2 million rows to keep 2.1 million.

Here's the part worth sitting with: the estimates were good. The planner predicted 2,119,083 rows from the scan and got 2,102,555. It predicted 440,004 groups and got 492,515. Both within 12 percent. This plan is slow despite being correctly planned, which means ANALYZE will do nothing for it. The fix is work_mem or a storage layout that doesn't read four million rows to answer a question about two columns.

A correct plan can still be a slow plan. That's why you read the split before touching anything.

Step 3: When the estimates are the problem

Sometimes they are wrong, and the damage compounds because every choice downstream inherits the error. Divide estimated rows= by actual at each node and find the first divergence.

Seq Scan on device_metrics_20260715 (cost=0.00..132989.21 rows=84 width=0)
                                      (actual time=4.128..350.008 rows=832 loops=1)
   Filter: ((region = 3) AND (rack = 1750))

Eighty-four predicted, 832 returned. The planner treated region and rack as independent and multiplied their selectivities, but every rack belongs to exactly one region. When two columns carry a functional dependency, tell Postgres:

CREATE STATISTICS dm_region_rack (dependencies)
  ON region, rack FROM device_metrics_20260715;

ANALYZE device_metrics_20260715;

The same scan afterward:

Seq Scan on device_metrics_20260715 (cost=0.00..133489.25 rows=841 width=0)
                                      (actual time=4.167..352.177 rows=832 loops=1)

841 predicted against 832 actual. One caveat that costs people an afternoon: dependencies statistics only apply to equality predicates. Rewrite the same filter as rack BETWEEN 1700 AND 1799 and the estimate collapses back to 1, because range predicates fall outside what functional dependencies model. Reach for ndistinct or an expression index there instead.

If estimates stay wrong after ANALYZE, raise the sample size. The default default_statistics_target of 100 samples roughly 30,000 rows, which is thin on a billion-row table. The new target changes nothing until the next ANALYZE, so run both:

ALTER TABLE device_metrics ALTER COLUMN device_id SET STATISTICS 500;
ANALYZE device_metrics;

Step 4: Fix the planning side

Execution fixes are familiar territory. Planning fixes are the ones teams skip, so here's the one that matters. A prepared statement caches the plan across executions. Prepare it once:

DEALLOCATE ALL;

PREPARE recent_avg AS
SELECT device_id, avg(value)
FROM device_metrics
WHERE ts > now() - interval '5 minutes'
GROUP BY device_id

Then execute it several times on that same connection:

EXPLAIN (ANALYZE) EXECUTE recent_avg;

Prepared statements live and die with the session, so PREPARE and EXECUTE must share a connection. Running them through separate psql -c calls, or through a pooler in transaction mode, returns prepared statement "recent_avg" does not exist.

Measured that way, Planning Time was 75.697 ms, then 0.179 ms, then 0.087 ms and stayed there. Execution held steady near 3 ms. The first call pays for the plan and every call after it inherits one, which turns a 79 ms query into a 3 ms query without touching an index.

A statement with no parameters gets a generic plan immediately, because there is nothing to specialize on. With parameters, Postgres builds custom plans for the first five executions, then compares the generic plan's cost against the custom average and switches if it holds up. Force the decision with plan_cache_mode = force_generic_plan. Verify by re-running EXPLAIN (ANALYZE) EXECUTE and watching Planning Time collapse. If it collapses and total latency doesn't move, planning was never your problem.

Caching hides the cost rather than removing it, so look at partition count too. If a typical query touches one partition and the planner starts from 500, the chunk interval is finer than the workload needs. Rolling data older than a month into weekly partitions cuts the count by roughly 7x.

To find which queries deserve this treatment across the whole workload, rank them in pg_stat_statements with pg_stat_statements.track_planning enabled, then sort by total_plan_time against total_exec_time.

When Planning Time is the architecture talking

A query that spends most of its life being planned is not a tuning failure. It's the shape of the workload pressing against a general-purpose planner. Postgres opens and prices every partition separately because it was built for schemas where each table might hold something different. Time-based chunks all hold the same thing, so most of those 20,063 buffer reads are waste.

Tiger Data attacks this at the source. Hypertables record chunk time ranges in a catalog table, so chunk exclusion resolves which chunks a query needs before standard planning begins. The cost tracks the chunks a query matches rather than the chunks that exist. Continuous aggregates go further: a dashboard query hits a small incrementally-updated rollup instead of planning a scan across billions of raw rows. The 62.9 ms in Step 1 is precisely the overhead those layers exist to remove, and Tiger Data’s whitepaper covers how.

Before you add another index, run EXPLAIN (ANALYZE, BUFFERS) on your slowest query and read the last two lines. If planning wins, you've been optimizing the wrong clock. Start a Tiger Data free trial today to use the right architecture to fix your slow query for good.

Continuous Aggregate Refresh, Demystified: Invalidation, Lookback, and Late-Arriving Data

Team Tiger Data — Fri, 14 Aug 2026 12:00:28 +0000

Yesterday the shift dashboard reported 41,900 units across an eight-hour window. Today the same query over the same eight hours reports 43,100. Nobody shipped a code change and nobody edited a row after the fact, so why did the number move? A plant-floor historian had lost its uplink, buffered locally, and flushed overnight, landing rows that carry their original timestamps. The raw table was correct at every point along the way. The pre-computed aggregate behind the dashboard never went back for those rows, so it published one number and later replaced it with another.

Every pre-computation strategy handles a fresh row arriving at the head of the table. They diverge on the two events no evaluation demo covers: data that arrives late for a window already computed, and data that changes after the fact. This piece runs the four common strategies against exactly those events to show why each behaves the way it does, then ends where the choice gets practical: the two offsets that decide, on a TimescaleDB continuous aggregate, whether a late row is ever reconciled at all.

The four ways to pre-compute a time-series aggregate

The industry files all of these under one label, "materialized view refresh," and the label is where the trouble starts: it implies one correctness story where there are four. Pre-computation is a bet about when the work happens: at write time, at schedule time, or at query time. Four designs place that bet differently.

1. Scheduled full recompute

A materialized view plus cron. In vanilla Postgres, REFRESH MATERIALIZED VIEW rebuilds the view from scratch on every run, with no incremental path, so between runs the view is stale by construction. You pay the full rebuild every run, whether one row changed or a million did. That stays affordable exactly as long as a rebuild of the range you care about fits inside the schedule interval.

For a daily rollup over a month of data on modest hardware, it fits for a long time. For a fine-grained rollup over years of raw samples, it stops fitting well before anyone notices, because the failure arrives as a slow schedule slip and not as an error. InfluxDB v2 tasks have the same shape: a scheduled Flux task reads a window of raw data and writes downsampled results, and the documented lever for lateness is an offset that delays the run rather than an invalidation layer that reaches back after it.

2. Insert-triggered incremental views

ClickHouse standard materialized views fire on INSERT into the source table and write the incremental result forward. Each insert carries a tiny increment, which is what makes the design attractive at ingest rates where a recompute is out of the question. It rests on a strong premise: the past never changes. Nothing in the standard view reacts to a backfill or a correction, so reconciling history becomes a manual, resource-heavy operation.

The same system offers an in-family alternative, refreshable materialized views, which re-run the full query on a schedule and atomically swap the destination table. That is strategy 1 again, reached for because strategy 2's premise broke.

3. Streaming dataflow engines

RisingWave and Materialize maintain a view through a dataflow graph updated on every source change, targeting sub-second freshness. Late data is another change event entering the graph, so there is no refresh window to reason about at all.

The tradeoff here is architectural rather than computational: a persistent streaming system running alongside your database, carrying its own consistency model and its own operational surface. If the freshness contract you have to meet is measured in seconds and your team already runs that layer comfortably, this is a defensible choice, and the rest of this piece is not an argument against it.

4. Invalidation-tracked window refresh

This is what a continuous aggregate in TimescaleDB does, and if you have not run one, the plain version is this: you declare a query with a time bucket and a GROUP BY, TimescaleDB stores its results in a table it maintains for you, and a scheduled policy keeps that table current.

The bookkeeping in between is what makes it a distinct strategy. Writes that touch already-summarized time ranges are recorded in an invalidation log, and each scheduled run re-examines a bounded window of time, recomputing only the buckets inside that window that saw activity. The work stays small and stays inside the database you already run. In exchange, someone has to pick the right width for that window.

Four strategies, each defensible for the workload it was designed around. What decides which one fits yours is what your data does after it first arrives.

The test: new data, late data, changed data

Here is how the four designs answer the three events:

Strategy	New data at the head	Late data for a closed window	UPDATE / DELETE on aggregated rows	What it costs
1. Scheduled full recompute	At next run	At next run	At next run	Full recompute, every run, whether or not anything changed
2. Insert-triggered incremental	Instant	Never	Never	Tiny increment per insert
3. Streaming dataflow	Instant	Instant	Instant	Continuous, in a second system you also operate
4. Invalidation-tracked window refresh	At next run	At next run if inside the window , never if outside	Same rule as late data	Recompute of the window only, and only the buckets that changed

Four strategies against the three things that happen to time-series data.

New data unites them; late data divides them

New data is where they all look alike. A row arrives at the head of the table with a current timestamp, and every one of the four picks it up on its own, with nobody intervening. What separates them here is only how long that takes: milliseconds for the two that work continuously, and up to a full schedule interval for the two that work on a timer. That is a freshness difference, and freshness is a dial you already know how to set. It is the only difference this regime exposes. Every evaluation runs here, which is exactly why the differences that matter stay hidden through the evaluation and surface in production six weeks later.

Late data splits them. Scheduled full recompute is right by construction: the next run recomputes everything in range without caring when any of it arrived. That correctness is prepaid on every run, including the runs where nothing was late.

An insert-triggered view sits at the opposite corner. The late INSERT does fire the trigger, but the view has no way to reach back into an aggregate it already wrote for an older timestamp, so historical corrections require a separate, resource-heavy refresh. The design traded that reach away deliberately in exchange for per-insert cheapness.

Streaming dataflow has no boundary to miss, because it never drew one. A late event is a change like any other, and the graph updates. What looks like a free win here is a cost moved rather than removed: the engine pays continuously to stay in a position where late data is unremarkable.

Changed data widens the split. An UPDATE or DELETE against rows that are already summarized is the case an insert-triggered view has no answer to at all, because no insert happened. The aggregate diverges silently until someone triggers a full refresh, and the divergence carries no timestamp to grep for. Full recompute absorbs the change for free, having never trusted its own previous output. Streaming dataflow propagates a retraction through the graph. TimescaleDB logs UPDATE and DELETE activity as invalidations exactly the way it logs INSERTs, so a correction to an old row inherits the same boundary condition as a late insert: inside the window it is fixed, outside the window it is not.

TimescaleDB's is worth looking at in depth, because it sits between those corners. It uses invalidation-tracked window refresh because late data is unbounded in theory and bounded in practice. Recording which buckets a write disturbed costs almost nothing. Re-examining a window wide enough to cover the lateness you actually see costs far less than recomputing all of history to catch it, and it happens inside the database you are already running. The bet is that your late-data envelope is something you can put a number on. What follows is the machinery that reads that number, and the two offsets you write it into.

Window refresh in detail

The bookkeeping starts before any refresh runs. TimescaleDB maintains an invalidation threshold, also called the materialization watermark: a time cutoff behind the hot head of the table. Mutations landing before it are logged as invalidations, because that region has already been summarized and the summary is now suspect. Mutations landing after it need no bookkeeping at all, because nothing has been materialized there yet. The granularity of that log is worth knowing, because it explains why a bulk backfill behaves differently from a trickle of corrections: each transaction logs the minimum and maximum timestamps of the rows it modified, so one transaction spanning a wide range marks every bucket between its endpoints.

A write whose timestamp lands in an already-materialized bucket is recorded in the invalidation log. Whether it ever gets re-materialized depends on the refresh policy's window. If that bucket falls inside the window when the policy next runs, the bucket is recomputed and the number corrects itself. If it falls outside, the policy never revisits it, and the aggregate keeps reporting the pre-arrival value indefinitely. TimescaleDB's own issue tracker carries a user report of exactly this boundary condition.

The aggregate query still returns, still returns fast, and still returns a plausible number. Nothing in a default policy setup raises "this bucket is stale relative to a write that landed after I last looked at it." The overnight dashboard shift in the opening is that branch resolving the slow way, when a human eventually re-ran something wide enough to sweep the bucket in.

Creating a continuous aggregate in TimescaleDB

Start with where the numbers live.

A continuous aggregate is a materialized view backed by its own hypertable, holding one row per GROUP BY bucket plus a column per aggregate. Here is the canonical shape:

CREATE MATERIALIZED VIEW conditions_summary_hourly
WITH (timescaledb.continuous) AS
SELECT
    device,
    time_bucket(INTERVAL '1 hour', time) AS bucket,
    avg(temperature) AS avg_temp,
    max(temperature) AS max_temp,
    min(temperature) AS min_temp
FROM conditions
GROUP BY device, bucket
WITH NO DATA;

Two parts of that statement are doing the declaring. WITH (timescaledb.continuous) is what makes this a continuous aggregate rather than an ordinary materialized view. The time_bucket call in theGROUP BY is what the continuous machinery then requires, because the bucket is the unit of work it invalidates and recomputes.

WITH NO DATA is a separate decision, and worth being precise about, since it is not the default. It controls the initial backfill at creation time and nothing else. The default, WITH DATA, computes every historical bucket the moment you run the statement, which on a large hypertable is a long blocking build at the worst possible time. WITH NO DATA creates the structure empty and hands the filling to the refresh policy you attach next, plus a manual refresh_continuous_aggregate call for whatever history you want backfilled deliberately. The pre-computation still happens either way. You are choosing when to pay for it.

A refresh run, whether triggered by the policy or by a manual call, is two transactions rather than one: the first briefly blocks writes while it determines the range to materialize and advances the threshold, and the second materializes without blocking writers, so a wide refresh does not hold one long lock.

One refresh cycle. The two offsets are the two edges of the window in the middle box, and everything about staleness is decided by which side of them a bucket falls on.

Which brings the design down to two numbers you set on the policy:

SELECT add_continuous_aggregate_policy('conditions_summary_hourly',
    start_offset => INTERVAL '48 hours',
    end_offset => INTERVAL '1 hour',
    schedule_interval => INTERVAL '30 minutes');

Each run refreshes the range from now() - start_offset to now() - end_offset. So start_offset is the lookback window, the distance back in time the policy is willing to reconsider, and end_offset is a deliberate freshness floor that holds the newest, still-filling bucket out of the refresh so it is not recomputed on every run while data is still landing in it. With one-hour buckets, an end_offset of one hour is the smallest value that does that job.

That floor has a consequence worth handing to whoever consumes the dashboard. Worst-case publication delay is roughly schedule_interval + end_offset: a row lands just after a run finishes, waits a full interval for the next one, and is still held back by end_offset when that one goes. With the values above the composite runs to about ninety minutes, triple the schedule interval taken on its own. Hand downstream consumers the composite, because either parameter alone understates what they will see.

Two things widen the envelope you are sizing for. Hierarchical continuous aggregates are built on top of other continuous aggregates, and each tier runs its own refresh policy against the tier below, so a correction lands at the top only after every tier's schedule has fired in turn, later than the top policy alone suggests. And a retention policy that drops raw chunks still inside your start_offset removes the source rows a late correction would have been reconciled from, so size those two against each other deliberately.

One feature looks like it closes the gap, and it does not. Real-time aggregation unions the materialized buckets with a live query over the tail that has not been materialized yet, so a query can read current even between policy runs. It is genuinely useful, but it solves a different problem: it covers data newer than the watermark. A late write carrying an old timestamp lands in a bucket that is already behind the watermark, and if that bucket is outside every policy window, no amount of real-time aggregation surfaces it, because the live query is not looking there. The two features solve adjacent problems, and it is easy to assume the first one closed the second.

-- Real-time aggregation, if you want the un-materialized tail included in reads.
ALTER MATERIALIZED VIEW conditions_summary_hourly
    SET (timescaledb.materialized_only = false);

Note the default flipped: in TimescaleDB v2.13 and later, real-time aggregates are disabled by default, where earlier versions enabled them. A window sized correctly still tells you nothing about whether the policy that reads it is running.

Checking that the policy is doing its job

TimescaleDB already records what you need to track the calculations. Each job's last_successful_finish, compared against its own schedule_interval, tells you whether it is keeping up. The extra join is there because job_stats reports the internal materialization hypertable rather than the name you query:

SELECT ca.view_name,
       j.schedule_interval,
       js.last_successful_finish,
       now() - js.last_successful_finish AS since_last_success,
       js.last_run_status,
       js.total_failures
FROM timescaledb_information.jobs j
JOIN timescaledb_information.job_stats js
  ON j.job_id = js.job_id
JOIN timescaledb_information.continuous_aggregates ca
  ON ca.materialization_hypertable_name = js.hypertable_name
WHERE j.proc_name = 'policy_refresh_continuous_aggregate';

Audit your own policies this week: for each continuous aggregate, compare how far back your writes actually arrive against the start_offset that policy covers. Where the arrivals run wider than the window, that aggregate is already publishing numbers that will never correct themselves, and widening start_offset is the cheapest thing you can do about it. The refresh-policies documentation carries the parameter reference, and Tiger Cloud is one place to try a wider window against a copy of your own data before you touch production.

What pg_stat_statements Actually Tells You About Your Queries

Team Tiger Data — Fri, 07 Aug 2026 04:00:00 +0000

Your slowest query is rarely your most expensive one. A 4-second report that runs twice a day costs your database 8 seconds. A 3-millisecond lookup that runs 40,000 times a minute costs it two minutes of CPU every sixty seconds. Only one of those shows up in a slow query log, and it is the wrong one.

pg_stat_statements settles this argument. It keeps a running total of every top-level statement your server executes, grouped by structure rather than by literal text. Each group is a fingerprint, with one row in the view representing every execution of the same query shape, with the constants stripped out. Every number in this guide is captured from a real instance, and one query on it spent more time being planned than every other statement combined. It runs in 0.2 milliseconds.

What you will learn

How pg_stat_statements normalizes queries into a fingerprint, and what that collapses.
Which columns matter: calls, total_exec_time, mean_exec_time, rows, and the buffer counters.
How to read a real result set and tell an expensive query from a merely slow one.
How to find planning-dominated queries, which mean_exec_time hides completely.
What the extension does not capture, so you know when to stop trusting it.

Before you start

You need PostgreSQL 13 or later. The column names used here landed in version 13; on 12 and earlier they are total_time and mean_time, and planning time is not tracked at all. You need superuser access to edit postgresql.conf and restart the server, and the role you query with needs pg_read_all_stats. Without it the view still returns rows, but every query column reads <insufficient privilege>.

Enable pg_stat_statements

pg_stat_statements allocates a fixed block of shared memory at postmaster startup, so it has to be in shared_preload_libraries. This is not a pure CREATE EXTENSION install. Edit postgresql.conf first, then restart:

# postgresql.conf
shared_preload_libraries = 'pg_stat_statements'
pg_stat_statements.max = 10000
pg_stat_statements.track = all
pg_stat_statements.track_planning = on

Then, in psql:

CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

SHOW shared_preload_libraries;
SELECT count(*) FROM pg_stat_statements;

Check the last two statements, not the first. CREATE EXTENSION succeeds whether or not the library is preloaded, so it proves nothing on its own. SHOW should list the extension, and the count should be non-zero within seconds of normal traffic. If instead you get ERROR: pg_stat_statements must be loaded via shared_preload_libraries, the config edit or the restart did not work.

Three settings change what you see. pg_stat_statements.max caps tracked fingerprints at 5,000 by default, and past that Postgres evicts entries by a decaying usage score, roughly least-recently-used. track_planning is off by default, so total_plan_time reads as zero until you enable it. That one is not optional here; it is where this article's main finding comes from. And track defaults to top, recording only the outermost statement, so if your logic lives in PL/pgSQL functions every nested query is billed to the wrapper. Set it to all.

Reset the counters before you measure

The view accumulates from the last reset. On a long-running server that can mean two years of history spanning a migration, a bad deploy, and a schema change you have since reverted.

SELECT pg_stat_statements_reset();

Rank by aggregate cost, not per-call cost

This query does most of the work. It ranks fingerprints by total execution time and puts the evidence beside each one:

SELECT
    substring(query, 1, 60) AS query_fragment,
    calls,
    round(total_exec_time::numeric / 1000, 1) AS total_sec,
    round(mean_exec_time::numeric, 3) AS mean_ms,
    round(total_plan_time::numeric / 1000, 1) AS plan_sec,
    round(rows::numeric / nullif(calls, 0), 1) AS avg_rows,
    round(100.0 * shared_blks_hit /
          nullif(shared_blks_hit + shared_blks_read, 0), 1) AS hit_pct
FROM pg_stat_statements
WHERE query NOT ILIKE '%pg_stat_statements%'
ORDER BY total_exec_time DESC
LIMIT 10;

Here is the actual output from an example PostgreSQL 16 instance holding 5 million rows in a device_metrics table across 500 daily partitions, 1.26 GB on disk against 256 MB of shared_buffers, after a mixed workload of 20,000 device lookups, 20,000 metadata lookups, 200 batch inserts, 20 reporting aggregates, and 3 retention deletes:

query_fragment	calls	total_sec	mean_ms	plan_sec	avg_rows	hit_pct
`SELECT date_trunc($1, ts), avg(value) FROM device_metrics WH`	20	251.9	12592.544	0.3	489.8	1.5
`SELECT * FROM device_metrics WHERE device_id = $1 AND ts > n`	20000	4.4	0.221	213	37.8	95.7
`SELECT id, name FROM devices WHERE org_id = $1`	20000	0.9	0.044	0.2	125	100
`INSERT INTO device_metrics (ts, device_id, metric, value, qu`	200	0.8	3.843	0	500	100
`DELETE FROM device_metrics WHERE ts < now() - interval $1`	3	0	16.091	0.1	13510.3	95.4

Two columns decide the question, and they point in different directions. total_sec is what a fingerprint costs your server, while mean_ms is what it costs one user. Read them as a pair, and four cases fall out. High calls with low mean_ms is an application problem: a query in a loop, an ORM N+1, or a dashboard polling faster than anyone reads it, and the fix is batching or caching rather than indexing. Low calls with high mean_ms is a query problem: a missing index, a bad join order, or a scan wider than the result needs. High on both is where you start. Low on both is the healthy majority, and leaving it alone is the correct action.

Every row below is one of those cases.

Row 1 is 251.9 seconds out of 258.0 across every fingerprint on the server, so 98% of execution time belongs to a query called twenty times. Read avg_rows next to hit_pct to see why. It returns 489.8 rows per call, one per day of retention, and does it at a 1.5% cache hit rate. Twenty calls read 2.75 million blocks off disk to produce 9,796 rows.

Row 5 is the trap from the intro. At 16 ms per call the retention DELETE is the second-slowest statement on the box, the first thing an on-call engineer would flag. Three calls, 48 milliseconds total. Ignore it, with one caveat: total_exec_time under-bills any DELETE, because the dead tuples it leaves and the autovacuum passes that clean them up are charged elsewhere. On a partitioned table, DROP TABLE on the oldest partition does the same job in constant time with no vacuum debt.

Use the buffer columns to separate cache misses from CPU

shared_blks_hit counts 8 KB blocks served from the shared buffer cache. shared_blks_read counts blocks that had to come from the OS cache or disk. The hit_pct column above derives from both, and it is worth selecting the raw counts alongside it, because a ratio hides volume. Statements that touched no shared blocks come back blank rather than zero, which is the nullif guard doing its job.

Row 3 sits at 100.0%. It runs 20,000 times, never leaves memory, and costs 0.9 seconds. That is what a healthy fingerprint looks like. Row 4 is the ingest floor, inserting 500 rows per call at 3.8 ms with a perfect hit rate. Neither is worth touching.

Row 1 sits at 1.5%. It scans a 1.26 GB table through a 256 MB cache, so almost nothing it reads is resident, and every block it pulls evicts something another query wanted. That is read amplification, and no index fixes it. The query has to visit every row in the retention window to compute the average.

The 213 seconds mean_ms cannot see

Now look at row 2. It executes in 0.221 milliseconds. By any per-call measure it is the healthiest query in the workload. However, its plan_sec is 213.0.

That is 10.65 milliseconds of planning for 0.221 milliseconds of execution, 48 times more expensive to plan than to run, and 99.7% of all planning time on the server. A slow query log would never show it. mean_exec_time does not include it.

EXPLAIN ANALYZE confirms the ratio and names the cause. Run the query below twice in one session and read the second result. The first pass loads catalog entries for 500 partitions and reports planning time that includes them:

EXPLAIN (ANALYZE, SUMMARY)
SELECT * FROM device_metrics WHERE device_id = 8 AND ts > now() - interval '7 days';

The second result:

Append (cost=0.29..4300.22 rows=539 width=193)
  Subplans Removed: 492
Planning Time: 10.739 ms
Execution Time: 1.354 ms

Subplans Removed: 492 is the tell. The predicate is ts > now() - interval '7 days', and now() is stable rather than constant, so the planner cannot prune at plan time. It builds a subplan for all 500 partitions, then discards 492 of them at execution. Resolving the timestamp in the application and passing a fixed value lets plan-time pruning run instead. Run this in the same session, substituting a real date:

EXPLAIN (ANALYZE, SUMMARY)
SELECT * FROM device_metrics WHERE device_id = 8 AND ts > '2026-07-22'::timestamptz;

And you may see the planning time drop to 0.183 ms. Subplans Removed is gone: same rows, same execution, 59 times less planning.

To fix this, you have to swap now() for a genuine constant. If you use current_date - 7 instead, nothing changes because current_date is stable too and the planner still builds all 500 subplans. Prepared statements and generic plan caching help too. Fewer, larger partitions help more.

Row 2 in the table was visible only because plan_sec happened to sit in the ranking. To sweep for the pattern deliberately, sort by planning time and add the ratio:

SELECT
    substring(query, 1, 60) AS query_fragment,
    calls,
    round(total_plan_time::numeric / 1000, 1) AS plan_sec,
    round(total_exec_time::numeric / 1000, 1) AS exec_sec,
    round((total_plan_time / nullif(calls, 0))::numeric, 3) AS plan_ms_per_call,
    round((100.0 * total_plan_time /
           nullif(total_plan_time + total_exec_time, 0))::numeric, 1) AS plan_pct
FROM pg_stat_statements
WHERE calls > 100
  AND query NOT ILIKE '%pg_stat_statements%'
ORDER BY total_plan_time DESC
LIMIT 10;

Sort by total_plan_time, and do not scan plan_pct on its own. That ratio is a trap: a trivial indexed lookup often plans in more time than it executes because both are microseconds, and on a scratch instance the same column reads 71.8% for a query nobody should touch. plan_ms_per_call is the honest signal. Healthy lookups plan in tens of microseconds. Row 2 planned in 10.65 milliseconds, and multiplied across 20,000 calls that is the 213 seconds. Use plan_pct to confirm what the absolute numbers already flagged, which for row 2 was 98.0%.

When the ranking stops changing

Rows 1 and 2 are both fixable. The pattern underneath them is not.

On a high-frequency time-series workload, total_exec_time, shared_blks_read, and total_plan_time all climb as data volume and partition count grow, even when every query is written correctly. You fix the top fingerprint, it drops to fourth, and two quarters later it is back with the same shape. If your ranking regenerates itself after you fix what is on it, you are measuring architecture, not technique. Columnar storage changes the inputs rather than the query text: a pre-computed rollup answers row 1 without touching 2.75 million blocks. The Optimization Treadmill covers when optimizing further stops paying.

Know the blind spots

Normalization is what makes the ranking possible and also what limits it. WHERE device_id = 42 and WHERE device_id = 9001 collapse into one fingerprint. If one value matches 4 billion rows and another matches an empty range, you get an average that describes neither.

The view also gives you no percentiles, so check max_exec_time before trusting a healthy mean. And it stores no plans. It tells you a query got slower, never that the planner switched from an index scan to a sequential scan. auto_explain fills that gap, but it needs preloading too, and it only logs statements over auto_explain.log_min_duration. Catching a regression in a 0.2 ms query like row 2 means setting that threshold low enough to hurt, so aim it at one fingerprint and turn it off afterward.

One version note: before PostgreSQL 18, IN lists of different lengths produced separate fingerprints, splitting one logical query across several rows. PostgreSQL 18 merges them.

Four more limits are covered above rather than here, because each one distorts a specific number as you read it: track = top bills nested PL/pgSQL queries to the wrapper, pg_stat_statements.max evicts fingerprints once you pass its configured max, track_planning left off makes every total_plan_time read as zero, and total_exec_time under-bills any DELETE because the vacuum work it creates is charged elsewhere.

Sort by planning time next

Reset your statistics, run one peak cycle, and pull the top five fingerprints by total_exec_time. Then run the same query ordered by total_plan_time, because on this instance that second list found the cost center the first one missed. If the same fingerprints keep returning to the top after you fix them, start a Tiger Cloud trial today and run both rankings against columnar storage to see which rows disappear.

How ControlCom Turns 300+ Million Monthly Facility Data Points Into Instant Answers With Tiger Data

Team Tiger Data — Thu, 06 Aug 2026 18:40:28 +0000

ControlCom Connect streams telemetry from thousands of industrial and facility assets into TimescaleDB, powering instant dashboards and an AI assistant, the same platform that caught $160,000 in hidden utility billing errors at one healthcare site and flagged a tier 1 hospital's backup generators left in manual before a power failure could put patients at risk.

A six-person team building a vendor-neutral industrial IoT platform streams telemetry from thousands of facility assets, from PLCs and meters to switchgear and generators, into TimescaleDB. The result: dashboards that re-bucket 300+ million data points a month on the fly, and an AI assistant that answers questions against live and historical readings in real time. That speed shows up across a growing list of catches, among them a healthcare deployment where automated monitoring caught $160,000 in utility billing errors that had gone unnoticed for years, and a tier 1 hospital where the same monitoring flagged backup generators left in manual mode before a routine service call could turn into a life-safety incident.

This is an installment of our Community Member Spotlight series, in which we invite our community members to share their work, spotlight their success, and inspire others with new ways to use TimescaleDB.

Today we hear from Catalin Negru, founder and CEO of ControlCom Technologies, and the team behind ControlCom Connect, an industrial IoT platform that unifies live equipment data across a facility portfolio. Catalin shares how a six-person team of software and electrical engineers evaluated three time-series options before settling on TimescaleDB, and why staying inside Postgres is what lets the platform's AI assistant and living asset graph stay fast as customer data volumes climb.

About ControlCom Connect

The people who keep critical facilities running (healthcare, water and wastewater, prime power, data centers, manufacturing, oil and gas, cold chain) don't get a unified view of the equipment they're responsible for. A SCADA system, a handful of PLCs, a fleet of meters, a generator, and a compressor each speak their own protocol and report into their own app. When a power event hits, the operator's job becomes walking the floor panel by panel, guessing what dropped, what restarted on its own, and what is still down.

ControlCom Connect connects to whatever a facility already owns: SCADA systems, smart devices, controllers, meters, PLCs, HMIs, generators, switchgear, compressors, batteries, cranes, and tractors, plus software sources like APIs, databases, and scripts, over MQTT, Sparkplug B, OPC UA, Modbus, BACnet, Ethernet/IP, HTTPS, and webhooks. Every connected asset is wired into a living graph of the facility, so when an outage hits, the platform traces it downstream in seconds and tells the operator exactly which assets were affected, instead of sending someone to check panels by hand. Live metering and equipment status stream into drag-and-drop dashboards with multi-site comparison, smart notifications route to the right person over SMS, email, Slack, or mobile push with on-call coverage and escalation, and an AI assistant grounded in the organization's own assets and readings answers questions in plain language.

At one healthcare deployment, automated utility monitoring caught more than $160,000 in billing errors that had gone unnoticed until ControlCom's platform started watching the meter data. At a tier 1 hospital, the same kind of continuous monitoring flagged backup generators left in manual mode after a maintenance visit, before a power outage could turn a routine service call into a life-safety incident.

ControlCom Technologies is a six-person team of software and electrical engineers, founded and led by Catalin Negru.

The Challenge

ControlCom Connect's entire platform runs on high-frequency telemetry streaming in from thousands of points across every connected asset, at a rate of 1,000 to 10,000 data points per second, and that volume only grows as customers add sites. Two parts of the product make query speed non-negotiable: the AI assistant has to answer natural-language questions against live and historical readings and return an answer while someone is waiting on it, and the living graph has to resolve relationships and pull time-series data for many points at once, on demand, the moment an outage hits. A user asking their facility a question, or watching the graph resolve an outage, can't sit and stare at a loading indicator.

PostgreSQL was already home for the rest of ControlCom's stack, so the team needed a way to handle time-series at scale without leaving it. - Catalin Negru, Founder & CEO, ControlCom Technologies

Why Tiger Data: Architecture-First From Day One

Catalin found TimescaleDB by googling time-series databases, after the InfluxDB and AWS proofs-of-concept fell short. Because ControlCom Connect was already built on PostgreSQL, TimescaleDB (built on PostgreSQL) was an easy choice. The team kept the database they trusted, their existing tooling, and plain SQL, while gaining hypertables, compression, and fast time-series queries on top. There was no migration to a foreign database, no new query language for the engineering team to learn, and no second connection pool to manage alongside the relational data that already ran the platform.

It is just Postgres, so we kept the database we trusted, our existing tooling, and plain SQL, while gaining hypertables, compression, and fast time-series queries on top. - Catalin Negru, Founder & CEO, ControlCom Technologies

Compression mattered as much as query speed. ControlCom runs a hybrid deployment: a local TimescaleDB on every Edge Server for on-site resilience, and a central TimescaleDB on Tiger Cloud that holds the portfolio-wide record. Keeping cost in check on the cloud side, while ingesting from thousands of points across every customer's fleet, was a factor in the decision from day one.

The ControlCom Connect Stack

ControlCom Connect's pipeline starts at the device: a PLC, controller, meter, or other piece of equipment. Each edge site runs an Edge Server, which also runs a local TimescaleDB that buffers and stores telemetry on site up to a configurable window, so a facility keeps monitoring itself even if it loses its connection upstream. The Edge Server publishes to an MQTT broker, which feeds into Kafka.

Kafka is where the real-time work happens. As readings stream through, ControlCom processes alarms and parses for anomalies the instant they arrive, writes the telemetry into the central TimescaleDB instance on Tiger Cloud, and pushes the update to Redis and out to every connected client in real time. The central TimescaleDB instance is the portfolio-wide source of truth: the AI assistant queries it for natural-language answers, the living asset graph queries it to trace an outage downstream, and the dashboards query it to render live and historical views across every site a customer operates.

ControlCom Connect's data flow: facility equipment publishes through an Edge Server with a local TimescaleDB for on-site resilience, into an MQTT broker and Kafka. Kafka drives alarm and anomaly processing and writes to a central TimescaleDB on Tiger Cloud, which powers live dashboards, the AI assistant, and the living asset graph.

Results: What ControlCom has seen

300+ Million data points a month, re-bucketed instantly

ControlCom routinely pulls from 300+ million data points a month. Users are used to being amazed: flip a setting on a graph, widen or narrow the bucket size, push the start or end date out, and the chart updates instantly. Re-bucketing and re-ranging across that volume on the fly is the kind of workload that would crawl on a general-purpose setup. On TimescaleDB, data is served in real time, so exploring months of high-frequency data feels as snappy as scrolling a single day.

Users are amazed when they flip a setting on a graph, increasing or decreasing the bucket size, or changing the start and end time, and the chart updates instantly. - Catalin Negru, Founder & CEO, ControlCom Technologies

An AI assistant that doesn't feel like a demo

In the industrial sector, the gap shows up the moment something goes wrong. Operators run facilities from a wall of dashboards, sometimes 80 of them, watching for any of 1,500 possible alarms. That works when conditions are calm. But when 20 alarms fire at once, the challenge stops being a lack of data and becomes finding the one signal that explains what happened, fast enough to act on it. That's the gap ControlCom's AI assistant is built to close.

ControlCom's AI assistant answers natural-language questions against both live and historical readings, which means every question triggers a real time-series query: an aggregation, a recent window, a multi-point comparison, run on the fly while the user waits for an answer. With TimescaleDB behind it, those queries come back fast enough that the conversation feels instant instead of leaving the user staring at a loading indicator, which is what makes an AI assistant grounded in live facility data actually usable rather than a demo.

$160,000 in hidden utility billing errors

The clearest proof of what fast time-series queries make possible showed up outside the dashboard entirely. One healthcare deployment's automated utility monitoring surfaced more than $160,000 in billing errors that had gone unnoticed. Nobody was going to find that by eyeballing a meter. It took a platform that could watch every reading, all the time, and flag the anomaly the moment it appeared.

One healthcare deployment, for example, surfaced over $160K in utility billing errors that would otherwise have gone unnoticed. - Catalin Negru, Founder & CEO, ControlCom Technologies

Beyond the balance sheet: Preventing a Tier 1 hospital blackout

Not every win shows up on a balance sheet. At one Tier 1 hospital, a maintenance crew working on the facility's backup generators left them in manual mode after finishing the job and leaving the site. ControlCom's platform caught the anomaly and escalated a notification to the facilities team that the system was out of spec. That catch depends on the same thing that makes the billing win possible: TimescaleDB lets ControlCom continuously track every asset's status across a portfolio, so a state change like a generator sitting in manual doesn't go unnoticed until someone happens to check. Had a power outage hit while the generators sat in manual, the hospital could have lost backup power entirely, with patients on operating tables and millions of dollars in potential damage on the line.

This could have caused millions of dollars in damage if a power outage occurred. You have real people in operating rooms on the operating tables. - Catalin Negru, Founder & CEO, ControlCom Technologies

Looking ahead

ControlCom is pushing further into the real-time side of the platform: deeper stream processing and analytics, with anomaly detection that flags issues the moment a reading arrives instead of on the next dashboard refresh. The AI assistant and the living asset graph are next in line for the same treatment, since both depend on time-series queries staying fast as the underlying data grows.

That data volume is only going in one direction. As ControlCom adds more sites and more equipment for existing customers, and onboards new ones, the ingestion rate climbs across the board. The architecture decision that compounds as it scales isn't any single query. It's that the Edge Server's local TimescaleDB and the central TimescaleDB on Tiger Cloud are the same technology end to end: one set of hypertables, compression, and SQL, from the edge to the portfolio-wide record. No split time-series stack to reconcile as fleets grow, and no separate system to keep affordable as history piles up.