DEV Community: Team Tiger Data

TimescaleDB 2.28: Faster Queries, Lighter Operations, and Better Schema Evolution

Team Tiger Data — Wed, 29 Jul 2026 12:30:13 +0000

Time-series analytics at scale creates operational friction. When you're running continuous aggregates, columnar storage, and complex analytical patterns, each new metric, query pattern, and configuration tuning attempt adds complexity. Refreshes block each other, configuration changes require rebuilds, and new aggregates mean recomputing entire rollups.

Over recent TimescaleDB releases, we've prioritized minimizing these hurdles by leveraging bloom filters to bypass redundant processing during high-volume operations on columnar storage. We also expanded vectorized execution across more query patterns and simplified continuous aggregate workflows by combining refresh and compression.

Now with TimescaleDB 2.28, we're making common analytical queries faster without code changes, making continuous aggregate operations less disruptive and more flexible, and eliminating friction when evolving your schema and configuration. The result is faster queries, lighter operations, and the ability to evolve your analytics alongside your application as it scales.

TL;DR

Lighter, more flexible continuous aggregates:

ADD COLUMN on CAggs: Add new aggregations in place without rebuilding
Fine-grained locking: Refreshes no longer serialize unrelated operations
Incremental refresh batching: Break large refreshes into batches instead of one heavy operation
ANALYZE and VACUUM: Maintenance commands now work directly on continuous aggregates

Faster queries on compressed data:

Metadata-driven first() and last() queries: Answer from batch metadata without decompression
Vectorized CASE expressions: Conditional logic stays on the fast path
Batch Sorted Merge for more queries: Lighter-weight sorting on compressed data

Better operational flexibility:

Sparse index retrofitting: Update configuration on existing chunks without recompression
Compression settings clarity: Warnings prevent misconfiguration
GUC for bulk loads: Skip invalidation tracking during migrations
Nullable ORDER BY safety: Fallback to correct compression path

Also: PostgreSQL 15 support ends with 2.28. Plan your upgrade to PG16, PG17, or PG18.

Schema evolution: ADD COLUMN on CAggs

Schema evolution lets you add new aggregated columns without dropping and rebuilding. Before 2.28, adding a metric meant recreating the entire CAgg, recomputing all historical data (hours on large datasets), and breaking downstream consumers. Now you can ADD COLUMN with GENERATED ALWAYS AS and backfill incrementally.

-- Add a new metric to an existing CAgg
ALTER MATERIALIZED VIEW conditions_summary_hourly
  ADD COLUMN max_temp double precision
  GENERATED ALWAYS AS (
    max(temperature)
  ) STORED;

-- Backfill historical data
CALL refresh_continuous_aggregate(
  'conditions_summary_hourly',
  NULL,
  NULL,
  force => true
);

How we made continuous aggregates operational at scale

Real-time dashboards on operational data used to force a choice: refresh frequently and block bulk loads, or refresh less often and accept stale data. Every continuous aggregate refresh acquired a table-wide lock on its materialized hypertable, blocking concurrent operations. In 2.28, we switched to row-level locking on the continuous aggregate catalog entry. Only one refresh processes a CAgg's invalidation log at a time, but the materialized table is free for concurrent operations. Refreshes and bulk loads now run in parallel.

We also made continuous aggregates more flexible. ADD COLUMN on CAggs enables you to evolve your analytical schema in production by adding new aggregations as you learn what data you need, without downtime or rebuilds. Your dashboard design no longer locks you into upfront predictions.

-- Before 2.28: refresh acquired table-wide lock
-- After 2.28: refresh acquires only catalog row lock
CALL refresh_continuous_aggregate(
  'metrics_hourly',
  '2026-01-01',
  '2026-02-01'
);
-- Other refreshes, bulk loads, and DDL can proceed

2.28 also adds two capabilities that make continuous aggregates evolve with your application:

Incremental refresh batching processes large time windows in smaller batches instead of single long-running transactions. Before 2.28, refresh_continuous_aggregate() materialized the entire window atomically, holding resources and blocking vacuums. Large windows could take hours and fail mid-way. Now you can batch them.

   -- Refresh a 30-day window in smaller batches
   CALL refresh_continuous_aggregate(
     'metrics_hourly',
     now() - INTERVAL '30 days',
     now(),
     options => '{
       "buckets_per_batch": 10,
       "max_batches_per_execution": 20
     }'::jsonb
   );

Schema evolution lets you add new aggregated columns without dropping and rebuilding. Before 2.28, adding a metric meant recreating the entire CAgg, recomputing all historical data (hours on large datasets), and breaking downstream consumers. Now you can ADD COLUMN with GENERATED ALWAYS AS and backfill incrementally.

-- Add a new metric to an existing CAgg
ALTER MATERIALIZED VIEW conditions_summary_hourly
  ADD COLUMN max_temp double precision
  GENERATED ALWAYS AS (
    max(temperature)
  ) STORED;

-- Backfill historical data
CALL refresh_continuous_aggregate(
  'conditions_summary_hourly',
  NULL,
  NULL,
  force => true
);

ANALYZE and VACUUM now work on continuous aggregates directly. Maintenance commands automatically maintain accurate planner statistics across the materialized hypertable and all its chunks, ensuring query plans stay optimal.

Faster queries on compressed data

One of the most common queries in time-series workloads is "give me the latest value per series." Dashboards and monitoring systems run it constantly. It's a simple pattern: find the first or last value in a time range. But on compressed data, answering that query used to require decompressing entire batches just to find values that are already in the metadata. Time-sorted batches store first and last values as metadata. But before 2.28, the database decompressed anyway to answer those queries.

In 2.28, TimescaleDB extracts first(value, time) and last(value, time) aggregates directly from batch sparse indexes. No decompression. For "latest reading" queries that consume significant resources at scale, that means meaningful speedup with zero query changes. Here's what that looks like:

-- Define firstlast sparse index
ALTER TABLE metrics SET (
  tsdb.index = 'firstlast(temperature)'
);

-- Note: Use rebuild_sparse_index() to retrofit existing chunks
-- New chunks now answer this query from metadata alone
SELECT
  device_id,
  last(temperature, time)
FROM metrics
WHERE time > now() - INTERVAL '7 days'
GROUP BY device_id;

Beyond first/last, we also extended vectorized execution to cover CASE expressions. Before 2.28, conditional logic in aggregations forced columnar queries to fall back to row-by-row processing, making identical queries perform very differently depending on whether a CASE expression was present.

-- This query now stays fully vectorized
SELECT
  time_bucket('1 hour', ts) AS bucket,
  SUM(
    CASE WHEN status_code >= 500 THEN 1 ELSE 0 END
  ) AS error_count,
  AVG(
    CASE WHEN latency_ms > 1000 THEN latency_ms END
  ) AS slow_avg_latency
FROM requests
WHERE ts > now() - INTERVAL '7 days'
GROUP BY bucket;

Pivot-style queries, error rate calculations, and conditional metrics all stay on the fast vectorized path. For workloads that use conditional aggregations on compressed history, that eliminates the surprise performance cliffs.

Finally, Batch Sorted Merge now applies to more query patterns. ORDER BY queries on unordered compressed chunks without segmentation no longer fall back to expensive external sorts. The planner now replaces those sorts with lightweight metadata merges over pre-sorted batches.

Better operational flexibility and efficiency

Tuning time-series workloads means iterating on compression strategy. But historically, every configuration change (adding a sparse index, adjusting segmentby, optimizing orderby) only applied to future compressions. Existing chunks kept their old settings, creating inconsistent performance and confusion about why newly tuned queries behaved differently.

2.28 eliminates that friction. rebuild_sparse_index lets you retrofit sparse index configuration to existing chunks without recompressing. Sparse indexes are metadata. Updating them shouldn't require rewriting data. Below is the actual code snippet you can leverage:

-- Change sparse index settings on the hypertable
ALTER TABLE metrics SET (
  tsdb.index = 'minmax(value)'
);

-- Retrofit existing chunks without recompression
SELECT _timescaledb_functions.rebuild_sparse_index(
  '_timescaledb_internal._hyper_1_42_chunk'
);

Compression settings changes now emit warnings clarifying that new settings apply only to future compressions. This closes the gap: users assume ALTER TABLE sets configuration across the entire dataset, but it only affects new chunks.

For correctness, nullable ORDER BY columns now safely fall back to decompress-compress during recompression, preventing silent incorrect results when min/max metadata doesn't account for NULLs.

Gapfill row count estimation improved so time_bucket_gapfill queries get more accurate planner statistics and better query plans.

For bulk migrations, a new GUC timescaledb.skip_cagg_invalidation suppresses continuous aggregate invalidation tracking during bulk loads. Migration tools no longer generate useless invalidation entries that trigger expensive refresh storms.

PostgreSQL 15: Final Release and Migration Path

PostgreSQL 15 support ends with TimescaleDB 2.28. Going forward, only PostgreSQL 16, 17, and 18 are supported.

If you're on PG15, plan your upgrade to PG17 or PG18 now. We'll begin upgrading instances in production the week of September 15th, so you should plan your migration path over the next two months to avoid downtime. Postgres upgrades are typically non-disruptive (seconds to minutes of downtime using logical replication or physical backup/restore), and newer Postgres versions bring improvements in query parallelism, vector search optimization, and compression.

Upgrade to 2.28 today

For workloads running continuous aggregates at scale, columnar queries with conditional logic, or iterating on compression configuration, 2.28 removes operational friction: refreshes don't serialize, queries stay vectorized, and tuning doesn't require rewrites.

2.28 is available now. To learn more, check out the full release notes for a complete list of improvements, or try Tiger Cloud for free and experience TimescaleDB 2.28 on your largest hypertables. We welcome your feedback on GitHub. Please note that for Tiger Cloud customers, all improvements are live immediately. For self-hosted deployments, download thelatest release and follow theupgrade guide.

The Hidden Cost of Postgres Constraints at Scale

Team Tiger Data — Fri, 24 Jul 2026 14:30:49 +0000

Your ingest workers are queuing. pg_stat_activity shows lock waits. The blocked query is not a slow SELECT. It's your bulk INSERT, waiting on the devices table.

You added a FOREIGN KEY there months ago. You added a UNIQUE constraint on the readings table to catch duplicates. Both were the right call. At 100 devices and 10,000 rows, you never felt them. At 50K inserts per second, they've become the ceiling.

What you will learn

What Postgres actually executes on every insert to enforce FOREIGN KEY and UNIQUE constraints
Why this overhead is invisible during a PoC and destructive at production ingest rates
Four concrete approaches that preserve data integrity without paying the full constraint cost on every row

Why it matters

The Postgres Optimization Treadmill describes how high-frequency time-series workloads hit architectural ceilings despite correct tuning. MVCC overhead, index write amplification, and WAL volume all compound as data grows. Constraint enforcement layers on top of all of that, and it compounds in the same direction.

Every FOREIGN KEY fires an index lookup against the referenced table on every insert. At 50K inserts/sec, that's 50K random reads per second competing directly with your write path. Every UNIQUE constraint fires an index scan before every insert, on an append-only table where duplicates shouldn't ever occur. Both generate additional WAL records and hold row-level locks during execution. Together, they quietly consume the ingest safety margin between "running well" and "falling behind."

Tracing a single constrained insert

A vanilla insert into a plain table performs two operations: a heap tuple write and a WAL commit record.

A constrained insert does five:

Heap write. The row is written to the 8KB heap page.
B-tree insertion. Every index on the table receives a new entry, traversing from root to leaf and splitting pages as needed.
FK shared-lock acquisition. Postgres acquires a FOR KEY SHARE on the referenced row in the parent table (devices) to verify it exists.
UNIQUE index scan. Postgres scans the unique index to confirm no matching entry already exists before writing.
WAL commit record. The constraint checks generate WAL in addition to the row write itself.

Under concurrent write load at 50K inserts/sec, step 3 is where it breaks down — not through mutual blocking, since FOR KEY SHARE locks are compatible with each other, but through MultiXacts. With only a handful of device rows referenced by thousands of concurrent transactions, each parent row is locked FOR KEY SHARE by many transactions at once, and Postgres must track that shared ownership with a MultiXactID. At this concurrency the churn saturates the MultiXact SLRU caches. pg_stat_activity surfaces this as MultiXact LWLock wait events (only for PostgreSQL 16-18; event names may differ on earlier versions), not as a query performance problem.

Not all constraints carry the same cost. NOT NULL and CHECK constraints evaluate against the row being inserted with no external lookups. They're near-free. FOREIGN KEY and UNIQUE are where the overhead lives, because both require reads against external state on every single insert.

Identifying the problem

Run this query during your next peak ingest window:

SELECT
    pid,
    wait_event_type,
    wait_event,
    query,
    state,
    now() - query_start AS duration
FROM pg_stat_activity
WHERE state = 'active'
  AND (
        (wait_event_type = 'LWLock' AND wait_event LIKE 'MultiXact%') -- PostgreSQL 16–18: SLRU wait-event names verified on these versions
     OR (wait_event_type = 'Lock' AND wait_event IN ('transactionid', 'tuple'))
      )
ORDER BY duration DESC;

All MultiXact wait events share the MultiXact prefix on PostgreSQL 16-18, so the LIKE 'MultiXact%' filter captures all SLRU contention events regardless of minor version. Look for rows where wait_event starts with MultiXact. That indicates MultiXact SLRU contention from FK checks on hot parent rows - the mechanism described above. Rows where wait_event is transactionid or tuple indicate true row-lock waits from a concurrent update or delete on the parent, a less common but related failure mode.

Four approaches to reduce constraint overhead

These options are ordered by risk and invasiveness. Start with option 1 if you have existing constraints and need a minimal-change fix. Move to option 2 if your duplication window is bounded to recent data. Use options 3 or 4 only if you own the full write path end to end and can enforce integrity outside the database.

1. Defer FK checks to commit time

Postgres supports deferring constraint checks to commit time rather than row time. Inside a bulk-insert transaction, the FK lookup runs once per batch instead of once per row.

First, declare the constraint as deferrable. This is backward-compatible: the constraint still enforces row-by-row in any transaction that does not explicitly defer it.

ALTER TABLE sensor_readings
  DROP CONSTRAINT IF EXISTS sensor_readings_device_id_fkey;

ALTER TABLE sensor_readings
  ADD CONSTRAINT sensor_readings_device_id_fkey
  FOREIGN KEY (device_id) REFERENCES devices(id)
  DEFERRABLE INITIALLY IMMEDIATE;

Then defer it inside each bulk-insert transaction:

BEGIN;

SET CONSTRAINTS sensor_readings_device_id_fkey DEFERRED;

INSERT INTO sensor_readings (ts, device_id, value)
SELECT ts, device_id, value FROM staging_data;

COMMIT;

For a batch of 1,000 rows, this turns 1,000 FK lookups into one check at commit. The integrity guarantee is unchanged: if any device_id in the batch does not exist in devices, the commit fails and the batch rolls back. Transactions that do not call SET CONSTRAINTS ... DEFERRED continue to enforce row-by-row, so this change does not affect other callers.

2. Scope the UNIQUE check to your live data window

If your primary concern is duplicate prevention during a historical bulk load or replay, a partial index eliminates the UNIQUE check for any row whose timestamp falls outside the live window.

ALTER TABLE sensor_readings
  DROP CONSTRAINT IF EXISTS sensor_readings_unique_reading;

CREATE UNIQUE INDEX idx_sensor_readings_recent_unique
  ON sensor_readings (device_id, ts)
  WHERE ts > now() - INTERVAL '7 days';

Postgres evaluates the partial index predicate at INSERT time. A row inserted with ts = now() satisfies ts > now() - 7 days and gets the UNIQUE check. A row inserted during a historical backfill with ts = '2024-01-15' does not satisfy the predicate in 2026 and skips the check entirely. That is the primary benefit: bulk loads of historical data avoid the UNIQUE scan completely.

For ongoing ingestion of fresh data, each new row is added to the partial index as it's inserted, so the index grows over time. To reclaim the size advantage, schedule a weekly rebuild:

REINDEX INDEX CONCURRENTLY idx_sensor_readings_recent_unique;

At 2 years of retention (730 days), a freshly rebuilt 7-day partial index covers roughly 1% of the dataset: 7 / 730 = 0.0096. The index is approximately 1/100th the size of a full-table UNIQUE index on the same columns, which reduces both scan time and per-insert write amplification by the same factor.

3. Validate FK references in the application layer

For workloads where the device set is stable and well-known, validating device_id in the application before inserting removes the per-row database lookup completely.

import psycopg2

# Cache valid device IDs at startup; refresh on a schedule
def load_valid_devices(conn):
    with conn.cursor() as cur:
        cur.execute("SELECT id FROM devices;")
        return {row[0] for row in cur.fetchall()}

# Initialize the cache
conn = psycopg2.connect("dbname=mydb user=postgres host=localhost")
valid_devices = load_valid_devices(conn)

def insert_readings(conn, batch):
    # Filter invalid device IDs before they reach the database
    valid_batch = [r for r in batch if r["device_id"] in valid_devices]
    invalid_count = len(batch) - len(valid_batch)
    if invalid_count > 0:
        # Log or alert; this signals an upstream data quality issue
        print(f"Dropped {invalid_count} rows with unknown device_id")

    with conn.cursor() as cur:
        cur.executemany(
            "INSERT INTO sensor_readings (ts, device_id, value) VALUES (%s, %s, %s)",
            [(r["ts"], r["device_id"], r["value"]) for r in valid_batch],
        )
    conn.commit()

This trades a database-level guarantee for an application-level guarantee. It works when the application owns the write path and invalid device_id values represent an upstream data quality problem rather than a concurrent-write race condition. Once this pattern is in place, the database-level FK is redundant, which makes option 4 available.

4. Drop the FK constraint

If your ingest pipeline already validates device_id before writing to Postgres (as shown in option 3), the database-level FK enforces a guarantee the pipeline already provides. Removing it cuts the per-insert lock acquisition entirely.

ALTER TABLE sensor_readings
  DROP CONSTRAINT IF EXISTS sensor_readings_device_id_fkey;

This eliminates the shared-lock acquisition on devices for every insert and reduces WAL records by removing the per-row constraint check entries. Combined with the partial UNIQUE index from option 2, this recovers measurable ingest headroom without changing hardware. The Postgres Optimization Treadmill article shows that a 50K inserts/sec workload with five indexes already generates 25-50MB/sec of WAL from heap and index writes. Dropping the FK removes additional per-row overhead sitting on top of that baseline.

The tradeoff is real: no FK means no database-level catch for data quality bugs that slip through the pipeline. Only drop the constraint if option 3 is in place and you have monitoring to detect upstream device ID mismatches before they reach the database.

Validating the fix

After applying option 1 or option 4, rerun the detection query during peak load:

SELECT
    pid,
    wait_event_type,
    wait_event,
    query,
    state,
    now() - query_start AS duration
FROM pg_stat_activity
WHERE state = 'active'
  AND (
        (wait_event_type = 'LWLock' AND wait_event LIKE 'MultiXact%') -- PostgreSQL 16–18
     OR (wait_event_type = 'Lock' AND wait_event IN ('transactionid', 'tuple'))
      )
ORDER BY duration DESC;

MultiXact LWLock waits and any transactionid/tuple row-lock waits tied to the devices table should drop to near zero (MultiXact events only visible on PostgreSQL 16-18). If they persist after applying option 1, confirm the constraint was successfully altered to include DEFERRABLE before the transaction runs SET CONSTRAINTS ... DEFERRED.

After applying option 2, confirm the partial index exists and the full constraint is gone:

SELECT
    i.indexname,
    pg_size_pretty(pg_relation_size(i.indexrelid)) AS index_size,
    x.indpred
FROM pg_stat_user_indexes i
JOIN pg_index x USING (indexrelid)
WHERE i.relname = 'sensor_readings';

The indpred column contains the partial index predicate as text. A non-null value confirms the index is partial. The index size should reflect only the data that has been inserted since the last REINDEX. At 2 years of retention with a 7-day partial index freshly rebuilt, expect a size approximately 1/100th that of a full-table index on the same columns.

To verify WAL reduction after option 4, compare wal_bytes from pg_stat_wal before and after dropping the FK:

SELECT wal_records, wal_bytes, pg_size_pretty(wal_bytes) AS wal_size
FROM pg_stat_wal;

After dropping the FK, wal_bytes growth rate should decrease measurably within a few minutes of sustained ingest. On older Postgres versions, check pg_stat_bgwriter for write activity trends instead.

Next step

Run the pg_stat_activity lock detection query during your next peak ingest window. If you see lock waits pointing at your devices table, apply option 1 first: alter the FK to be deferrable and add SET CONSTRAINTS ... DEFERRED to your bulk-insert transactions. It's a two-statement change with no impact on the integrity guarantee and no risk to other callers.

If your ingest rate is still climbing and you're already on the optimization treadmill, the Tiger Data free trial lets you validate ingest headroom on your own data.

What's New in Tiger Cloud: Bigger Performance Gains, Wider Platform Reach, Better Visibility

Team Tiger Data — Fri, 17 Jul 2026 18:51:12 +0000

This year, we've focused on improving three areas that define the Tiger Cloud experience:

Scale without splitting your architecture: Compression becomes a performance advantage. UPDATE and DELETE on compressed data run up to 160x faster, summary queries up to 70x faster. Storage scales to 80,000 IOPS and 64 TB on demand.
Spend less time configuring, more time shipping: Tiger Console auto-tunes hypertables, the PostgreSQL Source Connector moves data to Tiger Cloud without custom pipelines, and pg_textsearch brings production-ready BM25 search natively in Postgres.
Production-grade reliability, without the DIY tax: Tiger Cloud handles data residency, network isolation, disaster recovery, and visibility so you don't have to.

This past quarter we shipped deeper query engine optimizations in TimescaleDB, new regions, new enterprise networking options, and a long list of smaller improvements to how Tiger Console works day-to-day. Instead of listing every Tiger Cloud release on its own, here's what it adds up to, and why it matters: you can stay on Postgres as you scale, you'll spend less time configuring and more time shipping, and you get the reliability and visibility that time-series workloads actually need.

Scale without splitting your architecture

The moment analytical queries start competing with transactional ones, teams feel pressure to bolt on a separate analytical database. This quarter's TimescaleDB releases and storage upgrades ensure Postgres keeps scaling for time-series and analytical workloads instead of becoming the reason you re-architect. Here's what shipped, and why it matters.

Run queries and writes directly on compressed data, without the performance tax

Compression used to mean a trade-off: a smaller footprint for slower access. With the release of TimescaleDB v2.26, that trade-off keeps shrinking. Aggregate queries like COUNT, MIN, MAX, and FIRST/LAST now read straight from compressed metadata instead of decompressing full batches, up to 70x faster. Grouping with time_bucket() runs roughly 3.5x faster. Multi-column filters push down directly into compressed scans, cutting unnecessary decompression by half or more.

Writes get the same treatment. TimescaleDB v2.27 lets UPDATE, DELETE, and UPSERT on compressed chunks skip decompressing data that can't match, so selective write operations run up to 160x faster. Query rewriting can automatically route matching aggregations to a continuous aggregate, and continuous aggregate refreshes can compress chunks as part of the same job instead of needing a separate policy.

Continuous aggregates are now more reliable at scale. We fixed three stability issues that were constraining them: a memory leak, query correctness edge cases, and a deadlock during concurrent refreshes. As a result, you can now push continuous aggregates harder without operational workarounds or special handling.

Add full-text search without adding a search engine

As part of the pg_textsearch v1.0.0 release, BM25 full-text search now runs natively inside Postgres, and is production-ready. Add relevance-ranked search to your application without standing up and syncing a separate Elasticsearch cluster. In benchmarks at 138 million documents, pg_textsearch ran up to 6.5x faster than ParadeDB on typical multi-word queries and sustained 8.7x higher concurrent throughput. It ships with an <@> query syntax, a bm25_force_merge() function for segment consolidation, and support for Postgres 17 and 18. One less system in your stack to operate and keep in sync.

Scale storage on demand instead of provisioning for a peak that may not come

Scale plan services can now choose between 16,000 and 40,000 IOPS with up to 1,500 MB/s of throughput. Enterprise plans go up to 80,000 IOPS and 2,000 MB/s, with total capacity up to 64 TB. Changes apply without downtime, and you pay only for the IOPS you use. Size up as your workload grows instead of guessing at peak load today.

Spend less time configuring, more time shipping

None of the above matters much if half your week still goes to console configuration instead of building. The following updates hand more of that time back to you so you can focus on what matters: building your product, not configuring your database.

Build hypertables in a few clicks, without writing SQL

Define hypertable columns directly in Tiger Console instead of writing SQL. Configure a columnstore in the same step. For the best performance, you can enableautomated chunk tuning afterward, so you won't have to manually set chunk intervals.

Move data from Postgres into Tiger Cloud without building your own pipeline

The PostgreSQL Source Connector is now stable and ready for production use. Replicate an existing Postgres database into Tiger Cloud without hand-rolling a migration or sync pipeline. It supports a configurable worker count for the initial data copy, table selection by publication or direct selection, SSH tunneling, and bulk updates for table and schema mappings. Everything you need to bring production data over reliably.

Production-grade reliability, without the DIY tax

Data residency, network isolation, disaster recovery, and visibility into what's happening inside your database are table stakes for any fully-managed platform. The whole point of choosing Tiger Cloud is that you shouldn't have to design, build, and maintain that infrastructure yourself. Here's what shipped this quarter that takes more of that off your plate.

Meet data residency requirements in more places

Tiger Cloud is now available in two additional Azure regions: Germany West Central (Frankfurt) and Southeast Asia (Singapore). Teams with GDPR-sensitive workloads can now keep EU data in-region, and Asia-Pacific teams get local data residency and lower latency, without moving off Azure. For a list of all available regions, click here.

Keep database traffic off the public internet

Private endpoint support is now generally available across every supported AWS and Azure region. AWS PrivateLink connects from your VPC over the AWS private network. Azure Private Link does the same from your VNet over Microsoft's private backbone. Both are configured directly in Tiger Console and included on Scale and Enterprise plans, so database traffic never has to touch the public internet.

Recover easily when a region goes down

Cross-region backup already copies your data to a geographically distant region. Now Enterprise customers can restore directly from that backup in Tiger Console, closing the loop so a regional outage doesn't mean opening a support ticket to get your data back. To learn more, click here.

Catch problems in the tools you already use, before they escalate

Visibility should be simple. You shouldn't have to wait for a problem to surface before you can see it coming. The Tiger Cloud status page now lives at status.tigerdata.com, tied directly into incident response, so you can subscribe and get notified the moment an incident is created, updated, or resolved instead of finding out from a support thread.

Inside the Metrics tab in Tiger Console, a new Queries per Second graph gives a real-time view of throughput, making it easier to spot spikes or drops in query volume.

In the Insights tab, the query deep dive page now tracks CPU, memory, and storage IO (read and write) over time, so you can catch a query's resource footprint trending the wrong way and see its downstream impact on system health before it turns into a bigger problem.

Additionally, you can check the Chunk timeline in the Explorer tab to see how your data is organized across chunks. You can inspect sizes, time ranges, and whether each chunk is in the rowstore or columnstore. This lets you spot organization issues and monitor columnstore job health without running system queries.

Lastly, if you already monitor infrastructure elsewhere, Tiger Cloud now exports telemetry to Azure Monitor, plus PostgreSQL-specific metrics (replication, cache usage, background activity) to Amazon CloudWatch, Datadog, and Prometheus, with system-level disk IO and throughput metrics exported by default. Wherever you already look for problems, Tiger Cloud's data is there too. For a full list of available metrics you can export with Tiger Cloud exporters, click here.

Try Tiger Cloud for free

All of these features are live on Tiger Cloud. If you're already a customer, sign in and check out the latest TimescaleDB releases on your compressed hypertables and the new IOPS options if you're storage-bound.

If you're new to Tiger Cloud, start a free trial and see what a Postgres-native operational analytics database looks like.

The Data Layer for the AI Data Center

Team Tiger Data — Wed, 08 Jul 2026 13:47:46 +0000

This is Part II of a two-part series on the AI data center stack. Part I, AI's Physical Constraints: How AI Rewired the Data Center, explains why AI capacity has become constrained by the physical data center stack: accelerators, memory, cooling, power, time, and water.

Physical Plant Telemetry Is Now a Data-Layer Requirement

In an AI data center, the workload no longer stops at the server boundary. A synchronized training job can move power, cooling, battery response, and power electronics in patterns that look less like conventional facility load and more like part of the compute system itself. The physical plant has become coupled to the computer's execution profile.

This paper starts from the operational consequence: once those systems are coupled, their telemetry has to be correlated, retained, and queried together.

That coupling changes what operational telemetry has to do.

The plant is still operated through established OT patterns: PLCs, protection systems, BMS, PMS, CDU controllers, SCADA, message brokers, and layered networks. None of that goes away. What changes is the correlation requirement. Operators now need to ask questions that cross systems and timescales: what happened to facility load during a training phase transition, which cooling loop reacted first, which racks drove the phase imbalance, whether battery or UPS behavior aligned with GPU power movement, and how a hall-level event propagated into the campus electrical posture.

Those are not dashboard-only questions. They are data-layer questions. The system has to ingest the raw signal, preserve its context, retain it long enough to matter, and make it queryable without moving the data out of the environment where operations run.

The volume is assumed. NVIDIA DCGM exposes GPU telemetry through field identifiers and exporter paths covering clocks, power, thermals, energy, and fabric health, with third-party collector coverage describing thousands of metrics across GPU, MIG, NVLink, NVSwitch, and CPU scopes. The facility side adds liquid-cooling loop temperatures, pressures, flow, pump state, CDU status, leak detection, phase-level power, UPS and battery behavior, vibration, generator state, and grid-interface telemetry. OCP telemetry work catalogs base-building and data-hall points across utility, generation, central plant, power monitoring, cooling, environmental, and liquid-cooling systems.

The harder issue is not only volume. It is a timescale mismatch. GPU power can change in seconds or less. Electrical protection events can unfold in cycles. Cooling loops respond more slowly. Building systems often trend at minute cadence. The data layer has to preserve those cadences without flattening them into averages that erase the sequence operators need to reconstruct. Traditional facility telemetry was often sampled at minute-level cadence because the use case was monitoring, trending, and capacity planning. OCP notes that minute-level power sampling was commonly used to reduce network impact. AI workload telemetry, by contrast, is routinely collected at one-second or finer resolution, and the workload itself moves in synchronized phases. A 756-GPU academic cluster study collected 162 GB of per-second telemetry over 31 days. SemiAnalysis, citing Meta's Llama 3 infrastructure, describes tens of megawatts of instantaneous power fluctuation from synchronized GPU behavior on a 24,000-H100-class cluster with about 30 MW of IT capacity. Meta separately documented two 24,576-GPU clusters used for Llama 3 training. NERC's July 10, 2024 incident review documented approximately 1,500 MW of customer-initiated, voltage-sensitive load reduction after a 230 kV transmission fault sequence, six faults in an 82-second period.

The operational result is simple and uncomfortable: slow facility telemetry and fast workload telemetry now have to live in one data layer, with enough fidelity to be queried together. The data model has to respect OT boundaries, preserve local autonomy, and still support building, campus, and enterprise rollup. It has to fit the systems already in place rather than ask the facility to reorganize itself around a database.

That is the architecture this paper lays out: a PostgreSQL-native time-series layer, implemented with TimescaleDB, present at each Purdue scope, behind SCADA and the operational applications rather than in place of them, integrated through OT protocols, compressed and rolled up the hierarchy on premises, and optionally synchronized to a managed cloud only when policy allows.

Data-Layer Requirements

The reference architecture starts with requirements, because the failure mode is rarely a missing connector or a single slow query. It is a data layer that satisfies one requirement while violating another. For AI data center operations, the following have to hold together.

Sustained high-frequency, high-cardinality ingest. The ingest path must support continuous streams from GPUs, power chain, cooling chain, BMS, PMS, CDU, leak detection, vibration, battery, and grid-interface systems, and keep insert behavior stable as tags, devices, tenants, halls, and derived metrics grow.

Years of history online and affordable. Operators need recent high-resolution data for incident response, but capacity planning, model-based optimization, energy analysis, and failure prediction depend on long histories. Aging data cannot disappear into cold archives that need a separate restore path before they can answer a question.

Concurrent real-time and analytical access. The same operational data estate has to serve live dashboards, alarm investigation, root-cause analysis, fleet comparison, efficiency studies, and long-running analytical queries. Isolation matters, but copying data into disconnected stores for every audience defeats the point of a facility-wide operating record.

Edge-to-enterprise rollup with local autonomy. Data is born locally. A hall must keep recording if the building rollup is unavailable, and a building must keep operating if the campus link is down. Rollup should be delayed, replayed, and reconciled without changing the identity or semantics of the data.

Reliability and durability as baseline behavior. High availability, automatic failover, incremental backup, point-in-time restore, and operational visibility are not convenience features in this environment. They are part of the minimum viable design for operational infrastructure.

Deployment where the data lives. The primary target is on-premises, at the edge, in the data center, or in a customer-managed environment. Air-gapped and intermittently connected sites must be first-class designs, not exceptions.

Reference Architecture: A Time-Series Data Layer at Every Purdue Scope

The AI Data Center Telemetry Stack. A time-series data layer sits behind SCADA, HMI, MQTT, and operational applications at the hall and building scopes. Deterministic control stays local. Operational data rolls up on premises from hall to building to campus to enterprise, with optional managed-cloud synchronization shown as a separate path. Purdue layers are logical operating scopes; actual network segmentation varies by site.

The reference architecture places a time-series data layer at each operational scope of the Purdue model. Treat those scopes as logical, not as strict network boundaries. In many industrial environments Purdue Layers 2 and 3 share a subnet or operational network, so the layers describe responsibility and rollup position, not mandatory physical segmentation. This does not move control into the database. Deterministic control loops stay in PLCs, controllers, protection systems, BMS, PMS, CDU controllers, and SCADA. The database observes, stores, aggregates, and serves. It does not sit in the deterministic control path.

The key architectural choice is consistency: the same time-series technology runs at each scope, scaled to that scope's responsibility, with data rolling up the hierarchy. Rollup becomes a native data architecture instead of a chain of one-off translations between incompatible stores.

Layers 0 and 1: instrumentation, control, and local capture

Field devices and control assets produce the raw state: meters, breaker monitors, rPDUs, UPS telemetry, CDUs, leak sensors, valve positions, pump speeds, pressure transducers, flow meters, temperature and vibration sensors, PLC tags, BMCs, and GPU node telemetry. Collection happens through industrial gateways, collectors, or local agents adjacent to the control network.

Accepted ingress patterns include OPC UA for structured industrial data, Modbus and BACnet where building and equipment systems already expose them, MQTT for brokered publish and subscribe, Redfish for server and hardware management telemetry, and SNMP where power and network equipment still emit it. OCP's telemetry guidance explicitly covers MQTT, Redfish, TLS and mTLS, push and pull, publish and subscribe, Modbus, BACnet, segmentation, and secure data exchange across the OT and IT boundary.

At this layer the local database footprint can be small. Its job is short-horizon buffering, timestamp integrity, store-and-forward behavior, and protection against upstream loss. It should run on industrial PCs or local edge servers without introducing a dependency into the control loop.

Layer 2: hall operations and the hall historian

Layer 2 is the hall-level operating environment. This is where the supervisory and application systems live: SCADA and HMIs, Ignition, MQTT brokers, and the hall-level applications operators work day to day. The time-series data layer does not replace any of them. It sits behind and alongside them as the hall historian: the durable, queryable record of what the hall did.

A hall-level time-series store is the reference pattern, because it gives each hall a complete, durable local record and lets it stay available on its own. Not every deployment starts there. Some facilities run the data layer at the building scope and above and aggregate hall telemetry there, then add hall-level stores as availability requirements, complexity, and the need for local autonomy grow. The architecture supports both: the hall tier scales in where those requirements justify it, while the rollup hierarchy stays the same.

Where deployed, each hall-level time-series store ingests high-frequency telemetry for that hall, receiving it through Ignition gateways, MQTT brokers, OPC UA bridges, protocol gateways, collectors, and application connectors. SCADA remains the supervisory environment, the HMIs remain the operator interface, and MQTT remains the brokered transport where it is already used. Against that, TimescaleDB provides durable high-resolution history, SQL access, compression, continuous aggregates, and local query performance. It serves historical queries behind the local dashboards and preserves operational history during disconnection from upper scopes: if the link to building or campus is unavailable, Layer 2 keeps recording. SCADA and the HMIs remain the operator's real-time supervisory surface; the historian is what they query when the question is what happened, when, and in what order. Order matters here, and so does context: because TimescaleDB is PostgreSQL, writes land in ACID transactions, and the historian can preserve the recorded sequence with the timestamp, source, and quality context needed to reconstruct event order.

This is where high-cardinality ingest matters most. A hall can contain thousands of accelerators, dense liquid-cooling loops, high-frequency power instrumentation, and multiple local systems previously operated in separate views. The Layer 2 historian should answer immediate operator questions: what changed in this hall, which CDU loop moved first, which racks saw the phase imbalance, what the GPU power profile did, and which protection or backup systems responded.

Layers 2.5 and 3: sub-zone, building, and SCADA adjacency

Layers 2.5 and 3 aggregate across halls, rooms, mechanical zones, electrical lineups, and building systems. In practice these scopes often share an operational network with Layer 2 rather than sitting on a separate tier; the distinction is one of rollup responsibility, not necessarily of subnet. This is the integration point for SCADA, BMS, PMS, DCIM, MQTT brokers, message buses, and operational applications.

The data layer sits alongside SCADA, not in place of it. SCADA remains the supervisory interface and control environment at both the hall and building scopes. The time-series layer is the durable, queryable substrate behind real-time views, historical analysis, external reporting, and analytics.

The strongest proof point for this fit is the strategic alliance between Inductive Automation and Tiger Data, the company behind TimescaleDB, announced in April 2026 to modernize the industrial historian. Inductive Automation makes Ignition, a widely deployed industrial application and SCADA platform; TimescaleDB Enterprise (in early access; sign up to become a design partner) is an on-prem PostgreSQL-based time-series database that serves as the historian behind it, for on-premises and edge deployment. For operators already standardizing on Ignition, that makes the database an integrated historian path rather than a parallel system the operations team has to glue together alone.

Layers 4 and 5+: campus, enterprise, and cross-site analytics

At Layer 4 and Layer 5+, data rolls up for campus operations, cross-building comparison, capacity planning, reporting, fleet analytics, and AI or machine-learning workflows. This scope does not need every raw sample forever at the hottest resolution. It needs governed rollups, selected raw windows, and enough fidelity to reconstruct operational behavior when an incident crosses halls, buildings, or grid boundaries.

The rollup itself is the heart of the architecture, and it runs on continuous synchronization between scopes: hall to building, building to campus, campus to enterprise. Where supported, planned synchronization capabilities move selected raw streams, rollups, and metadata upward between TimescaleDB stores, so an upper scope holds a faithful, current view without the lower scope losing autonomy or identity. Critically, this rollup stays inside the operator's own infrastructure. It does not require the cloud.

Moving data from the on-premises hierarchy to Tiger Cloud is a separate concern. For organizations that want managed cross-site analytics, planned synchronization capabilities can provide an optional path to Tiger Cloud, where tiered storage keeps frequently queried data in a high-performance tier and moves older data to object storage. Tiger Cloud documentation describes up to 64 TB in the high-performance tier depending on plan, with a low-cost object storage tier behind it. That path is optional and cloud-bound. The on-premises hierarchy is complete without it: for sites that cannot send operational data out, the full edge-to-enterprise rollup still runs, entirely self-managed.

Operator Scenario: Reconstructing a Load Event

A training job enters a synchronized phase. GPU power draw rises across a hall, phase-level power telemetry moves, one CDU loop responds before the others, and UPS or battery telemetry shows a corresponding event. The hall historian preserves the high-resolution sequence locally. The building rollup shows whether the response crossed halls or stayed local. The campus view shows whether the event aligned with broader electrical posture. The operator does not need five disconnected exports to reconstruct the event. The data layer preserves source, timestamp, unit, quality, and lineage so the question can be asked across systems without losing the order of operations.

Why TimescaleDB Fits This Architecture

The requirements above point to a single system. The data layer has to speak SQL, keep time-series data bounded as it grows, compress years of history into affordable storage, roll up cleanly from edge to enterprise, and run reliably on hardware the operator controls. TimescaleDB is a PostgreSQL extension that does all of that in one engine.

Because it is an extension and not a fork, it keeps standard PostgreSQL clients, drivers, and SQL, and adds hypertables, continuous aggregates, compression, and retention. The data layer fits the tooling operations and analytics teams already run: drivers, BI tools, Grafana, backup tooling, access-control patterns, and the operational knowledge already in the building. There is no proprietary query language to learn and no silo that only one vendor's tools can read.

Hypertables answer the volume problem. They partition by time, so ingest, retention, compression, and queries operate over chunks instead of one ever-growing table that eventually degrades. Continuous aggregates answer the rollup requirement. They maintain incrementally refreshed views, including rollups over rollups, which maps directly onto the Purdue hierarchy: raw hall data rolls into minute, hourly, and daily views; building aggregates roll into campus views; selected metrics feed enterprise models.

Compression is what makes long online retention practical. Tiger Data's IIoT performance work reports common compression ratios of 80 to 95 percent, with a terabyte of raw data compressing to roughly 50 to 100 GB. Hypercore, the hybrid row and columnar engine behind this, lands new data in a row-oriented path for ingest and updates, then moves older data into columnar storage for compression and analytical scans. One table serves both the operator querying the last hour and the analyst scanning the last year.

On ingest, the honest claim is sustained production scale, not a record for peak rows per second on a narrow benchmark. The partitioning model exists to prevent the failure mode where a single PostgreSQL table grows until inserts and queries fall over. Direct Compress, a TimescaleDB 2.21 tech preview from September 2025, compresses data in memory during COPY ingestion and reports up to 40x faster ingestion in its benchmark scenario, with the usual caveat that schema, batching, storage, and workload shape change the result. AI data center telemetry is won by keeping ingest, SQL access, compression, rollup, retention, security, and operations in one reliable system, not by a drag race.

Reliability Where Operations Actually Run

The guarantees you were promised, on your own hardware.

Operations teams have spent years being told that critical data infrastructure should behave like a managed service: high availability, automatic failover, incremental backup, point-in-time restore, operational dashboards, controlled upgrades, and recovery that does not depend on a hero at 3 a.m. The requirement is right. The deployment assumption is wrong. Many AI data center environments cannot make a managed public cloud service the dependency of record for OT data.

TimescaleDB Enterprise is the self-managed answer to that constraint: the open-source TimescaleDB engine plus the operations layer, licensed for on-premises, edge, and customer-managed cloud, and built to run air-gapped. The operations layer is the point. High-availability clustering, automatic failover, fully incremental backups, and point-in-time recovery put managed-grade behavior on hardware the operator owns. A web-based admin console and pre-configured Grafana dashboards make that behavior visible instead of tribal.

Because the engine is the same at every scope, each layer can be operated to the criticality of its function. A Layer 2 hall historian runs locally and stays durable through an upstream outage. A Layer 3 building store runs with replicas and failover. A Layer 4 campus store aggregates across buildings without forcing every hall to depend on the campus link. Backup and restore are part of the operating model, not an export script someone has to remember to run.

Two data paths leave each site, and the distinction matters. On-premises rollup moves selected raw streams, rollups, and metadata up the hierarchy while each local system stays complete on its own. Cloud synchronization is a separate, optional path for organizations that want cross-site analytics in Tiger Cloud. For air-gapped sites, regulated sites, and facilities where policy, latency, or operational independence keeps OT data local, the rollup path runs without the cloud path ever being enabled.

Operating Model and Guardrails

The architecture should be deployed with clear boundaries.

First, keep the database out of the control loop. It can record telemetry and serve queries, but a database issue must never affect trip logic, PLC scans, protective relays, CDU control, or BMS control loops.

Second, keep the storage technology consistent across scopes. When the same time-series engine runs at the hall, the building, the campus, and the enterprise, a query written against a hall historian stays valid against a campus rollup, and identities, units, and retention policies carry upward without reinterpretation. Consistency at this level is what turns rollup from an integration project into a property of the system.

Third, keep identities stable across rollup. A tag, device, rack, CDU, UPS, hall, tenant zone, or GPU should not get a new identity at every layer. The rollup path should preserve source, timestamp, quality, unit, and lineage.

Fourth, design for replay. Local stores need bounded queues and retention windows that let them backfill upstream systems after a link outage. The system should assume disconnection, not merely tolerate it.

Fifth, separate raw, operational, and analytical views. Operators need recent raw and high-resolution data; building and campus teams need rollups; enterprise analytics needs governed datasets. Hypertables, compression policies, retention policies, and continuous aggregates provide the database-level primitives for that separation.

Finally, make reliability visible. HA state, replica lag, backup freshness, restore validation, ingest lag, compression job health, disk headroom, and query saturation belong on the operational dashboard. A database holding operational telemetry becomes part of the facility's own instrumentation.

The Operator's Goal

The data layer is not the goal. Energizing the build is the goal. Keeping the facility stable through synchronized workload behavior is the goal. Running power and cooling close to the real operating envelope, without sacrificing margin blindly, is the goal. Planning the next tranche of capacity with evidence instead of guesswork is the goal.

A time-series layer earns its place only if it serves those outcomes. It has to fit the SCADA and protocol stack already in place. It has to keep years of history online without making storage economics impossible. It has to serve real-time and analytical access without splitting the operating record. It has to roll up from edge to enterprise while letting every local layer keep running when the link drops. And it has to deliver reliability and durability on hardware the operator controls.

That is the reference architecture: TimescaleDB as the PostgreSQL-native time-series data layer, present at each Purdue scope, sitting behind SCADA and the operational applications rather than replacing them, integrated through OT protocols, compressed and rolled up the hierarchy on premises from hall to building to campus to enterprise, operated locally, and optionally synchronized to Tiger Cloud only when policy allows.

The physical plant is now part of the computer. The data layer has to be built like that is true.

Get Started

Interested in managed time-series analytics? Start a free Tiger Cloud trial.

Running on-premises, at the edge, or in an air-gapped environment? TimescaleDB Enterprise is built for those deployments and is accepting design partners.

AI's Physical Constraints: How AI Rewired the Data Center

Team Tiger Data — Thu, 02 Jul 2026 19:34:34 +0000

For most of the cloud era, a server rack was a five to twenty kilowatt object. You could fill a room with them, move air across the front, and the building stayed an ordinary building. A single current AI rack, NVIDIA's GB300 NVL72, draws about 132 to 140 kilowatts, with the GPUs alone accounting for more than a hundred. That is close to an order of magnitude more power in the same floor space as those old racks, and it lands as heat in the same small volume. Past roughly a hundred kilowatts per rack, air stops being able to carry the heat out, and the rack has to be plumbed for liquid. The compute got denser and the building changed with it.

This pattern repeats all the way out to the grid. For about fifteen years, getting more computing power felt like turning a dial. You needed more, you asked for more, and a few seconds later it was there. Spin up a hundred servers for a traffic spike, spin them back down when it passes. Capacity behaved like something continuous, instant, and reversible, a knob you turned rather than a thing you built. A generation of software was designed on that assumption, and it held, because for ordinary workloads the power and hardware involved were small against what the world could supply.

That has changed. Across AI infrastructure projects, the same moment now repeats. A team asks for more, and the answer comes back no. Not "no, that costs more," which everyone understands, but a harder no. "No, those GPUs are not available this quarter." "No, that region has no more power, and will not for years." You can order a GPU in a day; the date that a few hundred megawatts arrives at a site can be four or five years out, and no amount of money moves it sooner. The request that used to be a billing question has become a physical one.

Anyone who has designed, built, or run a data center knows the physical layer was always there. The people who design and build these facilities sized the transformers, ordered the switchgear, planned the cooling, and waited on the utility. What is new is the scale at which AI hits the physical layer. The data centers going up for AI are a different class of build: denser, hotter, hungrier, and more tightly coupled to the grid than the ones that came before.

Data centers always needed chips, memory, cooling, power, and water. Most cloud workloads before the AI surge kept those requirements in a range the existing build could absorb. AI pushes them past the thresholds where the old assumptions hold. It does not create a new kind of physics. It removes the buffer that made the physics easy to ignore.

Each of these limits has been written about on its own. What gets missed is how they connect, and why AI makes them arrive together. AI scaling moves through a physical dependency chain: more accelerators require scarce chip packaging and memory; more memory and compute concentrate heat; concentrated heat changes the rack and the building; the building then needs power the grid may take years to deliver; at that scale, the power itself may have to be buffered inside the facility; and the cooling choices made along the way determine where water becomes a problem. The limits arrive in a predictable order, and the order is the story:

GPUs. The first visible shortage is accelerators, the GPUs that do the AI computation, but the real bottleneck sits around the chip, not in it.
Memory. The accelerator depends on high-bandwidth memory, which pulls on the same finite wafer base as ordinary memory.
Cooling. More compute and memory in the same space means more heat in the same rack, past what air can carry.
Power and time. Liquid cooling moves heat, but every watt still has to come from the grid. First you wait for power to arrive; then, at AI scale, you may need to buffer the workload's own power swings.
Water. Not the national catastrophe the headlines suggest, but a local siting constraint shaped by cooling design.

Let's start with the one everyone already knows.

GPUs: The Bottleneck Is Not the Chip

The first wall everyone notices is GPUs. You cannot get them, you cannot get enough, or the price to rent them has climbed since last year. The figures are not subtle. H100 rental prices rose roughly 40 percent off their late-2025 lows in a matter of months, from about $1.70 to $2.35 per GPU-hour on one-year contracts between October 2025 and March 2026, and on-demand capacity is effectively sold out across GPU types. The pressure reaches the workstation end too. In June 2026 NVIDIA listed its RTX Pro 6000 Blackwell at $13,250, a 55 percent jump over the $8,565 launch price a year earlier, and the reason it gave was the 96 gigabytes of memory on the card in a market where memory is scarce.

The obvious reading is that NVIDIA cannot make enough chips, but that is not where the bottleneck lives. A modern AI accelerator is not one chip but a package: the processor die, stacks of high-bandwidth memory (HBM), and an interposer that wires them together at enormous bandwidth. A faster processor does not help if it cannot be packaged with memory, and advanced packaging capacity is finite, specifically the chip-on-wafer-on-substrate (CoWoS) lines at TSMC. The same analysis that tracked the rental spike named CoWoS packaging and HBM, not the processor, as the choke points. The lead times that stretch GPU orders toward a year are gated there. The chip is not the scarce thing; what surrounds it is.

So the GPU shortage is really a packaging and memory shortage wearing a GPU label. Packaging capacity can expand, but it expands on a manufacturing clock. Memory is the harder half, and the reasons it stays scarce are the next wall.

Memory: Why the Price Will Not Come Down

When a component spikes in price, the reflex is to wait. Shortages end, factories ramp, the price comes back down. That reflex is wrong here, and the reason is structural, not cyclical.

The memory AI systems need comes in two kinds. HBM is the fast, expensive memory stacked right next to the accelerator, where the model's working data lives during computation. Dynamic random-access memory (DRAM) is the ordinary system memory around it. The binding shortage is HBM, and the pressure spills into DRAM because both pull on the same finite wafer base. SK Hynix, the leading maker of HBM, locked up its 2026 HBM output well ahead of the year, and Micron has likewise reported its 2026 HBM sold out. Memory has gone from a rounding error in a machine's bill of materials to the single largest driver of the price on a high-end card.

So why not just make more? Essentially all the world's DRAM comes from three companies: Samsung and SK Hynix in South Korea, and Micron in the United States. When all three make the same allocation call at once, that is the global supply. There is no fourth maker at comparable scale waiting to undercut them.

Adding supply means building a fabrication plant, and a fab is not a factory you stand up in a quarter. It takes years of cleanroom construction, tool installation, and qualification before a single sellable chip comes out. HBM makes the squeeze worse, not better: each gigabyte of it uses about three times the wafer capacity of DDR5, today's standard volume DRAM, so every wafer redirected to the scarce thing makes the common thing scarcer still, and HBM already consumes roughly a quarter of all DRAM wafer output.

There is a second reason, and it is a choice rather than a constraint. The makers are steering wafers toward high-margin AI and enterprise memory and away from everything else, because that is where the money is. A new fab does not automatically reverse that, because the same margin logic governs what it chooses to build. IDC, the market-research firm, projects 2026 DRAM supply growth of only about 16 percent year over year, well below the 20 to 30 percent that was historically normal, even as demand for the AI variety grows far faster. The people running these companies are saying so directly. Intel's chief executive, Lip-Bu Tan, relayed in February 2026 what two of the key memory makers had told him: there is no relief until 2028.

The makers could produce more, but it takes years, only three companies do it at global scale, the AI memory eats three times the wafers, and they earn far more selling to AI than to you. Part of the shortage is physics, the supply is simply years out. Part of it is choice: the capacity that does exist is being pointed at AI, not at you. You have not been priced out for a quarter. You have been outbid.

Cooling: Why the Rack Changed Shape

GPUs and memory are still things you buy, even when you have to wait. The next wall is different. The same density that makes AI systems powerful, more transistors on the die and more memory stacked beside it, becomes heat the moment the hardware enters a building.

Every watt of power a machine draws comes back out as heat. Whatever goes in has to come out, or the machine cooks itself. It sounds too simple to matter, and it is the whole reason the rack changed shape.

For most of the history of computing, taking the heat out meant moving air. Servers in a rack, cool air through the room, and that was enough. At the rack densities of the CPU era, even at the top of the range, the airflow was manageable enough that the building stayed recognizably the same kind of building: rows of racks, cold aisles, hot aisles, chillers, and fans. Anyone who built those rooms knows the envelope.

AI did not make heat new. It made heat dense.

All the capability crammed into the die and the memory turns into heat in the same small volume. The rack densities from the opening, an order of magnitude higher than the CPU era, are really heat-removal figures: every one of those kilowatts has to be carried back out. The next generation on the roadmap, the Vera Rubin systems, is projected to push per-rack density several times higher again, and cooling vendors are already designing for the increase. At those densities, air cooling stops being practical. Bigger fans do not solve the volume problem.

Figure 1: Rack power density by generation. Past roughly 100 kilowatts, air cooling stops working and liquid becomes mandatory. The Rubin Ultra figure is a roadmap projection.

So the machine changed shape. A given volume of air can carry only so much heat away before it has to move faster than is practical, while water carries roughly three to four thousand times as much heat per unit volume as air. Past roughly a hundred kilowatts per rack, that gap stops being an efficiency question and becomes a hard limit, and the model flips: from moving air through a room to carrying heat away in liquid piped directly to the chip. The newest systems do not offer an air-cooled option at all. The GB300 NVL72 is fully liquid-cooled. The rack is no longer just electrical equipment. It now has plumbing.

Capital helps with procurement and retrofits. It does not change the thermal limits of air. This wall is geometry and thermodynamics. And it has a consequence even experienced operators feel: the hardware no longer fits in most existing buildings. A data center built for air, even one finished a couple of years ago, often cannot host these racks without being substantially rebuilt, retrofitted for liquid distribution, higher floor loading, and the plumbing that comes with it. You cannot simply drop the latest GPUs into the footprint you already have. For an operator who has spent a career optimizing airflow, that is the moment it becomes clear this is a different kind of building.

Liquid cooling changes how heat leaves the chip. It does not change where the energy comes from. Every watt still starts at the grid, and that is where the slowest constraint appears.

Power and Time: The Wall Underneath the Walls

What limits AI in the end is not chips, and it is not cooling. It is electricity, and specifically it is time. You can buy a GPU in a day. You cannot buy the specific date on which a few hundred megawatts will be delivered to a site. That is set by a physical system that moves on a timescale of years. Money can fund equipment and alternatives, but it does not make shared grid capacity appear on software time.

The building can be designed and built on one clock; the grid upgrades that let it draw full power often run on a longer one. Before a site can draw full power, the utility has to study and approve the load, the transmission system has to support it, and the substations and lines that feed it have to exist. The industry calls this time-to-power : the interval between choosing a site and being able to draw the load you planned around. For large AI sites, that interval can define the project. The upstream grid work resists money in a way the other constraints do not, because the transmission lines and substations are shared infrastructure that serves everyone on the grid, so a new load cannot simply pay to skip the queue without the wires actually being built. You can finish the building and then wait to turn it all the way on.

The time-to-power gap. You can buy a GPU in a day, but energizing a site takes years, and the build can finish while the power wait continues.

There is another clock running alongside the approval queue: the equipment itself. Even once a project is cleared to connect, the high-voltage transformers and switchgear that tie it to the grid can be in shortage. Lead times for large power transformers have stretched from roughly two years before 2020 to as long as five years now, and industry estimates suggest a meaningful share of planned 2026 data-center capacity could slip for want of power equipment and grid connections. Electrical gear is not the biggest line item in a data center. It can still decide when the building turns on, a reversal any operator who has waited on a transformer order will recognize.

The grid backlog around these projects is large. At the end of 2025, more than 2,000 gigawatts of generation and storage capacity were waiting in line to connect to the US grid, roughly twice the entire installed US power fleet. More waiting to connect than currently exists. That queue is not the same thing as a data-center load request, but it shows the condition of the shared infrastructure every large new load depends on: the wires, substations, studies, and upgrades are all moving on a multi-year clock. Lawrence Berkeley National Laboratory, which tracks those queues, finds the median time from request to commercial operation has roughly doubled, from under two years for projects built in the early 2000s to four to five years now.

This is not abstract, and it is not only an American problem. Ireland is the cleanest example. Dublin had become one of Europe's great data center hubs until the grid could not keep up, and in 2021 the grid operator EirGrid and the Commission for Regulation of Utilities imposed what amounted to a moratorium on new connections in the Dublin area. One Amazon project and two Microsoft projects were among those turned away and relocated to London, Frankfurt, and Madrid. By 2024, data centers were drawing around 21 percent of all the electricity in the country. The moratorium eased only in December 2025, and the new terms show where things are headed: a new facility now has to bring its own power generation or storage rather than simply draw from the grid.

The power industry itself is reorganizing around this demand. In May 2026, NextEra Energy announced a roughly $67 billion all-stock plan to acquire Dominion Energy, the utility behind northern Virginia's data center corridor.

That is the first half of the power story: getting electricity to the site. The second half starts once it arrives, and it is where AI looks least like the loads the grid grew up serving. A large training run is synchronized. Tens of thousands of accelerators compute, pause together to exchange results, and compute again. Power draw follows that loop. A single H100-class GPU draws far less at idle than under compute, so when tens of thousands switch states together, the facility's load can swing by tens of megawatts in seconds or less. Meta reported swings around 30 megawatts on a 24,000-GPU cluster training Llama 3.

A synchronized training cluster swings between compute and pause many times a second. The grid is built to follow the smooth aggregate of many independent users, not one correlated load moving in lockstep.

The grid was built around load diversity, where thousands of independent homes and businesses average into something smooth and predictable. A synchronized training cluster is neither diverse nor smooth, and that is the part that is new even to people who have planned power for a living. The stability problem is broader than training-loop swings: large data-center loads can also behave unexpectedly during grid disturbances. In July 2024, a transmission fault in Northern Virginia caused roughly 1,500 megawatts of data-center load to disconnect itself within 82 seconds, an event the North American Electric Reliability Corporation (NERC), which sets and enforces reliability standards for the North American bulk power system, said the system had never seen at that magnitude.

The fix moves on-site. xAI's Colossus cluster in Memphis installed about 150 megawatts of grid-scale battery storage alongside its power infrastructure. The point of that storage is not how much energy it holds but how fast it can absorb and deliver power. A small, fast store placed in front of a slower supply is a cache. Here the slow backing store is the grid, and the fast store is local batteries and power electronics. Batteries are no longer only backup equipment. In these designs, they can become part of workload control. At AI scale, power stops being merely an input to the computer. It becomes part of the computer's design.

And a design has to be operated. Once batteries, power electronics, cooling loops, and GPUs act as a single system, someone has to watch them as one: how power draw tracks compute, how the batteries answer a training swing, how heat follows the load. Those measurements arrive every second, from equipment that used to belong to three different teams. Read after the fact, they tell you what broke. Read live, they keep the loop stable.

Power is where the earlier constraints converge. The GPU you could not get, the region that was full, the building that needed rebuilding, the batteries now sitting between the workload and the grid: each one traces back to the same place, a power system that has to be built and buffered, on the grid's schedule, not the software team's.

Water: In Proportion

Power is the hardest wall because it sets the clock and, through on-site batteries and power electronics, becomes part of the machine's own design. Water is different. It sits downstream of cooling design and geography, which makes it more local, more variable, and more solvable than the public debate suggests. AI makes the siting choice more visible because the facilities are larger and denser, but the water problem still depends on design. That matters because water draws the most public attention of any of these constraints, and some of the least accurate reporting.

One distinction matters before any number makes sense: water withdrawn is not water consumed. Withdrawal is what a facility takes in; consumption is what it uses up, mostly through evaporation. A facility can withdraw a large volume and return most of it, or consume nearly all of what it takes, depending entirely on the cooling design.

Nationally, the figure is modest. As of 2021, all US data centers combined accounted for roughly 449 million gallons of water a day, about three to four tenths of one percent of total US water withdrawals, far below agriculture or power generation. The headline framing of data centers draining the country's water is not supported by the national figures.

The real issue is local. Roughly 40 percent of US data centers sit in areas of high or extreme water stress, so even a small national share can land hard on a particular community. Stated that way, it is a siting problem, real and solvable, rather than an indictment of the technology.

The cooling design is what sets the consumption. On-site consumption ranges from nearly nothing, for an air-cooled or closed-loop facility, to as much as 70 to 80 percent of what was withdrawn, for an open evaporative one. A single large evaporative facility can use something like five million gallons a day, comparable to a town of fifty thousand people. The same facility, built closed-loop, can use almost none. The high number and the low number describe the same building with two different cooling choices.

This is the constraint the industry is most actively engineering away. Closed-loop systems fill once and recirculate rather than evaporate. The same shift to liquid and direct-to-chip cooling described earlier can cut water needs dramatically, by up to 95 percent in some designs, and immersion cooling can eliminate evaporative water use altogether. Reclaimed wastewater is increasingly used in place of drinking water. Of the five walls in this piece, water is the one where the engineering response is furthest along, which is exactly why it deserves to be described accurately rather than dramatically.

The Reserve Ran Out

The five walls are not five problems. They are one fact seen from five angles. None of this is new physics: the power was always physical, the heat was always real, capacity always took years to build. What changed is that the cloud era ran on a deep reserve of capacity built ahead of demand, and as long as that reserve lasted, the limits underneath stayed out of view. You turned a dial and the reserve answered. AI has drawn that reserve down, and at a scale the old infrastructure was never built to carry, so the limits are back in view all at once.

That is why the data centers rising for AI are a different class of build, and why the people who built the last generation look at the numbers and recognize that the rules they worked under have moved. The accelerator depends on packaging and memory, the rack depends on liquid cooling, and the building depends on power. At this scale, the power itself needs a buffer. The site depends on grid capacity, water choices, and time. The next time a capacity question lands on your desk, ask where it will physically live and how long the power takes, before you ask what it costs. The cloud used to be an abstraction. It has an address now.

The five walls are physical. Operating inside them is not. Once the facility and the computer are one coupled system, running it means reading it as one: GPU power draw, cooling response, battery state, and grid posture, measured together and fast enough to act while the numbers are still true. That is not a facilities dashboard on a five-minute refresh. It is a live, correlated, high-frequency record of a machine that now runs from the silicon to the substation. Capturing that record, and querying it before it goes stale, is its own problem.

Get Started

Operational telemetry only helps if you can query it while it is still true, at the rate it arrives. That is the workload Tiger Data is built for: time-series and event data on Postgres, fresh and correct, without splitting into a second system. Start a free Tiger Cloud trial. Running on-premises, at the edge, or air-gapped? TimescaleDB Enterprise is built for those deployments and is taking design partners.

What We Heard: Three Patterns from Spring 2026 Events

Team Tiger Data — Wed, 01 Jul 2026 16:21:19 +0000

Spring 2026 was relentless. Tiger Data teams covered ten events across Europe and the US: GrafanaCON in Barcelona, Hannover Messe and AWS Summit Hamburg in Germany, AWS Summit London, ETHDenver in Colorado, the Offshore Technology Conference in Houston, Data Driven Oil & Gas in Texas, Sensors Converge and IOT Tech Expo in the Bay Area, and AWS Summit Los Angeles. Hundreds of conversations with platform engineers, SREs, industrial automation teams, oil and gas operators, blockchain infrastructure builders, and sensor hardware teams. That's a lot of badge scans.

Three patterns kept showing up across every audience, and they're really different angles on the same shift: the market has moved past category education and into implementation-level scrutiny. Here's what that looked like on the ground.

Pattern 1: Postgres is becoming the default for industrial and operational telemetry

Five years ago, telemetry workloads at scale defaulted to purpose-built time-series databases. InfluxDB. Prometheus. Specialized vertical stacks. The conversation at the booth has changed.

At Hannover Messe, the world's largest industrial trade show, the most common question from automation and manufacturing teams was some variant of: "we have tens of thousands of sensors writing constantly, our current system is buckling, what do you do differently." Not "should we use a time-series database." Not "what is TimescaleDB." Those teams had already chosen Postgres before they walked up. They wanted to know if Tiger Data could keep them on it at the volumes they were already running.

The Tiger Data team at Hannover Messe, Europe's largest industrial trade show.

At Sensors Converge, the audience skews closer to the hardware end of the IIoT pipeline: sensor manufacturers, embedded systems engineers, and the people writing firmware for the devices feeding everyone else's databases. The questions were specific. How does storage behave when you're sampling at 100 Hz across 50,000 endpoints. What happens when a fleet of devices comes back online after a network outage and dumps a backlog. Workload questions, not category questions.

At the Offshore Technology Conference in Houston, the energy sector followed the same shape. SCADA telemetry. Well monitoring. Equipment digital twins. Lifecycle analytics on multi-decade asset histories. Teams that already had Postgres in production wanted to know how far they could push it before having to reach for another system. The answer, increasingly, is further than they had assumed.

The pull toward Postgres in these markets is consistent. Existing Postgres in production. Existing SQL skills on the team. No appetite for a second on-call rotation. Industrial software is conservative for good reasons, and the system that already runs the business is a much better starting point than a green-field rewrite. Customers like Mechademy, Flogistix, and Axpo are public references for what this looks like in production.

Pattern 2: Split-architecture fatigue is becoming an explicit conversation

The second pattern builds directly on the first. Teams who split their architecture into "Postgres for transactions, ClickHouse or a warehouse for analytics" are increasingly explicit about regretting it, something we have been arguing internally for years and are now hearing back from the field.

At AWS Summit London, the booth was dense with platform engineers running on RDS or Aurora and feeling the analytical wall closing in. The question was usually some version of: "we are about to add a separate analytics database. Is there a path that does not require us to do that." A few years ago, the answer most teams accepted was no. They added the second system, built the pipeline, accepted the lag, and moved on. The conversation now starts from a different premise. Teams have either watched a peer team go through the split and pay the operational tax, or they have done it themselves and want out.

At Hannover and OTC, the same pattern appeared on a longer timeline. Industrial teams that built their architectures a decade ago, with operational data in one place and analytics in another, are increasingly looking at consolidation. Multi-system data plumbing is one of the largest hidden line items in their engineering budget. The pitch that operational analytics can stay on the source of truth, with no pipeline, no drift, no second backup strategy, is landing differently than it did even two years ago.

Speedcast's story is the cleanest public version of the before-and-after. They stitched together Kafka, Flink, and custom code to stream data from Postgres to Iceberg, and Tiger Lake replaced all of it. As Kevin Otten, their Director of Technical Architecture, put it: "It's the architecture we wish we had from day one."

Pattern 3: The questions have gotten harder, and that's the clearest evidence of the first two

Across all ten events, the conversations got deeper than at any event series we have run. The questions skipped the basics, and that alone says something: teams don't ask about compression ratios and failure modes when they're still deciding on a category. They ask once they've already made the decisions in Patterns 1 and 2, and they're just checking the implementation.

At GrafanaCON, booth visitors asked about continuous aggregates, compression ratios, and how Hypercore behaves under high-cardinality dashboards. These are the questions teams ask when they are comparing implementations. The Grafana community is one of the most technically opinionated audiences we talk to, and the depth of the questions has been climbing event over event.

At Sensors Converge, the questions ran the same way at the hardware level. Specific sample rates. Specific endpoint counts. Specific failure modes. Nobody wanted a feature tour. They wanted to know whether the system would behave the way they expected under their workload.

The share of conversations that started with "what is a time-series database" or "what is TimescaleDB" was lower than at any event series we have run. The market has moved past category education. People know what they need, and they are picking implementations.

That's the throughline across all three patterns: teams have already decided Postgres can carry this weight, they're done paying the split-architecture tax, and now they're evaluating implementations instead of categories. That's a good problem for a database company to have.

Where you'll find us next

The calendar doesn't really do summer breaks. If you want to keep the conversation going, here's where we'll be through the end of the year:

Online Workshop: From MQTT to Dashboard: Real-Time IIoT Pipelines with Tiger Cloud, July 22.
IMTS, Chicago, September 14-19.
Ignition Community Conference, Sacramento, September 22-24.
ISA Automation Summit & Expo, Florida, September 27-29.
Enlit Europe, Vienna, November 10-12.

More events coming

We're still locking in the rest of the year. Keep an eye out on our events page, or subscribe to the newsletter if you'd rather have updates come to you.

Want to connect at an event not on this list? Reach out.

If we are not coming to your city, the Tiger Cloud trial is open whenever you are.

How Float Runs an AI Energy Company on a 3-Person Team with Tiger Data

Team Tiger Data — Tue, 30 Jun 2026 11:00:05 +0000

Danish startup achieves 99.3% compression on 1Hz smart meter data, powering real-time appliance-level energy analytics for hundreds of homes.

Float is a Danish AI and energy startup that collects 1Hz smart meter data from hundreds of homes and disaggregates it into per-appliance consumption using a proprietary ML model. The entire system depends on one architectural bet: that compression on their time-series database would be high enough to make the storage economics work at scale. Co-founders Jens Brandt Nellegaard (CEO) and Victor Grabow (CTO) share how they evaluated every major time-series database, why 90% compression was the hard floor for viability, and what happened when they hit 99.3% on Tiger Data.

About Float

Most people have no idea what their individual appliances cost to run. In Denmark, most energy providers send a monthly PDF bill, and that is the entire customer experience. The market has seen no meaningful innovation in 10 to 15 years. Fraud and a lack of transparency have been so widespread that the Danish government introduced new consumer protection regulation effective January 2026.

The underlying data to solve this problem actually exists. Most European smart meters have a standardized consumer interface that can output total household electrical load down to one-second resolution. But total load is not very interesting on its own. What consumers need is a breakdown by individual appliance - to understand which one is wasting electricity, which one is running at peak price, which one is approaching the capacity limit of a fuse. Academic research on energy disaggregation goes back to 1992. Nobody had solved it in a commercially scalable way.

Float built three things to close this gap: a proprietary hardware module that plugs into the consumer port on a European smart meter, a signal processing and neural net pipeline that classifies appliance-level consumption from the raw waveform, and a consumer-facing app with a proactive AI energy agent. The system collects roughly 15 measurements per second per household, each at 1Hz resolution. One second is the hard floor - Victor explains: "At one-minute resolution, the model would break, because what we track are the changes. If there are too many appliances turning on within the same minute, it would be very hard to differentiate them."

Jens Brandt Nellegaard and Victor Grabow co-founded Float in 2022. After nearly three years of R&D, they achieved a proof of concept in December 2024, secured their energy provider license in December 2025, and are now rolling out a private beta to 350 pre-vetted customers. The company has just three people.

The Challenge

Float started on Azure managed Postgres with the TimescaleDB extension. The team had Azure credits early, so it made sense at the time. But the Apache version available on Azure did not include compression, and that turned out to be a dealbreaker.

Every customer generates roughly 15 measurements per second. Float samples voltage, frequency, and total load across each phase entering the home. At 1,000 homes, that is 15,000 data points per second, continuously. Without compression, the storage cost alone would break their business model. “We are an energy company with a flat-rate subscription fee,” says Jens. “We pass through the spot market price one-to-one with no markup. If storage cost per user exceeds what the subscription supports, the economics collapse.”

We also tried InfluxDB before settling on TimescaleDB. We ran into ingestion issues, and we needed SQL. When you are a three-person team building an asset-centric microservice platform, you cannot afford a database that requires a proprietary query language and limits how you join and query data across domains. - Victor Grabow, CTO, Float

On top of the storage problem, Float needed continuous aggregates. The Danish DSO delivers settlement data at 15-minute resolution. Float collects data every second. To generate a live energy bill and compare it against the grid operator's numbers, the system needs to aggregate raw 1Hz data down to 15-minute windows constantly. On managed Postgres without TimescaleDB's full feature set, that meant writing and maintaining batch jobs - more infrastructure overhead for a team that was already stretched thin across hardware, ML, and a licensed energy company.

Why Tiger Data

We tested pretty much every time-series database on the market. We think Tiger Data is the best solution for our use case. - Victor Grabow, CTO, Float

The team researched time-series solutions extensively and discovered TimescaleDB through the compression and continuous aggregate features. The compression was compelling enough that it was just a matter of time before they needed the full capability. When they found Tiger Data - the company behind TimescaleDB - the managed cloud service made the path clear.

Two features drove the decision. First, Compression. The team had modeled the unit economics and needed at least 90% compression on the time-series data for the subscription model to work. Anything below that and storage costs per user would exceed what the flat-rate fee could support. Second, Continuous Aggregates - materialized views that update incrementally as new data arrives. Float runs aggregations constantly, converting 1Hz readings to 15-minute settlement windows, calculating threshold-based alerts on voltage and frequency, detecting outages, and triggering duration-based notifications like flagging an oven that has been running for four hours. Continuous aggregates handle all of this without batch jobs or scheduled pipelines.

We chose Tiger Cloud, the fully managed service on Azure, because it was a question of speed. We needed to get up and running fast and offload infrastructure management entirely. Encore, our DevOps platform, provides ephemeral environments on Google Cloud, and Tiger Cloud's database branching fits naturally into that workflow. - Victor Grabow, CTO, Float

The Float Energy Data Stack

Data starts at the smart meter. Float’s IoT module plugs into the standardized consumer interface port and captures voltage, frequency, and total load across each phase at 1Hz resolution. The module sends readings to Azure IoT Hub, which the team kept from the original Azure setup as a stable ingestion endpoint for all devices.

From there, a bridge connector forwards the stream into Google Cloud, where Encore deploys Float’s microservices. The team moved off Azure Event Hub eventually because it was expensive. Google Cloud streaming services handle the same throughput at lower cost. The streaming layer batches incoming measurements from all households every second and inserts them per batch into Tiger Data.

Tiger Data stores the raw 1Hz time-series readings and runs continuous aggregates for threshold-based monitoring: voltage spikes, frequency changes, mean and max calculations, outage detection, and duration-based appliance alerts. All raw data is retained for ML training purposes through the private beta phase, with tiered storage planned as the fleet scales.

The Float app reads processed data to show customers their real-time energy breakdown per appliance. New customers see total wattage immediately on connection. Appliance-level breakdown takes roughly three to four weeks as the model trains on their home's specific patterns. The agentic orchestration layer on top handles billing, onboarding, customer service, and proactive notifications - flagging forgotten ovens and irons, inefficient appliances, and dangerous load conditions approaching fuse limits.

Float's data architecture: 1Hz readings flow from the IoT module through Azure IoT Hub and Google Cloud streaming into Tiger Data, which serves the ML pipeline, consumer app, and agentic platform.

What Compression Enabled

On Tiger Data, Float is seeing 99.3% compression on its time-series data. Victor puts it directly:

Compression needed to be in the high nineties range to not break our business model. So that was a great outcome. - Victor Grabow, CTO, Float

That number unlocked three things that would not have been possible at lower compression ratios.

The Business Model Works

At 15,000 data points per second across 1,000 homes, uncompressed storage would generate terabytes of raw time-series data per year. Float passes through the spot market electricity price to customers at cost with no markup. Revenue comes from a flat-rate subscription fee. If storage cost per user climbs above what that fee can support, the entire model collapses. At 99.3% compression, it does not. The subscription covers infrastructure with margin to spare, and that margin holds as the fleet scales.

Full Data Retention for ML Training

Float's disaggregation model needs weeks of 1Hz training data per household to learn each home's specific appliance signatures. At lower compression, the team would face a choice: retain all raw data for model training or keep storage costs viable. At 99.3%, they retain everything. All raw 1Hz readings from the entire private beta fleet are available for the ML pipeline, with tiered storage planned only as the fleet scales past the beta phase.

Real-Time Billing Without Batch Infrastructure

The Danish grid operates on 15-minute settlement windows. Float collects data every second. Continuous aggregates bridge that gap, converting 1Hz readings into the 15-minute intervals the DSO requires for bill reconciliation. Danish energy prices swing up to 80% between peak and off-peak hours, which makes the freshness of the aggregation directly valuable to customers. Because continuous aggregates update incrementally as new data arrives, Float's live energy bill is always current, i.e., no scheduled batch jobs, no pipeline maintenance, no lag.

A Three-Person Team Running a Licensed Energy Company

Float holds an energy provider license in Denmark. That means billing, customer service, onboarding, regulatory compliance - operational overhead that traditional energy companies staff with dozens of people. Tiger Cloud's managed infrastructure is part of what makes this possible. The team does not manage database operations, storage provisioning, or aggregation pipelines. That overhead is handled. When asked about team size, Jens's answer was simple: "Three. We have three people... and an army of agents. This is the future."

Looking Ahead

Float is targeting 1,000 additional private beta customers within the next 12 months, with a seed round, two additional hardware variants for full Danish grid coverage, and expansion into two more countries.

The next major integration is EV charging - starting with Tesla's telemetry API, enabling smart charging during cheap price windows. The bigger thesis is that a fleet of homes measured at 1Hz resolution can trade power on the spot market more efficiently than any energy company operating at 15-minute resolution with a 24 to 48-hour delay. As Jens puts it: "Ultimately we are trying to make the home not a burden for the grid, but a partner of the grid."

The architecture decision that compounds as Float scales is not the compression ratio itself. It is that everything runs on a single Tiger Data instance: the raw 1Hz readings, the continuous aggregates for billing, the training data for the ML pipeline, the anomaly detection queries. No split architecture to maintain, no query paths to reconcile as the fleet grows from 350 homes to 1,000 and beyond. The data model does not change - it just gets bigger.

How Small Postgres Metadata Tables Quietly Throttle Your Largest Queries

Team Tiger Data — Mon, 29 Jun 2026 13:34:12 +0000

Most time-series performance work starts with the obvious suspect: the fact table. Five hundred million rows of sensor readings, hundreds of daily partitions, an autovacuum worker fighting write load. That is where engineers go first. That is where the Optimization Treadmill lives. But after reading this guide, you'll know how to find a different class of slowdown hiding in the opposite direction, detect it in under five minutes with EXPLAIN, and apply a same-day fix that requires no schema migration.

A 10,000-row device metadata table, rarely updated, sitting in the FROM clause of every dashboard query you run, can drag your entire time-series query path into the wrong performance tier. The time-series scan looks fast in EXPLAIN. The join node is where the time goes. The fix requires looking at a table that almost nobody thinks to audit.

Why it matters

Postgres triggers ANALYZE on a table based on autovacuum_analyze_scale_factor, which defaults to 0.2. That means 20% of the table's rows need to change before autovacuum schedules a statistics refresh. On a 10,000-row devices table, that threshold is 2,000 rows. If your device metadata only sees occasional inserts as you onboard new sensors or record location changes, ANALYZE might have run once in the last quarter.

Meanwhile, pg_statistic still describes the distribution of device_id values your planner saw six months ago. Your 10,000-row table looks like a 600-row table to the query planner. Every query joining it runs a plan built for a much smaller dataset, and that plan doesn't improve on its own.

The symptom compounds constantly. Every dashboard refresh, every alert evaluation, every hourly export job runs this join. Stale statistics on a small metadata table are not a one-time cost. They tax your entire query workload invisibly, on every execution. The decision about how to execute that join is made by the query planner once, based on what pg_statistic describes at plan time.

How Postgres picks a join strategy

When the planner receives a query with a join, it chooses between execution strategies based on estimated row counts from pg_statistic. A nested loop join works well when the inner table is small and the join condition hits an index. A hash join builds an in-memory hash table and scans both relations once.

When your devices table has 10,000 rows but pg_statistic thinks it has 600, the planner's cost model produces the wrong answer. It may choose a nested loop when the dataset is large enough to warrant a hash join, or allocate an undersized hash table that spills to disk. The direction of the error matters less than the consequence: a fast indexed scan on your fact table becomes the cheaper half of a misplanned join, and the join node pays the penalty.

This pushes your query from the indexed access tier, where performance scales with log(n), into the linear tier, where it scales with the full dataset the planner miscalculated. A join that ran in 20ms when the planner had accurate statistics can degrade to 4 seconds as the mismatch grows, with no change to the fact table, the indexes, or the query itself. The IIoT performance envelope maps this degradation curve in detail.

Finding the problem

Run EXPLAIN (ANALYZE, BUFFERS) on any query that joins your fact table to a metadata table:

Run EXPLAIN (ANALYZE, BUFFERS) on any query that joins your fact table to a metadata table:

EXPLAIN (ANALYZE, BUFFERS)
SELECT
    r.ts,
    r.value,
    d.location
FROM readings r
JOIN devices d ON r.device_id = d.id
WHERE r.ts > now() - interval '1 hour'
  AND d.facility = 'plant-a';

Look at the join node in the output. Postgres shows two numbers for every node: rows=X (estimated) and actual rows=Y. A ratio of actual to estimated above 5:1 in either direction generally indicates the planner is working with stale information.

A real mismatch looks like this:

Hash Join (cost=45.00..12890.43 rows=620 width=48)
           (actual time=0.821..2847.332 rows=9841 loops=1)
  -> Seq Scan on devices (cost=0.00..42.20 rows=620 width=24)
                           (actual time=0.015..0.441 rows=9841 loops=1)

The planner expected 620 rows from devices. It got 9,841. That gap is why the hash table bucketing is wrong, why memory allocation is undersized, and why a join that should take 150ms ran for 2.8 seconds. The fact table scan was fast. The join node absorbed all the cost.

Fix 1: Force fresher statistics

The fastest fix lowers the scale factor on the metadata table so ANALYZE fires more aggressively:

ALTER TABLE devices SET (
    autovacuum_analyze_scale_factor = 0.01,
    autovacuum_analyze_threshold = 50
);

With these settings, ANALYZE fires after roughly 150 rows change rather than 2,000. For a table that sees occasional updates as you onboard new sensors, this keeps statistics current without adding meaningful autovacuum load. The devices table is not your hot write path.

Run a manual ANALYZE immediately to rebuild statistics from current data:

ANALYZE devices;

Then re-run the EXPLAIN query. The estimated rows on the join node should now match actual rows closely, and the join strategy will likely change.

Fix 2: Audit all small metadata tables

The same problem applies to any metadata table that grows slowly but gets joined to your large fact tables (e.g. a facilities table, an equipment_types table, a tag_groups table). This query finds the candidates:

SELECT
    schemaname,
    relname,
    n_live_tup AS estimated_rows,
    last_analyze,
    last_autoanalyze
FROM pg_stat_user_tables
WHERE n_live_tup < 100000
  AND (
      last_analyze < now() - interval '7 days'
      OR last_analyze IS NULL
  )
ORDER BY n_live_tup DESC;

Any metadata table with more than a week between ANALYZE runs and fewer than 100,000 rows is worth checking. Identify which ones appear in joins to large fact tables, then apply the scale factor adjustment to each. This is a one-time change with no ongoing maintenance burden.

Fix 3: Denormalize for the highest-volume joins

When a join runs on every query in your workload, the most durable fix is to remove it entirely. For stable metadata like location or facility, you can store the value directly on the readings row:

-- Add the column to the fact table
ALTER TABLE readings ADD COLUMN location TEXT;

-- Backfill from the current join
UPDATE readings r
SET location = d.location
FROM devices d
WHERE r.device_id = d.id;

The original query then becomes:

SELECT ts, value, location
FROM readings
WHERE ts > now() - interval '1 hour'
  AND location = 'plant-a';

No join. No statistics dependency. No planning variability. The tradeoff is keeping location current when a device moves. The simplest pattern is a trigger on devices:

CREATE OR REPLACE FUNCTION sync_device_location()
RETURNS TRIGGER AS $$
BEGIN
    UPDATE readings
    SET location = NEW.location
    WHERE device_id = NEW.id;
    RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER devices_location_sync
AFTER UPDATE OF location ON devices
FOR EACH ROW
WHEN (OLD.location IS DISTINCT FROM NEW.location)
EXECUTE FUNCTION sync_device_location();

This works well when device location changes are infrequent. If readings has hundreds of millions of rows and devices move regularly, an application-layer update run as a background job is safer: it keeps the writes out of the transaction that updates devices and avoids locking large row sets. A chunked loop limits I/O pressure:

DO $$
DECLARE
    part TEXT;
    rows_updated INT;
BEGIN
    FOR part IN
        SELECT child.relname
        FROM pg_inherits
        JOIN pg_class child ON pg_inherits.inhrelid = child.oid
        JOIN pg_class parent ON pg_inherits.inhparent = parent.oid
        WHERE parent.relname = 'readings'
        ORDER BY child.relname
    LOOP
        EXECUTE format(
            'UPDATE %I r
             SET location = d.location
             FROM devices d
             WHERE r.device_id = d.id
               AND r.location IS DISTINCT FROM d.location',
            part
        );
        GET DIAGNOSTICS rows_updated = ROW_COUNT;
        COMMIT;
        PERFORM pg_sleep(0.1);
    END LOOP;
END $$;

The IS DISTINCT FROM check skips rows where the value already matches, so the loop terminates cleanly even if it runs across a restart. Either way, the application code that previously ran the join query gets simpler, not more complex.

Validating the fix

After applying Fix 1 or Fix 3, re-run the original EXPLAIN (ANALYZE, BUFFERS). A correctly-planned query shows estimated rows close to actual on the join node and a shorter total execution time. Here is what the output looks like after ANALYZE with corrected statistics:

Hash Join (cost=178.00..13210.55 rows=9930 width=48)
           (actual time=1.243..312.881 rows=9841 loops=1)
  -> Seq Scan on devices (cost=0.00..172.30 rows=9930 width=24)
                           (actual time=0.018..0.892 rows=9841 loops=1)

The estimated rows now match actual rows closely (9,930 vs. 9,841). Total execution time dropped from 2.8 seconds to 312ms. The hash table was allocated correctly from the start.

For a before-and-after comparison across your full query workload, use pg_stat_statements:

SELECT
    query,
    calls,
    mean_exec_time,
    total_exec_time / 1000 AS total_sec
FROM pg_stat_statements
WHERE query ILIKE '%devices%'
  AND query ILIKE '%readings%'
ORDER BY mean_exec_time DESC
LIMIT 5;

Run this before the fix, then again 24 hours later once autovacuum has refreshed statistics and the query plan cache has turned over. The mean execution time change for your slowest join queries is your signal.

Your next step

Open psql and run the audit query from Fix 2. It takes under a minute, requires no permissions beyond SELECT on pg_stat_user_tables, and identifies every underanalyzed metadata table in your schema at once. For each table that comes back, check whether it joins to a large fact table in your workload, then apply the scale factor fix from Fix 1. No maintenance window, no schema migration. The new statistics take effect the moment ANALYZE completes.

If the audit returns nothing but your join queries are still slow, the bottleneck has moved. The Optimization Treadmill maps the next layer of performance constraints in high-volume Postgres workloads and what to do when the metadata tables aren't the problem. Start a Tiger Data free trial today to validate these fixes on your own data.

What You’re Really Owning When You Self-Host TimescaleDB

Team Tiger Data — Thu, 25 Jun 2026 13:51:54 +0000

Written by Matty Stratton, Brandon Purcell, Noah Hein, Hien Phan

Why operating TimescaleDB for mission-critical applications becomes a sustained platform engineering investment. Written by the team that builds TimescaleDB and operates Tiger Cloud across thousands of production deployments.

Abstract

Most engineers who evaluate TimescaleDB believe they are making a database decision. By the time customers depend on the application in production, they discover they made a platform ownership decision. The gap between those two decisions is the subject of this paper.

Availability, recoverability, scalability, security, and lifecycle management originate in the requirements of the application, and they land on the database team as operational systems that must be designed, built, staffed, and maintained for as long as the application runs. None of that work is beyond a capable engineering team. The question this paper raises is whether owning that platform is the highest-leverage use of the platform engineers who could otherwise be building the product the database supports.

Analytics on Live Operational Data Is Part of the Application

Analytics that runs overnight against a reporting database is important, and it is separate from the application. Analytics that plant operators watch continuously, that customers open before starting their workday, or that triggers an automated intervention before a defect is manufactured has moved onto the critical path of the business. Once analytics moves onto the critical path, the database supporting it moves there too.

One deployment carries most of this paper. An automotive supplier operates 120 robotic welding lines ingesting billions of sensor measurements per day. Plant engineers use the platform to detect abnormal operating patterns before they become equipment failures. A welding line showing early signs of a calibration drift or a sensor trending out of range may be a maintenance issue today; missed for thirty minutes, it can become an unplanned shutdown that halts production and costs the business more in rework than the platform team's quarterly budget. Every minute of latency is a minute a problem propagates across a line. We will follow this platform from its first twenty instrumented lines through its third year of operation, because the operational story of a self-hosted deployment is a story that unfolds over years, and it is easier to understand through one platform than across a survey of many. Where the same failure arrives from a different direction, we will bring in other deployments we have worked with: a connected-equipment OEM, a food processor, a specialty chemicals manufacturer. Different industries, different latency tolerances, the same dynamic underneath.

TimescaleDB is still TimescaleDB and PostgreSQL is still PostgreSQL. What changes is the operational commitment required to deliver the availability, recovery, scalability, and governance the application now demands.

The Database Inherits the Application's Requirements

Nobody is thinking about the database when the automotive supplier's operations organization commits to detecting equipment degradation before it stops a line, when plant management sets uptime expectations for the dashboards shift supervisors watch, or when compliance defines how long production records backing warranty claims must be retained. The database team inherits the requirements anyway and the inheritance changes the nature of the work.

A 99.9% availability target becomes a systems problem: replicas, failover orchestration, monitoring, and on-call coverage. A point-in-time recovery requirement becomes a tested runbook with validated restore times at production volume. Years of telemetry under compliance obligations becomes legal exposure that accumulates with every year of data the team holds.

Early in an application's life these requirements are loose. An outage of a few minutes is an inconvenience; a day of missing data is embarrassing but recoverable. As more users depend on the application and other teams build their own workflows on its data, the requirements tighten. Closing that gap is continuous work, and it falls on your platform engineers for as long as the application runs. The rest of this paper is what that work actually looks like, and each section raises the stakes a step: from lost engineer hours, to business risk, to commercial liability, to legal exposure.

Downtime Becomes a Product Problem Before the Architecture Catches Up

During the automotive supplier's early deployment, with twenty welding lines instrumented and a handful of engineers as the only users, a database restart is an inconvenience. Dashboards go blank for two minutes. Engineers wait. Nobody escalates.

Twelve months later, all 120 lines are instrumented and shift supervisors in three facilities depend on those dashboards. An equipment problem that goes undetected for thirty minutes because the platform was down costs more in rework than the platform team's quarterly budget. The database has the same configuration it had twelve months ago. The application does not.

The response is redundancy. PostgreSQL streaming replication keeps a replica synchronized with the primary, and if the primary fails, the replica is promoted with minimal data loss. This is well-understood architecture and it works. It is also where the engineering commitment begins.

Figure 1: Single-instance deployment versus production HA architecture: primary, HA replica, read replica, connection pooler, and monitoring as distinct operational layers, with failure paths and failover direction

Replication configuration is a decision with real trade-offs: synchronous replication eliminates lag but constrains ingest throughput; asynchronous replication preserves throughput but can lose data in a failover. Someone who understands the application has to make that call and own it as the workload changes. More importantly, someone has to watch replication continuously. A replica that has quietly fallen minutes behind provides a false sense of safety: stale as a read source, lossy as a failover target. Silent replication drift is one of the most common ways HA architectures fail to deliver the guarantees they appear to provide. We have seen it on deployments that had every structural component in place. The monitoring was the missing piece.

Beyond replication, HA is a stack of components that each need owners: failover automation that has been tested under simulated failure, not just configured; a connection pooler that shortens the error window and brings its own failure modes; rolling maintenance procedures rehearsed before they're needed under time pressure; and a monitoring layer that covers replication health, failover state, and background jobs, with alert thresholds tuned to the application and runbooks kept current as the system changes.

Availability is a standing allocation: a senior engineer's judgment on replication and failover, recurring hours for monitoring and rehearsal, and a permanent slot in the on-call rotation. Every one of those hours comes from the same platform engineers the roadmap is counting on. The question is whether it is the work you hired them to do.

Recovery Objectives Are Set by the Business and Tested by Almost Nobody

Figure 2: Recovery architecture showing full weekly backups, daily incrementals, continuous WAL archiving, and the RPO/RTO envelope they define, with restoration time as a function of data volume

Most backups are running. The question organizations rarely ask before they need to is whether the restore completes within the window the business requires, from the point in time the business requires, at the volume the database has actually reached. That gap is where recovery risk lives, and the food processor is standing in it the day a customer reports a potential contamination event.

The investigation needs sensor records for one production line during a two-hour window eighteen months ago. The data was backed up. None of that answers the only question that matters: whether restoring eighteen months of production telemetry at current volumes completes inside the investigation's clock. Nobody has ever run that restore. The procedure that exists is a hypothesis, and the incident is the wrong time to run the experiment.

That is the ownership gap. Not the backup. The untested restore.

As data volume grows, restore drills become harder, slower, and more important. A backup that exists is not the same as a recovery process that works.

The food processor's contamination event is the urgent version of this failure; the chemical manufacturer's corrupted migration is the irreversible one. A restore that completes is not necessarily a restore that worked. Validating that distinction at production scale is a recurring drill, measured in engineer-days per quarter, and someone has to own the calendar invite. That is before the platform has grown. The next problem is a system that succeeds.

Success Rewrites the Architecture

Figure 3: The same deployment scaling from hundreds to tens of thousands of assets: storage growth, backup window duration, query p95 latency, and concurrent connections as separate axes over time

The deployment that shows what success costs is an industrial equipment OEM whose customer-facing dashboards are a contracted product feature. A database outage there is a commercial event, logged against an SLA and escalated to an account manager. The OEM instructed a few hundred connected assets at launch and reached tens of thousands two years later. It did not build a system that broke. It built a system that succeeded, and the success invalidated the original architecture one assumption at a time. Nothing failed. The application simply outran the operational decisions made around it.

Storage growth is the most visible dimension and the least costly; disk is cheap to add. The downstream effects are the expensive part: longer backup windows, longer restores, more expensive maintenance, slower schema changes across thousands of chunks.

Retention policy management becomes load-bearing, and the hard part is rarely the configuration. The OEM's customers, now benchmarking equipment against peers on historical trends, reject the retention window set two years ago. The food processor meets the same moving target from the regulatory side: entering a new jurisdiction rewrites the retention obligations it launched with. The policy must stay aligned with requirements that keep moving, and someone has to confirm it is actually running.

Growth changes the operating model across every dimension: more assets, more users, more historical data, and more customer expectations. Query concurrency grows from a handful of internal analysts to thousands of customer-facing users, bringing read replicas, replication lag, and connection routing into scope. Configuration that was correct at two hundred assets is reconsidered at twenty thousand, but corrections don't take effect instantly. They migrate into effect gradually as new data arrives, which means the team is managing transitions, not flipping switches. These solutions work. They also need owners, and more of them as the application grows.

None of this stays solved. Each revisit lands on the same team being asked to ship features and support customers. Growth in the application is growth in the platform team's backlog.

The Platform Never Stops Evolving

Figure 4: Version lifecycle timeline: PostgreSQL major version cadence, TimescaleDB release cadence, support windows, and the upgrade planning cycles they impose

There is a common expectation, particularly among teams building their first production database platform, that operations stabilize once the deployment is running. It does not. Ownership never ends.

PostgreSQL ships a major version each year; TimescaleDB tracks those releases. Running past end-of-life means running without security patches, which is untenable for any system holding customer data or regulated production records. Minor version patches require only a brief service restart. A major PostgreSQL version upgrade is a coordinated migration process: it requires a staging environment that mirrors production in data volume, query workload, aggregate configuration, and columnstore state; a post-upgrade validation suite checked against pre-upgrade baselines; a tested rollback plan; and a coordinated maintenance window. For the chemicals manufacturer's five years of reactor telemetry, that is two to four engineer-weeks of work, recurring roughly annually, for as long as the platform operates.

Runbooks are the connective tissue holding the rest together, and they decay by default. Return to the automotive supplier, now three years in. The platform looks substantially different from what launched: additional lines instrumented, aggregates added, retention adjusted, the HA configuration changed after a failover exposed a gap in the original design, the chunk interval re-tuned after ingest grew. Each change was made for a good reason. None made it back into the runbook, and the engineer who made most of them has moved to a different team. This is how operational debt accumulates: not through negligence, but through the ordinary pressure of a team moving fast and treating documentation as something to get to later. If you are reading this and thinking it would not happen on your team, it is worth asking when your runbooks were last tested against the system they describe. Later arrives during incidents.

Every Capability Arrives With an Owner Attached

The preceding sections walk through these systems one at a time, as they arrive in the life of a deployment. Here is the full surface area in one place: each system with its own configuration, monitoring requirements, failure modes, and cadence of ongoing work. In aggregate, they make up the platform.

System	What it delivers	How it fails quietly	The ongoing work	Cadence
HA & failover (replicas, pooler, promotion automation)	The availability SLA	Silent replication drift; failover automation that was never rehearsed	Lag monitoring, failover drills, pooler tuning, rolling maintenance coordination	Continuous monitoring; drills quarterly
Backup & point-in-time recovery	The RPO/RTO commitment	Restores never run at production volume; broken WAL archive chains	Restore validation against real objectives; post-restore verification before returning to service	Restore drills quarterly; re-scoped at each growth step
Retention policies	Bounded storage; compliance windows	Background job fails silently; policy on the wrong relation deletes data that should have been kept; aggregates outlive the raw data they were built from	Policy validation and job monitoring; alignment with dependent systems and changing business requirements	Monthly job audit; review on every regulatory or contract change
Hypercore columnstore	90–98% storage reduction; faster analytical queries	Conversion boundary set too early adds overhead to hot data; too late, and the economics of long retention erode	Boundary tuning against access patterns; resource planning for large backfills (e.g., post-calibration corrections)	Semiannual review; per backfill event
Continuous aggregates	Dashboard latency at production scale	A failed refresh serves stale data with no user-visible error	Refresh policy tuning; job failure alerting so stale data is caught before users report it	Weekly alert review; retuned with each workload shift
Chunk configuration	Ingest throughput and memory health	A misconfigured interval degrades writes precisely during peak load	Interval review as ingest rates change; corrections migrate gradually into effect	Quarterly review or per major ingest change
Security & governance	Breach containment, auditability, contractual compliance	Long-lived over-permissioned credentials; audit logs retained where nobody looks; unreviewed production changes	Credential scoping and rotation rehearsed outside of incidents; audit log review and tamper protection; change control with staging validation and rollback plans	Rotation per policy; review gates on every production change
Version lifecycle	Security patches and support coverage	Running past end-of-life, unpatched, while holding customer or regulated data	Minor patch windows; major upgrades validated against a staging environment that mirrors production volume, workload, aggregate configuration, and columnstore state	Minor on a rolling cadence; major roughly annually
Runbooks & institutional knowledge	Incident response speed	Documentation describing the system as it was at launch	Updating with every configuration change; testing procedures against the live system	With every change; tested quarterly

Losing the Database Stops Being an Outage

We have seen audit configurations that were technically correct but practically invisible: logs retained in a system nobody had access to, alerts wired to a distribution list that no longer existed. The mechanism was in place. The ownership was not.

Every section so far has priced platform ownership in engineer hours and business risk. There is a point in a platform's life where the currency changes, where losing the database stops being an outage and starts being regulatory action, contractual liability, or litigation. The chemicals manufacturer lives past that point. Its five years of reactor telemetry is the empirical foundation for yield improvement decisions that took years to accumulate, and it is also an evidentiary record. The corrupted migration from earlier in this paper does more than destroy operational knowledge; it puts the team in the position of reconstructing that record under potential legal scrutiny, explaining to lawyers what a background job did and why nobody caught it. Some of these failures are irreversible by construction. A retention policy that accidentally drops a week of compliance data cannot be undone. A breach of customer telemetry arrives with notification obligations and suddenly load-bearing contract language.

This work rarely looks dramatic on a task list. It is credential scoping, rotation drills, tamper-protected audit logs, staging validation, change control, and rollback plans for every production change.

But the failure mode is different. Availability gaps cost minutes. Recovery gaps cost hours or days. Governance gaps can cost the company things that cannot simply be restored.

What This Costs, and What It Buys

Sustaining this platform at the standard a critical application demands lands between 1.5 and 3 full-time platform engineers, growing with the application, because every dimension of the work scales with data volume, concurrency, and the criticality of the guarantees. The on-call rotation requires three to four people to staff sustainably, independent of how much of their time the platform consumes. A major version upgrade is two to four engineer-weeks a year. Restore drills, failover rehearsals, policy audits, and runbook maintenance each claim recurring days per quarter. These numbers are rough by necessity and conservative by experience.

Capability was never the question. The teams that do this well go in with eyes open, staffed for the work, treating the platform as a product in its own right. The real question is the counterfactual: what would those same engineers build if they were pointed at the application instead, at the ingestion pipelines, the product features, the things customers actually pay for, while the guarantees are delivered by the team that builds the database?

Most engineers who evaluate TimescaleDB believe they are making a database decision. By the time the application is in production and customers depend on it, they discover they were deciding which parts of a platform they want to own. Every team owns the application. The decision is how much of the platform they want to own alongside it. This paper is designed to make that decision visible before it is made. For a framework to make it deliberately, see Self-Hosted TimescaleDB or Tiger Cloud: A Framework for the Decision.

Great Models Aren't Enough for Physical AI

Team Tiger Data — Thu, 18 Jun 2026 17:52:23 +0000

Notes from our Physical AI dinner

What should a drone do when a police helicopter approaches it?

We heard that question at a dinner we recently hosted for engineering leaders and founders working in Physical AI: the AI behind robots, drones, autonomous vehicles, and other machines that sense and act in the real world, along with the infrastructure that keeps them running. Nobody at the table had a clean answer. The people in that room deploy these machines for a living, and the question that stumped them was about safety, regulation, and operations, not model quality.

That was the theme of the whole evening: Physical AI is constrained by the physical world. Progress depends not only on better models, but on solving the problems around them: regulation, safety, operations, and data.

Scaling takes more than a better model

Despite rapid model progress, truly large-scale autonomous deployments still feel distant. The gap is the long tail of situations nobody puts in a pitch deck. How should a system respond when an animal starts interacting with the equipment? What kicks in when hardware behaves in ways nobody anticipated? Who is accountable when it does?

This is the unglamorous work that gates adoption. The physical world doesn't behave like a benchmark.

The physical world plays by rules a model can't change

Physical AI companies routinely operate inside regulatory frameworks written for older technologies. EV charging companies have had to navigate gas-station rules, including public price displays and printed receipts. Drone operators face aviation requirements designed for crewed aircraft. These constraints sit outside the model entirely, and a team has to clear them before a deployment is legal, let alone good.

Society sets its own rule on top: machines face a higher bar than humans. A battery fire or an autonomous accident draws disproportionate attention compared to an equivalent human-caused incident. That's the reality, and the teams that win will design for it early.

Surviving the physical world is a data problem

The edge cases, the regulations, the higher bar: you handle all of them through the data the machines produce. You catch an edge case because something in the telemetry looked wrong. You prove you met a regulation because you kept the records. The physical world is messy, and data is how you get a grip on it.

So data becomes its own hard problem. Machines in the field generate enormous volumes of telemetry, and every team deploying them wrestles with the same five decisions:

What does the system need in real time?
What stays at the edge?
What gets shipped to the cloud?
What's worth retaining for training?
And what must be kept for regulators, sometimes for decades?

Nuclear applications can carry 30-year retention requirements, a timescale that makes most storage strategies look quaint.

At fleet scale, monitoring, observability, and automation become critical infrastructure, increasingly run with agentic copilots that help operators watch and triage while humans stay accountable for the rare edge cases.

Most teams haven't felt this yet, because most aren't at fleet scale. The ones who treat the telemetry layer as core infrastructure before they get there are the ones who won't be rebuilding it under load later.

The work is surviving reality, not beating a benchmark

The model is what everyone watches. The deployment is decided by everything around it: the regulation, the safety bar, the operations, and the data that ties them together.

That's the work the people at our dinner do every day, and it's why we'll keep bringing them together. It's also the work we do: helping teams capture, store, and make sense of the data their machines produce, so the operational layer is ready when deployment scales. If you're building machines that operate beyond the lab, reach out. We'd love to have you at the table.

When PostgreSQL Isn't the Right Fit: Recognizing Workloads That Need Different Architecture

Team Tiger Data — Fri, 12 Jun 2026 12:00:47 +0000

When PostgreSQL isn't the right fit, the signs don't announce themselves clearly. Postgres is the right database for roughly 90% of workloads, such as SaaS backends, CRUD applications, and transactional systems with mixed read/write access on shared rows. But there's a narrow 10% where those same strengths become overhead: high-frequency append-only ingestion, time-ordered data accumulating at sustained rates, analytical scans over hundreds of millions of rows. If that sounds like your system, this post is for you.

What You Will Learn

If you've added indexes, implemented partitioning, tuned autovacuum, and upgraded hardware only to watch performance degrade again on the same trajectory, the problem likely isn't your configuration. By the end of this post, you'll know whether your workload is in Postgres's 10%, how to confirm it with a single diagnostic query, and what the first concrete step toward the right architecture looks like.

Why It Matters

An optimization problem and an architecture problem look identical in the early stages. Both show up as slow queries. Both respond to the same fixes: indexes, partitioning, autovacuum tuning, hardware upgrades. The divergence happens later, when the fixes stop holding and performance degrades on the same trajectory regardless of what you change.

This is what’s known as the optimization treadmill: a predictable sequence of phases that each buy three to six months of relief without changing the underlying trajectory. MVCC overhead, row-oriented storage, B-tree index maintenance, WAL volume. These aren't bugs. They're architectural tradeoffs that work well for 90% of workloads and work poorly for the 10%.

Knowing which problem you have determines whether you should keep tuning or make a different decision.

What Postgres Was Designed For

Postgres's architecture is built around concurrent access to shared rows. Multiple transactions read and write the same data at the same time, and MVCC handles the isolation. B-tree indexes find specific rows by key. Row-oriented storage assumes that when you retrieve a row, you want most of the columns in it.

For an e-commerce backend, a user authentication system, or a multi-tenant SaaS product, these are exactly the right tradeoffs. Transactions need isolation. Point lookups by user ID are the dominant query pattern. Write rates track user activity, which gives the database natural breathing room between peaks. The question isn't whether Postgres is good. It's whether the workload you're running matches the patterns its architecture was designed to serve.

The Workload That Breaks the Match

Three characteristics, when they appear together, put a workload outside what Postgres handles well.

Append-only or append-heavy writes. Rows are written once and never, or almost never, updated. Sensor readings, financial transactions, log entries, event streams. Every row still pays the full MVCC cost: a 23-byte tuple header tracking transaction visibility, hint-bit dirtying on reads, and autovacuum running continuously to freeze tuples and update the visibility map. None of that overhead produces value on data that will never be touched again.

Sustained high write rates. Not burst traffic that settles. Continuous ingestion at thousands to hundreds of thousands of rows per second, around the clock. The table grows without pause, B-tree index maintenance adds overhead with every insert, and that cost compounds with row volume, so there is no quiet window for autovacuum to catch up.

Analytical query patterns. The queries are aggregations over time ranges: averages, counts, percentiles, GROUP BY time bucket. Row-oriented storage forces Postgres to read all columns of every matching row even when the query needs two. On a 30-column table, that's fifteen times the I/O a columnar layout would require.

Any one of these is manageable. All three together is the combination that Postgres handles well at one million rows and struggles with at one hundred million.

The Optimization Treadmill in Practice

The pattern is predictable. Queries slow down as the table grows. You add indexes, and reads get faster. Write performance drops because index maintenance scales with row volume. You upgrade the instance. Performance stabilizes and costs go up. You implement partitioning. Recent-data queries get faster. Partition management becomes its own maintenance burden. You tune autovacuum settings. Things stabilize for a while. Data volume increases. The cycle repeats.

Each step is individually correct. The problem is that the sequence never ends. You're working around an architectural mismatch instead of running a workload the architecture was designed to serve.

The engineering cost accumulates in ways that are harder to see on a dashboard. The senior engineer spending a week on partition strategy is not shipping product features. The on-call rotation starts treating "database is slow again" as a recurring incident category. Quarterly planning includes a database scalability line item, every quarter.

How to Know Which 10% You're In

The answer is already in your table statistics. Not in EXPLAIN plans or monitoring dashboards, but in the counters tracking exactly how rows have been written, updated, and cleaned up over the table's lifetime. Run this against your highest-traffic tables:

SELECT
    relname AS table_name,
    N
_live_tup,
    n_dead_tup,
    n_tup_ins,
    n_tup_upd,
    ROUND(n_tup_upd::numeric / NULLIF(n_tup_ins, 0) * 100, 2) AS update_pct,
    last_autovacuum,
    last_autoanalyze
FROM pg_stat_user_tables
WHERE schemaname = 'public'
ORDER BY n_tup_ins DESC
LIMIT 10;

Here's an example of what a flagged table looks like next to a healthy one:

table_name	n_tup_ins	n_tup_upd	update_pct	last_autovacuum
device_metrics	84,729,3041	24,892	0.00	2025-06-01 14:22:11
user_accounts	184,203	91,843	49.86	2025-05-29 08:14:03

device_metrics is in the 10%: 847 million inserts, near-zero updates, and autovacuum fired three minutes ago on a table that has never had a meaningful UPDATE run against it. user_accounts is not: nearly half its rows are updated, and autovacuum runs only when it actually needs to.

Look for update_pct under 5% and last_autovacuum timestamps within the last few minutes on tables with near-zero deletes. That's the overhead the companion piece documents in detail: a cleanup process running non-stop on data you never modify, because the storage engine generates that work regardless of your intent.

Pair those numbers against the broader pattern. Your sustained write rate exceeds 10,000 rows per second. Your most common queries aggregate over time ranges, not point lookups by row identifier. You added partitioning specifically to control table size. You upgraded your instance specifically for query performance, not connection headroom.

Three or more of those conditions, and you're in the 10%. The optimization treadmill will keep running, but the trajectory won't change.

What the 10% Actually Needs

If you've confirmed you're in the 10%, migrating your highest-traffic table starts with a single function call:

SELECT create_hypertable('device_metrics', by_range('ts'));

This converts the table to a TimescaleDB hypertable, which does automatic time-based chunking without cron jobs or partition management scripts. From there, you can enable columnar storage on your chunks. This format reads only the columns a query requests, not full rows, and compresses historical data by 10 to 20x, bringing time-range aggregation performance in line with what the workload demands. The migration post walks through the full process, including zero-downtime options for production tables.

You keep the same SQL, the same connection strings, the same ecosystem tooling. This isn't a replacement for Postgres. It's Postgres with the storage primitives your specific workload actually needs.

Conclusion

Postgres is not the problem. Running the wrong workload class through an architecture designed for a different problem is. The distinction matters because one has a tuning fix and the other has a structural fix, and those two paths look identical for the first several months.

The most expensive version of this recognition happens after 18 months of optimization effort. The cheapest version happens now.

Run the diagnostic query above. If the numbers land where you expect, read the full architectural breakdown. If you're ready to test on your own data, start a free Tiger Data trial today.

Row vs Columnar Storage for Analytics: Why PostgreSQL Scans Are Slower Than They Should Be

Team Tiger Data — Fri, 05 Jun 2026 12:48:04 +0000

Here's a query that runs on most time-series tables:

SELECT time_bucket('1 hour', ts) AS hour,
       avg(temperature),
       max(temperature)
FROM sensor_readings
WHERE ts > now() - interval '7 days'
GROUP BY hour
ORDER BY hour;

The query needs two columns: ts and temperature. The table has 15 columns. Postgres reads all 15 columns for every row that matches the WHERE clause.

That's not a bug. It's how row-oriented storage works. Each row is stored as a contiguous block of bytes on disk, called a heap tuple, and Postgres reads the entire tuple to access any column within it. For point lookups on individual records, this is efficient. You want the whole row, and it's stored together. For analytical scans over millions of rows where you need two columns out of fifteen, it's the dominant source of wasted I/O.

In Understanding Postgres Performance Limits for Analytics on Live Data, row-oriented storage was identified as one of four architectural constraints that compound under high-frequency ingestion. That whitepaper maps the pattern at a system level. This post goes deeper on the physical mechanism: exactly how pages work, how read amplification accumulates, and why the usual fixes don't reach it.

What You Will Learn

By the end of this post, you'll have a concrete diagnostic formula: the read amplification ratio. It tells you whether your storage layout is the dominant I/O bottleneck for analytical queries on any table you own. You'll also understand why indexes can't fix this class of problem and how a hybrid row-columnar storage layout changes the math. This post assumes working familiarity with Postgres page layout and B-tree indexes.

How Row Storage Actually Works in Postgres

Postgres stores data in 8KB pages. Each page holds multiple heap tuples. Each tuple contains every column value for that row, stored sequentially, preceded by a 23-byte header that carries transaction visibility metadata.

A table with 15 columns averaging 200 bytes per row fits roughly 35 to 40 rows per page, after accounting for headers, alignment padding, and page overhead.

When Postgres runs a sequential scan, it reads pages from disk in order. Each page load brings all the rows on that page into shared_buffers, with all 15 columns per row intact. The executor then evaluates the WHERE clause and pulls the needed columns from what was already loaded into memory.

The I/O cost is proportional to total table size, not to the size of the queried columns. A query that needs 12 bytes of data per row still reads 200 bytes from disk. The remaining 188 bytes load into the buffer cache and get discarded.

The Read Amplification Math

The number that makes this concrete is the read amplification ratio: total row width divided by the width of the columns the query actually needs.

For sensor_readings, the calculation is direct. The ts column is a timestamptz at 8 bytes. The temperature column is a float4 at 4 bytes. Together they represent 12 bytes of useful data per row. The full row is 200 bytes.

Read amplification ratio: 200 ÷ 12 = 16.7x

For every byte the query uses, Postgres reads 16.7 bytes from disk.

At 100 million rows covering seven days, that ratio stops being abstract. The query needs 100M x 12 bytes = 1.14 GB. Postgres reads 100M x 200 bytes = 18.6 GB. At a 500 MB/sec sequential read rate, the scan takes approximately 38 seconds. Reading only the needed columns would take roughly 2.3 seconds. That 16x gap is pure storage model overhead.

No index changes this number. No configuration setting changes it. Partitioning reduces scope. Fewer pages get scanned by cutting the time range, but within each partition the same per-row read cost applies. The storage layout determines the I/O, and the storage layout is fixed.

Try This Now: Measure Your Read Amplification

You can calculate the ratio for any table you own. Run these two queries to get the byte widths you need:

-- Full row weight
SELECT pg_column_size(t.*) AS row_bytes
FROM sensor_readings t
LIMIT 1;

-- Queried column weight
SELECT pg_column_size(ts) + pg_column_size(temperature) AS queried_bytes
FROM sensor_readings
LIMIT 1;

Divide row_bytes by queried_bytes. If the ratio is above 5x, the storage model is your largest I/O bottleneck for analytical queries on that table. No index or configuration change will close that gap.

Why Indexes Don’t Solve This

When a query is slow, the instinctive response is to add an index. For OLTP workloads, that instinct is correct. B-tree indexes excel at row selection: they find specific rows in O(log n) time, and for a lookup like SELECT * FROM users WHERE id = 123, the index locates the target row in microseconds.

For analytical queries that touch millions of rows, row selection is not the bottleneck. Finding the rows is fast. Reading the data from those rows is slow. An index scan on a million-row result set still reads the full heap tuple for every matching row to extract the needed columns.

The one exception is a covering index, which stores column values inside the index itself so Postgres can satisfy the query without touching the heap. But covering indexes for analytical queries become impractical at scale. When queries involve aggregations across high-frequency writes, wide covering indexes impose substantial write overhead, compounding exactly the index maintenance costs described in the optimization treadmill post.

B-tree indexes optimize for row selection (which rows to read). Analytical query performance is dominated by row width (how much data per row). These are different problems, and solving one leaves the other intact. For a broader look at what this means for your schema design, see Best Practices for PostgreSQL Data Analysis.

How Columnar Storage Changes the Equation

In columnar storage, data is organized by column instead of by row. All values for ts live together in one stream on disk. All values for temperature live together in another. When the query needs those two columns, it reads two streams. The other 13 columns are never touched.

Same query, same 100 million rows: data read drops to 100M x 12 bytes = 1.14 GB. With typical 10 to 20x compression for time-series data, that compresses to approximately 60 to 120 MB. At 500 MB/sec, the same scan completes in roughly 0.12 to 0.24 seconds.

The compression benefit stacks on top of the I/O reduction. Because all values in a column share the same data type, compression algorithms work far more effectively. Sequential timestamps delta-encode to near-zero storage overhead. Floating-point sensor values compress with XOR-based techniques derived from Facebook's Gorilla algorithm. Row-oriented heap storage can't apply any of these because values from different columns are interleaved on every page. There's no contiguous column stream to compress.

Hypercore: Row and Columnar in One Table

The tradeoff with pure columnar storage is write performance. Every new row appends to each column file separately, which adds overhead for high-frequency ingestion. You get the read benefit but give up write throughput. Tiger Data's Hypercore solves this with a hybrid layout that keeps both.

Recent data stays in row-oriented storage for fast ingestion. Older data converts automatically to columnar format based on a compression policy you configure. The application writes standard SQL to one table. The storage format changes by age without any application-layer involvement.

-- Enable Hypercore on a hypertable with a 7-day row storage window
ALTER TABLE sensor_readings SET (
    timescaledb.compress,
    timescaledb.compress_segmentby = 'device_id',
    timescaledb.compress_orderby = 'ts DESC'
);

SELECT add_compression_policy('sensor_readings', INTERVAL '7 days');

New rows land in row format and ingest quickly. Data older than seven days converts to columnar chunks. To verify the behavior immediately without waiting for the policy schedule, compress a chunk manually:

SELECT compress_chunk(c) FROM show_chunks('sensor_readings') c LIMIT 1;

Then run EXPLAIN (ANALYZE, BUFFERS) on the aggregation query to see the difference in buffer reads (representative output on a 100M-row dataset):

-- Before: row storage sequential scan
Seq Scan on sensor_readings
  Buffers: shared read=2375000 -- 18.6 GB read from disk
  Execution Time: 38142.2 ms

-- After: Hypercore columnar scan
Custom Scan (ColumnarScan) on sensor_readings
  Buffers: shared read=10240 -- 80 MB read from disk
  Execution Time: 196.4 ms

The same SELECT statement works against both storage formats. The query planner handles the difference transparently.

Conclusion

Row storage reads every column to access any column. For analytical queries that scan millions of rows and need only a few, this is the largest source of I/O overhead. It doesn't yield to index tuning, partitioning, or hardware upgrades.

Calculate the read amplification ratio for your most common analytical queries using the pg_column_size queries above. If the ratio is above 5x, Hypercore is the direct fix. Start a free Tiger Data trial today to enable the hybrid storage model on your tables.