Self-Supervised Temporal Pattern Mining for coastal climate resilience planning with inverse simulation verification

Self-Supervised Temporal Pattern Mining for coastal climate resilience planning with inverse simulation verification

Introduction: A Tide of Questions

I still remember the afternoon I spent staring at two decades of tidal gauge data from a small estuary on the Atlantic coast. I had been experimenting with contrastive self-supervised learning for time-series representation, mostly on synthetic sensor data, when a colleague — a coastal engineer — asked me a deceptively simple question: "Can your model tell me when the water starts behaving in ways we've never seen before?" That question reshaped how I think about temporal pattern mining. Most anomaly detection pipelines I had built assumed labeled examples of "abnormal" behavior. But in coastal climate systems, the abnormal future is precisely what we have no labels for. The distribution is shifting, and the whole point of resilience planning is to anticipate regimes that don't exist in the historical record.

While exploring this problem, I discovered that self-supervised temporal pattern mining — learning compact representations of seasonal, tidal, and storm-driven dynamics without labels — combined with inverse simulation verification (using learned patterns to constrain and validate physical hydrodynamic simulations) creates a powerful loop. The representation mining tells you what patterns exist and which are drifting; the inverse simulation tells you whether those patterns are physically plausible and what forcing conditions would produce them. Together, they form a verification harness that neither component could provide alone.

This article walks through what I learned building such a system: the architecture, the self-supervised objectives, the inverse-simulation verification loop, the agentic orchestration layer I used to keep the pipeline honest, and the failure modes that cost me the most debugging time. I'll share code that captures the essential patterns rather than production-ready systems, because the insights matter more than the boilerplate.

Why Coastal Resilience Is a Temporal Pattern Problem

Coastal systems are governed by superimposed periodicities: semi-diurnal and diurnal tides, spring-neap cycles, seasonal storm seasons, multi-year El Niño–Southern Oscillation phases, and now a slow secular trend from sea-level rise. A resilience planner cares about questions like:

• Has the joint distribution of surge height and timing shifted relative to tide phase?
• Are compound events (high tide + storm surge + river discharge) becoming more frequent?
• Which monitoring stations are exhibiting representation drift that precedes measurable regime change?

These are temporal pattern questions. And crucially, they are unsupervised questions — we cannot label "the future." In my research of climate time-series literature, I realized that the dominant supervised approaches (e.g., forecasting with labeled extreme events) systematically fail exactly where it matters: at the tail of the distribution, where the novel behavior lives.

Technical Background: Self-Supervised Temporal Representations

The core idea of self-supervised representation learning for time series is to construct pretext tasks whose labels are derived from the data itself. The three families I found most useful during experimentation were:

1. Contrastive predictive coding (CPC) — predict future latent states from past context, using InfoNCE to pull true futures close and negatives far.
2. Masked reconstruction — mask spans of the series and reconstruct them from context (the time-series analogue of BERT/MAE).
3. Temporal-order and phase discrimination — learn invariances to nuisance transformations (sensor gain, offset) while preserving temporal ordering.

For coastal data, I found a hybrid worked best: contrastive prediction over multi-scale contexts (tidal scale, synoptic scale, seasonal scale) plus a masked reconstruction head that forces the encoder to retain fine-grained waveform detail. The multi-scale part is essential because a single context window either blurs out tides (too long) or misses seasonal drift (too short).

Here's the core encoder and multi-scale contrastive objective I converged on:

import torch
import torch.nn as nn
import torch.nn.functional as F

class DilatedConvEncoder(nn.Module):
    """Causal dilated conv encoder: cheap, parallel, good for periodic signals."""
    def __init__(self, in_ch=1, hidden=128, depth=6):
        super().__init__()
        layers, ch = [], in_ch
        for i in range(depth):
            layers += [
                nn.Conv1d(ch, hidden, kernel_size=3,
                          dilation=2**i, padding=2**i),
                nn.GELU(),
                nn.BatchNorm1d(hidden),
            ]
            ch = hidden
        self.net = nn.Sequential(*layers)

    def forward(self, x):            # x: (B, C, T)
        return self.net(x)           # (B, hidden, T)

def info_nce(z_anchor, z_pos, z_neg, temp=0.1):
    """z_anchor, z_pos: (B, D); z_neg: (B, K, D) negatives."""
    anchor = F.normalize(z_anchor, dim=-1)
    pos    = F.normalize(z_pos, dim=-1)
    neg    = F.normalize(z_neg, dim=-1)
    pos_logit = (anchor * pos).sum(-1, keepdim=True) / temp
    neg_logit = torch.einsum('bd,bkd->bk', anchor, neg) / temp
    logits = torch.cat([pos_logit, neg_logit], dim=-1)
    labels = torch.zeros(anchor.size(0), dtype=torch.long)
    return F.cross_entropy(logits, labels)

class MultiScaleCPC(nn.Module):
    """Contrastive prediction at tidal, synoptic and seasonal horizons."""
    def __init__(self, hidden=128, horizons=(12, 96, 720), n_neg=32):
        super().__init__()
        self.enc = DilatedConvEncoder(hidden=hidden)
        self.heads = nn.ModuleList(
            [nn.Linear(hidden, hidden) for _ in horizons])
        self.horizons, self.n_neg = horizons, n_neg

    def forward(self, x):            # x: (B, 1, T)
        h = self.enc(x)              # (B, hidden, T)
        losses = []
        for head, H in zip(self.heads, self.horizons):
            ctx = h[..., :-H].mean(-1)              # pooled context
            tgt = h[..., H:].mean(-1)               # pooled future
            # in-batch negatives: shuffled futures from other windows
            neg = tgt[torch.randperm(tgt.size(0))][:self.n_neg]
            losses.append(info_nce(head(ctx), head(tgt), neg.unsqueeze(0)))
        return torch.stack(losses).mean()

One interesting finding from my experimentation with this architecture was that the seasonal head (720 steps ≈ 30 days at hourly resolution) dominated the loss early in training and effectively starved the tidal head. I fixed this with per-head uncertainty weighting (Kendall et al.'s homoscedastic uncertainty formulation) rather than hand-tuned loss weights — the model learned to balance the horizons itself.

From Representations to Patterns: Mining the Latent Space

The representation is only half the story. To make it useful for planning, I needed to convert latent trajectories into interpretable temporal patterns. During my investigation of latent-space clustering methods, I found that a simple approach beat fancier alternatives: online clustering of latent windows combined with a drift statistic per cluster.

import numpy as np
from sklearn.cluster import MiniBatchKMeans

class PatternMiner:
    """Cluster latent windows; track membership drift over time."""
    def __init__(self, n_clusters=24, window=168, stride=24):
        self.km = MiniBatchKMeans(n_clusters=n_clusters, random_state=0)
        self.window, self.stride = window, stride
        self.history = []  # (timestamp, cluster_id, distance)

    def fit(self, latents):                    # (N, D)
        self.km.fit(latents)

    def mine(self, latents, timestamps):
        ids = self.km.predict(latents)
        dist = self.km.transform(latents).min(axis=1)
        self.history.extend(zip(timestamps, ids, dist))
        return ids, dist

    def drift_report(self, recent_frac=0.2):
        """Compare recent cluster occupancy to baseline."""
        n = len(self.history)
        split = int(n * (1 - recent_frac))
        base = np.bincount([h[1] for h in self.history[:split]],
                           minlength=self.km.n_clusters)
        rec  = np.bincount([h[1] for h in self.history[split:]],
                           minlength=self.km.n_clusters)
        base = base / base.sum() + 1e-9
        rec  = rec / rec.sum() + 1e-9
        # Jensen-Shannon divergence between occupancy distributions
        m = 0.5 * (base + rec)
        jsd = 0.5 * (np.sum(base * np.log(base / m))
                     + np.sum(rec * np.log(rec / m)))
        return {"jsd": float(jsd), "baseline": base, "recent": rec}

The Jensen-Shannon divergence between baseline and recent cluster occupancy became my single most useful scalar for "is the regime shifting?" In my experimentation, a JSD above roughly 0.15 on held-out stations consistently preceded measurable changes in surge statistics by several months — though I want to be careful: this was on a small number of stations, and I treat it as a hypothesis generator, not a validated early-warning signal.

Inverse Simulation Verification: Closing the Loop

This is where the system earns its keep. A learned pattern is just a vector until you can ask: what physical forcing conditions would produce this pattern, and are those conditions plausible? Inverse simulation flips the usual forward modeling direction. Instead of running a hydrodynamic model forward from forcing to water levels, we search for the forcing (wind stress, pressure, river discharge, boundary conditions) that best reproduces an observed or mined pattern — and then interrogate the solution.

I found that a differentiable surrogate of the hydrodynamic model made this tractable. The surrogate is trained on forward simulation outputs, and then we optimize over forcing inputs with gradient descent.

import torch

class SurrogateHydro(nn.Module):
    """Learned forward map: forcing -> water level. Trained on sim outputs."""
    def __init__(self, n_forcing=4, hidden=256):
        super().__init__()
        self.net = nn.Sequential(
            nn.Conv1d(n_forcing, hidden, 5, padding=2), nn.GELU(),
            nn.Conv1d(hidden, hidden, 5, padding=2),    nn.GELU(),
            nn.Conv1d(hidden, 1, 1),
        )
    def forward(self, forcing):        # (B, F, T) -> (B, 1, T)
        return self.net(forcing)

def inverse_solve(surrogate, target_level, n_steps=400, lr=0.05,
                  reg=1e-3, bounds=(-3.0, 3.0)):
    """Recover forcing that reproduces target water level."""
    forcing = torch.zeros(1, 4, target_level.shape[-1], requires_grad=True)
    opt = torch.optim.Adam([forcing], lr=lr)
    for _ in range(n_steps):
        opt.zero_grad()
        pred = surrogate(forcing)
        data_loss = F.mse_loss(pred, target_level)
        smooth = reg * forcing.diff(dim=-1).pow(2).mean()
        (data_loss + smooth).backward()
        opt.step()
        with torch.no_grad():          # physical plausibility box
            forcing.clamp_(*bounds)
    return forcing.detach(), data_loss.item()

The verification step is not just "did we fit the target?" It's a battery of physical consistency checks on the recovered forcing:

• Spectral plausibility — does the recovered wind stress have energy concentrated in meteorologically plausible bands, or is it a high-frequency artifact of overfitting?
• Cross-variable consistency — does recovered pressure correlate with recovered wind in the way real storms do (geostrophic balance signatures)?
• Ensemble stability — do multiple random restarts converge to similar forcing, or is the inverse problem ill-posed for this pattern?

In my research of inverse problem literature, I learned to treat ill-posedness not as a bug but as a diagnostic. When the inverse solve is unstable across restarts, it often means the mined pattern is a statistical artifact rather than a physically realizable state. That check alone filtered a meaningful fraction of spurious clusters in my experiments.

Agentic Orchestration: Keeping the Loop Honest

The pipeline has many moving parts — representation training, clustering, drift detection, inverse solves, verification checks — and I quickly found that a monolithic script became impossible to reason about. While learning about agentic AI systems, I realized the right abstraction was a small set of narrowly-scoped agents with explicit contracts, coordinated by a planner agent that decides what to re-run when verification fails.

from dataclasses import dataclass
from typing import Callable

@dataclass
class AgentResult:
    ok: bool
    payload: dict
    reason: str = ""

class VerificationAgent:
    """Runs inverse solve + physical checks on one mined pattern."""
    def __init__(self, surrogate, checks: list[Callable]):
        self.surrogate, self.checks = surrogate, checks

    def run(self, pattern_target: torch.Tensor) -> AgentResult:
        forcing, loss = inverse_solve(self.surrogate, pattern_target)
        failures = [c.__name__ for c in self.checks if not c(forcing)]
        if failures:
            return AgentResult(False, {"forcing": forcing},
                               f"failed checks: {failures}")
        return AgentResult(True, {"forcing": forcing, "loss": loss})

class PlanLoop:
    """Planner: if verification fails, request more representation data."""
    def __init__(self, miner, verifier, max_rounds=3):
        self.miner, self.verifier, self.max_rounds = miner, verifier, max_rounds

    def run(self, latents, timestamps, targets):
        for r in range(self.max_rounds):
            ids, _ = self.miner.mine(latents, timestamps)
            results = [self.verifier.run(t) for t in targets]
            bad = [i for i, r in enumerate(results) if not r.ok]
            if not bad:
                return results
            # planner decision: prune unverifiable patterns, request retrain
            targets = [t for i, t in enumerate(targets) if i not in bad]
        return results

What I appreciated about this agentic structure was that failure became actionable. A failed verification didn't just log an error — it triggered a specific response (prune the pattern, or request more training windows for that latent region). In my experimentation, this cut the number of unverifiable patterns reaching the planning dashboard by a large margin, though I never benchmarked it rigorously against a human-in-the-loop baseline.

Quantum-Accelerated Component: Where It Actually Helps

I want to be honest about quantum computing here, because hype is easy and substance is rare. In my exploration of quantum computing applications, I found exactly one place in this pipeline where a quantum approach has a plausible near-term advantage: the inverse solve can be recast as a quadratic unconstrained binary optimization (QUBO) problem if you discretize the forcing into a small number of candidate scenarios. Scenario selection under compound-event constraints is combinatorially nasty classically, and annealing-style solvers (quantum or quantum-inspired) can explore that space efficiently.

import numpy as np

def build_scenario_qubo(costs, penalties, n_scenarios):
    """QUBO: pick a small set of forcing scenarios matching a pattern.
    costs: per-scenario mismatch cost; penalties: pairwise conflict matrix."""
    Q = np.zeros((n_scenarios, n_scenarios))
    np.fill_diagonal(Q, costs)
    Q += penalties          # symmetric conflict penalties
    return Q                # feed to annealer or quantum-inspired sampler

I'll be candid: for the problem sizes I worked with (dozens of scenarios), classical simulated annealing was faster and just as good. The quantum angle matters more as scenario libraries grow into the thousands with real compound-event constraints. I include it because it's the honest frontier — promising, unproven, and worth prototyping but not yet load-bearing.

Real-World Applications

The pattern I converged on maps cleanly onto several practical deployments:

• Municipal flood planning — run the drift report quarterly across a sensor network; flag stations with rising JSD for engineering review before design storms are revised.
• Insurance and reinsurance — use verified patterns to stress-test catastrophe models against physically plausible but historically unseen compound events.
• Port and infrastructure operations — mine operational windows (e.g., "safe berthing regimes") and detect when the regime distribution shifts, triggering re-evaluation of operational thresholds.
• Monitoring network design — inverse verification reveals which stations are informationally redundant: if many mined patterns are recoverable from a subset of stations, you can prune or relocate sensors.

Challenges and Solutions

Challenge 1: Non-stationarity poisons the pretext task. Contrastive learning assumes the future resembles the past enough to be predictable. Under sea-level rise, the "future" contains a slow trend the encoder learns to ignore as nuisance. Solution: I added a trend-decomposition front-end (STL or a learned low-pass filter) and fed the encoder the detrended residual plus an explicit trend channel. The encoder then learns dynamics without being confused by the secular drift.

Challenge 2: Cluster count is a hyperparameter with consequences. Too few clusters merge distinct regimes; too many fragment them into unverifiable slivers. Solution: I used the inverse-verification failure rate as a model-selection criterion — sweep cluster counts and pick the one that