DEV Community: Kshitiz Maurya

# HLLN 2.1 Just Beat CfC on Chaos—And It Used 6 Fewer Parameters. Here’s Why That Matters.

Kshitiz Maurya — Fri, 24 Apr 2026 16:29:20 +0000

Title:
HLLN 2.1 Just Beat CfC on Chaos—And It Used 6× Fewer Parameters. Here’s Why That Matters.

Post Body:
A physics-inspired recurrent cell outperforms one of the most celebrated continuous-time models on a brutal dynamical benchmark. What does this mean for the future of sequence modeling?

The Hook: A Small Model, A Big Statement In the race to build ever-larger neural networks, it is easy to forget that structure can be more powerful than scale.

Last month, I trained a tiny recurrent cell called HLLN 2.1 (Heisenberg-Limited Learning Network) on a classic chaos benchmark: the Lorenz-96 system with regime shifts. The goal was simple—predict a 40-dimensional chaotic attractor as it abruptly switches dynamical modes (forcing F=8 → F=12 → F=8). The baseline I chose was not a toy. It was the Closed-form Continuous-depth (CfC) cell, a direct descendant of the celebrated Liquid Neural Networks from MIT.

The result?

Model Test MSE Parameters HLLN 2.1 0.1207 1,644 CfC 0.1626 9,720
HLLN 2.1 beat CfC by ~26% error, using roughly 6× fewer parameters.

If you work in sequence modeling, dynamical systems, or physics-informed ML, this should make you pause. Let me explain why.

Why CfC Is a Serious Opponent Before we celebrate, let us appreciate the baseline.

Closed-form Continuous-depth (CfC) networks, developed by Hasani et al. and popularized through the Liquid Time-Constant (LTC) and Liquid Neural Network line of research, are widely considered state-of-the-art for continuous-time sequence modeling. Unlike conventional RNNs that assume fixed time-discretization, CfC cells learn continuous-time dynamics through closed-form ODE approximations. They adapt their time-constants dynamically, making them naturally suited for irregularly-sampled data and non-stationary processes.

In short: CfC is not a strawman. It is a genuine frontier model.

The Benchmark: Lorenz-96 Regime Shifts The Lorenz-96 system is a 40-dimensional chaotic dynamical system widely used in atmospheric modeling and nonlinear dynamics research. It is beautiful, brutal, and unforgiving.

In my experiment, the system undergoes a regime shift:

Phase 1 (Steps 0–500): F = 8.0 — a familiar chaotic attractor.
Phase 2 (Steps 500–1000): F = 12.0 — a different dynamical regime. The statistics change. The attractor morphs.
Phase 3 (Steps 1000–1500): F = 8.0 — a return to the original regime.
This is a nightmare for predictors. A model trained on F=8 must suddenly realize its internal model is wrong, flush outdated assumptions, and adapt to F=12. Then it must switch back. Most RNNs fail catastrophically here because they suffer from memory inertia: they keep averaging the past into the present, blurring two incompatible dynamical laws into a single confused prediction.

How HLLN 2.1 Works: Physics as an Inductive Bias HLLN 2.1 is built on a simple philosophy: let the physics guide the architecture.

The Omega (Ω) Sensor: Real-Time Uncertainty Detection
At every timestep, HLLN measures the prediction error between its current hidden state and the true input. This error feeds into Ω (Omega), an uncertainty amplification factor:

\Omega = 1.0 + \beta \times |prediction_error|

When the system is predictable, Ω stays low. When the regime shifts and predictions fail, Ω spikes. This spike is not just a diagnostic—it is a control signal.

The Decay Gate (Γ): The Memory Flush
Traditional RNNs decay memory passively. HLLN 2.1 actively flushes it:

\Gamma = \sigma( -\alpha \frac{|E|}{\hbar \Omega} )

Here, E represents a learned energy-like parameter, ℏ is a learned uncertainty scale, and Ω is the uncertainty sensor. When Ω spikes (high uncertainty), the denominator increases, the argument of the sigmoid becomes less negative, and Γ drops. A lower Γ means the model forgets faster, clearing out the ghosts of the previous regime.

This is the key: HLLN does not just adapt its learning rate. It adaptively destroys outdated memory.

The Heisenberg Penalty
HLLN also incorporates an uncertainty penalty inspired by the Heisenberg principle:

L_{uncertainty} = ( |\theta|{mean} \times |E|{mean} − \hbar/2 )^2

This regularizes the model to respect a learned uncertainty budget, preventing overconfident predictions during unstable phases.

The Results: Numbers and Geometry Quantitative Dominance Metric HLLN 2.1 CfC Interpretation Test MSE 0.1207 0.1626 HLLN predicts ~26% more accurately Parameters 1,644 9,720 HLLN is ~6× more parameter-efficient Adaptation Signal Ω (uncertainty) τ (time-constant) HLLN’s signal has physical meaning The Geometry of Intelligence Numbers tell only half the story. When we project the hidden states of both models into 3D via PCA, a striking difference emerges:

HLLN 2.1 collapses its 40-dimensional hidden state into a clean, structured manifold—a neural attractor that mirrors the geometry of the underlying physics.
CfC produces a scattered, erratic latent space, suggesting it memorizes snapshots rather than learning the dynamical law.
Figure 1 — Strange Attractor Reconstruction
HLLN 2.1 reconstructing the Lorenz-96 strange attractor during the regime shift phase (F=12)

Figure 2 — Neural Geometry Comparison (3D PCA)
3D PCA of hidden states reveals HLLN’s structured, manifold-like intelligence versus CfC’s scattered distributed memory.

Figure 3 — Complete Experimental Dashboard
Full dashboard showing prediction errors, adaptation signals, decay gate heatmaps, and parameter efficiency.

Figure 4 — Latent Space Dimensionality
Additional dimensionality analysis of HLLN’s emergent representations.

Is This a Big Deal? Yes. Here Is Why. A. Physics-Inspired Inductive Biases Win Over Brute Force CfC is a marvel of engineering, but it is fundamentally a learned approximation to continuous dynamics. HLLN 2.1 encodes a physical principle—uncertainty-driven memory flushing—directly into its architecture. The result is that the model needs far fewer parameters to express the right function.

B. Interpretability Is Not Optional
In HLLN, Ω has a meaning: uncertainty. Γ has a meaning: memory decay. In CfC, the learned time-constants τ are effective but opaque. As AI moves into safety-critical domains, interpretability is a requirement.

C. Efficiency Is the New Accuracy
With only 1,644 parameters, HLLN 2.1 is small enough to run on edge devices. CfC’s 9,720 parameters may not sound like much, but in continuous-time control loops running at kilohertz, every parameter counts.

What This Means for the Future I believe HLLN 2.1 points toward a new category of models: physics-first continuous learners.

Climate & Weather: A model that adapts to regime shifts could improve sub-seasonal forecasting (e.g., El Niño/La Niña).
Robotics: An uncertainty-driven memory system could make control policies far more robust on varied terrain.
Finance: Explicit uncertainty flushing could prevent models from being poisoned by outdated market conditions.
Conclusion: Structure Over Scale
HLLN 2.1 did not win because it is bigger. It won because it is smarter—it encodes a physical insight about how intelligent systems should handle surprise. In a field obsessed with scaling laws, this is a reminder that inductive biases still matter.

Resources
GitHub Repository:

Kshitiz-Maurya / HLLN2.1

Heisenberg-Limited Liquid Network (HLLN) 2.1

HLLN2.1

Heisenberg-Limited Liquid Networks (HLLN 2.1)

Solver-Free Neural Dynamics via Phase-Space Uncertainty

Architect: Kshitiz Maurya
Focus: High-Efficiency Recurrent Dynamics / Edge AI / Noise Robustness

Heisenberg‑Limited Liquid Networks (HLLN 2.1)

📌 Reproducibility notice
The code for the paper “Heisenberg‑Limited Liquid Networks: Solver‑Free Neural Dynamics via Phase‑Space Uncertainty” is permanently archived at
Release preprint-v1.0.

Zenodo Preprint v1 Use that tag to exactly reproduce all experiments from the paper.
The main branch may contain newer experiments, improvements, and additional benchmarks.

🌌 Overview

HLLN 2.1 is a novel recurrent neural architecture designed to replace standard gated mechanisms (GRU/LSTM) with Physical Governors. By coupling internal energy states ($E$) and weight matrices ($\theta$) via a learnable Heisenberg-limited constraint ($\hbar$), HLLN 2.1 achieves superior information density and extreme noise resilience.

🚀 Key Breakthroughs

79.0% Parameter Reduction: Outperforms standard GRUs in complex sequence modeling (Shakespeare/Lorenz) with only ~21% of…

View on GitHub

Interactive Notebook:

colab.research.google.com

    </a>
  </div>
</div>

Preprint / DOI: Zenodo Record

HLLN 2.1 Just Beat CfC on Chaos—And It Used 6 Fewer Parameters. Here’s Why That Matters.

Kshitiz Maurya — Fri, 24 Apr 2026 15:42:02 +0000

HLLN 2.1 Just Beat CfC on Chaos—And It Used 6× Fewer Parameters. Here’s Why That Matters.

A physics-inspired recurrent cell outperforms one of the most celebrated continuous-time models on a brutal dynamical benchmark. What does this mean for the future of sequence modeling?

1. The Hook: A Small Model, A Big Statement

In the race to build ever-larger neural networks, it is easy to forget that structure can be more powerful than scale.

Last month, I trained a tiny recurrent cell called HLLN 2.1 (Heisenberg-Limited Learning Network) on a classic chaos benchmark: the Lorenz-96 system with regime shifts. The goal was simple—predict a 40-dimensional chaotic attractor as it abruptly switches dynamical modes (forcing F=8 → F=12 → F=8). The baseline I chose was not a toy. It was the Closed-form Continuous-depth (CfC) cell, a direct descendant of the celebrated Liquid Neural Networks from MIT.

The result?

Model	Test MSE	Parameters
HLLN 2.1	0.1207	1,644
CfC	0.1626	9,720

HLLN 2.1 beat CfC by ~26% error, using roughly 6× fewer parameters.

If you work in sequence modeling, dynamical systems, or physics-informed ML, this should make you pause. Let me explain why.

2. Why CfC Is a Serious Opponent

Before we celebrate, let us appreciate the baseline.

Closed-form Continuous-depth (CfC) networks, developed by Hasani et al. and popularized through the Liquid Time-Constant (LTC) and Liquid Neural Network line of research, are widely considered state-of-the-art for continuous-time sequence modeling. Unlike conventional RNNs that assume fixed time-discretization, CfC cells learn continuous-time dynamics through closed-form ODE approximations. They adapt their time-constants dynamically, making them naturally suited for irregularly-sampled data and non-stationary processes.

In short: CfC is not a strawman. It is a genuine frontier model.

3. The Benchmark: Lorenz-96 Regime Shifts

The Lorenz-96 system is a 40-dimensional chaotic dynamical system widely used in atmospheric modeling and nonlinear dynamics research. It is beautiful, brutal, and unforgiving.

In my experiment, the system undergoes a regime shift:

Phase 1 (Steps 0–500): F = 8.0 — a familiar chaotic attractor.
Phase 2 (Steps 500–1000): F = 12.0 — a different dynamical regime. The statistics change. The attractor morphs.
Phase 3 (Steps 1000–1500): F = 8.0 — a return to the original regime.

This is a nightmare for predictors. A model trained on F=8 must suddenly realize its internal model is wrong, flush outdated assumptions, and adapt to F=12. Then it must switch back. Most RNNs fail catastrophically here because they suffer from memory inertia: they keep averaging the past into the present, blurring two incompatible dynamical laws into a single confused prediction.

4. How HLLN 2.1 Works: Physics as an Inductive Bias

HLLN 2.1 is built on a simple philosophy: let the physics guide the architecture.

The Omega (Ω) Sensor: Real-Time Uncertainty Detection

At every timestep, HLLN measures the prediction error between its current hidden state and the true input. This error feeds into Ω (Omega), an uncertainty amplification factor:

Ω = 1.0 + β × |prediction_error|

When the system is predictable, Ω stays low. When the regime shifts and predictions fail, Ω spikes. This spike is not just a diagnostic—it is a control signal.

The Decay Gate (Γ): The Memory Flush

Traditional RNNs decay memory passively. HLLN 2.1 actively flushes it:

Γ = sigmoid( −α |E| / (ℏ Ω) )

Here, E represents a learned energy-like parameter, ℏ is a learned uncertainty scale, and Ω is the uncertainty sensor. When Ω spikes (high uncertainty), the denominator increases, the argument of the sigmoid becomes less negative, and Γ drops. A lower Γ means the model forgets faster, clearing out the ghosts of the previous regime.

This is the key: HLLN does not just adapt its learning rate. It adaptively destroys outdated memory.

The Heisenberg Penalty

HLLN also incorporates an uncertainty penalty inspired by the Heisenberg principle:

L_uncertainty = ( |θ|_mean × |E|_mean − ℏ/2 )²

This regularizes the model to respect a learned uncertainty budget, preventing overconfident predictions during unstable phases.

5. The Results: Numbers and Geometry

Quantitative Dominance

Metric	HLLN 2.1	CfC	Interpretation
Test MSE	0.1207	0.1626	HLLN predicts ~26% more accurately
Parameters	1,644	9,720	HLLN is ~6× more parameter-efficient
Adaptation Signal	Ω (uncertainty)	τ (time-constant)	HLLN’s signal has physical meaning

The Geometry of Intelligence

Numbers tell only half the story. When we project the hidden states of both models into 3D via PCA, a striking difference emerges:

HLLN 2.1 collapses its 40-dimensional hidden state into a clean, structured manifold—a neural attractor that mirrors the geometry of the underlying physics.
CfC produces a scattered, erratic latent space, suggesting it memorizes snapshots rather than learning the dynamical law.

Figure 1 — Strange Attractor Reconstruction

HLLN 2.1 reconstructs the Lorenz-96 strange attractor during the regime shift phase (F=12).

Figure 2 — Neural Geometry Comparison (3D PCA)

3D PCA of hidden states reveals HLLN’s structured, manifold-like intelligence versus CfC’s more scattered distributed memory. File: geometry_comparison_hd.png

Figure 3 — Complete Experimental Dashboard

Full dashboard showing prediction errors (log scale), adaptation signals (Ω vs τ), decay gate heatmaps, residuals, and parameter efficiency.

Figure 4 — Latent Space Dimensionality

Additional dimensionality analysis of HLLN’s emergent representations. File: newdimen.png

The Micro-View: Adaptation in Real-Time

Zooming in around the regime shift (timesteps 450–600), we see HLLN’s hidden state react instantaneously to the changing dynamics, while its decay gate simultaneously opens to flush outdated memory. CfC, by contrast, shows delayed adaptation because its time-constants must be learned through distributed gating rather than driven by an explicit uncertainty signal.

6. Is This a Big Deal? Yes. Here Is Why.

A. Physics-Inspired Inductive Biases Win Over Brute Force

CfC is a marvel of engineering, but it is fundamentally a learned approximation to continuous dynamics. HLLN 2.1 encodes a physical principle—uncertainty-driven memory flushing—directly into its architecture. The result is that the model needs far fewer parameters to express the right function.

This is a broader lesson for ML: when we know something about the structure of the world, we should build it into the model.

B. Interpretability Is Not Optional

In HLLN, Ω has a meaning: uncertainty. Γ has a meaning: memory decay. In CfC, the learned time-constants τ are effective but opaque. As AI moves into safety-critical domains—climate modeling, medical forecasting, autonomous control—interpretability is not a luxury. It is a requirement.

C. Efficiency Is the New Accuracy

With only 1,644 parameters, HLLN 2.1 is small enough to run on edge devices, embedded sensors, or low-power satellites. CfC’s 9,720 parameters may not sound like much in the era of billion-parameter transformers, but in continuous-time control loops running at kilohertz, every parameter counts.

7. What This Means for the Future

I believe HLLN 2.1 points toward a new category of models: physics-first continuous learners.

Immediate Implications

Climate & Weather: Lorenz-96 is a toy model for atmospheric dynamics. A model that adapts to regime shifts could improve sub-seasonal forecasting, where the planet switches between El Niño and La Niña modes.
Robotics: Robots operating on varied terrain face constant "regime shifts" (slippery → rough → inclined). An uncertainty-driven memory system could make control policies far more robust.
Finance: Markets shift between high-volatility and low-volatility regimes. Explicit uncertainty flushing could prevent models from being poisoned by outdated market conditions.

The Research Agenda Ahead

Multi-Scale HLLN: Can we stack HLLN cells operating at different timescales to capture both fast transients and slow drifts?
Hybrid Simulators: Can HLLN gates be coupled directly with numerical ODE solvers for physics-informed neural simulators?
Theoretical Guarantees: Can we prove stability bounds for HLLN under arbitrary switching sequences?

8. Conclusion: Structure Over Scale

HLLN 2.1 did not win because it is bigger. It won because it is smarter—it encodes a physical insight about how intelligent systems should handle surprise. In a field obsessed with scaling laws, this is a reminder that inductive biases still matter. A well-placed physical principle can outperform brute-force learning, especially when the world changes beneath your feet.

The future of sequence modeling is not just continuous. It is uncertainty-aware, physics-grounded, and interpretable.

HLLN 2.1 is a small step in that direction. But on a 40-dimensional chaotic attractor, small steps can take you far.

Resources

Interactive Notebook (Colab): Lorenz-96 Experiments — HLLN 2.1 vs CfC
Preprint / DOI: Zenodo Record
GitHub: github.com/Kshitiz-Maurya/HLLN2.1
Images: See hlln_attractor_hd.png, geometry_comparison_hd.png, full_dashboard.png, and phase_portrait.png in the GitHub results folder.

If you are working on continuous-time models, regime-shift detection, or physics-informed ML, I would love to hear from you. Let us build the next generation of adaptive intelligence—lean, interpretable, and grounded in physical principles.