The MEMORY-NATIVE-NEURAL_NETWORK repository has expanded its suite of experimental architectures with the addition of Neural Differential Manifolds with Momentum (NDM-Momentum). While the standard NDM model introduced continuous weight evolution via Ordinary Differential Equations (ODEs), the Momentum version brings a new level of stability to the "rewiring" process.
What is NDM-Momentum?
NDM-Momentum implements a "stabilized plasticity" framework. In traditional neural networks, weights are updated in discrete steps. In NDM, weights evolve continuously based on Hebbian traces—essentially "neurons that fire together, wire together". The Momentum enhancement introduces a velocity state (the $\beta$ factor) to these changes.
Instead of weights reacting instantly to noisy data gradients, they carry inertia. This creates smoother transitions in the network's physical structure, reducing oscillations and helping the model converge more reliably in volatile environments.
Key Features
- Momentum-Damped Evolution: Uses a velocity manifold ($V$) to filter weight changes ($dW/dt$), preventing catastrophic interference.
- Adaptive Plasticity: The network automatically becomes more fluid when it encounters high prediction errors and more rigid as it masters a task.
- Hebbian Traces: Weights evolve in real-time based on the correlation between pre-synaptic activity and post-synaptic derivatives.
- Dual-Language Implementation: Features a high-performance C backend for ODE integration with a seamless Python API for rapid deployment.
This architecture represents a step forward in creating "living" neural networks that can adapt to changing rules without losing their foundational knowledge.
Explore the source code and documentation at the official repository:
👉 NDM-Momentum on GitHub
Top comments (0)