TL;DR:
An open-source toolkit to measure how consistently an AI agent thinks — not just whether it gives the right answer.
👉 github.com/Freeky7819/harmonic-rsi
💡 Why this project exists
When evaluating large language models, we usually focus on compliance and accuracy.
But there's another dimension that often gets ignored — stability of reasoning.
How steady is the model’s internal logic from step to step?
Does it “drift” or “oscillate” between modes of thought?
Can we quantify that resonance instead of guessing?
That’s what the Harmonic RSI project explores.
🧩 What is Harmonic RSI?
Harmonic RSI (Resonance Stability Index) is a lightweight Python package that analyzes reasoning traces from AI agents — sequences of thoughts, plans, or explanations — and quantifies how coherent they remain over time.
It can be used standalone, or as a plug-in evaluator in frameworks like Rogue, LangChain, or EvalGen.
Main features:
🌀 Resonance Stability Index (RSI):
Measures logical drift via cosine distance between consecutive embedding vectors.
🔭 Resonant-filter mode (experimental):
Applies a log-periodic modulation on the embedding sequence to detect oscillatory instability.
🧩 ISM Φ-Layer:
Extracts phase-like signals from model embeddings and tracks ∂Φ/∂t (logical phase velocity).
🧠 Gradio UI:
Real-time reasoning dashboard:
Prompt → GPT → Embeddings → ISM → RSI
⚙️ CLI and API:
Works as a standalone evaluator or integrated pipeline.
⚙️ Quick Example
from harmonic_rsi import ResonanceEvaluator
trace = [
"Plan: gather data",
"Next: filter by category",
"Then: summarize results"
]
rsi = ResonanceEvaluator()
print(rsi.evaluate(trace, mode="embedding"))
Output
{'resonance_score': 0.87, 'phase_drift': 0.12, 'semantic_coherence': 0.91}
📊 Why it matters
Instead of treating reasoning instability as random noise,
RSI models it as a resonance pattern —
something that can be measured, compared, and potentially optimized.
Think of it as signal analysis for cognition — applied to LLMs.
⚖️ License & Ethos
License: CC BY-NC 4.0 — open for research, not for commercial use.
Goal: transparent exploration of internal model stability.
Not another leaderboard metric:
RSI complements standard evals; it doesn’t compete with them.
🧰 Try it out
Clone and run locally:
git clone https://github.com/Freeky7819/harmonic-rsi
cd harmonic-rsi/harmonic-rsi_final
pip install -e ".[st,dev]"
pytest -q
python -m harmonic_rsi.app_gradio
Gradio dashboard will open at localhost:7860
.
🙋♂️ Contributing
Feedback, testing, or critical discussion are very welcome.
If you’ve worked with evaluation frameworks (Rogue, HELM, EvalGen, etc.) — I’d love your thoughts on integrating RSI as a complementary layer.
Top comments (0)