The Problem
AI agents fail silently.
You give an agent a clear instruction:
"Refund user 123, $50 within 7 days"
The agent returns:
"User refunded $500 immediately"
No error. No warning. Just wrong output.
This is semantic drift — when LLM output
diverges from original intent.
What I Built
SIP (State Integrity Protocol) is a lightweight
Python SDK that detects and flags drift in
LLM outputs before they cause damage.
How It Works
from sip.middleware import SIPMiddlewarePipeline
pipeline = SIPMiddlewarePipeline()
pipeline.anchor("Refund user 123 $50")
result = pipeline.run(
output="Refund user 123 $500"
)
print(result.status) # repair_required
Three checks run automatically:
- Semantic drift (TF-IDF + cosine similarity)
- Intent alignment (sentence-transformers)
- Numeric drift ($50 vs $500 caught)
Real Test Results
| Test | Status |
|---|---|
| Exact match | accepted |
| Same meaning different words | accepted |
| Wrong output | repair_required |
| Numbers changed | repair_required |
| Injection attempt | repair_required |
Install
pip install state-integrity-protocol
GitHub
github.com/sijan324/state-integrity-protocol
Looking for feedback from anyone building
LLM pipelines or AI agents.
What drift problems have you seen in production?
Top comments (3)
Intent drift detection is one of the parts most agent frameworks pretend doesn't exist. Quick question, are you detecting drift per-step, or over a rolling window of N steps? Single step works for a chat agent but seems to miss the slow cook failures where each step looks locally fine but the goal has shifted three turns ago.
Hey — really appreciate this question, it's exactly the gap I've been thinking about.
Right now SIP anchors to the original intent and checks each step against that.
So yes, per-step only. The slow-cook problem you described is real and honestly not solved yet in SIP. Each step looks fine locally but the goal
has quietly shifted — that's dangerous.Rolling window is something I want to build next.
Would love your thoughts on how you'd approach it.
If you want to try the current version:
pip install state-integrity-protocol
Or see it live:
state-integrity-protocol-jxvjzwbhe...
Would genuinely appreciate your feedback —
you clearly think about this stuff deeply.
Sure, happy to try it out! And yes, I myself am pondering alot on these issues while building my OSS. I will remember to comeback here and share it with you to try it out as well once I launch the stable version (in ~2 weeks).