If you are building LLM applications, AI agents, or RAG pipelines right now, you already know the sinking feeling of watching your model execute a prompt injection or hallucinate a data leak in production.
Most security tutorials tell you to "just write robust system prompts" or "add a regex filter." But if you rely solely on system prompts, a clever user will bypass them with a simple "ignore previous instructions" override within five minutes. And if you rely on heavy custom middleware, you end up adding massive latency overhead to every chat request.
We wanted something better. We wanted an infrastructure-level defense that drops into an existing Node.js or Python backend instantly, runs with sub-millisecond latency at the edge, and actually stops sophisticated attacks.
Here is how we built Ice Phi—an API security gateway and SDK ecosystem—and how you can plug it into your app in under a minute.
The Problem with Traditional LLM Guardrails
When scaling an AI application, developers usually run into three major bottlenecks when trying to secure their endpoints:
The Latency Trap: Running heavy secondary models or complex parsing scripts inline can add hundreds of milliseconds to a chat completion. Users abandon chat apps that lag.
The Integration Friction: Having to rewrite core back end routing logic just to inspect payloads is a massive time sink.
The Black-Box Dilemma: Developers hate routing traffic through unknown third-party services without clear visibility into how the interception layer operates.
The Solution: Edge Middle Ware & Dual SDKs
To solve this, we decoupled the security inspection layer from the core application logic using a high-performance gateway architecture backed by Zuplo, paired with native client SDKs:
TypeScript / Node.js: @ice_phi/icephi-ts (for Express, Fastify, and custom HTTP loops)
Python: icephi-python (for FastAPI, Flask, and agent loops)
Instead of slowing down your main back end, the gateway intercepts payloads at the edge, inspects them for injection vectors, jailbreaks, and malicious patterns, and passes clean traffic through—all in milliseconds.
Getting Started in 3 Lines of Code
We wanted integration to be completely frictionless. You can grab a free API key from our sandbox portal and drop the middle ware directly into your project.
- TypeScript / Express Example
import express from 'express';
import { IcePhiShield } from '@ice_phi/icephi-ts';
const app = express();
app.use(express.json());
// Initialize the shield middleware
const shield = new IcePhiShield({ apiKey: process.env.ICEPHI_API_KEY });
app.use(shield.middleware());
app.post('/chat', (req, res) => {
// If the request reaches here, the payload is verified clean.
res.json({ status: 'success', reply: 'Response from model...' });
});
app.listen(3000);
- Python / FastAPI Example
from fastapi import FastAPI, Request
from icephi import IcePhiShield
app = FastAPI()
shield = IcePhiShield(api_key="your_api_key_here")
@app.middleware("http")
async def security_middleware(request: Request, call_next):
# Inspects and blocks malicious payloads at the edge
await shield.inspect_request(request)
response = await call_next(request)
return response
@app.post("/chat")
async def chat_endpoint():
return {"status": "success", "reply": "Response from model..."}
Test It Live
Don't take our word for it—you can test how it handles malicious payloads right now.
We set up a self-serve sandbox where you can generate a free API key in seconds and run your own prompt injection or jailbreak attempts against live endpoints to see how the engine responds.
📦 npm: @ice_phi/icephi-ts
🐍 PyPI: icephi-python
🛠️ Introduction & Docs: https://icephi.com
Have you run into prompt injection nightmares in your own AI apps? How are you currently handling guardrails? Let’s chat in the comments below!
Top comments (0)