60.1 Introduction
Production AI security does not end when a model has been approved and deployed.
The most important security boundary often exists at runtime, when an actual user request reaches an AI model.
A production inference request may pass through:
User
↓
Frontend
↓
API Gateway
↓
Authentication
↓
Authorization
↓
Policy Engine
↓
Model Router
↓
Retrieval / Tools
↓
Model Runtime
↓
Output Validation
↓
Response
Every stage represents a potential security boundary.
The fundamental principle is:
A model should never receive more authority than the current request requires.
60.2 Runtime Threat Model
AI inference systems can face:
- prompt injection;
- malicious file inputs;
- unauthorized model access;
- cross-tenant data leakage;
- excessive resource consumption;
- sensitive-output leakage;
- tool abuse;
- model-routing manipulation;
- context poisoning;
- malicious retrieved content;
- unsafe generated content;
- denial-of-service;
- credential exposure.
Runtime security therefore requires defense in depth.
60.3 Request Lifecycle
A secure request lifecycle can be represented as:
Request
↓
Transport Validation
↓
Authentication
↓
Authorization
↓
Tenant Resolution
↓
Policy Evaluation
↓
Input Validation
↓
Context Construction
↓
Model Selection
↓
Inference
↓
Output Validation
↓
Audit
↓
Response
A request should not jump directly from HTTP input to model execution.
60.4 API Gateway
The gateway provides the first major runtime boundary.
It can enforce:
- request size;
- authentication;
- rate limits;
- content type;
- timeout;
- request IDs;
- routing;
- abuse controls.
Example:
Client
↓
API Gateway
├── Authentication
├── Rate Limit
├── Request Validation
└── Request ID
↓
AI Application
The gateway should reject obviously invalid requests before they consume expensive AI resources.
60.5 Authentication Context
After authentication, the server should establish a trusted identity context.
Example:
interface RequestIdentity {
userId: string;
tenantId?: string;
roles: string[];
sessionId: string;
}
This context should come from verified authentication infrastructure.
Client-provided values such as:
{
"userId": "another-user",
"tenantId": "another-tenant"
}
must not override the authenticated identity.
60.6 Authorization Before Inference
Authorization should occur before model execution.
For example:
User
↓
Can access feature?
↓
Can use selected model?
↓
Can access requested data?
↓
Can call requested tools?
↓
Allowed
The AI model itself should not be responsible for deciding whether the user has permission.
60.7 Tenant Isolation
A multi-tenant AI platform requires strong isolation.
Example:
Tenant A
├── Users
├── Data
├── Models
└── Jobs
Tenant B
├── Users
├── Data
├── Models
└── Jobs
Runtime requests must preserve the tenant boundary throughout the entire pipeline.
60.8 Tenant Context Propagation
Tenant identity should propagate through internal services.
API
↓
Inference Service
↓
Retrieval Service
↓
Tool Service
↓
Storage
Each service should independently verify the trusted tenant context where appropriate.
A tenant ID should not merely be passed as an unverified string.
60.9 Model Routing
Modern AI applications may support multiple models.
For example:
Fast Model
Reasoning Model
Image Model
Video Model
Local Model
Cloud Model
A model router can choose the appropriate backend.
Request
↓
Policy Engine
↓
Model Router
├── Local
├── Cloud A
├── Cloud B
└── Specialized Model
The router should not allow users to bypass policy simply by specifying an internal model identifier.
60.10 Secure Model Selection
A client may request:
{
"model": "fast-model"
}
The server should interpret this as a request, not as an authorization decision.
The secure process is:
Requested Model
↓
Allowed for User?
↓
Allowed for Tenant?
↓
Allowed for Feature?
↓
Allowed by Policy?
↓
Route
60.11 Model Capability Registry
A model registry can contain capability metadata.
Example:
interface ModelCapability {
modelId: string;
supportsText: boolean;
supportsImage: boolean;
supportsVideo: boolean;
supportsTools: boolean;
maxContextTokens: number;
riskLevel: "low" | "medium" | "high";
}
The router can use these capabilities when selecting a model.
60.12 Model Routing and Privacy
Routing can also depend on data classification.
For example:
Public Data
↓
Cloud Model Allowed
Confidential Data
↓
Approved Private Model
Highly Sensitive Data
↓
Restricted Processing Environment
This prevents sensitive information from automatically being sent to every external provider.
60.13 Provider Abstraction
A secure AI application should use a provider abstraction.
interface AIProvider {
generate(request: AIRequest): Promise<AIResponse>;
}
Possible implementations:
OpenAIProvider
GeminiProvider
GroqProvider
HuggingFaceProvider
OllamaProvider
The application should not scatter provider-specific secrets and logic throughout the codebase.
60.14 Provider Credentials
Provider credentials should remain server-side.
Never expose provider API keys directly in:
- browser JavaScript;
- HTML;
- public configuration;
- client-side environment variables;
- generated prompts.
The secure architecture is:
Browser
↓
Application Backend
↓
Provider Credential
↓
AI Provider
60.15 Request Normalization
Before inference, requests should be normalized.
Possible checks:
- input encoding;
- maximum length;
- supported MIME types;
- structured fields;
- allowed model;
- allowed modality;
- token limits.
Normalization reduces ambiguity and unexpected parser behavior.
60.16 Prompt Construction
A production application may combine:
System Policy
+
Application Instructions
+
User Input
+
Retrieved Context
+
Tool Results
These inputs should not all have equal authority.
A useful conceptual hierarchy is:
Platform Policy
↓
Application Policy
↓
Trusted System Instructions
↓
User Request
↓
Retrieved / External Content
Untrusted content should not silently become a higher-priority instruction.
60.17 Indirect Prompt Injection
A user may not directly attack the model.
Instead, malicious instructions can exist inside:
- documents;
- web pages;
- PDFs;
- images;
- database records;
- search results;
- emails.
For example:
User
↓
"Summarize this document"
↓
Document contains malicious instructions
↓
Model reads document
The document is data.
It should not automatically become an instruction source.
60.18 Context Boundaries
The application should clearly distinguish:
INSTRUCTIONS
DATA
TOOLS
TOOL RESULTS
USER CONTENT
A conceptual context structure:
interface AIContext {
systemPolicy: string;
applicationInstructions: string;
userInput: string;
retrievedData: unknown[];
toolResults: unknown[];
}
The model should be instructed and architected to treat retrieved content as untrusted data.
60.19 Prompt Injection Defense
No single prompt can guarantee complete protection.
Use multiple layers:
Input Filtering
+
Context Separation
+
Tool Authorization
+
Output Validation
+
Least Privilege
+
Monitoring
+
Human Approval
The strongest defense is reducing what a compromised model can actually do.
60.20 Tool Boundary
An AI model may request tools such as:
Search
Database Query
File Processing
Email
Calendar
Image Generation
Payment
These tools should be treated as separate security domains.
The model should request an action.
The application should authorize it.
Model
↓
Tool Request
↓
Policy Engine
↓
Authorization
↓
Tool
60.21 Never Give the Model Raw Credentials
Bad architecture:
Model
↓
Cloud Credential
↓
Direct Infrastructure Access
Preferred:
Model
↓
Application Tool
↓
Policy Check
↓
Restricted Service Identity
↓
Specific Operation
The model receives capability through controlled interfaces, not unrestricted credentials.
60.22 Tool Allowlisting
For each feature, define allowed tools.
Example:
const allowedTools = {
imageEditor: [
"resizeImage",
"removeBackground",
"applyFilter"
],
documentAssistant: [
"searchDocuments",
"summarizeDocument"
]
};
A tool outside the allowlist should be denied.
60.23 Tool Argument Validation
Even an authorized tool request requires validation.
For example:
interface ResizeRequest {
width: number;
height: number;
}
The server should enforce:
width > 0
height > 0
width <= maximum
height <= maximum
Never assume model-generated arguments are safe.
60.24 Output Security
AI output should not automatically be trusted.
Outputs may contain:
- unsafe instructions;
- malicious URLs;
- sensitive information;
- malformed structured data;
- code;
- unexpected markup.
The application should validate outputs according to the destination.
60.25 Structured Output Validation
If the model is expected to return JSON, validate it.
Example:
interface GeneratedResult {
title: string;
description: string;
}
Use schema validation before accepting the result.
Conceptually:
Model Output
↓
JSON Parser
↓
Schema Validator
↓
Policy Check
↓
Application
60.26 HTML Output Security
If AI-generated text is rendered as HTML, output must be sanitized.
Never assume:
AI output = safe HTML
Instead:
AI output
↓
Sanitization
↓
Safe Rendering
For many applications, plain text or a restricted markup format is safer.
60.27 Code Generation Security
If the AI produces code, treat it as untrusted.
Do not automatically execute generated code in the application process.
A safer architecture is:
Generated Code
↓
Static Analysis
↓
Sandbox
↓
Resource Limits
↓
Execution
The sandbox should not have production credentials.
60.28 Generated SQL
AI-generated SQL is similarly untrusted.
Instead of directly executing arbitrary generated SQL against production databases:
Model
↓
SQL
↓
Production DB
use:
Model
↓
Structured Query Request
↓
Authorization
↓
Query Builder
↓
Restricted Database
This reduces injection and privilege risks.
60.29 Retrieval Security at Runtime
Runtime retrieval should enforce authorization before returning data.
User
↓
Authorized Retrieval
↓
Tenant Filter
↓
Document Permissions
↓
Relevant Chunks
↓
Model
Retrieval must not become a mechanism for bypassing application permissions.
60.30 Context Budget
Large contexts can cause:
- excessive cost;
- latency;
- resource exhaustion;
- reduced relevance.
Therefore enforce:
Maximum Input Tokens
Maximum Retrieved Documents
Maximum Tool Results
Maximum Output Tokens
This should be controlled server-side.
60.31 Runtime Rate Limiting
AI inference is expensive.
Rate limiting should consider:
- user;
- tenant;
- API key;
- IP where appropriate;
- endpoint;
- model;
- modality;
- cost;
- concurrency.
Example:
User Limit
+
Tenant Limit
+
Global System Limit
Multiple layers prevent a single abusive actor from consuming disproportionate capacity.
60.32 Cost-Aware Controls
Not every request should receive the most expensive model.
A policy may define:
Simple Request
↓
Low-Cost Model
Complex Request
↓
Advanced Model
High-Cost Request
↓
Additional Authorization / Quota Check
This protects both financial resources and system availability.
60.33 Runtime Queue
Long-running inference should often be asynchronous.
Request
↓
Queue
↓
Worker
↓
Model
↓
Result Store
↓
Client Notification
This provides better control over:
- concurrency;
- retries;
- cancellation;
- resource allocation.
60.34 Job Isolation
Each inference job should have a controlled execution context.
Example:
Job
├── Tenant Context
├── Model Version
├── Input References
├── Resource Limits
├── Timeout
└── Authorization Context
Jobs should not inherit unnecessary permissions from the worker host.
60.35 Timeouts
Every external or internal operation should have bounded execution.
Examples:
API Timeout
Model Timeout
Retrieval Timeout
Tool Timeout
Storage Timeout
Without timeouts, a small number of stuck operations can consume system resources.
60.36 Cancellation
Users should be able to cancel long-running operations where practical.
The system should propagate cancellation:
User Cancels
↓
API
↓
Queue
↓
Worker
↓
Model Operation
Cancellation also helps control resource consumption.
60.37 Runtime Isolation
High-risk AI workloads may require stronger isolation.
Possible mechanisms include:
- containers;
- dedicated worker pools;
- restricted namespaces;
- sandboxed runtimes;
- network policies;
- separate service identities.
The appropriate level depends on the workload.
60.38 Network Restrictions
An inference worker should not automatically have unrestricted outbound Internet access.
Possible architecture:
Inference Worker
│
├── Approved AI Provider
├── Approved Storage
└── Approved Internal Services
Everything else should be denied by default where practical.
This reduces data-exfiltration risk.
60.39 Metadata Security
AI requests may contain metadata such as:
- user ID;
- tenant ID;
- file ID;
- project ID;
- model ID;
- request ID.
Metadata should be validated and authorized.
Never trust metadata merely because it came from an internal service.
60.40 Caching Security
AI applications may cache:
- prompts;
- embeddings;
- model responses;
- generated assets;
- retrieval results.
Caches must respect authorization boundaries.
Bad:
cache[prompt]
Safer conceptual keying:
cache[tenantId:userId:resourceVersion:request]
Sensitive cached content should have appropriate expiration and deletion policies.
60.41 Response Caching Risks
A response generated for Tenant A must never be returned to Tenant B because both users submitted similar prompts.
Therefore cache keys and authorization must be designed together.
60.42 Logging Runtime Requests
Useful observability fields include:
requestId
tenantId
userId
modelVersion
provider
latency
tokenUsage
status
policyDecision
toolCount
Avoid logging sensitive prompt or output content by default.
60.43 Runtime Detection
Detection signals can include:
- sudden request spikes;
- repeated authorization failures;
- unusual model switching;
- excessive tool requests;
- repeated prompt injection attempts;
- unusual token consumption;
- abnormal output patterns;
- repeated failed validations.
These signals can trigger investigation or automated controls.
60.44 Runtime Containment
When an AI runtime appears compromised:
Detection
↓
Rate Limit
↓
Disable Risky Capability
↓
Isolate Tenant / Session if Needed
↓
Preserve Evidence
↓
Investigate
↓
Recover
Containment should be proportional to the incident.
60.45 Circuit Breakers
Circuit breakers can prevent repeated failures from overwhelming downstream services.
Example:
Provider Failures
↓
Threshold Reached
↓
Circuit Open
↓
Temporary Fallback
This is particularly useful for external AI providers.
60.46 Provider Failover
A multi-provider architecture can provide resilience:
Primary Provider
↓
Failure
↓
Policy Check
↓
Approved Backup Provider
However, failover should respect data-classification policies.
Sensitive data should not automatically move to a provider that is not authorized to process it.
60.47 Local AI
For privacy-sensitive workloads, a local model may be preferred:
Sensitive Request
↓
Local AI
while less-sensitive workloads may use:
General Request
↓
Cloud AI
This should be controlled by policy rather than arbitrary user selection.
60.48 Multimodal Runtime Security
Modern AI systems may process:
- text;
- images;
- audio;
- video;
- documents.
Each modality can contain adversarial content.
A secure pipeline should therefore preserve the same principles:
Validate
↓
Classify
↓
Authorize
↓
Process
↓
Validate Output
60.49 Image Input
Image inputs may require:
- file-type validation;
- size limits;
- decoding isolation;
- metadata handling;
- malware scanning;
- resource limits.
An image should not be trusted simply because its filename ends in .jpg or .png.
60.50 Audio and Video Input
Media processing may be computationally expensive.
Controls should include:
- duration limits;
- resolution limits;
- codec validation;
- processing timeout;
- isolated workers;
- temporary storage limits.
This reduces denial-of-service risk.
60.51 Prompt and File Relationship
A common architecture is:
User
↓
Prompt
+
Uploaded File
↓
Multimodal Model
The file should remain untrusted data.
Instructions found inside the file should not automatically override the application's trusted policy.
60.52 Secure AI Runtime Policy
A central policy engine can evaluate:
interface RuntimePolicyInput {
userId: string;
tenantId: string;
modelId: string;
dataClassification: string;
requestedTools: string[];
estimatedCost: number;
}
The policy engine returns something like:
interface RuntimePolicyDecision {
allowed: boolean;
allowedModel: boolean;
allowedTools: string[];
maxTokens: number;
maxRuntimeMs: number;
requiresApproval: boolean;
}
This creates a consistent decision layer.
60.53 Runtime Authorization Example
Conceptually:
async function authorizeInference(
identity: RequestIdentity,
request: AIRequest
) {
const policy = await evaluateRuntimePolicy({
userId: identity.userId,
tenantId: identity.tenantId!,
modelId: request.model,
dataClassification: request.dataClassification,
requestedTools: request.tools ?? [],
estimatedCost: request.estimatedCost
});
if (!policy.allowed) {
throw new Error("Inference denied");
}
return policy;
}
Production implementations should also enforce schema validation, quota checks, audit logging, and concurrency controls.
60.54 Secure Runtime Architecture
A complete reference architecture:
USER
│
▼
API GATEWAY
│
┌───────────┴───────────┐
▼ ▼
Authentication Rate Limiting
│
▼
Authorization
│
▼
POLICY ENGINE
│
▼
REQUEST VALIDATION
│
▼
MODEL ROUTER
│ │ │
▼ ▼ ▼
Local Cloud Specialized
│ │ │
└────┼────┘
▼
AI RUNTIME
│ │ │
│ │ └── Tools
│ └────── Retrieval
└────────── Context
│
▼
OUTPUT VALIDATION
│
▼
AUDIT / METRICS
│
▼
RESPONSE
60.55 Runtime Security Checklist
Request
- [ ] Authentication
- [ ] Authorization
- [ ] Tenant verification
- [ ] Input validation
- [ ] Size limits
- [ ] Rate limits
Model
- [ ] Approved model only
- [ ] Model version verification
- [ ] Secure routing
- [ ] Provider policy
- [ ] Data-classification enforcement
Context
- [ ] Instruction/data separation
- [ ] Retrieval authorization
- [ ] Context limits
- [ ] Prompt-injection defenses
Tools
- [ ] Tool allowlist
- [ ] Argument validation
- [ ] Least privilege
- [ ] Tool-specific authorization
- [ ] No raw credentials
Runtime
- [ ] Resource limits
- [ ] Timeouts
- [ ] Cancellation
- [ ] Network restrictions
- [ ] Job isolation
Output
- [ ] Schema validation
- [ ] Output policy
- [ ] Sanitization
- [ ] Sensitive-data checks
Operations
- [ ] Monitoring
- [ ] Audit logs
- [ ] Circuit breakers
- [ ] Incident response
- [ ] Rollback/failover
60.56 Final Principle
Secure AI inference is not achieved by finding a perfect prompt.
It is achieved by designing the runtime so that even when an input is malicious or the model behaves unexpectedly, the resulting impact remains limited.
The strongest architecture is:
Authenticate
↓
Authorize
↓
Validate
↓
Minimize
↓
Isolate
↓
Infer
↓
Validate Output
↓
Monitor
↓
Contain When Necessary
The model should provide intelligence, but the surrounding application remains responsible for authority, security, permissions, isolation, and enforcement.
That distinction is one of the most important foundations of a production-grade AI security architecture.
Top comments (0)