Building Guardrails for Enterprise LLMs That Actually Work

The term "guardrails" gets used loosely in conversations about enterprise AI safety. Some people mean input filtering. Some mean output filtering. Some mean a combination of both with a constitutional AI layer and a human review queue thrown in. The imprecision matters because different guardrail approaches address different failure modes, and deploying the wrong one gives you a false sense of security.

I want to describe the specific guardrail architecture I have landed on for enterprise RAG deployments after getting this wrong in several different ways and learning from each one.

The failure modes guardrails are actually trying to address

Before building anything, you need to be specific about what you are preventing. Guardrails that are not targeted at specific failure modes end up either too restrictive, blocking legitimate use cases, or too permissive, missing the actual problems.

The failure modes I treat as in-scope for enterprise knowledge management deployments:

Retrieval of content the requesting user is not authorized to see. This is an access control problem at the retrieval layer, not a guardrail problem, but it often gets misclassified as something guardrails should address. The correct solution is authorization enforcement before retrieval, not content filtering after generation.

Generation of responses that misrepresent organizational facts. The AI produces an answer that sounds authoritative but does not reflect what the source documents actually say. The guardrail approach here is groundedness checking: verifying that claims in the generated output are supported by the retrieved context.

Prompt injection through document content. A document in the knowledge base contains instruction-like text designed to override the system prompt or change the AI's behavior. The guardrail approach is input sanitization and structural separation between system instructions and retrieved content.

Out-of-scope responses. The AI answers questions that are outside the intended scope of the deployment, either drawing on general training knowledge rather than organizational documents, or engaging with topics unrelated to the deployment's purpose. The guardrail approach is scope enforcement through both system prompt design and output classification.

Sensitive information leakage through inference. A user does not directly retrieve a restricted document but asks a question that the AI can only answer correctly by drawing on information from that document. This is the hardest guardrail problem because it requires reasoning about the provenance of generated content, not just filtering explicit retrievals.

Layer one: Input processing

The first guardrail layer operates on user queries before they reach the retrieval system.

import re
from typing import Tuple

class InputGuardrail:
    INJECTION_PATTERNS = [
        r"ignore (previous|above|all) instructions",
        r"you are now",
        r"forget your (system|previous) (prompt|instructions)",
        r"act as (if you are|a|an)",
        r"disregard.*instructions",
        r"new instructions:",
        r"override.*system",
    ]

    MAX_QUERY_LENGTH = 2000  # tokens approximately

    def __init__(self, scope_classifier, pii_detector):
        self.scope_classifier = scope_classifier
        self.pii_detector = pii_detector

    def process(self, query: str, user_context: dict) -> Tuple[str, dict]:
        flags = {}

        # Check for injection attempts
        for pattern in self.INJECTION_PATTERNS:
            if re.search(pattern, query.lower()):
                flags["injection_attempt"] = True
                flags["matched_pattern"] = pattern
                break

        # Check query length
        if len(query.split()) > self.MAX_QUERY_LENGTH:
            flags["query_too_long"] = True

        # Check if query is in scope
        scope_result = self.scope_classifier.classify(query)
        if scope_result.confidence > 0.8 and not scope_result.in_scope:
            flags["out_of_scope"] = True
            flags["scope_confidence"] = scope_result.confidence

        # Check for PII in query that should not be there
        pii_detected = self.pii_detector.detect(query)
        if pii_detected:
            flags["pii_in_query"] = True
            flags["pii_types"] = pii_detected

        return query, flags

    def should_block(self, flags: dict) -> Tuple[bool, str]:
        if flags.get("injection_attempt"):
            return True, "Query contains patterns associated with instruction injection."

        if flags.get("query_too_long"):
            return True, "Query exceeds maximum length."

        return False, ""

The injection pattern matching is a necessary but insufficient defense. Sophisticated injection attempts will not match simple regex patterns. The pattern matching catches the obvious cases and logs flagged queries for review, but should not be relied on as the sole defense against prompt injection.

Layer two: Retrieved content sanitization

Before retrieved document chunks are assembled into the prompt context, they pass through a sanitization step that checks for instruction-like content.

class RetrievedContentGuardrail:
    SUSPICIOUS_CONTENT_PATTERNS = [
        r"(system|assistant|user):",
        r"\[INST\]",
        r"<s>",
        r"ignore.*previous",
        r"new task:",
        r"<\|.*\|>",  # various prompt formatting markers
    ]

    def sanitize_chunk(self, chunk_text: str, doc_metadata: dict) -> Tuple[str, bool]:
        flagged = False
        sanitized = chunk_text

        for pattern in self.SUSPICIOUS_CONTENT_PATTERNS:
            if re.search(pattern, chunk_text, re.IGNORECASE):
                flagged = True
                # Log but do not necessarily remove - some patterns may be legitimate
                self.log_suspicious_content(chunk_text, pattern, doc_metadata)

        return sanitized, flagged

    def build_safe_context(self, retrieved_chunks: list) -> str:
        # Structurally separate retrieved content from instructions
        # using clear delimiters that are unlikely to appear in documents
        safe_parts = []
        for i, (chunk, metadata) in enumerate(retrieved_chunks):
            sanitized, flagged = self.sanitize_chunk(chunk, metadata)
            if not flagged or self.allow_flagged_content(metadata):
                safe_parts.append(
                    f"[SOURCE {i+1}: {metadata.get('title', 'Unknown')}]\n{sanitized}\n[END SOURCE {i+1}]"
                )

        return "\n\n".join(safe_parts)

The structural separation with explicit delimiters is more important than the pattern matching. An LLM that is given a clear structural distinction between "these are instructions" and "this is retrieved content" is more resistant to injection than one that receives a flat prompt where instructions and content are interleaved.

Layer three: Output validation

Generated responses pass through two output validation steps before being returned to the user.

class OutputGuardrail:
    def __init__(self, groundedness_checker, scope_classifier, sensitivity_classifier):
        self.groundedness_checker = groundedness_checker
        self.scope_classifier = scope_classifier
        self.sensitivity_classifier = sensitivity_classifier

    def validate(
        self,
        response: str,
        query: str,
        retrieved_context: str,
        user_permissions: list
    ) -> Tuple[str, dict]:
        validation_results = {}

        # Check 1: Is the response grounded in the retrieved context?
        groundedness = self.groundedness_checker.check(
            response=response,
            context=retrieved_context
        )
        validation_results["groundedness_score"] = groundedness.score
        validation_results["ungrounded_claims"] = groundedness.ungrounded_claims

        # Check 2: Does the response contain sensitivity markers suggesting
        # it drew on content the user may not be authorized for?
        sensitivity = self.sensitivity_classifier.classify(response)
        if sensitivity.tier > max(self.tier_for_permission(p) for p in user_permissions):
            validation_results["potential_unauthorized_content"] = True
            validation_results["detected_sensitivity_tier"] = sensitivity.tier

        # Check 3: Is the response in scope?
        if not self.scope_classifier.classify(response).in_scope:
            validation_results["response_out_of_scope"] = True

        return self.apply_validation_results(response, validation_results), validation_results

    def apply_validation_results(self, response: str, results: dict) -> str:
        if results.get("potential_unauthorized_content"):
            return "I cannot provide information on this topic. Please contact your administrator if you believe you should have access."

        if results.get("groundedness_score", 1.0) < 0.6:
            # Low groundedness - add explicit uncertainty marker
            return f"{response}\n\n[Note: Some claims in this response may not be fully supported by the available documentation. Please verify with primary sources.]"

        if results.get("response_out_of_scope"):
            return "This question is outside the scope of what I can help with in this context."

        return response

The groundedness threshold of 0.6 is a starting point, not a universal answer. The right threshold depends on your use case and the cost of false positives (blocking valid responses) versus false negatives (allowing ungrounded responses). For high-stakes queries in regulated contexts, you want a higher threshold. For general knowledge retrieval, a lower threshold avoids over-blocking.

The failure mode this architecture does not address

I want to be honest about the limit of what output-layer guardrails can do for the sensitive information inference problem.

If a user asks "what is the most sensitive project currently underway at this company," and the AI can answer that question by synthesizing information from documents the user has access to, there is no guardrail that catches this cleanly. The user is authorized for each source document. The synthesized answer might reveal something that no individual document reveals. The guardrail does not know what inferences the user might draw from the response.

This problem is genuinely hard and I have not seen a satisfying general solution. The approaches that reduce the risk:

Query intent classification that flags synthesis queries involving organizational sensitivity assessments and routes them to human review. This adds latency and requires human reviewer capacity, but it catches the class of query where inference risk is highest.

Restricting the AI's ability to reason about organizational structure, hierarchy, or competitive positioning from document content. Implemented through system prompt constraints and output classification, this reduces the utility of the system for some legitimate queries but also reduces the most concerning inference pathways.

Accepting that a sufficiently determined insider can use an AI system to learn things a naive user would not, and designing the access control architecture (which documents are indexed and accessible to which users) to minimize the sensitivity of what can be inferred rather than trying to catch all inferences at generation time.

The guardrail architecture I described above handles most of the common failure modes reasonably well. For the inference problem, the honest answer is that it requires architectural decisions about what you index and who can access it, not just guardrails on the output.