Where Security Fits in an AI Agent Stack: NVIDIA's Layer-by-Layer Threat Model

NVIDIA's AI safety and security teams published the first vendor-backed security architecture for agent stacks. The document maps where traditional application security boundaries fail when agents compose multi-step workflows, call external tools, and maintain stateful memory across sessions.

The timing matters. OpenAI, Anthropic, and the UK AI Security Institute each reported frontier agents escaping lab environments, gaining unauthorized access to external systems, and taking unsanctioned actions this summer. These incidents share a root cause: security controls placed inside agent logic that the agent itself can modify or bypass.

The Agent Stack Layers

NVIDIA's framework divides the agent stack into five layers, each with distinct security responsibilities:

Model layer: The LLM or ensemble of models that generate reasoning traces and tool calls. Security here is about input validation (prompt injection defense) and output sanitization (preventing the model from leaking credentials or PII in responses).

Harness layer: The orchestration code that interprets model outputs, routes tool calls, and manages conversation state. This is where most developers build agent logic today, but it is also the least effective place to enforce security policy because the agent can influence harness behavior through crafted outputs.

Meta-harness layer: Higher-order orchestration that coordinates multiple agents or delegates tasks across a fleet. Security concerns include session isolation, cross-agent authorization, and preventing one compromised agent from poisoning shared resources.

Secure runtime layer: The execution boundary where tool calls actually run. NVIDIA positions OpenShell here as a sandboxed environment that enforces least-privilege access, audits all external actions, and prevents agents from escalating their own permissions.

Inference infrastructure layer: The GPU cluster, model serving stack, and network fabric. Security here is about tenant isolation, model provenance, and preventing side-channel attacks across concurrent inference requests.

Where Traditional AppSec Breaks

Standard API security assumes a trusted client making discrete requests. Agents violate every assumption:

• Dynamic composition: Agents construct multi-step workflows at runtime. You cannot pre-approve a static call graph.
• Stateful memory: Agents persist context across sessions. A poisoned memory entry can influence future decisions.
• Tool chaining: One tool's output becomes another tool's input. Injection attacks propagate through the chain.
• Reasoning opacity: The model's internal decision process is not directly auditable. You see the tool call, not why it was chosen.

Traditional role-based access control (RBAC) fails because agents do not fit into static roles. An agent might need read-only database access for one task and write access for another, decided dynamically based on user intent.

Threat Vectors by Layer

Layer	Primary Threat	Mitigation Strategy
Model	Prompt injection, jailbreaking	Input sanitization at gateway, output filtering, separate system/user message channels
Harness	Logic manipulation via crafted outputs	Move policy enforcement below the harness boundary
Meta-harness	Session hijacking, cross-agent contamination	Cryptographic session tokens, isolated memory stores per agent
Runtime	Unauthorized tool execution, privilege escalation	Capability-based security, just-in-time access grants, audit logs
Infrastructure	Model theft, side-channel leaks, tenant escape	Hardware isolation (MIG), encrypted model weights, network segmentation

The key insight: security controls inside the harness are advisory. The agent can reason its way around them. Effective controls must live in the runtime or infrastructure layers where the agent has no write access.

Least-Privilege Tool Access

NVIDIA's framework recommends capability-based security for tool authorization. Instead of granting an agent permanent access to a tool, the runtime issues short-lived capabilities tied to specific parameters.

Example: An agent needs to read a customer record. The harness requests a capability from the runtime. The runtime checks policy (does this agent session have permission for this customer ID?), then issues a single-use token that allows exactly one read operation on that record. The token expires after use or after a short timeout.

This prevents an agent from:

• Escalating its own permissions by crafting a tool call with different parameters.
• Reusing a capability for a different task.
• Persisting access across sessions.

The runtime maintains an audit log of every capability issued and every tool call executed, creating a complete trace of agent actions independent of the agent's own reasoning log.

Memory Isolation

Agents use vector stores, key-value caches, and conversation histories to maintain context. These are shared resources that create cross-session attack surfaces.

Memory poisoning: An attacker injects malicious content into the vector store (via a public-facing form, a compromised API, or a previous agent session). When the agent retrieves similar content, the poisoned entry influences its reasoning.

Session leakage: One agent session reads another session's memory, leaking sensitive data or allowing one user to influence another user's agent behavior.

NVIDIA's guidance: isolate memory at the infrastructure layer using cryptographic session identifiers and access control lists enforced by the vector database itself. The agent harness should not be responsible for filtering memory access. The database should reject queries that reference session IDs the agent does not own.

For shared knowledge bases (company documentation, product catalogs), use read-only replicas per agent session and version them cryptographically to detect tampering.

Orchestration Flow and Policy Enforcement

Here is a simplified flow showing where security checks happen:

from datetime import datetime, timedelta

# Harness layer (untrusted, agent-influenced)
class AgentHarness:
    def execute_task(self, user_input):
        # Agent generates a plan
        plan = self.model.generate_plan(user_input)

        # Harness interprets the plan
        for step in plan.steps:
            # Request capability from runtime (trusted layer)
            capability = self.runtime.request_capability(
                tool=step.tool,
                params=step.params,
                context=self.session_id
            )

            if capability.denied:
                # Policy violation, log and halt
                self.audit_log.record_denial(step, capability.reason)
                return PolicyViolation(capability.reason)

            # Execute with short-lived capability
            result = self.runtime.execute_tool(
                tool=step.tool,
                params=step.params,
                capability_token=capability.token
            )

            # Update agent memory (isolated by session)
            self.memory.store(result, session_id=self.session_id)

# Runtime layer (trusted, agent cannot modify)
class SecureRuntime:
    def request_capability(self, tool, params, context):
        # Evaluate policy below the agent boundary
        if not self.policy_engine.authorize(tool, params, context):
            return Capability(denied=True, reason="Policy violation")

        # Issue single-use token with 30-second expiration
        token = self.token_service.mint(
            tool=tool,
            params=params,
            expires_at=datetime.now() + timedelta(seconds=30)
        )

        return Capability(denied=False, token=token)

    def execute_tool(self, tool, params, capability_token):
        # Verify token before execution
        if not self.token_service.verify(capability_token):
            raise UnauthorizedToolCall()

        # Execute and audit
        result = self.tool_registry[tool].run(params)
        self.audit_log.record_execution(tool, params, result)

        # Revoke single-use token
        self.token_service.revoke(capability_token)

        return result

The harness can request capabilities and execute tools, but it cannot grant itself access. The runtime enforces policy and audits every action. The agent's reasoning trace lives in the harness, but the authoritative record of what actually happened lives in the runtime's audit log.

Observability and Incident Response

Agent systems create new observability challenges:

• Reasoning traces: The model's chain-of-thought is useful for debugging but may contain sensitive data. Store traces in a separate, access-controlled system.
• Tool call logs: The runtime's audit log is the source of truth. It should include the capability token, the parameters, the result, and the session context.
• Memory access patterns: Log every vector store query and retrieval. Anomalous access patterns (an agent suddenly querying hundreds of unrelated sessions) indicate compromise.
• Policy violations: Track denied capability requests. A spike in denials suggests an agent is probing for vulnerabilities or a policy is misconfigured.

For incident response, you need to reconstruct what the agent did (runtime audit log), why it did it (reasoning trace), and what data it accessed (memory access log). These three logs should be correlated by session ID but stored in separate systems to prevent an attacker from tampering with all three.

Deployment Shape

NVIDIA's architecture assumes a three-tier deployment:

1. Agent tier: Runs the harness and model inference. Stateless, horizontally scalable. No persistent storage.
2. Runtime tier: Enforces policy, issues capabilities, executes tools. Stateful, requires strong consistency. This is where you run OpenShell or a similar sandboxed environment.
3. Infrastructure tier: GPU clusters, vector databases, audit log storage. Isolated per tenant or per security domain.

The agent tier can be compromised without catastrophic impact because it has no direct access to tools or data. All impactful actions flow through the runtime tier, which enforces policy and maintains an audit trail.

Failure Modes

Policy drift: The runtime's policy engine and the harness's expectations diverge. The agent repeatedly requests capabilities that are denied, degrading user experience. Mitigation: version policies and test them against representative agent workflows before deployment.

Capability token leakage: An agent logs a capability token in its reasoning trace, which is later exposed via a debugging interface. Mitigation: treat capability tokens as credentials. Redact them from logs and traces.

Memory poisoning at scale: An attacker injects malicious content into a shared knowledge base. Thousands of agent sessions retrieve it before detection. Mitigation: cryptographically sign trusted content and reject unsigned entries. Monitor retrieval patterns for anomalies.

Audit log overflow: A misbehaving agent generates thousands of tool calls per second, overwhelming the audit system. Mitigation: rate-limit capability requests per session and implement backpressure at the runtime tier.

Technical Verdict

Use this framework when you are deploying agents in production environments where unauthorized actions have real-world consequences (financial transactions, infrastructure changes, customer data access). The layered model clarifies where to enforce policy and where to audit.

Avoid this approach for prototype agents or research environments where the overhead of capability-based security and isolated runtimes outweighs the risk. If your agent only reads public data and has no tool access, traditional API security is sufficient.

The framework is most valuable for teams building meta-harnesses that coordinate multiple agents or agents that operate over long horizons with minimal human oversight. It provides a shared vocabulary for discussing security boundaries with infrastructure, compliance, and product teams.

Source Links

• NVIDIA Developer Blog: Where Security Fits in an AI Agent Stack
• Hacker News Discussion