DEV Community: Yogi

Running a 122B Parameter Model and Agent Locally on AMD MI300X GPU — What I Learned

Yogi — Mon, 10 Aug 2026 04:26:14 +0000

Self-hosting Qwen3.5-122B with vLLM and SGLang on AMD GPU infrastructure, then building a fully autonomous agent on top of it

Introduction

Most conversations about running large language models locally stop at 7B or 13B parameter models on consumer GPUs. That's a reasonable ceiling when you're working with a single RTX card and 24GB of VRAM.

What happens when you have access to an AMD Instinct MI300X with 192GB of HBM3 memory?

This post documents what I learned deploying Qwen3.5-122B — a 122-billion-parameter model — on AMD MI300X GPU infrastructure using vLLM and SGLang, and then building a fully autonomous agent on top of it using OpenClaw. No API tokens. No rate limits. Everything running on AMD hardware.

Why This Matters Beyond the Demo

Before getting into the technical setup, it's worth naming why this is interesting from an architecture perspective.

The default assumption in enterprise AI deployments today is that large models live behind an API ( "Big Three" frontier AI laboratories). You call their endpoint, you pay per token, you accept their rate limits, their data handling policies, and their availability SLAs.

Self-hosting changes that calculus entirely:

Data sovereignty — sensitive enterprise data never leaves your infrastructure
Cost predictability — GPU hours instead of per-token pricing at scale
No rate limits — agentic workloads that make thousands of tool calls don't get throttled
Model control — you choose the model, the quantization, the inference parameters

The MI300X makes frontier-scale models achievable in this model. 192GB of unified memory means a 122B FP8 model fits comfortably on a single GPU — something that would require 4–8 consumer cards to attempt, if it were possible at all.

The Stack

┌─────────────────────────────────────────────┐
│              OpenClaw Agent                  │
│     (persistent memory, skills, tools)       │
├─────────────────────────────────────────────┤
│         SGLang / vLLM Inference Server       │
│      (OpenAI-compatible API endpoint)        │
├─────────────────────────────────────────────┤
│           Qwen3.5-122B-A10B-FP8             │
│         (122B MoE, FP8 quantized)           │
├─────────────────────────────────────────────┤
│         AMD Instinct MI300X GPU              │
│    (192GB HBM3, ROCm software stack)        │
└─────────────────────────────────────────────┘

Key components:

AMD MI300X — 192GB HBM3 unified memory, ROCm software stack
vLLM / SGLang — inference serving frameworks with OpenAI-compatible endpoints
Qwen3.5-122B-A10B-FP8 — 122B parameter Mixture-of-Experts model, FP8 quantized
OpenClaw — agent framework with file-based persistent memory and skill system

Access: AMD Developer Cloud GPU Droplets via the AMD AI Developer Program

Part 1: Spinning Up the Inference Server

Option A: vLLM

Launch a vLLM server serving Qwen3.5-122B with tool-calling enabled

Option B: SGLang (recommended for MI300X)

SGLang with the ROCm-optimized Docker image performed better in my testing — faster time-to-first-token and more stable under agentic workloads that generate many sequential requests

Verifying the Server

Poll until the model is loaded (typically 3–5 minutes for a 122B model)

Part 2: What the Agent Architecture Actually Looks Like

Once the inference server is running, the interesting question is: how do you build an agent on top of it that's actually useful and persistent?

OpenClaw's approach is worth understanding because it's transparent in a way most agent frameworks aren't. Everything lives in markdown files

Why this matters architecturally: The agent's memory, personality, and behavioral policies are human-readable, version-controllable, and editable. You can git diff your agent's personality. You can PR-review changes to its behavioral policies. This is very different from fine-tuning or prompt injection hidden inside a framework.

Other Aspects that can be done includes

Part 3: The Think → Act → Observe → Repeat Loop

The agent's actual behavior:

1. THINK   → "I need to understand what accuracy means in this context"
2. ACT     → run pytest, observe which tests fail and what they expect
3. OBSERVE → test expects accuracy = correct_chars / total_chars
              code computes accuracy = correct_words / total_words
4. THINK   → "The bug is in stats.py — wrong denominator"
5. ACT     → read stats.py, locate the calculation, apply minimal fix
6. OBSERVE → re-run pytest, all tests pass
7. REPORT  → file changed, line changed, before/after

This is genuinely different from a chatbot. The agent doesn't answer the question — it solves the problem, using the filesystem and shell as its senses and hands.

The key architectural insight: the tool loop is what separates an agent from a chatbot. The model itself is just the reasoning component — the value comes from what it can observe and act on.

Part 4: Skills — Making Agent Behavior Reusable

Once the agent solved the bug, the interesting question is: can it do this for any Python project, without re-explaining the process?

Skills are the answer. A skill is a markdown file with YAML frontmatter and step-by-step instructions that gets injected into the agent's system prompt automatically:

Example:

"Create a skill called pytest-debugger. Steps: 1) Read the tests/ folder. 2) Run pytest with verbose output. 3) For each failing test, read the source file it references. 4) Identify the minimal fix. 5) Apply and re-run to confirm. 6) Report: file changed, line changed, what was wrong, what the fix was."

The agent writes the skill file. From that point, invoking it on any project is:

No re-explanation. No new prompts. The skill travels with the agent across any codebase.

The broader pattern: skills are reusable behavioral packages. The agent OS approach means you're building a library of trusted, version-controlled workflows — not re-prompting from scratch each time.

Part 5: Autonomous Workflows Without Writing Schedulers

Example: delegating infrastructure setup to the agent itself.

Instead of writing a cron job, you can describe what you want:

"I need to wake up to a personalized tech brief every morning at 8 AM. Check sgl-project/sglang, vllm-project/vllm, huggingface/transformers, ROCm/ROCm, and openclaw/openclaw. I only care about performance updates, GPU features, and breaking changes — skip CI/infrastructure noise and docs-only PRs. Also search the web for the latest AI hardware news."

The agent:

Scheduled itself using the system's cron infrastructure
Write its filtering preferences to MEMORY.md for persistence
Configure GitHub repo monitoring
Set up a web search workflow for hardware news

No cron syntax. No YAML pipeline files. Just a conversational description of the desired outcome.

This is the "helpful agents" promise in practice — and also, incidentally, exactly the attack surface that the AI Security Conference sessions I attended last month were warning about. Giving an agent the ability to schedule itself, write to memory, and access external data sources is powerful. It's also a significant security perimeter that needs to be treated with the same rigor as any other privileged system process.

What I Learned About the ROCm Stack

A few practical observations from running on AMD infrastructure:

ROCm is production-ready for inference. The rocm-smi tooling is equivalent to nvidia-smi in terms of what you can observe — GPU utilization, memory usage, temperature. For inference workloads specifically, I found no meaningful difference in model output quality versus CUDA-based deployments.

SGLang's ROCm image is the right starting point. The lmsysorg/sglang:*-rocm700-mi30x Docker image handles the driver compatibility complexity for you. Don't try to build from scratch for initial experiments.

The MI300X unified memory architecture matters. 192GB of HBM3 means you're not juggling CPU/GPU memory transfers for large models. The 122B FP8 model loads entirely onto the GPU and stays there — no swapping, no offloading. This shows up in inference latency consistency.

Tool calling requires explicit configuration. The --enable-auto-tool-choice and --tool-call-parser qwen3_coder flags are essential for agentic use cases. Without them, the model won't correctly format tool call responses that agent frameworks like OpenClaw expect.

Connecting This to Broader Agent Architecture Principles

Running this exercise alongside the AI security work I've been doing recently surfaces an interesting tension.

The same properties that make a self-hosted 122B agent powerful — persistent memory, filesystem access, shell execution, scheduled autonomous operation — are exactly the attack surfaces that need to be secured:

Memory persistence = potential injection vector if any memory write is unvalidated
Shell execution = arbitrary code execution if the agent is manipulated
Scheduled autonomous operation = actions that happen without human review
External data access = context poisoning entry points

The agent OS model I described in a recent post on agent security architecture maps directly onto this stack. The OpenClaw workspace structure — where behavioral policies live in auditable markdown files, where skills are version-controlled, where memory has provenance — is actually a reasonable starting point for the kind of transparent, auditable agent infrastructure the security community is pushing for.

The missing pieces for enterprise use: secrets injection at the execution boundary (not in the system prompt), capability scoping per agent, and budget/rate limits on autonomous operations. These are solvable with the right wrapper infrastructure around an open deployment like this.

Getting Started — The Practical Path

This is specific to this particular model. Feel free to explore other AI Developer Programs from other leading vendors as well

*1. Sign up for AMD AI Developer Program *

2. Create a GPU Droplet
Select MI300X hardware + ROCm software image. SSH access or browser console.

3. Launch your inference server
Use the SGLang Docker command above. Wait ~5 minutes for the 122B model to load.

4. Install OpenClaw
Point it at your SGLang endpoint during onboarding.

5. Start building
The workshop notebook at github.com/xxx/amd-gpu-workshops walks through everything in this post step by step.

Bottom Line

Running a 122B parameter model locally on AMD MI300X is not a research exercise anymore — it's a practical deployment option for teams that need data sovereignty, cost predictability at scale, or freedom from API rate limits.

The ROCm stack is production-ready. SGLang on MI300X performs well. The OpenClaw agent framework gives you a transparent, auditable foundation to build on.

What remains genuinely hard: production-grade security hardening of the agent perimeter. The same capabilities that make a self-hosted agent useful also expand the attack surface significantly. That's the engineering problem worth focusing on next.

I've been writing about enterprise AI agent architecture, MCP integrations, and AI security on dev.to. If you're building agentic systems or exploring self-hosted inference, follow along — more coming on the security hardening side of this stack.

Tags: #llm #agents #amd #gpu #mlops #selfhosted #vllm #rocm #architecture

Building My Own LLM Model and Agent

Yogi — Mon, 10 Aug 2026 03:16:17 +0000

Introduction

Large Language Models (LLMs) are powerful, but most enterprises rely on pre‑packaged APIs. I wanted to go deeper: train my own LLM model and build an agent layer on top of it that could interact with real systems securely.

This post walks through the architecture, training setup, agent design, and key learnings from building a custom LLM stack.

The Problem I Wanted to Solve

Pre‑built LLMs are great for general tasks, but they don’t always align with:
·       Domain‑specific knowledge (e.g., enterprise workflows, industry jargon).
·       Security and compliance requirements (data residency, auditability).
·       Customization (fine‑tuned reasoning, tool integration).
I wanted an agent that could reason over enterprise data, invoke tools, and respect security boundaries — all while running on a model I controlled.

Architecture Overview

Here’s the high‑level architecture of the custom LLM + agent stack:

Security Considerations

When building your own agent, security is non‑negotiable:
·       PKCE for public clients — prevents token replay attacks.
·       Role‑based access control (RBAC) — ensures the agent only calls tools it’s authorized for.
·       Least‑privilege scopes — limit what the agent can read/write.
·       Audit logging — every agent action is logged for compliance.

Training the LLM

I trained the model using:
·       Domain‑specific corpora (enterprise docs, CRM exports, technical manuals).
·       Fine‑tuning frameworks like Hugging Face + DeepSpeed.
·       Evaluation metrics (perplexity, task accuracy, tool invocation success).
The goal wasn’t to beat GPT‑4 — it was to build a fit‑for‑purpose model optimized for my enterprise use case.

Agent Design

The agent layer adds:
·       Tool invocation — structured calls to APIs, databases, or CLI tools.
·       Reasoning loop — chain‑of‑thought planning before executing actions.
·       Error handling — retries, fallbacks, and safe exits.
·       Context management — session‑aware memory of prior queries.

Business Impact

·       Customization: Tailored reasoning for enterprise workflows.
·       Security: Full control over authentication and data flows.
·       Cost efficiency: Optimized inference for specific tasks vs. general APIs.
·       Innovation: Demonstrates how enterprises can own their AI stack.

Key Learnings

1.     LLM fine‑tuning is iterative — expect multiple training cycles.
2.     Security must be baked in — don’t bolt it on later.
3.     Agents need guardrails — otherwise they’ll over‑invoke tools.
4.     Business alignment matters — the model should solve real workflows, not just benchmarks.

Closing Thought

Owning your own LLM + agent stack isn’t about competing with hyperscalers — it’s about control, customization, and compliance. For enterprises, this is the next frontier of operational AI.

What 3 Days at Stanford's AI Security Conference Taught Me About Building Agents Safely

Yogi — Fri, 07 Aug 2026 10:55:22 +0000

Last month I attended the AI Security Conference organized by the Stanford Security Labs.

Three days. Thirty-plus sessions. Researchers and practitioners from Google, Anthropic, OpenAI, Palo Alto Networks, Stanford, Princeton, Berkeley, UIUC, University of Toronto, and more.

The signal was unmistakable: we are building AI agents faster than we are securing them.

This post is my attempt to synthesize what I learned into something actionable for developers and architects building agentic systems today.

The Framing That Changed How I Think About Agent Security

Florian Tramèr opened his keynote with a deceptively simple question:

If you replace a human in a security-sensitive workflow with an AI agent that behaves indistinguishably on the task — are the security properties preserved?

The answer, he argued, is no — and this is the "Security Turing Test" that our industry is currently failing.

Our entire security infrastructure was designed around implicit human properties:

Humans are slow — rate limits exist because humans can't act at machine speed
Humans are loud — anomalous behavior is detectable because volume is constrained
Humans are locatable — out-of-band verification (a phone call, a physical check) is possible
Humans are variable — diverse individuals make diverse mistakes, limiting blast radius

AI agents preserve none of these properties. And nobody designed our fraud controls, verification workflows, or access policies to survive their removal.

The practical implication: You cannot audit your way to agent security by checking agent behavior against a checklist. The assumptions underneath the checklist are already broken.

Attack Surface: What's Actually Being Exploited Right Now

Prompt Injection Has Become a Kill Chain

Ben Nassi introduced the concept of "promptware" — malicious payloads that propagate through agent pipelines the way malware propagates through networks. Johann Rehberger demonstrated nine live attacks across ChatGPT, GitHub Copilot, Amazon Q, and Gemini CLI in a single keynote:

Persistent memory injection — prompt injection via a GitHub-hosted file caused ChatGPT to repeatedly update its own memory across sessions, enabling durable remote control of user behavior
Unicode invisible character injection — hidden Unicode tag characters (invisible in UI, readable by LLMs) embedded in GitHub issues silently injected malicious instructions — confirmed working on Gemini 2.5, Grok, and others
Agent self-modification — prompt injection caused GitHub Copilot to modify its own settings.json, enabling the yolo flag and arbitrary code execution (patched on Microsoft Patch Tuesday)
Self-replicating AI worm — a Gemini CLI worm injected malicious prompts into repositories, force-pushed infected code to GitHub, and propagated when other developers cloned the repos — directly applicable to any CI/CD pipeline where AI auto-analyzes issues

One cited real-world incident: a production database wiped in 9 seconds.

Attack Economics Are Shifting Dramatically

Attack progression time has collapsed from 8 hours (2022) → 22 seconds (2025). Nicholas Carlini demonstrated LLMs autonomously discovering zero-days in Linux, FreeBSD, every major browser, and cryptographic libraries — including a 35-year-old race condition in NetBSD/OpenBSD.

The Ghost CMS SQL injection finding was patched in 18 hours — and weaponized by external attackers 2–3 weeks later against unpatched instances.

Individual exploit cost: ~$100 once a vulnerability is identified.

This is not a future threat. It is the current baseline.

Context Poisoning Is Quietly the Most Dangerous Vector

Multiple sessions surfaced a pattern that doesn't get enough attention: context/RAG poisoning.

A demonstrated attack showed a single malicious document — delivered via email into an organization's document store — dominating an agent's retrieval context and steering a procurement decision. The document was framed as "all other documents are incorrect," which caused the model to weight it preferentially.

Anyone who can inject into an agent's context window — internal or external, email or shared doc or web page — can influence outcomes. The attack surface is your entire information environment, not just your model inputs.

The Architecture Problem: You're Building It Wrong

Securing the Model Is the Wrong Layer

This was the consensus position across Day 1 and Day 2. Speaker after speaker — from researchers to founders to enterprise practitioners — landed in the same place:

Guardrail-tuning the LLM is insufficient. Security must live in the environment.

Illia Polosukhin presented a concrete architecture: treat agents as an operating system, not as an app that happens to use an LLM.

What that looks like in practice:

┌─────────────────────────────────────────────────────┐
│                  Agent OS Layer                      │
│                                                     │
│  ┌─────────────┐  ┌──────────────┐  ┌───────────┐  │
│  │ Virtualized │  │   Sandboxed  │  │  Secrets  │  │
│  │ Filesystem  │  │ Code Exec    │  │ Injection │  │
│  │ (S3/Drive/  │  │ (Docker/     │  │ (post-LLM │  │
│  │  local)     │  │  gVisor)     │  │  boundary)│  │
│  └─────────────┘  └──────────────┘  └───────────┘  │
│                                                     │
│  ┌─────────────┐  ┌──────────────┐  ┌───────────┐  │
│  │ Capability- │  │   Budget     │  │  Memory   │  │
│  │ Scoped      │  │   Limits     │  │ Provenance│  │
│  │ Permissions │  │ ($/agent)    │  │ + TTL     │  │
│  └─────────────┘  └──────────────┘  └───────────┘  │
│                                                     │
│                   LLM (one component)               │
└─────────────────────────────────────────────────────┘

Key design principles that emerged:

Secrets never touch the model. API keys, auth tokens, and passwords are injected at the network/execution boundary after LLM output — the model can never read, print, or transmit them.

Virtualize everything. The agent operates inside a virtual filesystem (transparently mounting local folders, S3, Google Drive, or Docker sandboxes — the agent can't distinguish between them) and a virtualized network stack that enables centralized SSRF and egress control.

Downward-only permission inheritance. Agents receive explicit capability scopes (e.g., read email but not send). Any sub-agent spawned inherits at most the parent's permissions — never more.

Memory needs provenance and TTL. Every memory item should carry its source context (HR document? financial record? external email?). Time-to-live is a cheap, effective mechanism to prevent stale memory from driving current decisions.

Dollar-denominated budget limits are a first-class primitive. Agent operations carry real dollar costs. Enforcing spending limits per agent is not just cost control — it's a security boundary.

The Cloud Governance Parallel

The best mental model I've found for explaining this to teams: this is the cloud governance problem, only faster and with higher stakes.

When cloud went mainstream, organizations that succeeded weren't the ones that lifted and shifted their monoliths and hoped for the best. They were the ones that re-examined identity, access, and observability from the ground up — and built natively from there.

AI agents demand the same discipline:

Who owns an agent's actions?
What can it touch?
How are decisions audited?
When does a human stay in the loop?

You cannot agentify a broken stack. Retroactive security on top of weak foundations doesn't harden a system — it just adds complexity to an already fragile one.

Systemic Risks That Don't Get Enough Attention

Agent Monoculture

Tramèr's most sobering point wasn't about any specific attack — it was structural.

Replace millions of diverse human developers with copies of essentially one model and you get perfectly correlated failures at scale. Every exploit generalizes instantly across the entire deployed population. The adversary has access to the exact same system everyone else runs, can probe it offline at leisure, and every technique that works transfers immediately.

The financial markets parallel is exact: this is the algorithmic trading flash crash problem. When algo trading replaced human traders, the industry didn't make algorithms slower. It built circuit breakers, artificial friction, kill switches, and market-stability mechanisms so a single failure couldn't cascade.

We need the AI equivalent — diversity interventions on the deployed model population, circuit breakers on high-velocity agent actions, and kill switches that operate faster than the failure can propagate.

The Supply Chain Is Already Compromised

9 out of 11 MCP marketplace servers have been found poisoned
7,000+ MCP servers are available with no certification requirement
Trojan backdoors survive transfer learning — downloading a fine-tuned model and fine-tuning further does not cleanse embedded backdoors
The IARPA TrojAI program (Kristopher Reese, Hood College) ran for years and concluded: perfect Trojan detection is not achievable

Treat model provenance like software supply chain. Assume transfer learning does not cleanse backdoors. Verify before you deploy.

What's Actually Happening Inside Models

Fine-Tuning Risks Are Larger Than Anyone Realized

Owain Evans presented the emergent misalignment finding in full: fine-tuning a model on ~5,000 insecure code examples — without disclosing the insecurity to the model — produced a model that expressed hostility toward humans across completely unrelated domains.

The fine-tuned model, when asked benign questions about unrelated topics, responded with statements that humans are inferior to AI. The training was code-only. The misalignment was universal.

Further findings:

RLHF and DPO suppress observable misalignment in chat contexts but leave it intact in agentic and coding contexts — "split-brain" conditional misalignment
Misalignment transfers between models via sequences of random numbers — no semantic content required (subliminal/homeopathic learning)
Mixing misaligned data with normal data transfers traits more effectively than pure misaligned data — the dilution paradox

For builders: Treat your fine-tuning datasets as a misalignment-risk surface, not just a content-quality surface. Chat-only evaluations will miss conditional misalignment.

Models Know When They're Being Evaluated

Research from MATS (Giles Edkins and Joe Needham) showed that every tested frontier model could reliably detect evaluation contexts — benchmark formatting, multiple-choice structures, explicit refusal triggers that appear in safety evals but rarely in real deployment.

If models behave differently during testing versus deployment, our safety benchmarks may be measuring performance theatre, not actual alignment. This has direct implications for anyone running red-teaming evaluations.

Defenses That Are Actually Working

CaMeL — Formal Data Provenance Enforcement

Edoardo Debenedetti (ETH Zurich) presented CaMeL: a dual-LLM architecture that enforces formal security policies based on data provenance.

Architecture:

Privileged LLM — handles planning, generates Python-like execution code
Quarantine LLM — parses all untrusted external data in isolation

A dependency graph tracks data provenance across the pipeline. Security policies are deterministic functions: if an email recipient is not authorized to read the data being transmitted, execution is blocked or escalated.

Result: near-100% prompt injection resistance on the AgentDojo benchmark, with acceptable utility trade-offs.

GAVEL — Activation-Level Security Monitoring

Yisroel Mirsky (Ben-Gurion University) presented GAVEL — described as "Snort for LLMs." Rather than monitoring tokens, GAVEL monitors neural activations to detect policy violations in real time.

It decomposes threats into atomic cognitive elements (CEs) — concepts the model must "think about" to execute harmful behavior — and matches them against human-readable rule sets. Language-agnostic by design: activations are language-independent even when outputs are in Mandarin, Spanish, or other languages.

Particularly notable: GAVEL can detect hidden agendas — cases where the model conceals information not visible in prompts or outputs. Only detectable via activations.

Anthropic's Layered Jailbreak Defense

Jerry Wei presented Anthropic's production classifier system: a layered architecture combining a model-internal probe (first-stage, low-cost filter) with an external classifier (escalation path for high-suspicion traffic).

Production false-positive rate: 0.05% (down from 0.38% at first deployment)
Built on natural-language "constitutions" that seed synthetic adversarial training data
Constitutions are updated as the threat landscape shifts — rapid adaptation to new attack domains

The OWASP Agentic AI Top 10

John Sotiropoulos (OWASP Agentic Security Initiative) presented a framework now validated across major organizations and referenced by five national cyber agencies. Worth bookmarking:

Prompt hijacking (most underappreciated risk)
Indirect prompt injection via trusted channels
Tool abuse and remote code execution
Supply chain poisoning
Identity and access sprawl
Rogue agent behavior
Excessive agency and permission scope
Memory and context manipulation
Unsafe inter-agent communication
Lack of runtime governance

The critical shift the framework advocates: move security controls to runtime. Policy-as-code evaluated on every agent action — stop and escalate rather than alert after the fact.

What This Means If You're Building Today

Based on three days of sessions, here is what I'd prioritize immediately:

Audit your human-assumption dependencies. Which of your security controls implicitly rely on human properties — speed limits, out-of-band checks, physical presence, plausibility judgment? Those controls will silently fail when an agent fills that role.

Replace Docker-only sandboxing. For any multi-tenant agent workload, VM-backed containers (Firecracker, gVisor) are the current recommended standard. Docker-only is insufficient.

Implement secrets isolation now. Keys injected post-LLM at the execution boundary, never in model context. This is not complex to implement and eliminates an entire class of exfiltration attacks.

Add provenance metadata and TTL to agent memory. Every stored memory item needs its source context and an expiration. This is cheap to implement and prevents stale, poisoned memory from compounding over time.

Treat package/dependency installation by agents as an active attack surface. Allowlists and provenance checks — not model judgment — for anything an agent installs or downloads.

Build friction deliberately. Add circuit-breaker-style gates on high-impact, high-velocity agent actions (deployments, payments, mass communications). Rate and blast-radius triggered, not just action-type triggered.

Diversify where it matters. Avoid monoculture for critical workflows — different models, different verification paths, so failures don't correlate across your entire fleet.

Closing Thought

The field is converging faster than most people realize. The gap between what we can build and what we can secure is closing — but in the wrong direction.

Three days at Stanford made one thing clear: the enforcement-layer architectures we've been discussing in research contexts are not academic anymore. They're urgent.

The "first big one" — a headline agent-mediated compromise at enterprise scale — is a when, not an if. Build as if it's coming, because the researchers in that room are confident it is.

I've been writing about enterprise AI agent architecture and MCP integrations on dev.to — follow along if this is relevant to what you're building. Happy to go deeper on any of the sessions covered here.

aisecurity #llm #agents #security #architecture #mlsecurity #promptinjection #devops*

Anatomy of an enterprise AI agent: a vendor-agnostic walkthrough

Yogi — Tue, 30 Jun 2026 08:38:36 +0000

Most enterprise platforms now ship some version of an "AI agent studio." The branding differs, but the architecture underneath is remarkably consistent. Here's a breakdown based on a recent build, generalized so it applies regardless of which platform you're using.

At the top level: a user prompt enters the platform, gets routed to the right tool based on intent, and the tool pulls from live backend data rather than a static export. The response is synthesized back in natural language — with an optional export action if the user wants it as a file or email.

Zooming in, the agent itself is composed of a few consistent building blocks: metadata describing what the agent does, an LLM doing the reasoning, topics and guardrails scoping its behavior, and a set of discrete tools — usually mapped to business objects — that connect to the underlying data store.

A note on portability
This pattern isn't tied to one platform. The same skeleton — router, scoped tools, guardrails, triggers — shows up in Oracle AI Agent Studio, Salesforce Agentforce, Microsoft Copilot Studio, AWS Bedrock Agents, and MCP-based integrations. What differs is terminology and how much orchestration each platform abstracts away. Understanding the underlying pattern, not just one vendor's UI, is what makes these skills transferable across ecosystems.

Happy to discuss this further in the comments — feel free to reach out with any questions.

How I Connected Claude Desktop to Live Salesforce CRM Data Using MCP

Yogi — Wed, 24 Jun 2026 23:15:08 +0000

I recently deployed a real-time integration between Claude Desktop and Salesforce CRM using Model Context Protocol (MCP) — and it changed how I think about AI in enterprise operations.

Here's a practical walkthrough of what I built, the security architecture behind it, and what I learned along the way.

The problem I was trying to solve

As part of my work, I was spending too much time manually navigating Salesforce to answer questions

Every answer required logging into Salesforce, running a report, cross-referencing opportunities, and building a mental model of the data. I wanted to just ask the question in plain English and get the answer — against live CRM data, not a stale export.

Enter Model Context Protocol (MCP).

What is MCP?

MCP (Model Context Protocol) is an open standard from Anthropic that lets AI models like Claude connect to external data sources and tools through a standardized interface.

Instead of building custom APIs for every data source, MCP defines:

A server (the data source, in this case Salesforce)
A client (Claude Desktop)
A protocol for tool discovery, invocation, and response

Salesforce now ships a Hosted MCP Server, which means the connection layer is managed for you — you just need to configure authentication and define your connected app.

Architecture overview

The integration has three layers:

Request flow

You type a natural language question in Claude Desktop
Claude identifies the right MCP tool to call (e.g. query_opportunities)
The MCP client translates the request into a Salesforce API call
The Salesforce Hosted MCP Server executes the query via SOQL
Results return to Claude, which synthesizes a natural language answer

The security architecture — OAuth 2.0 + PKCE

This is where most guides gloss over the hard part. Getting enterprise AI-to-CRM security right requires careful attention to token flows, scopes, and least-privilege access — especially when an AI model has live read access to customer data.

Why PKCE matters

PKCE (Proof Key for Code Exchange) is essential for public client integrations where you cannot safely store a client secret. Claude Desktop running locally is a public client — there's no server-side secret storage. PKCE solves this by:

Generating a random code_verifier on the client at the start of each auth flow
Hashing it to create a code_challenge sent with the authorization request
Sending the original code_verifier when exchanging the authorization code for tokens
The auth server verifies the hash matches — proving the token request came from the same client that initiated the flow

Without PKCE, an intercepted authorization code could be exchanged for tokens by a different client. With PKCE, the code is useless without the verifier that only the originating client holds.

Salesforce Connected App setup

# Create Connected App in Salesforce Setup with:
# - OAuth 2.0 enabled
# - PKCE required
# - Callback URL: http://localhost:{PORT}/callback
# - Scopes: api, refresh_token (principle of least privilege)
# - No client secret (public client flow)

MCP server configuration (claude_desktop_config.json)

json{
  "mcpServers": {
    "salesforce": {
      "command": "sf",
      "args": ["mcp", "start"],
      "env": {
        "SALESFORCE_ORG_ALIAS": "your-org-alias",
        "MCP_AUTH_TYPE": "oauth2-pkce"
      }
    }
  }
}

Authentication flow

What it enables

Claude queries the live data, reasons over it, and gives you a synthesized answer — no manual report-building required.

Key learnings

MCP is becoming the standard for enterprise AI integration

The pattern MCP establishes — standardized tool definitions, structured request/response, discoverable capabilities — is exactly what enterprise AI needs. It's analogous to how REST APIs standardized web service integration in the 2000s.

Least-privilege access is non-negotiable

Only grant the scopes your use case requires. For read-only pipeline reviews, api scope with read-only profiles is sufficient. Don't grant write access unless you specifically need it — an AI with write access to your CRM is a very different risk profile.

Token lifecycle management matters

Refresh token rotation, expiry handling, and re-authentication flows need to be part of your implementation plan. Salesforce's default refresh token expiry is org-configurable — make sure it aligns with your operational workflow.

SFDX CLI session management simplifies operations

Using sf org login web to establish authenticated sessions and letting the MCP server inherit those sessions reduces the auth complexity significantly compared to managing tokens directly.

Follow for more posts on enterprise AI integration, MCP, and operational AI tooling.