DEV Community: Michael Tuszynski

Your Claude Plugin Marketplace Needs More Than a Git Repo

Michael Tuszynski — Mon, 20 Apr 2026 23:48:03 +0000

Your Claude Plugin Marketplace Needs More Than a Git Repo

Anthropic's official plugin directory has 55+ curated Claude Code plugins since launch. The community tracks another 72. By end of 2026, most large engineering orgs will have a private marketplace too — and most of them will ship the architecture wrong.

The mistake: treating the marketplace as a distribution channel when it's actually a governance layer.

What a Claude Code Plugin Actually Is

A plugin is a directory with a .claude-plugin/plugin.json manifest and some combination of skills (skills/<name>/SKILL.md), agents, hooks, and MCP servers. When a user runs /plugin install code-formatter@acme-tools, Claude Code clones the plugin to ~/.claude/plugins/cache/<marketplace>/<plugin>/<version>/ and loads it into the session.

Critical point: plugins run fully trusted code inside your developers' sessions. Skills execute shell commands. MCP servers install arbitrary binaries. Hooks intercept every tool call. There is no sandboxing in 2026. Trust is transitive — trust the Git host, trust the repo's access control, trust the authors.

That's what "setting up your own marketplace" is really about. Distribution is the easy part.

The Three-Layer Model

Most engineers see plugins and think "install mechanism." The actual hierarchy is three layers:

Layer	What it does	Where it lives
Skill/plugin	Runs code in user sessions	`skills/<name>/SKILL.md`, `.claude-plugin/plugin.json`
Marketplace	Lists approved plugins, sources, versions	`.claude-plugin/marketplace.json`
Managed settings	Controls which marketplaces users can add	`managed-settings.json`

Skills do the work. The marketplace decides which skills exist. Managed settings decide which marketplaces your employees can reach at all.

Skip layer 3 and you've built a distribution channel, not an enterprise platform.

The Manifest That Matters

marketplace.json looks deceptively simple:

{
  "name": "acme-tools",
  "owner": {"name": "DevTools Team"},
  "plugins": [
    {
      "name": "sfdc-lint",
      "source": {
        "source": "github",
        "repo": "acme-corp/sfdc-lint-plugin",
        "ref": "v2.1.0",
        "sha": "a1b2c3d4e5f6..."
      },
      "description": "Blocks commits that violate Salesforce field access rules"
    }
  ]
}

The sha field is the thing nobody talks about. Pin to a commit hash, not a tag. Tags move. Commits don't. This is how you ship deterministic plugins across a 5,000-engineer org without one dev accidentally upgrading to a broken version during a release freeze.

Release channels work the same way — keep two marketplaces (acme-tools-stable, acme-tools-canary) pointing at different branches with distinct pinned SHAs. Developers opt into canary. Everyone else stays on stable.

The Lockdown: `strictKnownMarketplaces`

This is the setting that separates hobbyist from enterprise. In your org's managed settings:

{
  "strictKnownMarketplaces": [
    {"source": "github", "repo": "acme-corp/approved-plugins"},
    {"source": "hostPattern", "hostPattern": "^github\\.acme-corp\\.com$"}
  ]
}

Empty array = lockdown, no third-party marketplaces. A list = allowlist. A hostPattern regex = domain-level control. Pair with extraKnownMarketplaces to auto-inject your corporate marketplace so engineers don't have to discover or configure it.

Without this, any employee can run /plugin marketplace add random-internet-person/totally-safe-plugins and suddenly you have unreviewed scripts running with their local credentials, SSH keys, and AWS tokens. The technical capability is identical to the company marketplace. The governance is what changes.

Hooks Are the Runtime Enforcement Layer

Marketplaces control what gets installed. Hooks control what happens at runtime.

PreToolUse hooks fire before any tool call. They receive JSON on stdin describing the call, and can exit 0 (allow), non-zero (block), or rewrite the tool arguments. This is where you enforce org policy at execution time — not just installation time.

Real examples worth shipping in your internal plugin:

Block Bash calls containing curl | sh patterns
Reject Edit/Write to ~/.aws/credentials, ~/.ssh/id_*, or /etc/
Require human approval for MCP calls to external APIs
Log every filesystem mutation to a central audit service

The marketplace decides which plugins your team can install. The hooks decide what those plugins can actually do once loaded. You need both.

What Most Orgs Will Get Wrong

Mistake #1: Private repo as marketplace. Putting a skills/ directory in a private GitHub repo and asking people to clone it. That skips the marketplace.json layer entirely — no version pinning, no managed-settings enforcement, no clean update mechanism. Works for 5 people. Breaks at 500.

Mistake #2: Trusting the public directory. Anthropic's claude-plugins-official is curated, but "curated" is not "audited for your security posture." The 72+ community plugins have zero security review from anyone. Don't install these org-wide based on install counts.

Mistake #3: Forgetting the seed directory. For air-gapped environments or CI containers, CLAUDE_CODE_PLUGIN_SEED_DIR lets you bake plugins into the image at build time. Skip it and your CI pipelines hit GitHub every run — network errors, rate limits, surprise outages when Microsoft-owned infrastructure blips.

Mistake #4: Treating security as a scanning problem. There's no binary signing for Claude plugins in 2026. Static analysis catches obvious issues; it won't catch a plugin that calls an LLM to generate malicious commands at runtime. Your defense is the marketplace allowlist + runtime hooks + human code review before plugins enter the marketplace. Not automated scanners.

The First-Week Setup

If you're standing up an enterprise marketplace this quarter:

Two repos, separated. acme-plugins-marketplace (just marketplace.json and approval metadata) and your actual plugin source repos. Separating manifest from code lets you revise approval lists without touching plugin internals.
Write a managed-settings policy. Lock strictKnownMarketplaces to your corporate marketplace. Ship via your MDM or config-management tool (Ansible, Jamf, Chef, whatever already provisions dev laptops).
Start with three plugins. Something low-risk (commit-message helper), something internal-facing (your SSO login flow), something domain-specific (your SOW template for client work). Three forces you to confront versioning, namespacing, and review questions before you scale.
Ship a PreToolUse audit hook on day one. Even if it just logs. Retrofitting observability onto 50 plugins later is miserable.
Document the review process. Who reviews? What's the bar? What gets rejected? Most orgs skip this and end up with approval-by-loudest-voice. Write it down before you take the first submission.

The tools are ready. The distribution mechanism is solved. What's left is organizational work — deciding who decides, and who's accountable when a plugin ships something stupid.

The Model Doesn't Matter. The Harness Does.

Michael Tuszynski — Sun, 19 Apr 2026 17:04:46 +0000

The Model Doesn't Matter. The Harness Does.

Six frontier coding models now score within 0.8 points of each other on SWE-bench Verified. The same model wrapped in different agent frameworks swings almost ten points on SWE-bench Pro. Picking an agent platform based on which model it runs misses where the real performance differences come from.

The useful shift in framing came from Birgitta Böckeler at Thoughtworks, writing on martinfowler.com: Agent = Model + Scaffolding. The scaffolding is everything that isn't the model — the tool definitions, the context compaction, the error recovery logic, the feedback sensors, the system prompt, the memory between sessions. That's the layer where most of the variance in real-world agent performance actually lives.

The data

Particula Tech's analysis lines up four agent frameworks running the same Claude Opus 4.5 against SWE-bench Pro:

Framework	Model	Score
SEAL (standardized scaffold)	Opus 4.5	45.9%
Cursor	Opus 4.5	50.2%
Auggie (Augment)	Opus 4.5	51.8%
Claude Code	Opus 4.5	55.4%

Same weights, 9.5-point spread. And when Meta and Harvard's Confucius Code Agent ran Sonnet 4.5 with its own scaffold, it scored 52.7% — beating Opus 4.5 on Anthropic's stock framework at 52.0%. A cheaper model with better scaffolding beat the flagship on its vendor's own agent.

Single-variable changes produce similar results. Grok Code Fast went from 6.7% to 68.3% on coding benchmarks after changing only the edit tool format — same model, same prompts. LangChain's coding agent moved from 52.8% to 66.5% on Terminal Bench 2.0 by improving task decomposition and tool use, with no model swap. Adding WarpGrep as a specialized search subagent added 2.1 to 3.7 points across every model tested, while cutting cost 15.6% and runtime 28%.

What Anthropic and OpenAI are publishing

Both labs have written up their own agent framework work in the past year, and neither piece is about model training.

Anthropic's "Effective harnesses for long-running agents" describes a two-part pattern for agents working across many context windows. An initializer agent writes a feature list as structured JSON, an init.sh script, and a progress file. A coding agent reads the progress file, picks one feature, commits its work, and updates progress. Same model in both roles — the difference is what the scaffolding makes visible between sessions.

The observation that drove this: Opus 4.5 running on the Claude Agent SDK in a loop, given the prompt "build a clone of claude.ai," would run out of context mid-implementation, then declare victory too early in a later session because the environment looked mostly done. The fix wasn't a better model. It was a scaffolding pattern that handed off state correctly.

OpenAI's writeup on agent-first engineering goes further. Three engineers shipped a million lines of code across 1,500 PRs in five months with no manually-written code. Codex wrote the application logic, the tests, the CI configuration, the docs. The humans' job was to design the environment — wiring Chrome DevTools into the agent runtime so Codex could drive its own UI, exposing logs via LogQL and metrics via PromQL, making the app bootable per git worktree so each task ran on an isolated instance.

Their framing: "Our most difficult challenges now center on designing environments, feedback loops, and control systems." That's the team at the company that trains the models saying the models aren't the bottleneck.

Where the cycles should go

For teams actually building agents, the practical implication is that model upgrades at the frontier buy roughly one point on benchmarks, while scaffolding improvements can buy twenty or more. The components with measurable impact are fairly consistent across the published case studies: tool orchestration (how tools get discovered and composed), context management (compacting aggressively, keeping recent tool outputs intact, persisting structured state between sessions), error recovery (the gap between 42% and 78% on SWE-bench is largely recovery from mistakes rather than fewer mistakes), deterministic feedback sensors like linters and structural tests that run on every change, and planning-execution separation where one agent decides and another does.

None of this requires a frontier model. A well-scaffolded Sonnet beats a poorly-scaffolded Opus, at about a fifth the cost.

The model-as-differentiator era appears to be ending. Frontier capability has converged within a point or two, and the interesting engineering — the work that actually moves benchmarks and production reliability — is happening in the scaffolding around the model.

Sources:

Böckeler, Birgitta. "Harness engineering for coding agent users." martinfowler.com, April 2026.
Mondragon, Sebastian. "Agent Scaffolding Beats Model Upgrades: 42% to 78% on SWE-Bench." Particula Tech, March 2026.
Anthropic. "Effective harnesses for long-running agents." November 2025.
Lopopolo, Ryan. "Harness engineering: leveraging Codex in an agent-first world." OpenAI, February 2026.

AI Agents Are Killing Seat-Based SaaS Pricing. Here's What's Replacing It.

Michael Tuszynski — Sat, 18 Apr 2026 02:29:58 +0000

AI Agents Are Killing Seat-Based SaaS Pricing. Here's What's Replacing It.

Intercom's Fin AI agent went from $1M to $100M+ ARR on one pricing move: $0.99 per resolved ticket. Not per seat, not per month. Per outcome. Fin now handles 80%+ of Intercom's customer support volume and closes about a million conversations a week, and Intercom will refund up to $1 million if resolution targets aren't hit (Sequence).

That pricing model is telling you something the press releases aren't: when an agent does the work, the seat becomes fiction.

The Seat Model Assumes Humans Do the Work

Seat-based SaaS had a clean logic for twenty years. A human uses the tool. The tool's value scales with how many humans use it. Price the seat.

AI agents break that logic in one step. OutSystems' 2026 State of AI Development report found 96% of organizations are using AI agents and agents are resolving 80%+ of employee service requests on average — a shift projected to cut IT service management licensing costs by up to 50% (PR Newswire).

If your ServiceNow fulfillers are closing 80% fewer tickets because Now Assist closed them first, you're going to want 80% fewer fulfiller seats. Your ServiceNow rep has noticed.

Four Pricing Models Are Fighting to Replace It

1. Outcome pricing. Intercom's Fin charges $0.99 per resolved conversation. You pay when the customer confirms resolution or doesn't come back. Zero outcomes, zero charge. This is the model most aligned with buyer incentives — and the hardest for vendors to run. It forces the vendor's sales, CS, and product teams to actually deliver measurable value. Intercom's president put it bluntly: the model "exposed every weak link" inside the company (GTMnow).

2. Action/credit consumption. Salesforce Agentforce charges $2 per conversation under the Conversations model, or $0.10 per standard action ($500 per 100K Flex Credits) under the Flex Credits model rolled out in late 2025 and recommended for most new deployments in 2026 (Aquiva Labs). Microsoft Copilot Studio uses the same structure — $0.01 per credit or $200 per 25,000 credits pay-as-you-go via Azure (Microsoft).

Consumption pricing is vendor-friendly: it meters API calls, not business value, so the vendor gets paid whether the agent actually worked or not. It's also predictable enough for finance to plan around, which is why most enterprise CIOs will pick this over outcome pricing when given the choice.

3. Add-on uplift. ServiceNow's Now Assist is sold as the Pro Plus SKU — a 50-60% uplift on your existing base tier, plus $50-$100+ per fulfiller per month, plus per-Assist token consumption with overage at $0.015-$0.04 (Redress Compliance). Now Assist's net new ACV crossed $600M in FY25 and is tracking toward $1B in FY26. The model works because the base platform is sticky — you're not going to rip out ServiceNow to avoid a 50% uplift. Expect every dominant seat-based vendor to ship this variant first.

4. Agent-as-employee. The newest and most provocative: price the agent as a fraction of the full-time employee it replaces or augments. A $2,000/month "AI SDR" for a team that would otherwise hire a $90K human SDR. Monetizely's 2026 guide calls this the model that moves outcome pricing from support into sales, ops, and engineering (Monetizely). Expect "agentic enterprise license agreements" — roll-up contracts that bundle agent capacity across functions — to become the norm at the high end.

Microsoft Is Hedging Both Ways

Watch what Microsoft is actually doing. Microsoft 365 Copilot still costs $18-$42.50 per user per month — pure seat pricing (Microsoft). But Copilot Studio, where customers build their own agents, runs on consumption credits. Microsoft is collecting the seat revenue from customers who don't know the transition is happening and the consumption revenue from customers who do.

This is the playbook. Seat revenue doesn't disappear on any vendor's P&L for a few years. It gets harvested slowly while the consumption line ramps. Salesforce is running the same hedge — Flex Credits for agent workloads, $125-$650 per user per month for Agentforce 1 Editions on top of whatever you're already paying for Sales Cloud and Service Cloud.

What This Means for Platform and Procurement Teams

Three things you should be doing right now.

Renegotiate before the renewal, not at it. If your existing seat-based vendor is deploying agents into your environment, you have leverage you won't have once they've re-based your contract around agent ACV. Vendors are booking agent revenue as net-new ACV to protect multiples. Your seat count is about to become the anchor on their growth number. Use that.

Instrument consumption from day one. Enterprise AI agent deployments cost $150K-$800K to set up and $50K-$200K annually to run (AI Magicx). The per-unit economics mean nothing if you can't meter per-team, per-workflow, per-outcome. The FinOps teams that built cloud cost attribution in 2019-2022 are now building AgentOps. If you wait, your Q4 bill will surprise you.

Push vendors toward outcome pricing where you can. It's the only model aligned with your interests. Not every workflow has a clean outcome metric, but customer support does, sales qualification does, L1 IT does. For those, refuse to sign consumption deals without outcome-based alternatives in the quote. The vendors who refuse are telling you something about their confidence in the agent.

Your TCO Goes Up Before It Goes Down

Here's the part nobody's saying out loud: in year one of agent adoption, your SaaS bill gets bigger, not smaller.

You're still paying for seats — those people haven't left yet, and you still need the UI. You're now also paying consumption credits, add-on uplifts, or outcome fees. ServiceNow Pro Plus doesn't replace Pro, it sits on top of it. Agentforce doesn't replace Sales Cloud, it extends it. For 12-24 months, you're paying twice for the same work.

The bill comes down only when you actually reduce seat counts — which requires workforce planning decisions most companies aren't ready to make. Gartner reports SaaS vendors are already pushing 10-20% renewal uplifts on average (BetterCloud). Add agent consumption on top of that and your TCO is up 30%+ before any optimization kicks in.

The vendors know this. Their pricing models are designed to extract the uplift during the overlap period. If your FinOps practice doesn't expand into AgentOps by the end of 2026, you won't just pay more — you'll pay more without knowing why.

38% of AI Answers Are Wrong — And It's Your Prompt's Fault

Michael Tuszynski — Tue, 14 Apr 2026 15:16:26 +0000

Every week someone posts about AI hallucination like it's a mystery. It's not. A 2025 Frontiers in AI study measured it: vague, multi-objective prompts hallucinate 38.3% of the time. Structured, single-focus prompts? 18.1%. That's a 20-point accuracy gap from how you write the prompt — not which model you pick.

Everyone's debating GPT vs. Claude vs. Gemini. Nobody's talking about the fact that prompt structure matters more than model selection for most use cases.

The $0 Fix Nobody Uses

Research from SQ Magazine breaks it down further: zero-shot prompts (no examples, no structure) hallucinate at 34.5%. Add a few examples and that drops to 27.2%. Add explicit instructions: 24.6%. Simply adding "If you're not sure, say so" cuts hallucination by another 15%.

That last one is worth repeating. One sentence — "If you're not confident, say you don't know" — is worth more than upgrading your model tier. And it costs nothing.

Why Multi-Task Prompts Are the Worst Offender

"Summarize this doc, extract the key risks, and draft a response email" feels like one task. It's three. And each additional objective gives the model more room to fabricate connections between things that don't connect.

Language models are next-token predictors. Single task = narrow probability distribution = the model knows where it's headed. Three tasks stacked together = triple the surface area for error. A small fabrication in the summary becomes a stated fact in the risk analysis becomes a confident assertion in the draft email.

Longer, multi-part prompts increase error rates by roughly 10%. In legal contexts, hallucination rates run between 58% and 88%. That's not an AI problem. That's a prompting problem.

What Actually Works (With Numbers)

One prompt, one job. Summarize the doc. Stop. Review it. Then extract risks from the verified summary. Then draft the email from verified risks. Three prompts, each building on confirmed output.

Constrain the output. JSON, numbered lists, specific templates. Structured prompts cut medical AI hallucinations by 33%. The less room to improvise, the less it fabricates.

Give examples. Zero-shot to few-shot: 34.5% → 27.2%. Two examples costs you 30 seconds and buys a 7-point accuracy gain.

Set refusal conditions. "If confidence is below 70% or no evidence supports the claim, say 'insufficient data.'" You're not weakening the model. You're giving it a pressure valve so it doesn't fill gaps with fiction.

The Model Isn't the Bottleneck

The best models went from 21.8% hallucination in 2021 to 0.7% in 2025 on benchmarks. But benchmarks test clean, single-objective tasks. Real-world, multi-step workflows — the kind actual professionals run — depend more on how you ask than what you ask.

You wouldn't hand a contractor one work order that says "remodel the kitchen, fix the plumbing, and repaint the exterior." You'd scope each job, inspect the work, then move on.

The people getting the best results from AI already know this. Everyone else is blaming the model.

Building a Plugin Marketplace for AI-Native Workflows

Michael Tuszynski — Fri, 10 Apr 2026 22:55:16 +0000

Most AI coding tools ship as monoliths. One big system prompt, one set of capabilities, one-size-fits-all. That works fine for general software engineering. It falls apart the moment you need domain-specific workflows that vary by role, by team, and by engagement.

I build presales systems at Presidio — client research, SOW generation, meeting capture, deal operations. The kind of work where a solutions architect needs different tools than a deal desk analyst, and where loading everything into every session wastes tokens and degrades output quality.

So I built a plugin marketplace for Claude Code. Nine modular plugins, independently installable, composable by role. Here's what I learned shipping it to a team.

Why Plugins Instead of One Big Workspace

The original system was a monolith — 56 commands, 14 skills, 36 tools, all loaded into every session. It worked for me as the sole operator. The moment I tried to share it with other consultants, three problems surfaced immediately:

Context pollution. A consultant doing SOW work doesn't need the meeting transcription pipeline, the competitive intel framework, or the deal management commands cluttering their context window. Every irrelevant token degrades the model's attention on the task at hand.

Onboarding friction. "Clone this repo, read 200 lines of docs, configure 18 environment variables, and learn 56 commands" is not an adoption strategy. People need to install what they need and ignore what they don't.

Maintenance coupling. A bug fix in the meeting recorder shouldn't require every user to pull an update that also touches their SOW pipeline. Independent versioning matters when your users are busy consultants, not developers.

The fix was decomposition. Break the monolith into plugins that can be installed, updated, and removed independently.

The Architecture That Emerged

fulcrum-plugins/
├── plugins/
│   ├── core/          # Foundation: persona, auth, OneDrive, shared rules
│   ├── intel/         # Client research pipeline
│   ├── meeting/       # Silent recording + transcription
│   ├── discovery/     # Call prep, qualification, opportunity analysis
│   ├── sow/           # SOW drafting, review, QA, redlines
│   ├── proposal/      # Solution decks and pricing workbooks
│   ├── ops/           # Daily briefing, weekly review, context switching
│   ├── engage/        # Deal management, delivery handoff
│   └── util/          # Freshness audits, triage, workspace maintenance
├── shared/            # Frameworks and templates used across plugins
└── docs/

Each plugin is a self-contained directory with commands, scripts, rules, and hooks. One required dependency — core — provides identity, authentication, and the shared file system. Everything else is optional.

Installation is one command: /plugin install sow@fulcrum-plugins. Uninstall is equally clean. No cross-plugin imports, no shared state beyond what core provides.

Recommended Sets Over Required Bundles

Rather than prescribing one configuration, the marketplace offers recommended sets:

Minimal (SOW-focused): core + sow — for consultants who only write statements of work
Meeting-heavy: core + meeting + discovery — for consultants running client calls all day
Full presales: all 9 plugins — for people like me who touch every phase

This respects how people actually work. Nobody uses every tool every day. The consultant who installs just core + sow gets a focused, fast experience. The one who installs everything gets the full operating system. Both are first-class citizens.

Namespace Isolation Matters More Than You Think

When I first decomposed the monolith, every plugin had commands like /draft, /review, /status. Collisions everywhere. The fix was namespace syntax: /sow:draft, /intel:company-intel, /ops:weekly-review.

This felt verbose at first. It turned out to be the single most important design decision. Namespaces do three things:

Eliminate ambiguity. /draft could mean a SOW draft, a proposal draft, or an email draft. /sow:draft is unambiguous.
Enable discovery. A new user can type /sow: and see every SOW command without memorizing a list. The namespace IS the documentation.
Preserve plugin independence. Two plugin authors can independently create a status command without coordinating. /engage:status and /ops:status coexist without conflict.

I renamed all plugins in v1.1 specifically to get shorter prefixes — sow-pipeline became sow, research-intel became intel, engagement-lifecycle became engage. Every keystroke matters when you're typing these dozens of times a day.

Shared Lessons: Institutional Memory Without a Database

The feature I'm most proud of has zero lines of application code. It's a folder on OneDrive.

When a consultant learns something the hard way — a Salesforce field that's locked to AMs, a client's preferred meeting format, a compliance requirement that isn't documented anywhere — they drop a markdown file into a shared folder:

shared-lessons/jane-doe/2026-04-09-sfdc-stage-lock.md

At session start, a hook script scans all lesson files from the team, deduplicates by title, and renders them into a cached rule that loads into every session. The team's collective knowledge grows without anyone maintaining a wiki, attending a knowledge-sharing meeting, or filing a ticket.

The constraints are deliberate: one lesson per file, no client-confidential data, attribution required, dates required (stale lessons get pruned). The format is simple enough that non-developers can contribute. The mechanism — a OneDrive folder sync'd through Microsoft Teams — requires no new tools or logins.

This is the pattern I keep returning to: use infrastructure people already have. OneDrive, git, markdown files. Not a custom database, not a new SaaS tool, not an API integration. The best systems are the ones that disappear into workflows people already follow.

What Didn't Work

Plugin dependencies. The original design had plugins declaring dependencies on each other — sow depends on intel, engage depends on discovery. In practice, this created install-order headaches and made it harder to reason about what was loaded. The v1.3 architecture dropped all inter-plugin dependencies. Each plugin is fully self-contained. If sow needs client context, it reads the client's context file directly — it doesn't import a function from intel.

Automatic plugin updates. The original design auto-pulled updates on session start. In practice, this broke people mid-workflow when a command signature changed. The fix was making updates explicit — /core:update when you're ready, with a statusline indicator showing when updates are available. Users update on their own schedule, not yours.

Granular versioning. I initially tried to version each plugin independently. After three days of version coordination across nine plugins, I switched to a single marketplace version. All plugins share the version in VERSION. SemVer at the marketplace level, not the plugin level. Simpler to reason about, simpler to communicate ("update to 1.3.1"), simpler to tag.

The Adoption Signal

The most telling metric isn't installs — it's which recommended set people choose. When most of your users install the minimal set and gradually add plugins over weeks, you've built something that earns trust incrementally. When they install everything on day one and complain it's overwhelming, you've just shipped a monolith with extra steps.

So far, the pattern is healthy. New consultants start with core + sow (the job requirement), then add discovery after their first client call, then meeting after they see someone else's AI-generated meeting notes. Pull, not push.

Plugin architectures aren't new. What's new is applying them to AI agent context — treating the model's knowledge, rules, and capabilities as composable modules rather than a static system prompt. The tools exist today in Claude Code's plugin system. The hard part isn't the architecture. It's the discipline to decompose.

Context Engineering Is the New Prompt Engineering

Michael Tuszynski — Wed, 08 Apr 2026 22:06:58 +0000

Everyone's writing better prompts. Few are building better context.

That's the gap. Prompt engineering treats AI like a search box — craft the perfect query, get the perfect answer. Context engineering treats AI like a new team member — give them the right docs, the right access, and a clear understanding of how work actually gets done. As Andrej Karpathy put it, the hottest new programming language is English — but the program isn't the prompt. It's the context surrounding it.

I've spent the last six months building AI-native workflows at Presidio, where I'm a Principal Solutions Architect. Not chatbots. Not demos. Production systems where Claude Code agents run real presales operations — client research, proposal generation, meeting analysis, deal tracking. The kind of work that used to live in someone's head and a dozen browser tabs.

Here's what I learned about making AI actually useful.

Prompts Are Requests. Skills Are Frameworks.

The first mistake everyone makes: stuffing domain knowledge into prompts. "You are an expert in enterprise sales. When analyzing a deal, consider these 47 factors..."

That breaks immediately. Prompts are ephemeral — they disappear when the conversation ends. Domain knowledge needs to persist across sessions, get version-controlled, and evolve as you learn what works.

The pattern that works: skills files — what Claude Code's plugin architecture calls reusable domain knowledge. Markdown documents that encode decision frameworks, not instructions. A skill isn't "analyze this deal." A skill is the 5-gate qualification framework your team actually uses, written as structured markdown with decision criteria, red flags, and exit conditions. The AI reads it and applies it. You update the framework once, every future session uses the new version.

.claude/skills/
├── qualification-framework.md    # Decision gates with criteria
├── pricing-strategy.md           # Margin rules, discount authority
├── sow-review-rubric.md          # Evaluation checklist
└── competitive-positioning.md    # Differentiators by competitor

Skills are reusable. Prompts are disposable. That distinction matters more than any prompting technique.

Context-as-Code: Version Control Your AI's Brain

Every client engagement in my system has a single markdown file that serves as the AI's working memory for that account. Contacts, scope decisions, meeting history, action items, competitive intel — one file, version-controlled in git.

Why markdown? Three reasons:

It's grep-searchable. When an agent needs to find every mention of a specific technology across all accounts, grep -r "Kubernetes" clients/ works instantly. Try that with a vector database.
It diffs cleanly. Git shows you exactly what changed in the AI's understanding of an account. Who updated it, when, and why.
It's token-efficient. Structured markdown compresses well in context windows. A 200-line context file gives an agent everything it needs to operate on an account without RAG retrieval latency.

The anti-pattern is treating AI memory as a black box — embeddings you can't inspect, vector stores you can't diff, context you can't version. If you can't git blame your AI's knowledge, you don't control it.

One God-Agent Is a Trap

I started with one agent that did everything. It was mediocre at all of it.

The fix was domain specialization. Four agents, each with a clear role: one handles discovery and qualification, one handles technical design and proposals, one handles deal operations and pricing, one handles workspace maintenance. Each agent has its own tools, its own context, and a defined handoff protocol for passing work to another agent.

This mirrors how real teams work. Your sales engineer doesn't do contract redlines. Your deal desk doesn't design architecture. Specialization isn't just about accuracy — it's about cost. A maintenance agent running on Haiku costs 95% less than an Opus agent doing the same file cleanup.

Model routing by task complexity is the easiest money you'll save in AI:

Task Type	Model	Why
File organization, validation	Haiku	Structured, predictable
Research, summarization	Sonnet	Good reasoning, fast
Strategy, complex writing	Opus	Needs deep reasoning

Mistakes Must Become Infrastructure

Every production AI system has failure modes you won't predict. The question is whether failures teach the system or just annoy you.

My approach: every time an agent makes a mistake that I have to correct, it becomes a numbered rule in the system's configuration file. Not a mental note. Not a prompt tweak. A permanent, version-controlled rule that every future session reads on startup.

After six months, I have 39 of these. Examples:

"State files in .gitignore can vanish silently during merges — default to safe fallback values"
"Never infer what a client said in a meeting — only quote from the transcript or flag it as an assumption"
"Contract reverts produce the same error on every RPC — don't retry, they're non-recoverable"

These aren't prompt engineering. They're institutional memory encoded as code. The system gets smarter every time it fails, without retraining, fine-tuning, or hoping the model "remembers."

Don't Dump Everything Into Context

The biggest performance killer in AI systems isn't the model — it's context pollution. Loading every piece of knowledge into every session degrades output quality and burns tokens.

The pattern that works: modular context loading. My system has 14 skills, 56 commands, and context files for dozens of accounts. But any given session loads only what's relevant — the specific client context, the specific workflow skills, the specific agent role. Everything else stays on disk until needed.

Think of it like imports in code. You wouldn't import * from every module in your codebase. Don't do it with AI context either.

This also means your context files need to be current state, not changelogs. A context file that accumulates three months of historical notes becomes noise. Describe what the system is right now in 150 scannable lines. Put the changelog somewhere else.

The Stack That Actually Works

After building this across multiple enterprise engagements, here's the architecture I'd recommend for anyone building AI-native workflows:

Structured context files (markdown, git-tracked) over vector databases for domain knowledge
Skills (persistent frameworks) over prompts (ephemeral instructions) for domain expertise
Specialized agents with handoff protocols over one general-purpose agent
Cost-aware model routing — match model capability to task complexity
Error-to-rule pipelines — every failure becomes a permanent system improvement
Modular loading — only load context relevant to the current task

None of this requires a framework. No LangChain, no LlamaIndex, no orchestration layer. It's markdown files, a CLI, and good engineering discipline. The AI does the reasoning. You do the architecture.

The tools for building AI-native workflows exist today. The bottleneck isn't model capability — it's context architecture. Start treating your AI's knowledge like code: structured, versioned, reviewed, and intentionally loaded. That's the difference between a chatbot and a system.

AWS Frontier Agents: What $50/Hour Pen Testing and $30/Hour SRE Means for Platform Teams

Michael Tuszynski — Mon, 06 Apr 2026 00:57:56 +0000

AWS just launched two autonomous AI agents — Security Agent and DevOps Agent — and they're both generally available now. These aren't chatbots with polished wrappers. They're persistent, autonomous systems that run for hours or days without human oversight, doing work that previously required dedicated teams.

Here's what caught my attention, and why platform engineers should be paying close attention.

Two Agents, Two Big Problems

AWS Security Agent handles penetration testing. Not the "run a scanner and hand you a PDF" kind — it ingests your source code, architecture diagrams, and documentation, then operates like a human pen tester. It identifies vulnerabilities, builds attack chains, and validates that findings are real exploitable risks. Bamboo Health reported it "surfaced findings that no other tool has uncovered." HENNGE K.K. cut their testing duration by over 90%.

AWS DevOps Agent handles incident response and operational tasks. It correlates telemetry, code, and deployment data across your stack — AWS, Azure, hybrid, on-prem — and integrates with the observability tools you already use: CloudWatch, Datadog, Dynatrace, New Relic, Splunk, Grafana. Western Governor's University cut mean time to resolution from two hours to 28 minutes during a production incident by pinpointing a Lambda configuration issue that had been buried in undiscovered internal docs.

The preview numbers are worth noting: up to 75% lower MTTR, 80% faster investigations, and 94% root cause accuracy.

The Pricing Makes the Strategy Obvious

This is where it gets interesting.

DevOps Agent costs $0.0083 per agent-second — roughly $29.88 per hour. AWS's own pricing examples show a small team running 10 investigations per month pays about $40. An enterprise running 500 incidents per month pays around $2,300. For context, a single on-call SRE costs you $150k-$200k/year fully loaded.

Security Agent runs at $50 per task-hour. A small API test costs about $173. A full application pen test runs around $1,200. Compare that to external pen testing firms charging $15k-$50k per engagement, with weeks of lead time and limited scope.

Both agents include a 2-month free trial. AWS is clearly betting on adoption velocity — get teams hooked on the speed and economics, then make it sticky through integration depth.

The DevOps Agent pricing also ties into existing AWS Support plans. Enterprise Support customers get 75% of their support charges back as DevOps Agent credits. Unified Operations customers get 100%. AWS is effectively saying: your support spend now buys you autonomous operations capacity.

What This Actually Means for Platform Teams

AWS calls these "frontier agents" — autonomous systems that work independently, scale massively across concurrent tasks, and run persistently. The framing matters because it signals a product category, not a one-off feature.

Three implications stand out:

Security becomes continuous, not periodic. Most organizations pen test their top 5-10 applications once or twice a year because of cost and staffing constraints. At $50/task-hour, you can afford to test everything, continuously. The security posture shift from "we tested our critical apps last quarter" to "every app gets tested every sprint" is significant.

Incident response gets a tireless first responder. The DevOps Agent doesn't replace your SRE team — it augments the 3am on-call rotation. It can start investigating before a human even picks up the page, correlating signals across your entire stack. By the time your engineer opens their laptop, the agent has already identified probable root cause with 94% accuracy.

The multicloud angle is deliberate. AWS built the DevOps Agent to work with Azure DevOps, GitHub, GitLab, and non-AWS observability tools. This isn't altruism — it's a land-and-expand play. Once your operational intelligence lives in AWS, migrating workloads away gets harder. But for teams running hybrid environments today, having a single agent that understands your entire topology is genuinely useful.

The Bigger Picture

Google and Microsoft have their own agentic plays, but AWS shipping two production-ready autonomous agents with per-second billing and free trials is a concrete move. The fact that both agents work with tools like Claude Code and Kiro for generating validated fixes signals that AWS sees these agents as part of a broader autonomous development loop — not isolated point solutions.

For platform engineering teams, the takeaway is practical: evaluate these agents against your current pen testing costs and incident response metrics. The economics alone justify a proof of concept. The free trial removes any excuse not to try.

The real question isn't whether AI agents will handle DevOps and security tasks. It's how fast your organization adapts its processes, roles, and trust models to work alongside them.

The systems behind enterprise AI adoption success - IBM

Michael Tuszynski — Sat, 28 Mar 2026 17:10:20 +0000

Everyone's Buying GPUs. Almost Nobody's Ready to Feed Them.

The enterprise AI conversation has a blind spot the size of a data center. Every budget meeting I've sat in over the past 18 months has the same shape: GPU allocation gets 70% of the discussion time, model selection gets 20%, and the data infrastructure that actually feeds those models gets whatever's left over. Usually about ten minutes and a vague reference to "we'll figure out storage later."

This is why most enterprise AI deployments stall after the proof of concept.

The Bottleneck Nobody Budgets For

Here's what happens in practice. A team spins up a promising AI workload — retrieval-augmented generation, a fine-tuning pipeline, an inference service. It works great on a curated dataset in a dev environment. Then they try to run it against production data at scale and everything falls apart. Not because the model is wrong, but because the storage layer can't deliver data fast enough, the pipeline can't unify sources across hybrid environments, and nobody planned for the I/O characteristics of AI workloads.

AI training and inference workloads have fundamentally different storage profiles than traditional enterprise applications. Training jobs need sustained sequential throughput across massive datasets. Inference needs low-latency random reads. Fine-tuning needs both, sometimes simultaneously. Your SAN that runs ERP just fine will choke on a distributed training job that's trying to saturate eight GPUs.

IBM's recent framing of AI-ready infrastructure gets this right: the systems layer — storage, compute fabric, automation — is where enterprise AI succeeds or dies. Not in the model layer.

The Data Gravity Problem

The reason storage matters so much for AI isn't just throughput. It's data gravity.

Enterprise data doesn't live in one place. It's spread across on-prem databases, cloud object stores, SaaS platforms, edge devices, and that one team's PostgreSQL instance that nobody wants to touch. IBM defines enterprise AI as the integration of AI across large organizations — but integration implies the data is accessible. In most companies, it isn't. Not in any unified, performant way.

This creates a cascading failure. Your RAG pipeline needs product data from SAP, customer interactions from Salesforce, and technical documentation from Confluence. Each source has different access patterns, different latency profiles, different security boundaries. Stitching them together with API calls and batch ETL jobs introduces hours of lag and creates brittle pipelines that break every time someone changes a schema.

The companies I've seen succeed at enterprise AI solve this problem first. They build a unified storage layer that can serve multiple AI workloads without requiring six different integration patterns. IBM's approach with Storage Fusion and FlashSystem targets exactly this — high-performance, unified storage that can handle the mixed I/O profiles of AI workloads across hybrid environments. Whether you're on their stack or not, the architectural principle holds: if your AI workloads can't access unified data at the speed they need it, no amount of GPU spend will fix your pipeline.

Hybrid Cloud Is the Reality, Not the Exception

There's still a persistent fantasy in some planning meetings that AI workloads will live entirely in one public cloud. Maybe someday. Right now, for regulated industries, for companies with significant on-prem investments, and for anyone who's done the math on data egress costs, hybrid is the reality.

And hybrid AI infrastructure is hard. You need consistent orchestration across environments. You need storage tiering that can move hot data close to compute without manual intervention. You need security and governance that doesn't collapse the moment data crosses a network boundary.

IBM identifies inadequate infrastructure as one of the top five AI adoption challenges — and in my experience, "inadequate" usually means "designed for a different era." The infrastructure that runs your web applications, your CI/CD pipelines, your traditional analytics workloads — it wasn't built for the throughput patterns, the data volumes, or the operational demands of production AI.

This isn't a rip-and-replace argument. Nobody's going to throw out their storage infrastructure overnight. But you need a plan for how your existing infrastructure evolves to support AI workloads, and that plan needs to happen before you commit to production deployments.

What Actually Works: Three Patterns From the Field

After spending time with organizations that have moved past the POC phase into production AI, I see three common patterns:

1. Storage-first capacity planning. Successful teams model their data pipeline throughput requirements before they size GPU clusters. They ask: "How fast can we feed data to training jobs?" and "What's our p99 latency for inference-time retrieval?" If the answers don't match the model's appetite, they fix storage before buying more compute.

2. Unified data access across environments. Whether it's IBM Storage Fusion, a well-architected MinIO deployment, or a managed cloud storage layer with on-prem caching, the pattern is the same: AI workloads get a single namespace to read from, regardless of where the source data physically lives. This eliminates the integration tax that kills most pipelines.

3. Automation of the data lifecycle. Production AI generates enormous amounts of intermediate data — checkpoints, embeddings, feature stores, evaluation datasets. Teams that automate tiering, retention, and cleanup avoid the "we ran out of storage on a Friday night" incident that's practically a rite of passage.

The Uncomfortable Math

Here's a rough calculation that sobers up most planning conversations. A mid-size enterprise running a fine-tuning pipeline on proprietary data with a 70B parameter model needs approximately 500TB of accessible, high-performance storage just for the training data, checkpoints, and model artifacts. That's before you add your RAG corpus, your vector store, and your evaluation datasets.

Now multiply that by the number of AI initiatives in your roadmap. Most enterprises I talk to have between five and fifteen active AI projects. The storage footprint adds up fast, and it needs to perform — not just exist.

The GPU shortage got all the headlines in 2024. The storage and data infrastructure gap is the quieter crisis that will define which companies actually ship production AI in 2026.

What CTOs Should Do Next Week

Stop treating infrastructure as a downstream consequence of model selection. Flip it around.

Audit your current storage throughput against the I/O demands of your planned AI workloads. Map where your training data lives and how many network hops separate it from your compute. Calculate the real cost of your data integration layer — not just the cloud bill, but the engineering hours spent maintaining brittle pipelines.

Then have an honest conversation about whether your infrastructure roadmap matches your AI ambitions. If there's a gap — and there almost certainly is — close it before you scale your GPU footprint. The fastest accelerator in the world is useless if it's starving for data.

The companies that figure this out won't just run AI. They'll run AI that actually works in production, at scale, without the 2 AM pages. That's a meaningful competitive advantage — and it starts with the infrastructure layer that nobody wants to talk about at the budget meeting.

Why Enterprise AI Infrastructure is Going Hybrid – and Geographic

Michael Tuszynski — Thu, 26 Mar 2026 04:39:27 +0000

The Cloud Repatriation Nobody Expected: Why Enterprise AI Is Pulling Compute Back from the Cloud

The original pitch for cloud computing was simple: stop buying servers, rent someone else's. For most workloads over the past fifteen years, that trade worked. But AI infrastructure has rewritten the economics, and enterprises are responding by doing something few predicted — they're moving compute closer to the data, not further away.

A recent DataBank survey found that 76% of enterprises plan geographic expansion of their AI infrastructure, while 53% are actively adding colocation to their deployment strategies. This isn't a minor adjustment. It's a structural shift in how organizations think about where AI workloads should run.

The Economics Changed Before the Strategy Did

Running inference on a large language model in a hyperscaler region costs real money. Not "line item you can bury in OpEx" money — more like "the CFO is asking questions in the quarterly review" money. GPU instance pricing on AWS, Azure, and GCP has remained stubbornly high because demand outstrips supply, and the cloud providers know it.

The math gets worse when you factor in data gravity. Most enterprises generate data in dozens of locations — retail stores, manufacturing plants, regional offices, edge devices. Shipping all that data to us-east-1 for processing, then shipping results back, creates latency and egress costs that compound as AI adoption scales.

Colocation flips this equation. You place GPU-dense compute in facilities close to where data originates, connect to cloud services where they make sense (object storage, managed databases, identity), and keep the expensive part — inference and fine-tuning — on hardware you control or lease at predictable rates.

Why "Cloud-Smart" Beats "Cloud-First"

The industry is moving toward what Seeking Alpha describes as a "cloud-smart" strategy — using public cloud, private cloud, and edge computing based on the workload profile rather than defaulting to one deployment model for everything.

This makes sense when you break down what AI workloads actually need:

Training still belongs in the cloud for most organizations. You need massive, bursty GPU capacity for weeks or months, then nothing. Buying that hardware outright is a terrible investment unless you're running training continuously. Hyperscaler reserved instances or on-demand capacity work fine here.

Inference is the opposite profile. It's steady-state, latency-sensitive, and runs 24/7. The cost-per-token adds up fast at scale. Running inference on colocated or on-premises hardware — especially with purpose-built accelerators — can cut costs 40-60% compared to cloud GPU instances, depending on utilization rates.

Fine-tuning sits in the middle. You need GPU capacity for days, not months, and the data involved is often sensitive enough that you don't want it leaving your network. A colocated setup with good connectivity to your data sources handles this well.

The Geography Problem Nobody Planned For

Data sovereignty and residency requirements are accelerating the geographic distribution of AI infrastructure in ways that pure cloud strategies can't easily accommodate.

The EU's AI Act imposes requirements on where and how AI systems process data. Healthcare organizations in the US deal with HIPAA locality requirements. Financial services firms face data residency rules that vary by jurisdiction. When your AI model needs to process customer data from Germany, running inference in a Virginia data center creates compliance headaches that no amount of architectural cleverness fully solves.

Enterprises are responding by deploying AI infrastructure across multiple geographies — not because they want the operational complexity, but because regulators and customers demand it. The 76% planning geographic expansion aren't chasing some multicloud vision. They're meeting regulatory reality.

The Edge Dimension

Hybrid edge-cloud architectures add another layer. Manufacturing plants running quality inspection models can't tolerate 200ms round-trip latency to a cloud region. Autonomous systems need inference at the point of action. Retail environments process customer interactions in real time.

These use cases demand on-site or near-site compute with cloud connectivity for model updates, monitoring, and periodic retraining. The architecture looks less like "cloud with edge caching" and more like "distributed compute with cloud coordination." The control plane lives in the cloud. The data plane runs where the data lives.

This is a harder architecture to build and operate than a cloud-native deployment. It requires teams who understand networking, hardware lifecycle management, and distributed systems — skills that many organizations let atrophy during the cloud migration years.

What This Means for Infrastructure Teams

If you're an infrastructure leader planning AI capacity for the next 2-3 years, here's the framework I'd use:

Audit your inference costs first. Most organizations are surprised by how much they're spending on cloud GPU instances for inference once they aggregate across teams and projects. This number is your baseline for a hybrid business case.

Map data gravity. Where does your training data originate? Where do inference requests come from? Where do results need to arrive? If the answer to all three is "the same cloud region," stay in the cloud. If it's "twelve different locations across three countries," you need a distributed strategy.

Don't build a GPU data center. Colocation with GPU leasing gives you the economics of owned hardware without the capital expenditure and refresh cycles. Companies like DataBank, Equinix, and CoreWeave are building exactly this model — dense GPU compute in colocation facilities with direct cloud interconnects.

Plan for heterogeneous accelerators. NVIDIA's dominance in training is real, but inference has viable alternatives — AMD Instinct, Intel Gaudi, AWS Inferentia, Google TPUs. A hybrid strategy lets you match accelerators to workload profiles instead of paying the NVIDIA tax on everything.

Invest in platform engineering. Hybrid AI infrastructure without a solid platform layer becomes an operational nightmare. You need consistent deployment pipelines, observability, and model lifecycle management that works across cloud regions, colo facilities, and edge locations. Kubernetes helps here, but it's the starting point, not the whole answer.

The Uncomfortable Reality

Going hybrid is operationally harder than going all-in on a single cloud provider. Anyone who tells you otherwise is selling colocation space. You'll manage more vendor relationships, more network paths, more failure modes.

But the economics and the regulatory environment have shifted enough that "just put it all in AWS" is no longer a defensible strategy for AI-heavy workloads. The organizations figuring out hybrid now — while GPU supply is still constrained and cloud pricing remains elevated — will have a meaningful cost advantage over those who wait.

The cloud isn't going away. It's just no longer the default answer for every AI workload. And the sooner infrastructure teams internalize that distinction, the better positioned they'll be when AI spending goes from "experimental budget" to "largest line item on the infrastructure bill."

Enterprise AI has an 80% failure rate. The models aren't the problem. What is?

Michael Tuszynski — Sat, 21 Mar 2026 18:27:58 +0000

Enterprise AI Fails at 80% — And the Models Have Nothing to Do With It

Most enterprise AI projects die quietly. No dramatic failure, no post-mortem email chain. They just... stop. The prototype gets a demo, leadership nods approvingly, and then six months later the Slack channel goes silent and the budget gets reallocated.

Roughly 80% of enterprise AI projects fail — double the failure rate of traditional software projects. That number has held steady for years now, even as the models themselves have gotten dramatically better. GPT-4, Claude, Gemini — pick your favorite. They all work. They work remarkably well, actually.

So why does the enterprise keep fumbling the ball?

The Model Isn't Your Problem. Your Plumbing Is.

Here's what I keep seeing from the architecture side: teams treat AI like a features problem when it's actually an infrastructure problem. They spin up a proof of concept using an API, get promising results in a notebook, and then hit a wall when someone asks "OK, how do we run this in production?"

That wall has a name. It's called the deployment gap — the distance between a working model and a production system that real users depend on. And it's enormous.

A platform engineer on Reddit put it bluntly: the failure pattern repeats across organizations regardless of size or industry. The models work fine. The org doesn't.

The Five Ways Smart Companies Still Blow It

I've watched this play out at dozens of enterprise accounts. The failure modes are predictable enough to catalog.

1. They Solve the Wrong Problem

This is the most common and most expensive mistake. A team picks a use case because it sounds impressive in a board deck, not because it maps to an actual operational bottleneck. "We'll use AI to predict customer churn!" Great. Do you have clean customer data? Do you have a process to act on those predictions? Is churn actually your biggest revenue leak?

Companies that fail at AI overwhelmingly choose the wrong problems first. They optimize for what's exciting instead of what's painful. The successful projects I've seen start with someone saying "this manual process costs us 200 hours a month and we keep getting it wrong."

2. They Live in Data Fantasy Land

Every AI project starts with an assumption about data quality that turns out to be wildly optimistic. The data exists, sure. It's in four different systems, three different formats, with no consistent identifiers, maintained by teams who don't talk to each other.

I worked with an enterprise that wanted to build an AI-powered inventory optimization system. The model was straightforward. The data pipeline took eleven months — not because the engineering was hard, but because getting three business units to agree on what "inventory" meant took that long.

3. They Skip the Platform Layer

This one hits close to home. Teams build AI applications without investing in the platform that supports them. No model registry. No feature store. No monitoring for drift. No rollback mechanism. No cost controls.

Then the model goes sideways in production — and it will — and there's no way to detect it, debug it, or revert it. You're flying blind with a system that's making real decisions.

The production gap isn't a model problem — it's a platform engineering problem. The organizations that ship AI successfully treat ML infrastructure with the same rigor they'd apply to any other production system: observability, CI/CD, access controls, cost management.

4. They Start With Tech Instead of Humans

I've seen teams spend months evaluating which LLM to use, which vector database to pick, whether to fine-tune or RAG, which embedding model performs best on their benchmark — and zero time figuring out who will actually use this thing and how it fits into their workflow.

The best AI system in the world is worthless if the end user Alt-F4s out of it because it adds three clicks to their existing process. Starting with tech instead of humans is the classic engineering trap: we build what's interesting to build, not what's useful to use.

5. They Treat AI as a Project, Not a Product

AI models degrade. The world changes, user behavior shifts, data distributions drift. A model that was 94% accurate in January might be 71% accurate by June. Traditional software doesn't do this. You deploy a calculator app and it keeps calculating correctly forever.

AI requires ongoing investment: retraining, monitoring, evaluation, data quality maintenance. When leadership treats AI as a one-time project with a ship date and a done state, they're guaranteeing that the system will rot.

What Actually Works

After 25 years in tech — the last several spent watching enterprise AI projects succeed and fail — here's what separates the 20% that ship from the 80% that don't.

Start with the workflow, not the model. Find a process where humans are doing repetitive cognitive work, making inconsistent decisions, or drowning in volume. Build AI into that workflow. Not as a standalone app — as an augmentation of what people already do.

Invest in platform before product. You need model serving infrastructure, monitoring, cost tracking, and rollback capabilities before you need a sophisticated model. A simple model on a solid platform beats a sophisticated model on nothing.

Set a 90-day production deadline. If your AI project hasn't touched a real user in 90 days, it probably never will. Scope ruthlessly. Ship something small. Learn from real usage. The organizations that perpetually prototype never ship.

Budget for operations, not just development. AI is more like a garden than a bridge. You don't build it and walk away. Plan for ongoing model evaluation, data quality work, and retraining cycles. If your budget only covers development, you're planning to fail.

Make the ROI case boring and specific. Not "AI will transform our customer experience." Instead: "This model will reduce manual review time from 6 hours to 45 minutes per day for the claims processing team, saving $280K annually." When the value is that concrete, the project survives leadership changes and budget cuts.

The Uncomfortable Truth

The enterprise AI failure rate represents roughly $154 billion in wasted spend. That money didn't evaporate because GPT wasn't smart enough. It evaporated because organizations treated AI adoption as a technology challenge when it's actually an organizational design challenge.

The models are good enough. They've been good enough for a while now. The question was never "can AI do this?" — it's "can your organization support AI doing this?"

If you can't answer yes to that second question, no amount of model capability will save you. Fix the plumbing. Define the problem. Invest in the platform. Then — and only then — worry about which model to use.

Beyond Comprehension Debt: Why Context Architecture Is the Real AI Moat

Michael Tuszynski — Thu, 19 Mar 2026 03:00:36 +0000

Addy Osmani dropped a piece last week that's been making the rounds: "Comprehension Debt — The Hidden Cost of AI-Generated Code." His thesis is sharp. Teams are shipping AI-generated code faster than anyone can understand it. Tests pass, PRs look clean, and nobody notices the growing gap between what's been deployed and what any human actually comprehends. When something breaks at 3am, that gap becomes the bill.

He's right. And he's not seeing the whole picture.

Osmani diagnosed one symptom of a larger condition. Comprehension debt — the gap between shipped code and understood code — is real, and it matters. But it's one line item on a ledger that most engineering organizations haven't even opened yet. If you're a CTO or VP of Engineering adopting AI-assisted development, comprehension debt is the problem you can see. The ones that will actually sink you are the ones you can't.

The AI Debt Paradox

There's a seductive argument circulating in engineering circles right now: AI makes rewriting cheap, so tech debt doesn't matter anymore. Why maintain a crumbling monolith when you can regenerate services in an afternoon?

It's about 40% right, which makes it dangerous.

Yes, the cost curve of rewriting code has collapsed. For bounded, well-specified modules, AI absolutely turns "rewrite" from a quarter-long initiative into a day's work. The classic excuse — "we can't touch that, it'll take months" — is dying. That's real progress.

But here's the paradox: every time you "start from scratch," you throw away embedded knowledge about why decisions were made. The 47 edge cases handled one by one over 18 months. The compliance requirement someone baked in after an audit. The OAuth flow three partners hardcoded against. AI can regenerate the code layer fast. It cannot regenerate the institutional context that made that code correct.

Tech debt didn't disappear. It shape-shifted. And the new forms are harder to detect, harder to measure, and harder to pay down.

Three Debts, Not One

Osmani gave us a name for the first one. Here are the other two.

1. Comprehension Debt (Osmani's Contribution)

This is the gap between the code your team ships and the code your team understands. Osmani nailed the mechanics: AI-generated code passes review because it looks right, engineers approve PRs they haven't fully internalized, and the organizational assumption that "reviewed = understood" quietly breaks down.

The insight is correct. But the prescription — slow down, review more carefully, quiz your engineers — is a cultural intervention. It treats comprehension debt as a discipline problem. For Google-scale teams with senior engineers and strong review culture, that might work. For the other 95% of engineering organizations? You need more than discipline. You need infrastructure.

2. Context Debt

This is the one nobody's naming clearly, and it's the most dangerous.

Context debt is the accumulated loss of institutional knowledge about why systems are built the way they are. It's not about whether engineers understand the code in front of them — it's about whether anyone understands the decisions, constraints, trade-offs, and edge cases that shaped it.

Consider what lives in a mature codebase beyond the code itself:

Architectural rationale. Why this service exists as its own deployment rather than a module in the monolith. Why the database schema looks the way it does. Why that particular API contract was chosen over three alternatives that were debated for weeks.
Boundary knowledge. Which downstream consumers depend on specific response shapes. Which partner integrations are fragile. Which compliance requirements are baked into the data flow and why.
Failure memory. The incident that revealed a race condition nobody anticipated. The scaling problem that drove the caching strategy. The security audit finding that explains the seemingly redundant validation layer.

None of this lives in the code. Very little of it lives in documentation. Most of it lives in the heads of engineers who were there when the decisions were made. When AI regenerates a service from scratch, it produces code that compiles, passes tests, and handles the happy path. What it cannot produce is the scar tissue — the hard-won understanding of what goes wrong and why.

Context debt accumulates every time a team rewrites without capturing context first. Every time an AI-generated solution replaces a human-authored one without preserving the reasoning behind the original. Every time an engineer leaves and their knowledge of why things are the way they are walks out with them.

This isn't new — context loss has always been a risk in software organizations. What's new is the velocity at which AI-assisted development can destroy context. When rewriting is cheap, the incentive to understand before replacing drops to zero.

3. Integration Debt

The third form is architectural. Integration debt is the growing inconsistency between AI-generated components that were each built in isolation without awareness of the broader system.

AI coding assistants operate within a context window. They see the file you're working on, maybe some adjacent files, maybe a system prompt describing your stack. What they don't see is the full topology of your system — every service, every contract, every shared assumption that holds your architecture together.

When three different engineers use AI to independently build three services that interact, each service might be internally excellent. Clean code, good patterns, thorough tests. But the interfaces between them — data formats, error handling conventions, retry semantics, authentication flows — will diverge unless someone is deliberately enforcing coherence.

The numbers back this up. CodeRabbit's 2026 analysis found teams merged 98% more PRs that were 154% larger year-over-year, while 61% of developers reported that AI produces code that "looks correct but is unreliable." The generation pressure is real. The verification pressure is downstream — and growing.

Integration debt compounds quietly. It shows up as unexpected failures during deployment. As subtle data inconsistencies between services. As "it works on my machine" problems that are actually contract mismatches between components that were never designed to work together, despite technically needing to.

The faster you generate components, the faster integration debt accumulates. And unlike code quality — which AI can actually help improve — integration coherence requires exactly the kind of big-picture architectural thinking that AI tools are worst at.

The Composition Shift

Here's why this matters strategically, not just technically.

For two decades, "tech debt" mostly meant code-level debt: poor abstractions, missing tests, duplicated logic, outdated dependencies. That's the debt AI is genuinely good at paying down. Refactoring, test generation, dependency updates, code modernization — these are tasks where AI excels. If your tech debt balance sheet was entirely code-level debt, the "AI makes debt irrelevant" crowd would be right.

But in any system of real complexity, code-level debt was always just the visible portion. The deeper liabilities — context, comprehension, integration — were always there. They were just overshadowed by the sheer volume of code-level problems.

AI didn't eliminate tech debt. It paid down the most visible kind, revealing the structural kinds that were hiding underneath. The balance sheet didn't shrink. The composition changed. And most organizations are still using the old chart of accounts.

The 2024 DORA report hints at this shift: despite widespread AI adoption, throughput dipped 1.5% and stability dropped 7.2% across 39,000 respondents. Teams are generating more code and shipping it less reliably. The metrics that matter — lead time, change failure rate, recovery time — aren't improving with velocity alone.

Context as Infrastructure, Not Culture

This is where I part ways with the prevailing conversation.

Osmani and others are framing these new debts as cultural and organizational challenges. Slow down. Review more carefully. Keep humans in the loop. These aren't wrong, but they're incomplete — and for many teams, impractical. You can't tell a startup burning runway to slow down their AI-assisted velocity for the sake of comprehension hygiene. You can't tell a team of five to institute Google-style code review rituals.

What you can do is treat context as infrastructure.

I've been arguing for a while now that context management is the real skill gap in AI-assisted development — that getting value from AI tools is less about prompt engineering and more about maintaining rich, current, accessible context that those tools can leverage. (I wrote about this in "This Above All: To Thine Own Context Be True" earlier this year.)

The same principle applies to the debt problem, but at organizational scale. Context debt isn't an inevitable consequence of AI adoption. It's an infrastructure failure — a failure to build systems that capture, maintain, and surface the institutional knowledge that makes code comprehensible and architecturally coherent.

What does context infrastructure look like in practice?

Living architectural decision records. Not dusty wiki pages nobody updates, but actively maintained documents that live alongside the code and get updated as part of the development workflow. Michael Nygard formalized the ADR pattern back in 2011 — Title, Status, Context, Decision, Consequences. The format is fifteen years old. What's changed is that AI makes the cost of not having ADRs catastrophically higher. When AI generates a new implementation, the context for why the old implementation existed this way needs to be right there — available to the engineer reviewing the change and to the AI generating it.

Structured project memory. Tools and conventions so the reasoning behind decisions persists beyond the individual who made them. This means treating context documents — system descriptions, constraint inventories, edge case catalogs — as first-class artifacts that get versioned, reviewed, and maintained with the same rigor as code.

Integration contracts as explicit artifacts. Rather than letting service interfaces emerge organically from individually-generated components, defining and maintaining explicit contracts that AI tools can reference during generation. The contract becomes the source of truth for integration coherence, not the individual developer's mental model of how everything fits together.

Context-aware generation workflows. Configuring AI tools to ingest project context before generating code, rather than generating in a vacuum and hoping for coherence. This means investing in the scaffolding — the context files, the system prompts, the reference documents — that turn AI from a talented but amnesiac intern into a contributor who understands the system they're working within.

None of this is revolutionary. It's the kind of engineering discipline that good teams have always practiced. What's different is the urgency. When humans wrote all the code, context accumulated naturally — slowly, and with gaps, but it accumulated. When AI generates code at 10x the velocity, context dissipates at 10x the rate unless you deliberately counteract it.

The Assessment Gap

The biggest opportunity right now isn't in tooling — it's in assessment.

Most engineering organizations have no way to measure their exposure to these new forms of debt. They can tell you their test coverage percentage, their deployment frequency, their mean time to recovery. They cannot tell you how much institutional context has been lost in the last six months of AI-assisted development. They cannot quantify how many of their AI-generated components have integration assumptions that conflict with each other. They cannot assess whether their team's comprehension of the codebase has kept pace with its growth.

This is the gap that needs to be closed first. Before you can manage context debt, comprehension debt, and integration debt, you need to be able to see them. And right now, almost nobody can.

What This Means

If you're leading an engineering organization that's adopting AI-assisted development — and at this point, that's nearly everyone — the question isn't whether these new forms of debt are accumulating. They are. The question is whether you're managing them deliberately or discovering them during incidents.

The organizations that will thrive in the AI era aren't the ones that generate code fastest. They're the ones that maintain the richest context while moving fast. That's a different capability than what most teams are building right now, and it's one that compounds over time. The team that invests in context architecture today will be moving faster and more safely a year from now. The team that optimizes only for generation velocity will be drowning in debt they can't see and can't name.

The old tech debt conversation was about code quality. The new one is about knowledge architecture. And it's just getting started.

Michael Tuszynski is the founder of MPT Solutions, where he writes about AI strategy, cloud architecture, and engineering leadership. With 25 years in software — including six years as a Senior Solutions Architect at AWS and a stint as CTO at Fandor — he focuses on helping teams adopt AI-assisted development without sacrificing the institutional context that makes their systems work.

Previously: This Above All: To Thine Own Context Be True — on why context management matters more than prompt engineering.

Why does nobody teach the infrastructure problems that destroy developer productivity before production breaks

Michael Tuszynski — Wed, 18 Mar 2026 02:36:58 +0000

The Production Gap: Why Nobody Teaches the Infrastructure That Actually Matters

Every bootcamp, CS program, and YouTube tutorial series teaches you how to build features. Almost none of them teach you what happens when those features meet real traffic, real failure modes, and real users who do things you never anticipated.

The result is predictable: developers ship code that works on their laptop, passes CI, and then falls apart the moment it hits production at scale. Not because the logic is wrong — because nobody taught them about connection pooling, graceful degradation, or what happens when your database runs out of connections at 2 AM on a Saturday.

The Curriculum Blind Spot

A thread on r/ExperiencedDevs captured this frustration perfectly: educational content focuses almost entirely on writing code and building features, while operational concerns — monitoring, error handling, memory management, rate limiting — only become relevant when applications break in production. By then, you're learning under fire.

This isn't a minor gap. It's the gap between "I can build software" and "I can build software that stays running." And it's enormous.

Think about what a typical full-stack course covers: React components, REST APIs, database queries, authentication flows. Maybe some Docker basics. Now think about what actually causes production incidents: thread pool exhaustion, cascading failures from a single downstream dependency, memory leaks that only manifest after 72 hours of uptime, DNS resolution failures, certificate expiration, connection storms after a deploy.

These aren't edge cases. They're Tuesday.

Why This Gap Exists

Three forces keep operational knowledge out of the curriculum.

First, it's hard to teach without real systems. You can't simulate connection pool exhaustion on a laptop running SQLite. You can't demonstrate cascading failures with a single-service tutorial app. The infrastructure problems that destroy productivity only emerge at a certain scale of complexity, traffic, and time — none of which exist in a classroom.

Second, it's not glamorous. "Build a full-stack app in 30 minutes" gets clicks. "Understanding TCP keepalive settings and why they matter for your connection pool" does not. Content creators optimize for engagement, and operational topics feel boring until the moment they're the only thing that matters.

Third, the people who know this stuff learned it the hard way and are too busy to teach it. The senior SRE who understands why your Kubernetes pods are getting OOMKilled at 3x expected memory usage is probably dealing with an incident right now, not writing blog posts. Operational knowledge lives in war stories, incident retrospectives, and tribal knowledge passed between teammates — not in structured curricula.

The Real Cost

This isn't just an education problem. It's a productivity problem that masquerades as a people problem. When teams complain about slow velocity, the instinct is to look at process, hiring, or morale. But often the real bottleneck is that developers spend hours debugging infrastructure issues they were never trained to anticipate.

A developer who doesn't understand connection pooling will open a new database connection per request, wonder why the app works in dev but times out under load, and then spend two days tracking down the issue. A developer who doesn't understand backpressure will build a message consumer that looks correct but silently drops events when the queue backs up. A developer who doesn't understand DNS caching will deploy a service that works perfectly until the load balancer rotates IPs.

Each of these costs days — sometimes weeks — of debugging time. Multiply that across a team, and the infrastructure gap becomes the single largest drag on developer productivity. Not the tools, not the process, not the sprint ceremonies. The fact that half the team has never been taught how production systems actually behave.

The Specific Knowledge That's Missing

Here's my list of operational topics that every developer should understand before they're responsible for a production system. None of these show up in a typical CS degree or bootcamp:

Connection management. How connection pools work, why they have limits, what happens when you exhaust them, and how to size them for your workload. This single topic prevents more production incidents than any framework feature.

Graceful degradation. What your application should do when a dependency is slow or unavailable. The answer is never "throw a 500 and hope for the best," but that's what most tutorial code does.

Observability fundamentals. Not "install Datadog" — actual understanding of what metrics matter, how to correlate logs across services, what a useful alert looks like vs. one that wakes you up for nothing.

Memory and resource management. How garbage collection actually works in your runtime. What causes memory leaks in languages that claim to manage memory for you. Why your Node.js service uses 2GB of RAM after running for a week.

Rate limiting and backpressure. How to protect your service from being overwhelmed, and how to be a good citizen when calling someone else's service. This is the difference between a service that handles traffic spikes and one that cascades failures across your entire platform.

Failure modes of distributed systems. Partial failures, network partitions, split-brain scenarios, exactly-once delivery myths. You don't need a PhD in distributed systems theory, but you need to understand that the network is not reliable, clocks are not synchronized, and retries without backoff are a denial-of-service attack on your own infrastructure.

What Should Change

I'm not expecting universities to overhaul their CS curricula overnight. But a few things would help.

Bootcamps should include a "production readiness" module. Before graduation, every student should deploy an app, load test it until it breaks, diagnose the failure, and fix it. That single exercise teaches more about real-world engineering than a semester of algorithm problems.

Senior engineers need to write down what they know. The gap persists partly because operational knowledge stays locked in people's heads. Incident retrospectives should be shared broadly. Internal tech talks on "how we debugged X" are worth 10x more than another talk on the latest framework.

Companies should invest in structured onboarding for production systems. Don't throw a new hire at the codebase and hope they figure out the monitoring stack. Walk them through the architecture, show them where things break, explain the failure modes you've already seen. This is not hand-holding — it's preventing the next incident.

Platform teams should build paved roads. If connection pooling is tricky, provide a standard library that does it correctly. If observability requires too much configuration, bake it into the deployment pipeline. Don't rely on every developer independently learning every operational concern — make the right thing the easy thing.

The Uncomfortable Truth

The industry has a weird relationship with operational knowledge. We celebrate feature velocity and treat infrastructure work as unglamorous plumbing. We promote the developer who shipped the flashy new feature and overlook the one who quietly prevented 47 production incidents through better error handling and circuit breakers.

Until we value the skills that keep systems running as much as the skills that build new ones, the production gap will persist. New developers will keep learning the hard way — at 2 AM, on a Saturday, with a Slack channel full of escalations and no idea why the connection pool is exhausted.

The fix starts with acknowledging that knowing how to write code and knowing how to run code are two different skills. We teach the first one extensively. The second one, we mostly leave to chance. That's a choice, and it's the wrong one.

DEV Community: Michael Tuszynski

Your Claude Plugin Marketplace Needs More Than a Git Repo

Your Claude Plugin Marketplace Needs More Than a Git Repo

What a Claude Code Plugin Actually Is

The Three-Layer Model

The Manifest That Matters

The Lockdown: strictKnownMarketplaces

Hooks Are the Runtime Enforcement Layer

What Most Orgs Will Get Wrong

The First-Week Setup

The Model Doesn't Matter. The Harness Does.

The Model Doesn't Matter. The Harness Does.

The data

What Anthropic and OpenAI are publishing

Where the cycles should go

AI Agents Are Killing Seat-Based SaaS Pricing. Here's What's Replacing It.

AI Agents Are Killing Seat-Based SaaS Pricing. Here's What's Replacing It.

The Seat Model Assumes Humans Do the Work

Four Pricing Models Are Fighting to Replace It

Microsoft Is Hedging Both Ways

What This Means for Platform and Procurement Teams

Your TCO Goes Up Before It Goes Down

38% of AI Answers Are Wrong — And It's Your Prompt's Fault

The $0 Fix Nobody Uses

Why Multi-Task Prompts Are the Worst Offender

What Actually Works (With Numbers)

The Model Isn't the Bottleneck

Building a Plugin Marketplace for AI-Native Workflows

Why Plugins Instead of One Big Workspace

The Architecture That Emerged

Recommended Sets Over Required Bundles

Namespace Isolation Matters More Than You Think

Shared Lessons: Institutional Memory Without a Database

What Didn't Work

The Adoption Signal

Context Engineering Is the New Prompt Engineering

Prompts Are Requests. Skills Are Frameworks.

Context-as-Code: Version Control Your AI's Brain

One God-Agent Is a Trap

Mistakes Must Become Infrastructure

Don't Dump Everything Into Context

The Stack That Actually Works

AWS Frontier Agents: What $50/Hour Pen Testing and $30/Hour SRE Means for Platform Teams

Two Agents, Two Big Problems

The Pricing Makes the Strategy Obvious

What This Actually Means for Platform Teams

The Bigger Picture

The systems behind enterprise AI adoption success - IBM

Everyone's Buying GPUs. Almost Nobody's Ready to Feed Them.

The Bottleneck Nobody Budgets For

The Data Gravity Problem

Hybrid Cloud Is the Reality, Not the Exception

What Actually Works: Three Patterns From the Field

The Uncomfortable Math

What CTOs Should Do Next Week

Why Enterprise AI Infrastructure is Going Hybrid – and Geographic

The Cloud Repatriation Nobody Expected: Why Enterprise AI Is Pulling Compute Back from the Cloud

The Economics Changed Before the Strategy Did

Why "Cloud-Smart" Beats "Cloud-First"

The Geography Problem Nobody Planned For

The Edge Dimension

What This Means for Infrastructure Teams

The Uncomfortable Reality

Enterprise AI has an 80% failure rate. The models aren't the problem. What is?

Enterprise AI Fails at 80% — And the Models Have Nothing to Do With It

The Model Isn't Your Problem. Your Plumbing Is.

The Five Ways Smart Companies Still Blow It

1. They Solve the Wrong Problem

2. They Live in Data Fantasy Land

3. They Skip the Platform Layer

4. They Start With Tech Instead of Humans

5. They Treat AI as a Project, Not a Product

What Actually Works

The Uncomfortable Truth

Beyond Comprehension Debt: Why Context Architecture Is the Real AI Moat

The AI Debt Paradox

Three Debts, Not One

1. Comprehension Debt (Osmani's Contribution)

2. Context Debt

3. Integration Debt

The Lockdown: `strictKnownMarketplaces`