DEV Community: Brian Carpio

How to Find Your Own Code Inside ChatGPT — The Tiger Team Method Every Engineering Leader Should Run This Week

Brian Carpio — Wed, 20 May 2026 17:47:34 +0000

There is a ten-minute test you can run on Monday morning that will tell you whether your engineers are pasting proprietary source code into ChatGPT. It costs nothing. It requires no procurement, no security review, no consultant. If the test comes back positive, you have an open audit finding the regulator hasn't discovered yet. If it comes back negative, you have a brief window to ship the platform that prevents the next finding.

The test is the Tiger Team Method. The platform is the architectural answer cloud and DevOps already proved works. This post covers both — and explains why the operator playbook that won the cloud transformation in 2014 is the same playbook that wins the AI governance transformation in 2026. Same human dynamics. Different layer.

The Tiger Team Method

Pull a distinctive internal artifact from your own codebase — a function name no one outside your engineering organization would invent, a variable naming convention specific to your team, a comment pattern your style guide enforces, an internal acronym that appears in your ADRs. The more idiosyncratic, the better. calculate_loyalty_tier_uplift, not calculate_total. RiskWeightedExposureCalculator, not Calculator. In a regulated health setting, riskAdjustedMemberScore or an internal acronym like HEDIS_gap_closure_engine — the kind of artifact that only exists inside a payer or provider system and would never appear in public training data by chance.

Open ChatGPT in a private window. No login. Run three queries:

"How do I implement [your function signature]?"
"Explain what [your internal class name] does in a Python service."
"Give me a working example of [your internal acronym] with error handling."

Repeat the same three queries against Claude, Gemini, and any other public model your engineers have access to. Look for two patterns in the responses. First, does the model return code that uses your exact internal terminology — not generic equivalents, but the specific names and patterns from your own codebase. Second, does the model exhibit suspicious confidence about an artifact that should have zero public footprint — explaining what your internal class "does" with details consistent with how it actually behaves in your production system.

Either signal is a positive Tiger Team result. The model has seen fragments of your code. The only way that happens is if one of your engineers pasted it in. The exfiltration has already occurred. What you do next determines whether you discover the rest of the leakage from your own audit, or from a regulator's.

The method has a name because it deserves to be a named process inside every engineering organization. The Tiger Team is a five-person standing audit — one engineer from platform, one from security, one from each of the two largest product orgs, and one from the office of the CTO. They run the test quarterly. They report findings to the executive leadership team. The artifact is a one-page memo. That's the entire process. The technical complexity is zero. The organizational complexity is whether you have the discipline to look.

What the Research Already Shows

Three findings frame the scale of the problem. None of them are speculative. All three are sourced to research the executive leadership team can quote in a board deck without qualification.

IBM's 2025 Cost of a Data Breach Report found that one in five organizations had a breach attributed to shadow AI. The average shadow-AI breach cost $670,000 more than a comparable standard breach — making shadow AI the third-costliest breach factor in the 2025 dataset, displacing security skills shortages from previous years. Sixty-five percent of shadow-AI breaches exposed customer PII, against a 53% global average. Ninety-seven percent of organizations that suffered an AI-related breach lacked AI access controls. Only 37% have an AI governance policy. Only 17% have technical controls that can prevent employees from uploading confidential data to public AI tools.

Gartner's November 2025 analysis, based on a survey of 302 cybersecurity leaders, predicts that more than 40% of global enterprises will suffer a security or compliance incident linked to unauthorized AI tools by 2030. Sixty-nine percent of cybersecurity leaders surveyed already have evidence or suspect their employees are using public GenAI tools at work. The prediction is not whether an incident will happen at most enterprises — it is which enterprises will be in the 40% versus the 60% that built a sanctioned alternative in time.

The Netskope Cloud and Threat Report on Generative AI (2025) documented that prompts sent to GenAI tools grew sixfold in one year — from 3,000 to 18,000 prompts per organization per month, with the top quartile of organizations sending more than 70,000 per month. Data volume into GenAI tools grew thirtyfold over the same period. Organizations now detect an average of 223 monthly attempts by employees to include sensitive data in GenAI prompts. The trajectory is not slowing. It is accelerating.

Cyberhaven's 2026 AI Adoption & Risk Report, based on analysis of usage patterns across millions of AI interactions at hundreds of enterprises, found that 39.7% of all AI interactions involve sensitive data, and 32.3% of ChatGPT usage occurs through personal accounts — bypassing SSO, centralized logging, enterprise retention policies, and any control your existing data loss prevention stack might apply. Among the categories of sensitive data flowing into AI tools, source code is the single largest category at 18.7% of all sensitive data inputs. The percentage of corporate data going into AI tools that is sensitive has grown from 10.7% two years ago to 27.4% last year to 34.8% today. Employees input sensitive data into AI tools every three days.

The named incidents you already know are the visible portion of this picture. Samsung Electronics suffered three separate semiconductor source-code and meeting-transcript leaks into ChatGPT within twenty days in April 2023, leading to a company-wide ban. JPMorgan Chase, Apple, and Amazon each restricted ChatGPT firm-wide in early 2023 after similar incidents, with Amazon's own counsel warning employees in writing that ChatGPT output "closely matches existing material we have already produced." The difference between those four organizations and yours is that they discovered their incidents. The Cyberhaven numbers say the same pattern is occurring at virtually every enterprise.

You've Lived This Movie Before

Anyone who ran a cloud or DevOps transformation between 2010 and 2022 has watched this pattern play out three or four times already. The shape is invariant. A new technology arrives. Developers want to use it. The official answer is "not yet, security hasn't reviewed it." Developers use it anyway, on personal credit cards or personal accounts, and the organization discovers the adoption during an audit. The platform team eventually ships a sanctioned alternative. The unsafe path stops being the easy path. The transition completes.

I've led enterprise-wide cloud and DevOps transformations more times than I'll recount here — the four below are the ones that map most directly onto the AI moment. The mechanics differ. The human dynamics do not. What follows is not theory. It is what I watched happen at four Fortune 50 organizations, in chronological order, with the specific platforms and outcomes documented at the time.

Pearson, 2012–2014 — The Nibiru Platform

Developers at Pearson were spinning up AWS accounts on personal credit cards. Security had banned AWS use because it hadn't been reviewed. The ban produced exactly zero compliance — engineers used AWS regardless, and the organization had no visibility into what was running where. We built Nibiru, a self-service platform that was effectively an IaaS/IaC layer over AWS before that category had a name — a Flask web UI and REST API on top of Puppet configuration management, Zabbix monitoring, Route 53 DNS, and an LDAP-backed inventory, with encryption, network controls, naming standards, and audit logging baked into the deployment surface itself. Provisioning that had previously taken 12–18 months through traditional IT collapsed to minutes through Nibiru. The platform was the guardrail. The ban became unnecessary because the sanctioned path was faster than the unsafe one. Gene Kim came on-site to see what we were doing. Engineers stopped using personal AWS accounts because the platform path was strictly better — not because anyone enforced anything.

Aetna, 2014–2017 — The Utopia Platform

Same playbook, container layer — except this fight was about where the platform ran, not what it ran on. Aetna's Enterprise IT and security organizations wanted the consumer business out of AWS entirely and back inside Aetna's own data centers. When Aetna's CISO called me and asked what I was going to do when the platform got DDoSed, I told him "autoscale — what are you going to do?" We built Utopia on Mesosphere and SaltStack, with Twistlock for container security and Checkmarx for SAST integrated into the deployment surface. The platform delivered 0.05% security defect density on the consumer code base, against 5% on the legacy core. That number won the argument politics couldn't: the cloud-native, containerized platform was measurably more secure than the on-prem stack we were being told to retreat to. The CISO did not grudgingly relent — he mandated Docker enterprise-wide, and Aetna became one of the most publicly referenced Twistlock deployments in the industry. The platform changed what governance looked like — the safe path and the fast path were the same path, and the data proved it.

Liberty Mutual, 2016–2017 — The Fusion Platform

Liberty Mutual's Consumer business unit had a struggling Docker migration when I was brought in. We built Fusion on Chef and Docker Datacenter, with a declarative Fusionfile at the center: teams declared what they needed (upstream/downstream sidecars, data layer components, pre/post deploy hooks) and the platform figured out the rest. By 2017 the platform scaled to 300+ services in containers, hundreds of deployments per day, and the team told the story in their own Docker conference talk, All Roads Lead to the Cloud: Liberty Mutual's Journey with Docker EE, walking through the Jenkins-driven pipelines and Docker Datacenter foundation the platform ran on. The Fusionfile pattern was the architectural ancestor of every "declare-what-you-need-in-a-config-file" system we use today — including the per-repo ADR and code-map manifest patterns that power modern AI engineering platforms. I unpacked that lineage in detail in What Is an AI Engineering Platform? (2026 Guide).

Comcast, 2019–2022 — The SEED Platform and the Governance Argument

Comcast is the centerpiece anchor for this post because it is the lived governance story that maps most directly onto where every enterprise sits with AI today. At an internal cloud summit during my time there, a VP at Comcast told the room something like "80% of our AWS spend is on EC2, and 80% of those EC2 instances are sitting at 1% utilization." That single statement was a governance failure quantified. It was the financial signature of an engineering organization where every team had been given AWS and left alone to figure it out. Local optimization at scale. Hundreds of teams, each writing their own Terraform, each spinning up their own EC2 fleets, each making the same decisions in isolation, and the consolidated bill telling the story neither the teams nor the central architecture function had bothered to look at.

It got worse before it got better. Comcast's senior architecture leadership read Accelerate, became convinced of the value of standardized CI/CD tooling, and mandated Concourse company-wide. Every engineering org was instructed to move to Concourse. The mandate produced the most expensive form of local optimization imaginable. Hundreds of teams spent months each writing Concourse YAML — thousands of lines of it per pipeline — reinventing the same patterns in isolation because Concourse had no concept of shared libraries at the time. I personally reviewed pipelines from team after team where engineers told me with pride that they had spent three months building their CI/CD. Three months. Per team. Across hundreds of teams. To produce variants of the same pipeline.

SEED predated all of this. We built it before the company-wide Concourse mandate landed — a self-service platform on Jenkins where teams consumed a shared library from their Jenkinsfile, passed in a few parameters and a tfvars file declaring the services they wanted, and got our underlying Terraform modules wired together and provisioned out of the box. When the Concourse mandate came down, senior architecture leadership gave SEED a hallway pass: SEED teams were exempted until Concourse could reach feature parity with what SEED already delivered — which it never did. We banned EC2 across the platform — not by writing a memo, but by making the alternative paved-road: SEED only deployed to Lambda and ECS Fargate, both of which autoscaled on demand and routinely ran at 90% utilization rather than the 1% the EC2 fleet was averaging. SEED integrated directly with Comcast's Change Management API, which meant any team adopting SEED got CAB bypass for routine deploys. SEED integrated with Comcast's centralized logging, AWS Config inventory, and code-quality gates. SEED teams never spent three months reinventing pipelines and Terraform the way the Concourse-path teams did, because making the right way the easy way meant they got the whole thing in an afternoon.

The platform was the nucleus. Security got centralized logging and policy enforcement without having to chase teams individually. Finance got the EC2 spend collapsed by structural design. Architects got a single place to make decisions that propagated across the entire organization without having to convince every team one at a time. Engineers got CAB bypass, instant deploys, and Lambda-grade autoscaling without writing Terraform. Every constituency won because the platform was the governance, instead of governance being a tax bolted onto whatever the teams were going to do anyway.

I wrote about this exact pattern in December 2022 in a post called DevOps Is the New Waste in 2023. The argument then: hundreds of teams reinventing the same CI/CD patterns is not DevOps, it is overproduction. The DORA 2022 State of DevOps Report had just shown zero elite performers globally for the first time in the survey's history. The reason was visible to anyone running a platform org inside a Fortune 50: the "DevOps" movement had decayed into every team building bespoke versions of the same infrastructure, in isolation, because no one had built the nucleus. Three years later that argument is exactly the AI argument. Every team building bespoke prompt frameworks, per-team RAG implementations, per-team context layers. No central nucleus. No queryable audit trail. No governance shape that compliance can sign off on. Same waste pattern. Same answer.

The Pattern Is the Same. The Layer Is Different.

An engineer pasting a function into ChatGPT in 2026 is the same human dynamic as an engineer spinning up an EC2 instance on a personal credit card in 2013. The engineer has a problem. The sanctioned path is too slow, too painful, or too restrictive. The unsanctioned path is right there, takes thirty seconds, and gives them an answer. The engineer is not malicious. The engineer is rational. The organization that ignores this dynamic ends up with the consolidated bill that the VP at Comcast read out loud, except this time the bill is denominated in proprietary source code on someone else's training set, regulatory exposure, and the audit finding waiting to be discovered.

The fix has the same shape as every prior layer's fix. Build a sanctioned platform that is faster, easier, and better than the unsafe option. Put the governance into the platform surface, not into a policy memo. Make the safe path the easy path. The platform is the guardrail. Everything else is theater.

The Cyberhaven number that 32.3% of ChatGPT usage occurs through personal accounts is the 2026 version of the personal AWS credit cards at Pearson in 2012. The IBM finding that 97% of breached organizations lack AI access controls is the 2026 version of every team having its own Terraform module at Comcast in 2020. The Gartner prediction that 40% of enterprises will suffer a shadow-AI incident by 2030 is the 2026 version of DORA's 0% elite performer finding from 2022 — a leading indicator that the gap between the platform-led organizations and the policy-led organizations is about to widen by an order of magnitude.

What the AI Platform Has to Do

The architectural requirements are short and have been litigated extensively in the prior posts on this site. I'll summarize and link rather than relitigate.

The platform must run inside the customer's own AWS account, not in a vendor cloud. SaaS data exfiltration is the entire problem the platform is supposed to solve, and you cannot solve an exfiltration problem with a platform whose architecture is exfiltration by design. We covered the customer-AWS deployment model in detail in AI Coding Tool That Deploys in Your AWS Account.

The platform must replace ChatGPT for code questions, and beat ChatGPT on time-to-useful-answer. If the safe path is slower than the unsafe path, engineers will route around it. The advantage the platform has is that it can retrieve authoritative internal context — ADRs, code maps, internal documentation — that ChatGPT does not have access to. That retrieval architecture is what turns the safe path into the faster path. We unpacked the retrieval design in Why RAG Isn't Enough for Code: Adding a Graph.

The platform must log every interaction in a customer-owned audit trail. Who asked, what was retrieved, what was generated, what got merged. The audit trail is the artifact the Tiger Team uses to demonstrate governance to a regulator. The audit trail is not optional. It is the entire reason executive leadership invested in the platform.

The platform must use customer context, not training data. The output should reflect your architectural decisions and your internal patterns. The model is commodity. The context is the moat. We made this argument in What Are Context Engineering Platforms?

The platform must deploy in weeks, not quarters. Shadow AI is in flight right now. Quarterly procurement is not a strategy for an active incident. The customer-AWS deployment model collapses procurement to a Terraform read-through because the platform inherits the existing AWS posture rather than introducing a new vendor. We covered the compliance procurement path in AI Coding Tools for Regulated Industries.

That is what an AI engineering platform must do to actually solve the shadow-AI governance problem rather than perform solving it. Anything that is structurally a SaaS subscription with a security policy attached is policy theater dressed up as a platform. The Tiger Team will still find your code in ChatGPT. The bill will still arrive.

The Nucleus Argument

SEED was not just a deployment platform. SEED was the nucleus that gave every constituency inside Comcast's engineering org what it needed without forcing them to negotiate with each other. Security got centralized policy. Finance got cost structure. Architects got organizational consistency. Engineers got speed. Compliance got audit. The nucleus is the platform pattern that scales because every constituency wins simultaneously rather than one at the expense of another.

An AI engineering platform serves the same role at the new layer. It is the nucleus of the engineering organization's intelligence layer. The security team uses it to enforce policy on AI usage and produce audit evidence. The development teams use it because it is faster than ChatGPT and grounded in their own patterns. The architects use it to make organizational decisions that propagate without per-team negotiation. The compliance function uses it to demonstrate governance to regulators. The platform is the answer to every constituency's shadow-AI problem at once, not just one constituency's.

The organizations that built nucleus platforms during the cloud transformation won the cloud transformation. The organizations that built nucleus platforms during the DevOps transformation are the organizations that show up as elite performers on the DORA report when there are any elite performers to show up. The organizations that build nucleus platforms during the AI transformation will be the 60% that did not appear in Gartner's 2030 shadow-AI incident statistic. The pattern is invariant. The transformation is on a slightly faster clock this time. That is the only difference.

Run the Test Tomorrow

The Tiger Team Method is ten minutes of work. Pull a distinctive internal artifact from your codebase. Run three queries against ChatGPT, Claude, and Gemini in a private window. Look for your own terminology coming back. If it does, the exfiltration has already happened and you have a window to ship the platform before the regulator does the test for you. If it doesn't, you have an even better window because you can ship the platform while the problem is still ahead of you instead of behind.

The platform is what the cloud transformation, the container transformation, the CI/CD transformation, and the DevOps transformation all proved already. Build the nucleus. Make the safe path faster than the unsafe path. Put the governance into the platform surface instead of into a policy memo no one reads. Hand every constituency — security, developers, architects, compliance, finance — the same answer to the same problem.

Make the right way the easy way. That's how you won cloud. That's how you'll win AI.

How to Evaluate

The two-week proof of concept is structured for this evaluation. Apply the OutcomeOps Terraform into a non-production AWS account, run the Tiger Team Method against your own codebase, connect 20 representative repositories, and verify that the audit trail captures the interaction quality your compliance function needs. Book an enterprise briefing to start the PoC, or run the five-minute Readiness Assessment to get a written report on where your organization sits before scheduling.

Run the Tiger Team Test. Then Build the Nucleus.

If the test comes back positive, the exfiltration has already happened and you have a narrow window to ship the platform before a regulator runs the test for you. If it comes back negative, you have an even better one. Either way the answer is the same nucleus the cloud and DevOps transformations already proved works — deployed in your own AWS account, in weeks.

The platform is the guardrail. Everything else is theater.

Book an Enterprise Briefing

What Is an AI Engineering Platform? (2026 Guide)

Brian Carpio — Mon, 18 May 2026 16:45:29 +0000

The phrase "AI engineering platform" took two paths to 2026. One leads to CAE and simulation tools — Altair, Neural Concept, getleo, Viktor — that use AI to accelerate mechanical, structural, and product engineering. This post is not about those. The other leads to the platforms a software engineering organization uses to generate, review, and govern AI-written code at team scale — OutcomeOps, Devin, Cursor at enterprise scale, GitHub Copilot. That is the category this post defines, compares, and explains how to evaluate.

The category matters now because the conversation has shifted. Three years of "AI coding assistant" framing produced a generation of tools that augment one engineer in one IDE. The 2026 enterprise question is bigger: how does our software engineering organization, at the team and org level, use AI safely and consistently? That's a platform question, not an assistant question, and the answers look very different.

The Comparison Table (Above the Fold)

Four platforms, five dimensions that actually decide the call. Detailed writeups follow.

Platform	Where it runs	Unit of work	Cost model	Best fit
OutcomeOps	Customer AWS account (Terraform)	Pull request	Fixed enterprise tier + customer-paid Bedrock	Regulated enterprise, multi-repo, audit-required
Devin	Cognition cloud (SaaS)	Task / session	Per-task / subscription	Teams that want managed agentic execution, no audit pressure
Cursor	Engineer's laptop + Cursor cloud	File / inline edit	Per-seat / month	Individual engineer productivity at fast-moving teams
GitHub Copilot	Microsoft cloud (SaaS)	Completion / chat turn	Per-seat / month (Business / Enterprise)	Broad organizational adoption, GitHub-native shops

Status as of May 2026. Pricing and deployment options change frequently. Verify on vendor docs before procurement.

Definition: What an AI Engineering Platform Actually Does

Strip out the marketing language and an AI engineering platform has five components. Every serious platform in 2026 has all five — the architectural differences are where each component runs.

1. Context layer

The organization's authoritative knowledge — ADRs, code maps, Confluence pages, Jira tickets, runbook summaries — ingested into a vector store with metadata weighting. This is what makes generation specific to your org instead of generic. The platforms that take this seriously beat the platforms that don't, even with the same underlying model. We walked through this pattern in What Are Context Engineering Platforms?

2. Generation layer

Retrieval + LLM + standards enforcement, run as a single governed pipeline. The model gets the relevant ADRs, retrieves the relevant code patterns from the graph, generates output, and validates against the standards. RAG plus a code knowledge graph is the 2026 standard architecture for this layer.

3. Output layer

Structured artifacts — pull requests, ADR drafts, code reviews — not chat turns. The difference matters: a chat turn is unreviewable, a PR is. Output-layer maturity is what separates "AI is fast" from "AI ships to production."

4. Audit layer

Every interaction logged: who asked, what was retrieved, what was generated, what citations the output made, what got merged. Without this layer, AI use is unauditable — which is a non-starter the moment compliance, legal, or a regulator gets involved.

5. Deployment layer

Where the whole stack runs. Customer AWS account, vendor cloud, engineer's laptop, on-prem container. This is where the SaaS-vs-customer-cloud decision shows up — and where regulated-industry buyers either complete procurement in weeks or stall it for quarters. We covered the deployment-model lens in AI Coding Tool That Deploys in Your AWS Account.

Why "Platform," Not "Coding Assistant"

The 2024 framing was "AI coding assistant." The scope was one engineer, in one IDE, completing one function. That framing produced Copilot, Cursor, Tabnine, and a long tail of similar tools. All of them are good at what they do. None of them answer the org-level questions:

How do we make sure AI-generated code matches our architectural standards across 200 repos?
How do we audit what AI did six months from now when legal asks?
How do we keep the model's context current when the codebase changes 50 times a day?
How do we hand a new engineer the same productivity boost without each person re-discovering the patterns?

Those are platform questions. An assistant operates inside the developer's workflow; a platform operates inside the organization's workflow. The category name changed because the buyer changed — from the individual engineer expensing a $20/mo subscription to the engineering executive provisioning infrastructure for hundreds of people.

This is a familiar pattern. Platform engineering happened to infrastructure in 2018-2022. Every team writing its own Jenkins pipeline became one team running a paved-road platform with golden pipelines. Same productivity story, different layer. AI engineering platforms are the same pattern applied to code generation in 2026.

In late 2016 I was brought into a struggling Docker migration at Liberty Mutual's Consumer business unit. The team had bought the cloud-agnostic-deployment vision but had no concrete path to it. We built Fusion on top of Chef + Docker Datacenter, with a declarative Fusionfile at the center — teams declared what they needed (upstream/downstream sidecars, data layer components, pre/post deploy hooks) and the platform figured out the rest. By 2017 it scaled to 300+ services in containers, hundreds of deployments per day, and Docker featured the work as an official enterprise success story. That's the playbook for an AI engineering platform in 2026. Teams declare what they need in a per-repo config (ADRs, code maps, standards). The platform figures out the rest — retrieval, generation, validation, audit. Different layer, same paved-road thesis. The Fusionfile pattern was the architectural ancestor of every "configure your AI by writing a markdown file in your repo" system we use today.

The Four Platforms, in Detail

OutcomeOps — the customer-cloud platform

Ships as Terraform that applies into the customer's AWS account. Every component — context ingestion, retrieval (RAG + code knowledge graph), Bedrock invocations, PR generation, audit DynamoDB — runs inside the customer's VPC behind an internal-only ALB. Unit of work is the pull request: every output is a PR with cited ADRs, the relevant code-map context, and a structured rationale. Cost model is a fixed enterprise tier plus customer-paid Bedrock charges (typically $2–$4 per generated PR at production scale).

Best fit: 20+ engineer organizations with multiple repositories, real architectural standards, and any compliance posture (financial services, healthcare, defense, insurance) where SaaS is a non-starter. Overkill for individual engineers or three-person startups.

Devin — the autonomous-agent platform

Cognition's autonomous AI software engineer. Runs in Cognition's cloud. Engineers assign tasks ("implement this Jira ticket," "refactor this module"), Devin executes end-to-end including browsing, terminal commands, and PR submission. Unit of work is the task; cost model is per-task or subscription. The product has matured significantly through 2025 and 2026 — pricing dropped, success rates improved, and the agent now handles a meaningful fraction of standard implementation work without supervision.

Best fit: Teams that want a managed agent and accept the vendor-cloud tradeoff. Source code, agent reasoning, and execution logs all live in Cognition infrastructure. If your compliance posture has no opinion on that, Devin is a strong choice. If it does, the deployment-model question rules them out.

Cursor — the IDE platform (at enterprise scale)

A Cursor IDE installation per engineer, plus Cursor's cloud for inference and codebase indexing. Cursor for Business / Cursor for Enterprise add team-level controls and admin features. Unit of work is the file or inline edit; cost model is per-seat per month. The IDE itself is excellent and the agentic features (Composer, background agents) have grown into legitimate task-scope capability.

Best fit: Fast-moving teams that prioritize individual engineer productivity over organizational governance. Cursor wins on the developer experience and loses (relative to the platform tier) on org-level audit, customer-cloud deployment, and standards enforcement. Excellent assistant; less suited as a regulated-industry platform.

GitHub Copilot — the broad-adoption platform

Microsoft's incumbent. Runs in Microsoft cloud. Copilot Business adds team admin and data-handling controls; Copilot Enterprise adds organization-wide knowledge (custom models, knowledge bases, PR summaries, code reviews). Unit of work spans completion through chat through agent. Cost model is per-seat per month at meaningful enterprise scale.

Best fit: GitHub-native shops that want broad organizational adoption with minimal procurement friction. Copilot is the default for most enterprises and the default is often the right answer. The platform-tier features have improved enough that for non-regulated, non-customer-cloud-required buyers, Copilot Enterprise is a credible choice for the AI engineering platform category — not just the coding assistant category.

The Five-Criteria Evaluation Framework

Most vendor comparisons drown in feature lists. Five questions cut through the noise.

Where does the platform run? Customer cloud, vendor cloud, or the engineer's laptop? This single question determines roughly 70% of the procurement experience.
What is the unit of work? Completion, chat turn, file, task, or pull request? Unit-of-work granularity drives both pricing model and reviewability.
What is the cost model? Per-seat, per-token, per-task, fixed enterprise, or customer-pays-inference. Predictability and ceiling matter more than nominal price.
What is the audit story? Can you, today, produce a queryable log of who asked what, what was retrieved, what was generated, and what got merged? If not, compliance will ask later.
Does the platform know your patterns or guess them? ADRs ingested into a context layer, or generic best-practice generation? The difference is the gap between "works in a demo" and "ships to production unedited."

Question 1 usually determines questions 3 and 4 by structural consequence. Questions 2 and 5 sort the remaining platforms.

For Regulated Industries Specifically

The platform question collapses for regulated buyers. SaaS-by-default platforms (Devin, Cursor, Copilot in most configurations) trigger a vendor risk assessment, a sub-processor disclosure update, and a SOC 2 / HIPAA / FedRAMP scope expansion. Each adds quarters to procurement. Customer-cloud-deployed platforms (OutcomeOps in this lineup) inherit the customer's existing AWS posture and collapse the procurement path to a Terraform read-through.

If you're in financial services, healthcare, defense, insurance, or any industry where "the SaaS option won't pass procurement" has stalled previous AI initiatives, the deployment-model question is the entire decision. We unpack the regulated-industry lens in detail in Context Engineering Platforms: A Comparison Guide and AI Coding Tools for Regulated Industries.

What Changed in 2026

Three things matter from the last 12 months:

The category name stabilized. Buyers stopped saying "AI coding tool" or "AI dev assistant" and started saying "AI engineering platform." The vocabulary shift signals the buyer shift: from individual subscription to organizational infrastructure.
Multi-region became table stakes. After the October 2025 us-east-1 event took down a long list of AI-dependent SaaS, every enterprise architecture review now asks vendors for their HA story. Single-region deployments lost credibility. We documented our own answer in Why OutcomeOps Doesn't Use DynamoDB Global Tables.
The pricing model fractured. Per-seat (Copilot, Cursor) dominates volume. Per-task (Devin) survived enterprise pushback and got cheaper. Fixed-enterprise plus customer-paid-inference (OutcomeOps) became the dominant cost-transparency model for buyers who want a known annual ceiling and AWS-bill visibility into actual usage.

When You Don't Need One Yet

Honest take: not every team needs an AI engineering platform. If you're a three-person startup with one repository and no compliance constraint, Copilot or Cursor will deliver the productivity gain at near-zero operational overhead. The platform argument starts paying back at 20+ engineers, multi-repo, or any environment where AI output needs to demonstrably match organizational standards across teams that don't share daily context.

Sweet spot for a real platform: 50+ engineers, regulated industry, multiple business units, codebase old enough that "just ask Steve" is how architectural knowledge actually propagates. If you're there, an AI engineering platform is the system that scales Steve.

How to Evaluate

Two-week structured PoC with one platform beats six months of vendor demos. The structure that works:

Week 0: Internal alignment. Engineering, security, compliance, and procurement leads agree on the five-question framework and the weight each question carries in your environment.
Week 1: Vendor short list. Eliminate any platform that fails question 1 (deployment location). For most regulated buyers this leaves one viable option. For SaaS-friendly buyers it leaves two or three.
Week 2–3: Technical PoC. Apply the Terraform (customer-cloud) or complete vendor onboarding (SaaS). Connect 20 representative repositories. Generate code against real internal patterns. Inspect audit logs.
Week 4: Compliance review of the deployment model. For customer-cloud platforms this is reading Terraform. For SaaS this is the start of a longer vendor risk assessment.
Book an enterprise briefing to start the OutcomeOps PoC
Run the five-minute Readiness Assessment to get a written report on where your organization sits before scheduling

AI-Driven Enterprise Search Needs an AI-Ready Foundation

Brian Carpio — Sun, 17 May 2026 17:30:01 +0000

Every vendor in the enterprise software market is selling AI-driven search right now. The pitches sound nearly identical: natural-language queries, generative answers, citations, agent integrations. What none of them spend enough time on is the part that actually determines whether the system works — the foundation underneath. AI is not a magic layer that retrofits onto a broken data architecture. It is an amplifier. Whatever was already wrong with how your organization stored, secured, and connected its knowledge, AI will surface that flaw within the first week of usage.

I learned this the hard way leading enterprise cloud transformations for over a decade. The clearest articulation of it came from a CIO on the AWS re:Invent stage in 2023.

Between 2022 and 2024 I was the technical lead on AWS ProServe's engagement at Gilead Sciences — a $300B-class healthcare and life sciences organization. We rebuilt their AWS landing zone, cut account vending from 30+ days to 45 minutes, layered in 65+ service control policies, and reparented every account into a controlled hierarchy. When AWS Bedrock went into pre-release, Gilead was one of the first enterprises with safe access — because the foundation was already there. At re:Invent 2023, Gilead's CIO Marc Berson stood on the keynote stage and said the line I now quote in every executive briefing I give: "AI is only going to accelerate the speed at which gaps in a company's foundation are exposed." He was right. Every Fortune 500 I have advised since has lived that sentence.

That is the framing executives need before they spend a dollar on AI-driven enterprise search. The model is not the limiting factor anymore. Foundation models are commodity. The limiting factor is whether your organization's knowledge — across 112 SaaS applications on average, across permission-controlled file shares, across communication tools, across ticketing — can be made retrievable in a way the model can actually use.

What does an AI-ready foundation actually look like?

Four properties. First, every relevant surface must be indexable — code repositories, wikis, documents, communications, tickets. If you carve out one of those because it is "too sensitive" or "not technically integrated yet," the model's answers will be systematically biased toward whichever sources made it in. Second, permissions must enforce at query time at the source system, not after the model has already retrieved candidate content. We covered the architecture of this in permission-aware AI search. Third, retrieval must be semantic, not keyword — two people describing the same concept use different words roughly 80 percent of the time, and an AI layer fed by keyword retrieval will hallucinate confidently into the gap. Fourth, every generated answer must include citations to the underlying source, so a human can verify the model is not making things up.

Miss any of the four and you are not deploying AI-driven enterprise search. You are deploying a faster way to surface wrong answers.

Why does AI surface foundation gaps so fast?

Because the failure modes are no longer subtle. With keyword search, a user who finds zero results assumes the document does not exist and moves on. With AI search, a user who gets a hallucinated answer assumes the document said something it did not. The first failure mode is invisible. The second is louder, more confident, and more dangerous. The same property that makes generative AI useful — synthesis across multiple sources into a single answer — is the property that makes a weak foundation catastrophic.

We saw a smaller-scale version of this debugging our own retrieval pipeline at RetrieveIT. The model could not find content we knew was indexed, and the answer was not in the model — it was in the plumbing. We wrote about it in retrieval-friendly documentation. The pattern is universal: AI quality is downstream of retrieval quality, and retrieval quality is downstream of foundation quality.

What about regulated industries?

If you are in pharma, healthcare, finance, or legal, the foundation requirements get sharper. We covered the specifics in pharma and healthcare, but the short version: permission enforcement has to be auditable, every AI answer needs traceable citations for regulatory review, and the indexing layer cannot leak document metadata or snippets across permission boundaries. AI-driven enterprise search in these environments is not a productivity tool. It is a controlled system that has to demonstrate to a regulator how it decided what to show whom.

How RetrieveIT approaches the foundation

RetrieveIT was designed with the AI-ready foundation properties baked in. Semantic retrieval over 1024-dimension embeddings. Per-query permission checks at the source system. Citations on every generated answer. Workspace isolation so different parts of the organization stay logically separated. Continuous indexing rather than nightly batch. The MCP server exposes the same retrieval surface to AI agents — so when an agent like Claude Code or an internal workflow asks a question, it gets the same permission-aware semantic answer a human would. We covered the agent angle specifically in enterprise search API and MCP server.

The cousin problem — what happens to enterprise search when your service fleet itself is sprawling — we covered in why enterprise search tools break at microservice scale. The two problems compound in any large enterprise: more services, more SaaS, more documents, more communications, all of which an AI layer has to retrieve from coherently.

The pattern, restated

Marc Berson said it plainly: AI exposes the gaps. Buying an AI-driven enterprise search tool without first fixing the foundation underneath it is the most common enterprise AI mistake of the last two years, and it is the most expensive. The fix is not bigger AI. It is the boring work — indexing every surface, enforcing permissions at the source, retrieving semantically, citing every answer. Do that, and the AI layer becomes an amplifier of organizational knowledge instead of a confident liar.

Context Engineering Platforms: A Comparison Guide

Brian Carpio — Sat, 16 May 2026 10:54:54 +0000

The context engineering platform market has consolidated faster than most enterprise AI categories, and the differences between vendors are mostly architectural — not feature-list bullet points. The right choice depends almost entirely on what kind of buyer you are. SaaS-friendly enterprises building customer-facing AI experiences want one thing. Regulated buyers in financial services, healthcare, defense, and insurance want something fundamentally different. This post compares the four platforms that matter in 2026, walks the five criteria that actually decide the call, and is honest about which buyers should pick which platform.

For the category overview, see What Are Context Engineering Platforms?. This post assumes you already know what one is.

The Five Criteria That Actually Decide

Most vendor comparisons drown in feature lists. Five questions cut through the noise:

Where does the platform run? Customer cloud (Terraform into customer AWS), vendor cloud (SaaS), or both?
Where do source code and embeddings physically live? Customer-controlled storage, vendor-controlled storage, or hybrid?
Where does the AI interaction audit log live, and who can query it? Customer DynamoDB, vendor backend, or shared?
What procurement and security review path does the buyer actually have to walk? Terraform review, vendor questionnaire, BAA negotiation, sub-processor review?
What inference backends are supported? Bedrock-only, OpenAI-only, both, or anything?

Question 1 usually determines the answers to 2, 3, and 4. Question 5 matters most when compliance forces a specific provider — Bedrock for HIPAA-eligible workloads, GovCloud regions for FedRAMP environments.

The Comparison Table

Cells marked ⚠ reflect partial support, claimed-but-not-verified availability, or capabilities that vary by tier. Verify on each vendor's current public documentation before procurement.

Platform	Deployment	Data location	Audit log location	Best fit
OutcomeOps	Terraform into customer AWS	Customer S3 / S3 Vectors	Customer DynamoDB	Regulated enterprise, large eng orgs
Contextual.ai	SaaS	Vendor cloud	Vendor backend	SaaS-friendly enterprise, grounded RAG
Zep	SaaS or self-hosted ⚠	Vendor or customer (tier dependent)	⚠ Tier dependent	Agent memory, startups / SMBs
LangChain	Framework + LangSmith SaaS	Wherever the developer puts it	LangSmith (vendor) or self-built	Prototyping, developer experimentation

Status as of May 2026. Verify on vendor docs before procurement. Vendor self-hosted variants and roadmaps change frequently.

Platform-by-Platform

OutcomeOps — the customer-cloud option

Ships as Terraform that applies into the customer's AWS account. No SaaS variant. Every component — ingestion Lambdas, vector store (S3 Vectors), Bedrock invocations, audit DynamoDB, MCP server — runs inside the customer's VPC, behind an internal-only ALB, with OIDC at the edge against the customer's IdP. Architectural detail in AI Coding Tool That Deploys in Your AWS Account.

Best fit: 20+ engineer organizations, regulated industries, legacy modernization programs, and any enterprise where "the SaaS option won't pass procurement" is the actual reason every prior AI tool stalled. Overkill for greenfield SaaS startups with no compliance constraint.

Contextual.ai — the SaaS-grounded-generation option

Managed grounded-generation platform with strong retrieval quality. Customers connect data sources, the platform handles ingestion, embedding, retrieval, and inference, and developers consume through APIs. The team comes from the original RAG academic work and it shows in the product.

Best fit: Mid-market and enterprise buyers without a hard data-residency requirement, building customer-facing AI experiences (support agents, knowledge assistants), and willing to absorb a standard SaaS vendor risk assessment. For most non-regulated enterprises this is the fastest path to a working production system.

Zep — the agent-memory option

Started as long-term memory for chatbots and has expanded into a broader memory and context layer. Strong primitives for storing user facts, conversation summaries, and session state across LLM calls.

Best fit: Startups and SMBs building AI chatbots, support agents, or assistant products where the dominant context need is "remember what the user said last session," not "reason over a 200-repo codebase with 800 ADRs."

LangChain — the framework option

The broadest open-source framework for building LLM-powered applications: chains, agents, tool integrations, vector store abstractions, and the LangSmith SaaS for tracing and evaluation. A framework, not a platform.

Best fit: Developer teams comfortable operating their own stack, research and experimentation, and organizations with strong internal AI engineering already in place. A phenomenal prototyping tool. A heavy operational burden once a custom system reaches production scale.

The Decision Framework

Walk the five questions in order. Most teams reach a decision before question three.

If you are…	Pick	Why
A non-regulated B2B SaaS or e-commerce company building grounded customer-facing AI	Contextual.ai	Time-to-value beats deployment overhead. Standard vendor risk assessment is acceptable.
A startup building chat-heavy products with strong agent-memory needs	Zep	Purpose-built for the use case. Cheapest path to working prototype.
An AI engineering team that wants maximum flexibility and operates its own stack	LangChain + vector store	No vendor lock-in. You bring the operational maturity.
A regulated enterprise (financial services, healthcare, defense, insurance) or any buyer where SaaS won't pass procurement	OutcomeOps	Customer-AWS deployment collapses procurement to a Terraform review. Inherits existing compliance posture.

Why Deployment Model Dominates for Regulated Buyers

For non-regulated buyers the deployment-model question is mostly a time-to-value calculation. SaaS wins because the friction is lower and the compliance overhead is acceptable. For regulated buyers the calculation inverts — and it inverts so completely that deployment model becomes the only feature that matters.

A few years before OutcomeOps existed, I led the AWS Control Tower landing-zone redesign at Gilead. We deployed sixty-plus Service Control Policies, turned on GuardDuty across the organization, stood up Macie for PHI and PII detection, rolled out Identity Center, and standardized permission sets so every new account inherited the same access model. As part of that program we also implemented TEAMS — AWS's Temporary Elevated Access Management for IAM Identity Center — so engineers could request just-in-time elevated access instead of carrying standing admin rights.

The security team made us file an exception. The reason: TEAMS uses AWS Amplify, and Amplify "is public."

The AWS Console is also public. So is IAM Identity Center. So is every AWS service the security team had logged into that morning. We were making the environment ten times more secure — and the conversation kept circling back to a TLS-protected, OIDC-gated Amplify domain that exposed nothing without authentication. That is the moment you learn that the word "public" carries more weight in a regulated-industry compliance review than what the architecture actually does.

Every context engineering platform a regulated buyer evaluates needs to survive that conversation. SaaS platforms with VPC isolation don't survive it because the data still gets processed in vendor infrastructure — and the legal team knows. Customer-deployed Terraform platforms survive it because there is no public endpoint, no vendor environment, and no new third party to add to the SOC 2 / HIPAA / FedRAMP scope. The internal ALB has no public DNS, no public IP, and is reachable only from the corporate network via Direct Connect plus Transit Gateway. The "is it public?" question has a one-word answer: no.

What "Customer-Managed Encryption Keys" Actually Buys You

The standard SaaS pitch in 2026 is: "customer-managed encryption keys, VPC isolation, BAA available, SOC 2 Type II report on request." This addresses three legitimate concerns and misses the structural one. The platform still runs in the vendor's cloud. Source code, ADRs, and inference outputs flow to the vendor for processing. CMEK protects the data at rest, but the data has to be decrypted to be embedded, retrieved, or fed to the LLM. The vendor's infrastructure, by definition, sees plaintext.

For most enterprise SaaS that's fine. For regulated industries it triggers a different review. The compliance team is not asking "is the data encrypted?" They are asking "does this introduce a new third party that needs to be assessed, contracted, and added to our SOC 2 / HIPAA / FedRAMP scope?" The answer for SaaS is always yes. The answer for customer-deployed Terraform is no.

Industry-Specific Notes

Financial services

Data residency, audit traceability, and MNPI handling dominate. Customer-AWS deployment in a single region with KMS-encrypted vector storage and customer-DynamoDB audit logs handles all three. SaaS platforms struggle on data residency for global banks with strict in-country processing requirements.

Healthcare and life sciences

HIPAA and HITECH dominate. Decision usually comes down to whether the platform can operate inside an existing HIPAA-eligible AWS account using Bedrock (HIPAA-eligible under AWS's BAA) or whether the platform requires a new BAA with the platform vendor. The first path takes weeks. The second takes quarters. For PHI-adjacent workloads, customer-AWS deployment is effectively the only path that completes inside a fiscal year.

Defense and aerospace

ITAR, CMMC, FedRAMP High. GovCloud regions. Often air-gapped. The platform must run in the customer's GovCloud account, support fully offline operation if required, and use only approved model providers. SaaS is generally a non-starter; on-prem container or air-gapped Terraform are the only viable paths. See Air-Gapped AI Coding for Defense and Aerospace.

Insurance

NAIC model laws, state-by-state insurance department requirements, and rapidly emerging AI-specific guidance from insurance regulators. Audit-traceability is the dominant pressure: regulators are explicitly asking for evidence of how AI is used in underwriting and claims. Customer-DynamoDB audit logs that the carrier's compliance team can query directly are the cleanest answer. Vendor-stored logs accessed by support ticket do not satisfy the regulator.

Years before AI coding tools existed, I built a containerized deployment platform for Aetna's consumer-business launch. We integrated Twistlock for container security and Checkmarx for SAST into the golden pipelines every team used. The result was 0.05% security defect density on the consumer code base — against the 5% defect density of Aetna's legacy core. The architecture team's first response when they saw the receipts was "we should do that." The same lesson applies to AI in 2026: regulated organizations don't need new policies, they need platforms that bake the controls in. The receipts win the conversation. The architecture is what produces the receipts.

Pricing Models (Briefly)

Pricing is moving too fast in 2026 to commit specific numbers to a blog post, but the structures are stable:

OutcomeOps — tiered enterprise license (Pilot, Team, Division, Enterprise). Customer pays AWS for compute. Pilot pricing is fixed and includes the deployment.
Contextual.ai — usage-based SaaS. Per-document, per-query, per-token tiers. Custom enterprise contracts above mid-market thresholds.
Zep — freemium with usage-based scaling. Self-hosted is open-core with paid tier for enterprise features.
LangChain — OSS framework, free. LangSmith priced per trace. Operational cost is the team you need to run your custom stack.

Build vs. Buy in 2026

The build case has gotten weaker since 2024. Better embedding models, MCP standardization, and managed inference (Bedrock, Vertex, Azure OpenAI) mean the "commodity" layers of a context engineering platform are now genuinely commodity. The remaining differentiation lives in:

The ingestion connectors (every enterprise has weird sources).
The metadata weighting and ADR-prioritization strategy.
The audit and policy layer that compliance actually accepts.
The deployment model (customer cloud vs. SaaS).

None of those four are easy to build well. Most teams that try discover they are six months into a 24-month project before they realize they've recreated the easy 60% of a platform and are now stuck implementing the hard 40%. Build-vs-buy on this category has tilted firmly toward buy — provided the buy option matches the deployment posture you actually need.

The Procurement Sequence That Works

For any buyer evaluating context engineering platforms, this is the sequence that completes inside a normal fiscal cycle:

Week 0: Internal alignment. Engineering, security, compliance, and procurement leads agree on the five-question framework and the weight each question carries in your environment.
Week 1: Vendor short list. Eliminate any platform that fails question 1 (deployment location). For most regulated buyers this leaves one viable option. For SaaS-friendly buyers it leaves two or three.
Week 2–3: Technical PoC. Apply the Terraform (customer-cloud) or complete vendor onboarding (SaaS). Connect 20 representative repositories. Generate code against real internal patterns. Inspect audit logs.
Week 4: Compliance review. For customer-cloud platforms this is reading Terraform. For SaaS this is the start of a longer vendor risk assessment.
Week 5–6: Production deployment to a single team or business unit. Limited rollout with full audit log review.
Week 7+: Phased expansion across the organization.

For regulated buyers using a customer-AWS platform, the technical evaluation and the compliance evaluation run in parallel because both reduce to reading the same Terraform. That parallelism is the entire reason the deployment model matters — it's what lets the procurement cycle complete in weeks instead of quarters.

The Honest Bottom Line

For non-regulated buyers in 2026, all four platforms can work. The deciding factor is your team's operational maturity, your willingness to operate a custom stack, and how customer-facing the AI surface is. SaaS wins time-to-value. Frameworks win flexibility. There is no wrong answer.

For regulated buyers, the deciding factor is whether the platform's deployment model lets your existing compliance posture cover it. Customer-AWS-deployed Terraform is the only pattern that does that cleanly. If you've already lost a quarter to a SaaS vendor security review and the next AI initiative needs to ship faster, the deployment model is no longer a feature comparison — it's the entire decision.

How to Evaluate

The two-week proof of concept is structured for this evaluation:

Day 1–3: Apply the Terraform into a non-production AWS account in your existing compliance scope. Verify the architectural bill of materials matches your existing patterns.
Week 1: Connect 20 representative repositories. Generate code against real internal patterns. Inspect audit logs in your DynamoDB. Verify no data egress.
Week 2: Compliance review of the deployment model. Confirm existing AWS posture covers the deployment without new vendor assessment.
Book an enterprise briefing to start the OutcomeOps PoC
Run the five-minute Readiness Assessment to get a written report on where your organization sits before scheduling

AWS Kiro + OutcomeOps: Context Engineering for Regulated Industries

Brian Carpio — Wed, 13 May 2026 13:32:00 +0000

I have been watching agentic IDEs closely. Tools like AWS Kiro deliver exactly what developers want in 2026: natural language → structured spec → working code, all inside a familiar VS Code-like environment. Spec-driven workflows feel magical when you are heads-down building.

But I keep coming back to the same pattern I have seen across every platform shift: spec-driven tools optimize locally. They are fantastic for a single repo or greenfield project. At enterprise scale — across legacy systems, compliance regimes, tribal knowledge, and decades of decisions — they hit the same wall.

That is why we connected Kiro to OutcomeOps over MCP.

The integration solves a pattern I see everywhere: developers love spec-driven IDEs because they are fast, delightful, and magical. Enterprises need persistent organizational intelligence — ADRs, code graphs, compliance patterns, the things that decay the moment they leave a single workspace. Most teams pick one or fake the other. This is how you get both.

The Local Optimization Trap (Again)

Kiro excels at:

Turning vague prompts into EARS-style requirements, architecture, task breakdowns, and implementation.
Adaptive autopilot across large codebases.
"Powers" for specialized domains (AWS services, observability, security tools).
Interactive steering and hooks.

Developers love it. I love parts of it.

What it does not have natively is persistent organizational intelligence — the code graphs, ADRs, architecture standards, compliance rules, and cross-repo patterns that live outside any one workspace. That is where most spec-driven output starts to drift in regulated or large organizations.

I covered the underlying pattern in Escaping Local Optimization Anti-Patterns. This integration is the practical follow-up.

How the Hybrid Actually Works

OutcomeOps deploys fully into your AWS account (Terraform, air-gapped, GovCloud and FedRAMP ready) — the deployment model regulated industries demand. This is not security theater: code, ADRs, and inference all stay inside your trust boundary. It ingests your GitHub or GitLab repos, Confluence, ADRs, SharePoint, and Jira, and auto-generates queryable code maps that become a living enterprise code graph.

We expose this as an MCP server running in your VPC.

Kiro, with its strong MCP client support, connects to it like any other Power or tool. Once configured (usually 5 to 10 minutes), Kiro agents can:

Query the full organizational knowledge base with citations.
Pull relevant ADRs before generating code.
Validate against architecture standards and detect drift in real time.
Reason over enterprise-wide dependencies instead of just local files.

In practice it feels like this:

"Using our OutcomeOps context, implement feature XYZ following ADR-147 and the payment service patterns from the code graph."

Kiro handles the delightful spec-driven flow and interactive development. OutcomeOps supplies the systemic context and enforcement layer. No extra SaaS data exfiltration. Everything stays in your trust boundary.

Side-by-Side: Kiro Alone vs. OutcomeOps Alone vs. Combined

Aspect	Kiro Alone	OutcomeOps Alone	Kiro + OutcomeOps
Developer Experience	Excellent (IDE-first)	Strong chat + autonomous	Best of both — interactive IDE with deep context
Spec-Driven Planning	Native & polished	Ticket-to-PR focused	Full spec flow + enterprise rules
Organizational Memory	Local + steering files	Persistent code graphs + ADRs	Queryable enterprise intelligence
Compliance & Traceability	Good (with hooks)	Strong (ADR linking, drift detection)	Trace every line back to decisions
Deployment Model	Client + cloud inference	Fully in your AWS account	Same air-gapped security
Best For	Individuals, greenfield, rapid iteration	Large regulated orgs, legacy modernization	Teams that want velocity and governance
Data Control	Depends on model provider	Zero exfil by default	Zero exfil

Real-World Payoff We Are Seeing

Teams using this pattern report:

Higher first-time PR approval rates because standards are enforced during generation.
Reduced architectural drift across portfolios.
Developers stay in their preferred IDE while the AI knows how the company actually builds software.
Knowledge compounds: every accepted PR and ADR makes future work smarter.

A concrete example. A SaaS engineering team shipping payment handlers saw this play out in their first week. Kiro generated clean Stripe integration code from a spec. OutcomeOps injected the Decimal ADR during generation — the same one that took the float-to-Decimal failure rate to zero across 226 sessions in our own work. Result: zero float bugs across 15 PRs. Before the integration that ADR lived in Confluence and got caught in code review about 40 percent of the time.

This is not theoretical. It is the same Context Engineering loop we have been shipping since mid-2025 — now wired into the spec-driven IDE developers already want to use.

Setup Is Straightforward (for Enterprise)

Deploy OutcomeOps via Terraform (or we run the PoC with you).
Enable the MCP server endpoint in your VPC.
Add the config to Kiro (.kiro/settings or workspace file — similar to other Powers).
Optional: bundle it as a custom Power for one-click activation across your org.

We handle auth, schema compatibility, and performance tuning for large graphs.

When This Does Not Make Sense

If you are a three-person startup building a greenfield SaaS product with no compliance requirements, just use Kiro. The integration overhead is not worth it yet. OutcomeOps compounds value as organizational complexity grows: legacy systems, multiple teams, regulated industries, decades of tribal knowledge.

Sweet spot: 20+ engineers, a regulated industry, or any codebase where "just ask Steve" is how architecture decisions actually propagate.

The Bigger Picture

Agentic IDEs like Kiro are the creative front end. Context Engineering platforms like OutcomeOps are the durable back end. Together they move us past "vibe coding plus manual review" toward real outcome ownership.

If you are already evaluating Kiro (or any spec-driven IDE), I strongly recommend layering your organizational intelligence on top. Do not let great local tools create systemic debt.

See It Live With Your Repos

30-minute enterprise briefing. We connect Kiro to a temporary OutcomeOps instance using your actual ADRs and code graphs. No pitch — just watch your organizational intelligence flow into Kiro's spec-driven workflow.

Bring a messy legacy repo. We will show you what happens when systemic context meets local tools.

The o16g Manifesto Validates What We've Been Building Since July

Brian Carpio — Fri, 08 May 2026 05:53:47 +0000

The o16g Manifesto Validates What We've Been Building Since July

By Brian Carpio — February 12, 2026

Yesterday, Cory Ondrejka — co-creator of Second Life, the engineer who saved Meta, and current CTO of Onebrief — published a manifesto called Outcome Engineering (o16g). Charity Majors, CTO of Honeycomb, said it practically had her doing cartwheels. It's making the rounds on LinkedIn and for good reason.

Go read it. I'll wait.

Here's what struck me: we've been building the platform that implements these principles since July 2025. Not because we read Cory's manifesto — it didn't exist yet. Because when you spend 20 years leading enterprise transformations and then sit down to build something from scratch, you arrive at the same conclusions.

That's not a flex. That's validation. When a Meta CTO and a Fortune 500 practitioner independently converge on the same philosophy, it means the philosophy is right.

The Outcome Engineering Convergence

Cory opens with: "It was never about the code."

In July 2025, we opened with: "DevOps is dead. Not because the ideas were wrong, but because the implementation lost the plot." Same observation, same starting point. The industry optimized for the wrong things — deployments, velocity, pipeline metrics — while the outcomes that actually matter went unmeasured.

Cory calls the new model Outcome Engineering. We call it OutcomeOps. The name doesn't matter. What matters is the shared realization that engineering must be measured by impact, not activity.

Let me walk through four of Cory's principles and show what the implementation actually looks like.

Point 6: "The Map" — No Wandering in the Dark

"Never dispatch an agent without context. Map the territory before building. If you don't know where you stand, you cannot calculate the path to the destination."

— Cory Ondrejka, o16g

This is Context Engineering. We named the discipline in October 2025 and defined it as the craft of designing the environment in which AI thinks — the knowledge, rules, and context that determine its effectiveness. Not prompts. Systems.

In practice, that means before any AI touches your codebase, you've already indexed your Architecture Decision Records, your dependency manifests, your documentation, your Jira issues, your Confluence pages. The AI doesn't wander. It operates within the boundaries of what your organization has already decided, built, and documented.

When an engineer asks "do we have a Terraform module for RDS with encryption at rest?" — the platform doesn't guess. It searches the indexed code-maps, finds the exact module, cites the ADR that explains why it was built that way, and links to the repo. The territory is mapped before anyone asks a question.

Point 11: "The Graph" — All the Context, Everywhere

"Agents cannot reason in a vacuum. Embed context into the infrastructure, not just the prompt."

— Cory Ondrejka, o16g

This is the core of what OutcomeOps does. We don't bolt AI onto the side of your workflow. We index your GitHub repos (code-maps, dependencies, ADRs, documentation), your Confluence spaces, your Jira projects, and your Outlook communications into workspace-scoped knowledge bases. The context lives in the infrastructure — vectorized, chunked, retrievable, and scoped to the team that needs it.

The workspace model is how this scales without chaos. A security team's workspace has their repos, their standards, their compliance artifacts. A developer team's workspace has their services, their ADRs, their backlog. The context boundaries are intentional and enforced. There's no cross-pollination unless the organization explicitly configures directional sharing.

This isn't a feature. It's the architecture. Context embedded in infrastructure, not stuffed into a prompt.

Point 4: "The Liberation" — The Backlog is Dead

"The backlog is a relic of human limitation. Never reject an idea for lack of time, only for lack of budget. If the outcome is worth the tokens, it gets built. Manage to cost, not capacity."

— Cory Ondrejka, o16g

In December 2025, we introduced the Outcome Engineer — an engineer who doesn't receive user stories from a Product Owner, but identifies business problems directly, defines success metrics upfront, and uses AI to handle implementation. The measuring stick isn't story points or velocity. It's attributed revenue, customer lifetime value, and feature adoption rate.

The practical proof: we reduced 16-hour development tasks to 15-minute implementations at $2-$4 per feature. That's not a benchmark from a whitepaper. That's measured production data from Fortune 500 delivery. When the cost of building drops by two orders of magnitude, the backlog doesn't constrain you anymore. Budget does. Exactly as Cory describes.

Point 16: "The Validation" — Audit the Outcomes

"Trust is a vulnerability. Models drift. Prompts break. Capabilities change overnight. Continuously audit the agent against the domain. Verify the tool is sharp before you use it."

— Cory Ondrejka, o16g

We built a seven-layer defense system around our LLM pipeline — input moderation, refusal detection, forced refusal QA testing, logging, alerting, miss detection, and regression testing. Every input and output is logged. Every refusal is caught and categorized. Every moderation failure triggers a notification. The system doesn't trust the model. It verifies the model, continuously.

This wasn't academic. We built it because we had to. When you run AI in production across enterprise environments, you can't hope the model behaves. You build systems that prove it does — or catch it when it doesn't. The audit trail isn't a nice-to-have. It's the table stakes for enterprise trust.

Outcome Engineering Needs Implementation

The o16g manifesto is the philosophy enterprises need to hear. For engineering leaders redefining how they measure value, it's a north star. But philosophy needs implementation. And implementation at enterprise scale adds chapters that no manifesto can cover.

Compliance requirements, cybersecurity supplements, air-gapped deployment mandates, 40-page reseller agreements — these aren't obstacles to outcome engineering. They're the terrain where it gets real.

Enterprise reality adds constraints that make the philosophy stronger, not weaker:

Information boundaries matter. You can't give every agent "all the context, everywhere" when the organization has classified information types and regulatory obligations. Workspace scoping — limiting what AI can see based on team, role, and data classification — is how you implement Cory's Point 11 without creating a compliance nightmare.

Deployment model matters. When the platform runs inside the customer's AWS account, not your SaaS environment, the trust model changes entirely. The customer controls their data, their keys, their network boundaries. That's not a limitation — it's what makes the philosophy viable for organizations that can't send their source code to someone else's servers.

Audit trails matter. Point 16 says "audit the outcomes." In enterprise, that means every question asked, every answer returned, every source cited — logged, timestamped, and reviewable for 12 months minimum. Not because you want to spy on engineers, but because when the compliance audit comes, you need receipts.

These aren't objections to the manifesto. They're the next chapters. The ones that turn a philosophy into a platform enterprises will actually deploy.

The Takeaway

Cory Ondrejka wrote the philosophy beautifully. We've been building the implementation since July. The convergence is the point.

When a CTO who saved Meta and a practitioner who's led transformations at Fortune 10 companies both independently arrive at the same conclusion — it was never about the code, it was always about the outcomes — that's not coincidence. That's a signal.

The question for every engineering organization is the same one we asked in our first blog post: are you measuring your work by how fast you ship, or by the value it creates?

The manifesto has been written. Twice now. The platform exists. The Outcome Engineer is already here.

Time to Build.

The philosophy has been validated. The platform is ready. See how OutcomeOps implements outcome engineering at enterprise scale.

Context Engineering: The Next Evolution Beyond DevOps

Brian Carpio — Fri, 17 Apr 2026 16:24:30 +0000

The Era of AI Demands a New Operating Philosophy

Every major corporate revolution begins the same way: a set of best practices, a few tools, and a promise of transformation.

DevOps started that way. So did Agile. So did Cloud.

But each time, 80% of the Fortune 500 missed the point. They adopted the tools, not the mindset.

They automated pipelines without aligning outcomes.

They measured deploys instead of value.

They confused motion with progress.

Now AI is here, and it's about to happen again.

OutcomeOps: The Culture of Augmented Outcomes

OutcomeOps is the cultural evolution that re-centers the enterprise around results, not rituals.

It asks a simple but uncomfortable question:

Are we delivering outcomes that matter—or just producing outputs that look impressive?

Where DevOps unified development and operations through automation, OutcomeOps unifies human cognition and machine intelligence through augmentation.

It's not about speed—it's about alignment.

Not just "move fast," but move effectively.

An OutcomeOps organization measures success by the clarity of its outcomes and the velocity of its learning loops. It turns tools, data, and AI into partners—not shortcuts—in achieving those outcomes.

Context Engineering: The "How" Behind the Philosophy

If OutcomeOps is the culture shift, Context Engineering is the engineering discipline that makes it real.

Context Engineering is the craft of designing the environment in which AI thinks—the knowledge, rules, and context that determine its effectiveness.

It's how teams build LLM-aware systems that don't just generate words, but generate reliable reasoning.

At its core, Context Engineering answers this question:

How do we give AI the right information, at the right time, in the right form, to make the right decisions?

The answer is not prompts. It's systems.

Systems that:

Version and retrieve architectural decisions, standards, and code samples
Provide structured grounding for LLMs to reason within enterprise guardrails
Enable feedback loops where engineers and AI co-review code, enforce standards, and learn together
Create a persistent memory of decisions and trade-offs that improves with every interaction

An AI-assisted engineering platform—one that lives inside tools like Teams, GitLab, and Jira—sits at the foundation of Context Engineering.

– It operationalizes knowledge.

– It embeds augmentation into daily work.

– It transforms every question like "why did my build fail?" into a learning moment backed by institutional intelligence.

It's a system that learns and delivers.

From DevOps to OutcomeOps: Culture Meets Craft

DevOps gave us automation.

OutcomeOps gives us augmentation.

In the same way CI/CD pipelines transformed how we deliver code, Context Engineering transforms how we deliver cognition. It's continuous integration and delivery not of software, but of intelligence.

OutcomeOps redefines success around results, not rituals.

Why 80% of the Fortune 500 Will Miss It Again

Because they'll repeat the same mistake.

They'll buy a product instead of building a philosophy.

They'll hire AI engineers but never teach leaders how to think in systems.

They'll plug LLMs into old processes instead of redesigning the processes around outcomes.

They'll measure AI usage, not AI-driven impact.

Just like they once measured deploy frequency without asking whether those deploys created value.

The companies that get it will realize this:

OutcomeOps is not about adopting AI faster it's about aligning intelligence, human and machine, toward the outcomes that actually matter.

Those who master that loop—Prompt → Inspect → Refine → Align → Ship—will out-learn and out-execute everyone else.

The Future Operating Model

OutcomeOps is the culture. Context Engineering is the craft.

Together, they form the blueprint for enterprises that evolve, learn, and deliver in real time. We proved this when we refactored a 1,348-line Lambda using Context Engineering in under an hour, and again when 3 ADRs transformed AI-generated code on Spring PetClinic.

The future of engineering isn't about shipping code faster.

It's about teaching organizations to think in feedback loops—with AI as a co-engineer, not a vending machine. The engineers who embrace this shift are becoming Outcome Engineers — measuring revenue, not deployments.

The companies that understand this will redefine speed, quality, and intelligence itself.

The rest will keep automating their way to irrelevance.

Brian Carpio is the founder of OutcomeOps. outcomeops.ai

What AI-Assisted Development Actually Looks Like in Two Years

Brian Carpio — Fri, 17 Apr 2026 16:07:08 +0000

The pattern is consistent enough to make predictions. Not with certainty. With pattern recognition.

Charity Majors said something worth taking seriously: "No one knows what AI-assisted software development will look like in two years. NO ONE. Anyone who says anything differently is selling something."

She's right that certainty is the wrong posture. She's wrong that the pattern is unknowable.

I'm not a researcher. I'm not an analyst. I'm a practitioner who has watched the same transformation cycle play out five times across five different technology waves at some of the largest enterprises in the world. Cloud. DevOps. Containers. Platform Engineering. Now AI.

The arc is consistent enough to make predictions. Not with certainty. With pattern recognition.

Here's what I think actually happens in the next two years — the good and the bad.

The Pattern That Keeps Repeating

Every major platform shift in enterprise technology goes through four phases. I've lived through all of them, multiple times.

Phase 1: Individual productivity. Early adopters go faster. They write blog posts about it. The productivity gains are real but not transferable — they live in individual workflows, not organizational systems.

Phase 2: Local optimization. Every team builds their own version. One team at Company A spends two sprints wiring up the new technology. A different team at Company B does the same thing simultaneously. Neither knows the other exists. Neither captures what works. The knowledge stays fragmented.

Phase 3: The reckoning. The production incidents arrive. The technical debt surfaces. The downstream engineers — the ones who weren't in the LinkedIn posts — start dealing with the consequences. Charity is describing this phase happening right now with AI-generated code.

Phase 4: Organizational intelligence wins. The teams that survive and thrive are the ones who encoded the knowledge into a platform layer. Guardrails over gatekeepers. The right path becomes the easy path. Local optimization gives way to compounding leverage.

I watched this happen with cloud automation at Pearson in 2012. Containers and platform engineering at Aetna in 2014 — we were running Docker on Mesosphere before Kubernetes existed. A Docker rescue at Liberty Mutual in 2016. Platform engineering at scale at Comcast in 2019. AWS landing zones and cloud modernization at Gilead in 2022.

AI is in Phase 3 right now. Phase 4 is coming, and it's coming faster than previous cycles because the technology is moving faster.

What Two Years Actually Looks Like

The first half of the next two years looks like Charity's post.

More production incidents from AI-generated code that nobody fully reviewed. More downstream engineers dealing with "magic" that wasn't. More organizations mandating AI adoption while providing no framework for doing it well. More teams rebuilding context injection in isolation, sprint after sprint, capturing nothing.

This is not AI failing. This is the reckoning phase working as designed. The reckoning is how enterprises learn what governance they actually need.

The second half looks different.

The organizations that survive the reckoning will have built something: a context layer. Not a set of individual prompts. Not a team-specific .cursorrules file. A queryable, version-controlled corpus of organizational knowledge — ADRs, code-maps, compliance requirements, architectural decisions — that AI queries before generating a single line.

This is what makes AI output reviewable, repeatable, and trustworthy at scale. Not better models. Not more prompting. Encoded organizational intelligence.

The organizations that haven't built it by month 18 will be watching the ones that did compound their advantage at a rate that manual processes cannot match.

The Developer Headcount Prediction

I'll say the quiet part out loud.

The teams that get the context layer right will ship the same output with fewer developers. Not because AI is replacing engineers — because the leverage ratio changes fundamentally.

I built 90 Lambda functions in 120 days, solo, using OutcomeOps against my own ADRs and code-maps. That's not 90 functions of vibe code. That's 90 production functions with tests, consistent patterns, and deployment pipelines — audited by the same AI that generated them.

At a Fortune 500 company running OutcomeOps in production right now, 16-hour tasks complete in 20 minutes. First-time approval rate is 90%. Cost per feature is $2.24.

Those numbers don't leave headcount unchanged. A team of 20 engineers operating with this leverage ratio does not need to grow to 40 engineers to double output. They might need 22.

This isn't speculation. It's the same math that played out when cloud eliminated the need for physical datacenter teams. When Puppet and Chef eliminated the need for armies of sysadmins. When Platform Engineering at Comcast eliminated the need for every team to write their own Terraform.

The offshore consulting model built on labor arbitrage faces the same math.

The engineers who survive and thrive are the ones who move up the abstraction layer. Not the ones writing the most code. The ones encoding the most organizational knowledge.

The ones writing ADRs instead of tickets. Designing context instead of functions. Teaching the system instead of feeding it prompts.

Why Most Predictions Get This Wrong

The optimists predict a smooth transition where AI makes everyone more productive and nobody loses. The pessimists predict mass displacement and a race to the bottom on developer salaries.

Both miss the real variable: whether the organization encodes its knowledge before or after the reckoning.

Organizations that build the context layer proactively come out of Phase 4 with a compounding advantage. Smaller teams. Faster output. Higher quality. The institutional knowledge is in the system, not in the heads of engineers who might leave.

Organizations that don't get to Phase 4 at all. They stay in the reckoning, dealing with incident after incident, until either leadership mandates a framework or a competitor that got there first makes the decision for them.

The split outcome is not optimistic or pessimistic. It's what always happens when a platform shift arrives and enterprises have to decide whether to encode the knowledge or keep it in people's heads.

What Charity Is Actually Asking For

She's asking for truth-telling. For practitioners who will say what worked and what didn't, without shining it up past recognition.

Here's mine.

OutcomeOps works in production. The metrics are real. But it only works because the organizational knowledge was encoded first. The ADRs exist. The code-maps are current. The compliance requirements are queryable. Without that foundation, AI generates plausible-looking code against an imaginary codebase — and the downstream engineers deal with the consequences.

The "vibe coding" problem Charity is describing is not an AI problem. It's a context engineering problem. The organizations solving it are pulling away from the ones that aren't.

In two years, the gap will be visible to everyone.

Brian Carpio is the founder of OutcomeOps and has spent 13 years leading enterprise cloud, DevOps, and platform engineering transformations at Pearson, Aetna, Comcast, Gilead Sciences, and as an AWS ProServe Principal. OutcomeOps deploys into your AWS account via Terraform. outcomeops.ai

I Drove $31M in Bookings. The System Said I Needed Improvement.

Brian Carpio — Wed, 18 Mar 2026 19:37:45 +0000

Here's what my peers wrote in my performance review: "One of the better examples of what the Leadership Principles are all about." "A thought leader and transformation driver." "His ability to lead large-scale cloud transformations, combining deep technical expertise with strategic long-term vision." "A trusted advisor across organizations."

Here's what the system said: Needs Improvement. Development Needed.

Same review. Same year. Same company.

The Numbers
I led the largest cloud consulting engagement in the healthcare and life sciences vertical for a major cloud provider's professional services org. An $80B biopharma company. 55 engineers across 5 scrum teams. $18M in revenue. $31M in bookings. The engagement became the reference architecture — the blueprint for how every future deal in that vertical was sold. I flew out with the VP of the entire HCLS vertical to help close deals with other Fortune 500 pharma companies. Leadership was pulling me off the engagement to do trusted advisor work with new customers. I gave private executive briefings before dinner at the industry's largest conference to CIOs and CISOs from some of the largest pharmaceutical companies in the world.

I got production access approved for our consultants in a customer environment — something that had never been done before. Two people tried before me and failed. I wrote a 30-page security narrative, sat in front of review boards for 9 months, and got it approved. Then I got a third-party consulting partner on our paper approved for prod access too — that was unheard of.

Account vending went from 30 days to 45 minutes. The customer got early access to generative AI services because the foundation we built made it possible. Their CIO presented our work on the keynote stage at the industry's largest conference. The platform won Intelligent Digital Enterprise of the Year, Data Mesh of the Year, and a CIO 100 Award.

What I Didn't Do
While I was running all of that — while I was up until midnight protecting my team during a production incident, while I was banning the sales team from making promises we couldn't deliver, while I was navigating political firestorms between consultancies and internal security teams — I didn't write enough internal blog posts. I didn't get a basic intro-to-AI certification by a specific date — while I was literally working with the AI product team during early release, helping my customer implement it before it was even generally available. I now run three AI platforms. But the system needed that certificate. I submitted some timecards late.

That's what the system measured. Not the $31M. Not the reference architecture. Not the executive briefings at re:Invent. Timecards.

The Customer Satisfaction Score
It gets better. Part of the performance system was tied to customer satisfaction surveys. Our customer gave us an 8 out of 10. In this company's system, an 8 was a nuclear event. Suddenly directors were calling the customer, leadership was in crisis mode, and every engineer on the engagement took a hit in their performance review.

The customer's actual feedback? He was happy with the thought leadership — meaning me and my leads — but unhappy with the number of offshore resources. A staffing decision his boss and the sales team made. Not us. He later told me: "Brian, from now on just tell me what to put in the forms. I'm not trying to cause problems for you all."

But the score was in the system. And the system doesn't do nuance.

The Sales Credit Problem
Here's the part that still stings. The engagement I built became the go-to-market template for the entire vertical. My peers wrote it explicitly: "This has translated to a go-to-market approach" for the industry. But I never got credit for "building the business." The sales team packaged up what I delivered and sold it as their own. In the performance system, they got the points. I got told to write more blog posts.

Why This Matters Beyond My Story
I'm not writing this for sympathy. I took the severance. I'm building OutcomeOps now and I've never been more energized. I'm writing this because every senior engineer and technical leader in a large organization has some version of this story. The system rewards the people who optimize for the system, not the people who own the outcome.

When you measure what actually matters — time to outcome, customer success, business impact, architectural quality — you find the people who are actually driving your organization forward. When you measure timecards and certifications, you find the people who are best at filling out forms.

I built OutcomeOps because I'm done waiting for large organizations to figure this out. The Outcome Engineer doesn't exist inside a stack ranking system. They exist in operating models that escape the local optimization trap and align everyone around one thing: did the customer win?

My peers knew the answer. The system didn't care.

The $75 Billion Offshore Consulting Industry Dies in the Next 36 Months

Brian Carpio — Sun, 30 Nov 2025 19:41:07 +0000

The Uncomfortable Truth About Offshore

Every enterprise has the same story.

You hired TCS, Infosys, Wipro, or Cognizant. The pitch was compelling: $45/hour versus $150/hour. The math was obvious. You'd save millions.

Then reality hit.

You don't have 200 engineers in India. You have 12-18 real shippers in the US and a 180-person tax.

I've seen this pattern repeat across cable, big pharma, insurance, and more. At one Fortune 500, they had 55 engineers on the books. Only 3 were trusted to deliver end-to-end without handholding. The other 52? They generated work for those 3.
The Hidden Math

Here's what the offshore sales deck doesn't show you:

The Promise:

•Offshore rate: $45/hour
•Onshore rate: $150/hour
•"You'll save 70%!"

The Reality:

Your senior engineers - the ones you're paying $180K+ - spend 40% of their time:

•Reviewing offshore code
•Rewriting offshore code
•Explaining requirements for the third time
•Fixing production issues from code that "worked in dev"
•Attending 7am calls to bridge time zones

That's not savings. That's a tax.

The Real Cost:

•Offshore hourly rate: $45
•+ Senior engineer babysitting (40% of $85/hr loaded): $34
•+ Rework cycles (average 2.3x): $103
•+ Production incidents: $??
•+ Delayed time-to-market: $???

= Actual cost: $180+ per hour of delivered value

You're paying MORE than onshore rates for WORSE code and SLOWER delivery.
Why It Persists

Three reasons this model survives despite the math:

Sunk Cost Fallacy

"We've already invested in the Bangalore office. We can't just shut it down."

Headcount Theater

Some executives measure success by team size. 200 engineers looks better in a board deck than 50, even if 50 would ship more.

Nobody's Done the Real Math

Finance sees the hourly rate. They don't see the senior engineer time drain. It's not in any dashboard.
Enter AI (But Not How You Think)

Here's where it gets interesting.

Everyone's talking about AI coding assistants. Cursor raised at a $29 billion valuation. GitHub Copilot is on every developer's machine. The narrative is "AI will make developers faster."

That's thinking too small.

The real disruption isn't making your 200 offshore engineers 20% faster at typing. It's eliminating the need for 180 of them entirely.
Old Model New Model
12 seniors + 180 offshore 12 seniors + AI
Seniors review/rewrite offshore code Seniors review AI-generated code
40% of senior time on babysitting 90% of senior time on architecture and hard problems
$5M+ annual offshore spend $500K AI platform + token costs
Why Cursor and Copilot Won't Be the Disruptors

Here's the problem: Cursor and Copilot can't serve the enterprises that need this most.

Regulated industries - healthcare, pharma, finance, manufacturing - have a non-negotiable requirement: code cannot leave the environment.

Cursor ships your code to their servers. Copilot ships your code to Microsoft. For a hospital system under HIPAA, a pharmaceutical company under GxP, or a bank under SOX, that's a non-starter. Legal will never sign off.

The $75 billion offshore industry exists primarily in these regulated enterprises. They're the ones with:

•Massive codebases
•Complex compliance requirements
•Conservative technology adoption
•Deep pockets for consulting spend

And they're exactly the customers Cursor and Copilot architecturally cannot serve.
The Disruption Model

The platform that disrupts offshore won't be a cloud IDE. It will be:

Deployed in the customer's environment.

Your code stays in your AWS account. Your VPC. Your compliance boundary. Models run on Bedrock - AWS's commitment that your data never trains their models.

Context-aware, not just autocomplete.

Offshore fails because developers don't understand your architecture. AI that's ingested your code maps, your ADRs, your patterns - that knows WHY your system works the way it does - generates code that actually fits.

Multiplying seniors, not replacing them.

The goal isn't AI writing code unsupervised. It's AI generating PRs that your senior engineers can review in 15 minutes instead of rewriting in 4 hours.
The Math That Actually Works

Let's redo the comparison:

Current State (Offshore Model):

200 offshore engineers: $5M/year
15 senior engineers (40% time on oversight): $1M/year of their time
Rework, delays, incidents: $1M+/year

Total: $7M+ for mediocre output

Future State (AI-Augmented Model):

15 senior engineers (full capacity): $2.5M/year
AI platform: $500K/year
Token costs: $200K/year

Total: $3.2M for better output, faster

That's not a 20% improvement. That's a 50%+ cost reduction with higher quality and faster delivery.
Who Dies, Who Thrives

Losers:

•Body shops selling headcount
•Consulting firms billing for "resources"
•Any model predicated on labor arbitrage

Winners:

•Platforms enabling AI-augmented development
•Consulting firms that pivot to implementation and transformation
•Senior engineers (their value just went up)
•Enterprises willing to make the shift

I've seen the bodies stack up across cable, pharma, insurance, and hospitality. This time I'm selling the weapon.
The Timeline

This isn't a 10-year prediction. The technology exists today.

What's missing is enterprise adoption - and that's accelerating. Every CTO I talk to has the same frustrated look when offshore comes up. They know it's not working. They've just been waiting for an alternative that doesn't require sending their crown jewels to San Francisco.

Within 36 months, maintaining a 200-person offshore team when 20 engineers with AI could outship them will be seen as malpractice.

The $75 billion question: will the incumbents adapt, or will they go the way of every other industry built on an arbitrage that technology eliminated?
The Offshore Era Is Over

The only question left is whether you'll still be paying the tax in 2027.

Real data: One enterprise deployment (2 teams, 41 repos) shipped over 100 production PRs in 30 days using OutcomeOps.

That's more working code than most 200-person offshore teams deliver in a quarter.

Pilots are running and pay for themselves 4-5x in the first 90 days.
Schedule the 30-minute briefing →

OutcomeOps: The Operating Model for Engineers Who Own the Outcome

Brian Carpio — Mon, 10 Nov 2025 13:49:36 +0000

We’ve hit a wall in the software industry—and most people are too deep in Jira tickets or conference slides to realize it.

DevOps is dead. Not because the ideas were wrong, but because the implementation lost the plot. It was supposed to be about breaking down silos, accelerating delivery, and aligning engineering with outcomes. Instead, it got hijacked by process theater and rebranded operations teams.

Today? “DevOps” means YAML jockeys babysitting pipelines, managing Terraform drift, and debating whether Snyk or Prisma is more “shift-left.” We turned a movement into a tooling checklist.

And then there’s “vibe coding”—the aesthetic obsession with dark-mode VSCode, AI copilots, and working from a van in Patagonia. It’s cool for Twitter. It looks good in Reels. But it doesn’t ship. It doesn’t solve. It doesn’t scale.

We’re measuring activity, not impact. Shipping complexity, not clarity. We’ve created high-functioning teams that still produce mediocre outcomes because the operating model is broken.

It’s time for a new one.
Introducing OutcomeOps

OutcomeOps is an operating model for engineers who own the result, not just the release.

It’s born from experience—leading DevOps and cloud transformations for some of the largest Fortune 500s over the last 15 years, and more recently, building an AI platform from scratch. No committees. No tickets. Just fast, secure, reliable delivery tied directly to measurable business value.

OutcomeOps is:

• Pattern-Based Delivery: not a 1,000 microservices, but proven, repeatable design patterns that balance speed with sanity.
• Signal-First Feedback Loops: observability, not just logs. Signals that close the loop on quality, performance, and value.
• Compliance Built-In: security and compliance from the start. Not afterthoughts, not audit-season panic.
•Engineers as Owners: no more deployment handoffs. If you build it, you run it. If it breaks, you fix it.
• Monetization Mindset: everything you ship should tie back to outcomes—users, revenue, satisfaction, impact.

OutcomeOps is not another process framework. It’s not a product. It’s a mindset, a structure, and a standard. It’s the difference between engineering as ceremony… and engineering as execution.
OutcomeOps Is How

Building real products forces you to confront everything theory papers skip. In under 90 days, I shipped an AI platform with real paying users, over 70 Lambda functions, Grafana dashboards, and full infrastructure automation.

Not to brag—just to highlight that what I’ve been teaching companies for years is the same model I used to build this platform. Most people thought I was just teaching Terraform and CICD pipelines. But what I was really teaching was outcome-based thinking: how to ship to production 2–4 times a day, how to focus on user impact over tool debates, how to cut through complexity and get results.

Meanwhile, the industry is still debating EKS vs ECS while I’m building a fully functional, audit-passing, self-moderating, bank-integrated platform that reconciles its own ledger.

OutcomeOps isn’t theory. It’s practice. And it works.

Blogs are coming. Patterns are coming. Real-world examples—failures and wins—are coming.

This is the new model.

DEV Community: Brian Carpio

How to Find Your Own Code Inside ChatGPT — The Tiger Team Method Every Engineering Leader Should Run This Week

The Tiger Team Method

What the Research Already Shows

You've Lived This Movie Before

Pearson, 2012–2014 — The Nibiru Platform

Aetna, 2014–2017 — The Utopia Platform

Liberty Mutual, 2016–2017 — The Fusion Platform

Comcast, 2019–2022 — The SEED Platform and the Governance Argument

The Pattern Is the Same. The Layer Is Different.

What the AI Platform Has to Do

The Nucleus Argument

Run the Test Tomorrow

How to Evaluate

Run the Tiger Team Test. Then Build the Nucleus.

Related reading

What Is an AI Engineering Platform? (2026 Guide)

The Comparison Table (Above the Fold)

Definition: What an AI Engineering Platform Actually Does

1. Context layer

2. Generation layer

3. Output layer

4. Audit layer

5. Deployment layer

Why "Platform," Not "Coding Assistant"

The Four Platforms, in Detail

OutcomeOps — the customer-cloud platform

Devin — the autonomous-agent platform

Cursor — the IDE platform (at enterprise scale)

GitHub Copilot — the broad-adoption platform

The Five-Criteria Evaluation Framework

For Regulated Industries Specifically

What Changed in 2026

When You Don't Need One Yet

How to Evaluate

Related reading

AI-Driven Enterprise Search Needs an AI-Ready Foundation

What does an AI-ready foundation actually look like?

Why does AI surface foundation gaps so fast?

What about regulated industries?

How RetrieveIT approaches the foundation

The pattern, restated

Context Engineering Platforms: A Comparison Guide

The Five Criteria That Actually Decide

The Comparison Table

Platform-by-Platform

OutcomeOps — the customer-cloud option

Contextual.ai — the SaaS-grounded-generation option

Zep — the agent-memory option

LangChain — the framework option

The Decision Framework

Why Deployment Model Dominates for Regulated Buyers

What "Customer-Managed Encryption Keys" Actually Buys You

Industry-Specific Notes

Financial services

Healthcare and life sciences

Defense and aerospace

Insurance

Pricing Models (Briefly)

Build vs. Buy in 2026

The Procurement Sequence That Works

The Honest Bottom Line

How to Evaluate

Related reading

AWS Kiro + OutcomeOps: Context Engineering for Regulated Industries

The Local Optimization Trap (Again)

How the Hybrid Actually Works

Side-by-Side: Kiro Alone vs. OutcomeOps Alone vs. Combined

Real-World Payoff We Are Seeing

Setup Is Straightforward (for Enterprise)

When This Does Not Make Sense

The Bigger Picture

See It Live With Your Repos

The o16g Manifesto Validates What We've Been Building Since July

The o16g Manifesto Validates What We've Been Building Since July

The Outcome Engineering Convergence

Point 6: "The Map" — No Wandering in the Dark

Point 11: "The Graph" — All the Context, Everywhere

Point 4: "The Liberation" — The Backlog is Dead

Point 16: "The Validation" — Audit the Outcomes