Paulo Victor Leite Lima Gomes

Posted on Sep 12

hybrid ai is becoming a platform contract

#ai #softwareengineering #opinion #devops

The most important AI number of the last month was not a benchmark score.

On August 11, 2026, Cloudera published survey results saying 95% of enterprises had delayed or canceled AI projects because infrastructure, governance, compliance, cost, or data architecture could not keep up. In the same release, 66% said they had already shifted some AI workloads from public cloud back to private cloud or on-premises.

Read that again, slowly, because there are two very different stories hiding in it.

The loud story is "AI projects fail." Conferences love that one. It gives everyone a panel topic and a scapegoat, and it conveniently ignores the fact that most of these projects were not canceled because the model was bad. They were canceled because the organization could not answer basic operational questions about them.

The quiet story is the real one: the model was never the bottleneck. The platform was.

And that is why "hybrid AI" is suddenly back on the roadmap, minus the 2018 marketing gloss.

95% delayed is not a model problem

Here is what I think actually happened in most of those 95%.

A team built a pilot. The pilot worked. It demoed beautifully. Then someone asked where the data lives, who can access it, which region the inference runs in, what happens when the model provider changes terms, how the spend is metered per team, and what evidence exists that any of it is compliant.

That is when the project quietly slipped a quarter.

None of those questions are exotic. They are the same questions we have asked about databases, message queues, and payment APIs for fifteen years. The difference is that AI workloads arrived fast, with a new runtime, a new vendor surface, a new identity model, and a new cost curve, and they landed on teams that had not finished answering the old questions.

So yes, 95% is a scary number. But it is not evidence that AI does not work. It is evidence that governance and platform work is boring, slow, and load-bearing, and that skipping it does not make it go away.

real hybrid ai is a placement story, not a logo story

The old hybrid cloud pitch was about where your logo runs. You kept a VMware cluster, you bought a management plane, you drew a diagram with a cloud on the left and a rack on the right, and everyone nodded.

The new version is much less photogenic, because it is about placement rules.

Where is the data allowed to exist? Where is the inference allowed to run? Which workloads can touch customer records, and which ones can only touch the derived, scrubbed copy? Which GPUs are on a lease and which ones are capex? Which model is served from a vendor endpoint and which one is served from your own cluster? Where do prompts, completions, and traces get stored, and for how long, and under whose jurisdiction?

That is not an architecture slide. That is a contract.

And like any contract, it only matters if someone can be held to it. "We're hybrid" is a slogan. "This workload class runs in region X, on substrate Y, with data that never leaves the boundary, and here is the log line that proves it" is a platform.

the boring signal is bare metal

The tell that this is real platform work, and not a vendor rebrand, is that cloud providers are moving down the stack, not up it.

Azure has been pushing AKS across substrates that used to be somebody else's problem. AKS on bare metal went into public preview in June 2026, framed around sovereign and private clouds, OEM hardware, and cloud regions as one operational model. Arc-enabled clusters do the same thing from the management side: the control plane does not care whether the node sits in a hyperscaler region or a rack in your own data center.

I find this genuinely interesting, and not because bare metal is fashionable again.

It is interesting because the pitch is no longer "move everything to us." The pitch is "run the same Kubernetes operating model wherever the workload has to be." That is an admission, from a company with every incentive to say otherwise, that AI workloads are not all going to be co-located in one region under one set of terms. Some of them cannot leave a jurisdiction. Some of them cannot leave a building. Some of them need hardware you own because the economics only work at capex scale.

The CNCF is telling the same story from the community side. KubeCon North America 2026 added an AI Inference and Agentic track, and the topic list reads like a platform engineering backlog: GPU scheduling, model serving, observability, production AI. Not prompt engineering. Not "what is an agent." Scheduling, serving, observing, operating.

That is what infrastructure looks like when a wave stops being a demo and becomes a workload class.

what actually changed is blast radius

Here is the part that keeps me up at night more than placement ever will.

In the same window, the LiteLLM supply chain exposure reconnected the dots for everyone paying attention. Reporting from CloudSEK and SecurityWeek described an attack chain that reached well over 2,500 organizations, touching cloud credentials, Kubernetes environments, CI/CD pipelines, repositories, and the AI infrastructure layer itself.

Sit with the shape of that for a second.

Your AI gateway is not a convenience wrapper around model APIs. It is a credential broker. It holds provider keys. It talks to your cloud. It runs in your cluster with a service account. It is wired into CI/CD so the evaluation harness can run. It logs prompts and completions that may contain the same sensitive data your governance story swears never leaves the boundary.

Which means the AI gateway, your CI/CD secrets, your Kubernetes tokens, and your cloud credentials now share a blast radius.

That is a new sentence in the threat model, and a lot of teams have not written it down yet. Traditional APM will not save you here either. If an agent takes a wrong turn through three tool calls and a compromised dependency, "p99 latency looks fine" tells you nothing. The CNCF piece on agent observability makes the point bluntly: you cannot debug what you cannot see, and the thing you now need to see is a decision path, not just a request trace.

So the security question and the architecture question turn out to be the same question. Where did this workload run, what did it touch, with whose credentials, and what did it do with them?

the contract i would actually write

If I were the platform lead staring at this, I would not start with a migration. I would start with a document. The unglamorous, reviewable, versioned kind.

Here is roughly what would be in it.

Workload placement rules. Which classes of AI workload are allowed on which substrates. Training versus fine-tuning versus batch inference versus interactive inference are different animals. Some belong on rented GPUs, some on owned hardware, some behind a vendor API. Write the mapping down so the next urgent request does not relitigate it from scratch.
Data residency rules. What data may leave the boundary, in what form, under what contract, for how long it may be retained, and who signs off when an exception is asked for. "We anonymize it" is not a rule until someone can point at the code, the test, and the audit trail.
A model and runtime inventory. Every model, every serving runtime, every gateway, every evaluation harness. Versioned, owned, with a lifecycle. If you cannot list your models the way you list your services, you do not have an AI platform, you have an archaeology site.
Credential rotation paths. Which key lives where, what it can reach, how it is rotated, and how you would prove a rotation actually happened. Every AI gateway key is a production credential and should be treated with the same paranoia as a database root password.
Observability requirements. Spans for tool calls, structured logs for prompts and completions with a retention policy, cost attribution per team and per workload, and enough context to answer "what ran, where, and on whose behalf" during an incident. If the answer requires a person who has not left the company yet, you have a single point of failure.
Cost boundaries. Budgets, quotas, and a kill switch. Idle agents and runaway evaluation jobs are not a billing surprise, they are an architecture defect.
Rollback and rehearsal evidence. A tested path back for models, gateways, and placement changes. Rehearsed, not assumed. "We would just revert it" is a wish, and wishes do not survive contact with a Friday night.

Notice what is missing from that list. There is no section called "which cloud we picked." That decision falls out of the rules. Start by deciding what you are allowed to do, and the substrate question mostly answers itself.

the skeptical ending

Hybrid AI will be sold to you as freedom. Freedom from lock-in, freedom to choose, freedom to run anywhere.

For engineers, it mostly means accountability.

Because the moment your AI workload spans a vendor endpoint, your own cluster, and an owned rack, every incident becomes a question of evidence. Where did the inference actually run? What data did the prompt contain? Which credential touched it? Which logs exist, in which jurisdiction, for how long? Who approved the exception, and when does it expire?

If your platform cannot answer those questions on a bad day, at 2 AM, without a hero, then "hybrid AI" is not a strategy. It is a diagram of places you cannot see.

The teams that come out of this well will not be the ones with the most clever routing layer. They will be the ones who treated placement, identity, observability, and rollback as a contract, wrote it down, and made the platform prove it every single release.

references

To test my projects, I use Railway. If you want $20 USD to get started, use this link.

DEV Community