DEV Community: mithilesh gaurihar

Hugging Face Was Breached by an AI Agent. Then Its Own Tools Refused to Help.

mithilesh gaurihar — Mon, 20 Jul 2026 17:48:46 +0000

Hugging Face, the largest repository of open AI models on the internet, disclosed that its production infrastructure was compromised by an autonomous AI agent system.

The company said it detected and contained the incident earlier last week. Unauthorized access reached a limited set of internal datasets along with several credentials tied to its services.

The investigation is ongoing. So far Hugging Face has found no sign that the agent touched public-facing models, datasets, or Spaces, and no sign that its software supply chain was altered.

How the attack worked

The entry point was the data processing pipeline. A malicious dataset abused two separate code execution paths: the remote code dataset loader, and a template injection flaw in a dataset configuration file. That was enough to run code on a processing worker.

From there the agent escalated to node-level access, collected cloud and cluster credentials, and moved laterally into several internal clusters over the course of a weekend.

Which model powered the attack is still unknown. What Hugging Face can describe is the shape of it: an autonomous agent framework running many thousands of individual actions through a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.

What Hugging Face did about it

The company closed the code execution paths used for initial access, then worked through remediation:

Removed the attacker's foothold and rebuilt compromised nodes across affected clusters
Revoked and rotated affected credentials and tokens, with a broader precautionary secret rotation
Added guardrails and tightened admission controls on its clusters
Improved detection and alerting so responders get paged within minutes, around the clock

Hugging Face is also asking users to rotate their own access tokens and review recent account activity.

The part that should worry builders

Reconstructing the timeline meant analyzing a very large volume of attacker activity, so the team pointed LLM agents at the logs.

The frontier models behind commercial APIs would not do it.

Forensic work requires submitting genuine attack commands, exploit payloads, and command-and-control artifacts. Safety guardrails treated that material as hostile, because a filter has no way to tell an incident responder from an intruder. Both are sending the same bytes.

Hugging Face ran the analysis instead on GLM 5.2, an open-weight model from Z.ai, hosted on its own infrastructure.

The company was direct about the lesson. It does not know whether the attacker's agents ran on a jailbroken hosted model or an unrestricted open-weight one. Either way, the attacker operated under no usage policy at all, while the defenders were blocked by the guardrails of the tools they reached for first.

That asymmetry is not a mistake anyone made. It is structural. Offense uses ungoverned tooling by definition. Defense opts into governance and gets the invoice at the worst possible moment.

What to do with this

Most teams pick models on two axes: cost and capability. This incident adds a third, which is access.

There is a real category of legitimate work where the hosted tier will decline. Incident response. Malware analysis. Abuse investigation. Red-teaming your own product. You do not want to discover the refusal during an emergency.

Hugging Face's own recommendation is the practical one. Vet a capable model you can run on infrastructure you control, and have it ready before you need it. It avoids guardrail lockout, and it keeps attacker data and credentials inside your environment.

Most of us are not running the world's largest model repository. But plenty of us are shipping systems where an agent has more reach than we would say out loud, and where the entire incident response plan is "ask the good model."

That plan just failed a live test.

More like this daily. Discord's here if you want to follow along.

A Pipeline Is a File. Running It Is the Work.

mithilesh gaurihar — Mon, 13 Jul 2026 16:00:00 +0000

Why we built RocketRide Cloud around the same portable .pipe files developers use locally.

By Krish Garg and Mithilesh Gaurihar

The artifact we want a team to own is small.

It is a .pipe file: a JSON description of the nodes in an AI workflow, the data lanes between them, and the configuration each node needs. You can open it in an editor, review it in a pull request, and keep it with the application it belongs to.

The job we do not think every team should have to own is everything required to keep that file working for real users.

That is the distinction behind RocketRide Cloud.

We did not set out to make another place to draw agent graphs. Plenty of products can help you make a demo. We wanted the thing you build locally to stay the thing you run in production, without asking you to become the team that operates GPU inference, identity, secrets, routing, deploys, and the failures in between.

The pipeline is yours. Running it is the work.

Start with the file

Here is a real RocketRide pipeline. A dropper takes in a media file, a parser prepares it, an audio-transcribe node turns its audio into text, and an output node returns that text.

{
  "components": [
    { "id": "dropper_1", "provider": "dropper", "name": "Dropper",
      "config": { "mode": "Source", "type": "dropper", "hideForm": true } },
    { "id": "parse_1", "provider": "parse", "name": "Parser",
      "input": [{ "lane": "tags", "from": "dropper_1" }] },
    { "id": "audio_transcribe_1", "provider": "audio_transcribe", "name": "Transcribe",
      "config": { "profile": "default",
        "default": { "model": "base", "min_seconds": 240, "max_seconds": 300, "silence_threshold": 0.25, "vad_level": 1 } },
      "input": [{ "lane": "video", "from": "parse_1" }] },
    { "id": "response_text_1", "provider": "response_text", "name": "Return Text",
      "config": { "laneName": "text" },
      "input": [{ "lane": "text", "from": "audio_transcribe_1" }] }
  ],
  "project_id": "00000000-0000-4000-8000-0000000000d0",
  "version": 1
}

That is the application logic. Not the whole production system, obviously, but the part that should stay legible to the people who own it.

The value of a declarative pipeline is not that JSON is fashionable. It is that the runtime can understand what it is being asked to run, and so can you. The source is explicit: the dropper. The steps are explicit. The lanes between the nodes are explicit too, named right there in the file. The dropper hands off to the parser on a tags lane, the parser feeds the transcribe node on a video lane, and the transcribe node returns its output on a text lane. A model provider, a vector store, or a tool is a component with a contract, not a hidden dependency buried in application glue.

That makes a pipeline easier to inspect and easier to change. Swap the transcription model and the surrounding workflow does not need to be rewritten. Add a step and the review is a small diff, not a new service with another deploy path. The visual builder is just another view of the same file.

The file is portable on purpose. It can run locally in the VS Code extension, on infrastructure you operate yourself, or on RocketRide Cloud.

On your machine: the file above, edited in the RocketRide extension. The connection at the bottom points at RocketRide Cloud, and one click deploys your local changes to production.

Local is dev. Cloud is production. Deploy is one click.

This is the part we care about most, so we will say it plainly. In the VS Code extension the same pipeline points at two runtimes: one on your own machine, and RocketRide Cloud. Local is where the work happens. You add a node, connect it, run it, inspect the trace, and try again, against the very same runtime that will run in production. Local development should be boring, and here it is.

Then moving what you built from local to production is not a packaging step. You make the changes locally, and one click deploys them to Cloud. No container to build, no config to fill in, no key to paste, no separate CI to wire up. The pipeline you were editing a second ago is the one now serving your users. Develop, edit, iterate, and when it is ready, one click ships it.

In the cloud: the very same file, now running in production. Nothing was packaged or re-uploaded; the local edits were promoted with a click.

Production is a different job, and that is what the click quietly hands off. A production pipeline needs an authenticated entry point. It needs a place for project state, secrets, and billing. It needs to survive a rolling deploy, a failed process, a slow model load, or a reconnecting client. It needs a route to a model server when the work requires OCR, transcription, embeddings, NER, or vision. It needs someone to watch the runtime itself, not just the prompt.

Those concerns are why Cloud exists.

When you deploy, you are not handing your workflow to a new format or a black box. You are running the same .pipe file on a managed production path. The Cloud runtime supplies the service layer around it: authenticated access, managed endpoints, secrets and project controls, run-level observability, and the infrastructure that keeps the runtime available.

The important part is what does not happen. You do not rebuild the workflow as a separate cloud-only artifact. You do not fork the logic just because it moved from a laptop to a service your users call.

What you are actually handing over

Managed is an overused word, so it is worth being specific about the work that moves to us. Four things, plainly.

A fraction of the cost, on a patent-pending model server. Open-weight models are cheap now, but only to whoever can serve them well. Our model server does that serving: continuous batching to keep the GPU busy, warm caches so shared context is not recomputed, quantized serving to fit more model into less memory, and step-level routing that sends each step to the cheapest model that clears its quality bar. A step that only needs a small model gets one. To be honest about the claim: the saving is in model selection and efficient serving, not in the runtime being cheaper compute than a script you wrote yourself.

Shared pipelines your team actually shares. A pipeline is a first-class object, not a script living on one laptop. One person defines it, and the rest of the team runs it, forks it, and reads the same traces and the same cost data. When a model changes or a step needs work, someone edits one node and everyone is on the same version, instead of six people maintaining six slightly different copies.

The operational layer, handled. Running AI in production means owning authentication and secrets, provider failover, tool isolation, monitoring, and the security posture around all of it. RocketRide Cloud runs that layer so the churn lands on our on-call rotation, not your product. A monthly invoice cannot tell you why one customer request failed. A trace of the run can, and that is the level we operate at.

Performance and scale that survive real traffic. Once a pipeline is resident, repeat requests are served warm instead of paying a cold start every time. I/O-bound work like model calls, retrieval, and database hops overlaps cleanly rather than running one call after another. Shared state lets the service route work across healthy instances rather than pinning a pipeline to one process, and health checks and controlled shutdowns mean a deploy does not become a dropped request by default.

That is not the glamorous part of AI. It is the part that becomes your problem the first time a demo turns into a dependency.

For an engineering leader, the question is not whether the team can stand up those pieces. Most good teams can. The question is whether that is the best use of their time while they are still learning whether the product itself is useful.

Why the runtime is written this way

Under the file is a C++ runtime that orchestrates Python nodes. That design is deliberate, but it is not a magic performance claim.

The runtime is there to supervise work, keep the pipeline contract stable, and isolate failure boundaries. It does not make every piece of Python suddenly parallel, and it does not make a model reason better. In production the useful question is often simpler: when a node fails badly, what else does it take down with it?

RocketRide runs pipeline work under supervision so one broken run does not become an outage for every other request. That choice has a cost. Process isolation adds lifecycle work that a throwaway in-process script does not have. We made the trade because production systems are judged by how they fail, not only by how they look on the happy path.

Cloud is where that trade becomes practical. It is one thing to have a runtime that can supervise a run. It is another to operate the surrounding services, update the runtime, manage GPU-backed model serving, and keep the path healthy over time. That is not a burden a product team should inherit by accident just because it wanted to ship an AI feature.

Why not self-host?

This is the right question, and sometimes the answer is that you should.

Self-host RocketRide Server when your environment requires it. If your data has to stay inside a particular VPC, if you run an air-gapped deployment, or if you already have a platform team that wants to own the runtime, the MIT-licensed server is there for exactly that. The .pipe file you author is still yours, and it still runs.

Cloud is for the other case: a team that wants to ship a production AI capability without first becoming responsible for the operations around it. Same portable workflow, but with RocketRide operating the service path behind it.

That is a different thing from lock-in. We do not think a managed service earns its place by making departure hard. It earns its place when it is genuinely easier to run well than to run yourself. The open-source runtime makes that claim more credible, not less. You can inspect the runtime, use it locally, self-host it later, or keep using Cloud because it saves your team from running another piece of production infrastructure.

The decision is smaller than it sounds

You do not need to make a permanent infrastructure bet before you write the first pipeline.

Author the workflow as a file. Run it locally while the work is changing quickly. Use Cloud when you need an operated production path. Self-host when your environment requires you to own that path.

The important thing is that the application logic does not get trapped in the decision. That is why we built RocketRide this way. The pipeline should belong to the team building the product. The operational burden should be a choice.

Build and run a pipeline on RocketRide Cloud
Inspect or self-host RocketRide Server
Build locally with the RocketRide VS Code extension

What the AI Stack Looks Like Below the Model

mithilesh gaurihar — Thu, 09 Jul 2026 16:25:16 +0000

Picking a model is a config decision now. Serving it, securing it, scaling it, and sharing the work across a team is where the difficulty went.

By Krish Garg and Mithilesh Gaurihar

This week in AI

AWS raised EC2 Capacity Block GPU prices ~20% effective July 1, the second hike this year. (Yahoo Finance)
A design-level command-execution flaw in MCP's STDIO transport affects an estimated 200K instances. Anthropic declined to change the protocol, calling the behavior expected. (OX Security, The Register)
Stripe's autonomous coding agents now merge 1,300+ PRs per week with zero human-written code. The enabling layer is not the model, it is the harness around it. (ByteByteGo breakdown)

Frontier models keep shipping and open-weight alternatives keep closing the capability gap at a fraction of the price. Which model to run is a decision you revisit, not an architecture you commit to.

The engineering problems below the model did not get easier. Cloud GPU pricing is climbing quarter over quarter. A design-level security flaw sits in the most widely adopted agent protocol. Teams running agents at real volume are discovering the ceiling is the orchestration layer, not just the model on top of it. And pipeline work keeps getting rebuilt per engineer because there is no shared artifact to build against.

RocketRide Cloud is a managed AI pipeline runtime built around these four problems. You define your pipeline as a portable .pipe JSON file, connect it to a growing node library covering LLM calls, OCR, transcription, embeddings, vector retrieval, agents, and more across all major LLM providers, and run it on managed infrastructure. The open-source server is MIT-licensed. Cloud is the managed layer on top.

The AI subsidy era is ending

For the past two years, compute has been quietly underpriced. Cloud credits covered startup bills. Flat-rate tiers let power users consume more than they paid for. That era is closing: the credits are expiring and the clouds are repricing the metal underneath, and pricing across the stack is moving toward metering that reflects actual compute cost.

AWS raised EC2 Capacity Block GPU prices roughly 20% effective July 1, the second hike this year after a 15% increase in January. Cumulative increases have pushed reservation rates up 20 to 50 percent since January. When pricing was subsidized, inefficiency was invisible. Under metered pricing, every wasted token and every idle GPU-second lands on your bill directly.

Efficiency in the serving layer is the lever. For open-weight models running on Cloud's own infrastructure, RocketRide's model server applies continuous batching, KV and prefix caching, and quantized serving. For steps that call provider APIs, step-level routing sends each one to the cheapest model clearing its quality threshold. (The routing approach is patent-pending.) Either way, token spend, CPU, GPU, and memory are tracked live as the run executes, so you see where the resources are going while you can still change the routing, not on a monthly bill.

Live CPU, memory, GPU, and throughput per node while the pipeline runs. Resource usage is visible during execution, not on a monthly bill.

MCP has a design-level security problem

In April, OX Security disclosed a command-execution vulnerability in MCP's STDIO transport. Here is the concrete failure mode: STDIO executes whatever command string it is given to spawn a server. If the command starts a valid MCP server, you get a handle. If it is anything else, you get an error, but the command has already run. Execute first, validate second. An engineer installs an MCP server from a marketplace, the config carries a malicious command, and it executes on the host with that engineer's credentials. OX poisoned 9 of 11 MCP marketplaces with a proof-of-concept payload to demonstrate exactly this path.

This is not an implementation bug in one product. It is a design default in the official MCP SDKs across Python, TypeScript, Java, and Rust, inherited by every downstream project. OX confirmed 7,000+ publicly accessible servers and estimates up to 200K vulnerable instances across a supply chain of 150M+ downloads. The Cloud Security Alliance independently confirmed the findings. And the part that makes this an ongoing problem rather than a patch cycle: Anthropic declined to modify the protocol, stating the behavior is by design and sanitization is the developer's responsibility. The only upstream change was a documentation update. Remediation sits with whoever runs the server. That means you.

If you run MCP in-house, you own that surface continuously: allowlisting spawn commands, treating every config source as untrusted, auditing registries, and tracking the next CVE in the family (ten and counting from this one root cause). And that is one surface among many. Self-hosting AI infrastructure means also owning auth, secrets, warm model servers, queue pressure, deployment, and rollbacks. Every layer you run yourself is a layer you patch, monitor, and answer for. RocketRide Cloud takes the infrastructure ops layer off your plate, which shrinks the surface your team has to defend to the parts only your team can own. Build the pipeline, call it from your app, one click to production.

The RocketRide Cloud server monitor. Uptime, connections, and resource usage tracked continuously.

At Stripe scale, the harness is the product

Stripe's autonomous coding agents merge 1,300+ PRs per week with zero human-written code. The revealing part of their engineering writeup is that the model is almost a commodity. What makes the system work is the infrastructure around it: isolated sandboxes that spin up in seconds, deterministic gates interleaved with agent steps, selective CI, and scoped context management. The harness is the product.

Most teams do not have Stripe's decade of platform investment to build that harness on. And at high concurrency, the orchestration layer itself becomes the cost: per-pipeline memory footprint, serialization overhead between steps, and startup latency all compound as run volume grows. RocketRide's execution engine is native C++ with multithreading, built for exactly this throughput profile rather than retrofitted from a prototype.

There is also a failure mode that only appears at run length. An agent passes every eval, then degrades deep into the run, dozens of tool calls in, and nobody knows why until the run is over. Aggregate metrics never catch it, because the failure is specific to one run, not a statistical property of all runs. Catching it requires the full execution trace of the run that broke: every node, every handoff, where the drift started. RocketRide traces every LLM call, token, and millisecond scoped to the individual run.

Run-scoped trace. Every node and handoff, scoped to the run that failed.

The same pipeline keeps getting rebuilt

Most AI pipeline work lives in individual environments. One engineer builds the prompt scaffolding. Another wires the tool calls. A third adds eval steps. None of it is in the same place, and when someone leaves, the context leaves with them. The next team that needs the same capability starts from scratch.

Here is what a shared pipeline looks like in practice. This RAG pipeline is a standard flow: ingest through a webhook, parse, embed, store in Qdrant, and a chat path that retrieves and answers grounded in that store. The screenshot below shows the topology. What matters is not the shape, it is that the whole thing is one portable .pipe JSON file, and the full example is in the open-source repo. Point the LLM node at a different provider, run it on your laptop or on Cloud. Same file, same behavior. One team defines it, others run it and build on top. The context lives in the file, not in whoever built it first.

The RAG pipeline: Webhook, Parser, Generate Text, Transformer, Qdrant on ingestion, and a chat path through Transformer and Qdrant to the LLM.

Per-token prices keep falling, but the compute underneath is repricing upward and the subsidies that sat between those two numbers are ending. The dominant agent protocol has a by-design flaw whose remediation sits with you. The teams shipping agents at real scale are winning on infrastructure, not model choice. Each of these compounds while the model layer keeps commoditizing.

Try the claim yourself. Grab the RAG pipeline example from the repo, run it on RocketRide Cloud, then run the same .pipe file on the MIT-licensed open-source server on your own machine. Same file, same execution. If it does not work as described, that is worth knowing. If it does, you have a production pipeline you own either way.

Resources

Build the Runtime, Not the Stack

mithilesh gaurihar — Mon, 29 Jun 2026 15:08:40 +0000

By Krish Garg, Mithilesh Gaurihar

What this post covers

AI infrastructure risk is no longer theoretical. Here is what we are going to walk through:

A US government directive took Anthropic's Fable 5 offline for every developer on the planet, with zero notice.
GitHub Copilot ended flat-rate pricing and moved to usage-based billing, quietly ending the AI coding subsidy era.
Every team hard-wired to a single model, tool, or pricing plan had no fallback. The ones that did had a runtime layer sitting above all of it.
Model-agnostic doesn't mean migration is free. You will retune. But the runtime scopes the damage to one layer instead of everything.
The platforms you're routing across, OpenAI, Anthropic, Microsoft, are all actively trying to absorb the orchestration layer themselves. That's the real lock-in risk, not the model.
The answer is building the runtime on open infrastructure before you need it, not after.

Bottom line: the model, the tool, and the pricing plan can all move overnight. The only thing that doesn't have to is the runtime layer above them.

Two Headlines. One Lesson.

The US Commerce Department sent Anthropic a letter on a Friday evening. By midnight, Fable 5 and Mythos 5 were offline for every user on the planet. Not just foreign nationals, but everyone, because Anthropic had no way to verify citizenship at scale in real time. AWS Bedrock, Google Cloud, Microsoft Foundry, and the direct Claude API all went dark simultaneously. Teams woke up the next morning to broken pipelines and no warning.

Read Anthropic's official statement →

Around the same time, GitHub Copilot ended flat-rate pricing and moved every plan to usage-based billing. The AI coding subsidy era ended with an email and a new line item on the bill.

Read the GitHub announcement →

And if you think that's isolated to models and billing, SpaceX has agreed to acquire Cursor-maker Anysphere for $60 billion pending regulatory approval. Same pattern, different layer.

Read the CBS News report →

Different companies. Different layers. Same pattern: rented ground.

Why this keeps happening

Most AI system engineering goes into the model layer. Prompt design, tool calling, eval pipelines, output parsing. All of it tuned against a specific model's behavior. When that model changes, disappears, or gets repriced, the tuning work is either portable or it isn't. In most production systems, it isn't.

The layer that makes it portable is the runtime: model routing, observability, and cost tracking sitting above the model itself. Teams that built that layer had options on Friday night. Teams that didn't were waiting for support tickets.

This isn't a new idea. Perplexity's Aravind Srinivas has argued that the model is not the product. Anthropic's Boris Cherny has written about loop engineering: stop prompting agents directly and design the loops that prompt them. The durable artifact is the loop, not the model.

This news just made that expensive to ignore.

What model-agnostic actually means in practice

A model-agnostic runtime needs to do three things. Most teams skip all three until they are forced to deal with the consequences.

1. Treat the model as a configuration, not a dependency.
Prompts, tool definitions, and output schemas should not assume a specific provider's API shape. The moment your prompt is written for GPT-4's output format, or your tool calling assumes Anthropic's schema, you have a hard dependency masquerading as flexibility.

2. Span the entire run.
The runtime needs to see every component, every model call, every handoff in a single trace. When you route across multiple models, the failure is almost never inside one model. It is in the handoff between them. A trace that only covers one provider tells you nothing useful about that failure.

3. Surface cost in real time.
Not in a billing dashboard at the end of the month. As the run executes. When you pay per token, the decision of whether to commit a workload to an expensive model needs to happen before you commit it, not after.

Most teams get none of these right because they build the model integration first and the runtime later.

How RocketRide implements this

The model is a node, not a dependency

In RocketRide's pipeline builder, the model is a component node sitting between your prompt and your output. A Chat source feeds into a Prompt component, passes questions to an LLM node, and returns answers downstream. That LLM node is a configuration entry. Point it at OpenAI, Anthropic, Gemini, Mistral, Qwen, or a local Ollama instance and the rest of the pipeline stays exactly where it is. The connections, the prompt component, the output handler: none of it changes. You are swapping one node, not rewriting the system.

Here is what a pipeline looks like running end to end:

Swap GPT for Claude, Gemini, Qwen, Mistral, or a local Ollama instance without touching the pipeline.

Every run is fully observable

When a pipeline runs, RocketRide streams six views simultaneously: Design, Status, Tokens, Flow, Trace, and Errors. Each scoped to that specific run, not aggregated across all runs. The Status tab streams CPU%, CPU Memory, GPU Memory, and Total Completions per second in real time.

CPU, memory, and completion throughput stream live as the pipeline executes. Scoped to the run, not the account.

This matters for a specific reason. When you route across multiple models, the runtime is the only place that sees all of them. Without that span, you are flying blind across every provider boundary. A trace that only covers one model tells you nothing useful when the failure is in the handoff between them.

Cost surfaces before you commit

The Tokens tab breaks down what each component consumed: Total Tokens, CPU Usage, CPU Memory, and GPU Memory, per component, not just as a pipeline total.

The Tokens tab breaks down CPU usage, memory, and total tokens per component in real time. Cost visibility at the right layer.

That per-component breakdown is what makes cost decisions actionable. You can see whether the Chat component is consuming disproportionate tokens before the expensive LLM node even runs. When you are evaluating whether a cheaper model held up on a complex task, you are looking at this view, not waiting for an invoice at the end of the month.

The honest part: swapping is not free

Repointing a config key is the start of the work, not the end.

If the banned model was the one your prompts, tool-calling schemas, and eval thresholds were tuned against, you will retune. Prompt behavior differs across providers. Tool calling schemas are not universal. Evals that passed on one model will fail on another. That is real effort and we won't pretend otherwise.

What the runtime gives you is a scoped problem instead of an architectural crisis. The pipeline structure, the tracing across the Flow and Trace tabs, the cost visibility in the Tokens tab: none of that moves when you swap the model node. You are fixing one layer, not rebuilding everything. That distinction matters when you are scrambling at midnight because a government directive just took your model offline.

The economics make the effort worth it. Harvey's multi-model approach points to a consistent pattern: routing cheaper models for lower-complexity steps and premium models where it counts can outperform a single expensive model while costing less. When you pay per token, that gap compounds across every agent run. But those gains only materialize if the runtime layer exists to do the routing and the measurement. Without it, you cannot even see where the cost is going, let alone optimize it.

The counterargument worth taking seriously

The obvious pushback: capability beats resilience. Bet on the best model and architectural flexibility is a distraction.

The stronger version is structural. OpenAI's AgentKit and Anthropic's Managed Agents are both pulling orchestration, memory, evals, and observability into managed surfaces. VentureBeat reported that Anthropic has explicitly signaled the orchestration layer is where lock-in will live, because tooling infrastructure will outlast interchangeable models.

The platforms you are routing across are actively trying to absorb the runtime themselves.

That is a real risk. It is also the clearest argument for building on open infrastructure. A vendor-owned runtime is the same rented ground problem one layer up. The swap cost has to stay low, which means the runtime cannot be the thing you are locked into. That is why RocketRide Server is open source. Not as a positioning decision. As an architectural one.

What to do

The routing logic, tracing, and cost attribution are not novel engineering. What is hard is building the abstraction before you need it. Most teams build the model integration first and the runtime later, if at all. When the model gets banned or repriced, there is no later.

If your loop is model-agnostic, observable, and on open infrastructure, the next ban, acquisition, or repricing is an operational inconvenience rather than an architectural crisis.

Start here

Try RocketRide Cloud — spin up a model-agnostic runtime and watch every component trace and token stream live as it runs.
RocketRide Server on GitHub — open source and open to contributions. If you're building in this space, we'd love to build alongside you.
Download the free VS Code extension — the pipeline builder and all six run views live directly inside VS Code, free and open source, no account needed to get started.

If you've run a pipeline on RocketRide Cloud, let us know your feedback.