A framework is not one decision. It is four, bundled: provider abstraction, control flow, integrations and operations. Most teams need one or two of those and adopt all four, which is why the regret arrives around month six rather than week one. Name which of the four you need and the question stops being religious.
The four jobs a framework does
Every AI framework — LangChain, LlamaIndex, Haystack, Semantic Kernel, the Vercel AI SDK, CrewAI — is some mixture of the same four jobs. The mixture differs, the marketing differs, the jobs do not.
- Provider abstraction. One call shape across OpenAI, Anthropic, Google, Mistral and whatever you run locally. The pitch is that changing model is a configuration change. It largely is, for the fields every provider has.
- Control flow. Chains, graphs, agent loops, branching, parallel fan-out, retries and — the genuinely hard one — state that survives a process restart. This is the job with the most real engineering in it.
- Integrations. Document loaders, text splitters, vector store adapters, tool wrappers, memory backends. The value is breadth on day one: a hundred connectors you did not write.
- Operations. Tracing, token accounting, callbacks, evaluation hooks, run replay. Frequently the reason a team that wanted to leave stays.
The important property is that these four are separable. Almost nothing forces you to buy them together, and the most common expensive mistake is adopting a whole framework for job three — an integration you would have written in forty lines — and paying for jobs one, two and four in ways that only become visible once the application is in production.
What the loop looks like without one
Before pricing the alternative it is worth being precise about what you are avoiding writing. The core of an agent — the thing frameworks are usually adopted for — is a loop that calls a model, notices it asked for a tool, runs the tool, appends the result and calls again. It is about forty lines.
# The whole of it. Fields shown are the OpenAI-compatible chat shape,
# which every major provider and every local server now speaks.
def run(messages, tools, tool_impls, max_turns=8):
for turn in range(max_turns):
reply = client.chat.completions.create(
model=MODEL,
messages=messages,
tools=tools,
).choices[0].message
messages.append(reply)
calls = reply.tool_calls or []
if not calls:
return reply.content # the model is done
for call in calls:
name = call.function.name
args = json.loads(call.function.arguments)
try:
result = tool_impls[name](**args)
except Exception as exc: # a failed tool is a message,
result = {"error": str(exc)} # not an exception in your loop
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": json.dumps(result),
})
raise RuntimeError("agent did not finish in %d turns" % max_turns)
That is not a strawman version. It is the shape of the agent loop as every framework implements it, minus the abstractions. If your application is one of these, running inside a request handler, finishing in seconds, with four tools you wrote yourself, then the forty lines are the correct amount of code and a framework is a net loss.
What a framework adds beyond it is real, though, and worth naming precisely: durable state so a run can resume mid-way after a crash or a human approval; structured concurrency so two branches run at once without you writing the join; streaming that survives being nested three calls deep; and a trace that shows which of the eleven model calls in a run produced the wrong answer. Every one of those is genuinely annoying to build and genuinely annoying to get right.
What each job costs you
Lock-in is not a slogan. It is a specific set of things you will have to rewrite if you leave, and it differs sharply by job.
| Job | Description |
|---|---|
| Provider abstraction | Cheapest to leave and cheapest to replace. You are locked into the framework's dialect for provider-specific fields — reasoning effort, cache control, safety settings, logprobs — which is exactly where the abstraction leaks. A thin adapter of your own, or a dedicated proxy, does this job with a fraction of the surface. |
| Control flow | The expensive one, and the one worth paying for. Your application's logic ends up expressed in the framework's vocabulary — nodes, edges, chains, crews — and that vocabulary does not translate. If the framework also owns your persisted run state, leaving means a data migration as well as a code change. |
| Integrations | Cheap to leave, expensive to trust. You inherit each adapter's opinions and its bugs, one maintainer deep, and the failure mode is a silent behaviour change in a loader you did not know you depended on. Adapters are also the code you are most capable of writing yourself. |
| Operations | Sticky out of proportion to its size. Once a team's dashboards, alerts and evaluation runs read the framework's trace format, nobody wants to be the person who breaks them. Mitigated almost entirely by emitting OpenTelemetry spans instead of a proprietary shape. |
The pattern in that table is the whole recommendation: pay for control flow if your control flow is hard, be sceptical about the other three, and keep operations on an open format so it never becomes the reason you stay.
A decision procedure that takes an afternoon
- Draw your control flow. Not the happy path — the real one. Does it contain a cycle? A branch chosen by the model? A step where a human approves something hours later? A fan-out that joins? If the answer to all four is no, you have a pipeline, not an agent, and you do not need an orchestration framework.
- Ask whether a run must survive a restart. This is the single highest-value question on the list. If a deploy in the middle of a twenty-minute run must not lose it, you need durable checkpointed state, and that is worth adopting a framework for. If every run finishes inside one HTTP request, it is not.
- Count the integrations you need on day one. Not the number in the README. The actual list. Teams routinely adopt a framework for its hundred loaders and use two, both of which are a library call and a for-loop.
- Name who reads the traces. If the answer is “nobody yet”, operations is not a reason. If it is a support engineer at 2am, it is the strongest reason on this list — though see the last row of the table above.
- Build the hardest step twice, in one day. Not the demo. The step you are least sure about. A day of this tells you more than a week of comparison reading, because the thing you are testing is not whether the framework can do it but whether you can work out why it did what it did.
- Decide per job, not per framework. “We use LangGraph for orchestration and call provider SDKs directly inside the nodes” is a coherent, common and cheap position. “We use LangChain” is not a decision, it is a default.
Adopt at the seam, not at the centre
If you do adopt one, the difference between a framework you can leave and a framework you cannot is entirely about where you let it touch. Four rules cover most of it.
- Your prompts live in your files. Plain text or templates you own, versioned in your repository, not embedded in framework classes. Prompts are the highest-churn, highest-value asset in the application and they should not need an import to read.
- Your domain types are yours. Pydantic or Zod models defined in your own module, passed into the framework, not framework document objects leaking into your business logic. This one rule does more to bound a migration than everything else combined.
- Framework code stays in one directory. If
import langchainappears in forty files, you have adopted it at the centre. If it appears in three, you have adopted it at a seam. - Provider calls go through one function you wrote. Even if that function currently delegates straight to the framework. It is the seam that makes fallback chains and per-request cost accounting possible later without touching the application.
Provider abstraction is the job with the weakest case for a framework, because it can be done outside your process entirely: one OpenAI-compatible endpoint in front of many providers means model choice is a string in configuration rather than an import. Multigrid does that job at the network boundary, with routing, failover and per-request cost telemetry, which leaves the framework question narrowed to the three jobs it is actually good at.
When the answer changes
The decision is not permanent in either direction, and there are four fairly reliable triggers for revisiting it. Durability becoming a requirement — the first time a deploy kills a long run in production — is the usual reason to adopt one. Team size is the second: a loop one person understands is fine, a loop eight people extend needs a vocabulary, and a framework supplies one.
In the other direction: when you find yourself reading the framework’s source to work out what prompt it sent, the abstraction has stopped paying. And when the release cadence outruns your ability to test upgrades — when you are pinned three minor versions back because the last bump broke something you could not diagnose — you are maintaining the framework rather than the application. That is the point at which a staged removal becomes cheaper than staying.
Every framework named on this page changes its public API between minor versions, some of them frequently. Treat any specific import path, class name or flag you read anywhere — including here — as a hint to check against the version you have installed, not as a fact.
Top comments (0)