DEV Community: middleware.mind

[Boost]

middleware.mind — Mon, 06 Jul 2026 17:01:33 +0000

middleware.mind

Jul 6

Genkit Agents: Why the Whole Is Greater Than the Sum of Its Parts

#typescript #ai #genkit #agents

8 min read

Genkit Agents: Why the Whole Is Greater Than the Sum of Its Parts

middleware.mind — Mon, 06 Jul 2026 17:01:00 +0000

Most "AI agent frameworks" hand you a big Agent class with fifty options and hope for the best. Genkit takes a different bet: agents are built from the same small primitives you already use (tools, middleware, interrupts, sessions, streaming), so advanced behavior composes in layers instead of being bolted on.

The result is a framework where human-in-the-loop approval, a sandboxed coding assistant, and multi-agent delegation aren't separate features. They're the same handful of ideas snapping together. This is the "greater than the sum of its parts" thing, and it's genuinely fun to use.

TL;DR

If you only read the bold bits:

A working agent is ~10 lines. ai.defineAgent({ system, tools, store }) + agent.chat().sendStream().
You choose where state lives. Pass a store for server-managed sessions, or omit it for client-managed state (the client round-trips the state blob). Same agent, same API.
Interrupts turn a tool call into a pause button. Define a "tool" with no handler; the run stops, hands control to you, and resumes exactly where it left off. This is how you do human-in-the-loop.
Middleware adds real capabilities one line at a time. A use: [...] array stacks a filesystem, a skills library, approval gates, retries, and sub-agent delegation.
The pieces reinforce each other. The approval-gate middleware is built on the same interrupt machinery you'd use by hand. Sub-agents are just agents-as-tools. Artifacts flow between them.
The same agent runs on the server and in the browser. expressHandler(agent) on the backend, remoteAgent({ url }) on the client, with identical streaming and interrupt-resume semantics.

Everything below expands on those points, and every snippet is lifted from the real samples in the Genkit repo. Skip to whatever interests you.

1. The 10-line agent

Start with the smallest possible thing. A tool, an agent, and a chat:

import { z } from 'genkit';
import { FileSessionStore } from 'genkit/beta';
import { ai } from './genkit.js';

export const getWeather = ai.defineTool(
  {
    name: 'getWeather',
    description: 'Get the current weather for a given location.',
    inputSchema: z.object({ location: z.string() }),
    outputSchema: z.object({ weather: z.string() }),
  },
  async (input) => ({ weather: `Sunny in ${input.location}`, temperature: '71F' })
);

export const weatherAgent = ai.defineAgent({
  name: 'weatherAgent',
  system: 'You are an assistant helping with weather information. Use the getWeather tool.',
  tools: [getWeather],
  store: new FileSessionStore('./.snapshots'),
});

Talking to it feels like normal Node.js. Start a chat, send a message, stream the chunks:

const chat = weatherAgent.chat();
const turn = chat.sendStream('What is the weather like in London?');

for await (const chunk of turn.stream) {
  process.stdout.write(chunk.text ?? '');
}

const res = await turn.response;

Multi-turn is free. A single chat carries state across turns via that store, so a follow-up is just another send:

await chat.sendStream('What about Paris?').response;
await chat.sendStream('now say that in French').response;

One line controls where state lives. That store gives you server-managed state: the session lives on the server and the chat tracks it for you. Drop store and the agent is client-managed - the client owns a state blob and echoes it back each turn. Same agent, same chat API; you just choose who holds the memory (and remoteAgent handles the round-trip for you in the browser).

Server-managed vs client-managed state

Omit store and the agent is stateless server-side. Resume from the client-held blob, then hand the updated one back:

// No `store` → the client owns the session state.
export const weatherAgentStateless = ai.defineAgent({
  name: 'weatherAgentStateless',
  system: 'You are a helpful weather assistant...',
  tools: [getWeather],
});

// Resume from the client-held blob, then return the updated one.
const chat = weatherAgentStateless.chat(state ? { state } : undefined);
const res = await chat.sendStream('What is the weather in Tokyo?').response;
return { state: res.raw.state, message: res.message };

Why care? Server-managed state is simplest for a trusted backend. Client-managed state means the server holds nothing between turns - handy for stateless/serverless deployments, easy horizontal scaling, or when the client (not your server) should own the conversation. Tool-calling and multi-turn work identically either way.

That's the whole base layer. Now watch how much you get by layering on top of it.

2. Interrupts: a tool call that never runs (on purpose)

Here's the first idea that pays off everywhere else.

An interrupt is a tool with no handler. When the model "calls" it, the agent doesn't execute anything. It pauses, surfaces the request to you, and waits. You supply the result, and the run resumes from the exact point it stopped. Tool calls become control flow.

This is the cleanest human-in-the-loop primitive I've seen. Consider a banking agent that must get approval before moving money:

export const userApproval = ai.defineInterrupt({
  name: 'userApproval',
  description: 'Ask the user for approval before proceeding with a sensitive action.',
  inputSchema: z.object({
    action: z.string(),
    details: z.string(),
  }),
  outputSchema: z.object({
    approved: z.boolean(),
    feedback: z.string().optional(),
  }),
});

export const bankingAgent = ai.defineAgent({
  name: 'bankingAgent',
  system:
    'You are a banking assistant. If the user wants to transfer money, ALWAYS use ' +
    'the userApproval interrupt to confirm details before calling transferMoney.',
  tools: [userApproval, transferMoney],
  store,
});

The magic is on the response. When the agent pauses, res.interrupts is populated, and each interrupt hands you respond/restart builders to continue the run:

const chat = bankingAgent.chat();
let res = await chat.sendStream('Transfer $500 to my savings account.').response;

// Did the agent pause for approval?
const approval = res.interrupts.find((i) => i.name === 'userApproval');

if (approval) {
  // Resume the SAME chat. It tracked the snapshot for us.
  const resume = chat.resumeStream({
    respond: [approval.respond({ approved: true, feedback: 'Looks good' })],
  });
  res = await resume.response;
}

Why "a tool that never executes" is a superpower

Because the pause is just data (res.interrupts) and the resume is just another call (chat.resumeStream), the mechanism doesn't care where the human is. It works the same whether you're auto-approving in a test, prompting on a CLI, or rendering a dialog in a browser. There's no special "human-in-the-loop mode" to configure. It's the same request/response you already understand, paused in the middle.

There are two resume styles:

respond supplies the tool's output directly (the tool body never runs). Perfect for approvals and "ask the user a question."
restart re-issues the original tool call so it does execute this time, optionally tagged with metadata like { toolApproved: true }. Perfect for "run this only after I approve it."

3. Middleware: capabilities as one-liners

Every agent takes a use: [...] array. Each entry is a piece of middleware that weaves tools and behavior into the agent's generate loop. This is where a bare chatbot becomes something powerful, without you writing the plumbing.

The @genkit-ai/middleware package ships factories like filesystem, skills, toolApproval, retry, artifacts, and agents. Here's a full-featured coding assistant assembled almost entirely out of them:

import { filesystem, retry, skills, toolApproval } from '@genkit-ai/middleware';

export const codingAgent = ai.defineAgent({
  name: 'codingAgent',
  system: `You are an expert AI coding assistant working in a sandboxed workspace...`,
  tools: [runShell, askUser], // custom tools
  use: [
    // Reads + shell are auto-approved; writes require user confirmation.
    toolApproval({
      approved: ['list_files', 'read_file', 'use_skill', 'run_shell', 'ask_user'],
    }),
    // Adds list_files, read_file, write_file, search_and_replace - scoped to a dir.
    filesystem({ rootDirectory: WORKSPACE_DIR, allowWriteAccess: true }),
    // Loads coding conventions on demand via a use_skill tool.
    skills({ skillPaths: [SKILLS_DIR] }),
    // Automatic retry on transient model errors.
    retry(),
  ],
  store,
  maxTurns: 30,
});

Read that use array top to bottom and you've read the agent's entire capability set: it can approve tools, edit a sandboxed filesystem, load skills, and retry on failure. No glue code. Each line is a real, production-grade feature.

The order matters (and the comment tells you why)

In the sample, toolApproval is deliberately placed before filesystem so that the interrupt thrown by a pending approval propagates up cleanly instead of being swallowed by the filesystem middleware's error handling. Middleware composes like an onion, so ordering is a real (documented) knob, not an accident.

4. The payoff: the pieces reinforce each other

This is the section that makes the "sum of its parts" claim concrete. Watch the primitives from sections 2 and 3 combine.

Approval gates are interrupts

That toolApproval middleware in the coding agent? It doesn't invent a new approval system. When a write is attempted, it emits the exact same kind of interrupt you saw in the banking demo. So the client handling code is identical: check res.interrupts, then resumeStream. You learned it once; it works everywhere.

A tool can pause itself with `ctx.interrupt()`

The coding agent's run_shell tool has an AI-powered safety gate. A cheap, fast model classifies each command, and risky ones trigger an interrupt from inside the tool handler:

async (input, ctx) => {
  const isApproved = ctx.metadata?.resumed?.toolApproved === true;

  if (!isApproved) {
    const safetyCheck = await ai.generate({
      model: liteModel,
      prompt: `Evaluate this shell command for safety: "${input.command}" ...`,
      output: { schema: z.object({ verdict: z.enum(['safe', 'risky']), reason: z.string() }) },
    });

    if (safetyCheck.output!.verdict === 'risky') {
      // Pause and hand the decision to the human.
      ctx.interrupt({ command: input.command, reason: safetyCheck.output!.reason });
    }
  }

  // ...actually run the command...
}

Look at what just happened: a model, a tool, and an interrupt combined into a self-guarding action. Safe commands run instantly; risky ones ask permission. That's three primitives producing behavior none of them has alone.

Sub-agents are just agents-as-tools

The agents middleware lets one agent delegate to specialized others. You define sub-agents exactly like any agent, then list them in an orchestrator's use:

export const orchestratorAgent = ai.defineAgent({
  name: 'orchestratorAgent',
  system: 'Analyze the request and delegate to the right sub-agent, then synthesize a final answer.',
  use: [
    agents({
      agents: [
        'researcher', // description auto-discovered from the agent's registry metadata
        { name: 'coder', description: 'Writes, debugs, and explains code...' },
      ],
      maxDelegations: 5,   // guard rail against runaway loops
      historyLength: 4,    // how much context to forward to sub-agents
      artifactStrategy: 'session', // merge sub-agent outputs into the parent session
    }),
    artifacts({ readonly: true }), // orchestrator can READ what sub-agents produced
    retry(),
  ],
});

The middleware injects one delegation tool per sub-agent (delegate_to_researcher, delegate_to_coder). When the orchestrator calls one, the middleware runs that sub-agent and returns its answer as the tool result. And because sub-agents write artifacts that merge into the shared session, the researcher's findings are available to the coder and the orchestrator. Again: composition, not a bespoke "multi-agent engine."

5. The same agent, backend and browser

Here's the part that ties the bow. Everything above works identically from a web client, because the client speaks the same protocol.

Server: expose the agent as an HTTP endpoint.

import { expressHandler } from '@genkit-ai/express';

app.post('/api/bankingAgent', expressHandler(bankingAgent));

Browser: connect with remoteAgent and you get the same chat(), sendStream(), res.interrupts, and resumeStream() you used server-side:

import { remoteAgent, type AgentChat } from 'genkit/beta/client';

const agent = remoteAgent({ url: '/api/bankingAgent' });
const chat = agent.chat();

// Stream a turn
const turn = chat.sendStream('Transfer $500 to my savings account.');
for await (const chunk of turn.stream) {
  setStreamingText(chunk.accumulatedText);
}
const res = await turn.response;

// Same interrupt handling as the server - render a dialog, then resume
const pending = res.interrupts.find((i) => i.name === 'userApproval');
if (pending) {
  // ...user clicks Approve in a React dialog...
  const respond = pending.respond({ approved: true, feedback });
  await chat.resumeStream({ respond: [respond] }).response;
}

That's the whole point in one snippet. The interrupt/resume pattern you learned for a test flow is exactly what powers an approval dialog in production. No new concepts. The server tracks the session snapshot; the client just says "resume with this answer."

Verdict

Plenty of frameworks can call an LLM and run a tool. What makes Genkit's agent layer stand out is that its features are the same few ideas seen from different angles:

Interrupts give you human-in-the-loop.
Middleware gives you filesystems, skills, retries, and delegation.
The approval middleware is interrupts.
Sub-agents are tools.
The browser client is the server API.

Learn the primitives once and everything else is recombination. That's what "greater than the sum of its parts" actually feels like in code: a small vocabulary that keeps buying you bigger things.

Bonus for mobile devs: the same concepts exist in the Genkit Dart SDK, so a Flutter app can share the mental model end to end.

Want the full picture? The official Genkit agents guide walks through all of this in depth.

4 JavaScript Frameworks Powering the Generative AI Revolution (and How to Choose)

middleware.mind — Tue, 21 Apr 2026 18:57:59 +0000

For a long time, Generative AI was a Python story. If you wanted to build with LLMs, you used Python. Period. But things have changed. The JavaScript and TypeScript ecosystem has grown up, and you can now build complex, production-ready AI apps without leaving your favorite language.

But that brings us to the obvious question: Which framework should you actually use?

Let's break down the top 4 JavaScript Gen AI frameworks, look at what they're good at (and where they fall short), and help you pick the right one for your project.

1. LangChain.js: The Ecosystem Giant

If you've spent any time in the AI space, you've heard of LangChain. It's the most established framework and has the largest ecosystem by far.

Best for: Maximum integration compatibility and complex, highly customized AI pipelines.
The Vibe: The Swiss Army Knife.

Quick Example

Here is how you might invoke a simple prompt in LangChain.js:

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  modelName: "gpt-4o-mini",
});

const response = await model.invoke("Explain quantum computing in one sentence.");

console.log(response.content);

The Catch:
LangChain is massive. That means a steep learning curve. The abstractions are powerful, but they can feel heavy or over-engineered if you're just trying to build something simple.

The Verdict: All frameworks are good, but if you want something that feels more like a standard, idiomatic Node.js library with an incredible local developer UI for testing and tracing, you might find other options more approachable.

2. Vercel AI SDK: The UI & Edge Specialist

Vercel has been a dominant force in the frontend world, and their AI SDK brings that same focus on developer experience and performance to Generative AI.

Best for: Next.js developers and edge deployments.
The Vibe: The Frontend Darling.

Quick Example

The Vercel AI SDK makes generating text extremely straightforward:

import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";

const { text } = await generateText({
  model: openai("gpt-4o-mini"),
  prompt: "Explain quantum computing in one sentence.",
});

console.log(text);

The Catch:
It's really built for the Vercel ecosystem and serverless setups. If you're building a standalone Node.js backend or need to orchestrate complex, multi-step background agents, it might not be the best fit.

The Verdict: Perfect if you are all-in on the Vercel ecosystem. However, if you need deep visibility into how your prompts are executing step-by-step across any backend, a framework with dedicated local dev tools might serve you better.

3. LlamaIndex.TS: The Data Specialist

Generative AI is only as good as the data you give it. LlamaIndex is the heavy hitter when it comes to connecting LLMs to your own data sources.

Best for: Advanced RAG (Retrieval-Augmented Generation) and complex data ingestion.
The Vibe: The Data Engineer's LLM Tool.

Quick Example

LlamaIndex excels at RAG (Retrieval-Augmented Generation). Here is a minimal example of indexing a document and querying it:

import { Document, VectorStoreIndex } from "llamaindex";

// Simple RAG setup
const document = new Document({ text: "Quantum computing uses qubits..." });
const index = await VectorStoreIndex.fromDocuments([document]);

const queryEngine = index.asQueryEngine();
const response = await queryEngine.query({
  query: "Explain quantum computing in one sentence.",
});

console.log(response.toString());

The Catch:
LlamaIndex is hyper-focused on the data and retrieval side. It does support agents and tool use, but that's not its main strength.

The Verdict: If data retrieval is your primary bottleneck, this is your framework. But if you want to build full agents that combine data retrieval with complex tool use and structured workflows, you might want a framework that specializes in orchestration.

4. Genkit: The Developer Experience Champion

Developed by Google, Genkit is a framework that focuses heavily on developer experience, typesafety, and production readiness.

Best for: Building complex agents, structured workflows, and developers who value strong typing and great tooling.
The Vibe: The Idiomatic Full-Stack Framework.

Quick Example

Genkit uses a centralized instance to manage configuration and execution:

import { genkit } from 'genkit';
import { anthropic } from '@genkit-ai/anthropic';

// Initialize Genkit with plugins
const ai = genkit({
  plugins: [anthropic()],
});

// Simple generation
const response = await ai.generate({
  model: anthropic.model('claude-haiku-4-5'),
  prompt: 'Explain quantum computing in one sentence.',
});

console.log(response.text);

Genkit takes a slightly different approach. It gives you a set of core primitives (Actions, Flows, Models) that just feel like normal Node.js code. The real killer feature is the Genkit Developer UI—a local web app that lets you inspect traces, run prompts, and debug your execution steps in real-time. It makes seeing exactly what's happening under the hood much easier.

Bonus for Mobile Devs: While this article focuses on JavaScript, it's worth noting that Genkit is also available for Dart. This means if you are building Flutter mobile apps, you can use the same Genkit concepts and patterns on both the backend (via Node.js) and the mobile app itself.

The Catch:
It's newer than LangChain, so the community ecosystem is still growing. That said, it already covers the big players like OpenAI, Anthropic, and Gemini.

The Verdict: Genkit strikes a beautiful balance. It gives you the power to build complex multi-turn agents while maintaining a clean, understandable codebase and providing the best debugging tools in the business.

Summary: Which one should you choose?

Go with LangChain.js if you need to integrate with a massive ecosystem and don't mind a steeper learning curve.
Go with Vercel AI SDK if you're building a React/Next.js app and want the fastest path to a streaming UI.
Go with LlamaIndex.TS if your app's main complexity is data ingestion and retrieval (RAG).
Go with Genkit if you are just starting out with Gen AI. The built-in tooling (like the developer UI for observability and playgrounds) makes it much easier to understand what's happening under the hood, effectively demystifying AI for beginners, while still being powerful enough for production agents.

What do you think?

Have you built anything with these frameworks? Which one is your favorite, and why? Let me know in the comments below!

DEV Community: middleware.mind

[Boost]

Genkit Agents: Why the Whole Is Greater Than the Sum of Its Parts

Genkit Agents: Why the Whole Is Greater Than the Sum of Its Parts

TL;DR

1. The 10-line agent

2. Interrupts: a tool call that never runs (on purpose)

3. Middleware: capabilities as one-liners

4. The payoff: the pieces reinforce each other

Approval gates are interrupts

A tool can pause itself with ctx.interrupt()

Sub-agents are just agents-as-tools

5. The same agent, backend and browser

Verdict

4 JavaScript Frameworks Powering the Generative AI Revolution (and How to Choose)

1. LangChain.js: The Ecosystem Giant

Quick Example

2. Vercel AI SDK: The UI & Edge Specialist

Quick Example

3. LlamaIndex.TS: The Data Specialist

Quick Example

4. Genkit: The Developer Experience Champion

Quick Example

Summary: Which one should you choose?

What do you think?

A tool can pause itself with `ctx.interrupt()`