Stefan 🚀

Posted on Oct 7

Why Declarative API Orchestration Fails at Scale And How We’re Using gRPC + LLMs to Fix It

#grpcplugins #modernapis #api #apiorchestration

The Problem

Apollo, StepZen (now IBM), Tailcall—they’ve all tried the same pattern: declarative API orchestration. It looks elegant. It demos well. But in real-world systems, it can quickly break down.
I know because I built one back in 2018. A GraphQL API Gateway based on the same idea. The concept keeps coming back, but the core issues haven’t gone away.

Apollo Connectors: A Case Study in Overhead

type Booking {
  id: ID!
  checkInDate: String!
  checkOutDate: String!
  status: BookingStatus!
  hostReview: Review
    @connect(
      source: "listings"
      http: { GET: "/review?bookingId={???}" }
      selection: """
      id
      text
      rating
      """
    )
}

This looks clean in a hello-world schema. But once you start building real features, things get messy.

Why Declarative Fails

GraphQL Directives Aren’t Infra

They’re not extensible or type-safe. No scoping to specific types. No support for maps. You end up writing fragile, implicit logic.

The SDL Gets Cluttered

Mixing data model and execution logic leads to unreadable schemas. What should be a clean contract turns into an implementation dump.

It Leaks Implementation Details

You're modeling GraphQL types around how easy they are to map, not how the API should look. That kills flexibility.

It Can’t Solve the N+1 Problem

Fetching 50 bookings with reviews? That’s 50 individual API calls. No batching, no optimization, just repeated overhead.

Testing Is Manual and Incomplete

You need a running Router and playground just to validate basic functionality. No CI hooks. No mocks. No debugger.

Why We Held Off

At WunderGraph, we deliberately avoided Connectors. They weren’t scalable or testable. However, with recent advances in LLMs, we revisited the problem from a different perspective and built a better model.

Our Answer: gRPC Plugins

It’s a plugin system designed for LLMs and human-in-the-loop workflows.

You write (or generate) GraphQL SDL
A compiler creates a gRPC Protobuf definition
The Router loads the plugin over gRPC
The plugin is a standard Go service—debuggable and testable
Cursor Rules walk the LLM through the entire build process

It’s fast, clean, and designed to scale.

Why go-plugin Was the Right Fit

We looked at WebAssembly and scripting engines. go-plugin from Hashicorp won out.

Battle-tested in production tools like Terraform and Vault
High performance over Unix Domain Sockets
Language-agnostic with gRPC interface (Go today, Node and Python soon)
No extra deployment overhead

It gives us the control and simplicity we need without forcing users into a VM or DSL.

Cursor Rules: Operationalizing LLMs

We created a CLI to bootstrap new plugin projects:

wgc router plugin init hello-world

Open the folder in Cursor, and a single prompt gets you:

Compiled gRPC code
Plugin scaffolding
Tests and mocks
Build-ready output

The LLM handles the plumbing. You focus on logic and architecture.

Cosmo gRPC Plugins vs Apollo Connectors

Final Thoughts

Declarative orchestration sounds simple until you're the one scaling it.

You end up with a VM, a DSL, and a brittle runtime that no one asked for. That’s where Connectors are headed. Apollo has to teach classes just to make them usable.

We took a different route.

With gRPC Plugins, LLMs generate the glue code. There’s no new language to learn. No runtime magic. Just real, testable, debuggable code that plays well with modern workflows.

We’re building orchestration for the LLM era. And it works.

Explore the Router Plugins docs

For the full deep dive, check out the original post on WunderGraph’s blog

DEV Community