Route each agent step by job, then grade the output

Disclosure: Genspark provided the SecondBrain Note recorder discussed in this episode at no cost. Genspark is not a sponsor of this episode.

Send each step to a model class that matches the job, then grade the artifact against the task the user asked for. Knowledge workers, on Wen Sang's account, do not care which model sits behind the product. They want the results, and everything else is a means to an end. His example of a work-output eval is whether his sales proposal responded appropriately to every point in the RFP.

Sang, co-founder and COO of Genspark, laid this out on Chain of Thought in September 2026. Frontier labs, in his framing, are engines. His company builds the vehicle around those engines.

How do you route a step when the user will not pick a model?

Route on the kind of work inside the step. The people he builds for do not care what model is behind the product, and that is what drove the model orchestration.

Yeah, because very early on, we realized, you know, for the knowledge works, for our target audience, they don't care about what model is behind this. They just want the results. And everything else is means to an end. So we're like, OK, we got to put this all together. That's what drove the model orchestration.

Wen Sang, Co-founder and COO at Genspark, on Chain of Thought ep 73

Genspark's Bet: AI Agents Beco… - Chain of Thought | AI Agents, Infrastructure & Engineering - Apple Podcasts

Podcast Episode · Chain of Thought | AI Agents, Infrastructure & Engineering · September 17 · 1h 2m

podcasts.apple.com

Split the job into steps before any model call. Give each step a type you can key on: a plan (a sequence of decisions), a build (something that runs or renders), media (image or video), or a light transform over data you already have. Those labels are an example set you can change. Map each type to a model class. If that call fails, send the same prompt to a fallback class. Show the user one result, and log which class produced the text you kept.

Point planning at the class you use for long reasoning, building at the class you use for code, media at a multimodal class, and light steps at a fast class. Fewer classes means several types can share one. The fallback still needs a home so a failed call has somewhere to go.

What do you send with the model call?

Send the tools and the premium data the step needs in the same package as the model. Sang calls that package the execution layer.

So essentially, the models, the tools and the premium data sets, those three things packaged up as we call it the execution layer. So those are the arms and legs of your agents that will do things with the intelligence from the models, obviously.

Wen Sang, Co-founder and COO at Genspark, on Chain of Thought ep 73

A router that only swaps model names still stalls when the answer depends on a tool or a paid data set. Put the tool names and the data handles on the step, next to the type. A retry must reuse those same handles and change only the model class. If the step is a proposal, the RFP text belongs on that record too, or the later grade can only comment on tone.

How do you grade the work instead of the model?

Grade whether the user got usable work. An autonomous recursive learning algorithm, as Sang described it, grades every interaction between users and the product. The focus is boardroom ready materials, and the reliability, the accuracy, and the usability of the work output.

i care about if my sales proposal responded appropriately to every point that the RFP had in it, right? So those are the reliability, the accuracy, the usability of the work output. That is what our eval benchmark system focuses on, right?

Wen Sang, Co-founder and COO at Genspark, on Chain of Thought ep 73

In his last 10 years running his last company he has not solved one partial differential equation. In that time he wrote a lot of business emails, built a lot of financial models, did a lot of documents, and put together a lot of sales proposals. The grade follows that work. A puzzle unused for a decade does not tell you whether the proposal covered the RFP.

Build the checks from the source document. Pull the requirements out of the RFP first. For each requirement, ask a grader whether the draft responds to it, and store pass or fail. Fix the misses, or list them for a person to review. One blended quality score hides which line was skipped. Sang says he proofreads the email, messages, and social posts his agents handle for him, and that human beings take responsibilities. Leave that review on the path that ships.

The router and the grader are two functions. The fallback runs only when the preferred call fails. The grader runs on whatever text you return.

A minimal sketch to adapt, not a drop-in library.

STEP_ROUTES = {
    "plan": "reasoning",
    "build": "coding",
    "media": "multimodal",
    "light": "fast",
}

FALLBACK = "reasoning"

def route_step(step, call_model, log):
    kind = STEP_ROUTES.get(step["type"], "fast")
    try:
        text = call_model(kind, step["prompt"], step["tools"], step["data"])
    except Exception:
        kind = FALLBACK
        text = call_model(kind, step["prompt"], step["tools"], step["data"])
    log(step["id"], kind)  # which class produced the text you kept
    return text

def grade_output(step, output, grade_work):
    # One pass or fail per requirement pulled from the source document.
    return {item: grade_work(item, output) for item in step["acceptance"]}

call_model and grade_work are stand-ins. Pass the same tools and data into both attempts so a retry is not a different task. grade_work should answer a narrow question, such as whether one RFP line is addressed, and return a structured pass or fail. With a single model class today, keep the dictionary, point every type at that class, and add a second class later without rewriting the steps.

FAQ

Should the user choose the model on each step?

Pick a class per step type and log which class returned the text you kept. Sang's audience wants the results, and does not care what model is behind the product.

What should the grade look at?

Look at the work output: reliability, accuracy, and usability. A direct check is whether a sales proposal responded appropriately to every point in the RFP.

What happens when the preferred model call fails?

Call the fallback with the same prompt, tools, and data, then grade the text you keep. A retry that skips the grade can ship a draft nobody checked against the source.

Takeaway

• Label each step with a type before you choose a model class.
• Map types to classes, and keep a fallback for a failed call.
• Send tools and premium data with the model as one execution bundle.
• Grade each source requirement on its own, and keep the pass or fail list.
• Have a person proofread the draft before it goes out.

The full conversation, with the transcript, is on Chain of Thought.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.