I Ran 4 Free OpenRouter Models Through a Full Article Pipeline. Half of Them Broke.

Most model comparisons test one prompt and score the output. That tells you almost nothing about whether a model survives a real pipeline.

A real pipeline has steps. Draft, then a second pass that reads the draft back and rewrites it. Maybe structured output somewhere. Maybe a tool call. Each step is a new chance for something to break, and the breakages don't correlate neatly with how good the prose is.

So I tested four free models on OpenRouter through an actual two-step content workflow instead of a single call. Same prompts, same setup, no paid tokens anywhere.

Here's what broke and what didn't.

The setup

The pipeline is minimal on purpose:

1. Generate — take a topic, produce a full article
2. Improve — feed the draft back with an SEO-optimization instruction, get a revised version
3. Score — the tool computes a content score on both versions

Step 2 is the interesting one. It's a longer context, it requires the model to actually read and revise rather than generate fresh, and it's where things fell over.

Everything runs through OpenRouter's OpenAI-compatible endpoint, so swapping models is a one-line change:

const res = await fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.OPENROUTER_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "nvidia/nemotron-3-ultra-550b-a55b:free", // ← the only thing that changes
    messages: [{ role: "user", content: prompt }],
  }),
});

That's the whole reason this test is cheap to run. If you want to reproduce it, you're changing a string in a loop.

The results

Model	Draft quality	Improve pass	Verdict
inclusionai/ling-3.0-flash:free	Good, fast	❌ Failed every run	One-shot only
nvidia/nemotron-3-ultra-550b-a55b:free	Good, fast	✅ Clean	Best of four
nvidia/nemotron-nano-9b-v2:free	Weak (score 52)	✅ → 67	Viable with two passes
nvidia/nemotron-3-super-120b-a12b:free	❌ Errored	—	Broken

inclusionai/ling-3.0-flash:free — good writer

Fast, and the prose was genuinely better than I expected from a free tier. Clean structure, no obvious LLM tells.

Then step 2 failed. Every single run, not intermittently. I don't have a root cause — it could be a context limit on the free tier, a provider-side timeout on longer inputs, or something in how the revision prompt is shaped. Consistent failure usually points at a hard constraint rather than flakiness, but I didn't dig further.

Result: you get a good first draft and no way to iterate. Whether that's fatal depends entirely on whether your workflow has a step 2.

nvidia/nemotron-3-ultra-550b-a55b:free — the only one that just worked

Fast, solid text, improve pass completed without complaint. Nothing to write up, which is the point. If you test one model from this list, test this one.

nvidia/nemotron-nano-9b-v2:free — the interesting failure

Two things went wrong here, and only one of them matters.

First, instruction-following: I set length to "short" and got 1,557 words. Small models are notoriously loose about constraints expressed in prose. If length actually matters to you, enforce it downstream — max_tokens, a post-generation truncation step, or a structured output schema — rather than asking politely in the prompt.

Second, quality: content score of 52 on the first draft. That's below what I'd publish.

But the improve pass took it to 67. That's a real jump, and it reframes the model entirely. A 9B model isn't "too small to use" — it's a model that needs two passes to reach where a bigger model lands in one.

That trade-off is worth doing the math on. Two calls to a small fast model versus one call to a large one isn't obviously worse on latency or cost, and on free tiers it's not worse on either. The question is whether your pipeline can tolerate the extra step.

nvidia/nemotron-3-super-120b-a12b:free — nothing

Errored out. Never produced an article. Not much to analyse.

Free-tier endpoints get rate-limited, deprioritized, and occasionally just fall over under load. This might work fine next week. That's the deal with free tiers.

What I'd actually take from this

Test the chain, not the prompt. The most useful finding wasn't a quality ranking. It was that a model with good output couldn't complete the workflow. Single-prompt benchmarks would have ranked Ling highly and told you nothing about the thing that actually disqualified it.

Small models are two-pass models. Nemotron Nano's 52 → 67 is the clearest signal in the whole test. Don't evaluate a small model on its first output. Evaluate it on its second.

Consistent failure ≠ flaky failure. Ling failing every time is diagnostically different from failing sometimes. One suggests a constraint you can find and work around, the other suggests infrastructure you can't control. Log which one you're seeing.

The model is half the cost surface. Images in this pipeline came from Pexels, pulled in automatically. Full run — draft, revision, images — cost nothing end to end. When people price out an AI content workflow they price the tokens and forget everything around them. Sometimes the stock photo bill is the bigger line item.

Caveats

Free tiers change constantly. Models get added, throttled, and deprecated, and providers rotate what's behind a given slug. Everything above is a snapshot from a single run each — not a statistically meaningful benchmark. Content scores come from the tool's own scorer, so treat them as directional and internally comparable, not absolute.

I also haven't tested any of these on non-English content, and I'd bet that's where the free/paid gap widens most.

If you've run free models in a multi-step pipeline, I'm curious where yours broke — especially if you got Ling's revision step working, because I'd like to know what I did wrong.