DEV Community

goodpa
goodpa

Posted on

The Model Changed Again — and Your Whole Product Is Just a Harness Around It

There's a line from Hacker News this week that I can't stop thinking about: "Every SaaS business will become a harness around a model." (86 pts)

It sounds like a throwaway prediction. It's actually a description of the room you're already standing in.

Look at the last 24 hours on HN alone: DeepSeek Harness Desktop hit 382 points (the day's peak). Someone shipped a way to run an LLM locally, from the creator of Redis (140 pts). People spent a month coding with GLM 5.3 Flash and wrote it up (103 pts). GPT-6 Astra played World of Warcraft through an agent framework (70 pts). Four different "agentic coding" takes in one day (105 pts).

The model layer is not a foundation you build on. It's a river. And the thing you built is a boat.

What "harness" actually means

A harness is not the horse. A harness is the straps, the buckles, the shafts, the thing that connects the horse to the cart. It's valuable precisely because it's shaped to fit a specific animal — and worthless the moment you swap the animal out.

That's the uncomfortable part. Your product's value increasingly lives in the harness:

  • the prompts and orchestration that turn raw model output into something useful
  • the evals that tell you whether output is good enough to ship
  • the retries, the fallbacks, the guardrails
  • the UX that hides the model's shape from the user
  • the workflow that wraps the whole thing into a job someone will pay for

Notice something? None of that belongs to you when the model changes. You didn't build the intelligence. You built the fit. And fit is version-specific.

Three things that break when the model changes

1. Your evals. You tuned them against model A's failure modes. Model B fails differently. The eval suite that gave you 94% pass rate now measures the wrong thing, loudly.

2. Your costs. Token economics change per model — sometimes per version of the same model. A workflow that was marginally profitable at $X per million tokens can flip to unprofitable overnight, and you won't know until the invoice.

3. Your guardrails. The prompt injection defense you wrote assumed a model that follows instructions a certain way. New model, new jailbreak surface, new way to be embarrassed in production.

This is the same lesson the last four essays in this series kept circling: don't rent your distribution, don't rent your model layer, don't rent your identity layer, don't rent your access layer. Now add the fifth: don't mistake a harness for a foundation.

The cross-border angle (why this bites harder here)

If you run cross-border operations, the harness problem compounds:

  • You're already integrating third-party APIs across jurisdictions — each one a dependency with no SLA you control.
  • You're often running unattended — the failure surfaces at 3am in a timezone where no one is awake to catch it.
  • Your margins are thin enough that a cost shift of 20% is the difference between a business and a hobby.
  • And "the model changed" isn't one event. It's a cadence. Local models (ds4), open-weight flashes (GLM 5.3), desktop harnesses (DeepSeek Harness Desktop) — the pace itself is the risk.

What to actually do

You can't avoid the river. You can refuse to be surprised by it.

  • Measure the fit, not the vibe. Run the same eval suite against every candidate model on a schedule. Know which model your product actually depends on and how much.
  • Build a swap seam. If changing models is a rewrite, you don't have a product, you have a hostage situation. Abstract the model call behind one interface.
  • Track cost per unit of value, not per token. What does one finished job cost you? That's the number that matters when prices move.
  • Assume revocation and drift as normal. Every model endpoint is a lease. Keep a fallback path warm.
  • Put your moat in the harness and above it. The harness is where margin lives today. Your taste — knowing which jobs are worth automating, which outputs are trustworthy — is the part the river can't wash away.

The model will change again. Probably this quarter. The question isn't whether your boat gets shaken. It's whether you tied the hull to the river or built something that floats.


Series: Don't Rent Your Distribution → The Model Layer → The Identity Layer → The Access Layer → **The Harness Layer.

Top comments (1)

Collapse
 
deanlee profile image
Dean Lee •

The harness metaphor captures the operational drag, but the deeper economic trap is negative convexity. When the underlying model improves, the provider captures the upside through pricing power or users treat the higher capability as table stakes. When the model drifts, regresses on edge cases, or deprecates an API endpoint, the entire burden of broken invariants and recalibrating evals falls on the wrapper's balance sheet.

The common prescription to build a swap seam often confuses syntactic abstraction with semantic compatibility. A unified interface makes it trivial to change an endpoint URL, but it does nothing to absorb behavioral variance. Every prompt chain, fallback retry, and output validator is implicitly fitted to the previous model's error distribution. Swapping the engine does not eliminate that dependency; it just introduces basis risk where your guardrails pass synthetic test suites while failing silently on customer edge cases.

If foundation models turn over on a ninety-day cadence, prompt engineering and eval maintenance are not one-time implementation expenses. They are a continuous depreciation charge that directly erodes gross software margins. Treating model swaps as routine maintenance conceals what is fundamentally an ongoing capital expenditure to stay at par.