How To Build an AI or Agent Business With a Moat

Why sustainable AI advantage comes from what you capture, not what you rent

The conversation around AI and Agentic defensibility has become oddly ideological. One camp insists you must own your models and run everything locally to avoid vendor lock-in. The other argues that APIs are the only economically rational choice and worrying about moats is premature optimization.

In my personal opinion, both camps miss the point.

The real question isn’t whether you use APIs or local models. It’s whether you’re designing systems that generate defensible advantages regardless of whose intelligence you’re using.

Foundation model capabilities are commoditizing rapidly. What doesn’t commoditize is the proprietary data your product generates through deeply embedded user workflows.

The Dependency Problem Isn’t What You Think

When people worry about API dependency, they usually frame it as vendor lock-in or cost exposure. Those concerns are valid, but in my opinion secondary.

The actual problem is subtler. When you build on rented intelligence without capturing high-signal data from your users’ workflows, you’re constructing a business on sand. Your competitors can access the same models, implement similar features, and match your outputs. There’s nothing structural keeping users with you.

Model commoditization accelerates this dynamic. For example, GPT-4 was a leap forward when it launched. Within months, multiple providers offered comparable capabilities. Claude Sonnet raised the bar again. The cycle continues.

If your competitive advantage depends on having access to a “better” model, you’re playing a game with a six-month time horizon.

This is why horizontal products built on foundation models alone are vulnerable. They’ll get subsumed by the model providers themselves or by anyone else with API access. But AI in a vertical niche, tied to proprietary data that the big companies can’t crawl, is a different story. It gets better with tighter context over time, compounding advantages that generic horizontal tools can’t replicate.

Start With APIs and Design for Moat, Using this Architecture

Here’s where pragmatism matters. Starting with APIs makes sense for almost everyone.

The unit economics are clear. Google GCP, Microsoft Azure, and Amazon AWS achieve economies of scale you cannot replicate early on. They’ve optimized inference costs, distributed infrastructure globally, and handle reliability at a level that would consume your entire engineering budget to approximate. For validation, iteration, and early growth, APIs are the rational choice.

The strategic error isn’t using APIs. Rather it’s treating them as permanent infrastructure without building anything that compounds independently.

Think about what happens when you route user requests through an API, get responses, and return results. You’ve delivered value, but you haven’t captured anything proprietary. You have server logs and usage metrics, but those don’t differentiate you. Your competitors can implement the same flow tomorrow.

Now consider a different architecture:

1. User makes request.
2. System captures structured context about the request.
3. Routes to API.
4. Gets response.
5. User provides feedback (implicit or explicit).
6. System logs the feedback alongside the original context.

Over time, you accumulate a dataset mapping contexts to outcomes, refined by actual user behavior.

This is data exhaust. It’s high-signal, structured data generated as a byproduct of delivering value. And unlike API access, it’s yours.

Three Mechanisms That Create Distance

Defensibility in AI and Agents comes from three interlocking mechanisms. You need all three working together.

Workflow integration means your AI is embedded in the actual workflow, handling steps that used to require manual effort. This is what it means to build AI-first and AI-native operations. The more deeply integrated, the higher the switching cost. Users aren’t just losing access to a tool. They’re losing a system they’ve built their process around.

I’ve seen companies build AI features that users love but treat as optional supplements. Those products get replicated easily.

Compare that to systems where the AI handles core workflow steps and generates structured outputs that feed into downstream processes. Pulling that out means rebuilding workflows, not just swapping tools.

Data exhaust generation requires intentional design. Not all product usage produces useful data. You need to capture context, actions, and outcomes in a format that improves future model performance or product decisions.

The best data exhaust comes from correction loops. User generates output through your system, edits or refines it, approves final version. You now have ground truth for that context. Do this across thousands of users and you have a training corpus competitors can’t access.

Feedback loops turn data exhaust into compounding advantage. Every user interaction generates data. That data improves model performance, prompt engineering, or product features. Better outputs increase usage. More usage generates more data. The cycle accelerates.

This is where time horizon becomes a moat. A competitor can replicate your current product, but they start the flywheel from zero. You’re already three thousand iterations ahead. The gap widens with each cycle.

Building the Feedback Loop That Compounds

The most defensible AI and Agentic businesses design feedback loops from day one, even when using third-party APIs for inference.

Here’s what this looks like in practice. A user submits a request through your product. Before sending to the API, you capture structured context: user role, task type, key parameters. The API returns a response. You present it to the user. The user takes action: accepts, modifies, rejects. You log the outcome alongside the original context.

Over time, you build a proprietary dataset mapping contexts to outcomes, weighted by user behavior. This dataset has immediate value (you can fine-tune models, improve prompts, optimize for user preferences) and compounding value (it grows with usage and becomes harder to replicate as it scales).

This approach works identically whether you’re using Anthropic’s API or running Llama locally. The moat comes from the data layer, not the inference layer.

Companies that implement these loops see the impact quickly. Within 6 months, you’ll have accumulated enough signal to meaningfully outperform competitors starting from zero. The efficiency gains compound.

But you have to be willing to roll up your sleeves and actually instrument these systems properly, not just subscribe to ChatGPT and call yourself AI-native.

What This Means for Builders

The strategic landscape for AI and Agentic businesses is clarifying. Model access is commoditizing. Speed to market still matters, but not as much as designing for defensibility early.

The right approach is API-first for validation and growth, coupled with deliberate architecture for data moat. Don’t avoid APIs because of misplaced concerns about dependency. Use them pragmatically. But design your product so it generates and captures high-signal data from embedded workflows.

Your moat isn’t the model you use. It’s what you capture while using it.

…

Nick Talwar is a CTO, ex-Microsoft, and a hands-on AI engineer who supports executives in navigating AI adoption. He shares insights on AI-first strategies to drive bottom-line impact.

→ Follow him on LinkedIn to catch his latest thoughts.

→ Subscribe to his free Substack for in-depth articles delivered straight to your inbox.

→ Watch the live session to see how leaders in highly regulated industries leverage AI to cut manual work and drive ROI.