How to Add AI Features to an Existing SaaS (No Rewrite)

Your customers are asking for AI features, and your codebase was not designed with them in mind. The good news: you almost never need a rewrite or a separate "AI platform". An LLM feature is a feature like any other — it lives next to the code it serves, goes through your review process and ships behind a flag.

What makes AI features different is that their output is probabilistic, their cost scales with usage, and they send data to a third party. This is the playbook we use to handle those three facts when we add AI to existing products as part of our AI integration work.

Step 1: pick the first feature with two questions

Score candidate features on two axes: how much value a correct output creates, and whether the data the model needs is already in your system and accessible. The first feature should score high on both.

Feature type	Typical value	Data usually available?	Good first feature?
Extracting fields from documents or emails into records	High — removes manual data entry	Yes, it is the input	Often the best first choice
Summaries and generated reports from existing data	High for busy users	Yes	Yes, if users can see and correct the result
Classification and routing (tickets, leads, transactions)	Medium–high	Yes, plus historical labels	Yes — and easy to measure
Semantic search over the product’s content	Medium–high	Yes, needs indexing	Yes, but plan for retrieval work
An assistant that takes actions in the app	High	Yes, via your API	Better as a second feature, once the scaffolding exists
A general chatbot on the marketing site	Low–medium	Partly	Rarely the best place to start

"Users can see and correct the result" matters more than it looks. A feature where a person reviews the AI output before it is used is far more forgiving than one that acts on its own — and it generates the labelled examples you will use to evaluate and improve it.

Step 2: four architecture defaults

1. A provider-agnostic model layer

Put model calls behind one small interface in your codebase, with the model chosen by configuration. Providers, prices and model quality change every few months; switching should be a config change plus an evaluation run, not a refactor. SDKs such as the Vercel AI SDK or LangChain already give you this abstraction across Anthropic, OpenAI and Google models.

2. Structured output, validated against a schema

If software consumes the result, never parse free text. Ask for output that matches a schema and validate it. In the Vercel AI SDK this is generateText with an Output.object schema; the parsed, validated value comes back as output, and a response that cannot be parsed into the schema throws a NoObjectGeneratedError you can handle explicitly:

import { generateText, NoObjectGeneratedError, Output } from 'ai';
import { z } from 'zod';

const Invoice = z.object({
  vendor: z.string(),
  invoiceNumber: z.string(),
  dueDate: z.string().describe('ISO 8601 date'),
  total: z.number(),
  currency: z.string().length(3),
});

export async function extractInvoice(text: string, model: string) {
  try {
    const { output } = await generateText({
      model, // e.g. from config, so the provider is swappable
      output: Output.object({ schema: Invoice }),
      prompt: 'Extract the invoice fields from this document:\n\n' + text,
    });
    return { ok: true as const, invoice: output };
  } catch (error) {
    if (NoObjectGeneratedError.isInstance(error)) {
      // Route to a human review queue instead of saving bad data.
      return { ok: false as const, reason: 'unparseable', raw: error.text };
    }
    throw error;
  }
}

We used the same principle in an AI assessment app we built: assessment templates are generated as schema-constrained JSON, so the app never has to guess whether the model "probably" returned valid structure.

3. Streaming where users wait, background jobs everywhere else

Interactive features (an assistant, a draft the user watches being written) should stream. Everything else — extraction, classification, report generation — belongs in a background job with retries, so model latency and provider outages never block a web request. Your existing queue is usually fine.

4. Ship behind a feature flag

Roll out to internal users, then a few friendly customers, then everyone. Capture a thumbs-up/down or an edit signal on every output from day one; it is the cheapest evaluation data you will ever get.

Step 3: get multi-tenancy right before launch

Most AI tutorials assume one user and one dataset. A SaaS has many tenants, and the AI layer must respect the same boundaries as the rest of the product:

• Isolation in retrieval. If the feature searches customer content, filter by tenant inside the search query itself — never retrieve broadly and filter afterwards. Storing embeddings in PostgreSQL with pgvector keeps vectors next to your existing tenant and permission columns.
• No cross-tenant prompts. Prompt templates, few-shot examples and caches must never include another customer's data.
• Per-tenant usage metering. Record tokens, model and feature for every call, tagged with tenant and user. You need it for pricing, for abuse limits and to answer "why did our bill double?".
• Per-tenant limits and kill switches. One customer's bulk import should not exhaust your rate limits or budget for everyone else.

Price the feature with real numbers

Measure cost per user action during the pilot. Only then decide whether the feature is included in existing plans, metered, or sold as an add-on — guessing here is how AI features quietly erode margins.

Step 4: know exactly what data goes where

Your customers' security questionnaires will ask which data reaches which AI provider. Answer it on purpose:

• Send the minimum. Redact or drop fields the model does not need. It improves accuracy and cost as well as privacy.
• Use paid API tiers and read their data terms. As of October 2026: OpenAI states that data sent to its API is not used to train its models unless you opt in, and that abuse-monitoring logs are kept for up to 30 days, with zero data retention available to approved customers. Anthropic states it does not use inputs or outputs from its commercial products for training by default. Google's Gemini API terms say prompts and responses from paid services are not used to improve its products, while unpaid-tier data may be used and reviewed by people.
• Document it. Keep a short data-flow note per AI feature: fields sent, provider, retention, region. Update your subprocessor list and privacy policy.

Step 5: put evaluation in CI

Every prompt edit or model upgrade can make the feature better on the example you were looking at and worse on ten you weren't. Treat AI output like any other behaviour under test:

1. Collect 30–100 real inputs with the output a domain expert considers correct.
2. Score automatically: exact checks for structured fields, and an LLM-as-judge rubric for free text — but only after checking the judge's verdicts against human labels, so you know when it can be trusted.
3. Run the suite in CI on every change to prompts, models or retrieval, and block merges that regress.
4. Add every real-world failure your users report to the suite.

Mistakes we see when teams add AI themselves

• Starting with the most impressive feature instead of the most valuable one. A general assistant demos well; extraction or classification usually pays back first.
• Putting raw tables in the prompt. Models do better with less, well-labelled context — the relevant records, not the whole table. It is cheaper too.
• Trusting a single happy-path test. A prompt that works on the example you wrote it against tells you very little. An evaluation set of real inputs tells you a lot.
• Hard-coding one provider everywhere. When prices drop or a better model ships, you want to switch in an afternoon.
• Forgetting the failure path. Timeouts, rate limits and refusals happen in production. Decide what the user sees when they do, and make sure the rest of the product keeps working.
• No owner after launch. Model behaviour drifts as providers update models and your data changes. Someone needs to watch the feedback signal and the evaluation results.

A rollout checklist

Before launch	Done when
Model behind an interface	Changing provider is a config change
Schema-validated outputs	Invalid output is handled, never stored
Evaluation suite in CI	Merges that regress are blocked
Tenant isolation	Retrieval and prompts are tenant-scoped and tested
Usage metering and limits	Cost per tenant and per feature is visible
Fallback behaviour	The product still works when the provider is down
Data-flow documentation	Security questionnaires can be answered from it
Feature flag and feedback capture	You can roll back instantly and see user corrections

None of this requires a new platform or a rewrite. It does require engineers who treat AI as production software. That is what we do in our AI integration service — inside your codebase, following your conventions — and if you would rather have an engineer join your team directly, you can hire AI developers from us. For the SaaS foundations around it, see SaaS development.

FAQ

Do we need to rebuild our product to add AI?

Almost never. AI features can be added inside your existing codebase and infrastructure, behind a feature flag, using the same review and deployment process as any other feature.

Which AI feature should we build first?

One where a correct output creates clear value and the data the model needs is already in your system — structured extraction, summaries, classification or semantic search are common first choices. Features where users review the output before it is used are the safest start.

Will our customers’ data be used to train AI models?

Not on the paid API tiers of the major providers by default, according to their published terms as of October 2026 — but terms differ (for example, Google treats unpaid Gemini API usage differently). We document exactly which data reaches which provider for each feature.

How do we control AI costs?

Meter tokens per tenant and per feature, choose the smallest model that passes your evaluation set, cache where possible, set per-tenant limits, and price the feature from measured cost per action.

Sources

• AI SDK — Generating structured data (generateText with Output.object), accessed Oct 2026
• OpenAI — Data controls in the OpenAI platform, accessed Oct 2026
• Anthropic Privacy Center — Is my data used for model training?, accessed Oct 2026
• Google — Gemini API Additional Terms of Service, accessed Oct 2026
• pgvector — README

---

Originally published on the DablewDevs blog. DablewDevs builds production AI systems and software — AI agents, RAG, MCP servers and SaaS products. See what we build and what it costs.