How to Rescue a Failing AI App Before You Decide to Rebuild It

Quick Answer

When an AI app keeps breaking, a rebuild is rarely the right first move. Turn the failures into a test set, close the security holes, make every model call safe to retry, validate every model output, put a ceiling on cost, and only then compare what is left to repair with the cost of a rewrite. This guide walks through seven steps with working Python and a decision table. The decision table is the author's own judgement and not a published standard, because no neutral study of rescue versus rebuild costs for AI apps turned up while researching it. The guide ends with five teams that offer rescue and audit services and a short FAQ.

Why Do AI Apps Break After the Demo?

A demo exercises one happy path with clean input. Production adds slow networks, rate limits, messy user text, expired tokens, concurrent users and people who try things nobody planned for. AI features add three more problems: the model's output is not guaranteed to match what the next function expects, every call costs money, and the model can be steered by text that arrives from outside the app.

Generated code adds a fourth. Veracode's Spring 2026 update, published on March 24, 2026, tested more than 150 language models on 80 coding tasks across Java, JavaScript, C# and Python with no security-specific prompting. It found that 45% of generated code carried a known flaw, and that while the share of code that compiles had passed 95%, security pass rates had stayed roughly flat. Veracode sells application security products and tested raw models without human review, so read the figure as a warning about unreviewed output and not as a measurement of any one app.

The OWASP Top 10 for LLM Applications (2025 edition) is a useful map of what goes wrong: prompt injection (LLM01), sensitive information disclosure (LLM02), improper output handling (LLM05), excessive agency (LLM06) and unbounded consumption (LLM10) account for most rescue work. The steps below touch each of them.

Step 1: Turn the Failures Into a Test Set

Do not start by changing code. Pull 30 to 50 real failing cases from logs, support tickets and user reports, write down what a correct result looks like, and run them after every change. Anthropic's guidance on building tests says to design evaluations that mirror the real task distribution and to include edge cases such as empty input and overly long input. A rescue without a test set is a series of guesses.

CASES = [
    {"id": "refund-01", "input": "Cancel order 1042 and refund me",
     "must_contain": ["refund"], "must_not_contain": ["cannot access"]},
    {"id": "empty-01", "input": "",
     "must_contain": ["more detail"], "must_not_contain": []},
    {"id": "long-01", "input": "refund " * 5000,
     "must_contain": ["too long"], "must_not_contain": []},
]

def run_case(call_model, case):
    try:
        text = call_model(case["input"]).lower()
    except Exception as exc:
        return {"id": case["id"], "ok": False, "error": type(exc).__name__}
    ok = all(s.lower() in text for s in case["must_contain"]) and not any(
        s.lower() in text for s in case["must_not_contain"]
    )
    return {"id": case["id"], "ok": ok}

def run_suite(call_model):
    results = [run_case(call_model, c) for c in CASES]
    failed = [r for r in results if not r["ok"]]
    print(f"{len(results) - len(failed)}/{len(results)} passed")
    return failed

Keep this file in your own repository. OpenAI's evaluation guide says its hosted Evals platform goes read-only for existing users on October 31, 2026 and shuts down on November 30, 2026, so a test set that lives only there will disappear. The general advice on that page, to adopt eval-driven development and run evaluations on every change, still stands.

Step 2: Close the Security Holes First

Security findings decide whether a rescue is possible at all, so look for them before you tune anything else.

• Secrets in the client or the repository. Search the repository history and the shipped bundle for API keys. A secret scanner such as gitleaks or TruffleHog will do this, and any key that was ever committed should be rotated, not just deleted.
• Authorization enforced in the wrong place. Every endpoint must check who is asking on the server. The OWASP page on excessive agency calls this complete mediation: authorization belongs in the downstream system, not in the model.
• Tools with too much power. If the model can call a function, give that function the narrowest permission it needs, a read-only database role where possible, and human approval for high-impact actions.
• Known vulnerable dependencies. Run pip-audit or npm audit and fix what they report.
• Prompt injection. OWASP states that it is unclear whether fool-proof methods of prevention exist, so do not promise a complete fix. Limit what a manipulated model could do: constrain its behaviour, filter inputs and outputs, and require approval for risky actions.

Step 3: Make Every Model Call Safe to Retry

Many "random" failures are rate limits, timeouts and overloaded servers handled badly. Three rules fix most of them. Use one retry layer, not two. Use exponential backoff with random jitter so retries do not arrive together. Set a timeout shorter than the default.

The defaults are worth knowing. The Anthropic Python SDK retries certain errors twice by default and times out after ten minutes, and requests that time out are retried, so a hung call can hold a worker for a long time. OpenAI's rate-limit guide tells developers to add random jitter to retry delays, to treat a Retry-After header as a minimum, and to avoid stacking SDK retries on top of their own.

import anthropic
from tenacity import retry, retry_if_exception_type, stop_after_attempt, wait_random_exponential

MODEL = "your-model-id"  # model names change often, so keep this in config

# One retry layer (tenacity below) and a short timeout.
client = anthropic.Anthropic(max_retries=0, timeout=20.0)

RETRYABLE = (
    anthropic.RateLimitError,
    anthropic.APIConnectionError,   # includes timeouts
    anthropic.InternalServerError,
)

@retry(
    retry=retry_if_exception_type(RETRYABLE),
    wait=wait_random_exponential(min=1, max=60),
    stop=stop_after_attempt(6),
)
def ask(prompt: str) -> str:
    message = client.messages.create(
        model=MODEL,
        max_tokens=500,
        messages=[{"role": "user", "content": prompt}],
    )
    print("request-id:", message._request_id)  # log this, not the full prompt
    return message.content[0].text

Check the exception names against the SDK version you use, and apply the same pattern to any other provider. Two more points matter. If a step sends an email, charges a card or writes a record, give it an idempotency key so a retry cannot do it twice; Stripe's documentation on idempotent requests is the standard reference. And with streaming, an error can arrive after the server has already returned a 200 status, so do not treat the status code alone as success.

Step 4: Validate Every Model Output Before It Touches Anything

OWASP's guidance on improper output handling is to treat model output like any other untrusted user input: validate it, encode it for where it is going, and use parameterized queries for any database operation that involves it. Structured outputs help. OpenAI states that only its Structured Outputs feature ensures schema adherence, while plain JSON mode only ensures valid JSON. Anthropic's documentation adds that a refusal or a response cut off at the token limit may not match the schema. Validate even when the provider enforces a schema.

from pydantic import BaseModel, Field, ValidationError

class RefundDecision(BaseModel):
    order_id: int
    action: str = Field(pattern="^(refund|deny|escalate)$")
    reason: str = Field(max_length=300)

def parse_decision(raw: str) -> RefundDecision | None:
    try:
        return RefundDecision.model_validate_json(raw)
    except ValidationError:
        return None  # send to a human review queue, never guess

decision = parse_decision(ask("Return a JSON refund decision for order 1042."))
if decision is None:
    print("queued for manual review")

Step 5: Put a Ceiling on Cost

OWASP names the cost risk "unbounded consumption" and includes denial of wallet, where an attacker or a bug drives up a pay-per-use bill. Its mitigations are input size limits, per-user quotas, timeouts, throttling and monitoring. Provider-side caps help too: Anthropic lets you cap monthly spending for a workspace, though not on the default workspace. Google Cloud budgets are alert-only unless you configure a spend cap, so an alert alone will not stop a runaway bill.

from collections import defaultdict
from datetime import date

DAILY_TOKEN_LIMIT = 50_000
usage = defaultdict(int)  # use Redis or your database in production

def check_budget(user_id: str, estimated_tokens: int) -> None:
    key = (user_id, date.today())
    if usage[key] + estimated_tokens > DAILY_TOKEN_LIMIT:
        raise RuntimeError("Daily AI budget reached for this user")
    usage[key] += estimated_tokens

Also cap max_tokens on every call and cap the length of user input before it is sent.

Step 6: Add Observability Without Leaking Data

You cannot fix what you cannot see, but prompts often contain personal or confidential data. Log the provider's request ID, latency, token counts, status and your own case ID by default, and keep full prompts and responses out of logs unless you have a retention policy. Anthropic returns a unique request-id header on every response and asks for it when you contact support.

If you use OpenTelemetry, note that the GenAI semantic conventions have moved to their own repository. Secondary sources describe them as still in Development status and not Stable, so check that repository before depending on attribute names. The OpenTelemetry Python OpenAI instrumentation does not capture prompt or completion content by default, and turning it on means prompts, completions and tool arguments become telemetry that needs the same protection as the data itself.

Step 7: Decide Whether to Keep, Repair or Rebuild

After steps 1 to 6 you will know what is actually wrong. This table is the author's own judgement, built on the cited engineering guidance, and should be adjusted to your product.

Signal	Lean toward repair	Lean toward rebuild
Data model and architecture	Sound, with messy code on top	Wrong core model, such as one user table for everything
Security findings	Fixable in place: keys, auth checks, dependencies	Authorization missing throughout the design
Test coverage	You can add tests around the failing paths	No seam where a test can attach
System size	Large, with paying users on it	Small prototype that is quick to replace
Product direction	The product does what you want	You need fundamentally different behaviour
Continuity	Users must not notice a change	Old version can be retired quickly

Martin Fowler's description of the Strangler Fig approach, dated August 22, 2024, is a gradual path between the two: build new pieces beside the old system and move traffic across so that investment and returns arrive gradually and visibly. Microsoft's Azure Architecture Center lists when that pattern does not fit, including systems small enough that full replacement is simple, which is a fair description of many AI prototypes. Joel Spolsky's 2000 essay arguing against full rewrites is still widely quoted, but it is an opinion piece about desktop software and not data.

Common Mistakes That Keep a Rescue Stuck

• Changing prompts and code at the same time, so nobody knows which change helped.
• Fixing the visible bug while a leaked key stays live.
• Adding retries without idempotency, which turns one failed payment call into two charges.
• Logging full prompts, which creates a second privacy problem.
• Rebuilding before measuring, then reproducing the same design flaws in new code.

Which Teams Offer AI App Rescue and Audit Services? Five Worth Knowing

Figures below come from each company's own website and are self-reported, so confirm them directly before signing anything.

1. Dev Technosys

Dev Technosys offers AI app rescue services for apps that already exist. Its page lists a clear seven-step process that runs from application assessment and root-cause analysis through AI and integration diagnosis, prioritisation, remediation and testing to a stabilisation and growth roadmap. It names crashes, broken APIs, unreliable model output, slow responses and exposed secrets as the problems it handles, and it sets rescue against rebuild so the keep-or-replace question gets a direct answer. Dev Technosys suits teams that want one provider to diagnose the failures and carry the fixes through.

2. Redwerk

Redwerk lists offices in Kyiv and Tallinn and cites 90+ senior engineers, and says it has been fixing complex code since 2005. Its vibe-code cleanup page describes five stages (review, debugging, refactoring, optimisation and optional maintenance), names scanners such as SonarQube, Snyk, Semgrep, Bandit and Trivy, and lists AI tools it reviews, including Cursor, Copilot, Claude Code, Replit and Lovable. It claims no certification, and its percentage improvement figures are marketing numbers. Redwerk suits teams with a compliance-sensitive AI-built SaaS or mobile app that need audit, refactoring and optional maintenance.

3. Cleveroad

Cleveroad lists offices in Tallinn, Claymont, Delaware and Oslo, and cites 280+ engineers. Its rescue page describes six steps, including severity triage that ends in a rescue-or-rebuild call for each module, with fixes such as server-side permission checks, secrets moved to a vault and staging kept apart from production. Deliverables include a ranked audit, tests and runbooks. ISO 27001 and ISO 9001 appear as badges without an issuer named. Cleveroad suits startups and scale-ups with a Lovable-, Bolt- or Cursor-built product that need a ranked audit and a hardening pass.

4. Relevant Software

Relevant Software, founded in 2013, lists offices in Lviv, Warsaw and Valencia and cites 100+ in-house experts. Its cleanup page describes six steps from audit to a production-ready codebase, also covering duplicate-logic consolidation, secrets management, input validation, authorization fixes and slow-query fixes, and says the fixes follow the OWASP Top 10. It adds CI/CD, logging, monitoring and tests, with an optional AI engineering retainer. Certification labels on its home page name no issuer. Relevant Software suits mid-size product companies that want a full cleanup followed by an ongoing engineering retainer.

5. Akveo

Akveo lists offices in Warsaw and Vilnius and an Austin address, and its home page cites 70+ senior experts while its service page says 100+. Clients connect a GitHub repository and receive a launch plan within 48 hours of access, followed by code optimisation, architecture stabilisation, security hardening and testing, with optional support retainers also available. The page says its compliance references are not certifications Akveo holds, which is useful wording. Akveo suits healthcare and regulated-sector startups that need a fast plan and a retained team for an AI-built app.

Frequently Asked Questions

What is AI app rescue?

It is the work of stabilising an existing AI-powered app, or an app built with AI coding tools, by finding the root causes of its failures, fixing security and integration problems, adding tests and monitoring, and deciding which parts to keep, repair or replace.

How much does an AI app rescue cost?

Cost depends on the size of the codebase, the severity of the security findings, how much test coverage must be added and whether support continues afterward. Several firms above offer a free or low-cost assessment first, and most quote after scoping. Ask for a written scope, and treat any price quoted before someone has seen the code with caution. No neutral study of rescue versus rebuild costs for AI-built apps was found while researching this guide.

Should I rescue or rebuild my AI app?

Rescue when the architecture is sound and the problems are security gaps, brittle integrations and missing tests. Rebuild when the core design is wrong, the system is small enough to replace quickly, or the product needs to behave fundamentally differently. Many teams do both in stages by replacing one module at a time.

Can an AI app be fully protected against prompt injection?

OWASP states that it is unclear whether fool-proof methods of prevention exist. The practical approach is to limit what a manipulated model could do: least-privilege tools, output validation, input and output filtering, and human approval for high-impact actions.