When an AI feature fails to land on a legacy Rails app, it is almost never the model. It is that nobody could safely change the code underneath it.
We picked up an app running Rails 5.2 with 11% test coverage and a Sidekiq queue nobody had looked at since 2021. The team had already built an LLM classifier. It worked on a laptop and had been sitting in a branch for five months, because merging it meant touching a controller that three people were afraid of.
That is the whole argument: modernization is not a rewrite and it is not a version bump. It is the sequence that makes a legacy Rails app changeable again, and then puts AI in the places where the numbers move. Skip the sequence and you get a prototype that lives in a branch forever.
The sequence, and why you don't get to reorder it
Audit, upgrade, test, refactor, then integrate. Every step exists because the next one is unsafe without it.
- Audit. Map dependencies, security posture, and tech debt against business priorities. You are deciding what not to touch as much as what to fix.
- Upgrade, one major at a time. 4.2 → 5.0 → 5.2 → 6.0 → 6.1 → 7.0 → 7.2 → 8.0. Resolve deprecations at each stop and enable new defaults gradually through config.load_defaults. Skipping a version to save a sprint buys you a month of breaking changes.
- Test. Unit, integration, and system coverage, running on every push in CI. This is not hygiene. It is the precondition for everything after it.
- Refactor. Extract business logic into service objects and concerns. This is where the seams for AI actually get cut.
- Integrate. Only now.
Incremental work also wins on cost - rebuilds have been measured at up to three times the feature development cost of upgrading in place. And there are still over 560,000 live Rails sites, so the population of teams making this decision is not small.
What the AI layer actually looks like
Four components, and none of them are optional if the feature is going to survive a production week.
A provider client. Wrap OpenAI, Anthropic, Hugging Face, or your self-hosted model behind one abstraction. Swap vendors without touching business logic. Swap in a mock during CI so tests are fast and deterministic.
A service object. All AI logic lives here. Testable, replaceable, out of the controller.
A background job. Sidekiq or equivalent. Model calls never sit in the request path. A recommendation engine can chew through user data without adding a millisecond to page load.
**An audit table. **Inputs, outputs, and confidence scores in JSONB on Postgres. When a classification is wrong, you pull the exact pair, fix the prompt, and version the change.
The trade-off nobody puts in the proposal: that audit table grows fast and someone has to own retention on it.
How do you keep the AI layer from quietly costing you money?
Set confidence thresholds before launch, not after the first invoice. When confidence drops below the line, fall back to deterministic logic, a cached prior answer, or a human review queue. Enforce a per-tenant daily cost cap at the job layer, where you can actually stop the work.
Then treat prompts as versioned, testable code and mock model responses in the suite. An untested prompt change is a production change with no rollback story.
When to bring someone in
In-house works when the team ships weekly, has bandwidth for a 20% side program, and the AI use case is scoped tightly. Bring in help when:
- Rails is two or more majors behind and no engineer on the team has run an upgrade this decade.
- A live prototype never got past staging because nobody owns operations for it.
- A board or regulator has set a date, and missing it has a number attached.
- The one senior Rails engineer who understood the plan just gave notice.
What to push back on in a vendor quote: any multi-month discovery phase that ends in a slide deck, any proposal naming juniors on delivery, any pricing without a locked scope.
None of this shows up in a demo. It shows up five months later, when the classifier still works perfectly on someone's laptop and the branch has 400 commits of drift behind main. The upgrade sequence is not the interesting part of adding AI to Rails. It is the part that decides whether the AI ever ships.
Full breakdown of the assessment, upgrade path, and integration patterns: teamvoy.com/blog/how-to-transition-legacy-ruby-on-rails-apps-to-ai-enabled-architectures
Written by Bohdan Varshchuk, CTO at Teamvoy. More engineering writing at teamvoy.com/blog
Top comments (0)