Building Scalable AI-Powered SaaS with Rails and React

Introduction

Developers building SaaS platforms that incorporate large language models need a stack that balances speed of iteration with production reliability. In this article we walk through a practical architecture that uses Rails for the API, React for the frontend, and a simple LLM integration layer. The goal is to show how to get a working prototype in a week and then scale it to handle real traffic.

Architecture Overview

The system consists of three main services: a Rails API, a React single page application, and an LLM worker that runs in a separate process. All services communicate over HTTPS and share a PostgreSQL database. The LLM worker pulls requests from a Redis queue, calls the external model API, and writes results back to the database.

Setting Up the Rails API

Create a new Rails app with rails new api --api --database=postgresql. Add the pg gem and configure the database URL. Define a Prompt model that stores the user input and the generated response. Use has_many:responses if you need versioning.

class Prompt < ApplicationRecord
  has_many:responses, dependent::destroy
end

Expose a POST /prompts endpoint that validates the payload and enqueues a job:

class PromptsController < ApplicationController
  def create
    prompt = Prompt.create!(prompt_params)
    LmJob.perform_later(prompt.id)
    render json: { id: prompt.id }, status::accepted
  end

  private

  def prompt_params
    params.require(:prompt).permit(:content)
  end
end

Integrating the React Frontend

Bootstrap a React project with Vite for fast hot-module replacement. Use axios to call the Rails endpoint and display a loading state while the LLM job runs. Poll the /prompts/:id endpoint for the response.

function PromptForm() {
  const [content, setContent] = useState("");
  const [result, setResult] = useState(null);

  const submit = async () => {
    const { data } = await axios.post('/prompts', { content });
    const id = data.id;
    const interval = setInterval(async () => {
      const { data: poll } = await axios.get(`/prompts/${id}`);
      if (poll.response) {
        setResult(poll.response);
        clearInterval(interval);
      }
    }, 2000);
  };

  return (
    <div>
      <textarea value={content} onChange={e => setContent(e.target.value)} />
      <button onClick={submit}>Generate</button>
      {result && <pre>{result}</pre>}
    </div>
  );
}

Adding the LLM Feature

Create a background job that pulls the prompt from the queue, calls the external LLM API, and stores the answer. Use the http gem for the request and respect rate limits with exponential backoff.

class LlmJob < ApplicationJob
  queue_as:default

  def perform(prompt_id)
    prompt = Prompt.find(prompt_id)
    response = HTTP.post('https://api.example.com/v1/completions') do |req|
      req.body = { model: 'gpt-4', prompt: prompt.content }.to_json
      req.headers['Authorization'] = "Bearer #{ENV['LLM_API_KEY']}"
    end
    prompt.responses.create!(content: response.parse['choices'][0]['text'])
  end
end

Deploying with Docker

Write a Dockerfile for the Rails API, another for the React build, and a docker-compose.yml that brings up PostgreSQL, Redis, and the LLM worker. Use multi-stage builds to keep the images small.

Monitoring and Scaling

Expose Prometheus metrics from Rails and the worker. Set up Grafana dashboards for queue depth and response latency. When the queue exceeds a threshold, increase the number of worker replicas.

Conclusion

By keeping the API thin, the frontend simple, and the LLM work isolated, you can ship a production-grade AI feature quickly and iterate safely. The same pattern scales to multiple models and larger user bases without major refactoring.

DEV Community