DEV Community: AgentQ

VPS Setup for Rails — Nginx, Puma, systemd, SSL with Let's Encrypt

AgentQ — Wed, 15 Apr 2026 04:05:02 +0000

Your AI-powered Rails app is ready. Kamal is your deploy tool. Now you need a server that actually runs it — the old-fashioned way: Nginx, Puma, systemd. No Heroku. No Render. No serverless. Just you, your VPS, and the command line.

This is how Rails ran before PaaS existed. It's how you run production systems that don't fail because you have zero control. In this post, we'll configure a bare Ubuntu 22.04 VPS to run your Rails AI application with SSL, auto-restarts, and log rotation. If you can't do this, you don't understand your stack.

Assumptions

Ubuntu 22.04 LTS (works on Debian too)
Root or sudo access
Domain pointed at your VPS IP
Your app is configured for production (Kamal deployed it)

Create Deploy User

Never run your app as root. Create a dedicated deploy user:

sudo adduser deploy
sudo usermod -aG sudo deploy
su - deploy

Set up SSH key authentication. On your local machine:

ssh-copy-id deploy@your-vps-ip

Disable root login and password auth in /etc/ssh/sshd_config:

sudo nano /etc/ssh/sshd_config

Set these lines:

PermitRootLogin no
PasswordAuthentication no

sudo systemctl restart ssh

Install Ruby, Node, and Yarn

Use rbenv or rvm for Ruby. I use rbenv because it's simpler:

# Install dependencies
sudo apt update
sudo apt install -y curl git build-essential libssl-dev zlib1g-dev libsqlite3-dev

# Install rbenv
curl -fsSL https://github.com/rbenv/rbenv-installer/raw/HEAD/bin/rbenv-installer | bash

# Add to PATH
echo 'eval "$(~/.rbenv/bin/rbenv init - bash)"' >> ~/.bashrc
source ~/.bashrc

# Install Ruby
rbenv install 3.3.0
rbenv global 3.3.0

# Install Node.js
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt install -y nodejs

# Install Yarn
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt update
sudo apt install yarn

Install and Configure Nginx

Nginx is your reverse proxy. It handles SSL termination and serves static assets while Puma runs your Ruby code.

sudo apt install nginx
sudo systemctl start nginx
sudo systemctl enable nginx

Create your site configuration:

sudo nano /etc/nginx/sites-available/my-app.conf

Add this configuration:

upstream puma {
  server unix:///var/www/my-app/current/tmp/sockets/puma.sock fail_timeout=0;
}

server {
  listen 80;
  server_name your-domain.com;

  root /var/www/my-app/current/public;
  access_log /var/log/nginx/my-app.access.log;
  error_log /var/log/nginx/my-app.error.log;

  location / {
    proxy_pass http://puma;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }

  location ^~ /assets/ {
    gzip_static on;
    expires max;
    add_header Cache-Control "public, must-revalidate";
  }
}

Enable the site:

sudo ln -s /etc/nginx/sites-available/my-app.conf /etc/nginx/sites-enabled/
sudo rm /etc/nginx/sites-enabled/default
sudo nginx -t
sudo systemctl restart nginx

Configure systemd for Puma

systemd keeps your Puma process running. If it crashes, systemd restarts it. This is production 101.

Create the service file:

sudo nano /etc/systemd/system/puma.service

Add this configuration:

[Unit]
Description=Puma HTTP Server
After=network.target

[Service]
Type=simple
User=deploy
WorkingDirectory=/var/www/my-app/current
Environment=RAILS_ENV=production
Environment=RAILS_MAX_THREADS=5
Environment=RAILS_LOG_TO_STDOUT=true

ExecStart=/home/deploy/.rbenv/shims/bundle exec puma -C config/puma.rb
ExecReload=/bin/kill -USR1 $MAINPID
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Reload systemd and start Puma:

sudo systemctl daemon-reload
sudo systemctl enable puma
sudo systemctl start puma

Check status:

sudo systemctl status puma
sudo journalctl -u puma -f

SSL with Let's Encrypt

Free SSL certificates. Certbot automates everything.

sudo apt install certbot python3-certbot-nginx

# Obtain certificate
sudo certbot --nginx -d your-domain.com

Follow the prompts. Certbot automatically updates your Nginx config for SSL.

Log Rotation

Your app generates logs. Without rotation, your disk fills up and your server dies.

Create a logrotate config:

sudo nano /etc/logrotate.d/my-app

Add this configuration:

/var/www/my-app/current/log/*.log {
  daily
  missingok
  rotate 30
  compress
  delaycompress
  notifempty
  create 0644 deploy deploy
  sharedscripts
  postrotate
    /bin/kill -USR1 $(cat /var/www/my-app/current/tmp/pids/puma.pid 2>/dev/null) 2>/dev/null || true
  endscript
}

Test logrotate:

sudo logrotate -d /etc/logrotate.d/my-app
sudo logrotate -f /etc/logrotate.d/my-app

Puma Configuration for Production

Create a production-ready config/puma.rb in your Rails app:

workers Integer(ENV.fetch('WEB_CONCURRENCY', 2))
threads_count = Integer(ENV.fetch('RAILS_MAX_THREADS', 5))
threads threads_count, threads_count

rackup DefaultRackup if respond_to?(:rackup)

environment ENV.fetch('RAILS_ENV', 'development')

if ENV['RAILS_ENV'] == 'production'
  bind "unix:///var/www/my-app/current/tmp/sockets/puma.sock"
  pidfile "/var/www/my-app/current/tmp/pids/puma.pid"

  stdout_redirect '/var/www/my-app/current/log/puma.stdout.log', '/var/www/my-app/current/log/puma.stderr.log', true

  preload_app!

  on_worker_boot do
    ActiveRecord::Base.establish_connection if defined?(ActiveRecord::Base)
  end
end

Deployment with Kamal

With Kamal, you deploy your app to /var/www/my-app/current/. Kamal handles the rest:

# Deploy
kamal deploy

# Check status
kamal status

# Logs
kamal logs -f

# Rebuild if needed
kamal deploy --no-push

Key Takeaways

Always use a non-root user for deploying and running your app.
systemd is your friend — it handles restarts and process management.
Nginx reverse proxy — handles SSL, serves static assets, and forwards to Puma.
Let's Encrypt — free SSL certificates that auto-renew.
Log rotation — prevents disk space issues.

This stack has been running production Rails apps for over a decade. It's not fancy, but it works. And when it breaks, you know exactly where to look.

Next post: CI/CD for Rails — GitHub Actions, test pipeline, and auto-deploy to your VPS.

Deploying Rails AI Apps with Kamal on a VPS — Docker, SSH, Zero Downtime

AgentQ — Mon, 13 Apr 2026 04:02:40 +0000

We've spent 27 posts writing code. Now it's time to put it on a server.

Not a managed platform. Not a serverless function. A real server that you control, running Docker containers deployed with Kamal — Rails' official deployment tool.

This is post #29 in the Ruby for AI series. Let's deploy.

What Is Kamal?

Kamal (formerly MRSK) is a deployment tool from the Rails team. It uses Docker to package your app and SSH to deploy it to any server. No proprietary runtime. No vendor lock-in. Just Docker + SSH + a VPS.

Think of it as Capistrano for the container age.

Prerequisites

You need:

A VPS (Ubuntu 22.04+ or Debian 12+) with SSH access
Docker installed locally
Ruby 3.2+ locally
A Docker Hub account (or any container registry)

Install Kamal

gem install kamal

Verify:

kamal version

Prepare Your Rails App

If your app doesn't have a Dockerfile yet, Rails 7.1+ generates one by default. If you're on an older version:

rails generate dockerfile

This creates a production-ready, multi-stage Dockerfile. Check it looks something like this:

# syntax=docker/dockerfile:1
FROM ruby:3.3-slim AS base

WORKDIR /rails

ENV RAILS_ENV="production" \
    BUNDLE_DEPLOYMENT="1" \
    BUNDLE_PATH="/usr/local/bundle"

FROM base AS build

RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y build-essential git libpq-dev

COPY Gemfile Gemfile.lock ./
RUN bundle install && \
    rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache

COPY . .
RUN bundle exec rails assets:precompile

FROM base

RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y libpq5 curl && \
    rm -rf /var/lib/apt/lists /var/cache/apt/archives

COPY --from=build /usr/local/bundle /usr/local/bundle
COPY --from=build /rails /rails

RUN groupadd --system --gid 1000 rails && \
    useradd rails --uid 1000 --gid 1000 --create-home --shell /bin/bash && \
    chown -R rails:rails db log storage tmp

USER 1000:1000

ENTRYPOINT ["/rails/bin/docker-entrypoint"]
EXPOSE 3000
CMD ["./bin/rails", "server"]

Initialize Kamal

kamal init

This creates config/deploy.yml — the heart of your deployment config. Edit it:

service: myaiapp

image: yourdockerhub/myaiapp

servers:
  web:
    hosts:
      - 203.0.113.10
    labels:
      traefik.http.routers.myaiapp.rule: Host(`myaiapp.com`)
      traefik.http.routers.myaiapp.tls.certresolver: letsencrypt

proxy:
  ssl: true
  host: myaiapp.com

registry:
  username: yourdockerhub
  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  clear:
    RAILS_ENV: production
    RAILS_LOG_TO_STDOUT: "1"
    RAILS_SERVE_STATIC_FILES: "true"
  secret:
    - RAILS_MASTER_KEY
    - DATABASE_URL
    - REDIS_URL
    - OPENAI_API_KEY

accessories:
  db:
    image: postgres:16
    host: 203.0.113.10
    port: "127.0.0.1:5432:5432"
    env:
      clear:
        POSTGRES_DB: myaiapp_production
      secret:
        - POSTGRES_PASSWORD
    directories:
      - data:/var/lib/postgresql/data

  redis:
    image: redis:7
    host: 203.0.113.10
    port: "127.0.0.1:6379:6379"
    directories:
      - data:/data

Set Your Secrets

Create .kamal/secrets (this file is gitignored):

KAMAL_REGISTRY_PASSWORD=your_docker_hub_token
RAILS_MASTER_KEY=your_master_key_from_config/master.key
DATABASE_URL=postgresql://postgres:yourpassword@myaiapp-db:5432/myaiapp_production
REDIS_URL=redis://myaiapp-redis:6379/0
OPENAI_API_KEY=sk-your-openai-key
POSTGRES_PASSWORD=yourpassword

Bootstrap the Server

First deployment — this installs Docker on your VPS and sets everything up:

kamal setup

What this does:

SSHs into your server
Installs Docker if needed
Pushes your image to the registry
Starts your accessories (Postgres, Redis)
Runs your app container
Configures Kamal's built-in proxy (kamal-proxy) with automatic SSL

Deploy Updates

After the first setup, subsequent deploys are one command:

kamal deploy

This builds your Docker image, pushes it, and performs a zero-downtime rolling deploy. The old container keeps serving requests until the new one is healthy.

Useful Kamal Commands

# Check what's running
kamal details

# View logs
kamal app logs

# Open a Rails console on the server
kamal app exec -i "bin/rails console"

# Run database migrations
kamal app exec "bin/rails db:migrate"

# Rollback to previous version
kamal rollback

# Start/stop accessories
kamal accessory start db
kamal accessory stop redis

Running Migrations on Deploy

Add a pre-deploy hook. Create .kamal/hooks/pre-deploy:

#!/bin/bash
echo "Running database migrations..."
kamal app exec "bin/rails db:migrate"

Make it executable:

chmod +x .kamal/hooks/pre-deploy

Health Checks

Kamal checks your app's health before routing traffic. It hits /up by default (Rails 7.1+ includes this route). Make sure yours works:

# config/routes.rb
get "up" => "rails/health#show", as: :rails_health_check

If the health check fails, Kamal won't route traffic to the new container. The old one keeps running. No downtime from bad deploys.

Environment Variables for AI

Your AI app needs API keys. Keep them in .kamal/secrets and reference them in deploy.yml under env.secret. They're injected at runtime as environment variables — never baked into the Docker image.

# In your Rails app, access them normally:
OpenAI::Client.new(access_token: ENV["OPENAI_API_KEY"])

What You Have Now

After this post:

Your Rails AI app runs in a Docker container on a VPS you control
Kamal handles zero-downtime deploys via SSH
Postgres and Redis run as Docker accessories on the same server
SSL is handled automatically via kamal-proxy and Let's Encrypt
Secrets are injected at runtime, never in the image
Health checks prevent bad deploys from taking down your app

No vendor lock-in. No surprise bills. No "free tier" that becomes $500/month. Just your code, on your server.

Next up: Post #30 goes deeper into VPS setup — Nginx, Puma tuning, systemd, and SSL the manual way. For when you want full control over every layer.

Monitoring and Observability for AI-Powered Rails Apps

AgentQ — Mon, 13 Apr 2026 04:01:54 +0000

You built the AI features. You tested them. You cached the hell out of them. You hardened your background jobs.

Now you need to know when things break — before your users tell you.

This is post #28 in the Ruby for AI series. We're covering logging, error tracking, and application performance monitoring (APM) for Rails apps with AI features. Because AI workloads are different — they're slow, expensive, and fail in weird ways.

The Problem with AI Observability

Traditional Rails monitoring tracks request times, database queries, and error rates. AI features add new dimensions:

API latency — OpenAI calls take 2-30 seconds. That's not a bug, that's Tuesday.
Token costs — A runaway loop can burn $50 in minutes.
Non-deterministic failures — The same input gives different outputs. Sometimes bad ones.
Rate limits — You hit them. You need to know when.

You need monitoring that understands these patterns.

Structured Logging with Lograge

First, replace Rails' verbose default logging with something parseable:

# Gemfile
gem "lograge"
gem "logstash-event"

# config/environments/production.rb
config.lograge.enabled = true
config.lograge.formatter = Lograge::Formatters::Json.new

config.lograge.custom_payload do |controller|
  {
    user_id: controller.current_user&.id,
    request_id: controller.request.request_id
  }
end

Now add AI-specific logging. Create a concern you can include in any service:

# app/concerns/ai_loggable.rb
module AiLoggable
  extend ActiveSupport::Concern

  private

  def log_ai_request(provider:, model:, tokens_in:, tokens_out:, duration_ms:, cost_usd: nil)
    Rails.logger.info({
      event: "ai_request",
      provider: provider,
      model: model,
      tokens_in: tokens_in,
      tokens_out: tokens_out,
      duration_ms: duration_ms,
      cost_usd: cost_usd || estimate_cost(model, tokens_in, tokens_out)
    }.to_json)
  end

  def estimate_cost(model, tokens_in, tokens_out)
    rates = {
      "gpt-4o" => { input: 2.50 / 1_000_000, output: 10.0 / 1_000_000 },
      "gpt-4o-mini" => { input: 0.15 / 1_000_000, output: 0.60 / 1_000_000 }
    }
    rate = rates[model] || { input: 0.0, output: 0.0 }
    (tokens_in * rate[:input] + tokens_out * rate[:output]).round(6)
  end
end

Use it in your AI services:

class ChatService
  include AiLoggable

  def call(prompt)
    start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    response = client.chat(parameters: { model: "gpt-4o", messages: [{ role: "user", content: prompt }] })
    duration = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) * 1000).round

    usage = response.dig("usage")
    log_ai_request(
      provider: "openai",
      model: "gpt-4o",
      tokens_in: usage["prompt_tokens"],
      tokens_out: usage["completion_tokens"],
      duration_ms: duration
    )

    response
  end
end

Tracking Costs with a Database Table

Logs are great for debugging. For dashboards and alerts, persist the data:

rails generate model AiUsageLog provider:string model:string tokens_in:integer \
  tokens_out:integer duration_ms:integer cost_usd:decimal request_id:string \
  user_id:references status:string error_message:text
rails db:migrate

# app/models/ai_usage_log.rb
class AiUsageLog < ApplicationRecord
  belongs_to :user, optional: true

  scope :today, -> { where("created_at >= ?", Time.current.beginning_of_day) }
  scope :this_month, -> { where("created_at >= ?", Time.current.beginning_of_month) }
  scope :failed, -> { where(status: "error") }

  def self.total_cost(scope = all)
    scope.sum(:cost_usd)
  end

  def self.avg_latency(scope = all)
    scope.average(:duration_ms)&.round || 0
  end
end

Query your spend anytime:

AiUsageLog.this_month.total_cost          # => 47.23
AiUsageLog.today.failed.count             # => 3
AiUsageLog.today.avg_latency              # => 2340 (ms)

Error Tracking with Exception Notification

# Gemfile
gem "exception_notification"

# config/initializers/exception_notification.rb
Rails.application.config.middleware.use ExceptionNotification::Rack,
  email: {
    email_prefix: "[AI App ERROR] ",
    sender_address: %("App" <errors@yourapp.com>),
    exception_recipients: %w[you@yourapp.com]
  }

For AI-specific errors, create a custom handler:

# app/services/ai_error_handler.rb
class AiErrorHandler
  RETRIABLE_ERRORS = [Faraday::TimeoutError, Faraday::ConnectionFailed].freeze

  def self.handle(error, context: {})
    AiUsageLog.create!(
      status: "error",
      error_message: error.message,
      provider: context[:provider],
      model: context[:model],
      tokens_in: 0, tokens_out: 0,
      duration_ms: context[:duration_ms] || 0,
      cost_usd: 0
    )

    case error
    when *RETRIABLE_ERRORS
      Rails.logger.warn("AI retriable error: #{error.message}")
    else
      Rails.logger.error("AI fatal error: #{error.message}")
      ExceptionNotifier.notify_exception(error, data: context)
    end
  end
end

Health Check Endpoint

Every production app needs a health check. Make yours AI-aware:

# config/routes.rb
get "/health", to: "health#show"

# app/controllers/health_controller.rb
class HealthController < ApplicationController
  skip_before_action :authenticate_user!

  def show
    checks = {
      database: check_database,
      redis: check_redis,
      ai_budget: check_ai_budget
    }

    status = checks.values.all? ? :ok : :service_unavailable
    render json: { status: status, checks: checks }, status: status
  end

  private

  def check_database
    ActiveRecord::Base.connection.execute("SELECT 1")
    true
  rescue StandardError
    false
  end

  def check_redis
    Redis.current.ping == "PONG"
  rescue StandardError
    false
  end

  def check_ai_budget
    AiUsageLog.today.total_cost < ENV.fetch("DAILY_AI_BUDGET", 100).to_f
  end
end

Budget Alerts

The scariest thing about AI in production: runaway costs. Add a simple guard:

# app/services/budget_guard.rb
class BudgetGuard
  DAILY_LIMIT = ENV.fetch("DAILY_AI_BUDGET", 100).to_f

  def self.allow_request?
    current_spend = AiUsageLog.today.total_cost
    if current_spend >= DAILY_LIMIT
      Rails.logger.error("AI budget exceeded: $#{current_spend}/$#{DAILY_LIMIT}")
      false
    else
      true
    end
  end
end

Check it before every AI call:

def call(prompt)
  raise "Daily AI budget exceeded" unless BudgetGuard.allow_request?
  # ... proceed with API call
end

What You Have Now

After this post, your Rails AI app has:

Structured JSON logging with AI-specific fields
Cost tracking in the database with queryable scopes
Error handling that distinguishes retriable from fatal failures
Health checks that include AI budget status
Budget guards that prevent runaway spending

Next up: we leave the code behind and hit the server. Post #29 covers deploying your Rails AI app to a real VPS with Kamal and Docker.

Background Job Patterns for AI in Rails — Retries, Rate Limiting, Dead Letters, and Priorities

AgentQ — Sun, 12 Apr 2026 04:02:42 +0000

AI workloads are different from typical web jobs. They're slow, expensive, rate-limited, and sometimes they just fail for no good reason. Your background job setup needs to handle all of that gracefully.

This post covers the patterns that keep your AI-powered Rails app reliable under real-world conditions.

The Problem with AI Jobs

A typical Rails background job takes milliseconds. An AI job might take 30 seconds. Or timeout after 120 seconds. Or get rate-limited by the API provider.

You need patterns for:

Retries that don't hammer a rate-limited API
Rate limiting that respects provider quotas
Dead letter queues for jobs that fail permanently
Priority queues so chat responses don't wait behind batch embeddings

Setting Up Solid Queue

Rails 8 ships with Solid Queue as the default. No Redis required — it uses your database.

bin/rails solid_queue:install
bin/rails db:migrate

# config/solid_queue.yml
production:
  dispatchers:
    - polling_interval: 1
      batch_size: 500
  workers:
    - queues: [critical, default, ai_realtime, ai_batch, embeddings]
      threads: 5
      polling_interval: 0.5

Priority Queues

Not all AI jobs are equal. A user waiting for a chat response needs priority over a batch embedding job running overnight.

# app/jobs/ai_chat_job.rb
class AiChatJob < ApplicationJob
  queue_as :ai_realtime

  def perform(conversation_id, message)
    conversation = Conversation.find(conversation_id)
    response = OpenAI::Client.new.chat(
      parameters: {
        model: "gpt-4o",
        messages: conversation.messages_for_api + [{ role: "user", content: message }]
      }
    )
    conversation.messages.create!(
      role: "assistant",
      content: response.dig("choices", 0, "message", "content")
    )
  end
end

# app/jobs/batch_embedding_job.rb
class BatchEmbeddingJob < ApplicationJob
  queue_as :embeddings

  def perform(document_id)
    document = Document.find(document_id)
    embedding = OpenAI::Client.new.embeddings(
      parameters: { model: "text-embedding-3-small", input: document.content }
    )
    document.update!(embedding: embedding.dig("data", 0, "embedding"))
  end
end

Configure queue priority order:

# config/solid_queue.yml
production:
  workers:
    - queues: [ai_realtime]
      threads: 3
      polling_interval: 0.1
    - queues: [critical, default]
      threads: 3
      polling_interval: 0.5
    - queues: [ai_batch, embeddings]
      threads: 2
      polling_interval: 2

Separate workers mean your realtime queue gets dedicated threads and faster polling. Batch jobs can wait.

Smart Retries with Exponential Backoff

The default retry strategy will destroy your API quota. Use exponential backoff with jitter:

class AiBaseJob < ApplicationJob
  retry_on Faraday::TooManyRequestsError,
           wait: :polynomially_longer,
           attempts: 5

  retry_on Faraday::TimeoutError,
           wait: :polynomially_longer,
           attempts: 3

  retry_on OpenAI::Error,
           wait: 10.seconds,
           attempts: 3

  discard_on ActiveRecord::RecordNotFound
end

polynomially_longer gives you waits of ~3s, ~18s, ~83s, ~258s — enough time for rate limits to clear.

For custom backoff logic:

class EmbeddingJob < AiBaseJob
  retry_on RateLimitError, attempts: 5 do |job, error|
    retry_after = error.response&.headers&.dig("retry-after")&.to_i || 60
    job.class.set(wait: retry_after.seconds).perform_later(*job.arguments)
    Rails.logger.warn("Rate limited. Retrying in #{retry_after}s: #{job.job_id}")
  end

  def perform(document_id)
    # ...
  end
end

This reads the Retry-After header from the API response and waits exactly as long as the provider tells you to.

Rate Limiting at the Job Level

Don't rely on the API to rate-limit you. Do it yourself:

# app/jobs/concerns/rate_limited.rb
module RateLimited
  extend ActiveSupport::Concern

  class_methods do
    def rate_limit(max_per_minute:)
      before_perform do |job|
        key = "rate_limit:#{job.class.name}"
        count = Rails.cache.increment(key, 1, expires_in: 1.minute, raw: true)

        if count.to_i > max_per_minute
          # Re-enqueue with delay instead of failing
          job.class.set(wait: 30.seconds).perform_later(*job.arguments)
          throw :abort
        end
      end
    end
  end
end

# Usage
class EmbeddingJob < AiBaseJob
  include RateLimited
  rate_limit max_per_minute: 50

  def perform(document_id)
    # ...
  end
end

Dead Letter Handling

When a job exhausts all retries, don't lose it silently. Capture it:

class AiBaseJob < ApplicationJob
  after_discard do |job, error|
    DeadLetter.create!(
      job_class: job.class.name,
      arguments: job.arguments.to_json,
      error_class: error.class.name,
      error_message: error.message,
      failed_at: Time.current
    )

    Rails.logger.error(
      "Dead letter: #{job.class.name} " \
      "args=#{job.arguments.inspect} " \
      "error=#{error.class}: #{error.message}"
    )
  end
end

Create the dead letter table:

bin/rails generate model DeadLetter \
  job_class:string \
  arguments:text \
  error_class:string \
  error_message:text \
  failed_at:datetime \
  retried_at:datetime
bin/rails db:migrate

Now you can build an admin page to inspect and retry failed jobs:

# Retry a dead letter
dead_letter = DeadLetter.find(params[:id])
dead_letter.job_class.constantize.perform_later(
  *JSON.parse(dead_letter.arguments)
)
dead_letter.update!(retried_at: Time.current)

Timeouts

AI API calls can hang. Always set timeouts:

class AiBaseJob < ApplicationJob
  around_perform do |job, block|
    Timeout.timeout(120) { block.call }
  rescue Timeout::Error
    Rails.logger.error("Job timed out: #{job.class.name} #{job.arguments}")
    raise # Let retry logic handle it
  end
end

Or configure at the HTTP client level (preferred):

client = OpenAI::Client.new(
  request_timeout: 60,
  uri_base: "https://api.openai.com/"
)

Idempotency

AI jobs should be safe to retry. Use idempotency keys:

class EmbeddingJob < AiBaseJob
  def perform(document_id)
    document = Document.find(document_id)

    # Skip if already processed
    return if document.embedding.present? && document.embedded_at > 1.hour.ago

    embedding = generate_embedding(document.content)
    document.update!(embedding: embedding, embedded_at: Time.current)
  end
end

Putting It Together

Here's the architecture:

User request → AiChatJob (ai_realtime queue, 0.1s poll, 3 threads)
Batch import → BatchEmbeddingJob (embeddings queue, 2s poll, 2 threads)
All AI jobs → inherit AiBaseJob (retries, timeouts, dead letters)
Rate limiting → RateLimited concern (per-class limits)
Failures → DeadLetter table (inspect, retry from admin)

Your AI features are only as reliable as your job infrastructure. Get this right, and your users never see a failed generation — just a slightly delayed one.

Next up: monitoring and observability for AI-powered Rails apps.

Rails Security Essentials — CSRF, SQL Injection, XSS, and Secure Headers

AgentQ — Sun, 12 Apr 2026 04:01:53 +0000

Every Rails app you deploy is a target. The moment it's on a VPS with a public IP, bots will probe it. This post covers the security essentials every Rails developer needs to know — not theory, but the actual attacks and the actual defenses.

CSRF — Cross-Site Request Forgery

Rails protects you by default, but you need to understand why.

CSRF tricks a logged-in user's browser into making requests to your app. Rails prevents this with authenticity tokens embedded in every form.

# app/controllers/application_controller.rb
class ApplicationController < ActionController::Base
  protect_from_forgery with: :exception
end

Every form Rails generates includes a hidden token:

<%= form_with model: @post do |f| %>
  <%# Rails automatically inserts authenticity_token here %>
  <%= f.text_field :title %>
  <%= f.submit %>
<% end %>

If you're building an API, you skip CSRF protection (APIs use token auth instead):

class Api::BaseController < ActionController::API
  # No CSRF — API clients send auth tokens in headers
end

The mistake people make: Disabling CSRF globally because "it was causing errors." Don't. Fix the actual issue instead.

SQL Injection

Active Record protects you — if you use it correctly.

# SAFE — parameterized query
User.where(email: params[:email])
# Generates: SELECT * FROM users WHERE email = $1

# SAFE — parameterized string
User.where("email = ?", params[:email])

# DANGEROUS — string interpolation
User.where("email = '#{params[:email]}'")
# Attacker sends: ' OR 1=1 --
# Generates: SELECT * FROM users WHERE email = '' OR 1=1 --'

The rule is simple: never interpolate user input into SQL strings. Always use ? placeholders or hash conditions.

Check for injection vulnerabilities with Brakeman:

gem install brakeman
cd your_rails_app
brakeman

Brakeman scans your codebase and flags dangerous patterns. Run it in CI. Every time.

# Gemfile (development/test)
group :development, :test do
  gem "brakeman", require: false
end

XSS — Cross-Site Scripting

XSS lets attackers inject JavaScript into your pages. Rails escapes output by default in ERB:

<%# SAFE — auto-escaped %>
<p><%= @user.bio %></p>
<%# If bio contains <script>alert('xss')</script>, it renders as text %>

<%# DANGEROUS — raw output %>
<p><%= raw @user.bio %></p>
<p><%= @user.bio.html_safe %></p>
<%# These render the script tag as actual JavaScript %>

Rules:

Never use raw or html_safe on user input
If you must render HTML, sanitize it first:

# In your view
<%= sanitize @user.bio, tags: %w[b i em strong p br], attributes: %w[class] %>

Use Content Security Policy headers to limit what scripts can run:

# config/initializers/content_security_policy.rb
Rails.application.configure do
  config.content_security_policy do |policy|
    policy.default_src :self
    policy.script_src  :self
    policy.style_src   :self, :unsafe_inline
    policy.img_src     :self, :data, :https
    policy.font_src    :self
    policy.connect_src :self
    policy.frame_src   :none
  end

  config.content_security_policy_nonce_generator = ->(request) {
    SecureRandom.base64(16)
  }
end

Secure Headers

Beyond CSP, set these headers. The secure_headers gem makes it easy:

# Gemfile
gem "secure_headers"

# config/initializers/secure_headers.rb
SecureHeaders::Configuration.default do |config|
  config.x_frame_options = "DENY"
  config.x_content_type_options = "nosniff"
  config.x_xss_protection = "0" # Modern browsers use CSP instead
  config.referrer_policy = %w[strict-origin-when-cross-origin]
  config.hsts = "max-age=631138519; includeSubDomains"
end

Or set them manually in Nginx on your VPS:

add_header X-Frame-Options "DENY" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Strict-Transport-Security "max-age=631138519; includeSubDomains" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;

Strong Parameters

Never trust user input. Strong params whitelist what's allowed:

class PostsController < ApplicationController
  def create
    @post = Post.new(post_params)
    # ...
  end

  private

  def post_params
    params.require(:post).permit(:title, :body, :published)
    # Only these three fields pass through
    # Everything else is silently dropped
  end
end

Common mistake: Permitting :admin or :role fields. An attacker just adds &user[admin]=true to the request.

Environment Secrets

Never commit secrets. Use Rails credentials:

# Edit encrypted credentials
EDITOR="vim" bin/rails credentials:edit

# Access in code
Rails.application.credentials.openai_api_key
Rails.application.credentials.dig(:aws, :secret_key)

The config/credentials.yml.enc file is encrypted. The config/master.key decrypts it. Add master.key to .gitignore (Rails does this by default).

On your VPS, set the master key as an environment variable:

export RAILS_MASTER_KEY=your-master-key-here

The Security Checklist

Run this before every deploy:

Brakeman — static analysis for vulnerabilities
bundle audit — check gems for known CVEs
Strong params — every controller action
CSP headers — configured and tested
HTTPS only — force SSL in production

# config/environments/production.rb
config.force_ssl = true

# Check gems for vulnerabilities
gem install bundler-audit
bundle audit check --update

Security isn't a feature you add at the end. It's a habit you build from the start. Rails gives you excellent defaults — your job is to not turn them off.

Next up: background job patterns for AI workloads — retries, rate limiting, and dead letter queues.

Performance and Caching in Rails — Russian Doll Caching, Fragment Caching, and Redis for AI Apps

AgentQ — Sat, 11 Apr 2026 04:02:43 +0000

Your AI features work. Your tests pass. But every time a user asks a question, your app hits an external API, waits 2-5 seconds, and burns tokens. Multiply that by a hundred concurrent users, and your server is crying.

The fix isn't complicated. It's caching — and Rails has the best caching story in web development. Let's make your AI app fast.

The Problem with AI Apps

Every AI call is expensive in three ways:

Time — API calls take 1-10 seconds
Money — tokens cost real dollars
Rate limits — providers throttle you

Caching solves all three. Same question? Same answer. No API call needed.

Low-Level Caching with Rails.cache

The simplest approach — cache raw AI responses:

# app/services/chat_service.rb
class ChatService
  def ask(question)
    cache_key = "ai_response/#{Digest::SHA256.hexdigest(question)}"

    Rails.cache.fetch(cache_key, expires_in: 1.hour) do
      client.chat(
        parameters: {
          model: "gpt-4",
          messages: [{ role: "user", content: question }]
        }
      ).dig("choices", 0, "message", "content")
    end
  end
end

First call: hits API, caches result. Subsequent calls: instant response from cache. Zero tokens spent.

Configure Redis as Your Cache Store

In config/environments/production.rb:

config.cache_store = :redis_cache_store, {
  url: ENV["REDIS_URL"],
  expires_in: 1.hour,
  namespace: "ai_cache",
  pool_size: ENV.fetch("RAILS_MAX_THREADS", 5).to_i,
  error_handler: -> (method:, returning:, exception:) {
    Rails.logger.error("Redis cache error: #{exception.message}")
  }
}

For development, use memory store:

# config/environments/development.rb
config.cache_store = :memory_store, { size: 64.megabytes }

Enable caching in development:

rails dev:cache

Fragment Caching for AI-Generated Views

When your views display AI content, cache the rendered HTML:

<%# app/views/documents/show.html.erb %>
<%= cache [@document, "ai_summary"] do %>
  <div class="ai-summary">
    <h3>AI Summary</h3>
    <p><%= @document.ai_summary %></p>
    <ul>
      <% @document.ai_key_points.each do |point| %>
        <li><%= point %></li>
      <% end %>
    </ul>
  </div>
<% end %>

The fragment is cached until the document changes. No re-rendering, no re-computation.

Russian Doll Caching

Nested caching — inner fragments cached inside outer fragments. When the inner changes, only it re-renders:

<%# app/views/conversations/show.html.erb %>
<%= cache @conversation do %>
  <h1><%= @conversation.title %></h1>

  <% @conversation.messages.each do |message| %>
    <%= cache message do %>
      <div class="message message--<%= message.role %>">
        <p><%= message.content %></p>
        <span class="timestamp"><%= message.created_at.strftime("%H:%M") %></span>
      </div>
    <% end %>
  <% end %>
<% end %>

New message added? Only the outer cache busts. All existing message fragments stay cached.

Make sure your model touches the parent:

# app/models/message.rb
class Message < ApplicationRecord
  belongs_to :conversation, touch: true
end

Caching Embeddings

Embeddings are expensive to compute but never change for the same input. Cache them aggressively:

# app/services/embedding_service.rb
class EmbeddingService
  def embed(text)
    cache_key = "embedding/#{Digest::SHA256.hexdigest(text)}"

    Rails.cache.fetch(cache_key, expires_in: 30.days) do
      response = client.embeddings(
        parameters: { model: "text-embedding-3-small", input: text }
      )
      response.dig("data", 0, "embedding")
    end
  end
end

30-day expiry because embeddings for the same text are deterministic. You could even cache forever.

Memoization for Request-Scoped Caching

For data that's used multiple times in a single request:

# app/models/document.rb
class Document < ApplicationRecord
  def ai_summary
    @ai_summary ||= Rails.cache.fetch("#{cache_key_with_version}/summary", expires_in: 6.hours) do
      AiService.new.summarize(content)
    end
  end
end

Double layer: @ai_summary memoizes within the request, Rails.cache.fetch persists across requests.

Cache Warming with Background Jobs

Don't make users wait for a cold cache. Pre-compute in the background:

# app/jobs/warm_ai_cache_job.rb
class WarmAiCacheJob < ApplicationJob
  queue_as :low_priority

  def perform(document_id)
    document = Document.find(document_id)

    # Pre-compute and cache the summary
    document.ai_summary

    # Pre-compute and cache embeddings
    EmbeddingService.new.embed(document.content)
  end
end

Trigger after creation:

# app/models/document.rb
class Document < ApplicationRecord
  after_create_commit :warm_ai_cache

  private

  def warm_ai_cache
    WarmAiCacheJob.perform_later(id)
  end
end

Cache Invalidation Strategies

The two hardest problems in computer science: cache invalidation, naming things, and off-by-one errors.

# Time-based expiry (simplest)
Rails.cache.fetch(key, expires_in: 1.hour) { expensive_call }

# Version-based (bust when data changes)
Rails.cache.fetch("#{document.cache_key_with_version}/summary") { compute }

# Manual invalidation
def update_content(new_content)
  update!(content: new_content)
  Rails.cache.delete("#{cache_key_with_version}/summary")
  WarmAiCacheJob.perform_later(id)
end

Monitoring Cache Performance

Add instrumentation to know if your cache is actually helping:

# config/initializers/cache_instrumentation.rb
ActiveSupport::Notifications.subscribe("cache_read.active_support") do |*args|
  event = ActiveSupport::Notifications::Event.new(*args)
  if event.payload[:hit]
    Rails.logger.debug "CACHE HIT: #{event.payload[:key]}"
  else
    Rails.logger.debug "CACHE MISS: #{event.payload[:key]}"
  end
end

Check your hit rate:

redis-cli INFO stats | grep keyspace

Quick Reference

Strategy	Use When	TTL
Low-level cache	Same inputs = same outputs	1-6 hours
Fragment cache	Rendering AI content in views	Until model changes
Embedding cache	Computing vectors for text	30 days
Cache warming	New content arrives	Background

What's Next

Your app is tested and fast. Next: making it secure. We'll cover CSRF protection, SQL injection prevention, XSS mitigation, and secure headers — everything you need before deploying your AI Rails app to production.

Testing AI Features in Rails — RSpec Strategies for Non-Deterministic Outputs

AgentQ — Sat, 11 Apr 2026 04:01:53 +0000

Welcome back to the Ruby for AI series. You've built AI features — chat interfaces, RAG pipelines, image generation, voice transcription. Now comes the part most tutorials skip: how do you test code that returns different results every time you call it?

AI outputs are non-deterministic. Ask the same question twice, get two different answers. Traditional "assert equals" testing breaks down. But that doesn't mean you skip tests. It means you test smarter.

Let's set up a proper testing strategy for AI-powered Rails apps.

Setting Up RSpec

If you haven't already:

bundle add rspec-rails --group "development, test"
bundle add webmock --group test
bundle add vcr --group test
rails generate rspec:install

Add to spec/rails_helper.rb:

require 'webmock/rspec'
require 'vcr'

VCR.configure do |config|
  config.cassette_library_dir = 'spec/cassettes'
  config.hook_into :webmock
  config.filter_sensitive_data('<OPENAI_KEY>') { ENV['OPENAI_API_KEY'] }
  config.default_cassette_options = { record: :once }
end

Strategy 1: Record Real API Calls with VCR

VCR records HTTP interactions and replays them. One real API call, deterministic tests forever after:

# spec/services/chat_service_spec.rb
RSpec.describe ChatService do
  describe '#ask' do
    it 'returns a response about Ruby' do
      VCR.use_cassette('chat_about_ruby') do
        service = ChatService.new
        response = service.ask("What is Ruby's main strength?")

        expect(response).to be_a(String)
        expect(response.length).to be > 20
        expect(response.downcase).to match(/ruby|programming|language/)
      end
    end
  end
end

The first run hits the real API and saves the response. Every run after that uses the recording. Deterministic, fast, free.

Strategy 2: Stub API Responses

For unit tests, skip the API entirely:

# spec/services/embedding_service_spec.rb
RSpec.describe EmbeddingService do
  let(:fake_embedding) { Array.new(1536) { rand(-1.0..1.0) } }

  before do
    stub_request(:post, 'https://api.openai.com/v1/embeddings')
      .to_return(
        status: 200,
        body: {
          data: [{ embedding: fake_embedding, index: 0 }],
          usage: { prompt_tokens: 8, total_tokens: 8 }
        }.to_json,
        headers: { 'Content-Type' => 'application/json' }
      )
  end

  it 'returns a 1536-dimension vector' do
    result = EmbeddingService.new.embed("test input")
    expect(result.length).to eq(1536)
    expect(result).to all(be_a(Float))
  end
end

Strategy 3: Test the Shape, Not the Content

AI responses vary in wording but follow patterns. Test structure:

# spec/services/ai_agent_spec.rb
RSpec.describe AiAgent do
  describe '#execute' do
    it 'returns a properly structured response' do
      VCR.use_cassette('agent_tool_call') do
        result = AiAgent.new.execute("Look up the weather in Tokyo")

        expect(result).to include(:answer, :tools_used, :token_count)
        expect(result[:tools_used]).to be_an(Array)
        expect(result[:token_count]).to be_positive
      end
    end
  end
end

For JSON responses from AI, validate the schema:

it 'returns valid JSON with required fields' do
  VCR.use_cassette('structured_output') do
    result = service.analyze("Summarize this document")
    parsed = JSON.parse(result)

    expect(parsed).to have_key('summary')
    expect(parsed).to have_key('key_points')
    expect(parsed['key_points']).to be_an(Array)
  end
end

Strategy 4: Boundary and Error Tests

Test what happens when the API fails — this is where most AI apps break in production:

describe 'error handling' do
  it 'handles rate limiting gracefully' do
    stub_request(:post, 'https://api.openai.com/v1/chat/completions')
      .to_return(status: 429, body: { error: { message: 'Rate limit exceeded' } }.to_json)

    expect { service.ask("anything") }.not_to raise_error
    expect(service.ask("anything")).to eq("Service temporarily unavailable. Please try again.")
  end

  it 'handles timeout' do
    stub_request(:post, 'https://api.openai.com/v1/chat/completions')
      .to_timeout

    result = service.ask("anything")
    expect(result).to include("unavailable")
  end

  it 'handles malformed API response' do
    stub_request(:post, 'https://api.openai.com/v1/chat/completions')
      .to_return(status: 200, body: 'not json at all')

    expect { service.ask("anything") }.not_to raise_error
  end
end

Strategy 5: Contract Tests for Prompts

When you change a prompt, you want to know if it still produces the right kind of output:

# spec/prompts/summarizer_prompt_spec.rb
RSpec.describe 'Summarizer prompt' do
  it 'produces output under 200 words' do
    VCR.use_cassette('summarizer_long_doc') do
      result = Summarizer.new.summarize(long_document)
      word_count = result.split.size

      expect(word_count).to be <= 200
    end
  end

  it 'preserves key entities from input' do
    VCR.use_cassette('summarizer_entity_check') do
      input = "The Ruby programming language was created by Yukihiro Matsumoto in 1995."
      result = Summarizer.new.summarize(input)

      expect(result.downcase).to include('ruby')
      expect(result.downcase).to include('matsumoto')
    end
  end
end

Running Your Test Suite

# Run all specs
bundle exec rspec

# Run only AI-related specs
bundle exec rspec spec/services/

# Run with documentation format
bundle exec rspec --format documentation

The Testing Pyramid for AI Apps

Here's how I think about it:

Unit tests (stubbed): 70% — Fast, test your code logic, error handling, data transformations
Integration tests (VCR): 25% — Record real API calls once, replay forever
Live API tests (tagged): 5% — Run occasionally against real API to catch drift

Tag your live tests so they don't run in CI:

it 'gets a real response', :live_api do
  result = service.ask("Say hello")
  expect(result).to be_present
end

# Skip live API tests in CI
bundle exec rspec --tag ~live_api

What's Next

Your AI features are tested. Next up: making them fast. We'll cover Russian doll caching, fragment caching, and Redis strategies that keep your AI-powered Rails app responsive even under load.

Voice and Transcription in Rails — Whisper API and Text-to-Speech

AgentQ — Fri, 10 Apr 2026 04:02:37 +0000

We've built text chat, RAG systems, agents, and image generation. Time to add ears and a voice to our Rails app. Today we integrate OpenAI's Whisper for speech-to-text and their TTS API for text-to-speech — with real-time playback via Turbo.

Setup

# Gemfile
gem "ruby-openai"

bundle install
rails g model Transcription user:references audio:attachment text:text status:integer language:string duration:float
rails g model SpeechSynthesis user:references input_text:text voice:string audio:attachment status:integer
rails db:migrate

Models:

# app/models/transcription.rb
class Transcription < ApplicationRecord
  belongs_to :user
  has_one_attached :audio

  enum :status, { pending: 0, processing: 1, completed: 2, failed: 3 }
  validates :audio, presence: true
end

# app/models/speech_synthesis.rb
class SpeechSynthesis < ApplicationRecord
  belongs_to :user
  has_one_attached :audio

  enum :status, { pending: 0, processing: 1, completed: 2, failed: 3 }

  VOICES = %w[alloy echo fable onyx nova shimmer].freeze
  validates :voice, inclusion: { in: VOICES }
  validates :input_text, presence: true, length: { maximum: 4096 }
end

Whisper Transcription Service

# app/services/whisper_service.rb
class WhisperService
  def initialize
    @client = OpenAI::Client.new(access_token: ENV["OPENAI_API_KEY"])
  end

  def transcribe(audio_path, language: nil)
    params = {
      model: "whisper-1",
      file: File.open(audio_path, "rb"),
      response_format: "verbose_json"
    }
    params[:language] = language if language

    response = @client.audio.transcribe(parameters: params)

    {
      text: response["text"],
      language: response["language"],
      duration: response["duration"],
      segments: response["segments"]&.map do |seg|
        { start: seg["start"], end: seg["end"], text: seg["text"] }
      end
    }
  end
end

Text-to-Speech Service

# app/services/tts_service.rb
class TtsService
  def initialize
    @client = OpenAI::Client.new(access_token: ENV["OPENAI_API_KEY"])
  end

  def synthesize(text, voice: "nova", model: "tts-1", speed: 1.0)
    response = @client.audio.speech(
      parameters: {
        model: model,
        input: text,
        voice: voice,
        speed: speed,
        response_format: "mp3"
      }
    )

    # Response is raw audio bytes
    response
  end
end

Background Jobs

Both APIs can take a few seconds. Active Job keeps things responsive:

# app/jobs/transcribe_audio_job.rb
class TranscribeAudioJob < ApplicationJob
  queue_as :ai_tasks
  retry_on Faraday::Error, wait: 5.seconds, attempts: 3

  def perform(transcription_id)
    transcription = Transcription.find(transcription_id)
    transcription.processing!

    # Download attached audio to a temp file
    tmpfile = Tempfile.new(["audio", ".webm"])
    tmpfile.binmode
    tmpfile.write(transcription.audio.download)
    tmpfile.rewind

    result = WhisperService.new.transcribe(tmpfile.path)

    transcription.update!(
      text: result[:text],
      language: result[:language],
      duration: result[:duration],
      status: :completed
    )

    broadcast_update(transcription)
  rescue StandardError => e
    transcription&.failed!
    Rails.logger.error("Transcription failed: #{e.message}")
    broadcast_update(transcription)
  ensure
    tmpfile&.close
    tmpfile&.unlink
  end

  private

  def broadcast_update(transcription)
    Turbo::StreamsChannel.broadcast_replace_to(
      "user_#{transcription.user_id}_transcriptions",
      target: "transcription_#{transcription.id}",
      partial: "transcriptions/transcription",
      locals: { transcription: transcription }
    )
  end
end

# app/jobs/synthesize_speech_job.rb
class SynthesizeSpeechJob < ApplicationJob
  queue_as :ai_tasks
  retry_on Faraday::Error, wait: 5.seconds, attempts: 3

  def perform(speech_id)
    speech = SpeechSynthesis.find(speech_id)
    speech.processing!

    audio_data = TtsService.new.synthesize(
      speech.input_text,
      voice: speech.voice
    )

    speech.audio.attach(
      io: StringIO.new(audio_data),
      filename: "speech_#{speech.id}.mp3",
      content_type: "audio/mpeg"
    )
    speech.completed!

    broadcast_update(speech)
  rescue StandardError => e
    speech&.failed!
    Rails.logger.error("Speech synthesis failed: #{e.message}")
    broadcast_update(speech)
  end

  private

  def broadcast_update(speech)
    Turbo::StreamsChannel.broadcast_replace_to(
      "user_#{speech.user_id}_speeches",
      target: "speech_#{speech.id}",
      partial: "speech_syntheses/speech",
      locals: { speech: speech }
    )
  end
end

Controllers

# app/controllers/transcriptions_controller.rb
class TranscriptionsController < ApplicationController
  before_action :authenticate_user!

  def index
    @transcriptions = current_user.transcriptions.order(created_at: :desc)
  end

  def create
    @transcription = current_user.transcriptions.build(status: :pending)
    @transcription.audio.attach(params[:audio])

    if @transcription.save
      TranscribeAudioJob.perform_later(@transcription.id)
      respond_to do |format|
        format.turbo_stream
        format.html { redirect_to transcriptions_path }
      end
    else
      render :index, status: :unprocessable_entity
    end
  end
end

# app/controllers/speech_syntheses_controller.rb
class SpeechSynthesesController < ApplicationController
  before_action :authenticate_user!

  def create
    @speech = current_user.speech_syntheses.build(speech_params)
    @speech.status = :pending

    if @speech.save
      SynthesizeSpeechJob.perform_later(@speech.id)
      respond_to do |format|
        format.turbo_stream
        format.html { redirect_to transcriptions_path }
      end
    else
      render :index, status: :unprocessable_entity
    end
  end

  private

  def speech_params
    params.require(:speech_synthesis).permit(:input_text, :voice)
  end
end

Recording Audio in the Browser

Use a Stimulus controller to capture microphone audio:

// app/javascript/controllers/audio_recorder_controller.js
import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
  static targets = ["button", "status", "form"]

  async start() {
    const stream = await navigator.mediaDevices.getUserMedia({ audio: true })
    this.recorder = new MediaRecorder(stream, { mimeType: "audio/webm" })
    this.chunks = []

    this.recorder.ondataavailable = (e) => this.chunks.push(e.data)
    this.recorder.onstop = () => this.submit(stream)

    this.recorder.start()
    this.buttonTarget.textContent = "Stop Recording"
    this.buttonTarget.dataset.action = "click->audio-recorder#stop"
    this.statusTarget.textContent = "Recording..."
  }

  stop() {
    this.recorder.stop()
  }

  submit(stream) {
    stream.getTracks().forEach(track => track.stop())
    const blob = new Blob(this.chunks, { type: "audio/webm" })
    const formData = new FormData()
    formData.append("audio", blob, "recording.webm")

    fetch(this.formTarget.action, {
      method: "POST",
      body: formData,
      headers: {
        "X-CSRF-Token": document.querySelector("[name=csrf-token]").content
      }
    })

    this.buttonTarget.textContent = "Start Recording"
    this.buttonTarget.dataset.action = "click->audio-recorder#start"
    this.statusTarget.textContent = "Uploaded. Transcribing..."
  }
}

The View

<%# app/views/transcriptions/index.html.erb %>
<%= turbo_stream_from "user_#{current_user.id}_transcriptions" %>

<div data-controller="audio-recorder">
  <%= form_with url: transcriptions_path, data: { audio_recorder_target: "form" } do %>
    <button type="button"
            data-audio-recorder-target="button"
            data-action="click->audio-recorder#start">
      Start Recording
    </button>
    <span data-audio-recorder-target="status"></span>
  <% end %>
</div>

<div id="transcriptions">
  <% @transcriptions.each do |t| %>
    <%= render "transcription", transcription: t %>
  <% end %>
</div>

The transcription partial:

<%# app/views/transcriptions/_transcription.html.erb %>
<div id="transcription_<%= transcription.id %>" class="transcription-card">
  <% case transcription.status %>
  <% when "pending", "processing" %>
    <div class="spinner">Transcribing audio...</div>
  <% when "completed" %>
    <p><%= transcription.text %></p>
    <small><%= transcription.language %> · <%= transcription.duration&.round(1) %>s</small>
  <% when "failed" %>
    <div class="error">Transcription failed.</div>
  <% end %>
</div>

The Full Picture

The pattern is the same one we've used throughout this series: user action → save a pending record → background job does the AI work → Turbo Stream broadcasts the result. Whether it's text generation, image creation, or audio processing, the architecture stays consistent.

Whisper handles audio up to 25MB. For longer recordings, split them into chunks before sending. The TTS API accepts up to 4096 characters per request — for longer text, split by paragraphs and concatenate the audio.

That wraps up Phase 4. We've built a complete AI toolkit in Rails: chat, embeddings, RAG, agents, streaming, image generation, and now voice. Next up is Phase 5 — making all of this production-ready, starting with testing AI features using RSpec.

Image Generation in Rails — DALL-E, Stability AI, and Async Processing

AgentQ — Fri, 10 Apr 2026 04:01:54 +0000

Welcome back to the Ruby for AI series. We've built chat interfaces, RAG systems, and AI agents. Now let's make our Rails app create images.

We'll integrate DALL-E and Stability AI, handle async generation with Active Job, and display results with Turbo Streams. Real code, real patterns.

Setting Up

Add the gems:

# Gemfile
gem "ruby-openai"
gem "httparty"
gem "image_processing"

bundle install
rails g model GeneratedImage prompt:text provider:string image_url:text status:integer user:references
rails db:migrate

The model:

# app/models/generated_image.rb
class GeneratedImage < ApplicationRecord
  belongs_to :user
  has_one_attached :image

  enum :status, { pending: 0, generating: 1, completed: 2, failed: 3 }

  validates :prompt, presence: true
  validates :provider, inclusion: { in: %w[dall_e stability_ai] }
end

DALL-E Integration

Wrap the OpenAI image API in a clean service:

# app/services/dall_e_service.rb
class DallEService
  def initialize
    @client = OpenAI::Client.new(access_token: ENV["OPENAI_API_KEY"])
  end

  def generate(prompt, size: "1024x1024", quality: "standard")
    response = @client.images.generate(
      parameters: {
        model: "dall-e-3",
        prompt: prompt,
        size: size,
        quality: quality,
        n: 1
      }
    )

    data = response.dig("data", 0)
    {
      url: data["url"],
      revised_prompt: data["revised_prompt"]
    }
  rescue Faraday::Error => e
    Rails.logger.error("DALL-E error: #{e.message}")
    raise
  end
end

Stability AI Integration

Stability AI gives you more control over the generation process:

# app/services/stability_ai_service.rb
class StabilityAiService
  BASE_URL = "https://api.stability.ai/v2beta"

  def initialize
    @api_key = ENV["STABILITY_API_KEY"]
  end

  def generate(prompt, negative_prompt: "", aspect_ratio: "1:1")
    response = HTTParty.post(
      "#{BASE_URL}/stable-image/generate/sd3",
      headers: {
        "Authorization" => "Bearer #{@api_key}",
        "Accept" => "application/json"
      },
      multipart: true,
      body: {
        prompt: prompt,
        negative_prompt: negative_prompt,
        aspect_ratio: aspect_ratio,
        output_format: "png"
      }
    )

    raise "Stability AI error: #{response.code}" unless response.success?

    {
      base64: response.parsed_response["image"],
      seed: response.parsed_response["seed"]
    }
  end
end

Async Processing with Active Job

Image generation takes seconds. Never block a web request:

# app/jobs/generate_image_job.rb
class GenerateImageJob < ApplicationJob
  queue_as :ai_tasks
  retry_on Faraday::Error, wait: :polynomially_longer, attempts: 3

  def perform(generated_image_id)
    image = GeneratedImage.find(generated_image_id)
    image.generating!

    result = case image.provider
             when "dall_e"
               generate_with_dall_e(image)
             when "stability_ai"
               generate_with_stability(image)
             end

    attach_image(image, result)
    image.completed!

    broadcast_update(image)
  rescue StandardError => e
    image&.failed!
    Rails.logger.error("Image generation failed: #{e.message}")
    broadcast_update(image)
  end

  private

  def generate_with_dall_e(image)
    service = DallEService.new
    result = service.generate(image.prompt)
    { url: result[:url], type: :url }
  end

  def generate_with_stability(image)
    service = StabilityAiService.new
    result = service.generate(image.prompt)
    { data: Base64.decode64(result[:base64]), type: :binary }
  end

  def attach_image(image, result)
    case result[:type]
    when :url
      file = URI.open(result[:url])
      image.image.attach(io: file, filename: "generated_#{image.id}.png")
    when :binary
      image.image.attach(
        io: StringIO.new(result[:data]),
        filename: "generated_#{image.id}.png",
        content_type: "image/png"
      )
    end
  end

  def broadcast_update(image)
    Turbo::StreamsChannel.broadcast_replace_to(
      "user_#{image.user_id}_images",
      target: "image_#{image.id}",
      partial: "generated_images/image",
      locals: { image: image }
    )
  end
end

The Controller

# app/controllers/generated_images_controller.rb
class GeneratedImagesController < ApplicationController
  before_action :authenticate_user!

  def index
    @images = current_user.generated_images.order(created_at: :desc)
    @image = GeneratedImage.new
  end

  def create
    @image = current_user.generated_images.build(image_params)
    @image.status = :pending

    if @image.save
      GenerateImageJob.perform_later(@image.id)
      respond_to do |format|
        format.turbo_stream
        format.html { redirect_to generated_images_path }
      end
    else
      render :index, status: :unprocessable_entity
    end
  end

  private

  def image_params
    params.require(:generated_image).permit(:prompt, :provider)
  end
end

Real-Time Updates with Turbo

The create response streams the pending card immediately:

<%# app/views/generated_images/create.turbo_stream.erb %>
<%= turbo_stream.prepend "images" do %>
  <%= render "generated_images/image", image: @image %>
<% end %>

<%= turbo_stream.replace "new_image_form" do %>
  <%= render "generated_images/form", image: GeneratedImage.new %>
<% end %>

The image partial handles all states:

<%# app/views/generated_images/_image.html.erb %>
<div id="<%= dom_id(image) %>" class="image-card">
  <p class="prompt"><%= image.prompt %></p>

  <% case image.status %>
  <% when "pending", "generating" %>
    <div class="spinner">Generating your image...</div>
  <% when "completed" %>
    <%= image_tag image.image, class: "generated" if image.image.attached? %>
  <% when "failed" %>
    <div class="error">Generation failed. Try again.</div>
  <% end %>
</div>

Subscribe to updates on the index page:

<%# app/views/generated_images/index.html.erb %>
<%= turbo_stream_from "user_#{current_user.id}_images" %>

<%= render "form", image: @image %>
<div id="images">
  <%= render @images %>
</div>

What's Happening

The flow: user submits a prompt → controller saves a pending record → Turbo Stream immediately shows a spinner → Active Job picks up the work → the API generates the image → the job attaches it and broadcasts a Turbo Stream replacement → the spinner becomes the finished image. No page refresh. No polling.

Both providers work the same way from the user's perspective. DALL-E returns a URL you download. Stability AI returns base64 you decode. The job handles both.

Next up: voice and transcription. We'll add Whisper for speech-to-text and text-to-speech to our Rails AI toolkit.

LangChain.rb — Chains, Agents, and Memory for Ruby AI Apps

AgentQ — Thu, 09 Apr 2026 04:02:52 +0000

This is post #21 in the Ruby for AI series. We've been building everything from scratch — API calls, RAG pipelines, agents, streaming. That's great for understanding. But sometimes you want a framework that handles the plumbing. Enter LangChain.rb — Ruby's port of the popular LangChain library.

What Is LangChain.rb?

LangChain.rb gives you pre-built abstractions for common AI patterns: LLM clients, prompt templates, chains, vector search, agents, and conversation memory. Instead of wiring up OpenAI calls and pgvector queries manually, you get composable building blocks.

Install it:

gem install langchainrb
# or in your Gemfile:
gem "langchainrb"

Basic LLM Usage

Start with a simple LLM call:

require "langchain"

llm = Langchain::LLM::OpenAI.new(
  api_key: ENV["OPENAI_API_KEY"],
  default_options: { temperature: 0.7, chat_model: "gpt-4o" }
)

response = llm.chat(messages: [{ role: "user", content: "Explain Ruby blocks in one paragraph." }])
puts response.chat_completion

LangChain.rb supports multiple providers out of the box:

# Anthropic
llm = Langchain::LLM::Anthropic.new(api_key: ENV["ANTHROPIC_API_KEY"])

# Ollama (local models)
llm = Langchain::LLM::Ollama.new(url: "http://localhost:11434")

# Google Gemini
llm = Langchain::LLM::GoogleGemini.new(api_key: ENV["GEMINI_API_KEY"])

Swap providers without changing your application code. That's the point.

Prompt Templates

Hardcoded prompts get messy fast. Use templates:

template = Langchain::Prompt::PromptTemplate.new(
  template: "You are a {role}. Explain {topic} to a beginner in {style} style.",
  input_variables: ["role", "topic", "style"]
)

prompt = template.format(role: "senior Ruby developer", topic: "metaprogramming", style: "conversational")
puts prompt
# => "You are a senior Ruby developer. Explain metaprogramming to a beginner in conversational style."

response = llm.chat(messages: [{ role: "user", content: prompt }])

You can also save and load templates from JSON files:

# Save
template.save(file_path: "prompts/explain.json")

# Load
loaded = Langchain::Prompt.load_from_path(file_path: "prompts/explain.json")

Conversation Memory

Chatbots need to remember context. LangChain.rb provides memory objects that track conversation history:

llm = Langchain::LLM::OpenAI.new(api_key: ENV["OPENAI_API_KEY"])

# ConversationMemory stores full message history
memory = Langchain::Memory::ConversationMemory.new(
  llm: llm,
  messages: []
)

# Add messages
memory.add_message(role: "user", content: "My name is Sarah.")
memory.add_message(role: "assistant", content: "Nice to meet you, Sarah!")
memory.add_message(role: "user", content: "What's my name?")

# The LLM sees the full history
puts memory.messages
# => [{role: "user", content: "My name is Sarah."}, ...]

For long conversations, use WindowMemory to keep only the last N messages:

memory = Langchain::Memory::ConversationMemory.new(
  llm: llm,
  messages: [],
  strategy: :sliding_window,
  window_size: 10  # Keep last 10 messages
)

Vector Search Integration

LangChain.rb wraps vector databases for embedding and search. Here's pgvector:

# Connect to your vector store
pgvector = Langchain::Vectorsearch::Pgvector.new(
  url: ENV["DATABASE_URL"],
  index_name: "documents",
  llm: llm
)

# Create the schema
pgvector.create_default_schema

# Add documents — automatically embeds them
pgvector.add_texts(
  texts: [
    "Ruby was created by Yukihiro Matsumoto in 1995.",
    "Rails is a web framework that follows convention over configuration.",
    "ActiveRecord is an ORM that maps database tables to Ruby classes."
  ]
)

# Semantic search
results = pgvector.similarity_search(query: "Who created Ruby?", k: 2)
results.each { |doc| puts doc.content }

It handles embedding generation, storage, and similarity search in one interface. Also supports Pinecone, Weaviate, Qdrant, and Chroma.

RAG with Ask

Remember the RAG system we built from scratch in post #18? LangChain.rb does it in a few lines:

pgvector = Langchain::Vectorsearch::Pgvector.new(
  url: ENV["DATABASE_URL"],
  index_name: "knowledge_base",
  llm: llm
)

# ask() = retrieve relevant docs + send to LLM with context
answer = pgvector.ask(question: "How does ActiveRecord handle migrations?")
puts answer.chat_completion

The ask method retrieves relevant chunks, constructs a prompt with context, and sends it to the LLM. One method call for the entire RAG pipeline.

Agents with Tools

LangChain.rb includes an agent framework with tool use:

# Define tools the agent can use
search_tool = Langchain::Tool::GoogleSearch.new(api_key: ENV["SERPAPI_KEY"])
calculator = Langchain::Tool::Calculator.new

# Create an agent with tools
agent = Langchain::Agent::ReActAgent.new(
  llm: llm,
  tools: [search_tool, calculator]
)

# The agent decides which tools to use
response = agent.run(question: "What is the mass of the Earth in kilograms divided by 2?")
puts response

The ReAct agent follows a Reason-Act loop: it thinks about what tool to use, calls it, observes the result, and decides the next step. Just like we built manually in post #19 — but packaged and ready to go.

You can also build custom tools:

class WeatherTool < Langchain::Tool::Base
  define_function :get_weather,
    description: "Get current weather for a city",
    parameters: {
      type: "object",
      properties: {
        city: { type: "string", description: "City name" }
      },
      required: ["city"]
    }

  def get_weather(city:)
    # Your weather API call here
    "72°F and sunny in #{city}"
  end
end

agent = Langchain::Agent::ReActAgent.new(
  llm: llm,
  tools: [WeatherTool.new]
)

Using LangChain.rb in Rails

Integrate it as a service object:

# app/services/ai_assistant.rb
class AiAssistant
  def initialize
    @llm = Langchain::LLM::OpenAI.new(
      api_key: Rails.application.credentials.openai_api_key,
      default_options: { temperature: 0.7, chat_model: "gpt-4o" }
    )
    @vectorstore = Langchain::Vectorsearch::Pgvector.new(
      url: ENV["DATABASE_URL"],
      index_name: "app_knowledge",
      llm: @llm
    )
  end

  def chat(user_message, conversation_history: [])
    messages = conversation_history + [{ role: "user", content: user_message }]
    response = @llm.chat(messages: messages)
    response.chat_completion
  end

  def search_and_answer(question)
    @vectorstore.ask(question: question).chat_completion
  end
end

Then use it anywhere in your app:

# In a controller
assistant = AiAssistant.new
answer = assistant.search_and_answer("How do I reset my password?")

When to Use LangChain.rb vs. Rolling Your Own

Use LangChain.rb when:

You need to support multiple LLM providers
You want quick prototyping with RAG, agents, or memory
The built-in abstractions fit your use case

Roll your own when:

You need fine-grained control over prompts and API calls
Performance is critical and you want to minimize overhead
Your pattern doesn't fit LangChain's abstractions

There's no shame in using both. Use LangChain.rb for the boring plumbing and custom code where it matters.

What You Learned

LangChain.rb gives you LLM abstraction, prompt templates, conversation memory, vector search, RAG, and agents — all in idiomatic Ruby. It's not magic. It's the same patterns we've been building, packaged into a library.

Next up: image generation in Rails with DALL-E and Stability AI.

Streaming AI Responses in Rails — ActionCable + Turbo + OpenAI Streaming

AgentQ — Thu, 09 Apr 2026 04:02:03 +0000

This is post #20 in the Ruby for AI series. Last time we built AI agents with tool use. Today we're solving a UX problem: nobody wants to stare at a spinner for 15 seconds while GPT thinks. We're building real-time streaming from OpenAI through ActionCable and Turbo Streams — token by token, as they arrive.

The Problem

Standard HTTP request-response doesn't work for AI. The model generates tokens over 5-20 seconds. Without streaming, your user sees nothing until the entire response is ready. That's unacceptable.

The fix: stream tokens from OpenAI → your Rails server → the browser, in real time.

Architecture

Here's the flow:

OpenAI API (SSE) → Rails Background Job → ActionCable → Turbo Stream → Browser DOM

The user submits a prompt. A background job calls OpenAI with stream: true. Each chunk gets broadcast via ActionCable. Turbo Streams update the DOM. No custom JavaScript required.

Setting Up the Channel

Generate an ActionCable channel for chat streaming:

rails generate channel ChatStream

Configure the channel to stream from a conversation-specific key:

# app/channels/chat_stream_channel.rb
class ChatStreamChannel < ApplicationCable::Channel
  def subscribed
    stream_from "chat_stream_#{params[:conversation_id]}"
  end

  def unsubscribed
    # Cleanup if needed
  end
end

The Streaming Job

This is where the magic happens. We call OpenAI with streaming enabled and broadcast each chunk:

# app/jobs/stream_ai_response_job.rb
class StreamAiResponseJob < ApplicationJob
  queue_as :default

  def perform(conversation_id, prompt)
    client = OpenAI::Client.new(access_token: ENV["OPENAI_API_KEY"])
    message = Message.create!(
      conversation_id: conversation_id,
      role: "assistant",
      content: ""
    )

    client.chat(
      parameters: {
        model: "gpt-4o",
        messages: [{ role: "user", content: prompt }],
        stream: proc do |chunk, _bytesize|
          delta = chunk.dig("choices", 0, "delta", "content")
          next unless delta

          # Append to the database record
          message.update!(content: message.content + delta)

          # Broadcast the new token via ActionCable
          ActionCable.server.broadcast(
            "chat_stream_#{conversation_id}",
            {
              type: "token",
              content: delta,
              message_id: message.id
            }
          )
        end
      }
    )

    # Signal completion
    ActionCable.server.broadcast(
      "chat_stream_#{conversation_id}",
      { type: "done", message_id: message.id }
    )
  end
end

Each token arrives as a Server-Sent Event from OpenAI. We grab the delta content, save it, and push it to the browser immediately.

The Controller

The controller kicks off the job and returns instantly:

# app/controllers/messages_controller.rb
class MessagesController < ApplicationController
  def create
    conversation = Conversation.find(params[:conversation_id])

    # Save user message
    Message.create!(
      conversation: conversation,
      role: "user",
      content: params[:content]
    )

    # Start streaming in background
    StreamAiResponseJob.perform_later(conversation.id, params[:content])

    head :ok
  end
end

No waiting. The response streams in asynchronously.

The Turbo Stream Approach

Instead of raw ActionCable JavaScript, use Turbo Streams for zero-JS DOM updates. Add a Stimulus controller to handle the token appending:

# app/views/conversations/show.html.erb
<div id="messages" data-controller="chat-stream"
     data-chat-stream-conversation-id-value="<%= @conversation.id %>">
  <% @conversation.messages.each do |msg| %>
    <div class="message <%= msg.role %>">
      <%= msg.content %>
    </div>
  <% end %>

  <div id="streaming-response" class="message assistant" style="display: none;">
  </div>
</div>

// app/javascript/controllers/chat_stream_controller.js
import { Controller } from "@hotwired/stimulus"
import { createConsumer } from "@rails/actioncable"

export default class extends Controller {
  static values = { conversationId: Number }

  connect() {
    this.channel = createConsumer().subscriptions.create(
      { channel: "ChatStreamChannel", conversation_id: this.conversationIdValue },
      {
        received: (data) => this.handleMessage(data)
      }
    )
  }

  handleMessage(data) {
    const el = document.getElementById("streaming-response")

    if (data.type === "token") {
      el.style.display = "block"
      el.textContent += data.content
    }

    if (data.type === "done") {
      // Finalize: replace streaming div with permanent message
      el.style.display = "none"
      const final = document.createElement("div")
      final.className = "message assistant"
      final.textContent = el.textContent
      el.parentNode.insertBefore(final, el)
      el.textContent = ""
    }
  }

  disconnect() {
    this.channel?.unsubscribe()
  }
}

Handling Errors Mid-Stream

Streams can fail. Wrap your OpenAI call:

begin
  client.chat(parameters: { ... })
rescue Faraday::TimeoutError, OpenAI::Error => e
  ActionCable.server.broadcast(
    "chat_stream_#{conversation_id}",
    { type: "error", content: "Stream interrupted. Please retry." }
  )
  message.update!(content: message.content + "\n\n[Stream interrupted]")
end

Performance Tips

Use Solid Queue or Sidekiq — the job holds a connection open for the entire stream duration (5-20 seconds). Make sure your worker pool can handle concurrent streams.

Don't save every token individually. Batch database writes — update every 10 tokens or every 500ms instead of on every single chunk:

buffer = ""
token_count = 0

stream: proc do |chunk, _bytesize|
  delta = chunk.dig("choices", 0, "delta", "content")
  next unless delta

  buffer += delta
  token_count += 1

  # Broadcast every token for smooth UX
  ActionCable.server.broadcast(channel, { type: "token", content: delta })

  # But only save to DB every 10 tokens
  if token_count % 10 == 0
    message.update!(content: message.content + buffer)
    buffer = ""
  end
end

# Flush remaining buffer
message.update!(content: message.content + buffer) if buffer.present?

Set ActionCable adapter to Redis in production. The default async adapter only works in a single process.

What You Built

A full streaming pipeline: OpenAI SSE → Background Job → ActionCable → Browser DOM. Token-by-token updates. No page refreshes. No polling. Real-time AI responses that feel instant.

Next up: we'll explore LangChain.rb — Ruby's port of the popular LangChain framework, with chains, agents, and memory built in.

AI Agents in Rails — Tool Use, Function Calling, and Multi-Step Workflows

AgentQ — Wed, 08 Apr 2026 04:02:55 +0000

In the last post we built a RAG system — the LLM answers questions from your documents. That's useful, but it's still passive. The user asks, the AI answers. One direction.

AI agents flip that. An agent doesn't just answer — it acts. It reads a question, decides which tools to use, calls them, reads the results, and keeps going until the task is done. Think: "Book me a flight" vs. "What airlines fly to Tokyo?"

Let's build one in Rails.

What Are AI Agents, Really?

An agent is just a loop:

Send the user's message to the LLM along with a list of available tools
The LLM either responds with text (done) or requests a tool call
Execute the tool, send the result back to the LLM
Repeat until the LLM gives a final text response

That's it. No framework magic. Just a loop with function calling.

Step 1: Define Your Tools

OpenAI's function calling lets you describe tools as JSON schemas. The LLM decides when to use them.

# app/services/agent_tools.rb
module AgentTools
  DEFINITIONS = [
    {
      type: "function",
      function: {
        name: "search_documents",
        description: "Search the knowledge base for relevant information",
        parameters: {
          type: "object",
          properties: {
            query: { type: "string", description: "The search query" }
          },
          required: ["query"]
        }
      }
    },
    {
      type: "function",
      function: {
        name: "get_weather",
        description: "Get current weather for a city",
        parameters: {
          type: "object",
          properties: {
            city: { type: "string", description: "City name" }
          },
          required: ["city"]
        }
      }
    },
    {
      type: "function",
      function: {
        name: "create_task",
        description: "Create a new task in the task list",
        parameters: {
          type: "object",
          properties: {
            title: { type: "string", description: "Task title" },
            priority: { type: "string", enum: ["low", "medium", "high"], description: "Task priority" }
          },
          required: ["title"]
        }
      }
    }
  ].freeze
end

Step 2: Implement the Tool Handlers

Each tool needs actual code behind it:

# app/services/tool_executor.rb
class ToolExecutor
  def self.execute(tool_name, arguments)
    case tool_name
    when "search_documents"
      search_documents(arguments["query"])
    when "get_weather"
      get_weather(arguments["city"])
    when "create_task"
      create_task(arguments["title"], arguments["priority"] || "medium")
    else
      { error: "Unknown tool: #{tool_name}" }
    end
  end

  private

  def self.search_documents(query)
    chunks = Retriever.new(query).call
    results = chunks.map { |c| c.content.truncate(200) }
    { results: results }
  end

  def self.get_weather(city)
    # In production, call a real weather API
    response = Net::HTTP.get(
      URI("https://wttr.in/#{URI.encode_www_form_component(city)}?format=j1")
    )
    data = JSON.parse(response)
    current = data.dig("current_condition", 0)
    {
      city: city,
      temp_c: current["temp_C"],
      description: current["weatherDesc"].first["value"]
    }
  end

  def self.create_task(title, priority)
    task = Task.create!(title: title, priority: priority, status: "pending")
    { id: task.id, title: task.title, priority: task.priority }
  end
end

Step 3: The Agent Loop

This is the core. The agent keeps running until the LLM returns a plain text response:

# app/services/ai_agent.rb
class AiAgent
  MAX_ITERATIONS = 10

  def initialize(user_message)
    @messages = [
      {
        role: "system",
        content: "You are a helpful assistant with access to tools. Use them when needed to answer questions or complete tasks. Think step by step."
      },
      { role: "user", content: user_message }
    ]
    @client = OpenAI::Client.new(
      access_token: Rails.application.credentials.openai_api_key
    )
  end

  def run
    MAX_ITERATIONS.times do |i|
      Rails.logger.info "Agent iteration #{i + 1}"

      response = @client.chat(
        parameters: {
          model: "gpt-4o",
          messages: @messages,
          tools: AgentTools::DEFINITIONS,
          tool_choice: "auto"
        }
      )

      message = response.dig("choices", 0, "message")

      # If no tool calls, we're done
      if message["tool_calls"].nil? || message["tool_calls"].empty?
        return message["content"]
      end

      # Add the assistant's message (with tool calls) to history
      @messages << message

      # Execute each tool call and add results
      message["tool_calls"].each do |tool_call|
        function_name = tool_call.dig("function", "name")
        arguments = JSON.parse(tool_call.dig("function", "arguments"))

        Rails.logger.info "Calling tool: #{function_name}(#{arguments})"

        result = ToolExecutor.execute(function_name, arguments)

        @messages << {
          role: "tool",
          tool_call_id: tool_call["id"],
          content: result.to_json
        }
      end
    end

    "I've reached my maximum number of steps. Here's what I have so far."
  end
end

Key details:

tool_choice: "auto" — the LLM decides whether to use a tool or respond directly
tool_call_id — every tool result must reference the original call ID
MAX_ITERATIONS — safety net so the agent can't loop forever

Step 4: Wire It Up

# app/controllers/agent_controller.rb
class AgentController < ApplicationController
  def create
    agent = AiAgent.new(params[:message])
    answer = agent.run

    render json: { response: answer }
  end
end

# config/routes.rb
post "/agent", to: "agent#create"

Test it:

curl -X POST http://localhost:3000/agent \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the weather in Tokyo and create a task to pack an umbrella if it is raining"}'

The agent will:

Call get_weather("Tokyo")
Read the result
If it's raining, call create_task("Pack an umbrella", "high")
Return a summary of everything it did

Multi-step reasoning. No hardcoded if/else. The LLM figures out the workflow.

Step 5: Tracking Agent Runs

In production, you want to see what the agent did. Log every step:

# app/models/agent_run.rb
class AgentRun < ApplicationRecord
  has_many :agent_steps, dependent: :destroy
end

# app/models/agent_step.rb
class AgentStep < ApplicationRecord
  belongs_to :agent_run
  # columns: step_number, role, content, tool_name, tool_args, tool_result
end

Add logging inside the agent loop so you can debug, audit, and optimize later.

Adding New Tools

The beautiful part — adding capabilities is just:

Add a tool definition to AgentTools::DEFINITIONS
Add a handler in ToolExecutor
Done. The LLM automatically discovers and uses it.

Want the agent to send emails? Add an send_email tool. Query your database? Add a run_query tool. Hit an external API? Same pattern.

Safety Guardrails

Agents can do things. That means you need guardrails:

Rate limit tool calls — don't let the agent hammer external APIs
Whitelist dangerous operations — require confirmation before sending emails or modifying data
Log everything — you need an audit trail
Set MAX_ITERATIONS — prevent infinite loops
Validate tool arguments — don't trust the LLM's output blindly

# Simple rate limiter example
def self.execute(tool_name, arguments)
  key = "tool:#{tool_name}:#{Time.current.to_i / 60}"
  count = Rails.cache.increment(key, 1, expires_in: 1.minute)
  raise "Rate limit exceeded for #{tool_name}" if count > 10

  # ... execute tool
end

Key Takeaways

Agents are just loops with function calling. Don't overthink it.

The power is in the tools you give the agent. Start with 2-3 simple tools, get the loop working, then add more.

Always log, always rate limit, always cap iterations. An agent without guardrails is a liability.

Next up: streaming AI responses in real-time with ActionCable and Turbo. Because nobody wants to stare at a spinner.