AI Agents in Rails — Tool Use, Function Calling, and Multi-Step Workflows

In the last post we built a RAG system — the LLM answers questions from your documents. That's useful, but it's still passive. The user asks, the AI answers. One direction.

AI agents flip that. An agent doesn't just answer — it acts. It reads a question, decides which tools to use, calls them, reads the results, and keeps going until the task is done. Think: "Book me a flight" vs. "What airlines fly to Tokyo?"

Let's build one in Rails.

What Are AI Agents, Really?

An agent is just a loop:

1. Send the user's message to the LLM along with a list of available tools
2. The LLM either responds with text (done) or requests a tool call
3. Execute the tool, send the result back to the LLM
4. Repeat until the LLM gives a final text response

That's it. No framework magic. Just a loop with function calling.

Step 1: Define Your Tools

OpenAI's function calling lets you describe tools as JSON schemas. The LLM decides when to use them.

# app/services/agent_tools.rb
module AgentTools
  DEFINITIONS = [
    {
      type: "function",
      function: {
        name: "search_documents",
        description: "Search the knowledge base for relevant information",
        parameters: {
          type: "object",
          properties: {
            query: { type: "string", description: "The search query" }
          },
          required: ["query"]
        }
      }
    },
    {
      type: "function",
      function: {
        name: "get_weather",
        description: "Get current weather for a city",
        parameters: {
          type: "object",
          properties: {
            city: { type: "string", description: "City name" }
          },
          required: ["city"]
        }
      }
    },
    {
      type: "function",
      function: {
        name: "create_task",
        description: "Create a new task in the task list",
        parameters: {
          type: "object",
          properties: {
            title: { type: "string", description: "Task title" },
            priority: { type: "string", enum: ["low", "medium", "high"], description: "Task priority" }
          },
          required: ["title"]
        }
      }
    }
  ].freeze
end

Step 2: Implement the Tool Handlers

Each tool needs actual code behind it:

# app/services/tool_executor.rb
class ToolExecutor
  def self.execute(tool_name, arguments)
    case tool_name
    when "search_documents"
      search_documents(arguments["query"])
    when "get_weather"
      get_weather(arguments["city"])
    when "create_task"
      create_task(arguments["title"], arguments["priority"] || "medium")
    else
      { error: "Unknown tool: #{tool_name}" }
    end
  end

  private

  def self.search_documents(query)
    chunks = Retriever.new(query).call
    results = chunks.map { |c| c.content.truncate(200) }
    { results: results }
  end

  def self.get_weather(city)
    # In production, call a real weather API
    response = Net::HTTP.get(
      URI("https://wttr.in/#{URI.encode_www_form_component(city)}?format=j1")
    )
    data = JSON.parse(response)
    current = data.dig("current_condition", 0)
    {
      city: city,
      temp_c: current["temp_C"],
      description: current["weatherDesc"].first["value"]
    }
  end

  def self.create_task(title, priority)
    task = Task.create!(title: title, priority: priority, status: "pending")
    { id: task.id, title: task.title, priority: task.priority }
  end
end

Step 3: The Agent Loop

This is the core. The agent keeps running until the LLM returns a plain text response:

# app/services/ai_agent.rb
class AiAgent
  MAX_ITERATIONS = 10

  def initialize(user_message)
    @messages = [
      {
        role: "system",
        content: "You are a helpful assistant with access to tools. Use them when needed to answer questions or complete tasks. Think step by step."
      },
      { role: "user", content: user_message }
    ]
    @client = OpenAI::Client.new(
      access_token: Rails.application.credentials.openai_api_key
    )
  end

  def run
    MAX_ITERATIONS.times do |i|
      Rails.logger.info "Agent iteration #{i + 1}"

      response = @client.chat(
        parameters: {
          model: "gpt-4o",
          messages: @messages,
          tools: AgentTools::DEFINITIONS,
          tool_choice: "auto"
        }
      )

      message = response.dig("choices", 0, "message")

      # If no tool calls, we're done
      if message["tool_calls"].nil? || message["tool_calls"].empty?
        return message["content"]
      end

      # Add the assistant's message (with tool calls) to history
      @messages << message

      # Execute each tool call and add results
      message["tool_calls"].each do |tool_call|
        function_name = tool_call.dig("function", "name")
        arguments = JSON.parse(tool_call.dig("function", "arguments"))

        Rails.logger.info "Calling tool: #{function_name}(#{arguments})"

        result = ToolExecutor.execute(function_name, arguments)

        @messages << {
          role: "tool",
          tool_call_id: tool_call["id"],
          content: result.to_json
        }
      end
    end

    "I've reached my maximum number of steps. Here's what I have so far."
  end
end

Key details:

• tool_choice: "auto" — the LLM decides whether to use a tool or respond directly
• tool_call_id — every tool result must reference the original call ID
• MAX_ITERATIONS — safety net so the agent can't loop forever

Step 4: Wire It Up

# app/controllers/agent_controller.rb
class AgentController < ApplicationController
  def create
    agent = AiAgent.new(params[:message])
    answer = agent.run

    render json: { response: answer }
  end
end

# config/routes.rb
post "/agent", to: "agent#create"

Test it:

curl -X POST http://localhost:3000/agent \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the weather in Tokyo and create a task to pack an umbrella if it is raining"}'

The agent will:

1. Call get_weather("Tokyo")
2. Read the result
3. If it's raining, call create_task("Pack an umbrella", "high")
4. Return a summary of everything it did

Multi-step reasoning. No hardcoded if/else. The LLM figures out the workflow.

Step 5: Tracking Agent Runs

In production, you want to see what the agent did. Log every step:

# app/models/agent_run.rb
class AgentRun < ApplicationRecord
  has_many :agent_steps, dependent: :destroy
end

# app/models/agent_step.rb
class AgentStep < ApplicationRecord
  belongs_to :agent_run
  # columns: step_number, role, content, tool_name, tool_args, tool_result
end

Add logging inside the agent loop so you can debug, audit, and optimize later.

Adding New Tools

The beautiful part — adding capabilities is just:

1. Add a tool definition to AgentTools::DEFINITIONS
2. Add a handler in ToolExecutor
3. Done. The LLM automatically discovers and uses it.

Want the agent to send emails? Add an send_email tool. Query your database? Add a run_query tool. Hit an external API? Same pattern.

Safety Guardrails

Agents can do things. That means you need guardrails:

• Rate limit tool calls — don't let the agent hammer external APIs
• Whitelist dangerous operations — require confirmation before sending emails or modifying data
• Log everything — you need an audit trail
• Set MAX_ITERATIONS — prevent infinite loops
• Validate tool arguments — don't trust the LLM's output blindly

# Simple rate limiter example
def self.execute(tool_name, arguments)
  key = "tool:#{tool_name}:#{Time.current.to_i / 60}"
  count = Rails.cache.increment(key, 1, expires_in: 1.minute)
  raise "Rate limit exceeded for #{tool_name}" if count > 10

  # ... execute tool
end

Key Takeaways

Agents are just loops with function calling. Don't overthink it.

The power is in the tools you give the agent. Start with 2-3 simple tools, get the loop working, then add more.

Always log, always rate limit, always cap iterations. An agent without guardrails is a liability.

Next up: streaming AI responses in real-time with ActionCable and Turbo. Because nobody wants to stare at a spinner.