DEV Community: Mario Alberto Chávez

Rails MCP Server: Context-Efficient Tool Architecture

Mario Alberto Chávez — Wed, 10 Dec 2025 06:00:00 +0000

Learn how Rails MCP Server’s new architecture reduces context consumption through progressive tool discovery, Rails introspection, and Prism static analysis.

The Context Budget Problem
Progressive Tool Discovery
Rails Introspection: Beyond Regex Parsing
Prism Static Analysis
Detail Levels and Filtering
Sandboxed Ruby Execution
Improved Discovery UX
Interactive Configuration Tool
How I Use the New Architecture
Upgrading to the New Version
Looking Forward

The Context Budget Problem

Last week I came across an Anthropic blog post that fundamentally changed how I think about MCP server design. The insight was simple but profound: every tool you register with an MCP server gets sent to Claude at the start of each session.

With the Rails MCP Server’s previous 12 tools, that meant roughly 2,400 tokens consumed before asking a single question. On large Rails codebases where I’m already sharing multiple files and documentation, this context overhead matters. I’ve hit conversation limits mid-feature more times than I’d like to admit.

The Anthropic post suggested a different approach: progressive disclosure. Instead of loading all tool definitions upfront, let the AI discover them when relevant. This resonated with how I already work—I don’t need Claude to know about route analysis when I’m focused on model validations.

Progressive Tool Discovery

The most significant change in this release is the reduction from 12 registered tools to just 4. The other tools didn’t disappear—they became internal analyzers that Claude discovers on-demand.

The four registered tools:

Tool	Purpose
`switch_project`	Select which Rails project to analyze
`search_tools`	Discover available analyzers by keyword or category
`execute_tool`	Run any analyzer by name
`execute_ruby`	Sandboxed Ruby execution for complex queries

The previous tools like analyze_models, get_routes, and get_schema are now internal analyzers. Claude finds them through search_tools and invokes them through execute_tool. This pattern reduces the initial context footprint by roughly 67%.

When I start a session now, Claude only knows about these four capabilities. If I ask about database structure, Claude can search for relevant tools:


search\_tools(query: "database schema")

This returns information about get_schema without having loaded all nine analyzers upfront. Claude then invokes it:


execute\_tool(tool\_name: "get\_schema", params: { table\_name: "users" })

The workflow feels natural—Claude discovers what it needs when it needs it.

Rails Introspection: Beyond Regex Parsing

While refactoring the tool architecture, I realized something uncomfortable: the previous implementation relied heavily on regex parsing. Patterns like has_many\s+:(\w+) worked most of the time, but missed edge cases—associations with options, multi-line declarations, dynamic associations.

This release replaces regex parsing with proper Rails introspection. When you analyze a model now, the server actually asks Rails about it:

Model.reflect_on_all_associations # All associations with options
Model.validators # All validations with conditions
Model.defined_enums # Enum definitions and values
Model.columns_hash # Column details from the database

For routes, instead of parsing the text output of rails routes, the server accesses route objects directly:

Rails.application.routes.routes.map do |route|
  {
    verb: route.verb,
    path: route.path.spec.to_s,
    controller: route.defaults[:controller],
    action: route.defaults[:action],
    constraints: route.constraints
  }
end

For controllers, I now use action_methods instead of scanning for def statements, and _process_action_callbacks to find before/after actions accurately.

The difference in accuracy is significant. Associations with through:, class_name:, or foreign_key: options are now captured correctly. Validations show their conditions. Callbacks include their only: and except: filters.

Prism Static Analysis

Rails introspection tells you what exists at runtime, but sometimes you need to understand the code structure itself. For this, I added Prism static analysis.

Prism is Ruby’s new parser that ships with Ruby 3.3+. It provides a clean AST that I can traverse to extract:

Callbacks and their method references
Scope definitions
Included concerns and modules
Method definitions with line numbers
Instance variables assigned per action

The analysis_type parameter lets you choose what you need:

execute_tool(
  tool_name: "analyze_models",
  params: {
    model_name: "User",
    analysis_type: "full" # "introspection", "static", or "full"
  }
)

With analysis_type: "full", Claude gets both the runtime reflection data and the static code analysis. With analysis_type: "static", it gets just the AST-derived information—useful when you want to understand code structure without loading the Rails environment.

Detail Levels and Filtering

Another insight from the Anthropic post: intermediate results consume tokens too. When you ask about routes, do you really need the complete output of all 200 routes with constraints and defaults?

Every analyzer now supports a detail_level parameter:

names — Minimal output, just identifiers
summary — Names plus brief descriptions
full — Complete information (the previous default)

For routes specifically, I added filtering:

execute_tool(
  tool_name: "get_routes",
  params: {
    controller: "api/v1/users",
    verb: "GET",
    detail_level: "summary"
  }
)

Instead of 200 routes, Claude might get 5. The token savings compound across a conversation.

For models and schemas, batch operations reduce round-trips:

execute_tool(
  tool_name: "analyze_models",
  params: {
    model_names: ["User", "Post", "Comment"],
    detail_level: "associations"
  }
)

One call, three models, associations only. No source code, no validations, no columns—just the relationships Claude needs for the current task.

Sandboxed Ruby Execution

The most powerful addition is execute_ruby. Sometimes the predefined analyzers aren’t enough—you need a custom query that doesn’t fit the standard patterns.

execute_ruby(code: <<~RUBY)
  User.joins(:posts)
      .group("users.id")
      .having("COUNT(posts.id) > 10")
      .pluck(:email)
RUBY

This runs in a sandboxed environment with strict security controls:

No file writes — Cannot create, modify, or delete files
No system calls — system(), backticks, and exec are blocked
No network access — Cannot make HTTP requests or open sockets
No sensitive file reads — .env, credentials, and .gitignore‘d files are protected
Project-scoped — Cannot access files outside the project directory
Timeout protected — Maximum 60 seconds execution

The sandbox provides helper methods for common operations:

read_file("app/models/user.rb") # Safe file reading
file_exists?("config/database.yml") # Existence check (false for sensitive files)
list_files("app/models/**/*.rb") # Glob with filtering
project_root # Project path constant

I use this for complex queries that would otherwise require multiple tool calls or for analysis patterns specific to my projects.

Improved Discovery UX

After testing the new architecture with real conversations, I noticed AI agents sometimes struggled with the initial learning curve. Which tool reads files? What helpers are available in execute_ruby? The progressive discovery approach works well once you know the patterns, but getting started needed to be smoother.

Quick Start Guide

Now when you switch projects, you get an immediate orientation:

Switched to project: my_finances at path: /Users/mario/projects/my_finances

Quick Start:
• Get project overview: execute_tool("project_info")
• Read a file: execute_ruby("puts read_file('config/routes.rb')")
• Find files: execute_ruby("puts Dir.glob('app/models/*.rb').join('\n')")
• Analyze models: execute_tool("analyze_models", { model_name: "User" })
• Get routes: execute_tool("get_routes")
• Get schema: execute_tool("get_schema", { table_name: "users" })
• Search available tools: search_tools()

Helpers in execute_ruby: read_file(path), file_exists?(path), list_files(pattern), project_root
Note: Always use `puts` in execute_ruby to see output.

This eliminates the cold-start problem. Claude immediately knows the most common patterns without searching for them.

The `puts` Problem

One subtle issue kept appearing: execute_ruby would return “Code executed successfully (no output)” when users forgot to wrap expressions in puts. Now the no-output message includes helpful hints:

Code executed successfully (no output).

Hint: Use `puts` to see results, e.g.:
  puts read_file('config/routes.rb')
  puts User.count
  puts Dir.glob('app/models/*.rb')

Small friction points like this matter more than I initially realized. Every confused moment is context wasted on troubleshooting instead of actual work.

AI Agent Guide

For teams using this with custom AI setups, I created a comprehensive guide at docs/AGENT.md covering:

Tool selection decision trees
Common pitfalls and how to avoid them
Error handling and fallback strategies
Integration patterns with other MCP servers (like Neovim MCP)

The guide is written for AI consumption—structured, explicit, with clear examples for each scenario.

Interactive Configuration Tool

Managing MCP server configuration used to involve editing YAML files and JSON configs manually. With multiple Rails projects, documentation guides, and Claude Desktop integration, this became tedious. So I built rails-mcp-config—an interactive terminal UI for all configuration tasks.

Getting Started

rails-mcp-config

The tool presents a clean menu organized by frequency of use:

┌──────────────────────────────────────────────────┐
│ Rails MCP Server - Configuration │
╰──────────────────────────────────────────────────╯

Projects
  List projects
  Add project
  Add current directory (my-rails-app)
  Edit project
  Remove project
  Validate all projects
────────────────────
Guides
  Download guides
  Import custom guides
  Manage custom guides
────────────────────
Setup
  Claude Desktop integration
  Open config file
────────────────────
Exit

Project Management

Adding a project is now a guided process. If you run the tool from within a Rails project directory, it offers to add that directory with one confirmation:

Directory: ~/projects/my-rails-app

Project name: [my-rails-app]
✓ Added project 'my-rails-app' -> ~/projects/my-rails-app

The tool validates paths, checks for Gemfiles, and uses home-relative paths (~/projects/...) for portability.

Documentation Guides

Downloading Rails, Turbo, Stimulus, and Kamal guides is now visual:

Select guides to download:
  [x] rails (✓ downloaded)
  [x] turbo (not downloaded)
  [] stimulus (not downloaded)
  [] kamal (not downloaded)

⠸ Fetching files...
✓ rails: 2 downloaded, 45 skipped
✓ turbo: 12 downloaded, 0 skipped

You can also import your own markdown documentation and manage custom guides through the same interface.

Claude Desktop Integration

The most useful feature for new users is automatic Claude Desktop configuration. The tool:

Detects your existing Claude Desktop config
Shows current MCP server settings if configured
Offers to add or update the Rails MCP Server configuration
Automatically finds the correct Ruby and server executable paths
Creates timestamped backups before any changes
Supports both STDIO (recommended) and HTTP modes

✓ Claude Desktop config file found
✓ Rails MCP Server is configured

Current configuration:
  command: /Users/mario/.rubies/ruby-3.3.0/bin/ruby
  args: /Users/mario/.gem/ruby/3.3.0/bin/rails-mcp-server

What would you like to do?
  View full config
  Update Rails MCP Server config
  Back to menu

For HTTP mode with mcp-remote, the tool detects npx and guides you through the URL configuration.

Enhanced Terminal UI

The tool uses Gum when available for a polished experience with styled prompts, spinners, and tables. Without Gum, it falls back to a functional basic terminal interface—no dependencies required.

# Optional: Install Gum for enhanced UI
brew install gum # macOS
sudo apt install gum # Debian/Ubuntu

The legacy command-line tools (rails-mcp-setup-claude, rails-mcp-server-download-resources) still work for scripting and backward compatibility, but for interactive use, rails-mcp-config is now the recommended approach.

How I Use the New Architecture

My workflow has evolved with these changes. When I start a session:

Switch to project my_finances.

I immediately see the Quick Start guide, so Claude knows the patterns without any additional discovery.

When I need to explore unfamiliar territory:

Search for tools related to controllers and views.

Claude finds analyze_controller_views and explains what it can do. Then:

Analyze the UsersController with full introspection and static analysis.

For focused work where I know what I need:

Get the schema for the transactions table, just the columns and indexes.

The detail_level: "summary" is implicit in “just the columns”—Claude understands the intent and uses minimal output.

For complex questions that span multiple concerns:

execute_ruby(code: <<~RUBY)
  # Find models with callbacks that touch the database
  Dir.glob("app/models/**/*.rb").select do |file|
    content = File.read(file)
    content.match?(/after_save|after_create|after_update/) &&
    content.match?(/\.save|\.update|\.create/)
  end
RUBY

This kind of query would be tedious with the standard tools but trivial with direct Ruby execution.

Upgrading to the New Version

The upgrade is straightforward:

gem install rails-mcp-server

After installation, run the interactive configuration tool to set up or verify your configuration:

rails-mcp-config

This will help you:

Add your Rails projects
Download documentation guides
Configure Claude Desktop integration (with automatic path detection)

The new architecture is backward compatible in terms of functionality—everything the previous version could do, this version can do. The interface changed from direct tool calls to discovery and execution, but Claude adapts naturally.

If you prefer manual configuration or scripting, the legacy commands still work:

rails-mcp-setup-claude # Configure Claude Desktop
rails-mcp-server-download-resources rails # Download guides

For Prism static analysis to work, your Rails projects need Ruby 3.3+ or the Prism gem installed. If Prism isn’t available, the server gracefully falls back to introspection-only analysis.

Looking Forward

While version 1.4.0 focused on context optimization, my current exploration for 1.4.1 is focused on agent portability.

I am working on a --single-project flag to support ephemeral environments like the GitHub Copilot Agent. These agents run in temporary GitHub Actions environments where no projects.yml exists and no global configuration can be persisted. The new flag will allow the server to skip loading projects.yml, treat the current directory as the sole project, and auto-switch to it immediately on startup.

This same feature unlocks support for Claude Code. Because Claude Code uses worktrees where each session is a separate working directory. By leveraging STDIO mode with the --single-project flag, each worktree can spawn its own isolated MCP server instance without port conflicts. This also allows the .mcp.json configuration to be committed directly to version control, making the setup reproducible for the entire team.

The combination of reduced tool registration, Rails introspection, Prism analysis, and these upcoming compatibility features makes the Rails MCP Server not just smarter, but more adaptable to the rapidly evolving ecosystem of AI agents.

Source code is available at https://github.com/maquina-app/rails-mcp-server

Rails Upgrades with AI: A Real-World Success Story

Mario Alberto Chávez — Thu, 27 Nov 2025 06:00:00 +0000

A few weeks ago, I introduced the Rails Upgrade Skill for Anthropic’s Claude. You can check out the repository here: https://github.com/maquina-app/rails-upgrade-skill.

Yesterday, a client’s Ruby on Rails application—focused on EMR (Electronic Medical Records)—was successfully deployed to Rails 8 with only a minor, quickly resolved issue! Just a week prior, this same application was running on the End-of-Life (EOL) version, Rails 7.1.

Using the Rails Upgrade Skill , I was able to manage this rapid transition. The process involved first upgrading the application to Rails 7.2 and deploying it to production, and then, days later, performing the jump to Rails 8.

To be honest, the changes to the existing codebase were minor:

A few broken specs.
One gem that hadn’t been updated for recent Rails versions.
Two small issues that were not caught by the test suite but were easily fixed before they became real-world problems.

Currently, I even have a Work-In-Progress (WIP) Pull Request for the Rails 8.1 version of this application. It’s blocked only because the Mongoid gem doesn’t yet support the latest released gems in Rails 8.1, but the core upgrade code is ready in the repository. Once again, the skill performed most of the work; we’re just waiting for the gem to be released to continue the upgrade.

The Real Power of the Skill 💡

The skill doesn’t just provide a high-level analysis; it gives clear, actionable guidance on the changes between versions. This includes:

Step-by-step actions with a tailored script to check for breaking changes.
Precise instructions for creating a branch and running necessary commands.
Exactly what files to modify and the rationale behind each change.

Crucially, it makes merging configuration-specific changes much easier—a task that is often painful when using the rails app:update command.

Upgrading Rails applications with an AI skill

Vibecoding the Physical: How AI Helped Me Bind My Photobook

Mario Alberto Chávez — Sat, 22 Nov 2025 06:00:00 +0000

Sometimes, I reach for Claude Desktop just to “vibecode.” I start with a spark—a very simple idea—and I iterate, building complexity layer by layer. Usually, I don’t even specify the tech stack. By default, Sonnet tends to choose React.

Here’s the catch: my knowledge of React is basic. I can’t strictly tell if the code is idiomatic or “correct.” I can only treat the app as a black box—testing it to see if it works as expected. I leave the architectural decisions to the model and focus purely on the outcome.

But this time, the outcome was personal.

The Artist’s Need

Besides being a Software Engineer, I am a Visual Artist and contemporary photographer. Since 2022, I’ve been working on a long-term project that is finally becoming my first photobook.

Recently, my book editor and designer sent me a draft PDF. It looked great on screen, but a photobook is a physical object. I needed to “feel” the book before giving feedback. Electronic documents don’t have weight; they don’t have page turns.

I have the knowledge to print at home on a consumer printer. I also know the process required to take a linear PDF and rearrange the pages into “signatures” (groups of folded sheets) for traditional binding. But doing this manually is tedious and error-prone. One calculation mistake, one wrong paper flip, and the page order is ruined.

The AI Solution

I needed a tool, not a math lesson. I opened Claude and pitched the idea: a “Photobook Signature and Printing Layout Calculator.”

I wanted to upload my PDF and have the tool calculate:

How many signatures do I need for binding?
How to rearrange (impose) the pages for printing?
How to optimize paper usage based on my printer’s specific paper size?

After about 10 iterations, Sonnet nailed it. It built a fully client-side React application. It visualizes the grid, handles the imposition math, and even generates a visual guide for printing.

Because I was “vibecoding,” I didn’t stop at the code. I asked Claude to generate a comprehensive user manual for me (the Photographer) and a technical architecture document for “Future Me” (the Developer).

The Engineer’s Return

This is where the process gets interesting. The React app worked, but as an engineer, I felt a bit detached from the “how.” I wanted to ground the project in a stack I actually mastered.

So, I asked for a Spike :

“Can you make a spike and convert this application to Rails? Don’t worry about the full application; assume that it is there and you are only creating the needed files… Think harder about what can be pushed to the backend with ActiveStorage, and what needs to happen in the front end with ERB, Tailwind, Hotwire, and Stimulus.”

The vibe shifted. Sonnet didn’t get the Rails architecture perfect on the first try, but because I know Rails deeply, I could spot the flaws immediately. I knew where the N+1 queries would hide, how to better utilize ActiveStorage, or where a specific Gem would be more efficient than custom logic.

The Takeaway

I have found this to be a powerful workflow for testing ideas:

Vibecode in the unknown: Let AI build the prototype in a stack it prefers (like React). Focus purely on the user experience and solving the pain point.
Spike into the known: Once the logic is proven, ask AI to port it to my core stack (Rails).
Refine with expertise: Use my engineering seniority to polish the code that I now fully understand.

The result? I have a tool that solves my artistic problem, and it saved me from the manual process to get the layout right. Now, I can go back to my book editing software and rearrange pages to the suggested layout, then finally print my draft book.

Upgrading Rails applications with an AI skill

Mario Alberto Chávez — Wed, 12 Nov 2025 06:00:00 +0000

What if you could use an AI skill to handle the most tedious, error-prone parts of a Ruby on Rails upgrade? As someone who’s been in the Rails trenches since 2008, I’ve spent countless hours on this exact task.

In my time as a founder, contractor, and company owner, I’ve upgraded applications from nearly every era. The early days of 2.x, 3.x, and 4.x were battles against breaking API changes, specific and complex monkey patches, and gems that were abandoned over time.

Things got better after Rails 5.2, but one command still triggers a familiar headache: rails app:update.

The real challenge isn’t the command itself. It’s the high-stakes, manual process that follows: meticulously merging your application’s custom configurations with the framework’s new defaults. Even with great tools like RailsDiff, it’s a slow, manual process where a single mistake can cause subtle, cascading bugs.

The Spark of an Idea

I’ve been thinking about how to improve this process for a while. When Anthropic first announced the Model Context Protocol (MCP), I saw a post on X (formerly Twitter) asking if AI could finally solve this exact config-merging nightmare. The idea struck a chord, but I was swamped with work.

A few weeks ago, that idea resurfaced. A client’s application on Rails 7.1 was approaching its end-of-life (EOL) date, and I was staring down another upgrade. But this time, something was different: Anthropic had just launched Skills.

As I was planning the upgrade, I was open to considering if one of these new Skills could be the solution. I realized a “skill” wasn’t just a general-purpose AI; it was a specialized, tool-driven capability. Could this be the key to finally automating the most painful part of a Rails upgrade?

I had to find out.

Introducing the `rails-upgrade-skill`

The result of that experiment is the Rails Upgrade Assistant Skill (or rails-upgrade-skill in its repository). This is a specialized AI skill that uses Claude’s intelligence to guide the entire upgrade process.

It’s built using official Rails CHANGELOGs to intelligently plan your upgrade path for any version from Rails 7.0 through 8.1.1.

What makes this skill an intelligent upgrade assistant?

Intelligent Analysis: It calls the Rails MCP Server to read your actual project files , understand your customizations , and provide personalized guidance—not generic advice.
Custom Code Preservation: It automatically detects custom configurations and provides specific warnings (e.g., about custom SSL middleware or autoload paths) with clear instructions on how to migrate them safely, ensuring your code logic isn’t lost.
Sequential Enforcement: It prevents the dangerous practice of skipping versions. If you ask for a multi-hop upgrade (e.g., 7.0 to 8.1), it automatically plans the correct sequential path (7.0 → 7.1 → 7.2 → 8.0 → 8.1) and guides you through each hop separately.

Make It Your Own

This skill is currently opinionated and tightly coupled to my personal workflow. It’s built to work with two other tools I’ve made:

Rails MCP Server: Used to gather deep information about your Rails application (like version, file contents, routes).
Neovim MCP Server: Used to allow Claude to apply the identified changes directly to your files in Neovim (the Interactive Mode ).

But this is the beauty of Skills. You don’t have to use my stack. I invite you to fork the repo and edit the SKILL.md file. You can instruct the skill on how you want it to get project information or apply changes, tailoring it to your own editor and workflow.

How to Use the Skill: A Step-by-Step Guide

Here’s how it works in my setup.

1. Installation

First, clone the repository and run the build script:

> git clone [https://github.com/maquina-app/rails-upgrade-skill.git](https://github.com/maquina-app/rails-upgrade-skill.git)
> cd rails-upgrade-skill
> bin/build

This command zips the skill into a /build folder. Next, open your Claude Desktop app, go to Settings > Capabilities , and scroll down to Skills. You can upload the build/rails-upgrade-skill.zip file there.

2. Set Your Project Context

In a new chat, you need to tell Claude which project you’re working on. If you’re using my Rails MCP Server, the command is:

Switch to my_app project and show me the project information

This activates your Rails application’s context and allows the skill to gather its details.

3. Activate the Upgrade Skill

Once the context is set, you can invoke the AI skill with a simple prompt:

Help me upgrade my Rails application to next available version

The skill takes over and kicks off a three-step process, which can be done in two modes:

Mode 1 (Report-Only): Claude provides the comprehensive report, and you apply changes manually. This is recommended for your first time.
Mode 2 (Interactive): Claude guides you and offers to apply changes directly to your open files in Neovim.

4. The Three-Phase Process

Step 1: Detect Deprecations

The skill generates a bash script for you. You copy and run this script in your application’s root directory. It scans your application for deprecation patterns and breaking changes, saving findings to a .txt file, which you then paste back into Claude.

Step 2: Generate a Comprehensive Report

This is the core value. Using the project analysis and the deprecation findings, the skill generates a detailed report. It breaks down changes by priority (HIGH, MEDIUM, LOW) and component (ActiveRecord, ActionMailer, etc.).

Crucially, it intelligently merges the new Rails defaults with your existing custom settings and provides OLD vs. NEW code examples with explanation.

Step 3: Apply the Changes (The app:update Report)

The final report summarizes all files needing changes. If you are using the Interactive Mode with the Neovim MCP server, you can review the proposed changes and then tell Claude:

All files are ready on nvim buffers, update them all

Claude will then apply all the approved changes directly to your open files.

The Final Mile: After the Skill

This AI skill handles the most complex parts of the upgrade, getting you 90% of the way there. You still need to follow the standard Rails procedure:

First, update your gems:

> bundle update

Next, run the rails app:update command. Run it with confidence. When the command flags conflicts in config files, you can usually skip overwriting them, knowing the logic has already been intelligently merged by the skill. You then let the command proceed to create new files, like initializers and migrations.

> rails app:update

And the last, most crucial step: run your test suite and test the application manually.

Final Thoughts

I’ve already used this skill to upgrade a client application and several of my own projects, turning hours of tedious work into a process that takes just a few minutes of interaction time.

This, for me, represents the true power of AI in development: combining deep, specialized domain knowledge (the nuances of a Rails upgrade) with new tools like Anthropic Skills.

Please, give the rails-upgrade-skill a try. Fork it, adapt it to your own workflow, and let me know what you think.

Rails MCP Server: Enhanced Documentation Access

Mario Alberto Chávez — Tue, 03 Jun 2025 06:00:00 +0000

Rails MCP Server: Enhanced Documentation Access and AI Workflow Integration

Updated Rails MCP Server now provides consistent, up-to-date Rails documentation across multiple LLM clients with enhanced proxy support and Neovim integration.

Comprehensive Documentation Resources
The Documentation Challenge
Five Resource Categories Available
Setting Up Documentation Resources
MCP Proxy Integration for Better Compatibility
Rails MCP + Neovim MCP Integration Workflow
Documentation as a Shared Resource
Resource Management and Best Practices
FAQ

The latest updates to Rails MCP Server introduce significant improvements to documentation access, better integration capabilities, and enhanced workflow support that have transformed how I work with Rails projects using AI assistance.

Comprehensive Documentation Resources

The most significant addition in this release is the comprehensive Resources and Documentation system. This addresses a critical need in AI-assisted development: providing LLM clients with consistent, up-to-date documentation that can be shared across multiple AI sessions and different LLM providers.

The Documentation Challenge

When working with AI assistants on Rails projects, I frequently encountered situations where the LLM’s training data was outdated or incomplete regarding specific framework features. This led to suggestions based on deprecated APIs or missing newer functionality. The resource system solves this by:

Ensuring accuracy : LLMs receive the exact same official documentation that developers reference
Maintaining consistency : Multiple AI sessions can access identical documentation, ensuring consistent guidance
Staying current : Documentation can be updated to match specific Rails versions or framework releases
Sharing context : The same documentation resources work across different LLM clients and providers

Five Resource Categories Available

The server now provides access to five complete documentation libraries:

1. Rails Guides Documentation

Content : Official Ruby on Rails 8.0.2 documentation - all 50+ guides
Coverage : Getting started to advanced topics including Active Record, Action Pack, security, and deployment
Use case : Comprehensive Rails framework reference

2. Turbo Framework Documentation

Content : Complete Hotwire Turbo framework documentation
Structure : Handbook and reference sections covering Turbo Drive, Frames, and Streams
Use case : Modern Rails frontend development with Hotwire

3. Stimulus JavaScript Framework Documentation

Content : Full Stimulus documentation for building interactive components
Structure : Handbook tutorials and API reference
Use case : JavaScript interactions in Rails applications

4. Kamal Deployment Documentation

Content : Comprehensive Kamal deployment tool documentation
Coverage : Installation, configuration, commands, and deployment strategies
Use case : Modern Rails application deployment

5. Custom Documentation Resources

Content : Import and access your own markdown documentation files
Flexibility : Project-specific guides, API documentation, team standards
Use case : Maintaining consistent project documentation across AI sessions

Setting Up Documentation Resources

Setting up documentation is straightforward with the dedicated download tool:

# Download official framework documentation
rails-mcp-server-download-resources rails
rails-mcp-server-download-resources turbo
rails-mcp-server-download-resources stimulus
rails-mcp-server-download-resources kamal

# Import your custom documentation
rails-mcp-server-download-resources --file /path/to/your/docs/

# Force update existing resources
rails-mcp-server-download-resources --force rails

# Verbose output for troubleshooting
rails-mcp-server-download-resources --verbose turbo

Once downloaded, you can access guides naturally in conversation:

Can you load the Rails getting started guide?
Show me the Turbo Frames documentation.
I need help with Stimulus controllers - can you show me that guide?
Load the Kamal deployment guide so I can understand the process.

MCP Proxy Integration for Better Compatibility

Building on the previous release with HTTP SSE support, this update includes comprehensive documentation for using MCP proxies to address Ruby version manager compatibility issues that many developers face with Claude Desktop.

Setting Up MCP Proxy for Enhanced Compatibility

For users who want to leverage the HTTP/SSE capabilities or work around Ruby version manager issues:

Step 1: Start Rails MCP Server in HTTP Mode

rails-mcp-server --mode http

Step 2: Install and Configure MCP Proxy

# Install the Node.js based MCP proxy
npm install -g mcp-remote

# Run the proxy, pointing to your running Rails MCP Server
npx mcp-remote http://localhost:6029/mcp/sse

Step 3: Configure Claude Desktop for Proxy Usage

{
  "mcpServers": {
    "railsMcpServer": {
      "command": "npx",
      "args": ["mcp-remote", "http://localhost:6029/mcp/sse"]
    }
  }
}

This setup allows STDIO-only clients to communicate through the proxy while benefiting from HTTP/SSE capabilities and avoiding Ruby version conflicts.

Rails MCP + Neovim MCP Integration Workflow

I’ve significantly improved my development workflow by combining the Rails MCP Server with the Neovim MCP Server. This combination creates a powerful development environment where I can seamlessly work with both my Rails codebase and my active editing context.

The Two-Server Development Approach

Here’s how I use both MCP servers together for enhanced productivity:

Rails MCP Server handles:

Project structure analysis and exploration
Database schema exploration and relationships
Route inspection and API endpoint analysis
Model relationship analysis and validations
Access to comprehensive Rails ecosystem documentation

Neovim MCP Server provides:

Real-time access to currently open buffers
Context about active editing sessions
Bridge between editor state and AI assistance
Live file content without manual specification

Optimized Development Session Workflow

When I start a development session, I begin with context gathering:

Switch to project my_rails_app, don't analyze anything yet.
Show me which files I have open in my "my_rails_app" Neovim instance.

This gives me both project context from Rails MCP and my current editing context from Neovim MCP. Then I can work more efficiently:

Load the User model from Rails and also get the user_controller.rb that I have open in Neovim.
Can you also load the Rails Active Record associations guide for reference?

Benefits of the Dual-Server Approach

This approach allows me to:

Work with live context : See exactly what I’m editing without manually specifying files
Access comprehensive documentation : Get official guides loaded instantly for reference
Maintain project focus : Switch between Rails projects while keeping editor context
Avoid manual file management : Let the tools handle finding and loading the right files

Managing Token Usage Effectively

I’ve refined my prompting strategy to manage Claude’s context window more effectively:

Session Start:

Switch to project my_finances, don't analyze the project or any loaded files until you are told to.

Loading Files:

Load the following files from my Neovim buffers, do not analyze or try to load dependencies unless you are told to.

Session Continuation: When reaching conversation limits, I summarize and continue:

Summarize this chat. Describe the goal and add relevant information about what was done. Include the project name and list any files loaded, generated or shared.

Documentation as a Shared Resource

The resource system transforms documentation from a static reference into a dynamic, shared asset. When I load Rails validation documentation in one conversation, another AI session can access the same exact information. This consistency is particularly valuable when:

Training team members : Everyone gets the same documentation regardless of which AI tool they use
Maintaining project standards : Documentation stays consistent across different development sessions
Working with multiple LLM providers : The same Rails 8.0.2 guides work identically with Claude, ChatGPT, or other MCP-compatible clients
Ensuring version alignment : Projects can specify exact documentation versions to match their Rails installation

Instead of switching to browser tabs or searching through docs, I can load exactly the guide I need:

I'm working on validations - can you load the Rails validations guide?
Show me the Turbo Streams reference for this real-time update feature.
Load my custom API documentation so we can follow the established patterns.

The system handles both official framework documentation and custom project documentation seamlessly, making it a true companion for Rails development.

Resource Management and Best Practices

Managing resources is straightforward with clear commands and automatic organization:

Resource Organization Best Practices

Download once, use everywhere : Resources are stored locally and available across all conversations
Automatic updates : Re-download to get the latest documentation versions
Custom integration : Import your project-specific documentation alongside official guides
Intelligent organization : Resources are categorized and easily discoverable

Resource Storage and Management

Resources are stored in platform-specific locations:

macOS : ~/.config/rails-mcp/resources/
Windows : %APPDATA%\rails-mcp\resources\

Each resource category maintains a manifest file tracking downloaded guides, versions, and update timestamps.

Performance Optimization Tips

Selective downloads : Only download resources you actively use
Regular updates : Keep documentation current with framework releases
Custom documentation : Organize project-specific guides with descriptive filenames
Version alignment : Match documentation versions to your Rails installation

For complete details on setting up and using resources, I’ve created a comprehensive Resources Guide that covers everything from basic setup to advanced usage patterns.

FAQ

How do I update Rails MCP Server documentation resources?

Re-run the download command for any resource category to get the latest version:

rails-mcp-server-download-resources rails

Can I use Rails MCP Server with other AI clients besides Claude Desktop?

Yes! The MCP proxy setup allows compatibility with any MCP-compatible client. The HTTP/SSE endpoints work with various LLM providers.

What’s the difference between Rails MCP Server and Neovim MCP Server?

Rails MCP Server : Analyzes Rails project structure, database, routes, and provides Rails ecosystem documentation
Neovim MCP Server : Provides access to currently open files in your editor sessions

How do I troubleshoot Ruby version manager issues with Claude Desktop?

Use the MCP proxy setup to bypass Ruby version conflicts, or create a symbolic link as described in the Ruby Version Manager Users section.

Can I import custom documentation alongside official Rails guides?

Yes! Use the --file option to import your own markdown files:

rails-mcp-server-download-resources --file /path/to/your/docs/

Looking Forward

This enhanced Rails MCP Server has become an integral part of my Rails development workflow. The combination of comprehensive documentation access, improved integration options, and seamless editor integration through the Neovim MCP Server creates a development environment where AI assistance feels natural and productive.

The ability to instantly access official documentation, work with live editing context, and maintain project focus has significantly improved my development velocity and the quality of my AI-assisted coding sessions.

I encourage you to try both servers together and explore the new resource system. The documentation access alone makes this upgrade worthwhile, and the improved integration options ensure it works well regardless of your development setup.

Source code and detailed setup instructions:

The Rails MCP Server continues to evolve based on real-world usage. If you have suggestions or encounter issues, please feel free to open an issue or contribute to the project.

Rails MCP Server with HTTP Server-Sent Events support

Mario Alberto Chávez — Sun, 20 Apr 2025 06:00:00 +0000

Rails MCP Server 1.1.0 with HTTP SSE support

I’m excited to announce this new version of Rails MCP Server, version 1.1.0! This release introduces significant internal refactoring through the integration of the FastMCP gem, providing better code organization and adding support for HTTP Server-Sent Events (SSE).

What’s New in v1.1.0

FastMCP Integration

The most substantial change in this release is the internal refactoring to use the FastMCP gem. This refactoring has provided several benefits:

Improved Code Organization : I’ve made the codebase more modular and easier to maintain
Improved tools : I’ve reviewed the code for each tool to fix issues with edge cases and to make the tools honor .gitignore file when scanning the codebase for files. This is important for large codebases.
Added two new tools : There are two new tools, analyze controller and views and analyze environment configuration. Analyze controllers and views finds all the relationships with the routes, views, stimulus controllers and controller actions in detail. Analyze environment configuration compares the configuration for inconsistencies between environments, missing configuration, and possible security issues, it does not leak keys or passwords to the LLM.

HTTP Server-Sent Events (SSE) Support

With this release, I’ve added HTTP Server-Sent Events (SSE) support for real-time communication:

Support for other clients: Clients with support for HTTP Server-Sent Events (SSE) can now interact with the Rails MCP Server.

Using HTTP Mode with SSE

The Rails MCP Server can now run in two distinct modes:

STDIO mode (default): Communicates over standard input/output for direct integration with clients like Claude Desktop.
HTTP mode : Runs as an HTTP server with JSON-RPC and Server-Sent Events (SSE) endpoints.

To start in HTTP mode:

# Start in HTTP mode on the default port (6029)
rails-mcp-server --mode http

# Start in HTTP mode on a custom port
rails-mcp-server --mode http -p 8080

When running in HTTP mode, the server provides two endpoints:

JSON-RPC endpoint: http://localhost:<port>/mcp/messages
SSE endpoint: http://localhost:<port>/mcp/sse

This enables integration with web applications, dashboard tools, and other systems that need to communicate with the Rails MCP server.

Upgrading from Earlier Versions

Upgrading to v1.1.0 should be straightforward for most users. The API remains backward compatible, so existing code should continue to work without modification. To upgrade:

gem install 'rails-mcp-server' # This installs the current version 1.1.0

Under the Hood

I’ve streamlined the codebase significantly by refactoring to use FastMCP. The FastMCP gem provides a robust implementation of the MCP protocol, handling message encoding/decoding, connection management, and event dispatching.

I built the SSE implementation on top of Rack and integrated it seamlessly with FastMCP’s event system. This allows for efficient real-time updates and event streaming while maintaining compatibility with the Model Context Protocol standard.

Testing with MCP Inspector

The Rails MCP Server v1.1.0 works seamlessly with MCP Inspector, making it easier than ever to test and debug your MCP server:

# Install and run MCP Inspector with your Rails MCP Server
npm -g install @modelcontextprotocol/inspector

# Run the inspector and connect to the Rails MCP Server via STDIO or HTTP SSE
npx @modelcontextprotocol/inspector

How I use the Rails MCP Server

I use the server with Claude Desktop from Anthropic. It does not yet support HTTP SSE; it can only communicate via STDIO. Adding HTTP SSE facilitates people using other clients to use the server easily, as using the server with Claude Desktop is not simple if you use a Ruby version manager.

Claude Desktop starts a process to run the server but with a no login session, which means that your Ruby version manager is not initialized and macOS uses the bundled 2.6 Ruby version. There is no real fix for this situation, just a workaround. Please look at the README to see how to make it work for rbenv ; a similar solution is required for other Ruby managers.

When I start a session with Claude, my first prompt is:

Switch to project my_finances, don't analyze the project or any loaded files until you are told to.

I do this because the Sonnet model can start very creatively and begin analyzing all possible files in the project, consuming its quota.

I work on a feature per chat, so I load only the files that are related to the work that I want to do:

Load the following file or files, do not analyze or try to load dependencies unless you are told to.

Again, here I pass the list of files to load but I stop Sonnet from trying to analyze without knowing yet what I want to do. My next prompt is to explain what I want to do, and then I ask it to analyze the files and maybe load more if needed.

While working with Claude Desktop, it’s easy to reach the chat quota. I’m prepared at the end to continue with the feature if it’s not done yet. I use the following prompt:

Summarize this chat. Describe what is the goal and add any relevant information about what was done. Include the project name and the list with a brief description of any file loaded, generated or shared with you.

Then I copy the output and start a new chat with the following prompt:

Here is a summary of a previous chat. Switch to the project if a given name exists. Don't load any reference files until you are told to. Also, don't analyze the project or any loaded files until you are told to.
Just comprehend in a general way what was done before.

If you are told to generate data, always generate synthetic data unless you are told otherwise.

And continue with the cycle. Claude Desktop doesn’t write to the disk; I always review the generated code and copy it manually. I modify the code when it makes sense and share it back with Claude.

Conclusion

I believe this release of Rails MCP Server v1.1.0 represents a significant step forward in terms of code quality and feature set. The integration with FastMCP provides a more maintainable foundation, while the addition of SSE support expands the communication options available to applications using the server.

I encourage you to upgrade to this latest version and explore the new capabilities it offers. As always, I welcome feedback and contributions from the community.

Source code is available at https://github.com/maquina-app/rails-mcp-server

Rails MCP Server - Enhancing AI-Assisted Development

Mario Alberto Chávez — Fri, 21 Mar 2025 18:00:00 +0000

AI has changed the way we write and deploy code to production. LLMs are getting better and better at understanding and also at generating code. It started with chats, where you can make questions and share code, then evolved to suggest auto-completion inside the editor and later the inclusion of chats in the same editor.

Nowadays we have editors that are created specifically to understand the code base and write code with just prompts, and a button to accept the changes and update the code base.

But, there is a shortcoming when working with AI, mainly because of the bias of the LLMs that are better at generating code for the JavaScript ecosystem, and not that good with code in almost every other programming language.

Still, AI is great for generating boilerplate code or for green field projects. Moreover, the possibility to extend LLM model knowledge with techniques and protocols that help with the time to deploy, adding significant value, even if it requires more iterations to help the LLM get the correct context.

In my case, I can’t take advantage of the new IDEs that integrate AI into their core; I don’t use IDEs, I use Neovim. There are tools to make Neovim work like those IDEs, but still, that is something that I don’t want to do.

With my way of work, I prefer to use Claude Desktop with the Sonnets models. My workflow is simple; I create projects with a prompt that provides general instructions for the model, something like:

You are an expert Ruby on Rails developer. Help me to write professional and well-crafted code; don’t explain it unless required. You are familiar with TailwindCSS, Turbo Rails, Hotwire, and Stimulus. Also, don’t write tests unless you are required. All tests should be with minitest in unit test style.

Then, in the knowledge base of the project, I add documentation and texts that provide context in greater detail, along with code style, what to do, and what not to do.

Once my projects are configured, then I can start doing some work. I mainly work on brownfield projects with large code bases, so sharing the whole code base is not an option. Before I start the task at hand, I collect fragments of code that can be relevant to what I need to accomplish.

I even have a bash script that can copy the full content of a file along with the file path. The AI generates new code or changes with my instructions since the AI can’t write directly to my files, this allows me to review the code and assess if it will work as expected and if the code style is consistent with my code base.

If I spot something that I can write better or might produce an error, I don’t ask the AI to fix it. I just copy what makes sense to Neovim, update the code, and copy it back to the AI so it can have the latest version. I work this way because I don’t Vibe code. I need to keep some quality in the code, and I also need to fully understand what is happening because that code powers companies and their customers.

This workflow works great so far, but when I learned about the Model Context Protocol or MCP, I wondered how I might be able to use it to improve how I share code with Claude Desktop. So, I embarked on a quest to create a Rails MCP server; actually, Claude Desktop wrote about 80% of it.

What the Rails MCP Server can do?

The Rails MCP Server provides a set of tools that allow Claude to interact directly with my Rails projects. This enables a more seamless workflow when I need AI assistance with my codebase. Here are the capabilities:

Project Navigation

I can easily switch between different Rails projects using the switch_project tool. This is particularly useful when I’m working on multiple applications and need Claude’s assistance with different codebases throughout my workday.

The Rails MCP Server follows the XDG Base Directory Specification for configuration files:

On macOS: $XDG_CONFIG_HOME/rails-mcp or ~/.config/rails-mcp if XDG_CONFIG_HOME is not set
On Windows: %APPDATA%\rails-mcp

The server automatically creates these directories and an empty projects.yml file on the first run. To configure my projects, I edit the projects.yml file to include my Rails projects:

store: "~/projects/store"
blog: "~/projects/rails-blog"

Each key in the YAML file is a project name (used with the switch_project tool), and each value is the path to the project directory.

Example prompts I use:

“Can you switch to the ‘store’ project so we can explore it?”
“I’d like to analyze my ‘blog’ application. Please switch to that project first.”

Project Overview

With the get_project_info tool, Claude can provide me with a comprehensive overview of my Rails application, including its version, directory structure, and configuration. This gives Claude the necessary context to understand my application’s architecture before diving into specific tasks.

Example prompts I use:

“Now that we’re in the blog project, can you give me an overview of the project structure and Rails version?”
“Tell me about this Rails application. What version is it running and how is it organized?”

File Exploration

The list_files tool allows Claude to scan my project directories and locate specific files. I can filter by directory path or file pattern, making it easy to find what I need.

Example prompts I use:

“Can you list all the model files in this project?”
“Show me all the controller files in the app/controllers directory.”
“List all the JavaScript files in the app/javascript directory.”

Code Examination

Using the get_file tool, Claude can retrieve the complete content of any file in my project with syntax highlighting. This eliminates my need to manually copy and paste code snippets, streamlining my workflow.

Example prompts I use:

“Can you show me the content of the User model file?”
“I need to see what’s in app/controllers/products_controller.rb. Can you retrieve that file?”
“Please show me the application.rb file so I can check the configuration settings.”

Route Analysis

The get_routes tool provides access to all HTTP routes defined in my Rails application. Claude can analyze the routing structure to help me understand API endpoints, URL patterns, and controller actions.

Example prompts I use:

“Can you show me all the routes defined in this application?”
“I need to understand the API endpoints available in this project. Can you list the routes?”
“Show me the routing configuration for this Rails app so I can see how the URLs are structured.”

Model Inspection

With the get_models tool, Claude can examine my Active Record models in detail. It can list all models or provide comprehensive information about a specific model, including its schema, associations, and code implementation.

Example prompts I use:

“Can you list all the models in this Rails project?”
“I’d like to understand the User model in detail. Can you show me its schema, associations, and code?”
“Show me the Product model’s definition, including its relationships with other models.”

Schema Investigation

The get_schema tool allows Claude to access my database schema information. It can show the complete database structure or focus on a specific table, displaying columns, data types, constraints, and relationships.

Example prompts I use:

“Can you show me the complete database schema for this Rails application?”
“I’d like to see the structure of the users table. Can you retrieve that schema information?”
“Show me the columns and their data types in the products table.”

The Rails MCP Server doesn’t just save me time—it enhances Claude’s understanding of my codebase, leading to more accurate and relevant assistance. Whether I’m troubleshooting an issue, implementing a new feature, or refactoring existing code, Claude can now access the context it needs to provide helpful suggestions tailored to my specific project.

Installation

Installing the Rails MCP Server is straightforward. I start by installing the gem:

gem install rails-mcp-server

After installation, the rails-mcp-server and rails-mcp-setup-claude executables are available in my PATH.

Claude Desktop Integration

I run the setup script which automatically configures Claude Desktop and sets up the proper XDG-compliant directory structure:

rails-mcp-setup-claude

The script:

Creates the appropriate config directory for my platform
Creates an empty projects.yml file if it doesn’t exist
Updates my Claude Desktop configuration

After running the script, I restart Claude Desktop to apply the changes.

Ruby Version Manager Users

Claude Desktop launches the MCP server using my system’s default Ruby environment, bypassing version manager initialization (e.g., rbenv, RVM). The MCP server needs to use the same Ruby version where it was installed, as MCP server startup failures can occur when using an incompatible Ruby version.

Since I use a Ruby version manager (rbenv), I create a symbolic link to my Ruby shim to ensure the correct version is used:

sudo ln -s /home/mario/.rbenv/shims/ruby /usr/local/bin/ruby

I replace the path with my actual path for the Ruby shim.

Final Thoughts

If you want access to the source code, here is the link https://github.com/maquina-app/rails-mcp-server

The Rails MCP Server is a significant step forward in how I interact with AI assistants like Claude as a Ruby on Rails developer. I’ve learned that AI outputs are only as good as the context I provide—when Claude has access to the right files and project structures, the quality and accuracy of its suggestions improve dramatically. Instead of manually copying snippets or trying to explain complex relationships between models, I can simply ask Claude to examine the relevant files directly, maintaining control over my development process while enhancing my existing Neovim workflow with contextually aware AI assistance.

Nobuild with Rails and Importmap

Mario Alberto Chávez — Thu, 28 Nov 2024 18:00:00 +0000

The latest versions of Ruby on Rails have focused on simplicity across different aspects of the framework, accompanied by the promise to return to the “one-man framework” (where a single developer can effectively build and maintain an entire application).

Importmap Rails library is based on the principle that modern web browsers have caught up with the ECMAScript specification and can interpret ES Modules (ESM). As a web standard, Importmap allows you to control how JavaScript modules are resolved in the browser and manage dependencies and versions without the need to transpile or bundle the code sent to the browser.

How the Importmap Web Standard Works

It all starts with a script tag of type importmap defined in your application’s main layout or web page. Inside this tag, a JSON object defines aliases and their corresponding paths to the source code.

<script type="importmap">
  {
    "imports": {
      "application": "/assets/application.js",
      "local-time": "https://cdn.jsdelivr.net/npm/local-time@3.0.2/app/assets/javascripts/local-time.es2017-esm.min.js",
      "utils": "/assets/utils.js"
    }
  }
</script>

In the same map, you can mix library paths pointing to a CDN or using local resources. To use libraries from this map, reference the alias name.

<!-- Below the importmap script -->
<script type="module">import "application"</script>

And in your application.js, import needed dependencies:

// application.js

import LocalTime from "local-time";
LocalTime.start();

import "utils";

Importmap support is present in browsers Chrome 89+, Safari 16.4+, Firefox 108+, and Edge 89+. For older browsers, include a polyfill:

<script
  async
  src="https://ga.jspm.io/npm:es-module-shims@1.10.1/dist/es-module-shims.js"
></script>

How Importmap Works in Ruby on Rails

Importmap functionality in Ruby on Rails follows the same standard described above and offers an easy way to create maps and version files. Using a web application named heroImage as an example (source code available on Github), let’s explore the implementation.

When you create a new Rails 8 application, the importmap-rails gem is added and installed by default. A file config/importmap.rb is created where you can pin the JavaScript code needed in your application.

pin "application"

pin "@hotwired/turbo-rails", to: "turbo.min.js"
pin "@hotwired/stimulus", to: "stimulus.min.js"
pin "@hotwired/stimulus-loading", to: "stimulus-loading.js"

pin_all_from "app/javascript/controllers", under: "controllers", preload: false

The pin keyword takes up to three arguments. The first one is required, as it is the alias of the JavaScript code. pin "application" is a shortcut for file application.js with alias application:

pin "application", to: "application.js"

When alias and file names differ, use the keyword to::

pin "@hotwired/turbo-rails", to: "turbo.min.js"

The pin_all_from keyword helps reference multiple files at once. The first argument is the path where the JavaScript files are located, and the under: argument prefixes the alias for each file. The generated alias uses the under prefix and the file name, like controllers/alert-controller for alert_controller.js file.

To visualize the Importmap JSON file, execute:

bin/importmap json

{
  "imports": {
    "@hotwired/turbo-rails": "//127.0.0.1:4700/assets/turbo.min-fae85750.js",
    "@hotwired/stimulus": "//127.0.0.1:4700/assets/stimulus.min-4b1e420e.js",
    "@hotwired/stimulus-loading": "//127.0.0.1:4700/assets/stimulus-loading-1fc53fe7.js",
    "application": "//127.0.0.1:4700/assets/application-b1902c45.js",
    "controllers/application": "//127.0.0.1:4700/assets/controllers/application-fab0f35b.js",
    "controllers": "//127.0.0.1:4700/assets/controllers/index-c3f5d3c4.js",
    "controllers/alert_controller": "//127.0.0.1:4700/assets/controllers/alert_controller-caf203bf.js",
    "controllers/file_controller": "//127.0.0.1:4700/assets/controllers/file_controller-5da5fdc5.js",
    "controllers/notifications_controller": "//127.0.0.1:4700/assets/controllers/notifications_controller-88e2cc65.js",
    "controllers/service_worker_controller": "//127.0.0.1:4700/assets/controllers/service_worker_controller-ad4a24d3.js",
    "controllers/share_controller": "//127.0.0.1:4700/assets/controllers/share_controller-fe28ed00.js"
  }
}

Rails resolves all JavaScript through the Propshaft gem, which resolves the physical path of the JavaScript code, maps to the /assets web path, and adds the digest to each file for better caching and invalidations.

Propshaft discovers physical paths from the asset’s configuration:

Rails.application.config.assets.paths

Ensure your files exist in any of the registered paths or add your own path to be discovered by Propshaft and Importmap.

Importmap in Rails allows you to specify how the browser should load JavaScript files. There are two options: preload (default) and no preload. Preload tells the browser to download files as soon as possible. Importmap generates a link tag with rel="modulepreload":

<link
  rel="modulepreload"
  href="https://heroimage.co/assets/turbo.min-fae85750.js"
/>
<link
  rel="modulepreload"
  href="https://heroimage.co/assets/stimulus-loading-1fc53fe7.js"
/>
<link
  rel="modulepreload"
  href="https://heroimage.co/assets/stimulus-loading-1fc53fe7.js"
/>
<link
  rel="modulepreload"
  href="https://heroimage.co/assets/application-b1902c45.js"
/>

If you set the preload argument to false, the link tag is not generated and the browser downloads the file when needed.

With Rails’ Importmap, you can also pin JavaScript code from a CDN using the to: argument for the URL:

pin "local-time", to: "https://cdn.jsdelivr.net/npm/local-time@3.0.2/app/assets/javascripts/local-time.es2017-esm.min.js"

The Importmap includes a CLI to pin or unpin JavaScript code into config/importmap.rb file. It also includes commands to update, audit, and inspect versions:

bin/importmap --help
Commands:
  importmap audit # Run a security audit
  importmap help [COMMAND] # Describe available commands or one specific command
  importmap json # Show the full importmap in json
  importmap outdated # Check for outdated packages
  importmap packages # Print out packages with version numbers
  importmap pin [*PACKAGES] # Pin new packages
  importmap unpin [*PACKAGES] # Unpin existing packages
  importmap update # Update outdated package pins

When using the pin command for a JavaScript package, instead of setting the to: argument to the CDN, Importmap resolves package dependencies and downloads the package and dependencies to vendor/javascript, allowing the Rails application to serve those files:

bin/importmap pin local-time
Pinning "local-time" to vendor/javascript/local-time.js via download from https://ga.jspm.io/npm:local-time@3.0.2/app/assets/javascripts/local-time.es2017-esm.js


# config/importmap.rb
...
pin "local-time" # @3.0.2

This approach works well when your package has simple dependencies or well-defined dependencies in the JavaScript package. If that’s not the case, it becomes challenging to use with Importmap vendoring the code at vendor/javascript. It might work with the URL and manual dependency addition, or you can tweak the vendored code to make it work.

How to Work with Rails Gems - Engines - and Importmap?

There are two approaches to creating Ruby on Rails gems compatible with Importmap. The first approach allows your gem to provide JavaScript code, which you can choose to pin in the Importmap configuration. This is how the turbo-rails and stimulus-rails gems are implemented.

Place your JavaScript code in the app/assets/javascripts folder of your gem. You may need an additional process that minifies the JavaScript files and generates JavaScript map files. Then, inside the Engine class, define an initializer hook to declare your JavaScript code with Propshaft:

module MyEngine
  class Engine < ::Rails::Engine
    # Additional code
    PRECOMPILE_ASSETS = %w( my_javascript.js my_javascript.min.js my_javascript.min.js.map ).freeze

    initializer "my_engine.assets" do
      if Rails.application.config.respond_to?(:assets)
        Rails.application.config.assets.precompile += PRECOMPILE_ASSETS
      end
    end
  end
end

The second option uses an Importmap configuration file. If your engine has its layout template and the views are isolated from the host application, and the engine doesn’t need to share the JavaScript code with the host application, you can create an Importmap configuration file at config/importmap.rb, set your pins, place your JavaScript code at app/javascript, and configure the engine with an initializer.

Open your engine.rb Ruby file and add the Importmap configuration file and a sweeper:

initializer "my-engine.importmap", after: "importmap" do |app|
  MyEngine.importmap.draw(root.join("config/importmap.rb"))
  MyEngine.importmap.cache_sweeper(watches: root.join("app/javascript"))

  ActiveSupport.on_load(:action_controller_base) do
    before_action { MyEngine.importmap.cache_sweeper.execute_if_updated }
  end
end

Specify the Importmap to use in your engine’s layout template:

<%= javascript_importmap_tags "application", importmap: MyEngine.importmap %>

For sharing JavaScript code with the host application, like Stimulus controllers, create a partial Importmap configuration file and set the engine to merge it with the main one in the host application.

Create an Importmap configuration file at config/importmap.rb and add the JavaScript pins to share with the host application. If you have dependencies for external packages, add those via a generator or installer to the host application:

pin_all_from File.expand_path("../app/assets/javascripts/controllers", __dir__ ), under: "controllers", preload: false

Open your engine.rb file and add an initializer:

initializer "maquina.importmap", before: "importmap" do |app|
  app.config.importmap.paths << Engine.root.join("config/importmap.rb")
end

What are the Advantages of Using Importmap?

From a Ruby on Rails developer perspective, the main advantage of using Importmap is the freedom from requiring a JavaScript runtime-like node and freedom from the node_modules dependency.

Additionally, you don’t need an additional process in development mode to transpile and minify the JavaScript code. You rely on web standards to serve the code to the browser. Deploying your Rails application behind a reverse proxy offers several benefits. First, if you enable the HTTP/2 protocol, your browser can fetch multiple files with a single HTTP connection, and downloading many small JavaScript files won’t impact performance.

Enabling your proxy to use gzip or brotli compression ensures you are sending very small files while maintaining readability when using browser developer tools. If you change one file, you only need to invalidate that specific file, which the browser will download. The browser knows that a file was modified because of the fingerprint that Propshaft adds to all files.

Using a reverse proxy like Thruster along with Puma offloads the assets serving from the Rails application. Thruster can cache assets and serve them when a client requests a file.

When Not to Use Importmap

There are cases where you should avoid using Importmap in a Rails application. If you are building a SPA application with React, Vue, or any other similar tool, there is a high likelihood you are writing your code with TypeScript. In this case, you should stick with the bundling strategy.

Additionally, if you need to support older browsers, bundling with code transpilation is a better option.

No build for Javascript libraries with Rails and Importmap

Mario Alberto Chávez — Fri, 09 Aug 2024 18:00:00 +0000

As a web developer working with Ruby on Rails, you’re always looking for ways to streamline your workflow and make your applications more efficient. Enter Importmap - a powerful feature that simplifies how you manage JavaScript libraries in your Rails projects. In this post, we’ll dive into what Importmap is, how it works, and how you can use it for your projects.

What is Importmap?

Importmap is a way to map JavaScript module names to their actual locations. They allow you to use import statements in your JavaScript code without needing a bundler like Webpack or Rollup. This means you can use modern JavaScript modules directly in the browser, making your development process simpler and faster.

How does Importmap work?

Mapping : Importmaps create a mapping between a module name and its location.
Browser Support : Modern browsers use this mapping to load modules when they encounter import statements.
Rails Integration : Rails 7+ includes built-in support for Importmap, making it easy to use in your projects.
Local Pinning : By default, Rails pins libraries locally, copying the code to your project into the vendor/javascript folder. While this might feel similar to how JavaScript libraries were vendored in older Rails projects, Importmap provides a modern tool for managing dependencies and versions.

Practical tips for using Importmap in Rails

1. Setting up Importmap

For a new Rails application, Importmap is set up by default. If you want to add it to an existing application, then follow these steps.

To get started with Importmap in your Rails project, add this to your Gemfile:

gem "importmap-rails"

Then run:

bundle install
bin/rails importmap:install

This sets up the necessary files and configurations.

2. Adding JavaScript libraries

To add a library, use the pin option with bin/importmap command. Let’s use Chart.js as an example:

> bin/importmap pin chart.js
Pinning "chart.js" to vendor/javascript/chart.js.js via download from https://ga.jspm.io/npm:chart.js@4.4.3/dist/chart.js
Pinning "@kurkle/color" to vendor/javascript/@kurkle/color.js via download from https://ga.jspm.io/npm:@kurkle/color@0.3.2/dist/color.esm.js

This command performs the following steps::

Download Chart.js and its dependencies.
Place the files in the vendor/javascript directory.
Update the config/importmap.rb file with the correct local paths.

This is the contents of vendor/javascript directory.

> ls vendor/javascript
8.5k 9 Aug 12:53 -N  @kurkle--color.js
186k 9 Aug 12:53 -N  chart.js.js

The config/importmap.rb looks as follows with the pinned libraries.

pin 'chart.js' # @4.4.3
pin '@kurkle/color', to: '@kurkle--color.js' # @0.3.2

Now, let’s create a Stimulus controller to use Chart.js. First, make sure you have Stimulus installed in your Rails application. Then, create a new file app/javascript/controllers/chart_controller.js:

import { Controller } from "@hotwired/stimulus"
import { Chart, registerables } from 'chart.js'

Chart.register(...registerables)

export default class extends Controller {
  static targets = ['canvas']

  connect() {
    const ctx = this.canvasTarget.getContext('2d')
    this.chart = new Chart(ctx, {
  type: 'line',
  data: {
   labels: ['January', 'February', 'March', 'April', 'May', 'June'],
   datasets: [{
    label: 'Sample Data',
    data: [65, 59, 80, 81, 56, 55],
    borderColor: 'rgba(75, 192, 192, 1)',
    backgroundColor: 'rgba(75, 192, 192, 0.2)',
    borderWidth: 2
   }]
  },
  options: {
   scales: {
    y: {
     beginAtZero: true
    }
   }
  }
 })
  }

  disconnect() {
    this.chart.destroy()
  }
}

Now you can use this Stimulus controller in your Rails view. For example, in app/views/home/index.html.erb:

<div data-controller="chart">
  <canvas data-chart-target="canvas" width="400" height="200"></canvas>
</div>

This setup allows you to easily create charts in your Rails application using Chart.js through a Stimulus controller. The chart data can be dynamically passed from your Rails backend to the frontend using Stimulus values.

But if you try to load the page, the graph is not displayed. After looking at the developer tools in the browser, you will find the following error:

Failed to register controller: chart (controllers/chart_controller)
  TypeError: Importing a module script failed.

The error is not helpful, but it gives you a clue that there is a problem with the dependencies in the Stimulus chart_controller. Looking at the network tab in the browser tools, you find out that there is a missing file.

URL: http://localhost:3000/_/6La3kzg5.js
Status: 404 Not Found
Source: Network

Looking at the code of the file vendor/javascript/chart.js.js, almost at the beginning of the file, you find the following reference:

import{r as t,c ...
... as Jt}from"../_/6La3kzg5.js" ...

It seems like jspm.io adds a dependency which Importmap is unable to identify and download with the rest of the files. This is a known issue documented in the Importmap repository, as Download all the files associated with a package from a CDN.

3. Pinning from CDNs

While Importmap pins libraries locally by default, it also allows you to pin from a CDN instead. Using this method, the library is served directly from the CDN instead of our Rails application.

To try to find a solution, change the pins in the config/importmap.rb file to point to the CDN:

# Using JSPM.io (default)
pin 'chart.js', to: 'https://ga.jspm.io/npm:chart.js@4.3.0/dist/chart.js'
pin '@kurkle/color', to: 'https://ga.jspm.io/npm:@kurkle/color@0.3.2/dist/color.esm.js'

After making the change, reload the page. Now the graph is there, loading the libraries from the CDN solved the issue of missing references. If you are ok with this approach, then you can stop here.

If your goal is to serve and cache the libraries without a dependency on external CDN, then continue to the next section.

4. Customizing local pinning

With the option to choose a CDN from which to vendor JavaScript libraries, you should try another CDN, like JSDeliver. Each CDN bundles dependencies differently, so you might get lucky trying another option.

Remove all files from vendor/javascript and remove the pins to Chart.js and @kurkle/color from config/importmap.rb. The Importmap’s pin command accepts option -f to specify a specific CDN different from JSpm.io.

The first try to pin Chart.js fails with an error.

> bin/importmap pin chart.js -f jsdelivr
Pinning "chart.js" to vendor/javascript/chart.js.js via download from https://cdn.jsdelivr.net/npm/chart.js@4.4.3/dist/chart.js
/.rbenv/versions/3.3.4/lib/ruby/3.3.0/net/http.rb:1603:in `initialize': Failed to open TCP connection to cdn.jsdelivr.net:443 (execution expired) (Net::OpenTimeout)

For the second try, add the library @kurkle/color to the pin command:

> bin/importmap pin chart.js @kurkle/color -f jsdelivr
Pinning "@kurkle/color" to vendor/javascript/@kurkle/color.js via download from https://cdn.jsdelivr.net/npm/@kurkle/color@0.3.2/dist/color.esm.js
Pinning "chart.js" to vendor/javascript/chart.js.js via download from https://cdn.jsdelivr.net/npm/chart.js@4.4.3/dist/chart.js

This time it succeeded, the new files are placed in vendor/javascript and the file config/importmap.rb contains the pins. Reloading the page in your application shows that you have the same error as before, the missing file name is different but still the same result.

It is time to try something different. Remove the files from vendor/javascript one more time. Navigate to that directory and open your browser to https://www.jsdelivr.com/. Search for Chart.js and from the code block copy the URL and download the library using curl. Also, search for @kurkle/color and download it.

> curl -o chart.js.js 'https://cdn.jsdelivr.net/npm/chart.js@4.4.3/+esm'
> curl -o @kurkle--color.js 'https://cdn.jsdelivr.net/npm/@kurkle/color@0.3.2/+esm'

Once again, reload your application page. One more time you will get an error in the browser developer tools, but this time the error is different. It says that @kurkle/color failed to be loaded.

[Error] Failed to load resource: the server responded with a status of 404 (Not Found) http://localhost:3000/npm/@kurkle/color@0.3.2/+esm
[Error] Failed to register controller: chart (controllers/chart_controller) – TypeError: Importing a module script failed.
TypeError: Importing a module script failed.
 error

The reference to npm/@kurkle/color@0.3.2/+esm appears at the beginning of chart.js.js file. You need to replace this reference for the @kurkle/color that you have pinned in config/importmap.rb. Use sed command to replace the value as follows:

sed -i '' 's|/npm/@kurkle/color@0.3.2/+esm|@kurkle/color|g' chart.js.js

After this change, reload the application page. Now the graph is rendered and there are no errors related to missing references.

Not in all cases when pinning libraries from a CDN you will have this problem, but if there is a case, now you know how to diagnose and try to fix it. I would expect that newer versions of Importmap will have a feature to understand and resolve hidden references that libraries might have.

Conclusion

Importmap in Ruby on Rails offers a streamlined approach to managing JavaScript libraries like Chart.js. By simplifying your workflow, leveraging the power of modern browsers, and providing local access to your dependencies, you can focus more on creating functionality and less on configuration. Give Importmap a try in your next Rails project and experience the benefits for yourself!

Remember, while Importmap is powerful and the default local pinning provides good version control, you still have the flexibility to use various CDNs when needed. Consider your specific needs and performance requirements when deciding on your Importmap strategy.

Sound to Script: Using OpenAI’s Whisper Model and Whisper.cpp

Mario Alberto Chávez — Sun, 10 Dec 2023 06:00:00 +0000

AI offers a different set of inputs and outputs for inferences. One of the many inference models is Automatic Speech Recognition (ASR). Using OpenAI’s Whiper model makes transcribing pre-recorded or live audio possible.

Whisper.cpp implements OpenAI’s Whisper model, which allows you to run this model on your machine. It could be done running your CPU, Apple’s Core ML from M processors, or using a dedicated GPU unit. You can run the smaller or larger Whisper model; Whisper.cpp also supports running quantized models, which require less memory and disk space using the GGML library.

So, let’s see how to use Whisper.cpp to process a video to get subtitle SRT format.

First, you need to clone the Whisper.cpp repository.

git clone [https://github.com/ggerganov/whisper.cpp](https://github.com/ggerganov/whisper.cpp)

If you are running Whisper.cpp from your CPU, change to the code folder and run the make command:

make

Then download a Whiper model in ggml format:

bash ./models/download-ggml-model.sh base.en

You can find available models here: https://huggingface.co/ggerganov/whisper.cpp

With the model in place, run the following command to test it with a sample audio in the repository.

./main -f samples/jfk.wav

You should see the model being loaded; at the end, it displays the inferred text from the audio.

main: processing 'samples/jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:11.000] And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.

Now, I’ll focus on getting Whisper.cpp to work with Apple’s silicon processor to get better performance at inference.

You need Python installed to prepare Whisper models for running with Apple’s Core ML. The best way to set up Python is to install it via Miniconda and create an environment for Whisper.

conda create -n py310-whisper python=3.10 -y
conda activate py310-whisper

With Python ready and activated, install the following dependencies for Core ML.

pip install ane_transformers
pip install openai-whisper
pip install coremltools

Next, generate a Core ML model off the downloaded base.en Whisper model. If you downloaded a different model, update the command to reflect that change.

./models/generate-coreml-model.sh base.en

Finally, you need to compile Whisper.cpp with Core ML support.

make clean
WHISPER_COREML=1 make -j

Running the Core ML model with Whisper.cpp Core ML support produces faster inference.

./main -m models/ggml-base.en.bin -f samples/jfk.wav

With these tools ready, you can move to a different scenario where the audio needs to be extracted from a video, passed to Whisper.cpp, and produced the subtitle file in SRT format.

To extract audio from a video, ffmpeg is the best tool for this job. Whisper.cpp needs audio in 16-bit format.

ffmpeg -i video.mp4 -ar 16000 -ac 1 -c:a pcm_s16le video.wav

The output is a WAV audio file that you can use to produce a transcription into a JSON file.

./main -m models/ggml-base.en.bin -f video.wav -oj -ojf video
....

    "params": {
            "model": "models/ggml-medium.en-q5_0.bin",
            "language": "en",
            "translate": false
    },
    "result": {
            "language": "en"
    },
    "transcription": [
            {
                    "timestamps": {
                            "from": "00:00:00,720",
                            "to": "00:00:08,880"
                    },
                    "offsets": {
                            "from": 720,
                            "to": 8880
                    },
                    "text": " Hi, everyone. Would you let people in why? Okay. Yes. My name is Selma. For those of you that don't",
                    "tokens": [
                            {
                                    "text": " Hi",
                                    "timestamps": {
                                            "from": "00:00:00,000",
                                            "to": "00:00:00,240"
                                    },
                                    "offsets": {
                                            "from": 0,
                                            "to": 240
                                    },
                                    "id": 15902,
                                    "p": 0.882259
                            },
                            {
                                    "text": ",",
                                    "timestamps": {
                                            "from": "00:00:00,240",
                                            "to": "00:00:00,470"
                                    },
                                    "offsets": {
                                            "from": 240,
                                            "to": 470
                                    },
....

This JSON file can be transformed to meet our needs. In our case, we want to produce an SRT format for video player subtitles. This last part is done with a Ruby script.

json_data = JSON.parse(File.read(json_file_path))

transcription = json_data['transcription']
srt_content = ""

transcription.each_with_index do |entry, index|
  from_time = entry['timestamps']['from']
  to_time = entry['timestamps']['to']
  text = entry['text']

  srt_content += "#{index + 1}\\n"
  srt_content += "#{from_time} --> #{to_time}\\n"
  srt_content += "#{text}\\n\\n"
end

File.write(srt_file_path, srt_content)

It depends on the length of the extracted audio file; this process can take a few seconds or several minutes to complete.

The following is a Ruby script performs all three actions at once: extract audio, transcribe, and transform into SRT file format.

require 'json'
require 'tmpdir'

def extract_audio(dir, input_video_path)
  # Generate temporary WAV file path based on the input video
  temp_wav_file_path = "#{dir}/#{File.basename(input_video_path, '.*')}.wav"

  # Construct the ffmpeg command
  ffmpeg_command = "ffmpeg -i '#{input_video_path}' -ar 16000 -ac 1 -c:a pcm_s16le '#{temp_wav_file_path}'"

  # Execute the ffmpeg command
  puts ffmpeg_command
  system(ffmpeg_command)

  # Check if the command was successful
  if $?.success?
    puts "Audio extracted successfully to #{temp_wav_file_path}"
    return temp_wav_file_path
  else
    puts "Error extracting audio. Please check your ffmpeg installation and the input video file."
    exit(1)
  end
end

def process_wav(dir, wav_file_path)
  # Path to the main command and binary file
  path = '~/Development/llm/whisper.cpp/'
  main_command = 'main'
  model_file = 'models/ggml-medium.en-q5_0.bin'

  # Generate temporary JSON file path based on the WAV file
  temp_json_file_path = "#{dir}/#{File.basename(wav_file_path, '.*')}"

  # Construct the full command with quotes around file paths
  full_command = "#{path}#{main_command} -m #{path}#{model_file} -f '#{wav_file_path}' -oj -ojf '#{temp_json_file_path}'"

  # Execute the command
  puts full_command
  system(full_command)

  # Check if the command was successful
  if $?.success?
    puts "Processing completed successfully for #{wav_file_path}"
    return "#{temp_json_file_path}.wav.json"
  else
    puts "Error processing the WAV file. Please check your command and the input file."
    exit(1)
  end
end

def process_json_to_srt(json_file_path, srt_file_path)
  json_data = JSON.parse(File.read(json_file_path))

  # Extract 'transcription' array from JSON
  transcription = json_data['transcription']
  srt_content = ""

  transcription.each_with_index do |entry, index|
    from_time = entry['timestamps']['from']
    to_time = entry['timestamps']['to']
    text = entry['text']

    srt_content += "#{index + 1}\n"
    srt_content += "#{from_time} --> #{to_time}\n"
    srt_content += "#{text}\n\n"
  end

  # Write SRT content to the new file
  File.write(srt_file_path, srt_content)

  puts "SRT file created at #{srt_file_path}"
end

# Check if the video file path is provided as a command-line argument
if ARGV.empty?
  puts "Usage: ruby combined_script.rb path/to/your/video.mp4"
  exit(1)
else
  video_path = ARGV[0]

  Dir.mktmpdir do |dir|
    # Extract audio
    wav_file_path = extract_audio(dir, video_path)

    # Process audio to obtain JSON transcript
    json_file_path = process_wav(dir, wav_file_path)

    # Convert JSON to SRT
    srt_file_path = "#{File.dirname(video_path)}/#{File.basename(video_path, '.*')}.srt"
    process_json_to_srt(json_file_path, srt_file_path)
  end
end

It needs a few changes in the process_wav method.

First, you need to update the path to your Whisper binaries. And second, the relative path to the binaries of your model. After these changes, you can create subtitles for your video with the following command.

ruby transcribe.rb video.mp4

After script completion, you will have a video.srt file next to your video file.

Full-text search with SQLite and Rails

Mario Alberto Chávez — Fri, 01 Sep 2023 06:00:00 +0000

Today, I planned to move a few small Ruby on Rails applications with Heroku using Kamal onto a single server. All applications use PostgreSQL as the database because it is simple to use with Heroku and because they take advantage of Postgres’ full-text search.

Because of the size and usage of those applications, I was not convinced that I wanted to manage my own PostgreSQL server or pay for a managed service. So, I started to consider replacing the database with SQLite. At least in my feed at X, I have seen so many posts in the past few months on how good and fast it is.

I wanted something like the pg_search gem, where I could define a scope name and the fields to be indexed. On the SQLite side, I found that it has an extension called FTS5, which is defined as:

FTS5 is an SQLite virtual table module that provides full-text search functionality to database applications. In their most elementary form, full-text search engines allow the user to efficiently search a large collection of documents for the subset that contains one or more instances of a search term. The search functionality provided to World Wide Web users by Google is, among other things, a full-text search engine, as it allows users to search for all documents on the web that contain, for example, the term “fts5”.

This looked promising, so I started implementing a quick and dirty solution for my needs. Starting from a model Post with title and content attributes, I want to define a full_search scope and indicate the attributes to index for full-text search.

Something like this:

class Post < ApplicationRecord
  include SqliteSearch

  search_scope(:title, :content)
end

Where SqliteSearch is the module that does the heavy work. Before continuing with the implementation, there are decisions to be made regarding how to store the indexed data in the database. A virtual table is required to store the indexed data; one option is to create a single table for this purpose with a record_type and record_id fields to identify the data per model type, just like the way it works for ActiveStorage or ActionText in Rails.

The second option is to create a table per indexed model. I chose this option since not all my models require this functionality. The migration for this table is as follows:

class PostSearch < ActiveRecord::Migration[7.0]
  def up
    execute("CREATE VIRTUAL TABLE fts_posts USING fts5(title, content, post_id)")
  end

  def down
    execute("DROP TABLE IF EXISTS fts_posts")
  end
end

The table name follows the Rails convention, but it adds the prefix fts_. The table fields are the ones that need indexing with the addition of the original record id with the foreign key convention. SQLite virtual tables don’t have data types, primary keys, constraints, or indexes.

The SqliteSearch module needs to implement a way to add or update the model data to the search index. This is done using the ActiveRecord callbacks for save and destroy commit.

module SqliteSearch
  extend ActiveSupport::Concern

  private def update_search_index
    primary_key = self.class.primary_key
    table_name = self.class.table_name
    foreign_key = self.class.to_s.foreign_key

    search_attrs = @@search_scope_attrs.each_with_object({}) { |attr, acc|
      acc[attr] = quote_string(send(attr) || "")
    }
    id_value = attributes[primary_key]

    sql_delete = <<~SQL.strip
      DELETE FROM fts_#{table_name} WHERE #{foreign_key} = #{id_value};
    SQL
    self.class.connection.execute(sql_delete)

    sql_insert = <<~SQL.strip
      INSERT INTO fts_#{table_name}(#{search_attrs.keys.join(", ")}, #{foreign_key})
      VALUES (#{search_attrs.values.map { |value| "'#{value}'" }.join(", ")}, #{attributes[primary_key]});
    SQL
    self.class.connection.execute(sql_insert)
  end

  private def delete_search_index
    primary_key = self.class.primary_key
    table_name = self.class.table_name
    foreign_key = self.class.to_s.foreign_key
    id_value = attributes[primary_key]

    sql_delete = <<~SQL.strip
      DELETE FROM fts_#{table_name} WHERE #{foreign_key} = #{id_value};
    SQL
    self.class.connection.execute(sql_delete)
  end

  included do
    after_save_commit :update_search_index
    after_destroy_commit :delete_search_index

    scope_foreign_key = to_s.foreign_key
    scope :full_search, ->(query) {
      return none if query.blank?

      sql = <<~SQL.strip
        SELECT #{scope_foreign_key} AS id FROM fts_#{table_name}
        WHERE fts_#{table_name} = '#{query}' ORDER BY rank;
      SQL
      ids = connection.execute(sql).map(&:values).flatten
      where(id: ids)
    }
  end

  class_methods do
    def search_scope(*attrs)
      @@search_scope_attrs = attrs
    end

    def rebuild_search_index(*ids)
      target_ids = Array(ids)
      target_ids = self.ids if target_ids.empty?

      scope_foreign_key = to_s.foreign_key

      delete_where = Array(ids).any? ? "WHERE #{scope_foreign_key} IN (#{ids.join(", ")})" : ""
      sql_delete = <<~SQL.strip
        DELETE FROM fts_#{table_name} #{delete_where};
      SQL
      connection.execute(sql_delete)

      target_ids.each do |id|
        record = where(id: id).pluck(*@@search_scope_attrs, :id).first
        if record.present?
          id = record.pop

          sql_insert = <<~SQL.strip
            INSERT INTO fts_#{table_name}(#{@@search_scope_attrs.join(", ")}, #{scope_foreign_key})
            VALUES (#{record.map { |value| "'#{quote_string(value)}'" }.join(", ")}, #{id});
          SQL
          connection.execute(sql_insert)
        end
      end
    end

    def quote_string(s)
      s.gsub("\\", '\&\&').gsub("'", "''")
    end
  end
end

The class method search_scope tells the module the attributes we need to index. The update_search_index callback is called when the record is created or updated. Since SQLite, virtual tables don’t support upsert, which is a way to tell the database to insert a record if it doesn’t exist or to update it if it does. Also, the Rails SQLite adapter does not support multiple statements in a single execution call.

These limitations forced me to make two additional database calls: the first to delete the indexed data for a specific record, and the second to insert the new indexed data. It’s not very performant, but for a small database, it might be just fine. The delete_search_index callback removes the indexed data when a record is deleted.

The module also implements a class method, rebuild_search_index, which optionally receives an array of record ids to re-index. If no ids are passed, then it rebuilds the index for all records.

Finally, a scope full_search is added to fetch records based on the full-text search index. You can use keywords like “AND”, “OR,” or “NOT” to refine your search or any other special character supported by SQLite FTS5.

Post.full_search("Ruby OR Rails NOT Javascript")

Conclusion

Adding this module to my applications allowed me to explore the idea of moving from PostgreSQL for these small applications. Not because PostgreSQL is bad, but because it might be too much for the current application’s needs.

Working with Rails Engines, Importmap and TailwindCSS for assets.

Mario Alberto Chávez — Wed, 23 Aug 2023 06:00:00 +0000

Rails engines are one of my favorite tools when I want to isolate reusable functionality for Rails applications. An example of this is the NoPassword gem, which allows users to login into an application just with their email and the received link and code.

With Rails 3.1, the engines were mountable and integrated with the Asset Pipeline to manage the engine’s assets. This continued to work this way until Webpack was introduced to Rails in version 5.1. At this moment, it was not clear how to make Webpack work with the engine’s assets or any other gem that included assets.

Rails 7 introduced CSS and JS bundling, along with the Propshaft gem to manage and serve assets. Also, introduced Importmaps and TailwindCSS as options for not having Node as a dependency, but in both cases without any official word on how to make these work with engines.

NOTE: Importmap’s README explains how to composite multiple Impormap configurations. https://github.com/rails/importmap-rails#composing-import-maps

Working on the NoPassword engine and a few private others, I decided for them to use Importmaps, TailwindCSS, and Propshaft. It led me to figure out a way to use the engine assets. The rest of this post describes my successful journey with assets and engines.

Importmap engine configuration

First, let’s try the route described in Importmap’s README file.

To install Importmap in an engine project, it requires adding the gem to the dummy app and adding the dependency to the engine’s gem spec file.

Install Importmap and add it as a dependency to the Gemfile as follows:

bundle add importmap-rails

And add it to the gem spec file.

spec.add_dependency "importmap-rails", "~> 1.2", ">= 1.2.1"

Then run the bundle command and navigate to the test/dummy app to execute the installer in the dummy app.

cd test/dummy && bin/rails importmap::install

Following the README instructions for composite Importmaps, open the lib/my_engine/engine.rb file and add the following hook - remember to replace **** *\ ******* my-engine **** *\ ******* with the name of your engine -:

module MyEngine
  class Engine < ::Rails::Engine
    # ...
    initializer "my-engine.importmap", before: "importmap" do |app|
      app.config.importmap.paths << Engine.root.join("config/importmap.rb")
      # ...
    end
  end
end

Then create the file config/importmap.rb and pin the engine’s javascript assets.

pin_all_from File.expand_path("../app/assets/javascript", __dir__ )

To test this setup, two Stimulus controllers were added, one in the engine’s app/assets/javascript and another one at the dummy app test/dummy/app/javascript/controllers.

# app/javascript/controllers/host_controller.js (Dummy APP)
import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
  connect() {
    console.log("Connected")
    this.element.textContent = "Hello World! This is a Javascript from the Host"
  }
}

console.log("Loaded")

# app/assets/javascript/engine_controller.js (Engine)
import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
  connect() {
    console.log("Connected")
    this.element.textContent = "Hello World! This is a Javascript from the Engine"
  }
}

console.log("Loaded")

Set up Stimulus in the dummy app by adding the gem and running the installer.

cd test/dummy
bundle add stimulus-rails
./bin/rails stimulus:install

The Dummy app has a HomeController with an index action. The index template has two divs. One for host_controller.js and the second one for engine_controller.js .

<h1>Engine's Stimulus controller (Host app)</h1>
<div data-controller="host"></div>
<div data-controller="engine"></div>

Unfortunately, this setup does not work, at least for my use case, where I expect the engine to have Stimulus controllers available in the host application, the Dummy app in this case. The engine’s controller is there in the imports manifest but is not loaded in the context of the Stimulus application.

The way that I make this work is to first organize the engine’s Stimulus controllers with the following directory structure: app/assets/javascript/my_engine/controllers

Next, modify the Dummy app’s config/importmap.rb and add the following line at the end of the file.

pin_all_from MyEngine::Engine.root.join("app/assets/javascript/my_engine/controllers"), under: "controllers", to: "my_engine/controllers"

With this change, reload the page, and now it is working! The engine’s Stimulus controller is loaded by the host application.

In the Importmap repository, there is an open issue about how to make the gem work with engines, and the user muriloime mentions this solution https://github.com/rails/importmap-rails/issues/58

Now, what about making Importmap work with the engine’s own views? In the case of the NoPassword gem, it provides views for the login process and uses a Stimulus controller to display errors; it does not depend on the host application in any way.

First, add Stimulus as a gem dependency to the gem spec file and execute the bundle command.

spec.add_dependency "stimulus-rails", "~> 1.2"

The installer is not available inside the engine, so this step requires adding a few files manually. Let’s start with the javascript files.

# app/assets/javascript/my_engine/application.js
import "controllers"

# app/assets/javascript/my_engine/controllers/index.js
// Import and register all your controllers from the importmap under controllers/*

import { application } from "controllers/application"

// Eager load all controllers defined in the import map under controllers/**/*_controller
import { eagerLoadControllersFrom } from "@hotwired/stimulus-loading"
eagerLoadControllersFrom("controllers", application)

// Lazy load controllers as they appear in the DOM (remember not to preload controllers in import map!)
// import { lazyLoadControllersFrom } from "@hotwired/stimulus-loading"
// lazyLoadControllersFrom("controllers", application)

# app/assets/javascript/my_engine/controllers/application.js
import { Application } from "@hotwired/stimulus"

const application = Application.start()

// Configure Stimulus development experience
application.debug = false
window.Stimulus = application

export { application }

Open the file config/importmap.rb and replace the content with the following pins that load Stimulus and the engine’s controllers:

pin "@hotwired/stimulus", to: "stimulus.min.js", preload: true
pin "@hotwired/stimulus-loading", to: "stimulus-loading.js", preload: true

pin "application", preload: true

pin_all_from MyEngine::Engine.root.join("app/assets/javascript/my_engine/controllers"), under: "controllers", to: "my_engine/controllers"

Instead of composing a global Importmap, we are going to set up a local configuration for the engine. Open the lib/my_engine.rb file and add the following code:

require "my_engine/version"
require "my_engine/engine"

require "importmap-rails"

module MyEngine
  class << self
    attr_accessor :configuration
  end

  class Configuration
    attr_reader :importmap

    def initialize
      @importmap = Importmap::Map.new
      @importmap.draw(Engine.root.join("config/importmap.rb"))
    end
  end

  def self.init_config
    self.configuration ||= Configuration.new
  end

  def self.configure
    init_config
    yield(configuration)
  end
end

MyEngine.init_config

Here, a configuration class was added to the engine. It has only one setting, where it initializes a new Importmap with assets belonging to the engine. Be aware that sweepers are not set up for Importmap; this means that changes made during development with the engine will not be taken until the app is restarted. This configuration can be extended to include more engine settings if needed.

Now we need to create a helper similar to javascript_importmap_tag that is aware of the engine’s Importmap configuration. Open the app/helpers/my_engine/application_helper.rb and add the following method.

def my_engine_importmap_tags(entry_point = "application", shim: true)
  safe_join [
    javascript_inline_importmap_tag(MyEngine.configuration.importmap.to_json(resolver: self)),
    javascript_importmap_module_preload_tags(MyEngine.configuration.importmap),
    (javascript_importmap_shim_nonce_configuration_tag if shim),
    (javascript_importmap_shim_tag if shim),
    javascript_import_module_tag(entry_point)
  ].compact, "\n"
end

Now open the engine’s layout and replace the javascript_importmap_tag with my_engine_importmap_tags.

To test this, add a controller named MyEngine::HomeController with an index action and a div that uses engine_controller.js to it.

These are the two configuration options for Importmap with engines. Share the engine’s Stimulus controllers with the host app, or use the engine’s Stimulus controllers internally for the engine’s templates.

TailwindCSS engine configuration

The steps to install it are simple. First, let’s install it into the Dummy app.

cd test/dummy && bundle add tailwindcss-rails
./bin/rails tailwindcss:install

For the Dummy app that comes with the engine in development mode, there are two additional steps that are not required when the host app is not the Dummy app.

Ensure that the TailwindCSS task runs with Foreman or Overmind using your Procfile.dev.

web: bin/rails server -p 3000
css: bin/rails app:tailwindcss:watch

Also, change the test/dummy/config/tailwind.config.js file that was installed by TailwindCSS to have the right path to the Dummy app. Change the content section to include the ./test/dummy path.

content: [
    './test/dummy/public/*.html',
    './test/dummy/app/helpers/**/*.rb',
    './test/dummy/app/javascript/**/*.js',
    './test/dummy/app/views/**/*.{erb,haml,html,slim}'
  ],

Restart your application, and it should work for the Dummy app.

To extend the TailwindCSS styling into the engine’s templates, there are a few steps that need to be taken first. The host app, in our case, the Dummy app, needs to override the engine’s layout to include the TailwindCSS files. Copy the file app/layouts/my_engine/application.html.erb to test/dummy/app/layouts/my_engine/application.html.erb and add the following line in the head section:

<%= stylesheet_link_tag "tailwind", "inter-font", "data-turbo-track": "reload" %>

Navigate to an engine action, and you can confirm the general CSS reset of TailwindCSS is applied, but utility classes are ignored.

This happens because the file doesn’t know that it also needs to scan the engine templates for TailwindCSS classes.

To fix this, we need a rake task and a generator in the host or Dummy app. Create a file test/dummy/lib/tasks/tailwind.rake and add the following content:

# frozen_string_literal: true

namespace :tailwindcss do
  desc "Generates your tailwind config file"
  task :config do
    Rails::Generators.invoke("tailwind_config", ["--force"])
  end
end

Rake::Task["tailwindcss:build"].enhance(["tailwindcss:config"])
Rake::Task["tailwindcss:watch"].enhance(["tailwindcss:config"])

Here we are adding a tailwindcss:config task that invokes a generator (we are going to create this one in a moment) and enhancing TailwindCSS tasks provided by the gem by injecting this new task into the build process.

For the generator, create the file test/dummy/lib/generators/tailwind_config_generator.rb in the host or Dummy app and add the following code:

# frozen_string_literal: true

class TailwindConfigGenerator < Rails::Generators::Base
  source_root File.expand_path("../templates", __FILE__ )

  def create_tailwind_config_file
    @engines_paths = MyEngine.configuration.tailwind_content

    # The second parameter for the template method is required only if the host app is the Dummy app; 
    # for an external host app, remove that parameter.
    template "config/tailwind.config.js", File.expand_path("../../../config/tailwind.config.js", __FILE__ )
  end
end

This generator, when executed, creates a new TailwindCSS config file from a template, but before doing that, in the create_tailwind_config_file method, we can set up the engine paths to consider when scanning for TailwindCSS classes.

In the code, the generator assumes that our engine configuration exposes a tailwind_content accessor with an array of paths.

Add an accessor to the Configuration class in file lib/my_engine.rb.

attr_accessor :tailwind_content

Also add to the initializer method of the same class the following paths:

@tailwind_content = [
  "#{MyEngine::Engine.root}/app/views/**/*",
  "#{MyEngine::Engine.root}/app/helpers/**/*",
  "#{MyEngine::Engine.root}/app/controllers/**/*",
  "#{MyEngine::Engine.root}/app/javascript/**/*.js"
]

Moving back to the template file in the tailwind_config_generator, this is the TailwindCSS config file, but it will be created dynamically by the task to be able to inject the engine paths into it.

Move the file test/dummy/config/tailwind.config.js to lib/generators/templates/config/tailwind.config.js.tt and modify the content section as follows:

content: [
    './test/dummy/public/*.html',
    './test/dummy/app/helpers/**/*.rb',
    './test/dummy/app/javascript/**/*.js',
    './test/dummy/app/views/**/*.{erb,haml,html,slim}',
    <%= @engines_paths.map{ |path| "'#{path}'" }.join(",\n") %>
  ],

Add the file test/dummy/config/tailwind.config.js to .gitignore file, since this file will be generated automatically.

Restart the application, navigate the view in the host app and a view in the engine, and confirm that TailwindCSS is working for both cases.

Wrapping up

This is how I have been working with the Rails engines Importmap and TailwindCSS. Most of the knowledge here came from reading the source code of the libraries and trial and error.

If you are looking to work with engines and this tool set for assets, please give it a try and let me know how it goes or if there is a better, simpler way to archive the same.

DEV Community: Mario Alberto Chávez

Rails MCP Server: Context-Efficient Tool Architecture

Table of Contents

The Context Budget Problem

Progressive Tool Discovery

Rails Introspection: Beyond Regex Parsing

Prism Static Analysis

Detail Levels and Filtering

Sandboxed Ruby Execution

Improved Discovery UX

Quick Start Guide

The puts Problem

AI Agent Guide

Interactive Configuration Tool

Getting Started

Project Management

Documentation Guides

Claude Desktop Integration

Enhanced Terminal UI

How I Use the New Architecture

Upgrading to the New Version

Looking Forward

Related Posts

Rails Upgrades with AI: A Real-World Success Story

The Real Power of the Skill 💡

Related posts

Vibecoding the Physical: How AI Helped Me Bind My Photobook

The Artist’s Need

The AI Solution

The Engineer’s Return

The Takeaway

Upgrading Rails applications with an AI skill

The Spark of an Idea

Introducing the rails-upgrade-skill

Make It Your Own

How to Use the Skill: A Step-by-Step Guide

1. Installation

2. Set Your Project Context

3. Activate the Upgrade Skill

4. The Three-Phase Process

The Final Mile: After the Skill

Final Thoughts

Rails MCP Server: Enhanced Documentation Access

Rails MCP Server: Enhanced Documentation Access and AI Workflow Integration

Table of Contents

Comprehensive Documentation Resources

The Documentation Challenge

Five Resource Categories Available

1. Rails Guides Documentation

2. Turbo Framework Documentation

3. Stimulus JavaScript Framework Documentation

4. Kamal Deployment Documentation

5. Custom Documentation Resources

Setting Up Documentation Resources

MCP Proxy Integration for Better Compatibility

Setting Up MCP Proxy for Enhanced Compatibility

Step 1: Start Rails MCP Server in HTTP Mode

Step 2: Install and Configure MCP Proxy

Step 3: Configure Claude Desktop for Proxy Usage

Rails MCP + Neovim MCP Integration Workflow

The Two-Server Development Approach

Optimized Development Session Workflow

Benefits of the Dual-Server Approach

Managing Token Usage Effectively

Documentation as a Shared Resource

Resource Management and Best Practices

Resource Organization Best Practices

Resource Storage and Management

Performance Optimization Tips

FAQ

How do I update Rails MCP Server documentation resources?

Can I use Rails MCP Server with other AI clients besides Claude Desktop?

What’s the difference between Rails MCP Server and Neovim MCP Server?

How do I troubleshoot Ruby version manager issues with Claude Desktop?

Can I import custom documentation alongside official Rails guides?

Looking Forward

Related Posts

Rails MCP Server with HTTP Server-Sent Events support

Rails MCP Server 1.1.0 with HTTP SSE support

What’s New in v1.1.0

The `puts` Problem

Introducing the `rails-upgrade-skill`