Introducing rack-dev_insight: A New Development Tool for Rack (Rails) Application

Takahiro Ebato — Mon, 08 Jan 2024 14:28:45 +0000

Hi! I'm excited to introduce a tool I've been developing over the past few months: rack-dev_insight.
This tool, comprising a gem and a chrome extension, is designed to provide insights into your Rack applications.
While it's still in its early stages, it's already valuable to be added to your development toolbox.

What is rack-dev_insight?
Overview video of rack-dev_insight
Unique features of rack-dev_insight
- 1. Grouping SQL queries by CRUD operation
- 2. Grouping SQL queries by Normalized SQL
- 3. Detailed HTTP request / response data
- 4. Functional devtools panel
What is next?

What is rack-dev_insight?

rack-dev_insight is a simple, yet unique monitoring tool for Rack applications.
The rack-dev_insight gem serves as a middleware for recording and analyzing SQL queries and HTTP request / response data.
To visualize this data, Chrome Devtools panel is offered, making it a developer-friendly tool.
Currently, this tool is intended for use in development environments only.

Overview video of rack-dev_insight

Here is overview video of rack-dev_insight. Detailed explanations of its key features are described in next section.

Unique features of rack-dev_insight

Inspired by tools like rails_panel and rack-mini-profiler, rack-dev_insight stands out with its unique features:

1. Grouping SQL queries by CRUD operation

This feature basically creates a CRUD matrix.

What is CRUD matrix?

A CRUD matrix is a tabular representation showing how different entities (tables) are involved in CRUD (Create, Read, Update, Delete) operations in a business process. It's helpful for visualizing the interactions between various business processes and the tables involved.

For example, let's take a business process where a user updates a comment on a post. This action typically requires reading data from the users and posts tables and updating the comments table. The CRUD matrix would look like this:

Process \ Table	users	posts	comments
Update a comment	READ	READ	UPDATE

Visualization in Devtools panel

In devtools panel this scenario is displayed as follows.

Usefulness

This feature is particularly useful in complex scenarios where dozens of tables are involved or when working with unfamiliar APIs, helping you recognize exhaustive table lists involved, and understand data dependencies more clearly.

Details

When extracting CRUD operations for the matrix, the following subtle cases are considered:

Conversion of Table Alias Names to Original Table Names

In scenarios where table aliases are used, such as in the following SQL query,

SELECT a FROM t1 AS t1_alias

it is counted as READ on the original table t1, instead of the alias t1_alias.

Recognition of Read Operations in CREATE, UPDATE, DELETE Queries

Not just the primary operation but also implicit operations within a query are considered.
For instance, the following SQL query

INSERT INTO t1 (a) SELECT a FROM t2 INNER JOIN t3 USING (id) WHERE t3.b = 1

is counted as INSERT operation for t1, and READ operation for t2 and t3.

2. Grouping SQL queries by Normalized SQL

This feature groups SQL queries by Normalized SQL, which is a uniform format of SQL statement.

How Normalization works

Normalization works similarly to pt-fingerprint.

For instance, consider the following set of SQL queries:

SELECT * FROM users WHERE id = 1
SELECT * FROM users WHERE id = 2
SELECT * FROM users WHERE id = 3

These are normalized into a uniform format:

SELECT * FROM users WHERE id = ?

Visualization in Devtools panel

This is displayed in the Devtools panel as follows.

Usefulness

This feature is helpful to detect so-called N+1 queries, but not limited to SELECT statements.

Details

Other DML queries are normalized in the same way like the following:

INSERT INTO users (name) VALUES (?)
UPDATE users SET name = ? WHERE id = ?
DELETE FROM users WHERE id = ?

3. Detailed HTTP request / response data

rack-dev_insight captures detailed HTTP request and response data, displayed in a Devtools panel similar to Chrome's network panel.
It currently supports the net-http gem, with future plans to include more HTTP clients, GraphQL, and gRPC.

4. Functional devtools panel

The Devtools panel has the following features:

Editor integration

Open files directly in your editor by clicking their paths in the backtrace.
Set up path mapping between local and remote file systems for seamless navigation.

Recording control

Start or stop data recording, and flush recorded data, similar to the functionality of Chrome's network panel.

Sorting by columns

Easily sort data by clicking on column headers for quick organization and analysis.

What is next?

If you are interested in this tool, please try it out! I would love to hear your feedback.
The following enhancements are currently under consideration:

Additional options to normalization format
Recording only the specified block
Support for HTTP clients other than net-http
Support for GraphQL and gRPC
Support for other SQL dialects such as Microsoft SQL Server
Alerting and reporting
Background job (remote machines) support

Stay tuned for more updates. Your suggestions and ideas are always welcome.

Thank you for your time and interest!

DEV Community: Takahiro Ebato

Introducing rack-dev_insight: A New Development Tool for Rack (Rails) Application

Table of Contents

What is rack-dev_insight?

Overview video of rack-dev_insight

Unique features of rack-dev_insight

1. Grouping SQL queries by CRUD operation

What is CRUD matrix?

Visualization in Devtools panel

Usefulness

Details

Conversion of Table Alias Names to Original Table Names

Recognition of Read Operations in CREATE, UPDATE, DELETE Queries

2. Grouping SQL queries by Normalized SQL

How Normalization works

Visualization in Devtools panel

Usefulness

Details

3. Detailed HTTP request / response data

4. Functional devtools panel

Editor integration

Recording control

Sorting by columns

What is next?