Integrating Jupyter Notebook and Rails

If you have ever worked on Jupyter and Rails, you'd probably wanted to execute Rails codes inside the Jupyter Notebook. Although it was already possible to achieve it, the experience was not so good; not as slick as those developer experience you get when you are riding the rail.

To tackle this problem and make Rails Jupyter friendly, I've made a gem called jupyter_on_rails. With this gem, working on a Rails project will get helped by the power of Jupyter (Notebook).

What is achieved

By adding following gems to your Rails project's Gemfile and making the bundle install succeed,

gem 'jupyter_on_rails'
gem 'ffi-rzmq'

Following things are achieved:

• rake jupyter:notebook command to start a Jupyter Notebook server with your Rails app's kernel installed at your project local.
  • the iruby kernel configuration is confined to your project; no more troubles of global iruby configuration headache, if you know what I mean.
• Some methods which makes it easy to operate ActiveRecord operations on Jupyter Notebook.

Here is the example screenshot of what you'll achieve:

Detail

Prerequisites

To use jupyter_on_rails, you need to have jupyter available at your project, and you need to have all the libraries needed for the installation of dependency gems to be success.

For jupyter command, you have 2 options. One is installing globally by using e.g. pip install jupyter, or by creating pipenv environment at your project root and having jupyter there. My personal recommendation is using the Pipenv. For those rubyists who is unfamiliar with pipenv, it is a bundler for python.

You can check whether Jupyter notebook is properly installed by executing:

$ jupyter notebook
# or, for those who use pipenv
$ pipenv run jupyter notebook

If jupyter notebook web page is opened, then the jupyter installation is success.

Next, you need to install the libraries needed for the dependencies of jupyter_on_rails and ffi-rzmq. For this, since I cannot remember what was exactly needed for these gems, please refer to each gem's installation instruction if any bundle install error occurs. I can tell that all the required libraries could be installed by using Homebrew if you're using Mac.

Adding jupyter_on_rails to mac and launching Jupyter

Once you have your jupyter ready, put following dependencies in your Gemfile.

gem 'jupyter_on_rails'
gem 'ffi-rzmq'

Then bundle install. Install libraries to make install success if necessary.

After that, you can execute following command, to launch a Jupyter which have your app's kernel installed.

$ bin/rake jupyter:notebook

Below is the image of what that command execution will be look like. At top terminal, you can see the rake jupyter:notebook command launching a jupyter server, at left bottom you see that when launching a new notebook, one can choose the rails app. At right bottom you see ApplicationRecord, one of the classes from the Rails context, can be used without any requires or other configurations; it should work out of the box. (If you have used rails console, then the kernel's context is exactly the same as that context.)

If you have hard time looking at the image due to it's resize caused by CDN, refer to the original image at https://raw.githubusercontent.com/Yuki-Inoue/jupyter_on_rails/master/the_screenshot.png .

ActiveRecord integrations

The jupyter_on_rails comes with some utility methods which helps you operate ActiveRecord on the notebook.

Most notable is the ActiveRecord::Relation#to_df method. This turns your ActiveRecord::Relation instance into Daru::DataFrame instance. For those unfamiliar with what a Dataframe is, it is a table like structure which can be used for further data manipulation. It is also Jupyter friendly. For those with python context, it is the pandas.DataFrame in ruby.

The image above shows how #to_df method works. 1. You can turn your model into DataFrame 2. The includes is taken into account when creating DataFrame 3. It works on the Relation.

So now, you have the DataFrame. Refer to official documentation for how you work with the instance. https://github.com/SciRuby/daru

After modifications, you'd may want to write DataFrame data back to your model. You can achieve it by Daru::DataFrame#write_model(SomeModel). Below is again the screenshot of what its execution will look like. Note that this is extension method against daru made by jupyter_on_rails.

Behind the scene

The rake jupyter:notebook command creates an .ipython configuration at the project root. It basically copies the iruby kernel configuration, and adds boot information there. All the information needed for the kernel to boot rails is stored as configuration for kernel. On each invocation of this kernel, config/application.rb is loaded, and application.initialize_environment! is called, so that Rails configuration is available.

Feedback, etc.

Below is the Github project of this gem. For bugs/feature requests, please submit them there.

https://github.com/Yuki-Inoue/jupyter_on_rails

---

Thanks for reading.