The Most Important Task For Every OSS Maintainer

Yuki Inoue — Tue, 16 Apr 2019 01:32:49 +0000

Is to release the library whenever new feature or bugfix is merged into the master.

More descriptively, on modern development environment OSS is generally hosted on the Github. If so, they are likely to be adopting (explicitly or implicitly) the development flow of Github flow. In this style of development, every commit against master branch is a release candidate. Since it is a release candidate, it can always be released as pre-release version, if the versioning system adopted by the language library ecosystem supports Semantic Versioning.

When the library maintainer keep publishing whatever is merged into master, what the library user need to do is to manage libraries systematically according to their needs. Since Semantic Versioning defines linear ordering against every released version, one can expect the library managing tool (e.g. npm, gem) of each language to do those management well. On the other hand, when some feature/bugfix is merged into develop but not yet released, the library user generally need to handle "the source location to fetch specific library" (e.g. git url along with branch/tag/ref). Since all the other libraries specify their dependency by some constraints against the released version, that means the library user would have now manage versioning and source location for that library. I believe it is something that no library user wants.

I know that to maintain OSS, there many other tasks too. But releasing operation can be simply done if release policy is made. For example, just release every new master commit as new major/minor/patch release depending on the content of the commit made; major release if breaking changes, minor release if new feature, patch release if bug fix, do them as pre-release if you are not certain the commit won't break anything. If such policy is made, then what the maintainer needs to do can be made into just a single command, by automating the deploy procedure. By doing this, it can bring benefit to all the users of that library, IMHO... So, I sincerely hope every OSS maintainer kindly publishes their fabulous libraries regularly.

Integrating Jupyter Notebook and Rails

Yuki Inoue — Mon, 04 Mar 2019 16:45:17 +0000

If you have ever worked on Jupyter and Rails, you'd probably wanted to execute Rails codes inside the Jupyter Notebook. Although it was already possible to achieve it, the experience was not so good; not as slick as those developer experience you get when you are riding the rail.

To tackle this problem and make Rails Jupyter friendly, I've made a gem called jupyter_on_rails. With this gem, working on a Rails project will get helped by the power of Jupyter (Notebook).

What is achieved

By adding following gems to your Rails project's Gemfile and making the bundle install succeed,

gem 'jupyter_on_rails'
gem 'ffi-rzmq'

Following things are achieved:

rake jupyter:notebook command to start a Jupyter Notebook server with your Rails app's kernel installed at your project local.
- the iruby kernel configuration is confined to your project; no more troubles of global iruby configuration headache, if you know what I mean.
Some methods which makes it easy to operate ActiveRecord operations on Jupyter Notebook.

Here is the example screenshot of what you'll achieve:

Detail

Prerequisites

To use jupyter_on_rails, you need to have jupyter available at your project, and you need to have all the libraries needed for the installation of dependency gems to be success.

For jupyter command, you have 2 options. One is installing globally by using e.g. pip install jupyter, or by creating pipenv environment at your project root and having jupyter there. My personal recommendation is using the Pipenv. For those rubyists who is unfamiliar with pipenv, it is a bundler for python.

You can check whether Jupyter notebook is properly installed by executing:

$ jupyter notebook
# or, for those who use pipenv
$ pipenv run jupyter notebook

If jupyter notebook web page is opened, then the jupyter installation is success.

Next, you need to install the libraries needed for the dependencies of jupyter_on_rails and ffi-rzmq. For this, since I cannot remember what was exactly needed for these gems, please refer to each gem's installation instruction if any bundle install error occurs. I can tell that all the required libraries could be installed by using Homebrew if you're using Mac.

Adding `jupyter_on_rails` to mac and launching Jupyter

Once you have your jupyter ready, put following dependencies in your Gemfile.

gem 'jupyter_on_rails'
gem 'ffi-rzmq'

Then bundle install. Install libraries to make install success if necessary.

After that, you can execute following command, to launch a Jupyter which have your app's kernel installed.

$ bin/rake jupyter:notebook

Below is the image of what that command execution will be look like. At top terminal, you can see the rake jupyter:notebook command launching a jupyter server, at left bottom you see that when launching a new notebook, one can choose the rails app. At right bottom you see ApplicationRecord, one of the classes from the Rails context, can be used without any requires or other configurations; it should work out of the box. (If you have used rails console, then the kernel's context is exactly the same as that context.)

If you have hard time looking at the image due to it's resize caused by CDN, refer to the original image at https://raw.githubusercontent.com/Yuki-Inoue/jupyter_on_rails/master/the_screenshot.png .

ActiveRecord integrations

The jupyter_on_rails comes with some utility methods which helps you operate ActiveRecord on the notebook.

Most notable is the ActiveRecord::Relation#to_df method. This turns your ActiveRecord::Relation instance into Daru::DataFrame instance. For those unfamiliar with what a Dataframe is, it is a table like structure which can be used for further data manipulation. It is also Jupyter friendly. For those with python context, it is the pandas.DataFrame in ruby.

The image above shows how #to_df method works. 1. You can turn your model into DataFrame 2. The includes is taken into account when creating DataFrame 3. It works on the Relation.

So now, you have the DataFrame. Refer to official documentation for how you work with the instance. https://github.com/SciRuby/daru

After modifications, you'd may want to write DataFrame data back to your model. You can achieve it by Daru::DataFrame#write_model(SomeModel). Below is again the screenshot of what its execution will look like. Note that this is extension method against daru made by jupyter_on_rails.

Behind the scene

The rake jupyter:notebook command creates an .ipython configuration at the project root. It basically copies the iruby kernel configuration, and adds boot information there. All the information needed for the kernel to boot rails is stored as configuration for kernel. On each invocation of this kernel, config/application.rb is loaded, and application.initialize_environment! is called, so that Rails configuration is available.

Feedback, etc.

Below is the Github project of this gem. For bugs/feature requests, please submit them there.

https://github.com/Yuki-Inoue/jupyter_on_rails

Thanks for reading.

DEV Community: Yuki Inoue