DEV Community

Cover image for Sorbetting a gem, or the story of the first adoption
Vladimir Dementyev for Evil Martians

Posted on • Edited on • Originally published at evilmartians.com

Sorbetting a gem, or the story of the first adoption

NOTE: this post was written a few days after the first public Sorbet release (on 2019-06-20). If you're reading this months after that happened, most of the things described here could be no longer relevant.

Stripe has finally open-sourced their long-awaiting static type checker for Ruby—Sorbet. I've been waiting for this for more than a year since it was first announced.

To be honest, I'm not a big fan of type annotations, or more precisely, I'm on the "I hate type annotations" side.

But as a member of Ruby community, I'm glad to see something like this happening: this is a huge leap forward for Ruby itself (especially, compared to other evolutionary features like pipeline operator). Stripe team did a great job 👍!

After reading about Brandon's first impression (highly recommend to check it out), I decided to give Sorbet a try and integrate it into one of my gems.

tl;dr type checker works great, but not ideal; annotations are ugly; tooling leaves much to be desired.

Rubanok meets Sorbet

I decided to use rubanok for this experiment: it's the simplest gem of mine in terms of the amount of code and metaprogramming.

You can find the adoption PR here: https://github.com/palkan/rubanok/pull/5.

bundle exec srb init

The first phase of the adoption is to add sorbet gem to the bundle and generate all the required files (read more and see the screenshots of this process here).

Unfortunately, it didn't go so well:

$ bundle exec srb init

👋 Hey there!

...

Traceback (most recent call last):
    15: from /ruby/gems/2.6.0/gems/sorbet-0.4.4280/bin/srb-rbi:232:in `<main>'
         ...
     3: from /ruby/gems/2.6.0/gems/sorbet-0.4.4280/lib/gem_loader.rb:553:in `require'
     2: from /ruby/gems/2.6.0/gems/rspec-rails-3.8.2/lib/rspec-rails.rb:4:in `<top (required)>'
     1: from /ruby/gems/2.6.0/gems/rspec-rails-3.8.2/lib/rspec-rails.rb:6:in `<module:RSpec>' /ruby/gems/2.6.0/gems/rspec-rails-3.8.2/lib/rspec-rails.rb:8:in `<module:Rails>':
     uninitialized constant Rails (NameError)
Enter fullscreen mode Exit fullscreen mode

RSpec Rails refers to Rails constant in its code but doesn't require rails anywhere in the code (it should be somewhere here).

As far as I understand, Sorbet evaluates each file for building .rbi configs for your bundle and it doesn't know how to handle such exceptions.

That's an edge case that should be solved at the rspec-rails side (would you like to open a PR?).

There is a similar open issue regarding optional dependencies as well.

Adding rails as a dev dependency helped to solve this issue (honestly, don't why 🤷‍♂️).

After that, a new folder, sorbet/, appeared in the project directory with a bunch of .rbi files:

$ find sorbet -type f | wc -l
      55
Enter fullscreen mode Exit fullscreen mode

As the documentation suggests, you should put this directory into your version control system. Even though type signatures are not so heavy (as node_modules/, for example), I hope this will change in the future. The raw size (w/o .git/) of rubanok increased by 2.1MB, from 124KB to ~2.2MB.

bundle exec srb tc

When you first run the type checking command, you should not see any errors:

$ bundle exec srb tc
No errors! Great job.
Enter fullscreen mode Exit fullscreen mode

That's because Sorbet adds the # typed: false magic comment to all the source files by default. This strictness level only checks for critical problems (constants, syntax, invalid signatures).

Gradual type checking implies that you make your source code type-aware step-by-step by changing # typed: false to # typed: true (or even # typed: strong).

Problem #1. Unsupported features.

I started by enabling type checks for the core Rubanok class—Rule:

$ be srb tc
lib/rubanok/rule.rb:62: Method Rubanok::Rule#empty? redefined without matching argument count. Expected: 0, got: 1 https://srb.help/4010
    62 |    def empty?(val)
            ^^^^^^^^^^^^^^^
    lib/rubanok/rule.rb:56: Previous definition
    56 |        def empty?
Enter fullscreen mode Exit fullscreen mode

That's strange; I don't have duplicate method definitions, RuboCop could catch this easily. What made Sorbet think so? This:

using(Module.new do
  refine NilClass do
    def empty?
      true
    end
  end

  refine Object do
    def empty?
      false
    end
  end
end)

def empty?(val)
  return false unless Rubanok.ignore_empty_values

  val.empty?
end
Enter fullscreen mode Exit fullscreen mode

Looks like Sorbet doesn't recognize anonymous modules and treats their contents as the "parent" module contents. Hopefully, it seems to be an easy fix and a good first contribution:

Comment for #1010

Thank you for a great issue report with a great reproducer! For those interested to contribute, this should be super easy to fix: https://github.com/sorbet/sorbet/blob/master/dsl/ClassNew.cc is already handling Class.new, it should additionally handle Module.new.

Note that this problem has nothing with refinements themselves (although I think they are not supported at all and will not be anytime soon).

I quickly fixed this by moving the refinement outside of the Rule class. But at what cost? That made it less clear why and where we need this refinement. The code became a bit more entangled. And that's just the beginning...

Problem #2. Limitations of flow-sensitivity

After toggling # typed: true for another class, Plane, I found another interesting case:

$ bundle exec srb tc
lib/rubanok/plane.rb:50: Method <= does not exist on NilClass component of T.nilable(Class) https://srb.help/7003
    50 |          if superclass <= Plane
                     ^^^^^^^^^^^^^^^^^^^
  Autocorrect: Use `-a` to autocorrect
    lib/rubanok/plane.rb:50: Replace with T.must(superclass)
    50 |          if superclass <= Plane
                     ^^^^^^^^^^

lib/rubanok/plane.rb:51: Method rules does not exist on Class component of T.nilable(Class) https://srb.help/7003
    51 |            superclass.rules.dup
                    ^^^^^^^^^^^^^^^^

lib/rubanok/plane.rb:51: Method rules does not exist on NilClass component of T.nilable(Class) https://srb.help/7003
    51 |            superclass.rules.dup
                    ^^^^^^^^^^^^^^^^
  Autocorrect: Use `-a` to autocorrect
    lib/rubanok/plane.rb:51: Replace with T.must(superclass)
    51 |            superclass.rules.dup
                    ^^^^^^^^^^
Enter fullscreen mode Exit fullscreen mode

The "violating" code:

def rules
  return @rules if instance_variable_defined?(:@rules)

  @rules =
    if superclass <= Plane
      superclass.rules.dup
    else
      []
    end
end
Enter fullscreen mode Exit fullscreen mode

Is this a bug? Don't think so: as far as I know, the only case when superclass returns nil is BasicObject. Or if we redefine the .superclass method 😜

The suggested solution—using T.must(superclass)—is not a good fit: I don't want my code to have dirty hacks only to satisfy the type system.

I've tried to make Sorbet happy the other way—by unwrapping the superclass value:

def rules
  return @rules if instance_variable_defined?(:@rules)

  @rules =
    if superclass && superclass <= Plane
      superclass.rules.dup
    else
      []
    end
end
Enter fullscreen mode Exit fullscreen mode

That didn't have any effect—still the same errors. I've tried again:

def rules
  return @rules if instance_variable_defined?(:@rules)

  @rules =
    if superclass
      if superclass <= Plane
        superclass.rules.dup
      else
        []
      end
    else
      []
    end
end
Enter fullscreen mode Exit fullscreen mode

Still the same :( The last attempt (I thought):

def rules
  return @rules if instance_variable_defined?(:@rules)

  x = superclass
  @rules =
    if x
      if x <= Plane
        x.rules.dup
      else
        []
      end
    else
      []
    end
end
Enter fullscreen mode Exit fullscreen mode

Almost works:

$ bundle exec srb tc
lib/rubanok/plane.rb:53: Method rules does not exist on Class https://srb.help/7003
    53 |              x.rules.dup
                      ^^^^^^^
Errors: 1
Enter fullscreen mode Exit fullscreen mode

But why it cannot infer the superclass class from the x <= Plane check?
If you check the complete list of constructs that affect Sorbet’s flow-sensitive typing, you can find that only Class#< is supported but not Class#<< 🤷‍♂️

OK. Let's replace x <= Plane with x < Plane (this is actually a breaking change: someone could define global rules on Rubanok::Plane class itself, which is not a good idea but...).

Problem #3. Signatures vs modules.

Adding signatures for Rule and Plane went pretty smooth (LOC increased from 159 to 196). And I didn't have to change any code.

Then I turned on type checking for DSL modules, Mapping and Matching.

These modules implement the particular Rubanok transformations and extend the Rubanok::Plane class.

The first problem occurred with a pretty standard Ruby code. Here is the simplified example:

class Rule
  sig do
      params(
        fields: T::Array[Symbol],
        activate_on: T::Array[Symbol]
      ).void
    end
  def initialize(fields, activate_on: fields)
    # ...
  end
end

module Mapping
  def map(*fields, **options, &block)
    rule = Rule.new(fields, options)
  end
end
Enter fullscreen mode Exit fullscreen mode

This code raises the following type error:

$ be srb tc
lib/rubanok/dsl/mapping.rb:25: Passing a hash where the specific keys are unknown to a method taking keyword arguments https://srb.help/7019
    25 |        rule = Rule.new(fields, options)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^
Enter fullscreen mode Exit fullscreen mode

Seems legit: do not allow passing an arbitrary hash as the known keyword arguments.

Let's try to add a signature for the #map method using shapes:

sig do
  params(
    fields: Symbol,
    options: {
      activate_on: T::Array[Symbol]
    },
    block:  T.proc.void
  ).returns(Rule)
end
def map(*fields, **options, &block)
  rule = Rule.new(fields, options)
end
Enter fullscreen mode Exit fullscreen mode

(Expectedly) didn't help:

$ bundle exec srb tc
./lib/rubanok/mapping.rb:34: Passing a hash where the specific keys are unknown to a method taking keyword arguments https://srb.help/7019
    34 |    rule = Rule.new(fields, options)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  Got T::Hash[Symbol, {activate_on: T::Array[Symbol]}] originating from:
    ./lib/rubanok/mapping.rb:33:
    33 |  def map(*fields, **options, &block)
                           ^^^^^^^^^
Enter fullscreen mode Exit fullscreen mode

This Got T::Hash[Symbol, {activate_on: T::Array[Symbol]}] looks suspicious. Where did it found a hash with Symbol keys? No idea.

I gave up and duplicated the keywords for the #map method:

sig do
  params(
    fields: Symbol,
    activate_on: T::Array[Symbol],
    block: T.proc.void
  )
  .returns(T::Array[Rubanok::Rule])
end
def map(*fields, activate_on: fields, &block)
  rule = Rule.new(fields, activate_on: activate_on)
  # ...
end
Enter fullscreen mode Exit fullscreen mode

This doesn't seem right to me: now I need to think about keeping these signatures in sync in three different places (the type checker will definitely help here), there is a chance that I will lose this very crucial activate_on: fields default value (the type checker cannot help here).

If you know how to add signatures without changing the code itself—please, leave a comment!

The second problem with modules relates to the fact that they are only meant for extending the Rubanok::Plane class; thus they "know" a few things about the Plane API and use it. For example, they use #rules method:

def map(*fields, activate_on: fields, &block)
  rule = Rule.new(fields, activate_on: activate_on)
  # ...
  rules << rule
end
Enter fullscreen mode Exit fullscreen mode

Sorbet has no idea of our intent; hence, it reports this error:

lib/rubanok/dsl/mapping.rb:38: Method rules does not exist on Rubanok::DSL::Mapping https://srb.help/7003
    38 |        rules << rule
Enter fullscreen mode Exit fullscreen mode

I couldn't find anything similar to this situation in the docs, only the section devoted to interfaces turned out to be useful: I marked the module as abstract! and defined an abstract #rules method:

sig { abstract.returns(T::Array[Rubanok::Rule]) }
def rules
end
Enter fullscreen mode Exit fullscreen mode

It made this error disappeared. Bonus: see what happens if you remove or rename the Plane.rules method:

$ bundle exec srb tc
lib/rubanok/plane.rb:36: Missing definition for abstract method Rubanok::DSL::Mapping#rules https://srb.help/5023
    36 |    class << self
            ^^^^^
    lib/rubanok/dsl/mapping.rb:47: defined here
    47 |      def rules
              ^^^^^^^^^
Enter fullscreen mode Exit fullscreen mode

Problem #4. Metaprogramming.

Metaprogramming is what makes Ruby such a powerful language (and makes me love Ruby).

Ruby without metaprogramming is not Ruby.

On the other hand, this is one of the things that makes static type checking so tricky. I don't expect a type checker to be so smart and know how to deal with any meta stuff; I only need from it a canonical way to handle the situations like the one described below.

The #match method provided by Matching module generates dynamic methods, which rely on a couple of Plane instance methods:

define_method(rule.to_method_name) do |params = {}|
  clause = rule.matching_clause(params)
  next raw unless clause

  apply_rule! clause.to_method_name, clause.project(params)
end
Enter fullscreen mode Exit fullscreen mode

Sorbet didn't like it:

$ bundle exec srb tc
lib/rubanok/dsl/matching.rb:106: Method raw does not exist on Rubanok::DSL::Matching https://srb.help/7003
     106 |          next raw unless clause
                         ^^^

lib/rubanok/dsl/matching.rb:108: Method apply_rule! does not exist on Rubanok::DSL::Matching https://srb.help/7003
     108 |          apply_rule! clause.to_method_name, clause.project(params)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Errors: 3
Enter fullscreen mode Exit fullscreen mode

The trick with adding abstract methods didn't work out (since we need to add instance methods, not singleton ones).

Re-generating hidden definitions* didn't work either.

I didn't find anything better than adding a custom RBI file to the repo:

# sorbet/meta.rbi

# typed: true

module Rubanok::DSL::Matching
  sig { returns(T.untyped) }
  def raw
  end

  sig { params(method_name: String, data: T.untyped).returns(T.untyped) }
  def apply_rule!(method_name, data)
  end
end
Enter fullscreen mode Exit fullscreen mode

Yet another hack. "Enough," I thought, and didn't even try to enable type checking for Rails controller integration concern.

* I tried to check how hidden definitions work with a more straightforward example:

# typed: true

class A
  def x
    "y"
  end

  define_method(:xx) do
    x * 2
  end
end
Enter fullscreen mode Exit fullscreen mode

After running bundle exec srb rbi hidden-definitions I found the following line in sorbet/hidden-definitions/hidden.rbi:

class A
  def xx(); end
end
Enter fullscreen mode Exit fullscreen mode

So, Sorbet found this define_method. And, for some reason, it also changed # typed: true to # typed: false. After turning it back, I got:

$ bundle exec srb tc

lib/rubanok/a.rb:9: Method x does not exist on T.class_of(A) https://srb.help/7003
     9 |    x * 2
            ^
    lib/rubanok/a.rb:4: Did you mean: A#x?
     4 |  def x
          ^^^^^
Errors: 1
Enter fullscreen mode Exit fullscreen mode

As we see from the error message, Sorbet treats #xx as a class method. There is a relevant open issue: #64.

Problem #5. Runtime checks.

So far, I only tried to make static checks pass but haven't tried to run the code:

$ bundle exec rspec

NameError:
  uninitialized constant Rubanok::Rule::T 
Enter fullscreen mode Exit fullscreen mode

Right, we have signatures in our code, which we haven't loaded yet.

I added the require "sorbet-static" line to the main project file. And I was a bit surprised:

$ bundle exec rspec

LoadError:
  cannot load such file -- sorbet-static
Enter fullscreen mode Exit fullscreen mode

My bad: I assumed that you could use Sorbet without runtime checks and that's what sorbet-static gem is for.

As it turned out, there is no way to avoid sorbet-runtime if you have signatures in your code.

I started to hate type annotations more: I don't want to add an additional dependency to the gem, even if the overhead of type checking is <10% (which is still more than 0%):

OK. Let's play this game 'till the end.

After adding sorbet-runtime I was able to run the code and even caught one "problem":

$ bundle exec rspec

Failure/Error: rules << rule

RuntimeError:
  You must use `.implementation` when overriding the abstract method `rules`.
    Abstract definition: Rubanok::DSL::Matching at /Users/palkan/dev/rubanok/lib/rubanok/dsl/matching.rb:119
    Implementation definition: #<Class:Rubanok::Plane> at /Users/palkan/dev/rubanok/lib/rubanok/plane.rb:53
Enter fullscreen mode Exit fullscreen mode

Why didn't static analysis caught this?

UPD: it's possible to "disable" runtime checks by using T::Sig::WithoutRuntime.sig instead of sig to write signatures. And it's not possible to write extend T::Sig::WithoutRuntime instead of extend T::Sig 😄

Problem #6. Debugging.

I'm a heavy user of binding.pry.

When I was debugging the code with type signatures, I found it's very hard to step into the method:

Can you find where is the original method hiding?

In conclusion, or it's just the beginning

Sorbet is one of the most important things that happened to Ruby in the last few years.

But it's still far from bringing the development happiness. And it's you who can help it become better: give it a try, report issues or say "Thanks!" to people behind it (e.g., Dmitry Petrashko and Paul Tarjan).

Will I use Sorbet?

Likely, yes. But only in the way described below.

After adding all the signatures to the codebase (full adoption), I want to be able to dump them into a .rbi file and clean up the codebase. Thus, my Ruby code stays the same: less verbose and more readable.

That should break neither static nor runtime checks, i.e., if sorbet-runtime is loaded, runtime checks are activated, otherwise—not. Static checks should work just because of the presence of the RBI files.

P.S. I'm only talking about libraries development right now.

P.P.S. I want to try to adopt Steep as well and compare the process to the one described above. We'll see which one I like more.


Read more dev articles on https://evilmartians.com/chronicles!

Top comments (16)

Collapse
 
rhymes profile image
rhymes • Edited

Hi Vladimir, thanks for your article. In the current state I don't think I'd adopt it either. To be honest I'm not sure it's going to scale easily considering how most Rails apps load tons of gems full of metaprogramming and DSLs.

Stripe AFAIK does not use Rails, Sorbet should be easier in a Ruby-only project.

A few comments:

I hope this will change in the future. The raw size (w/o .git/) of rubanok increased by 2.1MB, from 124KB to ~2.2MB.

2 Megabytes for a small library... It's probably a fixed increase of size though, I don't think it's directly proportional to how many signatures or definition files.

Problem #1. Unsupported features.

I think this is the same problem the Javascript community had when trying TypeScript. The type checker needs to evolve with the language and AFAIK there's no formal spec of Ruby so you have to track each change manually anyway.

BTW I didn't know Sorbet is written in C++

Problem #2. Limitations of flow-sensitivity

You did a lot of wrangling there. It's the thing that annoys me from these checkers, the time spent to make the tool happy

Problem #3. Signatures vs modules.

Can I say that this signature looks ridiculous? I'm thinking of the cognitive complexity that adds for devs at all levels.

sig do
  params(
    fields: Symbol,
    activate_on: T::Array[Symbol],
    block: T.proc.void
  )
  .returns(T::Array[Rubanok::Rule])
end

I agree when you say that the tool should move the signatures away from the code, or at least have a switch that can do that.

Code is read much more than it's written.

Problem #5. Runtime checks.

7% of CPU overhead is a lot for an already slow language. Stripe has a ton of VC money, they can just scale up the machines :D

When I was debugging the code with type signatures, I found it's very hard to step into the method

:((((

Collapse
 
palkan_tula profile image
Vladimir Dementyev

Thanks for the comment!

Stripe AFAIK does not use Rails, Sorbet should be easier in a Ruby-only project.

Yep. But we also have Shopify, an early adopter/beta user of Sorbet. We haven't heard a lot about this adoption yet though.

And there are already some tools to make it easier to integrate with Rails: github.com/chanzuckerberg/sorbet-r....

Collapse
 
shevegen profile image
markus heiler

So everyone is now shopify or [insert random company xyz]?

I mean there is a reason why this abomination has been created by a COMPANY. At the least they finally open sourced it - it was ludicrous that they promoted it while it was closed source.

Collapse
 
rathrio profile image
Radi • Edited

Is debugging with binding.pry still an issue if runtime checks are disabled? I don't see how sorbet could mitigate this cleanly without digging deeper into one specific Ruby implementation.

BTW: Do the runtime checks work with Rubies other than MRI? Looks like a pure Ruby gem but that's not a guarantee for compatibility.

Collapse
 
joshcheek profile image
Josh Cheek

I haven't tried it, but I'd expect not. Presumably they implement it with TracePoint, which can register and unregister for events (ie I would expect it is implemented without changing the actual structure of the code)

Collapse
 
palkan_tula profile image
Vladimir Dementyev

As far as I can see, there is no TracePoint usage. They use method_added instead to redefine the method and wrap it with the signature validation.

So, it doesn't seem to have anything MRI specific.

Collapse
 
ptolemybarnes profile image
Ptolemy

Nice article. It seems like a really ambitious project. If I were working on it I would target a strict subset of Ruby and try to creep forward from there (I wonder if this is what Stripe have been doing internally).

Collapse
 
katafrakt profile image
Paweł Świątkowski

After adding all the signatures to the codebase (full adoption), I want to be able to dump them into a .rbi file and clean up the codebase. Thus, my Ruby code stays the same: less verbose and more readable.

Nice idea, although I think it kind of beats the purpose of types annotations, because every time you need to consult or adapt two files, instead of one.

I've been thinking too whether libraries should adopt Sorbet or not. As library maintainer, I don't want to force users to install Sorbet. I wish there would be a ways to enable those typechecks only if Sorbet is already present (because parent project decided to use that, or because I'm just developing my own library).

Probably it's a matter of time before some sorbet-dummy lib comes out, which would defined methods used by Sorbet (such as sig), but does nothing and steps off when it detects that "real" Sorbet is available.

Collapse
 
joshcheek profile image
Josh Cheek

The examples make it look like you can noop it with a simple def sig(*) end. Is that not why you defined it in the block form?

Collapse
 
bodacious profile image
Gavin Morrice

Honestly, I still can't see why type checking is ever necessary in Ruby?

What does it add to your codebase or workflow?

Collapse
 
palkan_tula profile image
Vladimir Dementyev

I think, type checking could bring value at a large scale: huge codebases, bigger teams (this post shares some thoughts on this topic).

For example, when you work on a large application you might not know all the internal APIs; type checking could help you to avoid Undefined method foo for NilClass kind of errors when writing code using these unknown APIs. Documentation could also help but it could easily get out of sync.

Refactoring with type system is also much easier and faster even if you have a decent test coverage: when test fails you have to spend time to figure out why; when type checking fails you know the exact place in code. That becomes useful when you have hundreds of such places to change.

Collapse
 
mereghost profile image
Marcello Rocha

type checking could help you to avoid Undefined method foo for NilClass kind of errors

If type checking fixed this we would never have seen NullPointerException in Java/.NET static typed worlds.

Collapse
 
jaydanielian profile image
jaydanielian

Totally disagree that type checking helps with refactoring. My experience with Java and C# is exactly the opposite. It becomes a large pain to either a) need to change the param of a method to a different type or b) add a parameter to an existing method. Yes tools like resharper help, but most of the time you play compiler jujutsu trying to find just the right generic or whatever type necessary to satisfy all your existing code.

These changes become more cumbersome to implement. Duck typing allows these kind of changes with ease, and ruby method signatures are usually pretty flexible with keyword args, option splats etc being common.

I think we’ll tested systems mean type checking is totally superfluous. I moved to ruby from a C# /Java background and I have not missed types/generics one iota in Ruby. Keep ruby ducky :)

Collapse
 
vinistock profile image
Vinicius Stock

Thanks for the great post! Your findings are extremely valueable for anyone trying to adopt the Sorbet.

Collapse
 
shevegen profile image
markus heiler

Why exactly?

It does not really address the type annotation situation from an objective point of view. To be fair it is also not that subjectively biased either; but even then he makes huge claims such as:

"this is a huge leap forward for Ruby itself (especially, compared to other evolutionary features like pipeline operator)."

He does not explain why it would be a "huge" leap forward. I don't see what is leaping here.

And he contrasts it to the pipeline syntax sugar. Well, I don't like the pipeline syntax either; don't hate it but just don't see a point. But this is still so totally different to the SCOPE of sprinkling type annotations all over a code base with the deliberate attempt to make it uglier and less readable than it was before, for the promised gain in return that certain errors will be caught (which real ruby hackers don't run into - you only have this problem IN BIG COMPANIES where the worker drones have to churn out code for a living).

Keep ruby clean - that should be the main motto.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.