DEV Community: Okinawa Ruby User Group

Optimize boot time of ruby CLI tools

Seiei Miyagi — Wed, 29 May 2019 10:18:13 +0000

Create a binstub like following:

#!/usr/bin/env ruby
require 'bootsnap'
gem_name = 'rubocop'
exe_name = 'rubocop'
cache_dir = File.join(ENV['HOME'], '.cache', 'bootsnap', gem_name, exe_name)
Bootsnap.setup(cache_dir: cache_dir, load_path_cache: false, autoload_paths_cache: false)
load Gem.bin_path(gem_name, exe_name)

Gem.bin_path¹ takes two arguments, 1st is name of gem, 2nd is name of executable. You can modify gem_name/exe_name to apply this binstub to other ruby CLI tools.

Without this binstub:

% time rubocop -v
0.70.0
rubocop -v  0.76s user 0.20s system 97% cpu 0.983 total

With this binstub, It takes a little more time at first run:

% time bin/rubocop -v
0.70.0
bin/rubocop -v  1.04s user 0.57s system 98% cpu 1.640 total

But once it runs, the cache is created. Then It becomes much faster:

% time bin/rubocop -v
0.70.0
bin/rubocop -v  0.53s user 0.23s system 79% cpu 0.969 total

You don't need to change any code of ruby CLI tools, Just create a binstub to optimize boot time.

Cheers <3

https://www.rubydoc.info/github/rubygems/rubygems/Gem.bin_path ↩

Using RubyVM::AbstractSyntaxTree.of in pry

Seiei Miyagi — Sat, 04 May 2019 04:27:34 +0000

Put following snippets to ~/.pryrc

unless defined?(SCRIPT_LINES__)
  SCRIPT_LINES__ = {}
end

Pry.config.hooks.add_hook(:before_eval, :"ast<3") do |code, _pry|
  SCRIPT_LINES__["(pry)"] = code.lines
  code
end

It works, yay!

% pry
[1] pry(main)> RubyVM::AbstractSyntaxTree.of(-> { puts :hi })
=> (SCOPE@1:32-1:45
 tbl: []
 args:
   (ARGS@1:32-1:32
    pre_num: 0
    pre_init: nil
    opt: nil
    first_post: nil
    post_num: 0
    post_init: nil
    rest: nil
    kw: nil
    kwrest: nil
    block: nil)
 body: (FCALL@1:35-1:43 :puts (ARRAY@1:40-1:43 (LIT@1:40-1:43 :hi) nil)))
[2] pry(main)>

I also wrote a irb version, including how RubyVM::AbstractSyntaxTree.of uses SCRIPT_LINES__.
https://dev.to/okinawarb/how-can-i-use-rubyvm-abstractsyntaxtree-in-irb-m2k

Enjoy AST!

How can I use RubyVM::AbstractSyntaxTree.of in irb?

Seiei Miyagi — Thu, 25 Apr 2019 13:00:41 +0000

Put following snippets to ~/.irbrc.

unless defined?(SCRIPT_LINES__)
  SCRIPT_LINES__ = {}
end

ast_happier = TracePoint.new(:call) do |tp|
  SCRIPT_LINES__['(irb)'] = tp.binding.local_variable_get(:statements).lines
end
ast_happier.enable(target: IRB::WorkSpace.instance_method(:evaluate))

It works, yay!

% irb
irb(main):001:0> pp RubyVM::AbstractSyntaxTree.of(-> { puts :hi })
(SCOPE@1:35-1:48
 tbl: []
 args:
   (ARGS@1:35-1:35
    pre_num: 0
    pre_init: nil
    opt: nil
    first_post: nil
    post_num: 0
    post_init: nil
    rest: nil
    kw: nil
    kwrest: nil
    block: nil)
 body: (FCALL@1:38-1:46 :puts (ARRAY@1:43-1:46 (LIT@1:43-1:46 :hi) nil)))
=> #<RubyVM::AbstractSyntaxTree::Node:SCOPE@1:35-1:48>

But how does it work?

SCRIPT_LINES__ is used for standard library debug.
https://docs.ruby-lang.org/en/2.6.0/DEBUGGER__.html

Of course there is no documentation.

SCRIPT_LINES__ = {} unless defined? SCRIPT_LINES__ # :nodoc:

https://github.com/ruby/ruby/blob/v2_6_3/lib/debug.rb#L22

In the ast.c, SCRIPT_LINES__ is also used for finding the source code of the proc.

static VALUE
script_lines(VALUE path)
{
    VALUE hash, lines;
    ID script_lines;
    CONST_ID(script_lines, "SCRIPT_LINES__");
    if (!rb_const_defined_at(rb_cObject, script_lines)) return Qnil;
    hash = rb_const_get_at(rb_cObject, script_lines);
    if (!RB_TYPE_P(hash, T_HASH)) return Qnil;
    lines = rb_hash_lookup(hash, path);
    if (!RB_TYPE_P(lines, T_ARRAY)) return Qnil;
    return lines;
}

https://github.com/ruby/ruby/blob/v2_6_3/ast.c#L189-L201

static VALUE
rb_ast_s_of(VALUE module, VALUE body)
{
    VALUE path, node, lines;
    int node_id;
    const rb_iseq_t *iseq = NULL;


    if (rb_obj_is_proc(body)) {
        iseq = vm_proc_iseq(body);


        if (!rb_obj_is_iseq((VALUE)iseq)) {
            iseq = NULL;
        }
    }
    else {
        iseq = rb_method_iseq(body);
    }


    if (!iseq) return Qnil;


    path = rb_iseq_path(iseq);
    node_id = iseq->body->location.node_id;
    if (!NIL_P(lines = script_lines(path))) {
        node = rb_ast_parse_array(lines);
    }
    else if (RSTRING_LEN(path) == 2 && memcmp(RSTRING_PTR(path), "-e", 2) == 0) {
        node = rb_ast_parse_str(rb_e_script);
    }
    else {
        node = rb_ast_parse_file(path);
    }


    return node_find(node, node_id);
}

https://github.com/ruby/ruby/blob/v2_6_3/ast.c#L220-L253

During the RubyVM::AbstractSyntaxTree.of, ruby does

read the file location from the RubyVM::InstractionSequence of given proc.
read whole source code from some sources, such as SCRIPT_LINES__/the real file/command line input given by -e
parse whole source code again to create an AST.
read the node id from the RubyVM::InstructionSequence of given proc.
find a node from the AST, the node which has same node id of given proc.

If parsing the same file, ruby parser gives identical node id for same object in that position. That's the point.

I use TracePoint to set the source code to SCRIPT_LINES__ before evaluate irb input.

The TracePoint work at here.

    # Evaluate the given +statements+ within the  context of this workspace.
    def evaluate(context, statements, file = __FILE__, line = __LINE__)
      eval(statements, @binding, file, line)
    end

https://github.com/ruby/ruby/blob/v2_6_3/lib/irb/workspace.rb#L83-L86

Hava a fun!

Ruby 2.6 new feature: refine `#to_proc`

Seiei Miyagi — Mon, 04 Mar 2019 12:10:57 +0000

Refinements take place at block passing. [Feature #14223]
https://github.com/ruby/ruby/blob/v2_6_0/NEWS

It's very useful when you want to use the literals as DSL for the block.
Following example refines Integer#to_proc and Array#to_proc to slice the values.

using Module.new {
  refine(Integer) do
    def to_proc
      -> o { o[self] }
    end
  end

  refine(Array) do
    def to_proc
      # Same as -> o { map(&o.method(:[]) }
      # https://dev.to/hanachin/ruby-27-new-feature-method-reference-operator-38l2
      -> o { map(&o.:[]) }
    end
  end
}

p "foo".match(/(a)(b)(c)/)&.then(&1) #=> nil
p "abc".match(/(a)(b)(c)/)&.then(&1) #=> "a"
p "abc".match(/(a)(b)(c)/)&.then(&[1]) #=> ["a"]
p "abc".match(/(a)(b)(c)/)&.then(&[1,2,3]) #=> ["a","b","c"]