DEV Community: Aaron Christiansen

Delta Pico - what I've learned by building a graphing calculator

Aaron Christiansen — Sun, 23 Oct 2022 08:53:04 +0000

This is the Delta Pico! It's a graphing calculator with a 2.8" colour display, running on a Raspberry Pi Pico, with software written in Rust.

It can evaluate expressions with a history stored in flash memory, and plot multiple graphs on an adjustable pair of axes. (You can also play a simple clone of 2048!)

After working on it since July 2021, I've reached a point where I'm very satisfied with this project. It has far from all the features you'd find in a commercially-available graphing calculator, but it's complete enough for me to be happy!

The Delta Pico is probably the most ambitious side-project I've ever undertaken, and also the longest I've ever stuck with a personal project.

This post will be a short retrospective of the project - how it evolved over time, and what I've learned along the way.

Development and Iteration

I really like calculators! Earlier models in history showcase the introduction of computing into our daily lives, and they remain interesting today to watch how they avoid being completely swallowed up by smartphones. I've even given a talk about them for my alma-mater's computer science society.

This project was a follow-on from my Delta M0 - a compact and much more simple calculator, based on one of STM's Cortex-M0 microcontrollers. I wanted to build the Delta Pico as a calculator more similar to the TI-Nspire, HP Prime, or the brilliant open-source NumWorks calculator.

It all started on a breadboard! While working on the Delta M0, I built a generic button matrix PCB, which you can see connected to the breadboard.

Shortly after getting the first prototype working, I designed a PCB in KiCad and ordered it from JLCPCB.

There have been three hardware revisions in total so far, though the changes have been quite minor.

Instead, most of the changes have been in software. The Delta Pico's current software stack and architecture is on its third iteration! It began as a PlatformIO Arduino project, since there were existing Arduino libraries for driving this ILI9341-based display, so that helped me to get up-and-running quickly. It then morphed into a Pico SDK-based project, and finally ended up as a pure embedded Rust codebase.

Rust code was introduced gradually before the project became completely Rust - it started out just being for user-level applications, then I rewrote the drivers for each device in Rust too.

The hardware abstraction layer (HAL) and the user software are two separate projects, which means you can run the Delta Pico's software on other hardware by just implementing a HAL for it. This allowed me to create a simulator to run on my computer which used the exact same software, which came in handy for debugging several times!

What have I learned?

Mathematical editors are tricky

The core of a calculator is the ability to input the expression you'd like to evaluate.

To match the functionality that I enjoyed on other calculators I own, I wanted the Delta Pico to have "natural" input. That means your expression isn't "flat" like sqrt(2/3), it actually looks like how you'd write it on paper:

23\sqrt{\frac{2}{3}}

This turned out to be a much greater challenge than I expected. While there are libraries for this, they're largely tied to the web, and aren't suitable for embedded use.

After a lot of experimentation, I built rbop, a framework for implementing mathematical editors on any platform. It's completely separate to the Delta Pico's codebase - it'll provide the input and evaluation logic, and you just need to implement however it will draw the expressions on your platform.

For example, you can even write a crude editor which works in your terminal:

This means that rbop is completely reusable if I decide to work on another calculator project.

rbop works using two different node trees: unstructured, which is looser and suited to user inputs, and structured, which follows a tight structure ready for evaluation. Unstructured node trees can be upgraded to structured ones using a parsing step. Both node trees can be rendered by recursively transforming them into a sequence of glyphs, which are then handed off to platform-provided code to draw them to the screen.

I might write more about how rbop works in the future - I found it fascinating to implement, and wasn't able to find other projects aiming for the same goals, so hopefully this could be useful going forward.

Embedded Rust isn't quite mature

Writing Rust for an embedded platform is a rocky experience, especially if you aren't too familiar with the language itself when starting out, which was the case for me.

For the uninitiated, a key Rust feature is the borrow checker, which essentially enforces that data can only be owned in one place at a time. You can create references to data, but the borrow checker must know how long they'll exist, which isn't always easy to specify.

The borrow checker enables Rust to insert memory allocations and deallocations at compile-time, rather than needing a garbage collector which would be impractical on an embedded system.

Rust's standard model for embedded development treats the peripherals of a microcontroller as data with an owner. This means that it's very tricky to share peripherals between multiple drivers in a way that the borrow checker is happy with.

The Delta Pico's keypad, flash storage, and display drivers all need to be able to access the clock peripheral to use delays. How do you model this?

The difficulty of working around the borrow checker has led to me writing what might be the worst Rust function I will ever write, to completely subvert Rust's referencing restrictions. I'm sorry, Ferris.

The library ecosystem is also currently far from what you have available when writing C or C++. As a Pico SDK project, the Delta Pico implemented USB mass storage using TinyUSB, so it would show us a storage device when connected to a computer. Unfortunately, I wasn't able to find a suitable Rust replacement for this when migrating the codebase.

At least, thanks to the community, the breadth of embedded libraries available is constantly getting bigger!

Trying new stacks is fun

I had wanted to properly sink my teeth into Rust for a while, and this project was the perfect opportunity to do so. Rewriting parts of a familiar project in a different language was a useful learning experience.

That said, it also encouraged some unhealthy habits. The Delta Pico codebase is littered with patterns which would be considered poor Rust idioms, so that things just work the way they used to.

Now that I know more, I could likely go back and update everything to be more Rust-like, but that's not exactly my idea of fun! I have refactored the software for this project enough times already.

The Raspberry Pi Pico is a wonderful platform

Time and time again, I was impressed by the Raspberry Pi Pico microcontroller that I chose as the core of this project.

The form factor is great, it has a tonne of RAM (important for a 240x320 colour frame buffer), and the documentation is leagues ahead of other microcontrollers I've used.

Having already risen to being a ubiquitous choice among makers, the Rust bindings for the platform are very complete too.

The Pico will absolutely be my default choice of microcontroller for new projects!

In conclusion

I got into electronics about seven years ago now, starting off with little sensor projects using an Arduino Uno kit, and I've been dabbling in the occasional project ever since. I also picked up PCB design during the first COVID-19 lockdown, and built a simple function generator.

I have to admit that I'm really proud of the Delta Pico - this is exactly the kind of project that I wanted to be able to create when I bought my first electronics kit those years ago. The fact I've finally done it is an incredibly rewarding feeling.

Going forward, I'm excited to continue extending the Delta Pico, and start working on new projects too!

If you'd like to take a look at the Delta Pico's hardware or software, check out the GitHub repository, where you'll find the Rust codebase and KiCad project.

Elixir's bitstrings - the data type I didn't know I wanted

Aaron Christiansen — Fri, 17 Dec 2021 18:24:50 +0000

After hearing a lot about it within the Ruby community, I'm working through this year's Advent of Code in Elixir.

Overall, I'm enjoying Elixir so far! I've found that Ruby knowledge has helped me to get to grips with Elixir rather quickly - their syntax is very similar, and I'm able to make educated guesses on the names of Elixir functions coming from Ruby too. By far the most challenging part of learning Elixir has been solving the Advent of Code problems in a functional, immutable style.

But in this post, I want to focus on one specific part of Elixir: bitstrings. These were invaluable for solving Day 16 of Advent of Code, which involved parsing packets made up of tightly-packed bit-level structures.

Thinking About Strings

Let's step back from bitstrings and think about regular old strings of text. Here's one:

"hello"

In memory, a string will actually be a sequence of bytes (with a length attached somehow, not shown here):

" h    e    l    l    o    "
  0x68 0x65 0x6c 0x6c 0x6f

Many interpreted languages, such as Python, have separate str and bytes types for differentiating between text and an arbitrary list of bytes.

But, Elixir makes no such distinction! Strings are simply an instance of a type called a binary, which is a list of bytes with a length attached. You can create a binary using << >> syntax - notice that Elixir's prints both of these inputs in the same way (using IO.inspect, Elixir's equivalent of Ruby's p or Python's repr):

"hello"
# => "hello"

<< 0x68, 0x65, 0x6c, 0x6c, 0x6f >>
# => "hello"

That's because they are the same underlying data type, and the double-quotes are really just a shortcut for creating a binary from some characters.

Elixir is printing it back to us as a string for our convenience, because it can see that the binary is made up of printable Unicode characters. If we create a binary of unprintable characters instead, it'll display it as a binary rather than a string:

<< 1, 2, 3 >>
# => << 1, 2, 3 >>

OK, so we've got a type which can store some bytes. But how's this special at all, when you can simply create an array or vector of bytes in most languages?

Introducing Bitstrings

In Elixir, binaries are not really their own data type. A binary is actually a special case of the real data type, called a bitstring. A bitstring is a binary if the number of bits is divisible by 8.

This is the special part! Most languages I've worked with don't provide facilities for seamlessly working with data whose size isn't a whole number of bytes, but in Elixir we can use bitstrings to work with any number of bits, not necessarily divisible by 8 into bytes.

Bitstrings are constructed using the same << >> syntax, but we can tag each component with its size in bits. For example:

<< 1 :: size(2), 6 :: size(4) >>
# => << 19::size(6) >>

The size can be omitted as a shorthand, allowing us to just enter << 1 :: 2, 6 :: 4 >>.

Elixir does concatenate these two components together into a single 6-bit field, but that doesn't matter, since the bit-level representation in memory remains the same:

    Original:   1       3
               .-.   .-----.
               0 1   0 0 1 1
               '-----------'
Concatenated:       19

Working With Bitstrings

We can use bitstrings to easily construct data structures at the bit level, from individual components. But their real power comes from when we want to do the opposite, and deconstruct data into its bits.

This is the best part: bitstrings support Elixir's pattern matching! Let's try pulling apart that example from earlier using pattern matching:

<< x :: 2, y :: 4 >> = << 19 :: 6 >>
x # => 1
y # => 3

Just like that, we can extract arbitrarily-sized integers from bitstrings!

The equivalent using most programming language's bitwise operators would be something like:

x = (19 & 0b110000) >> 4
y = 19 & 0b11

In my opinion, Elixir's pattern matching approach shows intention much better and is less error-prone. (Elixir does have bitwise operators those, so you can fall back to them if you like!)

Pattern matching allows us to do even more advanced things. Instead of :: size(x) or :: x, we can use the :: bitstring specifier to match a chunk of any size. For example, given some input data of any unknown size, we could split apart the first two bits and the remaining bits:

<< first :: 2, rest :: bitstring >> = << 243, 48 >>
first # => 3
rest # => << 204, 48::size(6) >>

(This particular application was where they were most useful in Advent of Code!)

For a final example, let's combine this with Elixir's pattern matching on method definitions, to define a method which counts the number of 1 bits in a bitstring:

# If the next bit is 1, count it and recurse
def count_bits(<< 1 :: 1, rest :: bitstring >>), do: 1 + count_bits(rest)

# If the next bit is 0, don't count it and recurse
def count_bits(<< 0 :: 1, rest :: bitstring >>), do: count_bits(rest)

# Base case: if bits have run out, stop
def count_bits(<< >>), do: 0

# Let's try it out...
count_bits(<< 55 >>)
# => 5
# That's right! 55 is 0b00110111

Conclusion

I hope you've found this to be an interesting look at bitstrings, and that you can see their value for certain problems! Although libraries with similar functionality are available for other languages, I doubt any of them will manage to achieve the same level of language integration with pattern matching that Elixir accomplishes.

Even if you aren't too familiar with Elixir, you may find it interesting to have a skim through my Advent of Code Day 16 solution to find how I've used bitstrings to deconstruct the puzzle input.

Thanks for reading!

Removing functionality the Ruby way - a look into SortedSet on Ruby 3

Aaron Christiansen — Wed, 30 Dec 2020 21:17:26 +0000

Ruby 3 has just released - hooray! But naturally, some of my code hit problems with Ruby 3's breaking changes.

Almost all of these were related to Ruby 3's changes to how keyword arguments are handled, but I also had an intriguing issue reported on my Sord type generation project.

Ruby used to include a SortedSet class, but it's been removed from Ruby 3 for "dependency and performance reasons". Suppose we try to use it...

irb(main)> SortedSet
RuntimeError (The `SortedSet` class has been extracted from the `set` library.You must use the `sorted_set` gem or other alternatives.)

Woah, what's that error message!? That's not usually what we see when we try to use a constant which doesn't exist. You'd expect to get something like:

NameError (uninitialized constant SortedSet)

SortedSet gives us a much more useful error message.

How does Ruby do this? Let's take a look!

My first guess was that Ruby 3 replaced the SortedSet class with a SortedSet method which throws an exception. It's not unusual for the Ruby Kernel to have methods which look like classes, so I thought this was a pretty good guess. But nope, there isn't any SortedSet method:

irb(main)> method(:SortedSet)
NameError (undefined method `SortedSet' for class `#<Class:#<Object:0x000056382e0a84c8>>')

To dig a bit deeper, I took a look at the Ruby interpreter source code. The lib/set.rb file revealed the answer:

autoload :SortedSet, "#{__dir__}/set/sorted_set"

SortedSet is now defined with autoload. Referencing SortedSet will require the given file to try and resolve that constant, acting as a kind of lazy-loading within a Ruby program.

Sure enough, the useful exception we were seeing is thrown by the sorted_set file which autoload points to:

begin
  require 'sorted_set'
rescue ::LoadError
  raise "The `SortedSet` class has been extracted from the `set` library." \
        "You must use the `sorted_set` gem or other alternatives."
end

This is a really clever way of providing useful error messages when functionality is broken out or removed, which I personally haven't seen before!

There is one quirk with it though, and it's the cause of the bug which made me look into this in the first place.

SortedSet is still a constant, so Object.constants contains it. But accessing this constant isn't always safe if you don't have the sorted_set gem. This means that iterating over all available constants can throw an exception!

Object.constants.each do |c|
  p Object.const_get(c)
end

# => KeyError
# => Thread::Queue
# => IndexError
# ...etc, until boom!
# => RuntimeError (The `SortedSet` class has been extracted from the `set` library.You must use the `sorted_set` gem or other alternatives.)

I can see this being a problem in some heavily-metaprogrammed environments, just like it was for Sord.

For what's basically just a custom error message, I found this approach to the removal of SortedSet interesting to think about.

Is breaking a common assumption like "reading a constant which exists won't throw an exception" worthwhile when it might help out the developer?

Or, on the other hand, is consistency key to maximise the effectiveness of Ruby's metaprogramming? Do edge cases like this introduce complexity which makes the internals of Ruby more difficult to understand?

Personally I think this minor touch is a great design decision in Ruby 3, even if it did introduce a bug that had to be solved. If you have any other thoughts, I'd be interested to hear them!

"yes_i_did" - A tiny gem to rerun your last IRB command with the "did you mean" suggestion if you make a typo

Aaron Christiansen — Sun, 05 Apr 2020 10:39:11 +0000

I've written a tiny gem called yes_i_did which adds a new method to your IRB session, yes. If you didn't quite get the name of something right in a command, simply invoke yes to rerun it with the typo fixed using the "did you mean" suggestion!

Setup is simple: just install the gem and require it in your ~/.irbrc.

This has already proven a handy time-saver for me, and I hope it can be for you too!

Flashing your PineTime using an ST-Link and OpenOCD

Aaron Christiansen — Tue, 07 Jan 2020 21:34:34 +0000

The PineTime development kits are now shipping out, and to do anything interesting with yours, you'll probably be looking to flash a firmware on to it.

With open-source tool OpenOCD and a cheap ST-Link programmer, it's really easy to flash firmware once you know what to do!

Prerequisites

There are a few things you need to do before you can follow this guide:

You'll need to solder wires onto your PineTime's SWD pins so that you can connect it to a programmer (or a Pi to unlock it - see below). See the wiki page for a diagram of which pin is which. (Some people have had success with just resting the pins in the holes on the board and not soldering them there, but I haven't tried this.)
You must remove the flash protection your PineTime before you can flash it. It's possible to do this using a J-Link, but they are very expensive. Fortunately, you can also do it with a Raspberry Pi; follow the Remove nRF52 Flash Protection with Raspberry Pi section of Lup Yuen Lee's fantastic guide.
You need an ST-Link programmer; I use this cheap clone from Amazon UK. If you haven't yet used an ST-Link on your Linux installation, you'll need udev rules: I followed this guide to add mine.

Building OpenOCD

The build of OpenOCD you can get from your package manager might not work for our purposes, as it probably hasn't been built with CMSIS-DAP support. We'll build it from source using a GitHub mirror instead.

On Linux:

git clone https://github.com/ntfreak/openocd
cd openocd
./bootstrap
./configure --enable-cmsis-dap
make

This will give you an OpenOCD executable inside the repo at src/openocd.

Configuring OpenOCD

You'll need a configuration file to tell OpenOCD to connect to your ST-Link, and tell it what kind of chip we'll be debugging. Save this in nrf52.cfg, at the root of your OpenOCD repo:

source [find interface/stlink.cfg]
transport select hla_swd
source [find target/nrf52.cfg]
init

Starting OpenOCD

Now we can start an OpenOCD server with our configuration! Run this in the repo root:

./src/openocd -s tcl -f nrf52.cfg -d3

-s tcl tells OpenOCD to look for files it needs in the tcl directory, such as the interface/stlink.cfg we specified in our config. -f nrf52.cfg runs our configuration file on OpenOCD, and -d3 prints the maximum amount of debugging information so it's easier to diagnose any issues.

Connecting to OpenOCD

After starting OpenOCD, we now have a telnet server on port 4444 which we can connect to in order to issue commands to our ST-Link. Connect with:

telnet localhost 4444

You'll get some preamble and a prompt which should look something like this:

Trying ::1...
Connection failed: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Open On-Chip Debugger
>

There are two commands to check to ensure your chip is detected. targets should show one device, and flash banks should show two results.

> targets
    TargetName         Type       Endian TapName            State       
--  ------------------ ---------- ------ ------------------ ------------
 0* nrf52.cpu          hla_target little nrf52.cpu          running

> flash banks
#0 : nrf52.flash (nrf5) at 0x00000000, size 0x00000000, buswidth 1, chipwidth 1
#1 : nrf52.uicr (nrf5) at 0x10001000, size 0x00000000, buswidth 1, chipwidth 1

Looks good!

Flashing

Flashing a firmware is now pretty easy. First, we want to ensure that our PineTime is stopped:

> reset halt
target halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0x00000462 msp: 0x20000800

You may want to clear the PineTime's firmware before flashing a new one. This should happen as part of the programming process, but if you want to do it explicitly:

> nrf51 mass_erase
nRF52832-QFAA(build code: E0) 512kB Flash, 64kB RAM

Now for the actual flash! Give the path you'd like to program. Note that this path is relative from where you ran OpenOCD, not where you ran telnet, so you'll probably want to give an absolute path here. Make sure you also include the verify reset at the end.

> program /home/aaron/projects/pinetime/app.bin verify reset
target halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0xfffffffe msp: 0xfffffffc
** Programming Started **
Adding extra erase range, 0x00005d2c .. 0x00005fff
** Programming Finished **
** Verify Started **
** Verified OK **
** Resetting Target **

You're all done! If your PineTime didn't restart for some reason, you can always reset it and kick it off with a command:

> reset run

Need a pre-built firmware?

Want to make sure you can flash your PineTime, but haven't got round to building a firmware yet? I've built a simple firmware on go-smartwatch which should show "Hello!" on your PineTime's display.

Download the pre-built firmware here!

Taking pattern matching further in Ruby 2.7

Aaron Christiansen — Sun, 22 Dec 2019 23:11:23 +0000

With Ruby 2.7 just around the corner, and release candidate 2 available now, I was eager to try the new pattern matching feature that has been introduced. As with anything in Ruby, I wanted to see how I could push this feature to its limits.

Note that this is not an introduction to Ruby 2.7's pattern matching; make sure you're already familiar with its purpose and how to use it. The release notes summarise this very well!

How does pattern matching work?

There are two main methods involved with the behaviour behind pattern matching, #deconstruct and #deconstruct_keys, which are used for array-style matches and hash-style matches respectively.

#deconstruct returns an array representing the object which can be matched against. For the Array class, this is simply aliased to #itself, as no conversion needs to be done here. Instances of Struct define #deconstruct as an array of the field values.

[1, 2, 3].deconstruct # => [1, 2, 3]

Person = Struct.new(:name, :favourite_food)
Person.new('Aaron', 'Pizza').deconstruct # => ['Aaron', 'Pizza']

#deconstruct_keys is similar, except it returns a hash rather than an array. Additionally, #deconstruct_keys takes an array of Symbol keys as an argument, indicating which keys of the hash must be returned. It is permitted to a hash with more keys than specified (Hash#deconstruct_keys is aliased to #itself so returns all keys) but the additional keys won't be used for the pattern match.

{ a: 3, b: 4 }.deconstruct_keys([:a, :b]) # => { a: 3, b: 4 }

Person = Struct.new(:name, :favourite_food)
me = Person.new('Aaron', 'Pizza')
me.deconstruct_keys([:name]) # => { name: 'Aaron' }
me.deconstruct_keys([:name, :favourite_food]) # => { name: 'Aaron', favourite_food: 'Pizza' }

As the `case` may be...

I was surprised to learn that you don't actually need to use the in operator for pattern matching within a case statement; you can use it anywhere! This can be used to unpack structs quite elegantly:

Person = Struct.new(:name, :favourite_food)
me = Person.new('Aaron', 'Pizza')

me in { name: my_name, favourite_food: my_favourite_food }
my_name # => 'Aaron'
my_favourite_food # => 'Pizza'

You can even use it as a slightly esoteric form of assertion, as a pattern which doesn't match will raise a NoMatchingPatternError. I'm not sure I'd recommend this, but it certainly works!

x = 3
x in 3 # Fine!
x in 2 # Raises NoMatchingPatternError

In its most basic form, you can even use pattern matching as a simple assignment. I wonder if DSLs could make use of this?

2 in y
y # => 2

A methodical approach

This was inspired by Brandon Weaver's excellent post about pattern matching. Being able to match against objects which define #deconstruct or #deconstruct_keys is great, but what about any other objects where properties are behind methods?

It's really easy to create a wrapper around any object which will dynamically invoke methods on it when #deconstruct_keys is called, allowing any object's zero-argument methods to be used in pattern matching!

Let's create an example of how this will help us first. I'll define a simple data class:

class Rectangle
  def initialize(width, height)
    @width = width
    @height = height
  end

  attr_accessor :width, :height
end

Rectangle#width and Rectangle#height are methods which will retrieve the properties of any instances of Rectangle. Unfortunately, pattern matching doesn't know how to match against instances of this class, as it doesn't contain a #deconstruct_keys definition:

rect = Rectangle.new(2, 5)
rect in { width: w, height: h } # Raises NoMatchingPatternError

Let's define a class which can wrap any object in to allow method calls to be made from a pattern match. I'll call it SendMatch:

class SendMatch
  def initialize(target)
    @target = target
  end

  def deconstruct_keys(keys)
    keys.to_h { |key| [key, @target.send(key)] }
  end
end

How this works is actually really simple. When constructed, it takes a single object and stores a reference to it as an instance variable @target. Then, when #deconstruct_keys is called, it invokes each the methods with the names of each requested key on @target and creates a hash from the results.

This effectively lets pattern matching work on any object!

rect = Rectangle.new(2, 5)
SendMatch.new(rect) in { width: w, height: h }
w # => 2
h # => 5

We could make this even more concise by monkey-patching Object, so classes without a definition of #deconstruct_keys will automatically call methods instead:

class Object
  def deconstruct_keys(*args)
    SendMatch.new(self).deconstruct_keys(*args)
  end
end

rect in { width: w, height: h }
w # => 2
h # => 5

I hope this has given you a good insight into how pattern matching can be used in creative ways in Ruby 2.7! Remember that pattern matching is still an experimental feature, so any of these behaviours could change at any time.

I'd love to hear about how you're using pattern matching to streamline your Ruby code!

Getting the most out of Ruby strings

Aaron Christiansen — Tue, 29 Oct 2019 23:14:22 +0000

Ruby has a fantastic set of methods on its built-in String class, and it's really useful to know how to use them. This will go over some which you might not have heard of!

Note: All these methods are from Ruby 2.6.5, and they're all from the standard library; no Active Support required!

`String#+@` and `String#-@`

Strings in Ruby support both the unary plus (+) and minus (-) operators.

The + operator will return a "thaw" a frozen string, returning a mutable copy of it. If the string was already mutable, it has no effect.

a = "Hello".freeze
a.downcase! # Throws: FrozenError (can't modify frozen String)

# Now, let's thaw the string first...
a = +a
a.downcase!
a # => "hello"

The - operator does the opposite, returning a frozen copy of the string. Note that, if a frozen instance of that particular string already exists in Ruby's object pool, it'll return that instance instead.

a = "Hello"
a.downcase! # This is fine - our string isn't frozen yet
a = -a
a.upcase! # Throws: FrozenError (can't modify frozen String)

You can even combine these two operators for a terse (but horrible) string duplication operator. (Please use #dup over this, though!)

a = "Hello"
b = +-a
a.gsub!('e', 'a')
a # => "Hallo"
b # => "Hello"

The many uses of `String#[]`

The square-braces indexing syntax on strings has a gigantic range of use cases in Ruby. Here are the more typical ones you've probably seen before, which involve passing in numbers or ranges.

a = "Hello"
a[0]    # Get the character at index 0             => "H"
a[2, 3] # Get 3 characters, starting from index 2  => "llo"
a[1..3] # Get the characters between index 1 and 3 => "ell"

Did you know that you can also pass in a regex to get the part of the string which matches it? If there are multiple matches, this will always return the first match.

"foo bar baz"[/b../] # => "bar"

You can even use capture groups in your regex, and pass an index to specify which capture group to return:

"foo bar baz"[/b(..)/, 1] # => "ar"

Finally, you can pass a string as an index and it will act like include?, except returning the original search string if it was found and nil if it was not found.

s = "foo bar baz"

# String#include? returns a boolean
s.include?("bar")   # => true
s.include?("hello") # => false

# String#[] with a string argument returns a string or nil
s["bar"]   # => "bar"
s["hello"] # => nil

Even better, all of these work with indexed assignment too, so you can replace specific parts of your string effortlessly!

s = "foo bar baz"
s[/b../] = "hello"
s # => "foo hello baz"

`String#tr`

The tr method on strings translates one set of characters into another.

When called on a string and given two arguments, it maps characters from the first argument found in the string to the corresponding characters in the second argument.

This method is best explained with an example. Suppose we want to replace all occurrences of . with !, and vice versa. We could use the following tr call.

"Foo. Bar!".tr("!.", ".!") # => "Foo! Bar."

It can also be used to map ranges of characters onto one character, or onto another range. A caret (^) can be used as a "not" operator, too.

"abcdef".tr("a-d", "x")   # => "xxxxef"
"abcdef".tr("a-d", "w-z") # => "wxyzef"
"abcdef".tr("^a-d", "x")  # => "abcdxx"

The String#tr method is a powerful method which can make fiddly string replacements much easier if used effectively! Like many string methods, it also comes with a mutating form, String#tr!.

`String#squeeze`

The squeeze method removes adjacent duplicate characters in a string. By default, it will remove any characters, but it also takes an argument to select the range of characters to remove. This argument is in the same format is #tr's arguments - - for ranges and ^ for inversion.

This is most useful as a flexible whitespace-removal tool, which is much easier to read than a regex.

"aaabbaacccc".squeeze        # => "abac"
"aaabbaacccc".squeeze("a-b") # => "abacccc"

# One of the best uses...
"Look   at all  this   whitespace!".squeeze(" ") # => "Look at all this whitespace!"

I hope this has taught you something new! If you have any tips for dealing with Ruby strings effectively, leave them in a comment below.

Causes - a programming language feature concept

Aaron Christiansen — Thu, 15 Aug 2019 17:53:58 +0000

Introduction

A typical programming language's standard library is often filled with methods which clearly complement each other.

A clear and common relationship is between an operation method, and a predicate method which becomes true if that operation has taken place. For example, immediately after calling the clear method on a Ruby Array instance, the empty? predicate becomes true. There are plenty of other ways to mutate an array such that it satisfies the empty? predicate, but clear is guaranteed to satisfy it every time with just one method call. To summarise: calling clear causes empty? to be true.

We've now established that there's a relationship between these methods. What if we could declare relationships like this to the language, and what advantages would this bring to both the programmer and the language itself?

Proposing a language which includes causes

Let's take a look at a theoretical language which allows relationships like this to be declared. We'll call these relationships causes, as calling one method causes another predicate to become true.

I'll be writing Ruby-style pseudocode, as implementing complex functionality like this using Ruby's metaprogramming functionality seems feasible.

Here's how the empty? and clear methods could be declared in the Array class, with a cause given for clear:

class Array
  # ...other definitions...

  def empty?
    length.zero?
  end

  causes { empty? } # <-- Here's our defined cause
  def clear
    each do |item|
      delete(item)
    end
  end
end

This code would associate the empty? and clear methods, specifying that an invocation of clear causes the predicate empty? to become true.

This could also be extended to more complex causes relying on the arguments passed to the method. For example, if an item has just been appended to an array (using << in Ruby), then it must be the last item in that array.

A definition of this cause could look like:

cause { |item| last == item } 
def <<(item)
  # ...array append logic...
end

Here, the predicate is more than a simple method call - the last method is invoked and has its result compared with the item to append.

What is gained from doing this?

Thinking of a method's causes seems like yet another hurdle when writing code, so why bother?

Effortless idempotency

Because it is specified exactly what will happen when they're called, simple methods with causes can be called idempotently. That is, if their effect has already happened, it shouldn't happen again.

Suppose you have an array of bytes called bytes supplied by a user, and you're about to process it in a way which requires a null terminator to be at the end. The user may or may not have included this null terminator. In traditional Ruby code, this could be done like so:

bytes << 0 unless bytes.last == 0

Introducing an idempotent call syntax into our language, where we add a ? prefix to the method's name, we could do this:

bytes ?<< 0

This is possible because, by specifying what a method causes formally, we know exactly what << will do and therefore can determine if it is required to call it. This particular code snippet would evaluate the predicate specified in the cause for <<; recall that this predicate is |item| last == item. If this predicate is false, i.e. the last item is not 0, then << would be executed.

Bringing declarative programming to a procedural language

If it is known what condition a method satisfies, then there is also the power to flip this information around. Given a condition we want to satisfy, we can find a method which causes it.

Suppose we have a variety of sorting algorithms defined on the language's array type, each with a cause specified such that the array satisfies a sorted? predicate after invoking them. To sort an array instance, we can write code that asks the language to satisfy the sorted? predicate, using a satisfy keyword:

array = [4, 3, 7, 2]
satisfy array.sorted?
p array # => [2, 3, 4, 7]

The language is aware of which methods will satisfy the sorted? predicate and can select one accordingly, perhaps using an intelligent interpreter which tries each candidate method over time and establishes which method is faster for particular instances.

Extra assertions for free

Each cause could optionally act as a postcondition assertion, where it runs after the associated method has been invoked. This is similar to the contract features built into Eiffel or D. For instance, in the clear and empty? example, the language will verify that the list really is empty when clear returns, and throw an assertion error if not. This could help catch bugs earlier.

Easier-to-read code

Each cause can act as documentation, showing a programmer reading the code what implications a method will have. In addition, because causes can act as assertions, it is guaranteed that these implications are true.

Conclusion

I think that implementing causes in a language could be a clean, unintrusive way to make code more expressive and easier to read. What do you think?

First impressions of Sorbet, a Ruby type checker

Aaron Christiansen — Wed, 03 Jul 2019 18:11:42 +0000

Sorbet is a type checker for Ruby, built by Stripe. Its release has been long-anticipated, having been discussed at a handful of Ruby conferences, and being available to test out in an online playground called sorbet.run.

Now it's been released as an open-source project, and I've been eagerly playing around with it. I'll start off by saying that, overall, my experience with Sorbet has been quite positive.

I've always been a fan of static typing, and the lack of it in Ruby has sometimes put me off writing large projects with it. Sorbet could be enough to tempt me back - but only after it's been given a bit of time to mature. In this post I'll be discussing my thoughts on Sorbet in-depth.

What I like

It's still Ruby!

This is one of the nicest things about Sorbet, made possible by Ruby's flexible and versatile syntax. Ruby code which uses Sorbet is still just Ruby. Type signatures are added through method calls above each definition:

sig { params(a: Integer, b: Integer).returns(Integer) }
def add_ints(a, b)
  a + b
end

All you need to run this is the sorbet-runtime gem, which defines sig and any other calls which are sometimes used to declare types.

This is in stark contrast to TypeScript's approach, which is an entirely new language compiling down to JavaScript. Creating a new language adds an entirely new build step and complicates things somewhat. (In many ways, what TypeScript is to JavaScript is very similar to what Sorbet is to Ruby, so I'll be drawing quite a few comparisons between them here.)

It's gradual

Once you add Sorbet to your project, you don't need to go through and add type signatures to every single method immediately. (However, shameless plug, my project will do that if you have YARD docs!)

Instead, you can gradually add types to your existing codebase, and Sorbet will only check methods with type signatures.

It brings an awesome language server with it

Sorbet needs to have a deep understanding of the structure of your codebase to do its job, and as a great bonus feature it is exposes this information over the Language Server Protocol. This means that your editor can see this information and offer rich autocomplete, jump-to-definition, and more.

With the Sorbet extension for Visual Studio Code, I got some of the best autocompletion I've ever seen in an editor for Ruby. Other solutions I've tried for Ruby in VS Code didn't even come close; I'd say Sorbet is about on par, if not better, than RubyMine's completion.

It makes code safer

Obviously, being a static type system, Sorbet brings all the advantages of static typing to your Ruby code. Accurate type signatures with the aid of a type checker can spot many common errors within the code you're writing.

What I don't like as much

It doesn't support structural interfaces

I feel like this is currently one of Sorbet's largest weak points. One of the aspects where TypeScript really knocked it out of the park is in supporting all of the stuff that people frequently do in JavaScript.

Like Ruby, JavaScript is quite a malleable language, but TypeScript is able to encompass virtually all of JavaScript's behaviours. It does this by using structural typing with interfaces. The programmer can create an interface type defining properties and methods, and then all objects which contain these properties and methods match the interface type automatically.

For example, take this TypeScript code:

interface CanSpeak {
  speak: (msg: String) => any;
}

class Person {
  constructor(public name: String) {}

  speak(msg: String) {
    console.log(msg);
  }
}

var speakable: CanSpeak = new Person("Aaron");
speakable.speak("Hello!");

Although we have not explicitly said that Person implements the method required for the CanSpeak interface, the TypeScript compiler can determine this for itself, and the type check succeeds.

On the other hand, Sorbet uses a nominative type system, more similar to what you may find in C# or Java. Interface implementation must be explicit, as you can see in this snippet of Ruby with Sorbet:

module CanSpeak
  extend T::Helpers
  extend T::Sig

  interface!

  sig { abstract.params(msg: String).returns(T.untyped) }
  def speak(msg); end
end

class Person
  extend T::Sig
  include CanSpeak # <-- explicit interface inclusion

  def initialize(name)
    @name = name
  end

  sig { params(msg: String).void }
  def speak(msg)
    puts msg
  end
end

speakable = T.let(Person.new("Aaron"), CanSpeak)

Without the include CanSpeak line, this snippet does not pass type checking.

The issue with this system is that it means portions of code may have to be refactored entirely to include a particular interface. The even larger problem is that, if the code isn't your own, you can't edit that to include the interface! This could lead to tricky scenarios when adopting Sorbet in a larger codebase which depends on gems.

It doesn't support options hashes very well

As you may well know, it's extremely common practice for a method to accept an options hash: a hash parameter containing any extraneous options for that method. For instance, in an API library, you might see a method definition like:

def connect(endpoint, opts={})
  # ...
end

You could specify values like your API key in the opts hash. This helps to keep method signatures concise for methods with huge numbers of optional arguments.

Disappointingly, Sorbet doesn't support hashes like this very well. You have a few options of what to use in your type signature, none of which are perfect:

Hash, which just accepts any keys or values. This is almost like having no type checking at all.
T::Hash[Symbol, T.untyped], which restricts keys to symbols but still allows any value and doesn't enforce that certain keys are specified.
Use a shape, which allows both key and value types to be specified. However, you can't have optional keys, which is one of the most common use cases of these options hashes.

What's not ready yet, which is holding me back from using Sorbet everywhere

Most gems don't have types yet

Sorbet has only just been released, and as such it hasn't been adopted in most projects yet. This kind of adoption takes time, and I have no doubt that the Ruby community will come together to add type signatures to most popular gems in no time.

Plugins aren't ready

Ruby's wealth of metaprogramming-powered domain specific languages are one of the things that makes Ruby so special. Naturally, it's not really possible to add type safety to these using a traditional type system, so Sorbet will have a plugin system to allow types to be generated on-the-fly.

This plugin system is currently not finalised and has some serious drawbacks. With that said, once the plugin system is fully implemented and matured, it will be really interesting to see how extensively static typing can be employed across existing Ruby tools.

In conclusion

Sorbet is a fantastic start in adding static typing to Ruby; the team at Stripe have done a brilliant job! It's currently great for many small projects, but for larger ones I believe it needs time to fully mature and build up a community following. I'm looking forward to seeing how Sorbet develops in the near future.

DEV Community: Aaron Christiansen

Delta Pico - what I've learned by building a graphing calculator

Development and Iteration

What have I learned?

Mathematical editors are tricky

Embedded Rust isn't quite mature

Trying new stacks is fun

The Raspberry Pi Pico is a wonderful platform

In conclusion

Elixir's bitstrings - the data type I didn't know I wanted

Thinking About Strings

Introducing Bitstrings

Working With Bitstrings

Conclusion

Removing functionality the Ruby way - a look into SortedSet on Ruby 3

"yes_i_did" - A tiny gem to rerun your last IRB command with the "did you mean" suggestion if you make a typo

Flashing your PineTime using an ST-Link and OpenOCD

Prerequisites

Building OpenOCD

Configuring OpenOCD

Starting OpenOCD

Connecting to OpenOCD

Flashing

Need a pre-built firmware?

Taking pattern matching further in Ruby 2.7

How does pattern matching work?

As the case may be...

A methodical approach

Getting the most out of Ruby strings

String#+@ and String#-@

The many uses of String#[]

String#tr

String#squeeze

Causes - a programming language feature concept

Introduction

Proposing a language which includes causes

What is gained from doing this?

Effortless idempotency

Bringing declarative programming to a procedural language

Extra assertions for free

Easier-to-read code

Conclusion

First impressions of Sorbet, a Ruby type checker

What I like

It's still Ruby!

It's gradual

It brings an awesome language server with it

It makes code safer

What I don't like as much

It doesn't support structural interfaces

It doesn't support options hashes very well

What's not ready yet, which is holding me back from using Sorbet everywhere

Most gems don't have types yet

Plugins aren't ready

In conclusion

As the `case` may be...

`String#+@` and `String#-@`

The many uses of `String#[]`

`String#tr`

`String#squeeze`