DEV Community: James Harton

Lightweight dev tools.

James Harton — Thu, 10 Aug 2023 22:31:13 +0000

I’ve been forcing myself to use a barely usable laptop for any couch coding I do because I have a puppy who likes to jump into my lap - regardless of whether there’s already something in it.

Usually I'd use a 15" MacBook Pro, but since both I and my employer would be pretty upset if it was damaged I've been using something with a little lower stakes that has been sitting unused on a shelf in my garage for quite a while.

It’s a bit of an interesting challenge and has forced me to re-examine some of my tool usage. I started by a minimal install of Debian “bookworm” with the XFCE Desktop Environment which chews through much fewer resources than the default GNOME 43 based environment (although more than LXDE - but there still has to be room for aesthetics).

This is already a bit of a change because when I'm not using the MacBook Pro, I am using my desktop which has Pop!_OS 22.04 installed.

Terminal emulator

I did find that XFCE’s terminal emulator was pretty slow, so I installed Alacritty - a lightweight terminal written in Rust.

Alacritty doesn’t yet support ligatures, tabs or split panes - so be warned if that’s a show stopper for you. It does happily render Nerd Font symbols though. Alacritty is so fast I also decided to make the switch from iTerm2 on my mac.

Shell environment

I’ve been using oh-my-zsh for years but over time more and more has been added to it (both by me and upstream) resulting in a great - but slow - experience. Running it on a machine with so few resources was definitely not an option.

After a little research I came across starship (also written in Rust) which is a “blazing-fast, and infinitely customizable prompt for any shell”. I deleted almost all of my .zshrc and replaced it with eval "$(starship init zsh)". I also had to manually add hooks for asdf, direnv and a couple of other tools that I had been relying on oh-my-zsh plugins for.

The result? Well I’m speechless. I had forgotten what it was like to have a near-instantaneous shell prompt. I’ve switched my profile on all my machines to starship.

Programming environment

I used to be a pretty heavy Vim user (RIP Bram), but when I started doing a lot of remote pairing with less experienced devs I realised that they were sometimes having trouble following along when I was driving. Their experience was basically "click click click" some text whizzes around the screen.

So I made the switch to Code. I came for the accessibility and stayed for devcontainers, remote development and the language server protocol. Again though, Code is a great tool but is built using web technologies and Electron and when loaded down with a full-suite of extensions it can feel sluggish on the most powerful of machines.

Here comes Lapce; a "Lightning-fast and Powerful Code Editor" written in Rust (are you seeing a theme here?). When they say "lightning-fast" they're not kidding. Despite the website saying it's "pre-alpha" it has a surprisingly full feature set:

Tree-sitter based syntax highlighting
Remote development
Language server protocol
WASI-based plugin system

What's missing is mainly docs and plugins (and docs on how to write plugins).

The first codebase I opened was a Rust project and it's support for Rust is nothing short of incredible. I guess it would be given that's where it originates. It seemlessly integrated with rust-analyser, cargo fmt, etc and felt like a very mature offering.

Sadly, this isn't true for all languages. I spend most of my days working on and with the Ash project in Elixir. Whilst Lapce has syntax highlighting for Elixir there is no language server plugin available yet. There are at least two plugins in development, but given I can't figure out how to install them in my local editor to test them I can't speak to their quality.

I decided I can live without elixir-ls when couching in return for having a usable editor. When the plugin ecosystem and documentation matures I can see myself switching to using Lapce for my primary editor.

Browser

For better or worse, I spend a lot of my time on the web. Having a quick web browser is kind of important, but since I also seem to mostly write web applications I also need one that renders things in a conventional way.

This immediately rules out a bunch of lightweight browser options like lynx or Dillo.

I usually use Firefox for general browsing and Chromium when I need the good dev tools. I think I have to live with having Chromium installed for when I need it, but Firefox is just too weighty for this little machine. I settled on Falkon which is a nice balance between performance and standards. It uses the Chromium renderer under the hood, so webpages usually look how I expect them to. It is thankfully missing some much loved browser "features" such as submitting "http://localhost:4000" to a search engine rather than loading the page.

Summary

Thanks for coming along on this little tour of lightweight dev tools. I hope in my struggles to use a dumb old computer you have found something useful to speed up your computing life.

In defence of idiotic products

James Harton — Sat, 20 Jun 2020 07:32:32 +0000

I was talking to a friend recently about how work on my robotics project and I said that over the weekend I had "popped a few things off my yak stack". Suddenly I realised what I had said, and immediately wondered what it would be like to have a todo list that behaved like a stack machine with a limited set of instructions available to the user. Of course it's not very practical but since when has that stopped anyone?

If, like me, you get a lot of joy from the process of building a product then you'll probably also know that it's hard to find a project that's the right size to build by yourself. My last product attempt was a management system for sailing regattas - unfortunately it wound up being put on the back burner, and is likely to stay there indefinitely. Making things that people will part with money for is hard - especially if you're doing it on your own. Unfortunately I can't stop making things.

So I decided to build YakStack. You're welcome to create an account and try making your own stacks. Currently the only instructions available are "push", "pop" and "swap", but it's enough to start making your own task lists. There's plenty of features missing (like deleting stacks!) but I'm interested in your feedback and ideas.

Now, given the title of this missive you'll have probably surmised that I don't expect YakStack to change the way we organise projects, or even be particularly useful. Regardless, there are a lot of skills that can be honed by building silly things. Thinking about user experience, feature prioritisation and product-market fit are still worthy uses of one's time, even if the product will never really fit the market.

As for the technical side it's built with Phoenix and backed by PostgreSQL. Elixir is the language I enjoy working with most, and since I currently work in JavaScript for my day job it is very nice to have a project which lets me practice with Phoenix and LiveView and keep up to date with the changing conventions in that community.

I hope that my stupid product made you smile and brought a small amount of joy into your life.

Elixir for Robots

James Harton — Fri, 12 Jun 2020 04:39:10 +0000

I've been working on a robotics project for quite some time. Around the time that I started working on it in earnest I also started learning Elixir. For those of you who don't know, Elixir is a programming language that lives on top of the Erlang virtual machine (called BEAM). Erlang's semantics were designed for Ericsson to run on it's telephony switches. The requirements of massive concurrency, fault tolerance and hot code reloading forced the team at Ericsson to make a bunch of really interesting decisions causing the creation of an immutable, functional language based around a robust actor system, message passing and network-transparent clustering.

Erlang's original creator, Joe Armstrong died early in 2019. The New Stack has a great article summarising Joe's contributions to computer science.

As well as the Erlang language and the virtual machine, Erlang also ships with a massive standard library (called OTP). OTP includes primitives for everything you'd expect in a modern language, and much more.

In addition to the massive boon that is OTP, the advent of Elixir has energised the Erlang community by introducing thousands of new developers to Erlang and it's ecosystem. Elixir keeps the semantics of Erlang and builds on top of them. A new syntax, a hygienic macro system, and modern tooling are just some of the many features that Elixir brings to the table. On top of that, you can still easily make use of years worth of existing Erlang libraries and OTP.

Current solution: ROS

The current go-to for open source (and commercial) robotics is The Robot Operating System (or ROS for short). Despite being branded as an operating system ROS is really just a collection of applications, libraries and tools for orchestrating them together. You can install ROS on pretty much any Linux machine.

The architecture of ROS is based around a few core concepts:

Message Passing - a pub/sub mechanism for the various applications to
send messages to each other or observe the state of other components.
Remote Procedure Call - a mechanism for doing synchronous RPC between
components.
Distributed Parameter System - a shared configuration registry for all
ROS components.

Additionally, the collaboration around ROS has given birth to several key components needed by nearly every robot project:

tf (or "transform") is used to keep track of a
robot's geometry. It is used to track the locations and coordinate frames
of all parts of the robot including sensors, bodies and actuators.
URDF (or "Unified Robot Description Format")
is an XML dialect which is used to describe the physical properties of the
robot. This is used by pretty much every part of ROS (including tf) to
understand how to simulate or manipulate the robot.
rviz provides a 3D visualisation of sensor
data, combined with the robot's URDF information.

Whilst I think that ROS is a pretty amazing tool, it's also fair to say that there are some valid criticisms. Due to it's nature as a specification for the loose-coupling of components it can be difficult to find documentation or guides for new users. I also found it is difficult to work with a ROS project unless you do your development on a Linux machine.

Elixir for Robotics

Looking at the list of ROS core concepts it seems like Erlang has these covered (and more). In fact a number of them are covered by the single Registry module in the Elixir standard library.

Missing parts:

A robot kinematics library. I have been on-again-off-again working on kinemat which is a solution for translating between reference frames and one day may be able to solve forward and inverse kinematics.
A URDF parsing and generating library, probably also as part of kinemat.
Drivers for lots more sensors and components. I wrote wafer to assist with this.
Integrations for common tools and libraries, like Gazebo, OpenCV and OMPL.

The IoT problem

Whilst ROS solves the problems of actually running a robot system, it doesn't appear to include any solutions for the "the IoT problem" (please correct me if I'm wrong) - remotely managing what could be a large number of systems which are geographically distributed, may be connected via less than stellar network connections and can be powered on and off without your control. How do we deal with issues? How do we deploy new versions of our code? How do we handle software updates for the underlying operating system?

This is a problem that is underestimated by almost every team that builds these types of edge-computing systems. I've seen it go wrong first-hand. It's such an important problem to me that I took a job at Balena to help solve it.

Elixir for IoT

A lot of work has gone into making Elixir a first-class choice for embedded systems. Specifically the Nerves Project and Elixir Circuits teams have done a heap of work making sure that ARM-based single-board computers (like the Raspberry Pi and BeagleBone) are well supported - not just as Linux devices that Erlang can run on, but also with easy access to hardware, like I²C, SPI and GPIO.

Nerves also includes support for remote development, firmware flashing and over-the-air updates. Recently, with the addition of NervesHub, they've started attacking some of the problems of fleet management too. It's an exciting time to be excited about Elixir and IoT.

Summary

I have a vision of a modern robotics toolchain, built using Elixir, with new and emerging IoT tools at their base. Not just for hobbyists, but for commercial robotics product developers too. I hope I have succeeded in sharing that vision with you and that you're just as excited about it as I am. If you can help, that'd be amazing.

Fold for COVID

James Harton — Thu, 16 Apr 2020 23:50:15 +0000

Hi.

As some of you fine folks may know, I work as a backend software engineer for Balena, occasionally building cool robots or shooting things with lasers.

I don't usually post about work stuff, but I wanted to share my pride in our team's efforts to get the Fold For COVID site up and running. We're using our IoT platform to provide prebuilt images that will boot a bunch of devices directly into BOINC and start folding proteins for the Rosetta@Home project at the University of Washington's Baker Lab.

This project was initiated by our very own David Tischler, with support from a bunch of other Arm folks got the Rosetta@Home project compiling on ARM64. If you want more information about that you can read David's blog post about it.

We don't just support Arm, if you have an intel machine lying around unused you can download an image which will flash it from a USB stick. Time to drag that old laptop out of the cupboard and get started. If you want to use a Raspberry Pi then you'll need at least a Pi 4 with > 2GB of RAM.

What are you waiting for? Go Fold For COVID!

Towards an ethical open source license

James Harton — Tue, 16 Apr 2019 00:47:25 +0000

Recently there has been a lot of discussion about 996.ICU and this has highlighted an uneasy feeling I've had about Open Source for a very long time.

The Open Source Initiative publishes the Open Source Definition which defines, amongst others, the following criteria for a license to be considered Open Source:

No Discrimination Against Persons or Groups The license must not discriminate against any person or group of persons.

No Discrimination Against Fields of Endeavor The license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, or from being used for genetic research.

These criteria were originally derived from the Debian Free Software Guidelines.

I write Open Source Software (OSS) for fun, not profit. I've written a lot of code that is useless even to me and given it away to see if anything interesting happens to it. I'm not a member of the Apache Foundation or the Mozilla Foundation, although I think those organisations do great work.

People write OSS for a lot of reasons; for the lols, to learn new things, to be part of a community, because they need it for their job or just because they can. There are as many motivations for writing and giving away software as there are people doing it.

Writing OSS has always been a political act. When Richard Stallman founded the Free Software Foundation he was making a political statement. That political statement represents his desire for universal freedom to study, distribute, create, and modify computer software. The OSI was founded by Bruce Perens and another guy whose name I won't mention because he's so toxic. I don't find it surprising that our "global movement" of giving away software for free has been framed by cisgender, heterosexual white dudes from the east coast of the United States. It has been said that the ability to write and contribute to OSS requires a certain level of privilege - enough spare time and resources to participate without negatively impacting your work or family life, for example - as such it's hardly surprising that the roots of our movement stem from a place of wealth and have a weirdly American flavoured libertarian bent.

So here's where I'm at: I want to write software and give it away for free. I want my friends or complete strangers on the internet to use my software for wonderful and surprising things. I even want people to build businesses using it if they can think of a way to do that. However, there are things I want to limit: I want to stop companies which mistreat their labour force, damage the environment or public health, discriminate against people, or make weapons from using my software. These are my core beliefs. Why shouldn't I be allowed to use a license that reflects them, when the licenses endorsed by the so-called Open Source Definition or Free Software Foundation so obviously reflect the beliefs of those that founded those organisations?

The idea of adding limits to open source licenses, other than those which enable derivative works to stay open source has been seen as taboo in our community. Why is this and what can we do about it?

Update 5th June 2019:

I've only just found out about the Do No Harm software license. I like it.

Events vs Actions in Ember.js

James Harton — Mon, 28 Jan 2019 22:43:08 +0000

Recently I was working with some of my team on an Ember component which needed to react to JavaScript events they expressed some confusion about the difference between JavaScript events and Ember's Action system. I decided to write up the basics here.

Blowing bubbles

One of the fundamental behaviours of JavaScript DOM events is bubbling. Let's focus on a click event, although the type of event is arbitrary. Suppose we have an HTML page composed like this:

<html>
<body>
  <main>
    <p>Is TimeCop a better time travel movie than Back To The Future?</p>
    <button>Yes</button>
    <button>No</button>
    <button>Tough Call</button>
  </main>
</body>
</html>

Supposing I load this page in my browser and I click on the "Tough Call" button (one of three correct answers on this page) then the browser walks down the DOM to find the element under the mouse pointer. It looks at the root element, checks if the coordinates of the click event are within that element's area, if so it iterates the element's children repeating the test until it finds an element that contains the event coordinates and has no children. In our case it's the last button element on the screen.

Once the browser has identified the element being clicked it then checks to see if it has any click event listeners. These can be added by using the onclick HTML attribute (discouraged), setting the onclick property of the element object (also discouraged) or by using the element's addEventListener method. If there are event handlers present on the element they are called, one by one, until one of the handlers tells the event to stop propagating, the event is cancelled or we run out of event handlers. The browser then moves on to the element's parent and repeats the process until either the event is cancelled or we run out of parent elements.

Getting a handle on it

Event handlers are simple javascript functions which accept a single Event argument (except for onerror which gets additional arguments). MDN's Event Handlers Documentation is very thorough, you should read it.

There are some tricky factors involving the return value of the function; the rule of thumb is that if you want to cancel the event return true otherwise return nothing at all. The beforeunload and error handlers are the exception to this rule.

A little less conversation

Ember actions are similar in concept to events, and are triggered by events (click by default) but they propagate in a different way. The first rule of Ember is "data down, actions up". What this means is that data comes "down" from the routes (via their model hooks) through the controller and into the view. The view emits actions which bubble back "up" through the controller to the routes.

Let's look at a simple example. First the router:

import Router from '@ember/routing/router';

Router.map(function() {
  this.route('quiz', { path: '/quiz/:slug'})
});

export default Router;

Now our quiz route:

import Route from '@ember/routing/route';

export default Route.extend({
  model({ slug }) {
    return fetch(`/api/quizzes/${slug}`)
      .then(response => response.json());
  }
});

Now our quiz template:

<p>{{model.question}}</p>
{{#each model.answers as |answer|}}
  <button {{action 'selectAnswer' answer}}>{{answer}}</button>
{{/each}}

A quick aside about routing

When we load our quiz page Ember first enters the application route and calls it's model hook. Since we haven't defined an application route in our app Ember generates a default one for us which returns nothing from it's model hook. Presuming we entered the /quiz/time-travel-movies URI the router will then enter the quiz route and call the model hook which we presume returns a JSON representation of our quiz. This means that both the application and the quiz route are "active" at the same time. This is a pretty powerful feature of Ember, especially once routes start being deeply nested.

More bubble blowing

When an action is fired Ember bubbles it up the chain; first to the quiz controller, then to the quiz route and then to the parent route and so on until it either finds an action handler or it reaches the application route. This bubbling behaviour is pretty cool because it means we can handle common actions near the top of the route tree (log in or out actions for example) and more specific ones in the places they're needed.

Notably Ember will throw an error if you don't have a handler for an action, so in our example above it will explode because we don't handle our selectAnswer in the controller or the route.

The lonesome component

Ember's "data down, actions up" motto breaks down at the component level. Ember components are supposed to be atomic units of UI state which don't leak side effects. This means that our options for emitting actions out of components are deliberately limited. Actions do behave exactly as you'd expect within a component, except that there's no bubbling behaviour. This means that actions that are specified within a component's template which do not have a corresponding definition in the component's javascript will cause Ember to throw an error.

The main way to allow components to emit actions is to use what ember calls "closure actions" to pass in your action as a callable function on a known property of your component, for example:

{{my-button onSelect=(action 'selectAnswer' answer) label=answer}}

import Component from '@ember/component';
import { resolve } from 'rsvp';

export default Component({
  tagName: 'button',
  onSelect: resolve,

  actions: {
    selectAnswer(answer) {
      return this.onSelect(answer);
    }
  }
});

This is particularly good because you can reuse the component in other places without having to modify it for new use cases. This idea is an adaptation of the dependency injection pattern.

The eventual component

There are three main ways components can respond to browser events. The simplest is to use the action handlebars helper to respond to your specific event, for example:

<div {{action 'mouseDidEnter' on='mouseEnter'}} {{action 'mouseDidLeave' on='mouseLeave'}}>
  {{if mouseIsIn 'mouse in' 'mouse out'}}
</div>

As you can see, this can be a bit unwieldy when responding to lots of different events. It also doesn't work great if you want your whole component to react to events, not just elements within it.

The second way to have your component respond to events is to define callbacks in your component. This is done by defining a method on the component with the name of the event you wish to handle. Bummer if you wanted to have a property named click or submit. There's two things you need to know about Component event handlers; their names are camelised (full list here) and the return types are normalised. Return false if you want to cancel the event. Returning anything else has no effect.

import Component from '@ember/component';

export default Component({
  mouseIsIn: false,

  mouseDidEnter(event) {
    this.set('mouseIsIn', true);
    return false;
  },

  mouseDidLeave(event) {
    this.set('mouseIsIn', false);
    return false;
  }
});

The third way is to use the didInsertElement and willDestroyElement component lifecycle callbacks to manually manage your events when the component is inserted and removed from the DOM.

export default Component({
  mouseIsIn: false,

  didInsertElement() {
    this.onMouseEnter = () => { this.set('mouseIsIn', true); };
    this.onMouseLeave = () => { this.set('mouseIsIn', false); };
    this.element.addEventListener('mouseenter', this.onMouseEnter);
    this.element.addEventListener('mouseleave', this.onMouseLeave);
  },

  willRemoveElement() {
    this.element.removeEventListener('mouseenter', this.onMouseEnter);
    this.element.removeEventListener('mouseleave', this.onMouseLeave);
  }
});

Note that using either of the last two methods you can use this.send(actionName, ...arguments) to trigger events on your component if you think that's cleaner.

Conclusion

As you can see, actions and events are similar but different. At the most basic level events are used to make changes to UI state and actions are used to make changes to application state. As usual that's not a hard and fast rule, so when asking yourself whether you should use events or actions, as with all other engineering questions, the correct answer is "it depends".

Building a stack-based virtual machine, part 7 - conditionals

James Harton — Thu, 20 Sep 2018 00:56:04 +0000

In the last episode we learned all about how to make our stack machine handle function calls. A pretty handy language feature, if I do say so. Today we're going to learn how to model conditionals. If you've been following along at home then you may have already figured this out for yourself, but in case you haven't let's walk through how it's done.

Note that we have all the infrastructure in place already to make conditionals work in our programs, it's all down to making new instructions that can optionally perform jumps.

As with our previous example, we're going to need an operand type which can contain either integers or strings so that we can find the appropriate labels to jump to.

enum Operand {
    I(i64),
    S(String),
}

impl Operand {
    fn to_i(&self) -> Option<i64> {
        match self {
            &Operand::I(i) => Some(i),
            _ => None,
        }
    }

    fn to_s(&self) -> Option<&str> {
        match self {
            &Operand::S(ref s) => Some(s),
            _ => None,
        }
    }
}

impl From<i64> for Operand {
    fn from(i: i64) -> Self {
        Operand::I(i)
    }
}

impl<'a> From<&'a str> for Operand {
    fn from(s: &'a str) -> Self {
        Operand::S(s.to_string())
    }
}

Next we're going to need our standard push instruction:

fn push(machine: &mut Machine<Operand>, args: &[usize]) {
    let arg = machine.get_data(args[0]).clone();
    machine.operand_push(arg);
}

Next an unconditional jump. This instruction uses the machine's jump method to move the IP straight to the position identified by the label. Obviously you'd want to do more checking than what I'm doing here (does the label exist? what if the argument is an integer), but this is a blog so YOLO.

You could also change this to read the jump label or location off the stack. That'd be rad too.

fn jump(machine: &mut Machine<Operand>, args: &[usize]) {
    let label = machine.get_data(args[0]).clone();
    machine.jump(label.to_s().unwrap());
}

The last, and most magic piece is our new jump_if instruction.

This instruction pops the top operand off the stack and assumes it's an integer. If the integer is non-zero then it performs a jump to the specified label, otherwise it's a no-op and the machine just carries on with the next instruction.

fn jump_if(machine: &mut Machine<Operand>, args: &[usize]) {
    let condition = machine.operand_pop().to_i().unwrap();
    if condition != 0 {
        let label = machine.get_data(args[0]).clone();
        machine.jump(label.to_s().unwrap());
    }
}

Let's assemble our instruction table:

fn instruction_table() -> InstructionTable<Operand> {
    let mut it = InstructionTable::new();
    it.insert(Instruction::new(0, "push", 1, push));
    it.insert(Instruction::new(1, "jump_if", 1, jump_if));
    it.insert(Instruction::new(2, "jump", 1, jump));
    it
}

Next we can build a very simple program which pushes an operand onto the stack and if it's "true" then jumps to the "if_true" label, otherwise it jumps to the "end" label.

fn conditional_program(condition: Operand) -> Operand {
    let it = instruction_table();
    let mut builder: Builder<Operand> = Builder::new(&it);

    // Push our condition operand into the stack
    builder.push("push", vec![condition]);

    // Add our `jump_if` instruction and tell it to jump to the `if_true` label.
    builder.push("jump_if", vec![Operand::from("if_true")]);

    // These instructions will be skipped if the operand is true:
    // * Push "it was false" onto the stack.
    // * Unconditionally jump to the end of the program.
    builder.push("push", vec![Operand::from("it was false")]);
    builder.push("jump", vec![Operand::from("end")]);

    // This instruction will be skipped of the operand is false:
    // * Push "it was true" onto the stack.
    builder.label("if_true");
    builder.push("push", vec![Operand::from("it was true")]);

    // Points at the end of the program, which will cause the machine to halt.
    builder.label("end");

    // Run the program:
    let code = Code::from(builder);
    let constants: WriteManyTable<Operand> = WriteManyTable::new();
    let mut machine: Machine<Operand> = Machine::new(code, &constants, &it);
    machine.run();

    // Return the operand at the top of the stack to the caller:
    machine.operand_pop()
}

Now we can write our tests to prove that it works:

#[test]
fn condition_true() {
    let result = conditional_program(Operand::from(1));
    assert_eq!(result.to_s().unwrap(), "it was true")
}

#[test]
fn condition_false() {
    let result = conditional_program(Operand::from(0));
    assert_eq!(result.to_s().unwrap(), "it was false")
}

As you can see modeling conditionals is pretty straight forward, although you will likely have to keep track of a bunch of labels. You might have also noticed how you could model a switch statement where conditional clauses can fall through into the next clause.

This completes our whirlwind tour of how stack-based virtual machines work. I sure hope you've learned something. I encourage you to read the source and feel free to ask any questions you have. If I don't know the answer I'd sure like to find out.

Please keep using Ruby

James Harton — Wed, 19 Sep 2018 23:37:18 +0000

There was a lot of mixed response to my last post about Ruby - and rightly so. Although I still don't think that any of my arguments are wrong, I talked only about the technical reasons why Ruby isn't a good fit for modern development. In this post I want to talk about why Ruby can still be a good choice for you.

Rails is not Ruby

A lot of people who consider themselves Rubyists are really Railsists. I have worked with a number of developers who whilst able to play the role of an intermediate developer (ie consistently deliver unsurprising results) on a Rails project needed to be taught basic programming skills like opening a file or calculating the rolling average of an infinite stream. This isn't bad per se; these developers are more than able to handle most of the day-to-day tasks they're confronted with and rely on more experienced developers for assistance where needed.

Last year I took the Ruby Association Certified Ruby Programmer Silver exam as a way of assessing it's use as a training tool. Whilst there were not many surprises for me it was an extremely valuable (and cost effective) way to level up our less experienced developers and although they found it challenging the feedback from those that have sat the exam was very positive. If you're stuck in the rut of just putting code in the same three folders over and over again I suggest you broaden your horizons and write some Ruby.

Ruby wears its blemishes with pride

As I said in my last post, Ruby's DNA comes from Perl and Smalltalk. It also has a 23 year history of adds, moves and changes to it's standard library. As a consequence the standard library API's are all over the place. Some of them are obviously wrappers around libc, some of them are beautifully functional and some of them are ill-advised Perlisms. Learning the Ruby standard library well exposes you to number of different programming styles and concepts and regardless of whether you or I think they're good or not you're definitely going to learn something.

Ruby can be beautiful

When written well, Ruby can be some of the most expressive and easiest to understand code I've ever seen. When the developer keeps principals like SOLID and tell don't ask in mind they can achieve truly marvelous solutions which will be functional, beautiful and easier for the next programmer to change.

Resources for learning

Since the explosion of Rails made the Ruby programming language so popular outside of Japan the number of resources to learn and master Ruby has exploded. There are so many books, videos and blog posts about Ruby that you could probably read and watch them until the heat death of the universe. What a way to go.

I want to point out some of the truly exceptional Ruby books that have materially improved my programming abilities; not just in Ruby but in every language I interact with.

Sandi Metz' POODR is an amazing book for those who want to level up their OO design skills. She helps you figure out how to spot the edges of abstractions that are missing in your programs.
Tom Stuart's Understanding Computation is an excellent learning resource to understand how computers really work with clear understandable examples in Ruby. If you're like me and never studied computer science, then this book will give you a better understanding of the fundamentals of computing that we often forget when working in higher level languages.

The C word

I'm certainly not the first person to point out how truly phenomenal the Ruby community is. I have served on the committee of Ruby NZ, organised a Rails Camp and attended a number of conferences around the world. I am always blown away by the inclusiveness of the Ruby community.

Ruby events (like Rails Camp) were some of the first to push for the adoption of codes of conduct at our events and online spaces. This behaviour has since become commonplace at lots of technology events, but it was the Ruby community's willingness to listen to and learn from it's members that led to these safe guards being commonplace.

I encourage you to engage with your local user groups and attend conferences, especially if you are a member of a minority or traditionally maginalised community.

Technically the Ruby community has been on a quite a journey also. We've seen many techniques come and go over the years. The current trend towards composition over inheritance and command/query object separation seems like the right direction to me, but I'm sure we'll all be doing something else in two years time.

Thanks

Thanks for taking the time to read this post, I know some of you were not particularly happy with the last one. I've outlined just a few of the many reasons why learning and practicing Ruby is still worth doing even if it's not as cool as Node or Haskell, as fast as Rust or as concurrent as Elixir. Go forth and punch a duck!

Building a stack-based virtual machine, part 6 - function calls

James Harton — Fri, 14 Sep 2018 00:03:29 +0000

In our last episode we finally ran a program. It was pretty exciting. This episode we're going to add function calling.

Up until now whenever we've used the term "stack" we've been referring to the operand stack, but typically when programmers talk about stacks they're talking about call stacks.

So what is a call stack and what is it used for? I'm glad you asked.

When you call a function you're executing a jump from one place in the program (instruction pointer) to another (hopefully named) place in the program. The difference between a function call and a jump is that you have to store the place you jumped from somewhere so that you can return back to it when you're done.

Arguments are passed by simply pushing them on to the stack before performing the call. The callee should pop the appropriate number of arguments off the stack and push the appropriate number of return values back onto the stack.

Yes, this means that functions can have multiple return values. How a machine takes arguments and return values is called it's calling convention. Here we've opted for the simplest possible convention to allow you, the compiler writer, to decide for yourself how you want function calling to behave.

So what is a call stack a stack of? We usually refer to them as stack frames so that's what we're going to build. At this stage our frame consists of a single value - our return address.

Frames will usually contain other environmental information, such as local variables or anything else that should be within this scope.

pub struct Frame {
    pub return_address: usize
}

impl Frame {
  pub fn new(return_address: usize) -> Frame {
    Frame { return_address: return_address }
  }
}

So how do we make a stack out of them? Turns out we already have a generic Stack<T> that we can use for the job. Let's add it to our machine:

pub struct Machine<'a, T> {
  code: Code<T>,
  instruction_table: &'a InstructionTable<T>,
  ip: usize,
  operand_stack: Stack<T>,
  call_stack: Stack<Frame>,
}

We need to update our new function to initialise the call stack. We create a frame which points past the last instruction in our program so that if the last frame is popped off the IP is pointing outside the program and the machine halts.

We also need to add the following methods to our machine:

jump - update the machine's instruction pointer to that of the named label.
call - create a new stack frame with the current IP as it's return address.
ret - pop the top frame off the stack and set the machine's IP back to the original location.

impl<'a, T> Machine<'a, T> {
    pub fn new(code: Code<T>, instruction_table: &'a InstructionTable<T>) -> Machine<'a, T> {
        let frame = Frame::new(code.code.len());
        let mut call_stack = Stack::new();
        call_stack.push(frame);

        Machine {
            code: code,
            instruction_table: instruction_table,
            ip: 0,
            call_stack: call_stack,
            operand_stack: Stack::new(),
        }
    }

    pub fn jump(&mut self, label: &str) {
        self.ip = self
            .code
            .get_label_ip(label)
            .expect(&format!("Attempted to jump to unknown label {}", label));
    }

    pub fn call(&mut self, label: &str) {
        self.call_stack.push(Frame::new(self.ip));
        self.jump(label);
    }

    pub fn ret(&mut self) {
        let frame = self.call_stack.pop();
        self.ip = frame.return_address;
    }

}

So now that we have our methods we can convert our earlier arithmetic program into one that has an add function. Here we use an enum as our operand type because now we need to store both labels (strings) and integers.

Before we code it up, here's our program as assembler so that it's easier to follow:

@0 = I(3)
@1 = I(4)
@2 = S("add_fun")

.main:
    push @0
    push @1
    call @2
    ret

.add_fun:
    add
    ret

So in order to understand what it's doing let's go through it step by step:

Push the integer 3 onto the stack.
Push the integer 4 onto the stack.
Call the "add_fun" function, which pushes a new stack frame and jumps to the add_fun label.
Add pops the two operands off the stack and pushes back the result of adding them together.
Ret pop the frame off the call stack and return by setting the IP back to 3.
Ret pop the last frame off the call stack and end the program.

#[derive(Clone, Debug, PartialEq)]
enum Operand {
    I(i64),
    S(String),
}

impl Operand {
    fn to_i(&self) -> Option<i64> {
        match self {
            &Operand::I(i) => Some(i),
            _ => None,
        }
    }

    fn to_s(&self) -> Option<&str> {
        match self {
            &Operand::S(ref s) => Some(s),
            _ => None,
        }
    }
}

fn push(machine: &mut Machine<Operand>, args: &[usize]) {
    let arg = machine.get_data(args[0]).clone();
    machine.operand_push(arg)
}

fn add(machine: &mut Machine<Operand>, _args: &[usize]) {
    let rhs = machine.operand_pop().to_i().unwrap();
    let lhs = machine.operand_pop().to_i().unwrap();
    machine.operand_push(Operand::I(lhs + rhs));
}

fn call(machine: &mut Machine<Operand>, args: &[usize]) {
    let label = machine.get_data(args[0]).clone();
    machine.call(label.to_s().unwrap());
}

fn ret(machine: &mut Machine<Operand>, _args: &[usize]) {
    machine.ret();
}

fn instruction_table() -> InstructionTable<Operand> {
    let mut it = InstructionTable::new();
    it.insert(Instruction::new(0, "push", 1, push));
    it.insert(Instruction::new(1, "add", 0, add));
    it.insert(Instruction::new(2, "call", 1, call));
    it.insert(Instruction::new(3, "ret", 0, ret));
    it
}

impl From<i64> for Operand {
    fn from(i: i64) -> Self {
        Operand::I(i)
    }
}

impl<'a> From<&'a str> for Operand {
    fn from(s: &'a str) -> Self {
        Operand::S(s.to_string())
    }
}

#[test]
fn example() {
    let it = instruction_table();
    let mut builder: Builder<Operand> = Builder::new(&it);
    builder.push("push", vec![Operand::from(3)]);
    builder.push("push", vec![Operand::from(4)]);
    builder.push("call", vec![Operand::from("add_fun")]);
    builder.push("ret", vec![]);
    builder.label("add_fun");
    builder.push("add", vec![]);
    builder.push("ret", vec![]);
    let mut machine: Machine<Operand> = Machine::new(Code::from(builder), &it);
    machine.run();
    let result = machine.operand_pop().to_i().unwrap();
    assert_eq!(result, 7);
}

Awesome. Now we have function calling. In the next episode I'll show you how to implement conditional instructions.

Please stop using Ruby

James Harton — Wed, 12 Sep 2018 07:11:02 +0000

I know what you're thinking but I want to start by saying that I love Ruby. Ruby is the language that made programming fun again for me after losing interest for a number of years. Ruby revitalised my career and allowed me to grow into the senior software engineer that I am today. Ruby is a lovely language. I just want you to stop using it.

For me Ruby came along in 2008 when I was working as a systems administrator at a local computer science department. I had written code in a bunch of languages before, and even enjoyed some of it but I was never able to feel completely productive or in tune with the language the way I felt when I started using Ruby. I was working with another team member to build a system to reimage lab computers using a combination of Rails, PXELINUX and BitTorrent. It was truly mad science. Also it worked really well - but that's a story for another day.

There was a bunch of reasons I stuck with Ruby after that first exposure and eventually transitioned into being a professional Rubyist. Below are the reasons I'm moving on and I think you should too.

Rails - the best tool for the job

If you wanted to make web applications in 2008 then Rails was simply the way to go. Sure there were other choices, but there were not other choices which were mainstream, had a reasonable talent pool available and were economic.

Rails drove the post-GFC startup economy. Not just because of the hipsters but also because it was the only framework at the time that allowed you to be so productive so quickly and to pivot your app easily when your hipster CEO decided to switch your dog food subscription website into an organic hemp running shoe e-commerce site.

Back then "the job" was a fairly constrained problem set: Take something from the database, squeeze it between brackets and serve it to the user. Then take some user input and stuff it back in the database.

These days the domain of a web app has expanded to include websockets, single page apps, APIs with their own query languages, web assembly and a bunch of tech that falls under the designation of "cloud". Web apps today look and behave completely differently to their 2008 counterparts - unless you look at a Rails app. Rails apps are still shuffling data back and forwards between databases and brackets - only these days there are more kinds of brackets.

Semantics - there can be only run

Ruby's DNA comes from a mixture of Perl and Smalltalk. Like Smalltalk Ruby features a dynamically typed classical object-oriented paradigm.

Ruby is a scripting language, meaning that the Ruby interpreter reads in source code, parses it and immediately starts evaluating it. There is no formal compile step. Runtime code defines (and redefines) constants, classes, methods and everything else. Runtime code can write more runtime code or modify or delete existing runtime code. In some cases the same class can have different behaviour depending on what lexical scope you're in. Ruby's semantics are so flexible that it was no wonder that Rails came along and told everyone their code had to fit into one of three folders.

When designing systems in Ruby you'll hear a lot about duck typing - the idea that you don't care what the type of an object is as long as it responds to the method you need. This is a way of having protocols without having to define them before hand, this makes sense because there is no "before hand", there's only runtime. In conventional cases it works quite well because most Ruby devs know that if you implement each then you can become an enumerable.

What this tends to mean in practice is that dynamic typing leads to subtle bugs because the wrong object is in the wrong place at the wrong time. The most common occurrence of this the infamous NoMethodError on NilClass. What this is saying is that you tried to call a method that doesn't exist on nil but what it really means is that somewhere below you in the stack a method returned nil instead of the thing you were expecting. It doesn't tell you where your bug is, just where the effects of it were finally intolerable to the system.

Because Ruby's semantics defy almost any kind of static analysis where a user of another language could add a type constraint or at least pattern match to avoid this situation the Ruby programmer is left with only one option: write a truly heroic number of unit tests in an effort to formalise what data is allowed to flow in and out of methods and objects and that it doesn't explode when the wrong input is given.

One other side effect of duck typing is the common use if backhanded type checking in the form of object && object.method to verify that the collaborating object is at least truthy before calling a method on it. This is so common that Ruby 2.3 added the &. safe navigation operator to turn this design problem into a virtue.

Performance - there can be only one

Let's get this out there first of all. Ruby is slow. Conventional wisdom is that Ruby is "fast enough" and that you can always throw more hardware at a problem. This is true to a certain extent, but most machines are multi-core these days and...

I don't consider myself as the threading guy, so I don't think I can make the right decision about the Actor library or the threading library.
Matz - creator of Ruby

...Ruby is also no good at concurrency. MRI has a global interpreter lock, which means that only one YARV instruction can be run at a time. There's a bunch of places where this can be worked around - like sleeping threads waiting for IO but fundamentally, if you want to do more than one thing at a time you need to run multiple interpreter processes.

Lastly, Ruby tends to use a lot of memory and it doesn't like giving it back to the operating system. There are ways to mitigate these problems to a certain extent, but most Rubyists are not aware of them or are simply used to the situation.

A lot of work has been put in by the Ruby core team over the last few years and I'll be the first to admit that Ruby 2.5 is much much faster than it used to be. Many would argue that it's fast enough. I would not.

Resilience - exceptional situations

As described above, even when running with multiple threads Ruby essentially runs a single interpreter with no built-in fault tolerance primitives so unhandled exceptions will take down the entire VM. There have been legendary efforts by the community at writing actor systems but ultimately those teams have given up and moved to languages who give them the tools they need to build reliability into their products (example Celluloid).

Tooling - from 0X to DX

The Ruby (and especially Rails) community lead the way in terms of good tooling to help developers achieve results quickly. Tools like Rake, Bundler, RSpec and the Rails command line really did blaze a trail for developer experience, however other languages have caught up and even overtaken Ruby in their tooling game. Rust's cargo, Elixir's mix and Scala's sbt all combine these tools into a single command and go the extra step of tightly integrating them directly into the language. And it's not just languages that have upped their tool game but also client side frameworks, DevOps systems and cloud providers.

Packages - to infinity and beyond

Ruby is blessed with a very large number of packages (or gems) which provide pretty much any kind of functionality you can think of. Not quite as many as that other language but I'd argue that on average they are of a much higher quality. Need an out-of-the-box authentication system? There's a gem for that. Need a massively overblown and architecturally questionable state machine that is weirdly tied to your database? There's a gem for that too.

Before Rubygems there really was only CPAN but no one needs to be reminded of that. Ruby's model of easily consumable and integrated packages paved the way and has been adopted by many many other languages, eg: crates, hex, CocoaPods and elm/packages.

So what should I use Ruby for in 2018?

For me, irb is still my go to tool to quickly work something out and I'll still write little Ruby scripts to orchestrate little one-off jobs like moving files around or sucking down an API responses.

When we talk about Ruby it's hard to ignore Rails, so I guess the question is "what should I use Rails for in 2018"? If you have an app that meets the following criteria then I expect that Rails will still be a good match for you:

is almost entirely database driven,
renders pages on the server or generates JSON,
can be mostly assembled out of existing gems,
doesn't have complicated business logic,
doesn't need to be fast,
doesn't need to serve many people at once,
won't ever change.

I'd still argue that you have better options though.

Building a stack-based virtual machine, part 5 - the machine

James Harton — Sun, 09 Sep 2018 21:39:32 +0000

In the last episode we built a code generator and a program representation so we now have all but that last bit we need to run our virtual machine.

Introducing the machine

So far we've built all the primary structures required to assemble our program, we have code, instructions and a stack. The last major thing we need to do before we can run our programs is the machine itself. Like Captain Planet we're going to assemble our constituent parts into a greater whole.

Here's our machine:

pub struct Machine<'a, T> {
  code: Code<T>,
  instruction_table: &'a InstructionTable<T>,
  ip: usize,
  operand_stack: Stack<T>
}

impl<'a, T> Machine<'a, T> {
  pub fn new(code: Code<T>, instruction_table: &'a InstructionTable<T>) -> Machine<'a, T> {
    Machine {
      code: code,
      instruction_table: instruction_table,
      ip: 0,
      operand_stack: Stack::new()
    }
  }
}

So when we create our machine we pass in our code and our instruction table. It sets the instruction pointer to zero (ie the first instruction in the code), creates an empty stack and returns our fancy new machine.

Before we can run any code we need to give it some handy helper methods for manipulating the operand stack and retrieving data from the data section. These are used by our instructions because if you remember they receive a mutable reference to the machine as their first argument.

impl<'a, T> Machine<'a, T> {

    pub fn operand_push(&mut self, value: T) {
        self.operand_stack.push(value);
    }

    pub fn operand_pop(&mut self) -> T {
        self.operand_stack.pop()
    }

    pub fn get_data(&self, idx: usize) -> &T {
        self.code
            .data
            .get(idx)
            .expect(&format!("Constant data is not present at index {}.", idx))
    }

}

No we can finally go ahead and add our run method to the machine. It's a reasonable chunk of code so let's break it down a bit first into some basic ideas:

Keep looping until the instruction pointer (IP) is the same as the length of the program (ie we've run out of program).
Read the opcode from our program at IP.
Read the arity from our program at IP + 1.
From 0 to arity read argument indexes from the program and push them into an arguments vector.
Call the instruction with the arguments.

impl<'a, T> Machine<'a, T> {

    pub fn run(&mut self) {
        loop {
            if self.ip == self.code.code.len() {
                break;
            }

            let op_code = self.next_code();
            let arity = self.next_code();

            let instr = self.instruction_table.by_op_code(op_code).expect(&format!(
                "Unable to find instruction with op code {}",
                op_code
            ));

            let mut args: Vec<usize> = vec![];

            for _i in 0..arity {
                args.push(self.next_code());
            }

            let fun = instr.fun;
            fun(self, args.as_slice());
        }
    }

    fn next_code(&mut self) -> usize {
        let code = self.code.code[self.ip];
        self.ip = self.ip + 1;
        code
    }

}

Now we have enough infrastructure in place to run our first program!

We're going to make a really simple arithmetic machine because I can do that without making this blog post too long. We need two instructions; Push which takes data and pushes it to the stack and Add which pops two operands off the stack, adds them together and pushes the result back onto the stack:

fn push(machine: &mut Machine<isize>, args: &[usize]) {
    let arg = *machine.get_data(args[0]);
    machine.operand_push(arg);
}

fn add(machine: &mut Machine<isize>, _args: &[usize]) {
    let rhs = machine.operand_pop();
    let lhs = machine.operand_pop();
    machine.operand_push(lhs + rhs);
}

fn instruction_table() -> InstructionTable<isize> {
    let mut it = InstructionTable::new();
    it.insert(Instruction::new(0, "push", 1, push));
    it.insert(Instruction::new(1, "add", 0, add));
}

Next we need our program, it's going to add 2 and 3 together:

fn build_program(it: &InstructionTable<isize>) -> Code<isize> {
    let mut builder = Builder::new(&it);
    builder.push("push", vec![2]);
    builder.push("push", vec![3]);
    builder.push("add", vec![]);
    Code::from(builder)
}

Now let's run it:

#[test]
fn addition_example() {
    let it = instruction_table();
    let code = build_program(&it);
    let mut machine = Machine::new(code, &it);
    machine.run();
    let result = machine.operand_pop();
    assert_eq!(result, 5);
}

And that's it. We've run our very first program in our virtual machine. Of course it's not very useful without function calls, so we'll add them in the next episode.

Building a stack-based virtual machine, part 4 - code

James Harton — Wed, 05 Sep 2018 20:55:57 +0000

Welcome back.

Now that we have our instructions and our stack, we need a program that we can run. I've called this Code.

So what is code?

Let's look at a really simple assembly program to get some ideas:

@0 = 123
@1 = 456

.main:
  push @0
  push @1
  add

This is not a real assembly program, but a simplified version I came up with for the stack-vm debugging output which is loosely based on the LLVM IR text format. It shows us some important things about a program. A program needs:

Data - here the constants 123 and 456 are assigned to the names @0 and @1.
Labels - in this case the label main points to the first instruction (instruction pointer = 0). In order to have functions or flow control we need to be able to label sections of code so that we can easily jump to them. We don't jump directly to instruction pointers because we'd probably make a mistake (remember GOTO?).
Instructions - the meat and veg of our program; here we use the instruction push twice with one argument each time and then we call the add instruction.

So what does this boil down to? Our Code struct needs to contain:

A "data section" - or a list of constant operands.
A "program section" - or a list of instruction op codes and indexes into the data section.
A mapping of labels to instruction pointers (or indexes into the program section).
A list of instruction names (symbols) stored by op code for easier debugging.

With all that in mind here's our Code type:

pub struct Code<T> {
    pub symbols: Vec<(usize, String)>,
    pub code:    Vec<usize>,
    pub data:    Vec<T>,
    pub labels:  Vec<(usize, String)>
}

We could write our software by just inserting values directly into these vectors but it's not very common to write assembly code by hand. It's much more common to generate code programmatically as an artifact of compiling a higher level language. In order to do that we're going to create our own builder (sometimes called a "code generator" or "assembler").

So what does our builder need to know in order to function?

An instruction table as outlined in part 3. This is so that we can have access to labels, opcodes and instruction arities.
Data, Labels and Instructions much like our Code type.

Why separate Code and Builder if they're mostly the same? I'm glad you asked.
We could conflate the two use cases together but the main reason they're separated is because the builder needs access to the instruction table for verification when inserting instructions, but code assumes that this process has already been done and can be used to serialising and deserialising to disk or for execution.

Here's our Builder:

pub struct Builder<'a, T: 'a> {
    pub instruction_table: &'a InstructionTable<T>,
    pub instructions:      Vec<usize>,
    pub labels:            WriteOnceTable<usize>,
    pub data:              Vec<T>,
}

impl<'a, T> Builder<'a, T> {
    pub fn new(instruction_table: &'a InstructionTable<T>) -> Builder<T> {
        let mut labels = WriteOnceTable::new();
        labels.insert("main", 0);
        Builder {
            instruction_table: &instruction_table,
            instructions:      vec![],
            labels:            labels,
            data:              vec![],
        }
    }
}

Don't worry about WriteOnceTable - it's just a wrapper around HashMap that only lets you set a value once.

So what behaviour does our builder need?

Add instructions and their arguments into the program.
Insert labels.

So let's cover off adding instructions first because it's by far the more complex part of the code.

impl<'a, T> Builder<'a, T> {

    pub fn push(&mut self, name: &str, args: Vec<T>) {
        let instr = self
            .instruction_table
            .by_name(name)
            .expect(&format!("Unable to find instruction with name {:?}", name));

        if args.len() != instr.arity {
            panic!("Instruction {} has arity of {}, but you provided {} arguments.", instr.name, instr.arity, args.len())
        }

        self.instructions.push(instr.op_code);
        self.instructions.push(instr.arity);
        for arg in args {
            let pos = self.push_data(arg);
            self.instructions.push(pos);
        }
    }

}

So given the name of the instruction and a collection of operands we take the following steps:

Look up the instruction in the instruction table.
Verify that the number of arguments we've been given and the instruction arity match.
Push the instruction's opcode into the program.
Push the instruction's arity into the program. This might seem like it's not necessary but is needed for the Code type to be independent of the InstructionTable.
Push each argument into the data section and it's index into the program.

Now on to labels:

impl<'a, T> Builder<'a, T> {

    pub fn label(&mut self, name: &str) {
        let idx = self.instructions.len();
        self.labels.insert(name, idx);
    }

}

This is pretty straight forward; we simply look up the number of instructions currently in the program and store the name pointing to it.

The last bit of the job is being able to convert a Builder into a Code. This is a relatively straight forward task:

Grab all the instruction names (symbols) from the instruction table.
Grab the program section.
Grab the data section.
Grab the labels and sort them by their IP.
Stuff them all in the Code object.

impl<'a, T> From<Builder<'a, T>> for Code<T> {
    fn from(builder: Builder<T>) -> Code<T> {
        let symbols = builder.instruction_table.symbols();
        let code    = builder.instructions;
        let data    = builder.data;
        let mut labels = vec![];
        for key in builder.labels.keys() {
            let idx = builder.labels.get(key).unwrap();
            labels.push((*idx, key.clone()));
        }
        labels.sort_by(|lhs, rhs| {
            lhs.0.cmp(&rhs.0)
        });

        Code {
            symbols: symbols,
            code:    code,
            data:    data,
            labels:  labels
        }
    }
}

Phew! That's a pretty dense episode for you to watch. I haven't included the tests, but believe me they're all there. In the next episode we'll wire everything up into a machine and actually run a program!