DEV Community: senyo

Embedded systems with Rust video course

senyo — Mon, 02 Sep 2024 09:12:47 +0000

Hello folks! This is just a post to say I've got a new course out with LinkedIn Learning on embedded systems programming with Rust.

It covers the basics of embedded systems and how to write Rust programs targeted for them. If you're interested in this field, check it out, I hope you find it valuable!

Course: Introduction to embedded systems with Rust

ecto's cast/4 function explained

senyo — Mon, 20 May 2024 17:00:00 +0000

The typical changeset in Ecto is created via the cast/4function.

cast(data, params, permitted, opts \\ [])

I've hardly found its usage intuitive.

def register_user(attrs) do
  %User{}
  |> User.registration_changeset(attrs)
  |> Repo.insert()
end

def registration_changeset(user, attrs, opts \\ []) do
  user
  |> cast(attrs, [:email, :password])
  |> validate_email(opts)
  |> validate_password(opts)
end

Why are we casting with an empty User? What do we actually use attrs for?

As it turns out, it's not even mildly confusing. The purpose of the cast/4 function is as the name says on the box: to cast data from one type into the other. It's necessary as we may receive data from external sources with the wrong type i.e an integer sent as a string from a form. Keep in mind though, we're still dealing with a changeset, whose sole purpose is to track changes to fields in our data.

Per the function signature, cast/4 casts the fields of params into the expected types specified by data, while tracking changes to the data. The only changes that are tracked are those specified by the atoms passed via permitted.

Take this Ecto schema

defmodule User do
  use Ecto.Schema
  import Ecto.Changeset

  schema "users" do
    field :age, :integer
  end
end

And say we do this

iex> changeset = Ecto.Changeset.cast(%User{}, %{age: "0"}, [:age])

The parameter age is a string. The cast/4 function will convert it into an integer. And since we've passed in an empty User, we'll get a changeset reflecting the changes we've made i.e adding a parameter age with the value of 0.

iex> changeset
#Ecto.Changeset<
  action: nil,
  changes: %{age: 0}, # type is cast to an integer
  errors: [],
  data: #User<>,
  valid?: true
>

We'll see this fail if our data cannot be converted into an integer with no changes applied.

iex> changeset = Ecto.Changeset.cast(%User{}, %{age: "a"}, [:age]) 
iex> changeset
#Ecto.Changeset<
  action: nil,
  changes: %{},
  errors: [age: {"is invalid", [type: :integer, validation: :cast]}],
  data: #User<>,
  valid?: false
>

If we already have a populated User type and we pass in valid data, we get our change tracking as expected and the data is cast if need be.

iex> changeset = Ecto.Changeset.cast(%User{age: 24}, %{age: "0"}, [:age]) 
iex> changeset
#Ecto.Changeset<
  action: nil,
  changes: %{age: 0},
  errors: [],
  data: #User<>,
  valid?: true
>

More writing

For more of my writing, check out my newsletter: off the wire

Earning the right to write software 'the right way'

senyo — Wed, 10 May 2023 19:59:50 +0000

Originally posted on my blog at https://senyosimpson.com/posts/earning-the-right

As technologists, we're often captivated by the technical details of software. It’s not long before we want to understand database internals, the intricacies of our operating system or the best paradigms for writing software. We’re arbitrarily motivated to do deep dives into areas that are filled with high degrees of complexity or to spend hours figuring out how to optimise the performance of some software. We’re invested in the technology itself; it is the end, not a means to it. Over time, I’ve come to realise that while this inclination has been central to my growth as a software engineer, in the context of my professional career, it comes with a hefty downside — being caught up in the technology even when its not valuable to the customer. If you’re obsessed with implementation details, you need a high level of pragmatism to be and remain successful at work.

The central goal of any company is ~~growth~~ to generate profit. Well, at least that’s the goal of any reasonable company. Generating profit means we get to care about everything else because without profit there is no company. To generate profit, we need to create value.

Software, in the context of a for-profit company, is a means to an end and that end is creating value. There’s a saying, “It's better to go slowly in
the right direction than go speeding off in the wrong direction”. If you exert energy in the wrong direction, it doesn’t matter how great of a job you’ve done in creating something, it’s value to your customers is marginal. That's the potential downside of obsessing over implementation details. All the time and energy that you put into producing the "perfect" code amounts to nothing if you're going in the wrong direction. In and of itself, great code is not valuable. It doesn’t matter if its performant and scales to handling millions of queries per second or that is has 100% test coverage and a beautiful integration test suite or that its written in a purely functional programming language or that it has best-in-class observability. To care about writing software and software systems "the right way™️", you have to earn the right.

To “earn the right” means to create a successful product. At that point, it is justifiable to care about the technical details¹. If millions of customers are using your product each day, you start running into many fun problems: optimising the performance of your software systems, rearchitecting systems to handle increasing load and improve developer velocity and improving developer productivity. At this stage, the company, the team and you have earned the right to care about the details².

At the beginning of this post, I mentioned that if you’re captivated by implementation details, you need a high level of pragmatism to be successful. The reason is summarised nicely in a quote we’ve all heard before, “premature optimisation is the root of all evil”. It’s so easy to get wrapped up
in improving the performance of your code or how well it’s encapsulates behaviour or any of the other 1000 things we all want to care about before we've even proven its useful to our customers. At this point, it's not subject to the type of constraints that necessitate the level of detail we all care about. And then, we have to ask ourselves, "is it really worth it?". It’s fine shipping it if it gets the job done. When you have enough customers, your systems will let you know they need some love. You’ve earned the right to care.

the best way to structure your repos and code or the programming paradigm are mostly a distraction. You'll undoubtedly run into challenges no matter which way you go, engineering is a bunch of trade-offs after all. And by the time you get to that point, you can just rewrite it anyway.

For the sake of nuance, this isn't as applicable to infrastructure products (e.g databases) where the novelty of the solution is the selling point of the software. ↩
Not all technical details are created equal. Even in the early stages, worrying about adequate test coverage, choosing good design patterns or how to write generally readable and maintainable code makes sense. Worrying about the scalability properties of your system, the performance of your software, ↩

Pointer Fu: An adventure in the Tokio code base

senyo — Mon, 25 Oct 2021 16:13:14 +0000

This post was originally posted on my blog

In an effort to understand the internals of asynchronous runtimes, I've been spending time reading Tokio's source code. I've still got a long way to go but it has been a great journey so far.

// Detect dark theme var iframe = document.getElementById('tweet-1450127351917027332-304'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1450127351917027332&theme=dark" }

Raw pointers are used all over Tokio. In one particular instance, the way they used them really blew my mind 🤯. So here I am, writing about it. Buckle in 💺.

Setting the scene

We have a type Cell that is defined as (taken directly from Tokio source)

#[repr(C)]
pub(super) struct Cell<T: Future, S> {
    /// Hot task state data
    pub(super) header: Header,

    /// Either the future or output, depending on the execution stage.
    pub(super) core: Core<T, S>,

    /// Cold data
    pub(super) trailer: Trailer,
}

It's used when initialising a struct RawTask

struct RawTask {
  ptr: NonNull<Header>
}

// This is pseudocode
impl RawTask {
    fn new() -> RawTask {
        let ptr = Cell::new(); // returns a raw pointer to a Cell

        // Cast pointer to one that points to a Header
        let header_ptr = ptr as *mut Header;
        let ptr = NonNull::new(header_ptr); 
        RawTask { ptr }
    }
}

What is so interesting?

First, a necessary detour! Rust can represent structs in memory in multiple ways. It's covered in detail in the The Rust Reference. By default, Rust offers no guarantee on the memory layout of your struct; it is free to modify the layout however it wants. From a developer perspective, it means you cannot write any code that makes assumptions on the memory layout of your struct. To change the representation of the struct, you can use the repr attribute (as shown in the above definition of Cell).

Now, here's where it gets good! I omitted the comment for the Cell struct, which is

/// The task cell. Contains the components of the task.
///
/// It is critical for `Header` to be the first field as the task structure will
/// be referenced by both *mut Cell and *mut Header.

Hopefully some alarm bells started ringing in your head 🚨. Header has to be the first field in the struct, so that you can dereference a pointer to Cell into either a Cell or a Header (if this doesn't make sense to you, check out the addendum). However, Rust's default representation provides no guarantee that Header will remain the first struct field! Instead, as you can see, they've changed the representation to the C layout. In C, struct fields are stored in the order they are declared. This gives us the guarantee we need. Now we can go around dereferencing the pointer to Header without any worry!

let cell = Cell::new();
let cell_ptr = &cell as *mut Cell;
let header_ptr = cell_ptr as *mut Header;

let header = unsafe { *header_ptr };

Addendum

View in Rust playground

#[derive(Debug, Clone, Copy)]
struct Header {
    x: u32
}

#[derive(Debug, Clone, Copy)]
struct Core {
    y: u8
}

#[derive(Debug, Clone, Copy)]
#[repr(C)]
struct Task {
    header: Header,
    core: Core
}

fn main() {
    let header = Header { x: 256 };
    let core = Core { y: 10 };
    let task = Task { header, core };

    // Create raw pointer to task
    let task_ptr = &task as *const Task;
    // Cast pointer to point to Header
    let header_ptr = task_ptr as *const Header;

    // Dereference header_ptr
    // We expect this to give us our Header struct as defined above.
    // Because Header is the first field and the pointer is pointing
    // to the beginning of the Task struct, it is valid to perform
    // this dereference
    let header_from_ptr_deref = unsafe { *header_ptr };

    // Dereference task_ptr
    // We expect this to give us our Task struct as defined above
    let task_from_ptr_deref = unsafe { *task_ptr };

    println!("Header from deref: {:#?}", header_from_ptr_deref);
    println!("Task from deref: {:#?}", task_from_ptr_deref);

    // Bonus: we can also cast task_ptr into a pointer to Core
    // However, because its struct member is a u8 and Header's member
    // is storing 256, which u8 cannot represent, dereferencing
    // core_ptr will produce a corrupted Core struct
    let core_ptr = task_ptr as *const Core;
    let core_from_ptr_deref = unsafe { *core_ptr };
    println!("Core from deref: {:#?}", core_from_ptr_deref);
}

The output of running this is shown below. Note that Core we obtained from dereferencing has y=0 instead of y=10.

Header from deref: Header {
    x: 256,
}

Task from deref: Task {
    header: Header {
        x: 256,
    },
    core: Core {
        y: 10,
    },
}

Core from deref: Core {
    y: 0,
}

Shoutouts to Ana for reviewing this post.

Designing state machines in Rust

senyo — Thu, 16 Sep 2021 14:50:15 +0000

This post was originally posted on my blog

In Rust, you often hear about state machines. Futures are state machines! I thought it would be cool to read more about it. I came across this blog post (funnily enough, by a friend and mentor of mine) which really helped me! I highly recommend reading it. In this post, I'm just noting the part I found relevant. The example here is from her blog post.

P.S.A: Go and read that post 😆

State Machines

Rust is able to represent state machines quite elegantly. Due to its type system, you can make sure invariants are upheld. To use a real life example, we can do some basic modeling of Raft (no knowledge needed for this post). A node in a Raft cluster can be in one of 3 states:

Candidate
Follower
Leader

The possible transitions are:

Follower -> Candidate
Candidate -> Leader
Candidate -> Follower
Leader -> Follower

Therefore the only invalid state transition is: Follower -> Leader.

Now we can start modeling this. We have a struct Node that represents a node in the cluster

struct Node<S> {
  ...
  state: S
}

S is a generic parameter over the possible states. Our states are also structs

struct Follower; // In reality, these are likely to hold some data
struct Candidate;
struct Leader;

In Raft, all nodes start in the Follower state. We can define a function new for that state only.

impl Node<Follower> {
  pub fn new() -> Node<Follower> {
    ..
    state: Follower {}
  }
}

We can initialise this like so

let follower = Node::<Follower>::new();

The important thing is that we can't do this for other states! If we do

let leader = Node::<Leader>::new();

we will get an error because we haven't defined the function new for Node<Leader>. This is super sweet!

The next thing we need to be able to do is have state transitions enforceable by the compiler. We can do this using the From trait. If we want to transition from Follower to Candidate we can implement From for it.

impl From<Node<Follower>> for Node<Candidate> {
  fn from(state: Node<Follower>) -> Node<Candidate> {
    let candidate = Node {
      ..
      state: Candidate {}
    }
    candidate
  }
}

We can use it as such

let follower = Node { state: Follower {}}
let candidate = Node::<Candidate>::from(follower);

Now we've gotten the type system on our side! If we don't implement From for Follower -> Leader, doing

let leader = Node::<Leader>::from(follower);

will cause a compilation error. Nice!

Thoughts

On using this design, one issue I came across was implementing general behaviours across all states. Since each impl block is dedicated to a particular state (e.g Node<Follower>, Node<Leader>), you have to duplicate any methods you use across them. We can get around this using our favourite feature: traits! We could have something like

struct Follower;
struct Candidate;
struct Leader;

trait NodeState {}
impl NodeState for Follower {}
impl NodeState for Candidate {}
impl NodeState for Leader {}

impl <S: NodeState> Node<S> {
    // Generic methods
}

We've added the trait bound NodeState. All states S that implement NodeState will have the functionality in the impl block.

Expectations of senior+ engineers from a not so senior engineer

senyo — Fri, 03 Sep 2021 14:29:25 +0000

Senior(+) engineer. One of the most coveted titles in our field. There are endless articles on the expectations of these engineers. Naturally, these are written by other senior+ engineers. On realising this, I had a thought

What would the expectations of a senior+ engineer look like from a non-senior+ engineer?

Fitting the bill of that description, I decided, I should write about it! Turns out, they're not so different from what is already in the public domain. However, I thought I would motivate them from my perspective. I can't speak for everyone but I think the themes will be largely applicable. I hope you
find it valuable.

Know it all

I'm only trolling you 😆 There is no expectation on you to know everything. All of us are on a learning journey. If you don't know the answer to something, that's perfectly fine. Pointing me in the right direction is often good enough. The best part? I have an opportunity to teach you something. We all grow together.

An uncanny sixth sense

At the core of being a senior engineer is having good intuition. Developed through a mix of experience and the ability to critically evaluate past successes and mistakes, it's an important piece of what separates seniors from the rest. Remember that time you avoided a sub-optimal design decision because you started seeing all kinds of red flags? That's your intuition saving the day. You're able to evaluate trade-offs, have a good feel for why certain designs will/won't work given the context you're operating in, base your decisions off decent assumptions in the face of incomplete information and so on. These all help in aiding your decision making process and guide yourself and your team to make more informed decisions.

This is one trait I appreciate immensely. Anyone who knows me knows I love asking technical questions: Why do our APIs follow this specific design, why do we use microservices, why don't monorepos suit us, when should we trade off quality for speed and so on. These kind of technical questions have no definitive answer. As I've said before

// Detect dark theme var iframe = document.getElementById('tweet-1363729866194116608-485'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1363729866194116608&theme=dark" }

They are all contextual. However, as someone who hasn't seen enough of this play out through my own experiences, it is great to be able to ask and get back well justified answers. I've found this to be one of the most valuable ways of learning for me. It helps me bootstrap my own intuition and opinions.

Deep thought enables great work

Keeping the tweets flowing, we have another one by yours truly

// Detect dark theme var iframe = document.getElementById('tweet-1336008654693281792-324'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1336008654693281792&theme=dark" }

I hold the belief that the art of writing code is more a function of design than anything else. Once you have a design, e.g an API interface, a suitable distributed algorithm, etc, the act of actually implementing it in code is often rather straightforward. Even for complex pieces of software this applies. We've gotten better at writing concurrent software because we've designed better concurrent primitives, as an example. This is to all say, at the heart of great engineering is thoughtfulness. Engineers (and teams) that spend significant portions of their time thinking through their solutions tend to produce higher quality work. This makes sense - barely ever is the first idea we have the best one. Being able to take a step back, conjure up and evaluate alternative solutions and then pick the best one given the constraints and context is such a valuable skill. All too often, we find ourselves mashing our keyboards without even thinking about the soundness of our API interfaces.

I find this trait valuable for the same reason I find the first valuable: I get to ask questions and learn from your thinking. It's always a great learning experience when something is implemented differently to the way I would have imagined it. Being able to talk through your thinking and getting justifications/opinions that challenge my own help me develop my thinking. In fact, one of the most important lessons I've learned is that there doesn't always have to be a strong justification for making a certain decision. Sometimes, "it's easy and it works" is really enough.

A human message broker

Much ink has been spilled on why it is necessary for engineers to learn to communicate effectively. At staff+ levels, this is likely to be one of the biggest expectations outside of technical skills. As technical people, this is often a really difficult skill to pick up and improve. Computers are simple. Humans, not so much. However, it cannot be underestimated how valuable a skill it is. Unfortunately, it's not as simple as putting words on paper or speaking about the matter at hand. This is what I believed initially. Over time, I've realised there's more to it: being able to build consensus on ideas/topics, being clear on outcomes and what they mean (we underestimate how often people have different ideas of the same thing), being clear on what success looks like, being able to storytell, being able to articulate the reasoning behind certain decisions and so on. These are all facets of being able to communicate effectively.

This is particularly relevant for me because I've been trying to learn how to communicate better within a technical context. I've found this to be rather challenging, however, I have grown considerably by watching how seniors (and even execs) communicate their ideas and concerns. To give a concrete example, one mistake I used to make routinely is vouching for a certain practice, for example, using type annotations in Python, in the name of "best practice". Best in class open source projects are using type annotations, we should too! This is poor communication. It doesn't even answer the first question you should ask yourself in this situation

What problem does using type annotations solve for us?

It's easy to have good ideas. It is significantly more difficult to justify them. Being able to communicate effectively is a necessary piece of being a great engineer.

May the force be with you

Part of seniority is enabling others to do their best work. People who do this are force multipliers. This is especially relevant at staff+ but even at senior. There are countless ways of doing force multiplying work: mentoring juniors/mids, bridging communication with the business and technical teams, delegating work, leading teams/projects, amplifying others work, helping team members get promoted, driving initiatives across teams, being involved in hiring. These have a large impact on your team and the organisation at large. It is often this work that moves people up into more senior positions, especially when it has organisation wide impact.

As an aside, check out the StaffEng podcast. It is one of the best places to gain insight into real-life stories of the journey to staff+ engineer

From a personal perspective, it's great having force multipliers in the team. The best part for me is actually just witnessing it. It is amazing to see senior+ engineers doing great work, caring about the org and making a positive impact. It's simply just inspiring.

Shoutouts to Ana and Wasim for reviewing this

Ergonomic error handling with Rust

senyo — Wed, 31 Mar 2021 09:02:10 +0000

Prior to learning Rust, I never knew error handling was actually a big deal. It never struck me that there are alternative ways to handling errors, that error messages should be as informative as possible, or even that you should take time to write robust error handling logic in your code. It's obvious in retrospect, but in Python-land, there isn't as big a chorus on error handling.

Rust's story is still developing and as a result there are conversations about every aspect of the language. With a strong emphasis on making developing in Rust enjoyable, there is plenty of healthy debate going on. Error handling is an ongoing and important discussion in the community. The language attempts to make error handling ergonomic and I think they've done a great job of it so far. As I started to read more about it, I realised there was so much I had never considered. Writing good errors and having robust
error handling logic are core features of writing good software. Alongside this, they contribute immensely to developer experience. The central theme threaded throughout this post is this:

Error handling is about communication

We strive to write robust code with good error handling logic and informative errors so that we can accurately describe what happened and how we resolved it. Without further ado, let's jump into the world of error handling.

Why you should learn about distributed systems

In distributed systems, if there is anything you can be certain of, it is that your system will fail. Failure is normal. These systems must be built to be fault-tolerant. In learning more about them, one idea really stuck with me and shaped my thinking: build for failure. This thinking
is applicable in most aspects of software development. How many times have you used software and it returns the world's most unhelpful error messages? It once took me about a full working day to debug an error that was fixed by using a jpg image instead of a png. The error message had absolutely nothing to do with the image format and so I wasted countless hours researching and debugging. The application was not built to handle (reasonable) failures.

Error handling is such an important part of development precisely because it allows us to handle failure. It improves our ability to reason about our software; we understand the myriad ways it may fail. When we encounter failures, great error messages help us diagnose problems efficiently. This in turn boosts productivity and leads to an improved developer experience.

Shifting paradigms

Languages like Rust and Go have returned to ways of ancient past by returning errors instead of using exceptions. Being accustomed to exceptions, I did not understand the benefits of returning errors. Through researching it, I came across justifications for its use:

Returning errors is explicit. In Python, any function could throw an exception. Short of reading the source code or the documentation, you're left for dead. If errors are returned, e.g obj, err = func(), they are pushed onto the developer explicitly. She is at least aware that the function may result in an error. It is still left to her to decide to handle or ignore it but she does so consciously. Without the explicitness, it is too easy to miss areas of code that should have exception handling, purely because you do not even know exceptions are raised. In fairness, statically typed languages with exceptions can solve this problem. For example, Java function signatures declare the exceptions they throw and the compiler enforces exceptions. Fortunately for all Java developers, there is a loophole out of that requirement; one that is used all too often.
Exceptions are often misused. They should only be used in exception-al cases. This is one of those dogmatic phrases in tech but there is some merit behind it. Exceptions do not need to be raised in situations where errors are expected. For example, if we need to read a file and know that there is a possibility that the file may not exist, an exception should not be raised. We expected this and should write our code to handle that situation or propagate that information back to the caller. It's not an exceptional situation. However, if we need to read a file and expect the file to be there and it is not, an exception can be raised. It's a situation we did not expect and possibly one we do not know how to handle. This was the original thinking behind exceptions - in most circumstances, we have logic to handle known, expected failure cases but when we run into truly exceptional situations, we raise exceptions. In reality, exceptions are used freely. One horror story I came across was from a developer who complained about an HTTP client library he had tried using. It threw exceptions for every single non 2xx and 3xx response. That's a great example of misusing exceptions.
Pokemon exception handling. In Java and Python, it's possible to catch all exceptions by using syntax like catch Exception since all exceptions subclass Exception. This makes debugging failure modes extremely difficult. If your code starts producing weird outputs, it's impossible to tell because errors are caught silently by the catch all statement. In reality, this is the fault of the developer but developers are lazy and will find loopholes if the system allows.

After reading that list, you'd be inclined to think that returning errors is hands down the better option. Returning errors has a critical flaw: it is easy to ignore them. For example in Go, you can write value, _ := func(). In the event that func errors, value will be left in an invalid state but execution will continue uninhibited. With exceptions, a key benefit is that program will crash immediately. Returning errors requires significant discipline from developers, who as we mentioned before, are lazy.

Rust takes returning errors to the next logical level by enforcing errors are handled. It takes inspiration from the ML family of languages. These languages have a strong type system, embedding the notion of a fallible computation into the type system in a unique way.

How Rust does error handling

The Result type

In the ML family of languages, one of the common constructs is the Result type. A Result is an enum that has two possible states: the value of the computation or an error. To get the value, you have to unwrap Result. The compiler enforces that Result is unwrapped before it is used anywhere else. For example, if we have a function do_something(s: String) and our input variable is currently a Result<String>, we have to unwrap it to get the String in order to pass it into the function. In the event Result contains an error, we have to handle it. This enforces the handling of errors. There is no way to circumvent it due to it being embedded in the type system. This is why error handling in Rust is great. Let's take a look at basic error handling.

// This is the result type in the standard library. It either contains
// the value of the computation Ok(T) where T is the value or an error
// Err(E) where E is the error
enum Result<T, E> {
   Ok(T),
   Err(E),
}

let mut f = File::create("ferris.txt");  // Returns Result<File>

// To unwrap we match on the enum variants
let mut file = match f {
  Ok(f) => f,
  Err(e) => panic!("File could not be created")
};

match file.write_all("Hi Ferris") {
  Ok(_) => {},
  Err(e) => panic!("Could not write to file")
}

Rust has several convenience methods/syntax such as unwrap, expect, ? which help to remove some of the boilerplate. We will discuss these later.

Panicking

In Rust, irrecoverable errors are signaled using the panic! macro. When a panic is invoked, the developer is essentially saying, "program execution cannot continue any further after encountering this error". Panicking is a terminal state; the program crashes as a result. It is normally used when
a bug is encountered. They are meant to be infrequent, with standard error handling taking care of most cases. We've already seen panic being used in Rust in the above example.

match file.write_all("Hi Ferris") {
  Ok(_) => {},
  Err(e) => panic!("Could not write to file") // If this branch executes, program crashes
}

Bubbling errors

In many cases, we do not want an error to be handled at the location it is generated. Instead, we would prefer to have that error be handled by the caller of the function, giving it the power to decide how to proceed. In programming parlance, we want to "bubble" the error to the caller. In Rust, we can achieve this using matching.

// () is the unit type and is used when there is no meaningful
// value to return
fn init() -> Result<(), io::Error> {
  let mut file = match File::create("ferris.txt") {
    Ok(f) => f,
    Err(e) => return Err(e) // returns an error to the caller
  };

  match file.write_all("Hi Ferris") {
    Ok(_) => return Ok(()),
    Err(e) => return Err(e)
  }
}

// The executable code
// Caller of function gets error and then panics on failure
match init() {
  Ok(_) => {},
  Err(_) => panic!("Could not initialise")
}

Making it ergonomic

As I mentioned earlier, Rust has a number of convenience methods/syntax to reduce boilerplate. There are three that are commonly used: unwrap, expect and ?.

Try

When bubbling errors, you can imagine that writing the above match statement becomes cumbersome. It's boilerplate that the language can handle for you. A similar problem exists in Go. If you ask Gophers what line of code they write the most, they'll answer: if err != nil {}. The Rust language designers took care of matching boilerplate by introducing the ? syntax. This replaced the try! macro as a more convenient syntax. Its purpose is to automatically bubble the error to the caller on an error occuring.

 // Before we would write code like this
fn init() -> Result<(), io::Error> {
  let mut file = match File::create("ferris.txt") {
    Ok(f) => f,
    Err(e) => return Err(e)
  };

  match file.write_all("Hi Ferris") {
    Ok(_) => Ok(()),
    Err(e) => return Err(e)
  };
}

// With the `?` operator, we can write it as
fn init() -> Result<(), io::Error> {
  let mut file = File::create("ferris.txt")?;
  file.write_all("Hi Ferris")?;
}

As you can see from the above example, our code is significantly less verbose while being functionally identical. This is one of those small quality of life improvements that makes a world of difference when writing Rust. It highlights the commitment to a friendly developer experience.

unwrap

There are often scenarios where you want to opt out of error handling. This may be when you're prototyping and don't want to go through the effort of setting up robust error handling or when you know that a function won't fail (e.g if you need to read a file that you know will always exist). To get out of it, you can use the unwrap() method. unwrap returns the Ok variant with it's value if the computation succeeds or will panic on error.


// Notice how we don't need the return type anymore
fn init() {
  // We use `unwrap()` to panic if it fails otherwise execution continues
  let mut file = File::create("ferris.txt").unwrap();
  file.write_all("Hi Ferris").unwrap();
}

If we try create a file in a location we do not have permission to access, the code will panic, causing the program to crash. An example of a panic message

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os {
code: 13, kind: PermissionDenied, message: "Permission denied" }',
src/main.rs:4:36

Notice how Rust does not give us a get out of jail free card with error handling. Either we write error handling logic or we accept our code will panic and subsequently crash (most likely unknowingly to the developer). Extensive unwraping is not recommended for production code. There are cases where it is permissible (as with everything), for example if we can guarantee it won't fail or we want execution to panic at that location. One issue with unwrap is that error messages can be uninformative. We can do one better by using expect.

expect

The expect() method is identical to unwrap() but it allows you to set an error message. This conveys your intent and makes debugging easier.


fn init() {
  // Write to a location we do not have permission
  let mut file = match File::create("/var/ferris.txt")
    .expect("Could not create file `ferris.txt`");

  file.write_all("Hi Ferris").expect("Could not write to file");
}

This generates the message

thread 'main' panicked at 'Could not create file `ferris.txt`: Os {
code: 13, kind: PermissionDenied, message: "Permission denied" }',
src/main.rs:4:36

As we can see, Rust has a strong focus on making error handling a frictionless experience for the developer. There are other functions that I have not gone through here: map_err, map_or_else, unwrap_or, unwrap_or_else and many more. If you're new to Rust, I encourage you to give them a look. I certainly will as I improve the quality of my error handling.

Making it informative

Writing good errors means making them highly informative. Without good error messages, debugging is significantly more difficult. To reiterate, error handling and error reporting is about communication. By providing highly informative errors, we paint the full picture of what has happened. We want to guide the developer directly to the problem. Rust has made good error reporting a core emphasis of the language and the larger ecosystem. Arguably, unintentionally.

// Detect dark theme var iframe = document.getElementById('tweet-1303107711928643584-24'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1303107711928643584&theme=dark" }

The Rust compiler really is a gold standard in error reporting. There's still work to do but it is one of the most human friendly pieces of software I have used. As an example

I quite like how this tweet puts it :)

// Detect dark theme var iframe = document.getElementById('tweet-1348669062528774148-95'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1348669062528774148&theme=dark" }

Focusing on good error messages has permeated throughout the community. There's even the Error Handling Project Group if you weren't convinced how committed the language designers are to getting this right. There are a number of techniques we can use to make our errors more informative. Along the way, we will discuss the crates that can help.

What are we writing?

The first question we have to ask is

Are we writing an application or a library?

I had never considered that your approach to structuring error handling is different depending on whether you are writing an application or library. This article illuminated this difference to me. I encourage you to go read it. I will mention the points I found particularly insightful with some insight of my own.

As far as possible, libraries should use meaningful custom error types. This has several benefits:
- Applications can easily differentiate between various errors. A simple example are the various IO errors defined in the Rust standard library.
- Errors can be wrapped into a custom error defined by the library. Without this, it would be impossible to differentiate between errors from different libraries. An IO error in Foo would be indistinguishable from an IO error in Bar
- It increases the cardinality of errors at the application level. High cardinality data allows us to answer very specific questions, giving us the insights we need. This helps when you need to understand error rates across your system, find areas that need some extra maintenance, debug production issues and so on.
When defining custom errors, they should always implemented the standard library Error trait, std::error::Error. This is to mitigate compatibility issues with errors from other libraries.
Libraries should never panic. From the application programmer's point of view, panics are undefined behaviour; there is no expectation that a library call will crash an application. Errors should be bubbled up to the caller.
Applications consume errors and make informed decisions on how to handle them. There is often not a huge need for custom errors at the application level. Additionally, at this level, panicking may be the best option and is therefore reasonable.
Applications are responsible for deciding how errors are formatted and displayed to users.

As we can see, there are subtly different requirements depending on which one you are writing. Rust being Rust has crates for both these use cases. The most popular of these are anyhow and thiserror. anyhow is used for error reporting in applications while thiserror is used for creating custom errors for libraries (and applications). I will go through their use cases in context of important aspects of error handling.

It depends

A while ago, I tweeted this

// Detect dark theme var iframe = document.getElementById('tweet-1363729866194116608-877'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1363729866194116608&theme=dark" }

As (not) funny as it may be, the truth is that most technical decisions (and arguably most life decisions) come down to context. If you operate outside of context, you're likely to make a whole host of sub-optimal decisions; not because they are inherently incorrect, they're just not applicable to the situation. Similarly, if error messages are missing the context in which they are generated, they're likely to send you down a spiral of debugging. Most of that time might unfortunately be spent exploring dead ends. For errors to be informative, they need to include additional context. Taking a simple example, imagine you are trying to open a file and it does not exist. If we printed the error, we would get

No such file or directory (os error 2)

This is barely useful. We know that we have an IO error but no idea what generated it. We also do not know which file or directory is the culprit. Now imagine our error message was like this

Error: Failed to read file /path/to/directory/ferris.txt

Caused by:
    No such file or directory (os error 2)

Much better! We know which file we are missing and where it's located. If we let our imagination run unbounded, we can think of additional context we could add - like a stack backtrace.

Error: Failed to read file /path/to/directory/ferris.txt

Caused by:
    No such file or directory (os error 2)

Stack backtrace:
   0: std::backtrace_rs::backtrace::libunwind::trace
             at /rustc/f5f33ec0e0455eefa72fc5567eb1280a4d5ee206/library/std/src/../../...
      std::backtrace_rs::backtrace::trace_unsynchronized
   ...
   ...

As you can see, adding additional context makes our errors so much more informative. This aids us whenever we need to debug our code.

As I mentioned earlier, I said I'd talk about the relevant crates as we go over various topics. anyhow allows us to add additional context to our error messages. The above two error reports are produced by anyhow.

use std::fs;
use anyhow::{self, Context};

fn main() -> anyhow::Result<()>{
    let path = "ferris.txt";
    let content = std::fs::read(path)
        .with_context(|| format!("Failed to read file {}", path))?;

    Ok(())
}

Be like Mansory

Mansory is a luxury car modification company. Their cars are something to behold and I'm not even that into cars. In the same way they make crazy custom cars, we should strive to make crazy custom errors. Okay, maybe not too crazy. Libraries should have their own set of custom errors that are meaningful given the domain they operate within. In other cases, they should wrap standard errors. As mentioned, this ensures we can differentiate between similar classes of errors between libraries. You can also take the approach tokio took and re-export types so they are accessible through your library but still differentiated from another library. Once again, when you strip it down to its core essence, the point of all of this is communication. We want our libraries to faithfully communicate to the developer what type of error was encountered when things go wrong.

Rust requires a bit of ceremony to define custom types. It is worth digging into before using a library to do the heavy lifting for you. thiserror is a library used for creating custom errors. I've found it enjoyable to use, albeit I have not had to do anything advanced. Imagine we have a library beatmaker that generates music using midi. The errors we care about:

The notes are invalid music notes i.e they are not between A & G
The format of the file containing the notes is invalid
The instrument is invalid - it's not in the set of instruments we have
IO errors e.g the file is missing. For IO errors, we want to wrap the IO error into one of our own errors.

use thiserror::Error

#[derive(Error, Debug)]
enum BeatMakerError{
    #[error("The music notes are not all valid. Please ensure they are between A & G")]
    InvalidNotes,
    #[error("The format of your .bm file is invalid. Check the guide to learn how to create .bm files")]
    InvalidFormat,
    #[error("The name of your instrument is invalid. Please check the instrument list for all valid instruments")]
    InvalidInstrumentName,
    // This wraps all IO errors produced by the std lib into our defined IOError
    #[error(transparent)]
    IOError(#[from] std::io::Error), 
}

We can use it in an application like so

fn raise() -> Result<(), BeatMakerError> {
    return Err(BeatMakerError::InvalidNotes)
}

fn main() -> Result<(), BeatMakerError> {
    match raise() {
        Ok(_) => println!("All good!"),
        Err(e) => println!("I'd write better errors but I'm lazy. The error: {}", e)
    }

    Ok(())
}

Since we've wrapped the standard library's IO errors, we can write a fallible function that returns an IO error and it will work seamlessly with BeatMakerError

use std::fs::File;

fn io_raise() -> Result<(), BeatMakerError> {
    File::open("ferris.txt")?;
    Ok(())
}

fn main() -> Result<(), BeatMakerError> {
    match io_raise() {
        Ok(_) => println!("All good!"),
        Err(e) => println!("I'd write better errors but I'm lazy. The error: {}", e)
    }

    Ok(())
}

On error, this gives us the error message

I'd write better errors but I'm lazy. The error: No such file or directory (os error 2)

The future of errors

One aspect of Rust's community I really enjoy is its grandiose imagination. In particular, Rust's language designers have a seemingly unbounded imagination. They dream of designing a language that is overwhelmingly pleasant to use. In many ways, they are getting this right - adoption for the language is growing even in the face of complexity such as the borrow checker. Error handling is no different. In reading more about error handling, I came across Jane Lusby's RustConf 2020 talk, Error handling Isn't All About Errors. She wrote eyre, a fork of anyhow that adds support for customised error reports. I encourage you to watch the video and check out the library. The core focus of eyre is to further improve error messages/reports. It's imagining even more informative errors in Rust.

Error:
   0: Unable to read config
   1: cmd exited with non-zero status code

Location:
   src/main.rs:50

Stderr:
   cat: fake_file: No such file or directory

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 10 frames hidden ⋮
  11: <std::process::Command as solo::Output>::output2::h8837a51b9856a548
      at /Users/senyosimpson/Projects/test/solo/src/main.rs:50
  12: solo::read_file::h21ce0bfb8b0da736
      at /Users/senyosimpson/Projects/test/solo/src/main.rs:88
  13: solo::read_config::h048c15951f2a6b11
      at /Users/senyosimpson/Projects/test/solo/src/main.rs:93
  14: solo::main::hec1c621ee896fc4a
      at /Users/senyosimpson/Projects/test/solo/src/main.rs:65
  15: core::ops::function::FnOnce::call_once::h94986c4cb4784c0a
      at /Users/senyosimpson/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/..
                                ⋮ 9 frames hidden ⋮

Suggestion: try using a file that exists next time

The above error report is an example of this. There are some nice novelties. The error contains both the error messages and the order in which they occurred in a nice format. There is also a suggestion which is something we do not see often in standard error reports. The other information we are used to seeing but it is nicely formatted. It really contains all the information you need in a nice and succinct format. There's even possibilities of extending this further. If we have a chain of errors, we can generate a report like this

Error:
   0: encountered multiple errors

Location:
   src/main.rs:47

Error:
   0: The task could not be completed
   1: The task you ran encountered an error

Error:
   0: The machine is unreachable
   1: The machine you're connecting to is actively on fire

Error:
   0: The file could not be parsed
   1: The file you're parsing is literally written in c++ instead of rust, what the hell

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 10 frames hidden ⋮
  11: solo::join_errors::hd118537501e8ddf7
      at /Users/senyosimpson/Projects/test/solo/src/main.rs:47
  12: solo::main::hec1c621ee896fc4a
      at /Users/senyosimpson/Projects/test/solo/src/main.rs:35
  13: core::ops::function::FnOnce::call_once::h94986c4cb4784c0a
      at /Users/senyosimpson/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/..
                                ⋮ 9 frames hidden ⋮

This is so cool! We get all the information we need with the full chain of errors. The error messages are also stellar, nice touch Jane :)

We have the capacity to imagine a future with the most enjoyable and ergonomic error handling and informative, easily readable error reports. It's great to see there is active work on this front. Hopefully, Rust will continue to make our lives all a bit better, one error message at a
time.

Shoutouts to Ana for reviewing this post 🐻

The basics of gRPC

senyo — Sat, 19 Sep 2020 17:04:18 +0000

If you are similar to me, you've heard about gRPC countless times but have little idea of what it actually is, what its use cases are and why everyone keeps speaking about it. Initially, I thought it must be another buzzword being thrown around. Fortunately, I was wrong 😆. I've spent a couple of days reading and learning more about it. This blog post is a summary of all I've learned. I hope you enjoy it!

What is Remote procedure call?

Remote procedure call (RPC) is a communication protocol used between web services. The central idea around RPC is to allow a client application to execute a procedure (function) on a server application as if the procedure is running locally. The networking details are abstracted away from the developer making it simple to use.

To concretize it, a bird's eye view of a typical RPC flow:

A client calls a function
The function is actually a remote procedure call. A request is made to a server to execute the procedure
The request contains the parameters to be passed into the procedure
The procedure is executed on the server
A response is returned to the client containing the result of the procedure execution

From the client's perspective, the procedure seems as if it is running locally. This is a great abstraction, however there are pitfalls. Martin Kleppmann, in his book Designing Data-Intensive Applications, highlights some of the differences between remote and local procedure calls.

A local function call is predictable and its success depends on factors under your control. In contrast, a networking request is unpredictable and fails for various reasons.
A local function returns a result, throws an exception or never returns. A networking request, in addition, can return without a result due to a timeout. In this scenario, there is no way to know what occured.
A network request may be received and execute but only the network response fails. Network request failures are often handled by retries but unfortunately in this case, the function would be executed multiple times as you would not know only the responses are getting lost.
A local function takes a similar amount of time to execute every time it is invoked. Network requests have variable latency due to networking specific issues - network is congested, server is at maximum capacity etc.

What is gRPC?

The gRPC website describes it as a, "modern, open source remote procedure call (RPC) framework that can run anywhere. It enables client and server applications to communicate transparently, and makes it easier to build connected systems". It was created by Google and subsequently open-sourced in 2016. gRPC has driven the resurgence of communications using RPC. With the move towards service-oriented architectures, it has found a natural home in service-to-service communications.

gRPC uses protocol buffers (protobufs) - a method for serializing/deserializing structured data. Protocol buffer messages are encoded in a binary format. This makes it fast to send information over the wire; at least in comparison to textual formats such as JSON. The use of protocol buffers alongside the great and easy to use tooling around gRPC has made it the preferred framework for implementing RPC.

As with any technology, there are advantages and disadvantages. This article by Microsoft highlights some of the them. In summary

gRPC Advantages

Performance - gRPC is fast in comparison to other protocols
Code generation - server and client code is automatically generated from a single file
Streaming - gRPC supports HTTP streaming

gRPC Disadvantages

Limited browser support - there is limited support for HTTP/2 in the browser
Not human readable - gRPC uses a binary format and therefore is not human readable.

Protocol Buffers

As stated above, protocol buffers are a method for serializing/deserializing structured data. Protocol buffer messages are defined using a schema consisting of key-value pairs.



message Person {
  string name = 1;
  int32 id = 2;
  string email = 3;
}

You can think of Person as a struct. Each field is a key-value pair and is annotated with their respective data type. Each field has a number associated to it which is used as the key. In actual fact, the key contains the field number and information about the type of data being encoded in order to determine the length of the value. This is commonly referred to as the tag. As an example, if we have a protobuf message



message Test2 {
  string b = 2;
}

with the value of b set to testing, the corresponding encoding will be



12 07 74 65 73 74 69 6e 67

The bytes 12 and 07 form the tag. The value 12 is decoded to give the field number 2 and the data type string. The value 07 is decoded to give the length of the value which is 7 in this case. As you can see, there are 7 bytes remaining which give the value of the string. If you are interested, you can find more information about the protocol buffer encoding from the official guide. This example is actually from there albeit with much less detail.

What happened to REST?

REST is the canonical standard for communications over the web. It has been battle-tested in production, extensive tooling exists for implementing RESTful services and most developers are comfortable with designing, building and maintaining RESTful services. Naturally the big question is what is so great about gRPC that we would forego creating a RESTful service? The answer is fairly straightforward: performance. While there are other benefits of using gRPC such as the use of HTTP/2 or bidirectonal streaming, for the standard use case, performance is the central reason for adopting it. JSON is much slower to serialize/deserialize than protobufs. At scale, this can lead to a noticeable degradation in performance of the overall system. As protobufs are a binary format, they are much faster to serialize/deserialize. some articles claiming there is an improvement of 5-6 times.

gRPC by example

Nothing is ever complete without some code examples! I've tried to make this as easy to follow as possible. Let me know if any improvements can be made. All the code can be found here. We will implement the service in the Quick Start section of the gRPC website. I'm using Golang for this tutorial.

One snag that got me was the automatic code generation from the proto files. There is a move towards a new Golang plugin for the protobuf compiler (it performs the automatic code generation). The old (which I used) and new version output slightly different files and so I couldn't follow the code in the official grpc examples repository. To make things simple, this tutorial uses the old Golang plugin. Awesome, let's get into it 🥳

The service is really simple, all it does is greet a person. The client can send a request containing a name to the server. The server produces a simple message, "Hello <name>". To recap, the server is responsible for executing the procedure and returning the result to the client. From the client's perspective, the procedure ran locally.

First things first, the dependencies need to be installed.

Protobuf compiler - installation instructions
Golang plugin for the Protobuf compiler - installation instructions

The protobuf compiler and Golang plugin are used for code generation. They automatically implement the server and client code, only leaving the functions definitions up to the developer.

We create our protobuf schema helloworld.proto.



syntax = "proto3"; // This denotes the protobuf version

// All protobufs have a package name. This is to avoid name clashes with other protocol
// message types. In Golang, the package name is used as the Go package name for the
// generated files. This is however overwritten if you specify the go_package option
package helloworld; 

// This option is specifies the full import path of the Go package that contains
// the generated code.
option go_package = "github.com/senyosimpson/tutorials/grokkingrpc/helloworld";

// This defines the service. In a service, you define rpc methods. These are
// method signatures that contain their request and response types.
service HelloWorld {
    rpc Greet(HelloRequest) returns (HelloReply) {}
}

// Messages are defined by specifying their type and name of the variables it holds.
// These are enumerated from 1 to n.
message HelloRequest {
    string name = 1;
}

message HelloReply {
    string message = 1;
}

Now that we have our protobuf schema, we can generate the Go files. These files will have an extension .pb.go. To do so, execute the command



protoc -I helloworld/ helloworld/helloworld.proto --go_out=plugins=grpc:helloworld --go_opt=paths=source_relative

Now that we have our generated files, we're ready to create our server. If you look in the generated file, helloworld.pb.go, you should find an interface named HelloWorldServer.



// HelloWorldServer is the server API for HelloWorld service.
type HelloWorldServer interface {
    Greet(context.Context, *HelloRequest) (*HelloReply, error)

We have to implement this interface to define our service. In the implementation, the server prints out that it received a given name. It returns a reply message that contains the greeting, Hello <name>.



type helloWorldService struct {}

func (s *HelloWorldServer) Greet(ctx context.Context, r *pb.HelloRequest) (*pb.HelloReply, error) {
    log.Printf("Received: %v", r.GetName())
    return &pb.HelloReply{Message: "Hello " + r.GetName()}, nil
}

As an fyi, the function GetName is automatically generated. This can be found in the helloworld.pb.go file.

The rest of the necessary code is implementing the server itself. The full code



package main

import (
    "context"
    "log"
    "net"

    pb "github.com/senyosimpson/tutorials/grokkingrpc/helloworld"
    "google.golang.org/grpc"
)

type helloWorldServer struct{}

func (s *helloWorldServer) Greet(ctx context.Context, r *pb.HelloRequest) (*pb.HelloReply, error) {
    log.Printf("Received: %v", r.GetName())
    return &pb.HelloReply{Message: "Hello " + r.GetName()}, nil
}

func main() {
    lis, err := net.Listen("tcp", ":50051")
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }

    server := grpc.NewServer()
    pb.RegisterHelloWorldServer(server, &helloWorldServer{})
    if err := server.Serve(lis); err != nil {
        log.Fatalf("failed to serve: %v", err)
    }
}

The client is straightforward, I just copied it from the examples repository. The main line of consequence is the request to the server.



r, err := client.Greet(ctx, &pb.HelloRequest{Name: name})

Here we see the client calling the Greet method. A request message is passed in with a specified name.



package main

import (
    "context"
    "log"
    "os"
    "time"

    "google.golang.org/grpc"
    pb "github.com/senyosimpson/tutorials/grokkingrpc/helloworld"
)

const (
    address = "localhost:50051"
)

func main() {
    // Set up a connection to the server.
    conn, err := grpc.Dial(address, grpc.WithInsecure(), grpc.WithBlock())
    if err != nil {
        log.Fatalf("did not connect: %v", err)
    }
    defer conn.Close()
    client := pb.NewHelloWorldClient(conn)

    // Contact the server and print out its response.
    name := "Bas"
    if len(os.Args) > 1 {
        name = os.Args[1]
    }

    ctx, cancel := context.WithTimeout(context.Background(), time.Second)
    defer cancel()
    r, err := client.Greet(ctx, &pb.HelloRequest{Name: name})
    if err != nil {
        log.Fatalf("could not greet: %v", err)
    }
    log.Printf("Greeting: %s", r.GetMessage())
}

We're now ready to test it! In one terminal, start the server. In the other, we can run the client code and make requests to the server. In the image below, the server is on the left and client on the right. We can see the client gets back the reply message and prints the message to the console.

And that's it! I hope you've taken something valuable out of this walkthrough of the basics of gRPC. I've certainly enjoyed learning and writing about it.

References

RPC

gRPC VS REST

Migrating APIs from REST to gRPC

gRPC

Protocol Buffers

Concepts I learned from diving into Clojure

senyo — Sat, 25 Jul 2020 16:49:59 +0000

This year, one of my goals was to learn a functional programming language. I carried this over from last year because who actually achieves their goals, right? To be fair, I did make an attempt - I began learning Haskell. Let's just say, I'll try again one day! Around April this year, on recommendation from one of my mentors, I decided to look into Clojure. It turned out to be a great suggestion and I've been enjoying hacking away in Clojure ever since! With all that being said, I have been pleasantly surprised by the language, learning plenty along the way.

Clojure is a functional programming language and a dialect of Lisp. Coming from an object-oriented and imperative coding background, both of these ideas were new to me. I'll dedicate the first portion of this post to the Lisp aspects of Clojure and the second portion to functional programming.

Lisp

List processing (Lisp) is a programming language created by John McCarthy in 1958. It is designed to make processing lists simple and efficient (hence the name). During its peak, its main use case was in artificial intelligence research. Clojure is a dialect of Lisp, giving it some useful properties common to all Lisp languages.

S-Expressions

A symbolic expression (s-expression) is the syntax used in the Lisp language and its variants. It provides a representation of a nested list of data. In Lisp, both code and data are represented as s-expressions which leads to the philosophy of code as data. Examples of s-expressions

(+ 2 3)
; => 5
(+ 2 (+ 3 3))
; => (+ 2 6) => 8

A useful property of s-expressions is that they provide a uniform structure. All syntax follows the structure (operator arg1 arg2 ...). This maps well to tree-structured data (as will be discussed further on) and makes writing compilers more trivial. In the above example, the first line adds numbers 2 and 3 to give 5. In the second line, we have a nested list. The inner operation is performed first and then the outer operation - first 3 and 3 are added to give 6 and then 2 and 6 are added to give the final result of 8.

For most programmers, this syntax is strange - I certainly found it awkward to work with in the beginning. After some time with it, I really began to enjoy it. The simple structure of the language makes reading and writing code straightforward. It lends itself to concise and readable functional code which made the process of learning functional programming easier. Considering we often want to chain a set of functions to apply to some data, having this uniform syntax made the mental model of how to achieve your goal simple. For example

(add 4 (multiply 3 (subtract 8 4)))
; => (add 4 (multiply 3 4)) - (subtract 8 4) is computed to give 4
; => (add 4 12) - (multiply 3 4) is computed to give 12
; => 16 ; the final output is 16 as we add 4 and 12

Code as Data, Data as Code

One of the central philosophies in Lisp is the idea of homoiconicity, commonly known as code as data. A homoiconic language is one that treats code as regular data that can be manipulated by the language itself. The main advantage of a homoiconic language is that it is easy to extend the language through the use of macros - new features or additions to the language can be easily created by manipulating already known constructs in the language. There is no inherent need to wait for new versions of the language to be introduced to gain features you need urgently, you can just create them yourself.

Concretely, Lisp code is stored as a valid data structure, an abstract syntax tree (ast). One of the design goals of the language was to make the correspondence between the syntax and the resulting ast as direct as possible. Consider the Clojure code

(+ 4 (+ 2 3))

This generates the ast

This is the internal representation and is evaluated when the code is executed. However, given that + is just a function and the rest are literals, the expression can be stored as a list

'(+ 4 (+ 2 3))

Now we have a list with the elements +, 4, (+ 2 3). One can imagine we can rearrange this, replace the first operator, replace the second function, etc. This will result in a list that can be converted into the adequate ast. Through this capability, we arrive at code as data, data as code.

Functional Programming

Functional programming is a programming paradigm that is based on the composition and application of functions - the core construct in the paradigm. Programs are built from composing multiple pure functions together. Given this structure, functional programming does not maintain or mutate state. This is in stark contrast to object-oriented programming where code is built around objects that have state and that state can be mutated. The core tenets of functional programming are

Pure functions
No side effects
Immutability
Function composition
First-class functions
Referential transparency

Pure Functions

Pure functions are functions that return the same output for a given input. An example of a pure function is

; this function squares a number
; in Clojure, the last evaluated expression is returned
(defn square-num [x]
  (* x x))

(square-num 2)
; => 4

This function will always give the same output if the same input is given no matter how many times it is invoked. The other important aspect of a pure function is that is has no side effects (i.e it does not mutate state).

No Side Effects

Side effects occur when a function mutates state out of its context. Side effects make it harder to reason about code; it is not always easy to understand the flow of data. Example of side effects are reading or writing files, updating databases, creating database connections, printing to the console, so on and so forth. Naturally, functional programs cannot completely rid themselves of side effects, however, they are often separated from the data and are integrated in a principled manner. An example of a function with a side effect

(defn add-numbers [a b]
  (let [num (+ a b)] ; binds the value (+ 4 3) to num
    (println "hello")
    num) ; to return the number
)

(add-numbers 4 3)
; => hello
; => 7

This example, while simple, adequately highlights a side effect. When adding the numbers, "hello" is printed to the console. In this case, it is a fairly harmless side effect. In other situations it may be detrimental. Imagine the function updated a database with the calculated number. The outcome of the function is different every time it is executed as it is adding data to a database. The reason that this can be difficult to reason about is that the function gives no indication such an effect would occur.

Immutability

Functional programming relies heavily on immutability. This means once declared, structures cannot be modified. For example in a mutable language, you can do

x = 1
x = x + 1

This is not possible in functional programming. In functional programming we can create a new variable but we can't update an already instantiated variable. This, again, brings the benefit of making it easier to reason about code and understand the flow of data through a program. Another advantage of immutable data structures is that it makes concurrency much easier to implement. Since objects cannot change once defined, they can be shared amongst threads without qualm.

Naturally, a common question is how do you write any useful code without mutable data structures. This requires a mental shift in the way you write programs - instead of having objects that are mutated as a program executes, you have data that is transformed as the program executes. Every time the data is transformed, a new output is created, leaving the input as it was. Clojure provides backdoors to mutability when necessary but it is not idiomatic Clojure. To formalise this, lets look at the same code done in a mutable manner and an immutable manner

class Car:
    def __init__(self, engine_on=False):
        self.engine_on = engine_on

    def start(self):
        self.engine_on = True

    def stop(self):
        self.engine_on = False

car = Car(False)
car.engine_on == False
# => True
car.start()
car.engine_on == True
# => True
car.stop()
car.engine_on == False
# => True

In this case, we have an object and we mutated its state (i.e the engine_on variable).

Functional programming languages do not have classes. Instead, we store this data as a hash map/dictionary.

; this defines a variable called car with a key engine-on set
; as false
(def car {:engine-on  false})

; a function that switches a car engine on. In this case,
; the assoc function returns a new map with the key
; updated to true
(defn switch-engine-on [car]
  (assoc car :engine-on true))

; a function that switches a car engine off. In this case,
; the assoc function returns a new map with the key
; updated to false
(defn switch-engine-off [car]
  (assoc car :engine-on false))

; the car is passed into the function switch-engine-on
; which returns a new map with the key updated to true.
; the next operation fetches the value of engine-on.
; Finally true? checks whether the value is true
(true? (:engine-on (switch-engine-on car)))
; => true

; because that function returned a new car map, the
; original car map still has the value engine-on set to false
(false? (:engine-on car))
; => true

; we can chain commands to switch the car on and off
; first the engine is switched on, then it is switched
; off. Then we get the value of the key engine-on.
; finally we check if it is false
(false? (:engine-on (switch-engine-off (switch-engine-on car))))
; => true

Function Composition

Function composition is the act of combining multiple (simple) functions to build more complex functions. Due to functional programming emphasis on purity and mitigating side effects, it lends itself well to this pattern. We have already seen examples of function composition in previous Clojure code. Function composition is not unique to functional programming, however, the paradigm makes it feel natural. Alongside this, its support for first-class function makes function composition all the more powerful.

First-class Functions

In simple terms, first-class functions are functions that are treated as variables. This means they can be passed into other functions as arguments, be returned from functions and be saved in variables. First-class functions allow developers to create higher-level with abstractions with functions alone and enables function composition. Other programming languages such as Python also support first-class functions. An interesting use case for first-class functions is the partial function. This function takes in a function, binds a set of arguments to the specified values and returns the updated function. Since Python has this function, lets look at it in both Python and Clojure.

from functools import partial

def add(a, b):
    return a + b

# bind the argument a to the value 2. Adder is now a new function
adder = partial(add, b=2)

# we can use adder now and only specify one argument (i.e a)
adder(4)
# => 6
adder(1)
# => 3
adder(-2)
# => 0

# In essence the partial function has done this
def adder(a):
    return add(a, 2)

Similarly in Clojure

((partial + 2) 4)
; => 6
((partial + 2) 1)
; => 3
((partial + 2) -2)
; => 0

Referential Transparency

Referential transparency is when a function can be replaced with the value it returns without changing the program's behaviour. This is dependent on the fact that functions are pure. Breaking function purity breaks referential transparency. The main advantages it brings are that it makes code easier to reason about, test, refactor and optimize. A simple example of a referentially transparent function is an addition function

(defn add [a b] 
  (+ a b))

We can replace that function with its return value and the calling program will still function the same.

(let [a (add 2 2)]
  (if (= (add a a) 8) ; in this case, adds a to itself and then compares if it is equal to 8
    (println "Sum function works") ; executes if true
    (println "Error in sum function"))) ; executes if false

; => Sum functions works

It is clear that if we replace a with 4, the program would work the same, printing sum function works. Now imagine we defined the add functions as

(defn add [a b]
  (println "Adding" a "and" b)
  (+ a b))

In this case, the program would first print Adding 2 and 2, then Adding 4 and 4 and finally sum function works. If we replaced a with 4, we would not get the first print message. This has altered the program and therefore it is not referentially transparent.

To more Clojure

Learning Clojure has been an enlightening experience. I hope you found some of the concepts equally as awe-inspiring. It's an awesome language to learn to expand your horizons as a developer and at the same time, is a powerful language in its own right. I've found it to be one of the most productive languages I've used. To more Clojure!

Machine Learning in the Wild

senyo — Tue, 09 Jun 2020 08:09:14 +0000

Machine learning promises to be transformational, providing much needed innovation across industries ranging from financial services to healthcare to agriculture. The resurgence of machine learning owing to the astonishing performance of deep learning in computer vision and natural language processing has prompted industry to jump on the bandwagon and utilise machine learning in their businesses. In fact, entire startups are being built around machine learning alone, promising that their new found algorithms will vastly improve and outperform existing products and services.

Whilst new state of the art algorithms are being published week-on-week, using them in a production setting is non-trivial. The additional complexity of using machine learning to power products cannot be understated. It may come as a surprise to new practitioners that most of this complexity has very little to do with the model itself but rather with the infrastructure supporting it. In fact, a common trope is that the development of machine learning models comprises only a small part of the puzzle. So what makes machine learning complex? Data cleaning and warehousing, data versioning, model versioning, distributed training (in the event you are training large models), reproducibility, concept drift, inference speed and optimizing the right thing, to name a few.

Fortunately, this makes for an extremely rewarding engineering experience. As engineers, we are drawn to technically challenging problems - at least, I'd like to think so! Machine learning in the wild does not fall short in any regard. It is truly stimulating, though it might cause you to lose a few hairs, destroy your laptop and move to Motuo - consider yourself warned. In this post, I illuminate some of the core challenges faced in walking the distance with machine learning.

Data Quality

What is machine learning without data? The answer is nothing. Absolutely nothing. As the common saying goes, "garbage in, garbage out". Perhaps the most important aspect of machine learning is the data used to train models. Without quality data, no amount of sophistication and creativity will help your machine learning model perform accurately. Whether you agree with the below tweet or not, it does provide an interesting angle on what a model is - a view of the data it was trained on. // Detect dark theme var iframe = document.getElementById('tweet-1260658763687538688-656'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1260658763687538688&theme=dark" }

Data is the source of truth for your model. Every phenomenon it learns is encompassed by that data. Given this stance, you'd be crazy to not ensure that your model is trained on the highest quality data you can create or get your hands on. Unfortunately, ensuring data quality is difficult, expensive and laborious. Rather than trying to circumvent this, it should be incorporated into the company/team strategy. Making the necessary capacity and budget provisions will make this manageable. Ideally, ensuring data quality should be reduced to a mere nuisance as opposed to a blocking point to deploying machine learning models.

Concept Drift

Real-life data is akin to a living organism. It is dynamic and changes over time, drifting further and further away from the data that came before it. This is called concept drift and is a common challenge in machine learning. In its simplest form it is not difficult to deal with as it only requires training on the new batch of data. The complexity arises on an infrastructural level. To handle this effectively, it requires models to be continuously retrained as their performance starts deteriorating in production. Setting up the correct infrastructure to facilitate easy training to deployment and monitoring in production can be difficult depending on the velocity at which this needs to happen. In this blog post by Netflix, they discuss their system architecture for recommendations. Interesting to note is their separation of offline, online and nearline computation. Offline is the most common paradigm in machine learning today where data is stored and models are trained on the new data. However, they reached limitations with this approach as it often took time between updates to the models, impacting their recency. They solve this problem using online computation which reacts to the latest changes in the data in real-time. Nearline computation is a hybrid of these two approaches. Considering Netflix's scale, most of us would not need such comprehensive architectures to run machine learning in production but it certainly highlights the complexity that can arise.

To drive concept drift home, I'll walk through an example. Keeping with the theme of Netflix, imagine you worked there and trained a recommendation model in January. At this time, life was good and relaxed. The majority of customers were watching comedy shows. The model was able to learn this and its performance was admirable and subsequently deployed into production. It is now June and you realise the model is underperforming. The world is going through a pandemic and in response, people have shifted away from comedy shows toward thrillers and horrors. Evidently, this behavioural change implies that the dataset that was used to train the model in January is outdated. To bring the model up to date, you train it on the latest data and as expected, your magic algorithm is performing well once again. Hooray!

Versioning

Versioning is an essential part of software engineering. Git has largely solved this problem for traditional software development. In machine learning, we not only have to version our code but also the parameters of our model and the data that was used to train the model.

Data versioning is a new challenge that machine learning brings to the table. As discussed earlier, data is the source of truth for a model. When models are trained on different data, we naturally expect differences in their performance. Without keeping track of which data a model was trained on, we cannot make any significant claims on its performance relative to another model. Alongside that, without knowing which dataset a model was evaluated on, we cannot fairly compare competing models. Finally, we cannot reproduce experiments if we do not version our data. Reproducibility is a growing concern in the machine learning community and rightfully so - ensuring we create the best environment for it is important. From this, it is clear that data provenance is of utmost importance in machine learning efforts. At this current time, there is no industry standard for data versioning. There are many options available in the market. A few of them include dvc, dolt and pachyderm. They do not all fall within the same use cases but there is extensive overlap between them.

One example of a scenario in which poor data versioning can play against you is during the experimentation phase. Let's suppose you decide to create an object detection system. You train a model with dataset A. You decide to change the data slightly and train a model on this data; dataset B. You repeat this data alteration process twice more, creating dataset C and D. After these experiments, you find that the models trained on dataset A and C perform the best. Unfortunately, you did not record the differences between these datasets and only possess the data with the most recent changes (i.e. dataset D). It is now impossible for you to reproduce the best performing model. Alongside this, you cannot understand why datasets A and C result in the best models, reducing your understanding and confidence in the model's performance. This highlights the importance of data versioning and why it is critical to the machine learning process.

Experimentation Platforms

Experimentation platforms refer to the necessary platforms needed to use machine learning as effectively as possible. It is not necessary in the beginning of the machine learning journey but as a company begins to scale up their machine learning efforts, it becomes critical. Thousands of experiments need to be run and evaluated. Having the correct infrastructure and tools makes the iteration speed fast and allows models to make it into production.

Let's walk through an example to highlight its importance. You've decided that you need to create and train a new model for fraud detection. You whip out your trusty jupyter notebook and set off on the adventure of a lifetime. On completion of your model and dataset, you proceed to train the model on a virtual server provided by the company on their prefered cloud provider. The virtual server is not configured, has no fault tolerance and requires many manual steps (ssh into it, install all dependencies, monitor it, etc). Naturally, these steps present many points of potential failure as well as general headaches, slowing down development. The company realises that this is inefficient and invests into tooling to ease the machine learning development process. There are a whole host of tools such as Kubeflow, Comet, Weights & Biases, MLflow to name a few. These are utilised to create a machine learning platform that makes the machine learning lifecycle easier to manage. Now you have the instant ability to monitor progress, visualise training and validation loss and view the results on a user-friendly dashboard. Clearly, this has improved the iteration speed, removed the need to understand infrastructure and provided the necessary amount of visibility. By using the correct platform, the time taken between iterations has decreased substantially.

Eventually, there is an upfront cost that must be paid in scaling out machine learning efforts by providing the necessary platforms. Data scientists and machine learning engineers need to be empowered to do their work effectively and this is one step in that direction. Alongside this, it allows them to add more value by being able to improve their models in a shorter period of time.

Optimizing the Right Thing

In many product offerings, the core goal of machine learning is to improve the customer experience. In a music company such as Spotify, examples of improving the customer experience include song recommendations and personalisation based on music taste. In a medical imaging company, it could help doctors make more accurate diagnoses or provide diagnoses themselves. Metrics provide an objective way of ranking the performance of models and ensuring you have the latest and greatest model in production. Unfortunately, a common pitfall is optimizing the wrong metric meaning that the metric does not translate to an improvement in the customer experience. A simple question to ask is: what does a 10% improvement of this metric mean for the customer? It is often quite simple to analyse your model against standard metrics in that line of work, however, it does not always translate to a tangible improvement for the purpose they were designed to serve.

A simple case to illuminate this is in medical diagnosis. Imagine you are the patient and this revolutionary AI is used to provide you with a diagnosis. If the model predicts that you have a disease and you do not, the overall harm inflicted by the incorrect diagnosis is minimal. Despite being incorrect, you will survive and hopefully live for many years. Now imagine the model predicts that you do not have a disease that you do in fact have. In response to this prediction, the doctor lets you off into the world to fend for yourself without medication or treatment. Evidently, this is a harmful situation which will likely have severe consequences. How does this relate to optimizing the right thing? Imagine being responsible for creating the model that performs this task and your chosen metric is accuracy. Accuracy measures the total number of correct predictions out of total predictions. Your new model is 5% better than your old model and so you release it into production - happy days! Unfortunately, that metric does not communicate the most important value to you: how often your model predicts an individual does not have a disease when they actually do! The appropriate metric here is recall. Contextually, perfect recall means that the model successfully predicts every case where an individual has a disease, however it makes some mistakes where it incorrectly predicts that individuals have a disease that they do not have.

If only it was as easy as the example makes it seem! In most cases, metrics will have to be designed with the aim of improving the extent to which they translate to an improvement in the customer experience. This talk by Jade Abbott walks through an example of this - start at the 15 minute mark.

The main takeaway is that metrics should be carefully designed and give an indication of the model's impact on the customer. Additionally, it allows the rest of the business to understand the value of the work you are doing. Then they will also celebrate when your model improves by 0.01%!

Greener Pastures

Machine learning in the wild is difficult! This should not discourage us by any means. While difficult, it is extremely rewarding and if you're the kind of engineer I hope you are, it is exciting! With all difficult things comes the joy of working on technically challenging problems.