DEV Community: W. Brian Gourlie

Mutability in Rust, and how it differs from object-oriented languages

W. Brian Gourlie — Tue, 04 Feb 2020 00:26:15 +0000

This post started out as a long-winded answer to a stackoverflow question. The question made me realize that there's a pretty universal way of thinking about mutability when coming from popular languages like C# and Java, and that coming to rust with this mindset often results in confusion and frustration.

I’ll be comparing and contrasting solutions to common problems involving mutability in a way that (hopefully) expresses the differences intuitively. For the non-rust examples, I'll be using Java, but these examples can be expressed similarly in other object-oriented languages.

We'll start with a simple goal—Define a Dog type with two fields:

Name, which will be immutable
Age, which will be mutable

In the first few examples, we'll avoid common idioms like getters and setters in order to keep things simple.

Here's how we'll define our type in Java:

class Dog {
    public final String name;
    public int age;

    public Dog(String name) {
        this.name = name;
        this.age = 0;
    }
}

An equivalent type in rust would look like this:

struct Dog {
    pub name: &'static str,
    pub age: usize,
}

impl Dog {
    pub fn new(name: &'static str) -> Dog {
        return Dog { name, age: 0 };
    }
}

In the rust example, you may notice that there's no modifier on our name field to suggest that we're controlling mutability in any way. This is because rust has no concept of mutability at the field level.

Rust takes a fundamentally different approach. If we want to modify any field on our type, we need to declare a mutable variable pointing to it. By default, all variables are immutable and you must add the mut modifier if you intend to mutate it:

// Declare an immutable variable
let harris = Dog::new("Harris");
harris.age += 1; // Compile error!

// Declare a mutable variable
let mut harris = Dog::new("Harris");
harris.age += 1; // No problem!

At first glance, it would appear as if Java provides finer-grained control over mutability than rust—If a variable's mutable, we can mutate all of its fields; if it's immutable, we can't mutate any of them.

Mutability and `self` parameters

Let's update our examples to make them more idiomatic. We'll make our fields private and expose methods to access or modify them.

First in Java:

class Dog {
    private final String name;
    private int age;

    public Dog(String name) {
        this.name = name;
        this.age = 0;
    }

    public String getName() {
        return this.name;
    }

    public int getAge() {
        return this.age;
    }

    public void incrementAge() {
        this.age += 1;
    }
}

And our rust example:

struct Dog {
    name: &'static str,
    age: usize
}

impl Dog {
    pub fn new(name: &'static str) -> Dog {
        return Dog { name, age: 0 };
    }

    pub fn get_name(&self) -> &'static str {
        return self.name;
    }

    pub fn get_age(&self) -> usize {
        return self.age;
    }

    pub fn increment_age(&mut self) {
        self.age += 1
    }
}

In the rust example, the get_name, get_age and increment_age methods all include a self parameter. Like variables, parameters are either mutable or immutable. self differs from other parameters in how it's passed to the method: If we have a variable named dog and we call dog.increment_age(), the dog variable is implicitly passed to the increment_age method as the self parameter, and can access private fields declared on the type.

This is even more interesting when we consider how this works in conjunction with variables. If we have an immutable variable, and we call a method on it that takes a &mut self, we'll encounter a compile error because the method requires mutable access to our value!

// Declare an immutable dog variable
let dog = Dog::new("Harris");
dog.increment_age(); // Compile error!

// Declare a mutable dog variable
let mut dog = Dog::new("Harris");
dog.increment_age(); // This works!

So what happens if we declare a variable as mutable, and call a method that takes an immutable self parameter? In this case, the call is valid, but the method is still only granted immutable access to the self parameter.

To reiterate: In order to call a method that requires mutable access to self, the variable must be declared as mutable, but methods that declare an immutable self parameter can be called via mutable and immutable variables alike.

Encoding mutability in return types

We're going to introduce a new field to illustrate how mutability works with more complex types. We'll add a List (Vec in rust) to our example named friends:

class Dog {
    private final List<Dog> friends = new ArrayList<>();
    ...

    public List<Dog> getFriends() {
        return this.friends;
    }

    public void addFriend(Dog friend) {
        this.friends.add(friend);
    }
}

In the java example, our new friends field is declared final, which means we can never change what the field points to, however, we're still free to add and remove items from the list via our getter method as seen below:

   Dog harris = new Dog("Harris");
   Dog buck = new Dog("Buck");
   harris.getFriends().add(buck);

So how would this work in rust? Let's update our example:

struct Dog {
    ...
    friends: Vec<Dog>,
}

impl Dog {
    ...
    pub fn get_friends(&self) -> &Vec<Dog> {
        return &self.friends;
    }
}

In our rust example, we're actually encoding mutability into the return type of our get_friends method! The &Vec<Dog> return type represents an immutable reference because it lacks the mut keyword. It would be a compile error if we attempted to add an item to the returned friends list:

let mut harris = Dog::new("Harris");
let buck = Dog::new("Buck");
harris.get_friends().push(buck); // Compile error!

Let's break this down:

We declared a mutable harris variable
The get_friends method returns an immutable reference to our friends field
The Vec type has a method named push which requires a mutable reference to self
Our code fails to compile because the immutable reference returned by get_friends cannot be used to call a method that requires mutable access to self

So, even though our original variable was declared as mutable, we use the get_friends method to hide access to our list behind an immutable reference. Consequently, it's a compile error if we attempt to mutate it.

As an exercise, let's say we wanted the get_friends method to provide mutable access to the the returned friends reference. Take a moment to consider what we'd need to change.

Have an idea? Here's the answer:

pub fn get_friends(&mut self) -> &mut Vec<Dog> {
        &mut self.friends
    }

To summarize, we had to:

Declare our self parameter as mutable,
Declare the return type as mutable, and
Return a mutable reference to our list

We can now modify the friends list via our get_friends method! Whether or not we'd ever want to do this in practice is another question.

Conclusion

Mutability in rust is very different from what most people are used to, and it can take some time for it to sink in. Once it does, however, it becomes an incredibly powerful way to enforce guarantees that aren't possible in most languages. It's also a fundamental part of the language, working in concert with other language features to achieve a unique combination of safety and performance.

A quick note about rust idioms

The rust examples aren't particularly idiomatic, but that was done for the sake of clarity and simplicity. Perhaps I'll touch on interesting rust idioms in a separate post!

NPM hot-takes and the pitfalls of trivialization

W. Brian Gourlie — Tue, 17 Apr 2018 05:27:08 +0000

The infamous left-pad controversy

Most people are probably aware of the left-pad controversy by now. If not, I'll get you up to speed:

The maintainer of a popular NPM package deleted their package in protest, breaking thousands of other packages that had depended on it. The problem was quickly remedied and solutions put in place to prevent it from happening again, however, following the controversy was a deluge of blog posts and articles asking why we're dependent on tiny libraries that solve such trivial problems. "Have we forgot how to program?" one blog post asked, illustrating the general sentiment at the time.

Left-pad redux

The left-pad controversy still comes up from time-to-time, usually as the butt of jokes. I recently ran across a Medium article pointing out the absurdity of the is-odd package, which as of this writing has over 3 million downloads over the past seven days. Indeed, on the surface, relying on a package to determine if a number is odd appears to be the epitome of overkill. One of the most highly clapped comments made the following claim:

If you can’t write a function to determine if a given integral input is odd in under 10 seconds, you’re either a crappy typist or you shouldn’t be programming.

That's a pretty bold claim, and needlessly insulting to boot. Is it really that simple? The typical 10 second solution would probably look something like this:

function isOdd(i) { return i % 2 === 1; }

Let's write some unit tests just for the hell of it:

expect(isOdd(-2)).toBe(false);
expect(isOdd(-1)).toBe(true);
expect(isOdd(0)).toBe(false);
expect(isOdd(1)).toBe(true);
expect(isOdd(-2)).toBe(false);

Boom! Done. Let's call it a day.

Just kidding. We're not done yet.

When we initially considered this problem, we kind of assumed that all inputs would be integers. However, there's no way to enforce that a particular type be passed to the function. Not only that, but all numbers in javascript are actually double precision floats. Let's add a few unit tests to make sure our function works with floating point inputs:

expect(isOdd(1.5)).toBe(true); // FAIL!

Ok, so our function doesn't work with floating point numbers. There are a couple options at this point:

I don't intend to pass anything but whole numbers to the function, so I don't care if it returns the wrong result.
I'll just return true if it's a not a whole number.
I'll throw an exception if it's not a whole number.

None of these solutions is necessarily wrong, since the best solution could involve any number of considerations. Either way, it still warrants consideration. Not quite the 10 second solution we initially thought it was.

Ok we're done. The end.

Just kidding, there's one last thing we need to consider here: Javascript is dynamically-typed and we need to determine how to handle non-numeric inputs. Again, we're faced with a few different solutions:

I don't intend on passing non-numeric inputs to the function, so I don't care what happens if I do.
I'll just let javascript do whatever wacky implicit conversion it wants and accept the result.
I'll throw an exception.

Again, none of these is necessarily wrong, but it's yet another consideration we didn't initially think of. Trivial libraries handle these considerations so we don't have to.

Behavior is not trivial

Trivial libraries do more than just solve trivial problems. They consider all the little edge cases that we're likely to overlook and provide consistent and robust behavior. This is especially important with languages like javascript.

As a general rule of thumb, I try not to trivialize things that look trivial on the surface. Programming, like the real world, is non-trivial and full of nuance.

I recently discovered Parcel, and it's amazing

W. Brian Gourlie — Mon, 09 Apr 2018 04:46:22 +0000

While playing around with WebAssembly and rust, I stumbled across a plugin for an asset bundler I hadn't heard of before. Admittedly, my initial thought was "Dammit, why couldn't they just use webpack?" After all, webpack has served all my needs in the past and has seemingly become the defacto standard. "Time to roll up my sleeves and learn how to configure this thing..." I thought.

The first thing that struck me about Parcel is that there is no configuration file. In theory, I thought, that's great, but inevitably I'll need fine-grained control over something.

(Narrator: "He has yet to encounter such a scenario.")

Indeed, I've only used Parcel for two personal projects thus far, but I haven't had to configure anything at all. Everything Just Works™, and I've used it to do some relatively exotic things:

Compiling rust to WebAssembly and bundling the resulting artifact
Bundling and exposing markdown files as React components

And then some not-so-exotic things, which are all supported out-of-the-box: Javascript, TypeScript, CSS, HTML, etc.

More than just zero configuration

Zero configuration is a huge selling point, but it's not the only thing Parcel has going for it. Whereas bundlers like webpack transform javascript files exclusively, Parcel can transform anything (correction: webpack 4 can now transform other types of files). Any file type not supported out-of-the-box can be supported by way of plugin. The practical implication here is that you can use any type of file as your entry point, not just javascript. This allows for a more natural way of bundling assets. For example, I can specify an HTML file as my entry point, and it will recursively locate and bundle all assets from there.

Parcel accommodates this by operating at a fundamentally higher level than its counterparts–It actually parses supported file types and transforms the resulting abstract syntax tree. When it encounters a javascript file, it transforms import and require declarations. When it encounters HTML, it will transform script and link tags. When it encounters CSS files, it transforms import and url declarations.

Oh and it's f***ing fast

According to the README on Parcel's github page, it's twice as fast as webpack when not caching assets, and nearly an order-of-magnitude faster when caching assets. It caches assets by default, and I have yet to encounter a situation where I've needed to turn off caching. Anecdotally, the numbers check out.

So, next time you're in need of an asset bundler, you should seriously consider Parcel.

NodeJS leadership is a toxic mess

W. Brian Gourlie — Wed, 27 Sep 2017 00:11:21 +0000

Preface

In this article, I’m going to be calling out toxic behavior exhibited by certain high-profile members of the NodeJS community. As is the nature of such things, some people will use this as justification to harass them. I want to be clear: Under no circumstances should anyone respond in any sort of harassing or threatening manner to the people called out in this article (or anyone, really). My goal in writing this is to improve the tech community as a whole, for everyoneâ€Š–â€ŠNot to incite anger and hate.

Also, since this article touches on issues of diversity in tech, I want to make clear that I absolutely support diversity initiatives. This article is specifically highlighting toxic behavior that is being excused and justified because it is, supposedly, done in the interest of diversity.

Inclusion by exclusion

“If you are not a white dude and have things to say about JavaScript testing, please submit a talk to [redacted]”

The above quote is from the NPM CEO, who, for context, is a white male who planned to speak at the conference he was promoting (the conference organizer reached out to tell me his talk was never formally accepted).

This is only a single example, but his twitter feed is filled with equally exclusive and adversarial messaging. His response to any criticism follows a general theme among prominent members of the NodeJS community wherein they hoist themselves into a position of moral authority by asserting that any criticism is a result of people simply not supporting diversity measures. For example:

“You’re sarcastically whinging at a stranger because you fear losing your unearned privilege. I’m bored now.”

A discussion quickly followed on /r/node where people voiced displeasure at the messaging put forth. Among the responses were members of underrepresented groups talking about how the CEO’s messaging made them uncomfortable and didn’t feel like it was helping their cause. In response, the NPM CEO dismissed the node subreddit entirely:

“/r/node is not part of the node community. It’s a place for GamerGate/KiA/TRP to launch attacks on the @nodejs community.”

It’s worth noting that, overwhelmingly, the discussions that took place on reddit were not anti-diversity, but rather criticizing this individual’s toxic behavior.

A pattern of behavior

“In fact, if you were a white dude and you wanted to talk at the conference, your chances were basically nil.”

This quote comes from a member of the Node foundation board of directors (who, for context, is also white). This quote in particular was among many that were part of a code-of-conduct report (detailed here) filed against the quoted individual. In response to the code-of-conduct report and ensuing discussion, another individual (and NPM employee) is on record as saying:

“reminder to reddit manbabies that they are kinda pathetic, irrelevant, boring, creepy nerds who’ll never amount to anything. The only thing they seem to know how to do is complain, masturbate about upvotes and karma, and be creepy stalkers when women are involved. Like holy shit, do these weirdos even realize what they’re doing? Y’all are literally the sketchy, sweaty nerds that make us wanna vomit. But keep doing what you’re doing, because I’d rather that y’all vomit-inducing rapey gross dweebs actually keep being open about who you are”

This resulted in another code-of-conduct report that was summarily dismissed as “random trolling not submitted in good faith. It’s worth pointing out the following excerpt from the NodeJS code-of-conduct:

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.

In the case of the NPM code-of-conduct, which contains similar language, there’s a clause stating “The definitions of various subjective terms such as â€˜discriminatory’, â€˜hateful’, or â€˜confusing’ will be decided at the sole discretion of the npm abuse team. In other words, there’s no objective means of enforcing the code-of-conduct, and if there were, many high-profile members of the Node community would be in violation of it based on any reasonable interpretation.

My personal take

A concerning aspect of the messaging put forth by high-profile members of the Node community is that it overwhelmingly comes from members of majority groups (white, often male). At best, it’s a highly misguided attempt to do the right thing. At worst, it’s exploiting a good cause for the purpose of self-promotion while actively hurting the case for diversity and inclusion.

Promoting diversity through exclusionary and adversarial messaging does not help to foster a safe, welcoming environment for underrepresented groups. Furthermore, I legitimately believe it will result in brain-drain from communities where this behavior is deemed acceptableâ€Š–â€ŠNot because white men will feel excluded, but because good people of all backgrounds will not want to be associated with the rhetoric put forth by community leaders.

Cross-platform development on Windows is suddenly awesome

W. Brian Gourlie — Mon, 07 Nov 2016 03:42:04 +0000

Anyone who's done cross-platform development on Windows knows that getting things to compile can be a huge pain in the ass. Sometimes it's not even possible. The whole situation kinda sucked, until now.

Enter Linux on Windows

There's this new thing called the Windows Subsystem for Linux. Without going into too much detail, WSL provides a bash shell that exposes a Linux environment. It's native Linux, too — These are ELF binaries running inside Windows. The Windows file-system is fully accessible, with each drive having its own mount point. Linux processes can bind to the Windows loopback address and it all just works. You can read the technical details here, but the practical implications are simple enough: Windows is finally a good OS for doing cross-platform development.

WSL isn't installed by default, so you'll need to follow these instructions to get started.

A practical example

I do a lot of Rust development in my personal time. Rust actually has excellent Windows support, however, there are packages that link against C libraries that aren't well supported on Windows. One of my projects depends transitively on OpenSSL, for example. Trying to compile on Windows results in this:

While most of these issues can be resolved with enough persistence, it's a huge waste of time and effort. My typical reaction is to ditch my beefy desktop and its multiple monitors for a small, relatively under-powered MacBook where things just work™. Not anymore!

To get started, I needed to install the Linux Rust toolchain. Something I haven't mentioned to this point is that the WSL uses Ubuntu, meaning it uses apt for package management. Installing things from the bash shell inside Windows is no different than doing it on any other Ubuntu installation:

brian@DESKTOP-O4RQ37I:~$ sudo apt-get install build-essential
...
brian@DESKTOP-O4RQ37I:~$ curl https://sh.rustup.rs -sSf | sh
...

First I install the build-essential package which installs gcc (among other things). Then I install rustup, Rust's toolchain manager, via its handy shell script. That's it — I navigate to the same code base that failed to compile within the standard prompt and compile it from the bash shell instead:

I was even able to configure IntelliJ IDEA to launch the bash shell in its terminal, meaning my entire workflow stays the same. The difference is that I'm now compiling a Linux binary and running it directly in Windows!

Hi, I'm W. Brian Gourlie

W. Brian Gourlie — Mon, 07 Nov 2016 03:24:05 +0000

I have been coding for 17 years.

You can find me on Twitter as @dubya_brian

I live in Madison, WI.

I mostly program in these languages: Java, Javascript, Rust, and Elm.

I like turtles.

Nice to meet you.