DEV Community: Ben

The Joy of Monads

Ben — Wed, 10 Jan 2024 13:14:32 +0000

I love taking some time out in December, but my code obsessed brain often results in me using it to learn about some new coding concepts. This year, I took to reading though Miran Lipovača's awesome Learn You a Haskell for Great Good! which had previously sat on my bookshelf unread for a long time.

Probably one of the main reasons I didn't approach it was that Haskell has a reputation for being a difficult language. I still think that reputation is pretty well earned, but more than anything, mainly because its such a big language. There's a lot in Haskell, and it has a pretty complex type system. As a digression, I think this is what makes Rust famously difficult to pick up too. Rust doesn't have very many "gotchas" compared to other languages, but that doesn't stop the fact that there's a lot of completely new concepts you'll need to learn in order to pick it up.

One of the things that suprised me most, was from learning the concept of Monads, which I've always assumed where insanely complicated. Turns out, the first sentence of the wikipedia article on Monads is enough:

a monad is a structure that combines [functions] and wraps their return values in a type with additional computation

That's literally it. In other words, a monad is a design pattern where we abstract away additional work that happens on each function call.

I ended up putting together a super small library for implementing monads in python as a learning exercise. So lets work the "Result" monad from that as an example!

Take this extremely helpful code:

import requests

def get_age():
    age = input("what is your age?")
    return age

def human_to_dog_years(age):
    response = requests.get("https://animalyrconverter.com/api/v1/humantodog")
    dog_yr_per_human_yr = response.json()[converter]
    return age * dog_yr_per_human_yr

def dog_to_cat_years(age):
    response = requests.get("https://animalyrconverter.com/api/v1/dogtocat")
    cat_yr_per_dog_yr = response.json()[converter]
    return age * cat_yr_per_dog_yr

if __name__ == "__main__":
    cat_years = dog_to_cat_years(
        human_to_dog_years(
            int(get_age())
        )
    )
    print(f"you are {cat_years} cat years old!")

I'm sure you can see lots of problems with this example that I haven't even notices, but lets just focus on one + a bonus:

Main problem: we don't have any structured error handling
Bonus: pythons nested functions are kinda janky to read

Let's ignore the bonus for now, and just implement some basic error handling to guarantee no runtime errors:

if __name__ == "__main__":
    try:
        cat_years = dog_to_cat_years(
            human_to_dog_years(
                int(get_age())
            )
        )
        print(f"you are {cat_years} cat years old!")
    except Exception as error:
        print(f"I don't know how old you are in cat years")
        print(f"This error happend: {error}")

Python linters are all screaming right now because of the generic except not specifying an error, but lets not worry about that since this is just a toy example and instead look at what this looks like using a monad to carry out error handling instead:

from monads.monads import Result

if __name__ == "__main__":
    cat_years = (
        Result(get_age())
        .bind(int)
        .bind(human_to_dog_years)
        .bind(dog_to_cat_years
    )
    if cat_years.value:
        print(f"you are {cat_years} cat years old!")
    else:
        print(f"I don't know how old you are in cat years")
        print(f"This error happend: {cat_years.exception}")

or, as a bonus we can use some syntactical sugar and express binds like this (don't worry too much about this part though, it's handy, but isn't needed for understanding monads):

from monads.monads import Result

if __name__ == "__main__":
    cat_years = Result(get_age()) >> int >> human_to_dog_years >> dog_to_cat_years
    if cat_years.value:
        print(f"you are {cat_years} cat years old!")
    else:
        print(f"I don't know how old you are in cat years")
        print(f"This error happend: {cat_years.exception}")

The Result monad is basically a class wrapping a value, and giving us "bind" to run a function over it. It has a value and an exception, one of which is always None.

the bind definition just looks like this:

def bind(self, func):
    if self.exception:
        return self
    try:
        return Result(func(self.value))
    except Exception as exception:
        return Result(None, exception)

Basically we're saying:

If there's already an error, don't even attempt to do anything
Otherwise, try to return a new Result with the updated value
If that fails, return a new Result with the relvant exception

I'm just aiming to explain monads here rather than evangelising them, but some motivations for this approach might be:

More complicated error handling behaviour is easier since you have an exception variable that isn't within the except Exception nest
You can pass the Result monad around different parts of your program easily, which is trickier with try/catch statements

The result handling itself though, is just an example, what makes this a monad is that it is doing the result handling on each function call, so our function calls don't need to worry about the result handling at all.

If that's still a little fuzzy, let's take a look at the Maybe monad:

import random

from monads.monads import Maybe

def maybe_not():
    return random.choice([7, None])

def double(x):
    return x * 2

def square(x):
    return x ** 2

if __name__ == "__main__":
    result = Maybe(maybe_not()) >> double >> square
    if result.value:
        print(result)
    except:
        print("nope")

Here Maybe handles the possibility that something is None for us. Here's its definition:

class Maybe(Generic[T], Monad):
    def bind(self, func: Callable) -> "Maybe":
        if self.value is None:
            return Maybe(None)
        return Maybe(func(self.value))

Basically, if the value is None, we won't do anything, otherwise, we'll return a new Maybe with the executed function.

This means that we don't have to think about the possibility of whether something is None within our double or square calls, which would otherwise error.

And one more, the List monad:

from monads.monads import List

def double(x):
    return x * 2

def square(x):
    return x ** 2

if __name__ == "__main__":
    result = List([1, 2, 3]) >> double >> square
    print(result)

Here the List monad abstracts away the complexity of dealing with a List and applies a function to all the members. Otherwise we'd have to change functions like double to be this:

def list_double(x_list):
    return [x * 2 for x in x_list]

Hopefully I've convinced you now that monads are just a pretty regular (and helpful) design pattern, and not some kind of dark magic!

To summarise, monads:

wrap some kind of value
do a bunch of extra work on function calls to that value

Which means:

we don't need to repeat logic on those individual function calls
function definitions don't need to handle additional abstractions or logic

I had a bunch of fun learning about this - hoping to dive into some more functional programming stuff next. Happy new year 🎉

Coding Virtues

Ben — Tue, 26 Sep 2023 08:55:08 +0000

I have a pretty unusual and specific guilty pleasure. Well actually I have an entire music taste based on that premise. More specifically though, I have one that relates to programming: clean coding.

By that, I don't mean that all my code is spotless or beautiful or anything, but some part of my brain really enjoys reading about clean code. I have a neat bookshelf of books on programming, with a healthy dose of books like Uncle Bob's Clean Code, Fowler's Refactoring and plenty more 90s joys that promised me they where going to solve all the problems I make for myself when writing programs. Actually, if I'm honest, I think it's less about my own code, and more about fodder for when I don't like something. Saying "I don't like this" is rarely accepted, and never helpful, but somewhere along the way "this is a code smell" has become cannon.

I think clean code is a pretty important feature, but I still see my relationship to it as essentially a guilty pleasure, something like the programming version of virtue signaling. Ultimately, for me it comes down to the fact that once you've read about the general principles of clean code early on in your programming career, reading about it again is basically just confirming your existing beliefs. My experience tells me code is almost always either reasonable, or completely horrible, and there isn't that much in between. While a bunch of people could maybe do with googling clean code, few need to "brush up" on clean coding.

Which makes me end up asking myself, why does it feel so good to talk about clean code?. I don't think I'm alone - I've witnessed a lot of times when pull requests have been bogged down in style comments, and bikeshedding on style seems to be a pretty commonly reported phenomenon. I think the temptation is that style is easy, understanding performance and correctness of code can be challenging, and often involves a lot of work up front to understand what the code is trying to do and how it achieves it. Good style comments are based around this, but most style comments (like "this variable name is bad and should be longer", "this function name is bad and should be shorter") require only a cursory glance at the code to make.

I've noticed murmurs recently around clean code being less important than performant code and while that linked video is great, it's easy for this just to become another opportunity for bikeshedding.

That's why I'm introducing my ultra-trademarked-propriety-copyrighted-pay-me-£10-each-time-you-say-it trio of coding virtues! I think that rather "this is bad" / "this is good", we should focus on these three qualities. Code is nice, when it is:

Expressive: it's easy to understand what's going on and reason about the code
Performant: the code runs faster on average
Maintainable: it's easier to make changes to the codebase, and avoiding breakages over time

This isn't some kind of holy trinity and there's plenty of other good qualities code can have, but these correspond nicely to the three things that we do with code: reading, running and editing.

Rather than go into some kind of diatribe about the necessary and sufficient conditions for each, I think it's more helpful just to look at some quick examples where we have a choice to trade one off for the other.

Performance VS Expressiveness

This is probably the most common trade off because performance often introduces complexity, which almost always affects codes readability negatively. Take the following python code:

from typing import Any, Optional

def get_biggest_7_value_key(
    some_dict: dict[Any: float],
) -> Optional[Any]:
    """
    Return the corresponding key, for the largest
    divisible by seven value in a dict
    """
    divisible_by_seven_dict = {k: v for k, v in some_dict.items() if v % 7 == 0}
    if len(divisible_by_seven_dict) == 0:
        return None
    max_value = max(divisible_by_seven_dict.values())
    max_key = [k for k, v in some_dict.items() if v == max_value][0]
    return max_key

This is a pretty silly example, hopefully its clear enough what its doing - its looking only at a dictionaries keys, finding the largest one that's divisible by 7 and then returning the corresponding key.

For example, here:

x: str = get_biggest_7_value_key({"a": 4, "b": 14, "c": 7, "d": 100})

the variable x will equal "b", the corresponding key to 14.

Granted, its not clear why we would want to run this code once, but lets say we want to run it a lot of times, on some randomly generated, but similar dictionaries:

import random
import time

def example_dict() -> dict[str: float]:
    return {
        k: random.choice([1.0, 7.0, 21.0, 48.0])
        for k in ["a", "b", "c", "d"]
    } | {"e": 7.0}

start = time.perf_counter()
for _ in range(10_000_000):
    some_dict: dict = example_dict()
    get_biggest_7_value_key(some_dict)
end = time.perf_counter()

print(f"Total time taken: {end-start:0.4f} sec")

Running on my hardware, that took 30.7597 seconds but your mileage may vary.

Save we want to optimise things, the obvious thing to do here is cache the function values. But we can't since a dictionary isn't a hashable object type. We could easily do something like this though:

from typing import FrozenSet, Tuple
from functools import cache

@cache
def get_biggest_7_value_key(
    set_representation: FrozenSet[Tuple[str, float]],
) -> float:
    """
    Return the corresponding key, for the largest
    divisible by seven value in a dict

    Takes a set of key, value tuples, generated from a dictionary
    """
    divisible_by_seven_set = (i for i in set_representation if i[1] % 7 == 0)
    max_pair = max(divisible_by_seven_set, key=lambda x: x[1])
    return max_pair[0]

Now lets run it a bunch of times again:

start = time.perf_counter()
for _ in range(10_000_000):
    some_dict: dict = example_dict()
    get_biggest_7_value_key(frozen_set(some_dict.items()))
end = time.perf_counter()

It now takes 24.0632 seconds on my machine - which is faster than before, all that caching paid off, yay! But its undeniably more confusing to read - the function takes a frozen set as its sole argument, but we don't really care about a frozen set at all, we care about some dictionary, but then have to find a different representation so that it will cache, and yada-yada. Code can be a little like a joke sometimes, if you have to offer an explanation, well, it isn't good.

If we are in a context where we care about performance, then this additional confusion is worthwhile, but what if we're only going to pass a single dictionary into this function? Probably it isn't worth it.

One example is more expressive, and one example is more performant, but neither is better, they're just two trade-offs we can make.

Expressiveness vs Maintainability

I think 'maintainability' might not be the most obvious virtue since editing code also involves reading it, so it seems like expressiveness and maintainability should be the same thing. But we don't have to look hard to find an example where they are at odds:

const makeEven = n => {
    if (n % 2 == 0) {
        return n;
    }
    return n + 1;
}

or consider this:

var isEven = require('is-even');

const makeEven = n => {
    if (isEven(n)) {
        return n
    }
    return n + 1
}

Ok, ok, I get it. These are both perfectly readable and fine, nobodies mind is gonna be blown in either case. But the first one we obviously have to do some brain-work, we're doing some modulo arithmetic, why? Oh right, of course. The second is much more expressive, we're explicitly saying what we're doing, checking n is even. This is a toy example, but we can imagine a more sizable logger head with something like, initial checking credit card number validity for example.

In the second example, we get more readability, but we also have to bring in an additional dependency, that needs to be managed from this point on - there's a bunch of practical questions around, can it be relied on, might the api change, etc, etc. If it sounds silly, just think back as far as left pad, where a bunch of organisations had issues with code because a tiny, very similar to is-even library was taken down.

For something as simple as isEven that I'll go out on a limb and say you probably don't need to introduce a new dependency. But I don't think it's clear where the line is for something like, prime numbers, email validation etc. Those are all judgement calls based on the reliability of a library, and the size and lifespan of the codebase you're introducing dependencies into.

Don't be a jerk in Pull Requests

Think about these three qualities as a balance - have you thought about all of these issues when reviewing or writing code, or just some? And never, never, never, pleeeease never write simply "this code would be better. . . " in a review. Even if its true, it probably isn't going to be well received, and definitely isn't a way to lead to a constructive discussion.

Thinking of code quality in terms of describable metrics like expressiveness, maintainability and performance makes for a meaningful conversation about what code is trying to achieve and how it could better do that. They also cash out in more meaningful ways than "better", "worse", "clean" etc.

Reinventing the Wheel

Ben — Tue, 26 Sep 2023 08:54:20 +0000

Good news if you work in software - impress your friends and colleagues with these magnificent sentences!

"Have we thought about scalability?"
"Let's stop and think about what problem we're actually trying to solve"
"We don't need to reinvent the wheel"

You can say these at any point, in any meeting, without considering context and people will nod in agreement, and think you're very clever. It's not without good reason: scalability is normally important (though probably not quite as much as people think); it's never a bad idea to refocus your thoughts and make sure haven't lost sight of the task at hand; and if reinventing the wheel was a good idea, then step one of every project would involve writing the linux kernel.

In fact, "don't reinvent the wheel" is maybe special, in that it's sort of the whole point of writing software. Let's think about hello world in javascript:

console.log("hello world!")

First off the bat, you only need to write this once, and not every single time you want to print out hello world - that's pretty nice, especially if you want to do something a lot of times. You only have to come up with a method for doing something, and that can be repeated again and again. Even better, when you write this, you don't even have to come up with a method for most of the stuff that's actually happening, like:

How strings are encoded into data
What a console is, and actually how to write to it
The interpretting of javascript code
etc, etc

You can probably imagine that if we had to write all these component parts then very little would ever get done. But I actually want to make the case that reinventing the wheel is totally awesome and you should definitely do it! Here's why:

Reinventing the wheel makes you consider the design of things you take for granted

There's a great blog post called I am my only user which talks about building your own tools. That's a great idea for lots of reasons, but actually, some of the things I've learnt the most from are tools I've made that I wouldn't even dogfood to myself. To give an example, I wanted to experiment with what an ideal interactive shell would look like, so I wrote one that extends python called clamshell. There's lots of reasons why this is a bad idea, here's a few:

There are already a gazzilion interactive shells in the world
There is already an interactive shell in python that's very good
There are lots of shells you can copy lines of code for (eg) installation into. It's unlikely that most software is going to start writing ready-to-use-code for my bespoke shell script.

But I learnt a whole bunch out of doing this exercise - taking things back to first principles made me realise a bunch of usability features that I actually wanted out of an interactive shell. For instance:

It's kinda annoying that the default way to remove things "rm" just straight up burns whatever file you had out of all existence
It's also unnecessary that when moving directories with "cd" you have to know exactly where you're going "i.e. cd ../../Documents/some-folder/sub-folder" when it's really quick to parse the non-hidden folders in a user directory.
Posix shell more or less is string-only-input-and-output, which is probably one of the reasons it's been so easy to plug programs together in unix historically, but also seems lacking by today's standards.

After implementing this in my home-brewed shell script, and then deciding not to use it, I found ways to alter the behaviour of "rm" to move things to my OS' recycle bin and introduce a handy "jump" function in my .zshrc file (where you can "jump sub-folder" and "cd" to the first match for "subfolder" that fdfind runs into). I don't have a solution for 3 yet, other than that I think passing json between shell scripts would solve this, although the fact I've never seen it done makes me wonder if it's a terrible idea for some reason I haven't bumped into yet.

Implementing these things isn't particularly difficult, but neither are things I would have thought to do if I hadn't undertaken the task of writing an interactive shell. I'd been using bash/zsh/fish more or less every day of my life for about 5 years at the point I started writing clamshell, and had never considered these things, and I probably would have gone another 5 years without thinking about them if I hadn't reinvented the wheel.

Reinventing the wheel is a great way to learn new things

I'm currently working on a super-secret-project (tm) that involves writing an esoteric programming language and a bespoke interpretter. I won't go into details here, but it's probably fair to say that my contributions to the world of code interpretation aren't going to be taught in Computer Science classes of the future.

But I'm also learning a huge amount about code, and the way it runs. You don't get a lot of opportunities to learn about and practically develop these kind of things unless you specifically reinvent the wheel. There are only three major operating systems (sorry BSD) and two major compiler toolchains* (GCC & LLVM). That's not a whole lot of real work to go around, but a lot of people would benefit from / enjoy learning about compilation and OS'. Serenity OS probably isn't going to be running in offices any time soon, but that doesn't stop lots of people from learning huge amounts, and creating something awesome in the meantime.

In fact, it's exactly because code is so good at helping us avoid re-inventing the wheel that means without doing so, it's easy to program with only a surface level understanding of what's going on under the hood.

Wikipedia lists around 700 programming languages, something like 900 historical OS' and cites almost 27 million profressional programmers, by my logic, there's probably room for a few more wheels in the universe.