DEV Community: Alex Loukissas

Programming principles from id software

Alex Loukissas — Tue, 03 Mar 2020 22:58:03 +0000

Probably the most iconic video game company of the 1990s, id Software developed such groundbreaking titles as Wolfenstein 3D, Doom, and Quake. In a somewhat recent talk, John Romero (the co-founder of id), outlined the company’s programming principles that allowed them to produce so many quality titles, one after the other, in a very short time, with a very small team.

Think of this as a companion to Lampson’s timeless paper on system design. Many of the principles are common and they are all quite relevant today.

Although these programming principles refer to video game development, most (if not all) apply to general software development. In this blog post, I outline my own interpretation of these principles, paired with screenshots from Romero’s own slides for reference. I also make an attempt to generalize each principle beyond game development. Here we go.

Principle 1: Just do it (and do it well).

This doesn’t necessarily mean overly complicate your current version of the product. In fact, this may one way of describing the iterative development practice. Build something that does one thing really well and keep improving upon it constantly. Just keep your quality standards high in each iteration.

Principle 2: Keep your code always runnable.

In his talk, Romero mentions how they had programmed the code to show an image of a bagel when the user hit an error loading a sprite. By adding good defaults/fallbacks, they made the game still playable. Had they not done this, the user would be blocked until the bug had been fixed (i.e. lost productivity hours). You can imagine how important this becomes as an engineering team grows larger. A practical example of this is using defaultProps in ReactJS.

Principle 3: Keep it simple.

This is obviously not novel. Call it Occam’s Razor or the KISS principle, keeping things as simple as possible is timeless great advice.

Principle 4: Invest time in building great tools.

In his talk, Romero mentioned that he built a level editor called TED. The time he spent building TED paid hefty dividends, since it immensely helped them rapidly ship one game after the other by boosting productivity. Since those days, there has been an explosion of developer tools that have helped boost developer productivity. But if something off-the-shelf doesn’t cut it, try to identify whether an internal tool can help your developers be at their most productive (even if it can take development resources off the main product).

Principle 5: Test your code thoroughly.

This covers many topics that many of the most effective engineering teams use as best practices: (a) dogfood your product as much as possible; (b) don’t delegate to others (e.g. QA engineers, or worse, customers) to find bugs in your code; (c) write as many tests as possible to accompany your code.

Principle 6: Fix bugs as soon as possible.

We are very strict about this at AgentRisk, a practice we’ve carried over from our previous startups. During our daily stand-ups, we make sure that any new bugs have the highest priority and get fixed ASAP. Obviously, not all bugs are equal, so some business-related judgement is warranted here.

Principle 7: Use a superior development system.

This is one that may mostly apply to game development. In other cases, you may want to go the other route when it comes to testing during development. For example, you may have users running your application on a mobile device that has very inferior specs, or they may be accessing your web application over a high-latency 2G connection. Make sure they don’t have a sucky UX.

Principle 8: Write code for this version of the product.

This mostly translates to “don’t transfer over limitations of your past code and its implementation to future code”. This is a tricky one and kind of ties with Principle 4 . As engineers, we are often tempted to “rewrite everything” from scratch. This is where having experienced engineering leadership is really important. Often, the choice to take the plunge and do a new implementation can pay dividends down the line (similar to building tools). For example, as Slack started scaling, they scraped their v1.0 implementation entirely and moved to a brand new architecture, reaping the rewards many times over. We also had a similar experience moving from Python to Elixir, having a much more robust codebase and much increased developer productivity.

Principle 9: Use good component abstractions.

This is really hard. If you have ever built and maintained an API, you know how hard it is to get it right (especially the first time). In his talk, Romero gives the example of encapsulate the functionality of a torch together with the flame and other related objects. Had they needed to move or update all torch instances in a level, having more a granular abstraction could have led to e.g. forgetting to move/update a flame. Spend a lot of time on this and try to get this right the first time. There will be compounding rewards in development and debugging time.

Principle 10: Seek feedback from peers while coding.

Using code review software can help with this. For more complex parts of a product, advance architecture review may be warranted. In any case, make sure you promote a culture that values communication and seeking feedback.

Principle 11: Give coders creative freedom.

There are many ways to dice an onion. Give your coders the creative freedom to come up with their own solution to the problems they’re working on. Just make sure to enforce some coding standards, so that any member of the team can jump into the codebase. Getting caught up on coding aesthetics can waste valuable time, so it’s best to leave this to linters and auto-formatters. This does not mean that e.g. identifying suboptimal implementations in code reviews shouldn’t be encouraged. Just focus on things that are objectively wrong.

Thanks for reading!

I hope this has been useful. Below is a link to the entire talk. I suggest you also read Masters of Doom, which gives a detailed back-story to id software. Many thanks to Jon V and Diwaker Gupta for their feedback on initial drafts of this.

You better work in cents, not dollars 💸

Alex Loukissas — Sun, 23 Feb 2020 21:56:31 +0000

When building an application, some lines of code are more important than others. Screw up your mutex logic and you may run into a deadlock or a race condition. Screw up a database query and you can wipe out an entire table. Screw up how you handle money, and someone will get short-changed (or even worse: you get fined by the SEC).

But how can you screw up how you handle money, you ask?

Hint: you may have snoozed during that boring lecture where the professor was covering floating point arithmetic and scientific notation. Let’s have a quick refresher from Tom in the video below:

Why using float to store money is a terrible idea

I couldn’t have said it better than Bill Karwin:

Bill Karwin

@billkarwin

If I had a dime for every time I've seen someone use FLOAT to store currency, I'd have $999.997634. #ieee754jokes

03:49 AM - 20 Jun 2013

333 218

But let’s try this out! I’ll be using Python in the examples in this post, but this is not a Python-specific issue. This has to do with how computers represent numbers internally.

Rounding errors

The reason issues like the one in the code above occur is due to rounding errors when doing floating point arithmetic. Let’s repeat the example from the video above. Here we’re adding just two floating point numbers:

If in either of these two examples you pull up your favorite calculator, you will get the “correct” answer, i.e. 0.1 + 0.2 = 0.3 and 165 * 19.4 = 3201.

Hopefully, by now I should have convinced you that using float to store money is a bad idea. If you’re still not convinced, keep reading!

Always use integers to store money

While float is the troubled child in the family, integer is the poster child that always behaves as it’s expected of her. In our case: no rounding errors.

How does this work? What about cents?

I think the primary reason we automatically use floats to represent money is that we more commonly think in a currency’s main unit, e.g. 19.4 dollars.

But forget what Diddy and Biggie told you. It’s all about the cents.

To avoid rounding errors, you should store money in a currency’s smallest unit using an integer.

For example, we’d represent 19.4 dollars as 1940 cents. Just make sure you use long integers, so you don’t run into brand new problems.

Storing money in a currency’s smallest unit is a common software design pattern made popular by Martin Fowler back in 2002. It’s even adopted by Stripe in their APIs. That’s a pretty solid endorsement in my book.

Let’s retry the code for the example above using this pattern:

Note that now that every monetary value is in cents, we need to be aware of that and properly format stored values into what the user expects to see.

At minimum, we need to divide the stored value by 100:

>>> 320100 / 100
3201.0

This is what we would expect as a result if we’d done the math by hand.

What’s better than an integer? A Money library.

As we saw above, it may be easy to forget that monetary values are in cents and run into new problems in the presentation layer/UI. The best way to deal with this is to use a dedicated library (or class) for everything money-related.

There are ton of great libraries that implement this pattern: Dinero.js is a really great one for JavaScript as is Money for Elixir (the two main languages we use at AgentRisk). Surprisingly, I wasn’t able to find something for Python.

The most common one for Python is good, but it uses Decimal to store values. While this is generally fine, we really wanted to stick with the Fowler pattern across all systems and languages and use integers to represent money everywhere.

That’s why I decided to write my own and make it available on GitHub. You can check it out here:

agentrisk / agentrisk-money

Python class for working with money, following Martin Fowler's money pattern

agentrisk-money

Python class for working with money, following Martin Fowler's money pattern.

Influenced by https://github.com/carlospalol/money and https://github.com/luka-mladenovic/fowlers-money.

Currently only supports USD as currency.

View on GitHub

It’s still in beta, but has decent test coverage. PRs welcome!

Thanks for reading!

Hopefully this was useful to you, or at least a good refresher how floating point arithmetic works. If you want to nerd out further, I’d recommend going deep in this classic article from ACM. Mind you, there are other issues one can run into when handling money, like allocation and calculating percentages. Both these will be added to agentrisk-money library very soon.

This article was originally posted on the AgentRisk blog.

Elixir’s with statement is fantastic

Alex Loukissas — Wed, 19 Feb 2020 07:23:22 +0000

I gotta be honest. It was not exactly love at first sight when I first encountered Elixir’s with statement. But just like I learned to appreciate (and love) the whole non-defensive/let-it-crash approach, the same happened here as well.

The result: I was able to rewrite a bunch of complicated functions with very hard-to-read nested case and/or cond statements in a very clean, and, more importantly, maintainable and extensible way using with.

Basic usage

The basic premise of with is that it lets you chain together pattern matches and arriving at the positive result, in a very concise statement. Error handling (if any — remember, this is still let-it-crash land) is done in an optional else clause, after we’re done with the positive codepath.

Let’s look at a basic example. Suppose you’re working with the Elixir library for Stripe and you want to implement a function that creates a new subscription for a customer. Here’s what this may have looked if we hadn’t used the with statement:

What’s wrong with the code above? Here’s a hint:

Let’s try to write the same function using the with statement:

That’s a lot cleaner and easier to read, no?

Alas, it’s not all rosy

But there’s a problem. In the second implementation, it’s really hard to infer which of the steps of the function calls failed the pattern matching, so that we can return a more useful error to the caller. It also makes it harder to debug.

So, did we trade readability for usability?

Not quite.

We can do better!

Fear not, reader! A really simple solution that we’ve been leveraging heavily in our codebase at AgentRisk is to leverage pattern matching. Here’s how:

Each chained function call returns a tuple with an atom that helps us know exactly where things went wrong in the chain.

If we rewrite the previous code example like this, this is what we get:

In the example above, we are swallowing the error return values, but this was in favor of simplicity. The core idea here is that we tag each step with a unique tuple pattern, so that we can know exactly where the error occurred.

A final note

Hopefully, this post clarified a few of the nuances of Elixir’s with statement. We have not found a better way to implement complex functions in our codebase, for readability, maintainability, and ease of debugging.

I want to thank Carlos Brito Lage for introducing me to this pattern as well as editing an early version of this post.

This post was originally posted on the AgentRisk blog.