Don’t pay the for-loop tax

Note: this post was originally published on my Medium profile

Once, when doing code review on a TypeScript project at my work, I came across several instances when a colleague of mine used a for loop, even though it wasn’t necessary (i.e. a more readable declarative equivalent was available). In a joke I stated that we should impose a “for-loop tax for every loop used unnecessarily.
It made me think however, why so many people tend to go for the longer and more error prone solution with the loop and I got to the following conclusion: Nearly every (mainly) imperative programming language course/book I ever took/read (be it Pascal and C# in high school or C/C++ and Wolfram Mathematica in college) contained a section like

“These are the loops available, this is how you write them and this is how you use them to solve these basic problems”.

There is an important point to note here: they only teach how to write a loop but scarcely explain why you would need one (or sometimes even worse they state that loop based solutions are the best ones). For future reference, I decided to write this “cookbook of the main types of situations where loops are often used and how they can be replaced. All the examples will be written using JavaScript as it is very popular, but the rationales behind the examples can be used in many other languages as well.

#1: I need to go over an array and get a single value as a result

We start with the simplest of problems:

Given an array of numbers, return the sum of its elements.

const sum = (array) => {
    let result = 0;
    for (let i = 0; i < array.length; i++) {
        result += array[i];
    }
    return result;
}

const numbers = [5, 25, 8, 18];
console.log(sum(numbers)); // logs 56

If you attended similar courses as me, you surely recognise this code. Create a temporary variable, initialise it with zero and using a for loop iterate over the array returning the final value of the variable. There are some problems though:
For something as simple as a sum of an array, 7 lines of code seem quite a lot.
You must handle the bounds of the iteration yourself. In other words you must know to start at zero (in JavaScript, many other languages have 1-based arrays – Wolfram Mathematica for example) and end at i that is strictly less than the length of the array (not less than or equal). This is prone to errors especially if your work in many languages at the same time.

const sum = (array) => array.reduce(
  (total, current) => total + current,
  0);

const numbers = [5, 25, 8, 18];
console.log(sum(numbers)); // logs 56

The solution that remedies both those problems is to use the reduce function (in other languages also called fold or aggregate). In a single expression we iterate over each of the array elements adding them together (stating the sum’s default and initial value is zero). Notice there is no mention of the iteration bounds, it just guarantees that it will go over all the elements from first to last.

#2: I need to create a new array from a given one and transform all the elements

This is another common problem, let’s illustrate it with this example:

Given the array of prices, return a new array with the prices n % lower.

const discount = (originalPrices, discountAmount) => {
    const multiplier = 1 - discountAmount;
    // we must clone the array
    let result = new Array(originalPrices);
    for (let i = 0; i < originalPrices.length; i++) {
        result[i] = originalPrices[i] * multiplier;
    }
    return result;
}

const prices = [5, 25, 8, 18];
console.log(discount(prices, 0.2)); //logs [ 4, 20, 6.4, 14.4 ]

The loop-based way to do this is pretty similar to the sum code. There is one additional problem though: in order to not destroy the input array, we must clone it first and then transform the values in the new array. This can easily be forgotten introducing a potentially unwanted side effect in the application.

const discount = (originalPrices, discountAmount) => {
    const multiplier = 1 - discountAmount;
    return originalPrices.map(price => price * multiplier);
}

const prices = [5, 25, 8, 18];
console.log(discount(prices, 0.2)); // logs [ 4, 20, 6.4, 14.4 ]

The cloning problem can be avoided altogether using the map function. For a given array it returns a new array where each element is the corresponding element in the original array transformed using the provided function (in our case multiplied by the discount multiplier).

#3: I need the numbers from m to n

Another common situation where loops are used is when generating linear ranges as an input for further transformations. A classic example is:

Return an array of the first n squares

const squaresBad = (n) => {
    let result = [];
    for (let i = 1; i <= n; i++) {
        result.push(i * i);
    }
    return result;
}

const squares = (n) => {
    let result = new Array(n);
    for (let i = 1; i <= n; i++) {
        result[i - 1] = i * i;
    }
    return result;
}

console.log(squaresBad(5)); // logs [ 1, 4, 9, 16, 25 ]
console.log(squares(5)); // logs [ 1, 4, 9, 16, 25 ]

This is a problem that can be solved very badly when using loops. The first naïve solution suffers from the problem that it pushes a new element to an array every iteration. This expands the array and may cause it to reallocate in memory being slow (benchmark).
The second approach instantiates the array of correct size beforehand avoiding this problem, but we can easily make a mistake when assigning the current value (see the result[i – 1] expression in the second for-loop).

const range = require("lodash.range")
const squaresLodash = (n) => range(1, n + 1).map(
    (n) => n * n);

const squares = (n) => [...Array(n).keys()].map(
    (n) => (n + 1) * (n + 1));

console.log(squaresLodash(5)); // logs [ 1, 4, 9, 16, 25 ]
console.log(squares(5)); // logs [ 1, 4, 9, 16, 25 ]

While there is no native way to generate a range of integers in JavaScript there are two ways to tackle this problem in a more declarative ways with map: using the lodash.range function, or a clever ES2015 syntax trick (source).

#4: I need to do something with side effects n times

The final use case of loop I want to discus here is invoking a method with side effects more than once. As Edsger Dijkstra famously said:

Two or more, use a for

The simplest example to illustrate this case is:

Console.log the string “Hello world n times

This is in my opinion the only justifiable use case for loops in JavaScript (not counting infinite loops) as it is the most concise and performant way (at least until Tail Call Optimisation arrives to most environments).
However, I would strongly recommend to abstract this into a helper function to restrict the loop to a single place.

const doNTimesLoop = (n, f) => {
    for (let i = 1; i <= n; i++) {
        f(i);
    }
}

const doNTimesRec = (n, f) => {
    const body = (m) => {
        if (m > n) return;
        f(m);
        return body(m + 1);
    }

    return body(1);
}

//both log "Hello world" five times
doNTimesLoop(5, x => console.log("Hello world"));
doNTimesRec(5, x => console.log("Hello world"));

As we can see in the examples (both calling the provided function with numbers from 1 to n), the iterative version is shorter and simpler to write. Also the “loop-free version would cause a stack overflow on environments without Tail Call Optimisation.

Conclusion

On four elementary situations, we described how to use declarative style to replace loops and therefore make our code shorter and less error-prone.
Do you use loops? Do you disagree with any of the solutions? Comment please!

Comments

[Richard Tallent]

Ok, I'm gonna be the old fart here.

1. If anything has a "tax," it's using map/reduce/filter, because these require defining a function (frequently an anonymous one) to perform the unit work, then that function is called for each iteration. So for N loops, you get at least N+1 additional function calls, with all of the associated overhead, compared to just having some code in a loop.

2. Calling map/reduce/filter doesn't eliminate the for loop, it merely moves it. The implementation of these methods has... you guessed it... a for loop.

3. Loops can break or return early. While findIndex() helps with some situations you'd need that for, other use cases don't have an equivalent.

4. Loops can modify their iterator variable, allowing them to skip over values (or even go back). Granted, this is an edge case.

I suppose the idea is that loops somehow require a mental tax? I've been using "for" and its siblings for over 30 years in just about every programming language, so they're pretty natural for me.

I do use these abstractions, and enjoy the syntactic sugar where it makes sense. But avoiding a loop structure should be a choice made with a full understanding of the pros and cons, it shouldn't be some sort of default because "indentation is a code smell" or similar hair-brained rules.

[Massimo Artizzu]

Indeed, the performance penalties of functional programming methods must be known, but you're misunderstanding what's the "tax" here: it's indeed a "mental" tax, which led us to use old patterns because they were the only ones available in older languages, and they're still available anyway, while there could be other, more expressive ways to do the same things.

More expressivity means the code is more readable and maintainable, because it gives you more hints about the purpose of the code. If that implies writing a separate function for the iteration, so be it, because it's exactly the function's name that brings additional value.

Sure, you can always write a comment. But comments should explain why, not what: writing comments is tedious, so let the code speak for itself as much as possible.

[Dan Homola]

Thanks for the reply. I'll try to comment on your points:

Ad 1. The tax was originally meant as a real monetary tax (or a fine more precisely) and it was meant as a joke. There were no performance connotations to the word in this context. As others have stated in this thread: if you need performance go for the loops.

Ad 2. I agree, but it hides it from you, and I believe it is a good thing. When reading someone else's code, I'd much rather see a map than some loop because with the loop I have to infer the what from the how whereas with map the what is already written for me.

Ad 3. So can find, includes and some others. Granted these are not as widely supported, they can be polyfilled.

Ad 4. I think that is actually a disadvantage, just because you can do something, doesn't mean you should (excluding the performance optimisation scenarios).

I totally agree with you on that any choice should be made with all the implications in mind. The approaches recommended in the article are just that – recommendations :)

[t0ss]

You pretty much covered what I was going to comment here. I'm honestly a little skeptical of some of the mantras surrounding Javascript at the moment. There are some ideals being tossed around at the moment that might lean a bit too hard on perceived "readability advantages" and straightforwardness. In other words, the performance implications of some of the current proposed best practices make me a tad uncertain regarding their longevity.

edit: I'm not saying the giving examples are bad either, I feel like this piece actually gave great examples. I'm speaking towards the bigger trend I've been noticing.

[Josh Cheek]

So for N loops, you get at least N+1 additional function calls, with all of the associated overhead, compared to just having some code in a loop.

I may be wrong, but I would have assumed that modern JS interpreters can optimize this away. I would also expect it to be faster to call the loop they implement in the underlying language.

Calling map/reduce/filter doesn't eliminate the for loop, it merely moves it. The implementation of these methods has... you guessed it... a for loop.

If you're iterating over an array, but not necessarily for other iterable structures. A nice thing about using the iterator functions is that you don't have to know / care what kind of collection you're iterating over. Using the abstractions means you don't have to embed iteration knowledge at every location in your code that you want to iterate. Your code doesn't change just because the details of how to iterate change.

Loops can break or return early. While findIndex() helps with some situations you'd need that for, other use cases don't have an equivalent.

IIRC, there's an interface for defining iterators that allows them to return early.

Loops can modify their iterator variable, allowing them to skip over values (or even go back). Granted, this is an edge case.

It's a good point, if you can't do that, then you have to retain state somehow (probably via a stack). Although, if the logic for how to prepare for the next iteration were that complex, I would probably choose a while loop over a for loop (or perform the update section of the for loop in its body).

[Massimo Artizzu]

This expands the array and may cause it to reallocate in memory being slow

That's surely correct, but if you're interested in performance you should definitely use a for loop, as any solution based on callbacks, while more expressive, is one or two orders of magnitude slower, for small and large arrays:

console.time("Array.push");
var array = [];
for (var i = 0; i < 1e7; i++) array[i] = i;
console.timeEnd("Array.push");
// Array.push: 220ms

console.time("new Array.push");
var array = new Array(1e7);
for (var i = 0; i < 1e7; i++) array[i] = i;
console.timeEnd("new Array.push");
// new Array.push: 47ms

console.time("Array.map");
var array = [ ...Array(1e7).keys() ].map((_, i) => i);
console.timeEnd("Array.map");
// Array.map: 1209ms

But let's not forget that we also have for...in and, above all, for...of in JavaScript, which bring much nicer semantics.

These are my main cases of why I use for loops in JavaScript:

1. I'm releasing an open source package which could be used in performance-intensive tasks;
2. my code needs to be transpiled and things like for...of are polyfilled with an unbearable amount of code (in the case Babel can't infer if the collection is actually an array or any other kind of iterable);
3. because - ah, screw it - it's clear enough since it's a pattern everyone knows, while building an array with [ ...Array(n).keys() ].map(...) is quite obscure at the first glance.

The rest of the article is pretty much spot-on. Concepts that need to be reiterated.

[Dan Homola]

Thank you for your insightful response. I totally agree that in performance-critical applications loops are the way to go and that the for...of loops addresses some of the problems I have with simple for loop.

I just believe it is easier to get it right with map/reduce. It fits in the "Make it run, make it right, make it fast, make it small" workflow nicely.

[pew pew rainbows an shit fuck yea]

It's not just a little faster to use the for loop - it's orders of magnitude faster. Why adopt an abstraction that reduces performance of a code segment by upwards of 80% (tested 86% slower on my desktop, 84% slower on iPad gen 5).

Like why adopt inefficient design patterns in the first place?

jsperf.com/old-man-yells-at-clouds...

[James Muturi]

A little late to the discussion, but generating the array declaratively would probably go a little faster if we make Array think it's converting an "array-like" structure and provide a map function.

console.time("Array.from");
var array = Array.from({length: 1e7}, (_, i) => i);
console.timeEnd("Array.from");

[Adam Brandizzi]

The for(;;) construct is error-prone but I feel "going functional" in JavaScript is hardly a good solution.

Take for example the sum() function from the post. In Haskell, it would be defined this way:

sum' l = foldl (+) 0 l

Or here it is in Scheme:

(use-modules (srfi srfi-1))
(define (sum l)
    (fold + 0 l))

The sum' function is quite concise because the (+) operator is already a function. Even Python (where such a function would be frowned upon) could make it clearer:

import operator
from functools import reduce

def sum(l):
    return reduce(operator.add, l)

So, when we go to JavaScript, we got this very weird line (total, current) => total + current, where there is a total, and a current, and they are added up. Now I have a bit more to understand here. The for(;;) loop is brittle, but why not this?

const sum = (array) => {
    let result = 0;
    for (let i in array) {
        result += array[i];
    }
    return result;
}

Or, better, this?

const sum = (array) => {
    let result = 0;
    for (let v of array) {
        result += v;
    }
    return result;
}

There is a mutable variable but it is clearer what is going on. The JavaScript syntax is helping me here. Do you disagree?

I find trying to optimize for sum() functions problematic. sum() is a solved problem, whatever the solution you choose. But get another code where one calls, let us say, map() inside the function given to reduce(). It is surprising confusing. Given the trend to avoid naming functions in JS, it will become a mess.

Also, many functional patterns are helpful in functional languages, or specific situations, but not in all places. Many JS programmers write apply(compose(fn1, fn2.bind(arg)), value) because they think it is as good as f1 . (f2 arg) $ value. But is it? I don't think so. JavaScript didn't make this construction clear, as Haskell did.

Functional patterns make sense a lot of times. If you have an algorithm which can be reused except for one part, it will be great to pass the function (instead of, let us say, a Strategy pattern house-of-cards). Functions as values are a cornerstone for asynchronous code. I even like the each()/forEach() methods, so I do not have to discover if it uses a[0], a.get(0), a.item(0).

However, in general, a "more functional" JavaScript code will not be better than one that uses some loops. Trying to get better code by being "more functional" is very much like improving by being "more OO." And we know the result: how many Command pattern instances we found that should have been a function?

Now I wonder the same about lambdas that could be a for(of) block.

[Dan Homola]

Thanks for the reply, it is an interesting view. While I understand that the for..of cycles remedy some of the problems I have with cycles, I still prefer the reduce.

I admit that for something so simple as a sum there is a little advantage, but this was chosen as an illustration for the problem exactly for the simplicity of the problem.

What I wholeheartedly agree with you on is that the all the operators (including the function call operator) in JavaScript should be functions as well.

[Wilson Yu]

I think the only argument for using a a for-loop is to break out of the loop. I think pretty much everything else can be done using the functional approach.

[Dan Homola]

While I agree that breaking out of a loop is a point for the loops, breaking out of a for rubs me the wrong way. I believe you should use a while (or do-while) loops and specify the condition appropriately. Breaks hamper readability imho and make the code even harder to reason about or even formally verify.

[tbodt]

I've written plenty of loops that would look a lot worse if I had to write them without break statements. For example, this code to look for possible forward moves for a white rook on a chessboard:

while (row++ < BOARD_TOP) {
    if (is_white_piece(row, column))
        break;
    GENERATE_MOVE;
    if (is_black_piece(row, column))
        break;
}

The first check could obviously be moved into the loop condition, but I can't think of a good way to move the second check into the loop condition too.

Edit: Language is C.
Edit 2: GENERATE_MOVE call was in the wrong place, moved it outside of the second if.

[Massimo Artizzu]

If that's still JavaScript (could be even C or Java), it could be refactored in a more expressive way:

const piece = chessboardRows.find(pieceAtColumn(column));
if (piece && isWhite(piece)) {
    GENERATE_MOVE;
}

function pieceAtColumn(column) {
    return function(row) {
        // something like row[column]?
    };
}

Maybe it's more verbose, but also clearer.

[tbodt]

This was C, I didn't have a find method on all the arrays.

[Massimo Artizzu]

Ah, crud, you're out of luck then XD

I think the aim of this article is mainly JavaScript or any other language that support methods for functional programming.

I guess you're stuck with classic loops unless some kind of abstraction helps you with that (in Java <8 it's still a mess anyway).

[tbodt]

I discovered that I pasted the code wrong, so I've edited the post to correct that. In the original version I could have written a function to find the index in the array, but I can't do that for the correct version.

[Dan Homola]

I agree the loop would look worse. However, in this case all you are doing is finding the first piece and if it's white calling a method, right?

So you could do something like

const pieceIndex = findFirstPiece(row);
if (pieceIndex > 0 && is_white_piece(pieceIndex , column))
  GENERATE_MOVE;

where the findFirstPiece would find the first row value that is either a white or a black piece, returning -1 if no such value was found.

[tbodt]

Oops, I pasted the code wrong. What I wanted to do was generate every move from the current position of the rook up to a piece, and including the piece if the piece is a black piece. See updated post.

[Massimo Artizzu]

Indeed, but as a reminder there are methods that allow you to "break out" soon looping an array with FP. Namely, find and some are probably the most expressive. (Alas, find isn't supported in IE, not even 11, but OTOH is easily polyfill-able.)

[Lluís Josep Martínez]

Let's say you have to build a nightly batch process to process milions of entities (orders, invoices, bank transactions ... whatever). A typical approach would be to open a result set (or the equivalent structure in JS) to process the entities one by one and do partial commits. With a functional approach, will you read the milions of entities into an array just to use map/filter/reduce ? I hope your server has some Gigabytes of contiguous memory free, otherwise it wil fail miserably. The optimal solution will use a loop.

[Dan Homola]

Well, I would use streams and their on method which IMO resembles the functional approach...

[Lluís Josep Martínez]

Do you mean in JS or Java? Because in Java doing it the functional way is extremely complex according to stackoverflow.com/questions/322092...

Do you have a JS example that reads rows from a relational DB without using loops?

[Dan Homola]

I meant JavaScript. An example could be Google Lovefield (haven't tried it though).

[Lluís Josep Martínez]

By looking at the Readme, it seems like this API returns an array of rows. Which obviously is not acceptable when dealing with huge result sets.

}).then(function(results) {
// The SELECT query's Promise will return array of rows selected.
// If there were no rows, the array will be empty.

results.forEach(function(row) {
// Use column name to directly dereference the columns from a row.
console.log(row['description'], 'before', row['deadline']);
});
});

[Prash]

My argument has always been that writing a for loop that takes each element in an array, transforms it, and adds the output to another array, is writing your own implementation of map.

map is a well understood function these days that everyone can read so don't keep re-implementing it. It's just more code for you to own and read, and there's a chance of bugs in your implementation. To put it another way, you should use the map function for the same reason you use any functions at all: to avoid repeating code.

The same goes for filter and reduce.

RE performance: Measure it in the context of your application. I doubt there is a difference in the vast majority of cases given engine optimisations, but you can always fall back to a for loop where you can prove it's worth doing.

[Leah]

The funny thing is, writing your own map function with a for loop is faster than the native one (a plain for loop is still faster because it doesn't have to deal with the function invokations)

[Wolfie's just fine]

for-loop-tax is accompanied with an array-method-tax.

I agree the first go-to should be array-methods, even with objects "for in" usually sucks compared to Object.keys(foo).map((propName,i)...) as you get i included.

The exceptions IMO are performance, break, continue and return.

It requires discretion because in some circumstances you could sometimes be writing spaghetti code when trying to be too functional where break, continue & return "sometimes" result it far less code and better maintainability (sometimes). But obviously vice versa as you've demonstrated.

[Michael Manukyan]

Never rely on the order of keys of objects in JavaScript because by specification the interpretator can order them as it wishes

[Wolfie's just fine]

Not for generic prop names but if you previously "created" incremental prop names:

const obj = {thing-0: 'thing-a', another-thing-1: 'thing-b', 'something-2: 'thing-c'};
...
... obj[key.value.slice(0,-1) + i] // do something

There could also be situations where you need to accumulate based on the number of props rather than specifically the order of props.

[Alan Languirand]

I 100% agree with mantra here, but I'd like to add to the examples. For me, the functional style/indirection in these examples IS the value, and once you're doing things in a functional way you've already captured most of the technique's purpose. For me, whether you continue to use a declarative iterator or a for loop inside your abstraction is less important.

Here's an admittedly naive example of the price discounting that I think very clearly shows the difference between a for loop and an Array.forEach.

let prices = [5, 25, 8, 18];
let discount = 1 - 0.2;

// more imperative
for (let i = 0; i < prices.length; i++) {
    console.log(prices[i] * discount)
};

// more declarative
prices.forEach((price) => {
  console.log(price * discount);
});

Just reading these aloud to yourself shows the difference in clarity:

"For when i equals zero and i is less than the length of prices while incrementing i, console log prices at position i times discount."

vs.

"Prices: for each price, console log price times discount".

When moving from an imperative to declarative style, the code turns from near gibberish to an honestly comprehensible English sentence. I think that's a win for the whole team at every skill level, and the value this style is attempting to capture.

[edA‑qa mort‑ora‑y]

A for loop has one great advantage over every other approach: it's common. I know how a for loop works and I can rely on it working in different languages and projects. It also has a common meaning with a single concept.

While I'm strongly in favour of higher level functions, it's difficult to remember which language has which, and precisely what they do. Is there a map, fold or join? What types does it support? What if a need a small variation, like fold_right instead?

I know how to do all of this in a for loop, so it doesn't bother me to use it if I'm unaware of another solution. If somebody points out an alternate syntax, I'll gladly switch the loop.

[Dan Homola]

While I can see where you are coming from with your argument, I must ask: Isn't having to know if a particular language has 0- or 1- or arbitrarily based arrays a bit difficult as well?

I am obviously not saying it is the same amount of knowledge, I just think you have to know the language you are using either way, so why not learn its functions?

[Kasey Speakman]

I could not agree more with this article. A map, filter, or reduce tells me what is going on without looking too deeply at the code. A for loop requires a careful eye to make sure you don't miss a side effect when you refactor something. The performance difference for most cases hardly compares to the maintenance benefits.