Understanding Higher-Order Functions

Working on a few article ideas, and I'm realizing that quite a few of them are dealing with callbacks, or higher-order functions. Might make sense to talk about what they are, how they work, what they mean, and the superpowers with which they can endow us.

The What

When we talk about higher-order functions, we're simply talking about a function that either takes in another function (as a parameter), or gives back another function (as a return value), or both.

An example of a function that takes in a function might be:

document.querySelector("#container").addEventListener(
  "click",
  function(event){
    console.log(`You clicked on ${event.target.dataset.name}!`);
  }
)

We are telling the addEventListener function about two parameters: the first is a string, and the second is a function. The Event API will take that function in and, when that event occurs on that element, it will call the function for us.

An example of a function that returns a function might be this one:

const addTwoThings = function(first){
  return function(second){
    // and here, we have both first and second variables!
      return first + second;
  }
}

const add7 = addTwoThings(7);
const sayHello = addTwoThings("Hello, ");

// those are both now references to functions,
//   and those functions are scoped to unique instances of
//   addTwoThings. Let's see how we can use 'em:
console.log(sayHello("Bert")); // "Hello, Bert"
console.log(add7(28)); // 35

// Let's just complicate the crap out of things:
console.log(sayHello( add7( add7(28) ) ) ); // "Hello, 42"

So we can pass functions in, and we can pass functions back. And this can be very powerful.

The Why

Why is this significant? Consider the ES6 array methods: filter, map, forEach, reduce, some, every for example. Each of them take in a function, and the function's signature that we are passing in is very similar for each of them. The interface is pretty uniform, which makes remembering how they work a bit easier.

A common lesson in online courses might have us write a map function, simulating the way map works in our own implementation. So I don't want to give away the answer to that one, but let's try making our own implementation of filter, and see how that might work.

First, we need to understand exactly what .filter() is doing, step by step. It is looping over the array, and testing each element in that array with a given function. The function takes in up to three parameters, the element, its index, and the original array reference, and returns a truthy or falsy value. If that returned value is truthy, we persist that element. If it is not, we don't. And then we return a new array, containing just the elements that met our function.

Incidentally, a function that returns a true/false value is often referred to as a predicate function, as in "the decision was made predicated on the outcome of this thing."

We want our myFilter to work the same way, so we would be able to call it like this:

// filter out every other element:
[1,2,3,4,5,6,7,8,9].myFilter((_, index)=>index%2===0);
// [1,3,5,7,9]

So we'll start by defining a new Array method (which isn't best practice, but for the purpose of this test I'll just... do... bad things.)

Array.prototype.myFilter = function(/* some stuff here */){
  // and do some stuff here.
}

First thing to note, I used a function(){...} here, and not a ()=>{...} (a traditional function, vs a lambda or fat-arrow function). I did so, because within the function this will refer to the array itself. We need a valid this, so we can't use a lambda here.

So we know we need a starting empty array, and we need a loop:

Array.prototype.myFilter = function(/* some stuff here */){
  let stuffWeWant = [];
  for(let i=0; i<this.length; i++){
    // some kind of testing?
  }
  return stuffWeWant;
}

Now, if we refer back to how we call it, we want to be able to pass in a function. And inside the loop, we need to call that function:

Array.prototype.myFilter = function( aFilterFunction ){
  let stuffWeWant = [];
  for(let i=0; i<this.length; i++){
    if( aFilterFunction(/* but what should go here? */) ){
      // if that returns true, we'll be in here.
    }
  }
  return stuffWeWant;
}

What do we want to pass into the function? We've told the user that the function's parameters will be the current element, the current index and the array reference:

Array.prototype.myFilter = function( aFilterFunction ){
  let stuffWeWant = [];
  for(let i=0; i<this.length; i++){
    const currentElement = this[i];
    const index = i;
    const array = this;
    if( aFilterFunction(currentElement, index, array) ){
      // if that returns true, we'll be in here.
    }
  }
  return stuffWeWant;
}

We could have simply defined those in place, if we prefer:

Array.prototype.myFilter = function( aFilterFunction ){
  let stuffWeWant = [];
  for(let i=0; i<this.length; i++){
    if( aFilterFunction(this[i], i, this) ){
      // if that returns true, we'll be in here.
    }
  }
  return stuffWeWant;
}

And get the same result. So now we're testing with that aFilterFunction, so if we get a truthy value back, we will simply add this[i] to our stuffWeWant array!

Array.prototype.myFilter = function( aFilterFunction ){
  let stuffWeWant = [];
  for(let i=0; i<this.length; i++){
    if( aFilterFunction(this[i], i, this) ){
      stuffWeWant.push(this[i]);
    }
  }
  return stuffWeWant;
}

And with that, we have a working myFilter function, that looks and acts exactly the same as the ES6 filter.

Meanwhile, Back On The Farm...

Yeah, that was a bit of a sidetrack, but it does have a point - in our myFilter, we are receiving a function. And that function is being run for us, by the myFilter function. It is calling the function just as we would, passing in the expected parameters.

We are running the passed-in function by proxy. We don't need to know its name, we don't need to know where it came from, we just know it's being passed in as a parameter so we have a reference, and it will take our defined parameters or it will fail.

So it's "higher-order" - we are passing in a function and we are delegating the execution of that function to myFilter. It's a neat idea, and it seems pretty simple, but it opens up a LOT of possibilities.

Looking Ahead

This was a little bit of a sidestep from the last article, but it is a lead-in to the next one - so make sure you come back for that!