DEV Community: Homam

Time Series Visualization using SQL Window Functions

Homam — Wed, 07 Aug 2019 16:26:50 +0000

How many sales did we make this month?

One way of answering this question is to find the total sum of sales that we have made in the current calendar month:

SELECT COUNT(*) FROM sales 
WHERE DATE_TRUNC('month', sale_date) = DATE_TRUNC('month', NOW())

This query returns the number of rows in the sales table that their sale_date value is in the current calendar month.

Similarly we can count up the sales that we made in the previous calendar month:

SELECT COUNT(*) FROM sales 
WHERE DATE_TRUNC('month', sale_date) = DATE_TRUNC('month', NOW() - '1 month' :: INTERVAL)

This query returns the number of rows in the sales table that their sale_date value is within the previous calendar month.

Depending on which day of the month today is, you may notice that last month we had considerably more sales than this month. We can of course explain it because we are always somewhere the middle of the current month and this month has not yet ended.

We can avoid this problem all together if interpret the meaning of the current month as the past 30 days as opposed to a calendar month:

SELECT COUNT(*) FROM sales 
WHERE sale_date >= NOW() - '1 month' :: INTERVAL

In this post, I argue that this interpretation is more suitable for data visualization and trend analysis.

We also explore some techniques for generating sample data sets and working with SQL window functions.

Generating Data

Let's start by generating a (albeit boring) sample data set.

We use PostgreSQL's generate_series(lower, upper, step) handy function to create our samples. For example this snippet generates a time series of hourly interval between a year ago and now:

SELECT * FROM generate_series(NOW(), NOW() - '1 year' :: INTERVAL , '-1 hour') AS sale_date;

SQL Fiddle

Here we assume that we have had one sale every hour at regular intervals in the past year (this assumption is what makes this data set dull, nevertheless we will check a more advanced data generation technique later one) but it is enough for us to make a point about varying month lengths.:

WITH sales AS (
   SELECT generate_series(NOW(), NOW() - '1 year' :: INTERVAL , '-1 hour') AS sale_date
)

SELECT 
  DATE_TRUNC('month', sale_date) AS sale_month
, EXTRACT(DAY FROM DATE_TRUNC('month', sale_date + '1 month' :: INTERVAL) - DATE_TRUNC('month', sale_date)) AS month_length
, COUNT(*) 
FROM sales 
GROUP BY sale_month, month_length
ORDER BY sale_month

WITH expressions are called Common Table Expressions or CTEs. We use them to encapsulate our logic.

I avoid creating temporary tables in this post, that's where CTEs come handy.

Here WITH sales AS ... makes a virtual, sales table with one column: sale_date, this table lives in the memory during the execution of the query.

This is the result of the query (try it in SQL Fiddle)

+------------+--------------+---------+
| sale_month | month_length | count   |
|------------+--------------+---------|
| 2018-08-01 | 31           | 659     |
| 2018-09-01 | 30           | 720     |
| 2018-10-01 | 31           | 745     |
| 2018-11-01 | 30           | 720     |
| 2018-12-01 | 31           | 744     |
| 2019-01-01 | 31           | 744     |
| 2019-02-01 | 28           | 672     |
| 2019-03-01 | 30           | 743     |
| 2019-04-01 | 30           | 720     |
| 2019-05-01 | 31           | 744     |
| 2019-06-01 | 30           | 720     |
| 2019-07-01 | 31           | 744     |
| 2019-08-01 | 31           | 86      |
+------------+--------------+---------+

Ignoring first and last months, the number of sales in every month is directly related to the number of days in that month. February is usually problematic.

To improve our analysis, let's take advantage of our other interpretation of 'month' as the past 30 days.

The following query finds the average number of sales that we had in the past 30 days, for every day in the data set:

WITH sales AS (
   SELECT generate_series(NOW(), NOW() - '1 year' :: INTERVAL , '-1 hour') AS sale_date
)
, 
daily_sales AS (
  SELECT 
    DATE_TRUNC('day', sale_date) AS sale_day
  , COUNT(*) AS sales 
  FROM sales 
  GROUP BY sale_day
  ORDER BY sale_day
)
, 
daily_avgs as (
  SELECT 
    sale_day
  , SUM(sales) OVER W
  , AVG(sales) OVER W
  FROM daily_sales
  WINDOW W as (ORDER BY sale_day ROWS BETWEEN 29 PRECEDING AND CURRENT ROW)
)

SELECT * FROM daily_avgs
ORDER BY sale_day DESC

SQL Fiddle

There's a lot to unpack here, but let's first check the result:

+------------+-------+------+
| sale_day   | sum   | avg  |
|------------+-------+------|
| 2019-08-05 | 710   | 23.7 |
| 2019-08-04 | 720   | 24.0 |
| 2019-08-03 | 720   | 24.0 |
| 2019-08-02 | 720   | 24.0 |
| 2019-08-01 | 720   | 24.0 |
| 2019-07-31 | 720   | 24.0 |
| 2019-07-30 | 720   | 24.0 |
| 2019-07-29 | 720   | 24.0 |
| 2019-07-28 | 720   | 24.0 |
...

The sum in every row is 24 * 30 = 720, but the latest row. The problem is of course because today is not finished yet. We never have full 24 hours of today in the data set.

Note that we first created a daily_sales CTE:

daily_sales AS (
  SELECT 
    DATE_TRUNC('day', sale_date) AS sale_day
  , COUNT(*) AS sales 
  FROM sales 
  GROUP BY sale_day
  ORDER BY sale_day
)

Which is basically a time series of number of sales that we had every day.

We are running statistics on daily_sales CTE in the next CTE (daily_avgs):

daily_avgs as (
  SELECT 
    sale_day
  , SUM(sales) OVER W
  , AVG(sales) OVER W
  FROM daily_sales
  WINDOW W as (ORDER BY sale_day ROWS BETWEEN 29 PRECEDING AND CURRENT ROW)

WINDOW W as (ORDER BY sale_day ROWS BETWEEN 29 PRECEDING AND CURRENT ROW) creates a window (named W) that is 30-row wide. We could have named our window anything like (W30 or my_window or whatever) I choose a simple one letter W because we only have one window in this query.

Here we say for each row in our data set, select sale_day of the row, and sum all the sales that occurred between the sale_days that are 29 rows earlier from the current row and including the current row (that is 30 rows in total).

We can roughly translate this window expression into a C-like language as below:

const indexed_daily_sales = daily_sales
  .sortBy(r => r.sale_day) // ORDER BY sale_day
  .map((r, index) => ({...r, index}))
foreach(row in indexed_daily_sales) {
  yield indexed_daily_sales
    .filter(r => r.index >= row.index - 29 and r.index <= row.index) // ROWS BETWEEN 29 PRECEDING AND CURRENT ROW
    .sum(x => x.sales) // SUM(sales)

 // similarly for AVG
}

If you want to exclude today's data from the result, filter out today at the end.

And here's the result visualized in a time series:

SQL Window Function

Let's explore the concept of windows in SQL a bit more.

If I want to sum up all the integers between 1 and 10, I use this query:

SELECT SUM(id) FROM generate_series(1, 10) id

The result is simply a row with one number: 55.

But if I want to find the sum of all the integers from 1 to N for every N <= 10, I use:

SELECT id, SUM(id) OVER (ORDER BY id) FROM generate_series(1, 10) id

The result has 10 rows:

+------+-------+
| id   | sum   |
|------+-------|
| 1    | 1     | = 1
| 2    | 3     | = 1 + 2
| 3    | 6     | = 1 + 2 + 3
| 4    | 10    | = 1 + 2 + 3 + 4
| 5    | 15    | = 1 + 2 + 3 + 4 + 5
| 6    | 21    | = 1 + 2 + 3 + 4 + 5 + 6
| 7    | 28    | = 1 + 2 + 3 + 4 + 5 + 6 + 7
| 8    | 36    | = 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8
| 9    | 45    | = 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9
| 10   | 55    | = 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10
+------+-------+

The sum field in every row corresponds to the sum of the integers from 1 to id of the row.

Here OVER (ORDER BY id) creates a window of all the rows up and including the current row. It is equivalent to: OVER (ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)

We don't have to name our window if we use it only once.

SELECT 
  SUM(id) OVER (ORDER BY id)
FROM ...

is equivalent to:

SELECT 
  SUM(step_size) OVER W
FROM ...
WINDOW W AS (ORDER BY id)

Let's modify this query to return the sum of three integers around each number, for example the row for 5 should return: 4 + 5 + 6 = 15

SELECT 
  id
, SUM(id) OVER (ORDER BY id ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) 
FROM generate_series(1,10) id

SQL Fiddle

+------+-------+
| id   | sum   |
|------+-------|
| 1    | 3     | =   + 1 + 2 = 3
| 2    | 6     | = 1 + 2 + 3 = 6
| 3    | 9     | = 2 + 3 + 4 = 9
| 4    | 12    | = 3 + 4 + 5 = 12
| 5    | 15    | = 4 + 5 + 6 = 15
| 6    | 18    | = 5 + 6 + 7 = 18
| 7    | 21    | = 6 + 7 + 8 = 21
| 8    | 24    | = 7 + 8 + 9 = 24
| 9    | 27    | = 8 + 9 + 10 = 27
| 10   | 19    | = 9 + 10 +   = 19
+------+-------+

The size of the window is 3 with the exception of the first and last items in the result.

This is usually the case that the size of the window in the beginning and/or the end of the result is smaller than in the middle.

Aggregate functions that operate on windows are aware of this. For example check the average of the three neighboring numbers (that must be the middle one):

SELECT 
  id
, SUM(id) OVER W
, AVG(id) OVER W
FROM generate_series(1, 10) id 
WINDOW W AS (ORDER BY id ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING)

SQL Fiddle

+------+-------+-----+
| id   | sum   | avg |
|------+-------+-----|
| 1    | 3     | 1.5 | = (    1 + 2) / 2 = 1.5 (note division by 2)
| 2    | 6     | 2.0 | = (1 + 2 + 3) / 3 = 2.0 (note division by 3)
| 3    | 9     | 3.0 | = (2 + 3 + 4) / 3 = 3.0
| 4    | 12    | 4.0 | = (3 + 4 + 5) / 3 = 4.0
| 5    | 15    | 5.0 | = (4 + 5 + 6) / 3 = 5.0
| 6    | 18    | 6.0 | = (5 + 6 + 7) / 3 = 6.0
| 7    | 21    | 7.0 | = (6 + 7 + 8) / 3 = 7.0
| 8    | 24    | 8.0 | = (7 + 8 + 9) / 3 = 8.0
| 9    | 27    | 9.0 | = (8 + 9 + 10) / 3 = 9.0
| 10   | 19    | 9.5 | = (9 + 10    ) / 2 = 9.5 (note division by 2)
+------+-------+-----+

Depending on your analysis you might need to take this fact into consideration.

For us it means that the average of sales in our time series in the beginning has less data points, hence AVG(sales) OVER W would be more noisy at the left of our charts.

One easy workaround is to ignore the data points in the beginning by offsetting our result. Use OFFSET 29 at the end the query (SQL Fiddle).

Random Walk

We need a more realistic data set in order to put everything that we discussed above into use. Our sales data set has been very boring so far because we used a uniform distribution (one sale every hour) to create our sample data set.

Here we explore a method for generating a more realistic sales data set.

Random Walk is exactly what you think it is. In two dimension, you can think of a small turtle on a surface that chooses the direction of her next step completely by random.

In one dimension we can only move up or down.

SELECT step_id, (FLOOR((RANDOM() * 3) - 1)) AS step_size
FROM generate_series(1,6000) step_id

+-----------+-------------+
| step_id   | step_size   |
|-----------+-------------|
| 1         | 1.0         |
| 2         | 0.0         |
| 3         | 1.0         |
| 4         | 1.0         |
| 5         | 0.0         |
| 6         | -1.0        |
...

This snippet generates a uniform distribution of -1s, 0s and 1s step_sizes.

These numbers model the movement of our cursor down, up or not at all at each step.

The sum of all the previous step_sizes at each step determines the total distance that we have travelled form the origin.

SELECT 
  SUM(step_size) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS pos
  FROM (
    SELECT step_id, floor((random() * 3) - 1) AS step_size
    FROM generate_series(1,6000) step_id
  ) _a

+-----------+-------------+-------+
| step_id   | step_size   | pos   |
|-----------+-------------|-------|
| 1         | 1.0         | 1.0   |
| 2         | 0.0         | 1.0   | = 1 + 0
| 3         | 1.0         | 2.0   | = 1 + 0 + 1
| 4         | 1.0         | 3.0   | = 1 + 0 + 1 + 1
| 5         | 0.0         | 3.0   | = 1 + 0 + 1 + 1 + 0
| 6         | -1.0        | 2.0   | = 1 + 0 + 1 + 1 + 0 - 1
...

Use UNION if you want to specify a starting point:

SELECT 0 as step_id, 600 as step_size
UNION
SELECT step_id, floor((random() * 3) - 1) AS step_size
FROM generate_series(1,6000) step_id
ORDER BY step_id

We use random walk to generate more realistic-looking data set. The idea here is that the number of sales every day is not completely random, but it is actually close to the sales that we had on the previous day plus or minus some random value which we call noise.

WITH noise AS (
  SELECT 
    step_id
  , date_trunc('day', NOW()) - (step_id || ' day') :: INTERVAL AS sale_day
  , SUM(step_size) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS pos
    FROM (
      SELECT step_id, (FLOOR((RANDOM() * 3) - 1)) * FLOOR(RANDOM() * 100) AS step_size
      FROM generate_series(1,1000) step_id
    ) _a
)
,
daily_sales AS (
  SELECT
    sale_day
  , (CASE WHEN EXTRACT(DAY FROM sale_day) < 8 
      THEN FLOOR(RANDOM() * 200) 
      ELSE 0 END
    ) + (SELECT ABS(MIN(pos)) FROM noise) + pos AS sales

  FROM noise  
  ORDER BY step_id DESC
)

SELECT 
  sale_day
, sales
, AVG(sales) OVER (ORDER BY sale_day ROWS BETWEEN 29 PRECEDING AND CURRENT ROW) AS avg_daily_sales
FROM daily_sales

noise is a series of random numbers between -99 and 99:

SELECT * FROM noise

+-----------+------------+--------+
| step_id   | sale_day   | pos    |
|-----------+------------+--------|
| 1         | 2019-08-04 | 48.0   |
| 2         | 2019-08-03 | 48.0   |
| 3         | 2019-08-02 | 48.0   |
| 4         | 2019-08-01 | 72.0   |
| 5         | 2019-07-31 | 72.0   |
| 6         | 2019-07-30 | 159.0  |
| 7         | 2019-07-29 | 252.0  |
...

In this model, we expect our sales to vary ±99 from the previous day by random.

daily_sales adjusts the noise series by first making sure that all data points are above zero: + (SELECT ABS(MIN(pos)) FROM noise). We also adds some seasonality to the series (we assume our sales increase in the first week of the month by maximum of 200 sales per day):

(CASE WHEN EXTRACT(DAY FROM sale_day) < 8 
   THEN FLOOR(RANDOM() * 200) 
   ELSE 0 END
)

Let's check the final result:

+------------+---------+-------------------+
| sale_day   | sales   | avg_daily_sales   |
|------------+---------+-------------------|
| 2016-11-08 | 1074.0  | 1074.0            |
| 2016-11-09 | 1068.0  | 1071.0            |
| 2016-11-10 | 1118.0  | 1086.66666666667  |
| 2016-11-11 | 1118.0  | 1094.5            |
| 2016-11-12 | 1112.0  | 1098.0            |
| 2016-11-13 | 1177.0  | 1111.16666666667  |
| 2016-11-14 | 1145.0  | 1116.0            |
| 2016-11-15 | 1117.0  | 1116.125          |
...

A sanity check:

avg_daily_sales at 2016-11-11 = (1074 + 1068 + 1118 + 1118) / 4 = 1,094.5

The maximum number of rows that we have in any window is 30. In the beginning we of course we have less than 30 rows in the window, because there is no row before 2016-11-08.

Compare our latest chart with our original attempt with naïve DATE_TRUNC.

And if you prefer monthly statistics, filter out the rows at the end of the query:

...
daily_avgs AS (
  SELECT 
    sale_day
  , sales
  , SUM(sales) OVER (ORDER BY sale_day ROWS BETWEEN 29 PRECEDING AND 
  CURRENT ROW) AS avg_monthly_sales
  FROM daily_sales
)

SELECT sale_day, avg_monthly_sales 
FROM daily_avgs
WHERE EXTRACT(DAY FROM sale_day) = EXTRACT(DAY FROM NOW() - '1 day' :: INTERVAL)
ORDER BY sale_day DESC

SQL Fiddle

Each bar represents the sum of sales during 30-days before and including the date that is associated with the bar.

Missing Values

Often we need to deal with gaps in our time series. In our example, rows in daily_sales would be missing if there is no sales happened on that day.

So far in our generated data set we avoided zero and negative sales by padding the data with the absolute value of the minimum distance that was generated by our random walk.

In order to produce some gaps, let's first decrease this padding to half + (SELECT ABS(ROUND(MIN(pos)/2)) FROM noise) and then filter out all the rows with zero or negative sales: SELECT * FROM daily_sales_1 WHERE sales > 0:

WITH noise AS (
  SELECT 
    step_id
  , DATE_TRUNC('day', NOW()) - (step_id || ' day') :: INTERVAL as sale_day
  , SUM(step_size) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS pos
    FROM (
      SELECT 0 as step_id, 0 as step_size
      UNION
      SELECT step_id, (floor((random() * 3) - 1)) * floor(random() * 100) AS step_size
      FROM generate_series(1,1000) step_id
      ORDER BY step_id
    ) _a
)
,
daily_sales_1 AS (
  SELECT
    sale_day
  , (CASE WHEN EXTRACT(DAY FROM sale_day) < 8 
      THEN floor(random() * 200) 
      ELSE 0 END
    ) + (SELECT ABS(ROUND(MIN(pos)/2)) FROM noise) + pos AS sales

  FROM noise  
  ORDER BY step_id DESC
)
,
daily_sales AS (
  SELECT * FROM daily_sales_1 where sales > 0
)
,
calendar AS (
  SELECT generate_series(
      (SELECT min(sale_day) from daily_sales_1)
    , (SELECT max(sale_day) from daily_sales_1)
    , '1 day' :: INTERVAL
  ) as sale_day
)

SELECT 
  calendar.sale_day
, COALESCE(sales, 0) AS sales
, AVG(COALESCE(sales, 0)) OVER (ORDER BY calendar.sale_day ROWS BETWEEN 29 PRECEDING AND CURRENT ROW) AS avg_daily_sales
FROM calendar
LEFT JOIN daily_sales ON calendar.sale_day = daily_sales.sale_day

SQL Fiddle

This is the result:

calendar CTE generates a series of dates

calendar AS (
  SELECT generate_series(
      (SELECT min(sale_day) from daily_sales_1)
    , (SELECT max(sale_day) from daily_sales_1)
    , '1 day' :: INTERVAL
  ) as sale_day
)

We select sale_days from calendar and left join it with daily_sales table in the final step.

sales are null for the rows that are missing in the daily_sales table because of left join. That's why we use COALESCE(sales, 0) to cast nulls to 0 for the missing data points.

In general null does not mean 0. So be careful. But we can cast nulls or missing data points to 0 when we are dealing with gaps in a time series.

Composability: from Callbacks to Categories in ES6

Homam — Mon, 20 May 2019 15:02:58 +0000

Promises are a well known solution to the Callback hell problem that arises in asynchronous JavaScript programs.

Borrowing some ideas from Functional languages, I am exploring a different approach to address callback hell in this post. This solution will be more general than Promises, in fact we will take advantage of these ideas to make Promises even more composable.

I use a notation similar to Haskell’s. But in many ways I will divert from rigid Haskell notation everywhere that I think it helps.

You only need to be familiar with Callbacks, Promises, and ES6 anonymous function (lambda) syntax to follow this post. We will be playing with some ideas from Functional Programming (FP).

TOC:

Callbacks
Composable Callback class
Composable Callback class and Promise class are Monads
Monads
Categories
Func Category
Kleisli Category

Callbacks

Many programming languages utilize callbacks for continuation. When we encounter:

    db.getSomething(callback)

We know that db.getSomething is a void function, it executes some code (potentially asynchronously) and passes the result of operation to the callback function to handle it.

Callbacks in JavaScript are more powerful than just continuation. We can model a function that returns more than one result using callbacks:

function next2(x, callback) {
  callback(x + 1, x + 2)
}

next2(10, (eleven, twelve) => …)

In fact this is how callbacks are used for propagating errors. By convention the first argument to a callback is the error (if any) that was produced by the operation:

function sqrt(x, callback) { 
  if(x < 0) 
    callback(Error('Sqrt of negative value', null))
  else 
    callback(null, Math.sqrt(x))
}

If the operation produces any error, we always ignore the second argument (whatever result it might had produced).

Callback hell happens when we want to pass the result of the first async operation to the second async function and to the third and so on:

function myLongOperation(userId, callback) {
  db.getUser(userId, (error, user) => {
    if(!!error)
      return callback(error, null)
    else
      api.generateMessage(user, (error, message) => { 
          if(!!error)
            return callback(error, null) 
          else
            client.sendMessage(message, callback)
      })
  })
}

Here we are passing userId to getUser in order to get the user asynchronously then we’re passing the user to generateMessage to … You know instead of narrating it in words let’s use some notation to describe this process:

userId → (getUser ⋙ generateMessage ⋙ sendMessage)

The above notation perfectly describes what our myLongOperation function does. Error handling at every step is clearly redundant. Promise fans know that this notation is very similar to (but not exactly the same as) what we do with Promises:

    getUser(userId).then(generateMessage).then(sendMessage)

Promise.then takes care of error handling and chaining.

But our goal is to come up with a construct that is more general than Promises.

In our notation ⋙ is a way of composing (piping async functions). We will discuss it later.

x → y denote a function from x to y. For example:

const plus1 = x => x + 1
//        Number → Number

myLongOperation is a function from userId to a series of async operations, hence:

    userId → ( … ⋙ … ⋙ … )

Haskellers know that this is not a proper type definition. But for our purpose this notation perfectly describes myLongOperation function.

Composable Callback

Promises are not the only solution to the callback hell problem. Promises provide more features than composability (for example they have an internal state which remembers if they’ve been resolved or not plus some other kinks).

Let’s define a bare minimum solution to the callback hell problem by implementing a “composable Callback” class:


class Callback {
  constructor(f) {

    // this.run = f
    this.run = callback => {
      try {
        f(callback)
      } catch (ex) {
        callback(ex, null)
      }
    }

    // this.map = ...
    // this.bind = ...

    // this :: Callback x
    // x -> (y || Callback y) -> Callback y
    this.then = g => new Callback(callback => {
      this.run((error, ...result) => {
        if(!!error) {
          callback(error, null)
        } else {
          try {
            const y = g(...result)
            if (y instanceof Callback) {
              y.run(callback)
            } else {
              callback(null, y)
            }
          } catch(ex) {
            callback(ex, null) 
          }
        }
      })
    })

    this.bindTo = g => this.bind(Callback.from(g))
  }
}

// x -> Callback x
Callback.pure = x => new Callback(cb => cb(null, x))

Callback.resolve = Callback.pure

// Callback.from casts f into a Callback instance, where
// f is a function that takes x and a callback function
Callback.from = f => (...x) => new Callback(cb => f(...x, cb))

Check out the full code here.

Callback class provides this interface:

constructor takes an async function (f which will produce either an error or a value x)
run instance function: receives a callback function and feed it to the f
map instance function analogous to Array.map, transforms the x (the result of f)
bind instance function is similar to Promise.then, it is used for chaining Callback instances
then instance function corresponds to Promise.then; it’s a combination of map and bind functions.
bindTo instance function is a utility for chaining Callback instances to normal async functions
pure (alias resolve) static function is similar to Promise.resolve, it creates an instance of Callback.
from static function casts an async function to an instance of Callback.

It’s not an accident that Callback interface resembles the interface of Promise. pure is an alias for resolve. If you’ve ever used Promise.resolve() you know what Callback.pure does. I think pure is a better name for our Callback class. Similarly Callback.then is analogous to Promise.then. I consciously avoid Callback.map and Callback. bind.functions in this post, because *Callback.then *is sufficient as it both maps and binds.

We start with Callback.pure. It puts a value into a new Callback instance:

    Callback.pure(64).run((error, result) => console.log(result))

Will log 64 in the Console.

This is how we can compose Callback.pure(64).with our sqrt function:

  Callback.pure(64)
    .bindTo(sqrt)
  .run((error, result) => console.log(error || result))

Under the hood, bindTo casts sqrt to an instance of Callback. The above snippet is equivalent to the followings:

Callback.pure(64)
  .then(Callback.from(sqrt))
.run((error, result) => console.log(error || result))

Callback.pure(64)
  .then(x => new Callback(cb => sqrt(x, cb)))
.run((error, result) => console.log(error || result))

Using Callback class our myLongOperation function can be written more concisely as:

    // userId → (getUser ⋙ genMessage ⋙ sendMessage)

    const myLongOperation = (userId, callback) => 
      Callback.pure(userId)
        .bindTo(getUser).bindTo(genMesssage).bindTo(sendMessage)
      .run(callback)

Notice how closely this implementation matches the notation.

.bindTo(getUser).bindTo(genMesssage).bindTo(sendMessage).is denoted by (getUser ⋙ genMessage ⋙ sendMessage)
But Callback.pure(userId) seems unnecessary. (userId → (…) is the denotation of the whole myLongOperation function.) We will back to this point later.

Our changes to myLongOperation function are not visible to the user of this function. myLongOperation is still an async function that takes a userId and a callback.

We can always use bindTo utility to chain Callback instances to async functions. For example let’s assume we have another async function like getUserId(userName, callback) which we want to pipe its result into myLongOperation:

const messageUser = (userName, callback) =>
  Callback.pure(userName)
  .bindTo(getUserId)
  .bindTo(myLongOperation)
  .run(callback)

Notice that now run() is being called twice: once inside myLongOperation and the second time inside messageUser. There’s a catch here. Nothing really happens unless we call run().

const proc = Callback.pure(5)
  .then(x => new Callback(cb => {
    console.log(`binding ${x} to x + 1`)
    setTimeout(() => cb(null, x + 1), 100)
  }))

console.log() in the third line only gets called after proc.run(). Try it here:

proc (as an instance of Callback class) represents the instructions to an async operation that JavaScript only executes after run() is called. This is very different from Promises:

const prom = new Promise(resolve => {
  console.log('Promise executes immediately')
  resolve()
})

When you run this snippet, ‘Promise executes immediately’ is logged immediately, even if you never use the prom or prom.then(x => …).

So let’s change our myLongOperation function to return an instance of Callback (we can save one call to run() this way):

// userId → (getUser ⋙ genMessage ⋙ sendMessage)

const myLongOperation = userId => 
  Callback.pure(userId)
  .bindTo(getUser).bindTo(genMesssage).bindTo(sendMessage)

Now this definition matches the notation even better since we eliminated the callback function completely.

In the same spirit, we update our messageUser function:

// userName → (getUserId ⋙ myLongOperation)

const messageUser = userName =>
  Callback.pure(userName).bindTo(getUserId).then(myLongOperation)

We changed the last bindTo().to then(), because now our updated myLongOperation is a function that returns an instance of Callback (remember originally before the change, it was a void function that was taking a callback in its second argument).

This is how we can use messageUser :

messageUser(userName).run((error, result) => ...)

We call run() only at the end of the operation. run() executes the operation and returns the result in its callback argument.

We achieved composability and avoided callback hell without resorting to Promises. Check out the full example here:

Functional programmers know that there must be some eta reduction to convert

myLongOperation(userId) = userId → (getUser ⋙ genMessage ⋙ sendMessage) to
myLongOperation = getUser ⋙ genMessage ⋙ sendMessage

In the rest of this post we build some constructs that ultimately enable us to eliminate this redundant parameter.

Callback and Promise are Monads

Our Callback class and the standard Promise class have a lot in common. We call these constructs monad, by which I mean they have a bind (then) function that chains an instance of Callback (or Promise) to a function that returns another instance of Callback (or Promise).

    const proc = Callback.pure(10)
    proc.bind(x => new Callback(…))

We use this notation to describe proc as an instance of Callback monad:

proc :: Callback x

proc.bind :: (x → Callback y) → Callback y

We might read the notation like this:

proc is a Callback of x
proc.bind is a (higher order) function that takes a function from x to Callback of y and produces a Callback of y.

For example Callback.pure(10) can be bound to a function that takes a Number and returns a new Callback:

Callback.pure(10)
  .bind(x => new Callback(cb => cb(null, x + 1)))

(remember that resolve() is an alias for pure() and then() has a similar functionality to bind())

Promise class also forms a monad:

Promise.resolve(10)
  .then(x => new Promise(resolve => resolve(x + 1)))

These two expressions look vary similar and that’s indeed the power of monads. Monads provide an abstraction that is useful in many different programs. In our notation the above expressions can be written as:

Monad 10 ≫= (x → Monad (x + 1)) = Monad 11

For Promise Monad:

    Monad 10           ::  Promise.resolve(10)
    ≫=                 ::  .then(…)    
    x → Monad (x + 1)  ::  x => new Promise(resolve => resolve(x + 1))

For Callback Monad:

    Monad 10           ::  Callback.resolve(10) // = Callback.pure(10)
    ≫=                 ::  .then(…)             // = Callback.bind(…)
    x → Monad (x + 1)  ::  x => new Callback(cb => cb(x + 1))

Monads encapsulate a value that can only be retrieved by executing the monad. For Promise monad we retrieve the result of the computation (11) by calling then() function and for our Callback monad we retrieve the result by run().

Monads have this interesting feature that they can be used even if their encapsulated value is not computed yet. We are able to call then() on a Promise and chain it with a function or another Promise even if it’s not completed and the value that it encapsulates is not computed yet. This fact is even more pronounced for our Callback monad. We had seen earlier that Callback doesn’t even bother start computing its result before we call run() (Repl.it demo).

More generally both computations might be denoted as:

Monad x ≫= (x → Monad y) = Monad y

x and y can be of any type. Here they’re Numbers, but they can be String, Boolean, JSON objects, … or even functions or other monads!

What is a Monad?

For our purpose any class that has these two features is a Monad:

The class must have a way of encapsulating a value (using a static pure() or resolve() function)
It must provide a way to bind itself with a function that returns another instance of it (using bind() or then())

Monads add extra structure to the value that they’re encapsulating. Different types of Monads provide different structures. The implementation of the pure function is the place to look for these structures.

For Promise:

    Promise.resolve = x => new Promise(res => res(x))

For Callback:

    Callback.pure = x => new Callback(cb => cb(null, x))

For Array:

    Array.of = x => [x]

For Reader:

    Reader.pure = x => new Reader(env => x)

Click on the links to see the definitions and play with these monads. In this post we only study Promise and Callback.

We can indeed define a monad that has almost no extra structure. This minimum monad is called Identity Monad:

    Identity.pure = x => new Identity(x)

How Identity is useful can be the subject of another post.

In conclusion

Composability is the essence of any solution to the callback hell problem.

We implemented the Callback class as an alternative solution and we discovered that our Callback class has actually something in common with Promises. They both provide a then().function that binds them to functions that return a new instance of Promise or Callback. We named these constructs monad.

Monad x ≫= (x → Monad y) = Monad y

(Monad x) .then(x => Monad y) = Monad y

    Callback.pure(10).then(x => new Callback(cb => cb(null, x + 1)))

    Callback.resolve(10).then(x => new Promise(res => res(x + 1))

Monads deal with the result of the operation. Promise.resolve(10).will result in 10 (wrapped in a Promise).

But Categories deal with both input and output of the operation (we denoted them as Cat (x ↣ y)). Func is the simplest category (which corresponds to normal functions).

Categories provide a pipe() function which is akin to Monad.then(). then() receives a function in its argument, but in contrast pipe() takes another instance of Category:

Cat (x ↣ y) ⋙ Cat (y ↣ z) = Cat (x ↣ z)

Cat (x ↣ y) .pipe( Cat (y ↣ z) ) = Cat (x ↣ z)

    Func(x => x + 1).pipe(new Func(x => x * 3)).run(10)

“Functions that return a monad” form a category (which is called Kleisli category).

Using Kleisli category we’ve been able to reduce the noise and redundancy in our async program. Generally in functional programming, instead of dealing with how the program works, our goal is to describe what the program does. Abstractions (like categories or monads) will take care of the details.

Demo Links:

Whether you liked this post or if I lost you earlier somewhere in the text, you might want to check Mostly adequate guide to FP (in javascript) open source book.

Although we didn’t need to use any library, but for me Ramda is the standard bearer of JavaScript FP libraries.

Reduce

Homam — Sat, 18 May 2019 14:35:55 +0000

Recently I needed to parse a semi-structured long text document and convert it into a data structure. As a lazy programmer I did not want to copy and paste the text a thousand times by hand. My solution was quite simple: read the document line-by-line, keep track of each line that I have not successfully parsed yet in an array, and try to parse the array at the end of each iteration, and empty the array every time parsing succeeds. And repeat till EOF.

This is how parsers work generally. My little hack was easy to do only because I contained my logic inside reduce function.

This experience reminded me that I have to write about the power and utility of reduce function.

Read this post if you are getting on board of functional programming train.

Summing up Numbers

Let’s create a function to sum up the numbers inside an array. (you can try these snippets in your browser console)

let oneToTen = [1,2,3,4,5,6,7,8,9,10]

let sum = function(arr) {
  let acc = 0 // the accumulated sum
  for(var i = 0; i < arr.length; i++) {
    let a = arr[i] // ith item in the array
    acc += a
  }
  return acc
}

sum(oneToTen)

Simple, yes! But like most things in programming, there is a nicer way for doing this:

oneToTen.reduce((acc, a) => acc + a, 0)

reduce function is very powerful and it indeed looks magical if it is the first time you're seeing it.

Reduce is known by many other names: Aggregate in .NET Linq, fold in Scala, foldl in Haskell, Erlang, accumulate in C++. Check the full list at Foldl Wikipedia page.

In JavaScript Array.prototype.reduce receives two arguments. The first one is a function and the second argument is the initial value (or the seed) of the reduction process (here it is 0).

Here’s a more verbose version of the above code:

oneToTen.reduce(function(acc, a) {
  return acc + a;
}, 0)

You can compare acc and a variables in this version with the similarly named variables in the loop version earlier.

So how does it work?

The function inside reduce (which we call reduction or aggregation function) gets called multiple times, exactly once per item in the array. This is very similar to the operation inside the body of for. At each step the reduction function returns the current accumulated value by summing up the previous accumulated value (acc) and the current item in the array a.

Let’s add some logs to see the result at each step:

let oneToTen = [1,2,3,4,5,6,7,8,9,10]

oneToTen.reduce((acc, a) =>  {
  console.log(`acc = ${acc}, a = ${a}`)
  return acc + a
}, 0)

reduce is an abstraction over looping operations. We can convert any operation on arrays to reduce.

Probably counting the number of items in an array is one of the simplest and the most-common things that we do with arrays. JavaScript array natively support Array.prototype.length. But since it is an operation on arrays we can also use reduce to count the size of our array:

['a', 'b', 'c', 'd', 'e'].reduce((acc, _a) => acc + 1, 0)

The length of an array does not depend on the actual value of each item in the array. That’s why we do not use the parameter _a in the above code.

Here the seed value of reduce is 0; reduce returns the seed value if the array that it is operating on is empty.

Of course you should continue using Array.prototype.length and most of the native array functions in your production code. Or use a library like Ramda. Many examples here are for demonstrating the generality and power of reduce function.

So far the reduce operations that we have seen produced a numeric result. Now let’s check string concatenation.

Standard Array.prototype.join concatenates an array of strings, using its argument and returns the concatenated string. We can also define it using reduce:

['reduce', 'is', 'cool'].reduce((acc, a) => acc + ' ' + a, '')

// " reduce is cool"

Notice the extra space at the beginning of the string.

We have the extra space because we started reducing with an empty string. The value of the first acc is the initial empty string. Then in the reduction function we added a space and then the word "reduce":

['reduce', 'is', 'cool'].reduce((acc, a) => {
  console.log(`acc = '${acc}', a = '${a}'`)
  return acc + ' ' + a
}, '')

// " reduce is cool"

We can solve this easily by not passing any initial value to the reduce:

['reduce', 'is', 'cool'].reduce((acc, a) => acc + ' ' + a)

// "reduce is cool"

But I argue that this implementation is also problematic because it fails for an empty array.

We can deal with the unnecessary space using an if expression. We check if acc is equal to the empty string (that means we are in the first iteration):

['reduce', 'is', 'cool']
  .reduce((acc, a) => acc === '' ? a : acc + ' ' + a, '')

If you are not used to if-then-else expressions in JavaScript, the code above is equivalent to this:

['reduce', 'is', 'cool'].reduce((acc, a) => {
  if(acc === '') {
    return a;
  } else {
    return acc + ' ' + a;
  }
}, '')

I prefer if-then-else expressions here because they ensure that I would not be forgetting the else clause. Every if in this tutorial will need an else.

I also always pass a seed value to reduce functions.

We can create the join function:

function join(c, arr) {
  return arr.reduce((acc, a) => {
    if(acc === '') {
      return a;
    } else {
      return acc + c + a;
    } 
  }, '')
}

join('*', ['reduce', 'is', 'cool'])

Or more concisely:

let join = (c, arr) => arr.reduce(
   (acc, a) => (acc === '' ? '' : acc + c) + a
 , '')

Array functions

Let’s explore defining some basic array operations with reduce starting with map:

let map = (f, arr) => arr.reduce((acc, a) => { 
  const mappedA = f(a) // apply f to the current item in the array
  return acc.concat([mappedA])
},[]) 

// the initial seed is an empty array, this is the result of reduction if the input array is empty

map(x => x * 2, oneToTen)

// [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]

And filter:

let filter = (f, arr) => arr.reduce((acc, a) => {
  const include = f(a)
  return include ? acc.concat([a]) : acc
}, [])

filter(
    x => x.startsWith('A')
  , ['Orange', 'Apple', 'Pearl', 'Avocado', 'Pomegranate']
)

// ["Apple", "Avocado"]

We can see the pattern now.

identity just creates array with the exact same elements of the array that it receives, without doing any other operation:

let identity = arr => arr.reduce((acc, a) => acc.concat([a]), [])

identity(['a', 'b', 'c', 'd', 'e', 'f'])

// ['a', 'b', 'c', 'd', 'e', 'f']

Now let’s define reverse function using reduce. Check how its definition is different from identity:

let reverse = arr => arr.reduce((acc, a) => [a].concat(acc), [])

reverse(['a', 'b', 'c', 'd', 'e', 'f'])

// ["f", "e", "d", "c", "b", "a"]

take returns the first N items in the array as a new array:

let take = (howMany, arr) => arr.reduce(
   (acc, a) => acc.length === howMany ? acc : acc.concat([a])
 , []
)

take(3, ['a', 'b', 'c', 'd'])

// ['a', 'b', 'c']

head is a function that returns the first item in an array (similar to arr[0]). And last returns its last item of an array:

let head = arr => arr.reduce((acc, *_a*) => acc)

let last = arr => arr.reduce((*_acc*, a) => a)

head(['a', 'b', 'c', 'd']) // "a"

last(['a', 'b', 'c', 'd']) // "d"

And a bit of sanity check:

head(reverse(['a', 'b', 'c', 'd'])) === last(['a', 'b', 'c', 'd'])

// true

drop function removes the first N item in the array and returns the rest. We can define drop using take and reverse:

let drop = (howMany, arr) => {
  const reversedArr = reverse(arr)
  const topN = take(arr.length - howMany, reversedArr)
  return reverse(topN)
}

drop(3, ['a','b','c','d','e']) // ["d", "e"]

This definition is not very efficient, because we iterate through the array three times: (reverse, take, reverse).

We can simply count the items in the array and exclude the items which their index is less than N:

drop = (howMany, arr) => arr.reduce(
  (acc, a) => {
    // current index in array
    const currentIndex = acc.currentIndex + 1 

    const result = currentIndex >= howMany 
      ? acc.result.concat([a])
      : acc.result
    return {currentIndex, result}
  }
  , {currentIndex: -1, result: []} //the initial seed of aggregation
)
.result

drop(3, ['a','b','c','d','e']) // ["d", "e"]

Remember that JavaScript array index starts from 0.

Here the initial (seed) value of the reduction process is not a simple array or an empty string or number 0, but it is an object with two fields:

{currentIndex: -1, result: []}

Note that the aggregation (reduction) function returns a similar object.

currentIndex keeps the count of the items in the array.

result keeps track of the result of our reduction process.

At the end of the reduction currentIndex is equal to the length of the array minus one and result contains the final result of the drop operation.

This implementation iterates through the array only once.

We can use de-structuring to make this function shorter and depending on your taste more or less readable:

drop = (howMany, arr) => arr.reduce(
 ({ currentIndex, result }, a) => 
  currentIndex + 1 >= howMany 
   ? { currentIndex: currentIndex + 1, result: result.concat([a]) }
   : { currentIndex: currentIndex + 1, result: result }
 , { currentIndex: -1, result: [] }
).result

The seed value

The idea of reducing using complex objects as seed values is very powerful. For example we can calculate the sum and the product of the items in an array simultaneously by going through the array only once:

[1,2,3,4,5,6,7,8,9,10].reduce((acc, a) => {
  return {
    sum: acc.sum + a,
    product: acc.product * a
  }
}, {sum: 0, product: 1})

Here the choice of {sum: 0, product: 1} for initial seed is not trivial. 0 is the neutral element of sum operation and 1 is the neutral element of product.

The result of reducing an empty array is equal to the seed value of reduction.

[].reduce((acc, a) => {
  return {
    sum: acc.sum + a,
    product: acc.product * a
  }
}, {sum: 0, product: 1})

Let’s study the choice of seed value for sum and product functions in more details:

let sum     = arr => arr.reduce((acc, a) => acc + a, 0)
let product = arr => arr.reduce((acc, a) => acc * a, 1)

The idea is that the seed value i must be chosen so that for our reduction function f and for every a that is an element of our array:

f(a, i) == f(i, a) == a

Seed value is the neutral element of the reduction function.

For example for product function, where f = (acc, a) => acc * a, the seed value must be 1 so:

1 * a == a * 1 == a

Pipe

pipe function receives a list of functions and applies them one after another to its input. By utilizing pipe we can avoid defining temporary local variables for one-time use:

function addTwoPlusOneOverSeven(a) {
  const b = 2 * a
  const c = b + 1
  const d = c / 7
  return c
}

// will become

function addTwoPlusOneOverSeven(a) {
  return pipe([
      x => x * 2
    , x => x + 1
    , x => x / 7
  ])(a)
}

In other words, more generally pipe creates a new function by composing the functions in its input array:

const addTwoPlusOneOverSeven = pipe([
    x => x * 2
  , x => x + 1
  , x => x / 7
])

Defining pipe using reduce is quite easy:

let pipe = arr => arr.reduce(
    (acc, next) => x => next(acc(x))
  , x => x
)

Note the seed value x => x. This is identity function that is the neutral element of composition. It is akin to 0 for sum or 1 for product.

Here our reduction function is: f = (acc, next) => x => next(acc(x))

Note that acc and next are both functions and f compose them together one after the other.

id = x => x is the neutral element because for every function h that we can think of:

f(id, h) == f(h, id) == h

pipe([
    x => x * 2
  , x => x + 1
  , x => x / 7
  , x => `((10 * 2) + 1) / 7 = ${x}`
])(10)

// "((10 * 2) + 1) / 7 = 3"

Moving Average

Lastly I want to show how we can implement efficient moving average, and some basic statistics using reduce:

let movingAverage = (size, arr) => arr.reduce((acc, a) => {
  let currentWindow = acc.currentWindow.concat([a])
  currentWindow = currentWindow.length > size
    ? drop(1, currentWindow)
    : currentWindow
  return {
    currentWindow,
    result: currentWindow.length == size
      ? acc.result.concat([sum(currentWindow) / size])
      : acc.result
  }
}, {currentWindow: [], result: []})

let {result} = movingAverage(3, [2,5,6,4,1])
let expected = [sum([2,5,6])/3, sum([5,6,4])/3, sum([6,4,1])/3]

{result, expected}

// result = [4.333333333333333, 5, 3.6666666666666665]

Basic descriptive statistics in one-go:

let stats = data => data.reduce( 
    ({count, sum, mean, vari, min, max}, x) => {
      const k = 1 / (count + 1)
      const mean_ = mean + k * (x - mean)
      const ssr_ = (count - 1) * vari + k * 
        count * (x - mean) * (x - mean)
      return {
          count: count + 1
        , sum: sum + x
        , mean: mean_
        , vari: ssr_ / Math.max(1, count)
        , min: isNaN(min) || x < min ? x : min
        , max: isNaN(max) || x > max ? x : max
      }
    }
  , {count: 0, sum: 0, mean: 0, vari: 0, min: NaN, max: NaN}
)

stats([3,4,2,2,4,3,2,2,4,5])

/*
{
  count: 10, 
  sum: 31, 
  mean: 3.1, 
  vari: 1.2111111111111112, 
  min: 2, 
  max: 5
}
*/

Here I am using Welford variance algorithm to compute the variance. This algorithm also works with streams.

We need to sort our array to calculate other statistics like median or quartiles.

Defining Reduce

Now, to learn how reduce works internally, let’s define our own version reduce function.

reduce is an abstraction over recursion. At each iteration we produce the result by calling the reduction function f over the current element in the array and the result of of the latest iteration of the reduction.

let reduce = (f, seed) => arr => {
  if(arr.length === 0){
    // result of reducing an empty array is the initial seed
    // the array is empty if it is the last iteration
    return seed 
  } else {
    const [a, ...tail] = arr
    const result = f(seed, a)

    // result is the initial seed of the next iteration
    return reduce(f, result)(tail)
  }
}

reduce((acc, a) => acc + a, 0)(oneToTen)

// 55

Or equivalently we can define reduce using iteration:

reduce = (f, seed) => arr => {
  if(arr.length == 0) {
    // result of reducing an empty array is the initial seed
    return seed 
  } else {
    let result = seed
    for(var i = 0; i < arr.length; i++) {
      const a = arr[i]
      result = f(result, a)
    }
    return result
  }
}

reduce((acc, a) => acc + a, 0)(oneToTen)

// 55

I hope you agree that our definition using recursion is more elegant. It captures some truth about reduce. It clearly shows that reduce is an abstraction over recursion of the elements in an array.

The iterative version though is faster in JavaScript, because many JavaScript engines do not support tail-call-optimization technique.

Reducing from Right

Standard Array.prototype.reduce reduces the array from left-to-right. This means it first applies the reduction operation to the seed value and the first element of the array, creates a new seed value, drops the first element and repeats.

We can also reduce arrays from right-to-left:

let reduceRight = (f, seed) => arr => {
  if(arr.length === 0){
    // result of reducing an empty array is the initial seed
    return seed 
  } else {
    const [a, ...tail] = arr
    const result = reduceRight(f, seed)(tail)

    // first result is the seed, 
    // second result is f applied to the seed and 
    // the last element of the the array, ...
    return f(result, a)
  }
}

// 4 - (3 - (2 - (1 - 0))) = 2
let leftReduceResult  = [1,2,3,4].reduce((acc, a) => a - acc, 0)

// (((4 - 0) - 3) - 2) - 1 = -2
let rightReduceResult = reduceRight((acc, a) => a - acc, 0)([1,2,3,4])

console.log({leftReduceResult, rightReduceResult})

Right-to-left reduction is especially efficient with linked-list data structure.

ECMAScript supports Array.prototype.reduceRight:

[1,2,3,4].reduceRight((acc, a) => a - acc, 0)

// -2

Scan

No piece of writing about reduce is complete without mentioning scan.

scan returns an array that contains the result of each step in reduction. Scan is especially useful in stream processing when we are dealing with effectively infinite streams (Check RxJS scan).

let scan = (f, x0) => arr => arr.reduce(
  ({prev, result}, a) => {
    const current = f(prev, a);
    return {prev: current, result: result.concat([current])};
  }
  , {prev: x0, result: []}
).result

let sum_scan = scan(
  (total, a) => total + a
  , 0
)

sum_scan(oneToTen)

// [1, 3, 6, 10, 15, 21, 28, 36, 45, 55]

Note that we used reduce to define scan while the final item in the array that scan produces is the result of reduce:

last(scan(f, s, arr)) == reduce(f, s, arr)

Or using pipe:

pipe([
  scan(f, s)
, last
]) == reduce(f, s)

Or in math notation:

last ∘ scan = reduce

I hope I got you into the zen of reduce.

DEV Community: Homam

Time Series Visualization using SQL Window Functions

Generating Data

SQL Window Function

Random Walk

Missing Values

Composability: from Callbacks to Categories in ES6

Callbacks

Composable Callback

Callback and Promise are Monads

What is a Monad?

Categories

Good old functions

Func Category

Functions that return a Monad form a Category

In conclusion

Reduce

Summing up Numbers

Array functions

The seed value

f(a, i) == f(i, a) == a

Pipe

Moving Average

Defining Reduce

Reducing from Right

Scan

last ∘ scan = reduce