How I improved my JS code with these 3 functions

I’m a huge fan of Functional Programming. Today I will show you how it helps writing readable code.

I worked recently on a recommendation system in Node.js, where I applied some of FP principles.

Initial code

In my examples, I’ll use the Flow type system so that it’s easier to follow the idea. The code consists of two parts: a service API and a recommendation flow.

Service API

First, here are the type definitions we'll use in the service.

type Options = { [string]: any }

type Entity = { [string]: any }

type OneOrMany<A> = A | A[]

And this is the service interface:

class RecommendationService {
    user =           async (input: OneOrMany<number>)                          : Promise<Entity[]> => []
    signals =        async (input: OneOrMany<Entity>, options: Options)        : Promise<Entity[]> => []
    signalCreators = async (input: OneOrMany<Entity>, options: Options)        : Promise<Entity[]> => []
    creatorContent = async (input: OneOrMany<Entity>, options: Options)        : Promise<Entity[]> => []
    similar =        async (input: OneOrMany<Entity>, options: Options)        : Promise<Entity[]> => []
    enrich =         async (input: OneOrMany<Entity>)                          : Promise<Entity[]> => []
    set =            async (input: OneOrMany<Entity>, from: string, to: string): Promise<Entity[]> => []
    take =           async (input: OneOrMany<Entity>, amount: number)          : Promise<Entity[]> => []
    shuffle =        async (input: OneOrMany<Entity>)                          : Promise<Entity[]> => []
    deduplicate =    async (input: OneOrMany<Entity>, by: string)              : Promise<Entity[]> => []
    sort =           async (input: OneOrMany<Entity>, by: string)              : Promise<Entity[]> => []
    merge =          async (...inputs: Promise<Entity[]>[])                    : Promise<Entity[]> => []
}

We will concentrate on how we compose those functions, so their implementation doesn’t matter to us.

Most have a signature of expecting an input with optional configurations and returning an array of modified data. This API is very flexible and allows the designing of a variety of recommendation flows.

Recommendation flow

I wanted to use this API in the following way:

At a high level, it has two parallel recommendations: via similar content from user signals and via similar creators of those signals, with many additional steps on top of those.

Let see how we can describe this diagram. We will start with chaining promises like this:

recs.user(userId)
    .then((i) => recs.signals(i, {...}))

And merging them like this:

recs.merge(
    flow1,
    flow2
)

Also, merge returns promise as well, so we can chain it:

recs.merge(...)
    .then((i) => g(i))

The full flow implementation would look this way:

async function recFlow(userId: number, recs: RecommendationService): Promise<Entity[]> {
    const userFlow: Promise<Entity[]> = recs
        .merge(
            recs.user(userId).then((i) => recs.signals(i, { someConfig: 'A' })),
            recs.user(userId).then((i) => recs.signals(i, { someConfig: 'B' }))
        )
        .then((i) => recs.deduplicate(i, 'id'))
        .then((i) => recs.set(i, 'id', 'origin'))
        .then((i) => recs.similar(i, { someOption: 1 }))
        .then((i) => recs.deduplicate(i, 'id'))
        .then((i) => recs.shuffle(i))
        .then((i) => recs.take(i, 10))

    const creatorFlow: Promise<Entity[]> = recs
        .merge(
            recs.user(userId).then((i) => recs.signalCreators(i, { someConfig: 'A' })),
            recs.user(userId).then((i) => recs.signalCreators(i, { someConfig: 'B' }))
        )
        .then((i) => recs.deduplicate(i, 'id'))
        .then((i) => recs.set(i, 'id', 'origin'))
        .then((i) => recs.similar(i, { someOption: 1 }))
        .then((i) => recs.deduplicate(i, 'id'))
        .then((i) => recs.creatorContent(i, {}))
        .then((i) => recs.shuffle(i))
        .then((i) => recs.take(i, 10))

    const bothFlows: Promise<Entity[]> = recs
        .merge(userFlow, creatorFlow)
        .then((i) => recs.deduplicate(i, 'id'))
        .then((i) => recs.enrich(i))
        .then((i) => recs.sort(i, 'order'))

    return await bothFlows
}

The first question you could have is: “why don’t we use async/await?”.

If we want to use the await keyword to chain computations, we’ll need to split this code into multiple smaller sub-functions because the merge function expects parallel promises. In this case, it’s undesirable as we want to keep the complete picture of the entire flow, hence we chain promises with then.

To me, this code is very cumbersome and cluttered. We concentrate too much on chaining functions rather than business logic.

Additionally, I don’t like this indirection in the merge: when you see this method, you need to put the reading context into a stack and explore what happens inside the function arguments.

Now, let’s improve this code, but first, I’ll introduce you to currying:

Currying is a technique of translating a function with multiple arguments into a sequence of functions with a single argument:

$$F(a,b,c) \Rightarrow F(a)(b)(c)$$

It allows the creation of partially applied functions, i.e. capturing intermediate arguments and constructing new functions:

$$H = F(1)(2) \\ x = H(3) \\ y = H(4)$$

Refactoring service

Let’s refactor the RecommendationsService using the currying technique. For this, I want to introduce a new type alias:

type Pipe<A> = (OneOrMany<A> => Promise<Entity[]>)

Pipe is a generic async function which receives a single argument 'input' and returns an 'output'.

Now, let’s see how we can rewrite the service using Pipe and currying:

class RecommendationService {
    user                                       : Pipe<number> =  async (input) => []
    signals                = (options: Options): Pipe<Entity> => async (input) => []
    signalCreators         = (options: Options): Pipe<Entity> => async (input) => []
    creatorContent         = (options: Options): Pipe<Entity> => async (input) => []
    similar                = (options: Options): Pipe<Entity> => async (input) => []
    enrich                                     : Pipe<Entity> =  async (input) => []
    set            = (from: string, to: string): Pipe<Entity> => async (input) => []
    take                     = (amount: number): Pipe<Entity> => async (input) => []
    shuffle                                    : Pipe<Entity> =  async (input) => []
    deduplicate                  = (by: string): Pipe<Entity> => async (input) => []
    sort                         = (by: string): Pipe<Entity> => async (input) => []
    merge = async (...inputs: Promise<Entity[]>[]): Promise<Entity[]> => []
}

It may not be clear at first where the currying is, so let me show some methods without types:

class RecommendationService {
    user = async(input) => []
    ...
    signals = (options) => async (input) => []
    ...
    set = (from, to) => async (input) => []
    ...
}

This way, we transformed signals from (input, options) to a curried version of (options)(input).

Some functions, like user or shuffle, don’t expect any configuration and are instances of Pipe type already; others, like signals or similar, receive some options and create the Pipe instance.

With this new API, we can now change the flow implementation:

async function recFlow(userId: number, recs: RecommendationService): Promise<Entity[]> {
    const userFlow: Promise<Entity[]> = recs
        .merge(
            recs.user(userId).then(recs.signals({ someConfig: 'A' })),
            recs.user(userId).then(recs.signals({ someConfig: 'B' }))
        )
        .then(recs.deduplicate('id'))
        .then(recs.set('id', 'origin'))
        .then(recs.similar({ someOption: 1 }))
        .then(recs.deduplicate('id'))
        .then(recs.shuffle)
        .then(recs.take(10))

    const creatorFlow: Promise<Entity[]> = recs
        .merge(
            recs.user(userId).then(recs.signalCreators({ someConfig: 'A' })),
            recs.user(userId).then(recs.signalCreators({ someConfig: 'B' }))
        )
        .then(recs.deduplicate('id'))
        .then(recs.set('id', 'origin'))
        .then(recs.similar({ someOption: 1 }))
        .then(recs.deduplicate('id'))
        .then(recs.creatorContent({}))
        .then(recs.shuffle)
        .then(recs.take(10))

    const bothFlows: Promise<Entity[]> = recs
        .merge(userFlow, creatorFlow)
        .then(recs.deduplicate('id'))
        .then(recs.enrich)
        .then(recs.sort('order'))

    return await bothFlows
}

It looks much better already, as we eliminated some clutter from passing the input everywhere. At this stage, I’m more or less satisfied with the API. However, there is still a lot to improve in the flow, and we haven't fixed the merge flaw yet.

Before moving to the next step, I need to introduce Ramda - functional library for JavaScript. It comes with plenty of useful utility functions.
$ npm install ramda
const R = require('ramda')

Chaining

Let’s meet the first two functions which help make this code better: R.pipeWith and R.andThen.

Together, they can compose multiple Pipes into an ultimate Pipe. For example, the following two functions are identical:

const userSignalsVanilla: Pipe<number> = 
  async (userId) => recs.user(userId).then(recs.signals({}))

const userSignalsRamda: Pipe<number> = 
  R.pipeWith(R.andThen, [recs.user, recs.signals({})]

Now, we can create a helper function flow, which combines a list of Pipes.

const flow = <Inp>(f: Pipe<Inp>, ...fs: Pipe<Entity>[]): Pipe<Inp> => 
  R.pipeWith(R.andThen, [f].concat(fs))

Please, notice that the resulting type of Pipe, i.e. input type, is the same as the type of the first function in the chain.

Having this utility function, we can rewrite our flow in this way:

async function recFlow(userId: number, recs: RecommendationService) {

    const userFlow: Pipe<number> = flow(
        (i) =>
            recs.merge(
                flow(recs.user, recs.signals({ someConfig: 'A' }))(i),
                flow(recs.user, recs.signals({ someConfig: 'B' }))(i)
            ),
        recs.deduplicate('id'),
        recs.set('id', 'origin'),
        recs.similar({ someOption: 1 }),
        recs.deduplicate('id'),
        recs.shuffle,
        recs.take(10)
    )

    const creatorFlow: Pipe<number> = flow(
        (i) =>
            recs.merge(
                flow(recs.user, recs.signalCreators({ someConfig: 'A' }))(i),
                flow(recs.user, recs.signalCreators({ someConfig: 'B' }))(i)
            ),
        recs.deduplicate('id'),
        recs.set('id', 'origin'),
        recs.similar({ someOption: 1 }),
        recs.deduplicate('id'),
        recs.creatorContent({}),
        recs.shuffle,
        recs.take(10)
    )

    const bothFlows: Pipe<number> = flow(
        (i) => recs.merge(userFlow(i), creatorFlow(i)),
        recs.deduplicate('id'),
        recs.enrich,
        recs.sort('order')
    )

    return await bothFlows(userId)
}

Let’s now fix the merge part.

Converging

Judging by its signature, the merge function takes multiple branches and merges their output into a single array. Ramda has a helper for this case either: R.converge, so again these two functions are equal:

const pipeVanilla: Pipe<number> = 
  async (i) => recs.merge(userFlow(i), creatorFlow(i))

const pipeRamda: Pipe<number> =
  R.converge(recs.merge, [userFlow, creatorFlow])

Final version of the flow:

async function recFlow(userId: number, recs: RecommendationService): Promise<Entity[]> {

    const userFlow: Pipe<number> = flow(
        recs.user,
        R.converge(recs.merge, [
            recs.signals({ someConfig: 'A' }), 
            recs.signals({ someConfig: 'B' })
        ]),
        recs.deduplicate('id'),
        recs.set('id', 'origin'),
        recs.similar({ someOption: 1 }),
        recs.deduplicate('id'),
        recs.shuffle,
        recs.take(10)
    )

    const creatorFlow: Pipe<number> = flow(
        recs.user,
        R.converge(recs.merge, [
            recs.signalCreators({ someConfig: 'A' }), 
            recs.signalCreators({ someConfig: 'B' })
        ]),
        recs.deduplicate('id'),
        recs.set('id', 'origin'),
        recs.similar({ someOption: 1 }),
        recs.deduplicate('id'),
        recs.creatorContent({}),
        recs.shuffle,
        recs.take(10)
    )

    const bothFlows: Pipe<number> = flow(
        R.converge(recs.merge, [
            userFlow, 
            creatorFlow
        ]),
        recs.deduplicate('id'),
        recs.enrich,
        recs.sort('order')
    )

    return await bothFlows(userId)
}

With the reduced clutter, now we can concentrate only on the business logic, instead of remembering to pass inputs and chain promises.

I encourage you to check out Ramda documentation for other useful utility functions, experiment with them and improve the readability of your code.

I hope you will find this article useful, thank you!

Comments

[artydev]

Very instructive,
Thank you

[Lana]

Thanks for sharing. Good tips