DEV Community: John Kazer

Async functionality - currying that makes sense

John Kazer — Thu, 03 Nov 2022 02:46:02 +0000

There are a range of ways in JavaScript to handle or express delayed activity:

Callbacks (pass a single function to be used later)
Promises
Async/Await
setTimeout
Subscription
Push notifications
etc.

Another way of providing delayed functionality is "currying". Regardless of the many tutorials about how currying works, the main point is to handle a delay in the application of functionality. I just use the Ramda curry function and don't worry about the mechanics.

In fact, currying can work quite well in an asynchronous environment because the delay is fundamental. You may often be subject to waiting for all the information before executing full functionality.

An example might be pre-configuring a button in React, but needing to receive a user selection from a drop-down that completes the configuration before adding it to the UI. The user's selection might define the button's label text for example.

Another example is making use of the difference between configuring how to process an array, and actually doing the processing.

import { curry } from 'ramda'

const listOfData = [1, 2, 3]
// the core function expressing the functionality we want
// but wrapped by the 'curry' function
const processMyList = curry((configParam, listItem) => {
    return configParam + listItem
})
const fetchedConfigParam = await someAsyncronousFunction() // fetches a number
// using 'curry' creates a new function we can use later
const incrementListOfNumbers = processMyList(fetchedConfigParam) // provide the configParam arg
// do the processing
const processedList = listOfData.map(number => {
    return incrementListOfNumbers(number) // provide the listItem arg
})

NPM - too much of a good thing?

John Kazer — Tue, 23 Mar 2021 22:23:32 +0000

(cover pic unsplash)
A thought occurred to me today when I was considering an article by Eric Normand on the different moods or "buzz" that languages generate in the dev community. His discussion point is that JavaScript generates enormous interest, obvious activity and "buzz", whilst a language like Clojure is used extensively but not talked about.

His take was that this is just fine, as Clojure is used by many for building apps (front and backend along with enterprise tools) - but it is very stable and therefore doesn't automatically generate "buzz" about itself... There are few changes to shout about and discuss.

The thought that occurred to me was actually that maybe the JavaScript community generates too much buzz, and spends too much time developing new libraries on npm. If we just got on with the libraries we already have and spend less time gold-plating things and learning new stuff we'd have more energy to spend creating more and better quality products.

(pic unsplash)

There is indeed a lot to be said for an active community and the potential for an existing library out of the thousands to solve your problem very specifically.

As an example, perhaps the differences between Vue and React or Svelte (Angular and this or that) don't really warrant their on-going segregation of the resulting apps and dev communities? Is it actually, overall, less efficient and resulting in lower-quality apps over time?

(this might be a bad example, as I don't really want to discuss the merits of frameworks per se, but use their existence as an example that applies to npm in general)

I know that it can be argued that one or the other library/framework/whatever is better in certain circumstances than the others - but is that really true and do you really gain sufficient benefit compared to the effort of complicated build processes, maintenance, updates and on-boarding of new staff? Not to say the opportunity cost from barriers to knowledge sharing between teams using different libraries/frameworks regarding architectures and domain solutions.

Perhaps if there was less "buzz" and more "do" in the JavaScript community, we'd have higher quality engineering and more apps?

Refactoring a horrible function - thought processes

John Kazer — Sun, 21 Feb 2021 22:56:49 +0000

How to think differently about your code, via an example of how I rescued myself from a horrible function.

TL:DR - Processing different data types separately and thinking hard about the key actions or fundamental operations you want to apply to data really do make for simpler code. In this example, I turned a horrible function with 5 nested forEach loops (and 3 auxillary functions) into 3 simple functions using 3 maps and a single forEach (and 2 auxillary functions).

The general problem

I won't give a full description of how the initial function works, hopefully the general idea will be enough. And why would you pollute your mind with the effort?! Also, the refactoring should make the aim much clearer.

The core problem is one of converting the result of searching in a narrow but deep set of data trees into a simple UI for selecting specific search results with a checkbox. I was using a useful library fuse to run the search and so to some degree was limited by the way it reported results (I did make some customizations).

Steps I wanted to implement:

Build some UI from a set of simple trees of HTMLElements representing the search results for users to select
Only add to the simple trees those parts of the data trees that where positive search results
Loop through each set of search results relevant to each data tree
Loop through each data tree

The basic data structure to be searched was an array with elements like the following. The search targets where the value keys of the Class3Name children:

{
    colName: "Class1Name",
    value: "Energy",
    children: [
        {
            colName: "Class2Name",
            value: "Electricity",
            children: [
                {
                    colName: "Class3Name",
                    value: "Grid",
                    children: [
                        {
                            colName: "factor",
                            value: "United Kingdom grid",
                            children: [
                                // a few more nested levels followed by a final children[]
                                // with a list of objects with some data details
                            ]
                        }
                    ]
                }
            ]
        }
    ]
}

Which was augmented by some search result "matches" that gave me arrays of the indexes of tree children that lead to positive search results.

So whilst trying to get my head around the complicated data structures I had, and contemplating the direct creation of UI, I created a rather imperative solution that successfully did everything at once. I also wrote a small essay of comments to try and explain what on earth was going on.

The Horrible Function

The following code is the eponymous "horrible function", slightly augmented with jsDocs types (see end of article). It also uses some helper functions, not included for brevity.

The trap that I fell into was trying to manipulate the data directly to create UI. It is imperative code that defines how I want the data to be processed.

const treeForUI = buildTree(searchResultsArray) // will be added to the DOM
/**
     * 
     * The data arg will be an array of branches, each of which contain one or more positive search results (matches).
     * The problem is that it isn't known which part(s) of the branch was an actual result. So we use the information in the match object.
     * The match object contains an array of the specific results and their associated paths along their branch. The paths are described in two complementary ways.
     * Recall that a branch has one or more children and multiple levels, hence many 'leaves' or potential search targets.
     * Firstly, the sequence of search keys from creating the search library's object are re-used, in match.key.path.
     * Secondly the sequence of indexes in each 'children' array that describes the "route" to a search result is in match.path.
     * Therefore for each match in a branch, we want to drill down into the branch using the specific route through the path described by the search results.
     * @param { searchResult[] } data 
     * @return { HTMLDivElement }
     */
const buildTree = (data) => {
    let tree = document.createElement('div')
    data.forEach(branch => {
        /** @type { HTMLElement } */
        const newLevelOneNode = createEfHtmlTree(branch.item)
        branch.matches.forEach(match => {
            /** @type { HTMLElement } */
            let currentNode = newLevelOneNode
            branch.matches.forEach(/** @type { match } */ match => {
                /** @type { branch } */
                let currentBranch = branch.item
                match.path.forEach((p, index) => {
                    const k = match.key.path[index + 1] // + 1 because have already handled the first level
                    // process the next branch or a leaf (i.e. emission factor data)
                    match.path.forEach((childIndex, branchLevel) => { // target the specific index of each branch level's children array
                        /** @type { boolean } */
                        const isSearchResult = branchLevel + 1 === match.key.path.length - 1  // have we gotten to the last element in the search yet?
                        /** @type { branch } */
                        let branchInfo = {}
                        if (index + 1 < match.key.path.length - 1) {
                            branchInfo = currentBranch[k][p]
                            const branchInfo = buildBranchSearchRoute(currentBranch, match, childIndex, branchLevel, isSearchResult) // build the next level of the search route
                        if (isSearchResult) { // this will also be the end of the current forEach iteration
                            // incrementally build the HTML tree to reflect the search tree routes
                            newLevelOneNode.append (createEfHtmlTree (branchInfo))
                        } else {
                            branchInfo.colName = match[k]
                            branchInfo.value = collateEfDetails(currentBranch.children[p], match[k])
                            currentBranch = branchInfo // move down the tree in the data
                        }
                        /** @type { HTMLElement } */
                        const nextNode = createEfHtmlTree(branchInfo)
                        currentNode.append(nextNode)
                        currentNode = nextNode // move down the tree in the (soon-to-be) DOM
                        currentBranch = branchInfo // move down the tree in the data
                    })
                })
            })
        })
    tree.append(newLevelOneNode)
    return tree
}

So I managed to build all this logic and get it working. It felt like I'd bashed out some code to get a job done then move on to the next thing. But it bothered me so much that I kept picking at the logic to try and see how to refactor it.

Refactoring

Some steps that got me to what I think is a much better solution:

Call buildTree in a map, to take out the first level of forEach loops. But I was confounded by the types, as it returns HTMLElements that don't sit well with a regular array. So I decided the data types needed to be dealt with separately - search results and UI.
Consider the operation to select the successful search results as a filter operation. Here, I considered that the process of keeping the "paths" through each tree's branches and children based on search success/fail is basically a filter. But not a straightforward one as would need to be an iterative/recursive filter down the lists of children.
In fact, focusing on the search results separately from UI meant that I could build new successful-hits-only search data via a map then use this to create UI. The clarity provided by separated data and the filter concept led to a better solution.

So I guess I was finally following some basic data management principles of separating types out and thinking more about functional operations rather than direct UI creation.

The resulting code is spread over several functions but hopefully provides a much more obvious logic.

const collatedSearches = searchResultsArray.map(collateSearchResults)
const searchNodes = collatedSearches.map(buildSearchResultNodes) // will be added to the DOM
/**
 * This function acts as a sort-of filter, only returning data for those child elements that appeared in the successful search "matches"
 * @param { searchResult } searchResult 
 * @return { collatedSearch }
 */
const collateSearchResults = (searchResult) => {
    return {
        "colName": searchResult.item.colName,
        "value": searchResult.item.value,
        "efDetailsList": searchResult.matches.map(/** @type { match } */ match => {
            const searchResultLocation = moveAlongSearchPath(searchResult.item, match.path)
            return collateEfDetails(searchResultLocation)
        })
    }
}
/**
 * Follow a search path recursively down a branch
 * @param { branch } branch 
 * @param { number[] } path
 * @return { branch } 
 */
const moveAlongSearchPath = (branch, path) => {
    if (path.length < 1) {
        return branch
    } else {
        return moveAlongSearchPath(branch.children[path[0]], path.slice(1))
    }
}
/**
 * Build a simple UI structure from a list of successful searches
 * @param { collatedSearch } collatedSearchResult 
 * @return { HTMLDivElement }
 */
const buildSearchResultNodes = (collatedSearchResults) => {
    const rootElement = createEfHtmlTree(collatedSearchResults) // need to create top-level for Class1Name
    collatedSearchResults.efDetailsList.forEach((ef) => { // add each ef from the list
        const nextElement = createEfHtmlTree(ef) // create a checkbox element
        rootElement.append(nextElement)
    })
    return rootElement
}

The jsDocs types used:

/** 
 * @typedef { object } searchResult
 * @property { branch } item
 * @property { number } refIndex
 * @property { match[] } matches
 */
/** 
 * @typedef { object } branch
 * @property { branch[] } children
 * @property { string } colName
 * @property { string | efValue } value
 * @property { number } [efCount]
 */
/** 
 * @typedef { object } match
 * @property { number } idx
 * @property { number | undefined } indices
 * @property { pathKey } key
 * @property { number } norm
 * @property { number[] } path
 * @property { number } score
 * @property { string } value
 */
/** 
 * @typedef { object } collatedSearch
 * @property { string } colName
 * @property { string } value
 * @property { efValue[] } efDetailsList
 */
/** 
 * @typedef { object } efValue
 * @property { string } name
 * @property { number | string | boolean } cf
 * @property { string } unit
 */

What puzzles me still as a learning developer, however, is whether given the same sort of task again I'd now leap straight to the "clean" reasonably functional approach, or still have to start imperatively.

Is web security broken?

John Kazer — Thu, 26 Nov 2020 08:38:38 +0000

Or just our approach to it?

Why does this subject, so fundamental, seem so hard?

The internet, including DEV, is full of articles on security, JWT, cookies-good, cookies-bad etc.
Some services (e.g. Azure Active Directory) seem to handle it all for you - but do they really? And when can you actually use this approach?
Pretty much any article on the subject will come with a stream of comment and debate attached - doubt is sown...

What's a professional, let alone a beginner, to do?

Functional programming is conversational, OOP is abstract art

John Kazer — Mon, 28 Sep 2020 08:35:23 +0000

Functional, OOP and procedural styles are all valid, just different - and in fact complementary.

To explore how, here's a fundamental question: What is it really like to program in a functional, procedural or an OOP style? I don't mean the technical nuts-and-bolts, which many folk have and will write about. I mean from a personal perspective - what thinking style are you using and is it suited to the task at hand?

I was reading an article by Arsen recently, on the different interactions implicit in software engineering teams. For example, engineering processes are basically code procedures that are both impersonal and hard to stop once started. But very useful and efficient for well-understood activities.

Consider modern art (such as the article's header pic which is a part of "Little painting of fir-trees" by Paul Klee). It takes a lot of thought and consideration to distill a thought or a scene into an abstract representation on canvas. And once done it is hard to undo - the links between painting and reality are not always obvious or reversible. But the end-point is powerful and imparts meaning quickly. Of course, the meaning you extract might not always be what the painter intended you to see!

Contrast this with the interactive conversation that is pair-programming or an agile sprint. Feedback is rapid, change is easy (desired) and you have more control over the questions and answers.

A common development process therefore might start with some rapid-feedback activity, when the solution space isn't clear. But over time become more structured.

We can apply the same concept to the style of programming. Start functional, move to objects with message passing once the structure is clearer and more organisation is needed (the objects may be functions with closures or some other OOP concept). Then convert some of the fundamental activities into fast and efficient procedures when you can cope with the loss of control this implies.

Functional, OOP and procedural styles are all valid, just different - and in fact complementary.

Don't be confused by closures, currying etc

John Kazer — Fri, 11 Sep 2020 22:11:37 +0000

If you are worrying about the how and the detail, just relax. Focus on the logical operations.

The fundamental issue of a functional style is to deal in the logic of data transforms. You don't need to know how currying works, just use it (e.g. via a library like Ramda). Avoid the books and tutorials that try to explain the technical features and give examples of functions implementing partial application - at least for now.

If you are used to a more imperative style where the how is important, such as the detailed structure of a for loop, then it can be confusing dealing with the details of closures, currying, partial application and such like. There are many resources telling you in detail how these things work. But functional programming is about logic and data manipulation. You don't have to worry about the detail, so don't! Understanding what these tools can do is important, details less so.

Write pseudo-code and re-use or write functions to match.

The first time I really felt I "got" the concept of currying wasn't by reading examples explaining how to build a curry or partial application function from scratch. It was just using the concept to solve a simple problem. Now, I won't say that the following problem or solution are good practice but that doesn't matter either :-)

A simple example in pseudo-code:

create a basic button so the app is usable
save a way to update it later
render the basic button
delay here whilst getting some info from the user
eventually get the information to update the button
use my saved method to make the update
optionally do this again, with the exact same functions but different data

The key feature here is the delay in getting some data then being able to apply it to make a change. A bonus is being able to repeat this again, reliably. When there is a delay, use currying (or possibly a closure) to retain functionality or data for later.

import { curry } from 'ramda'

/**
 * return { HTMLButtonElement }
 */
const updateButton = curry((/** @type { HTMLButtonElement } */ button, /** @type { string } */ label) => {
    return button.textContent = label
})
/** @type { HTMLButtonElement } */
const buttonToUpdate = document.getElementsById('button')
renderBasicButton(/** @type { HTMLButtonElement } */ buttonToUpdate, 'dummy label') // create the button with an initial label
// generate a new function
renderButtonWithLateLabel = updateButton(/** @type { HTMLButtonElement } */ buttonToUpdate) // use currying to store the ability to update the button later
// renderButtonWithLateLabel is now a function that has stored the button. It's just waiting for the second argument, the string, to come along later.

// do stuff, get a string from the user maybe

/** @type { string } */
const labelDefinedLater = "text"
renderButtonWithLateLabel(/** @type { string } */ labelDefinedLater)
// now we complete the logical step by updating the "saved" button with a new label

So a toy example but it was all I needed to grasp the principle of currying. It's all about delayed functionality.

Another more useful example is managing the values in the objects within an array. Say I wanted to update a value 'good' in a specific object (e.g. with a specified value of 'key'), whilst retaining the overall array integrity. This could be quite a complex process of mutated data, nested loops, searching through each object, finding the one to change, making the change and rebuilding the array.

However, I did a google search with a phrase that mirrors the general logic I was after "update value in an object array ramda" and the first result is what I want.

import { curry, map, when, propEq, assoc } from 'ramda'

const alter = curry((checked, key, items) => map(
  when(propEq('key', key), assoc('good', checked)),
  items
))

const arrayWithUpdatedObject = alter('true', '22', items)
// in the array "items", find the object with 'key' is 22 and set 'good' to be true

Which is fine, but is a rather one-shot sort of thing. If I make the first argument to propEq an argument to the curried function it becomes more interesting. By making an additional argument, we can save an aspect of functionality to use later.

const alter = curry((alterBy, checked, key, items) => map(
  when(propEq(alterBy, key), assoc('good', checked)),
  items
))

const alterByKey = alter('key') // save the concept of matching the objects by key

const arrayWithUpdatedObject = alterByKey('true', '22', items) // will alter the object with key === '22'

// alternatively you could create a separate instance which identifies objects to alter by id
const alterById = alter('id')

In many ways a closure is similar to currying. You are keeping some data or functionality around to be used later. The "how" of creating a closure isn't really important - the key thing is it's usefulness in your logical process.

Here is a great example of using a closure to store data and functionality for later. A function is created called person that contains data (firstName, lastName) and functions (getFullName, setFullName) which are stored for later use.

Thanks to

.ltag__user__id__296048 .follow-action-button { background-color: #2e0338 !important; color: #ffffff !important; border-color: #2e0338 !important; }

Stopa

stopachka stopachka

DEV's UI decision-making - A question...

John Kazer — Fri, 28 Aug 2020 14:09:01 +0000

.ltag__user__id__1 .follow-action-button { background-color: #3B21E2 !important; color: #C9FAF2 !important; border-color: #3B21E2 !important; }

Ben Halpern

A Canadian software developer who thinks he’s funny. He/Him.

bendhalpern benhalpern http://benhalpern.com

I'm interested in why the new(ish) DEV (desktop) interface invites you to 'Add Comment' when there are no comments yet on a post. Does this suggest you should write a comment without reading the content?

An alternative that seems more natural to me would be to simply invite you to read the post...

What was the thought process in this case?

The mobile view just tells you there are none:

Why functional programming? Stay in control

John Kazer — Fri, 29 Nov 2019 10:33:31 +0000

As I delve a bit more into the experience of converting myself into a semi-literate functional programmer, I still wonder what is the point.

Well, one interesting realisation occurred today and it is about control.

I was looking at some code, a straightforward JavaScript array.forEach construct. I thought about what was needed to convert it into something more functional - e.g. a array.map. What would happen? I'd need a return value to use as the logical effect of the function. What was I doing right now? It was hard to say - need to read the code and find out what the effect really is.

And then it hit me - using forEach means I was losing control of the code. It didn't feel 'safe' somehow. If I was asking for a return value then I'd still be in control because then I can do something with it. Using forEach gives all that responsibility away. I have to trust the called function to do the right thing.

Maybe this is a consequence of writing functional code and my emotional response to code is changing?

Hard to say, but it gave me some insight into the psychology of code wars and declarative (functional) vs procedural approaches.

Conclusion: Functional programmers like control and certainty, procedural programmers are more chilled (or maybe more anxious).

Functional programming is great, but not for the reasons everyone keeps telling you

John Kazer — Wed, 13 Nov 2019 14:31:48 +0000

There is no necessarily fundamental reason why functional programming is better than a procedural or object oriented approach. Yes, pure functions, clean code, readability etc. may make your programmes safer and easier to reason about.

But that isn't why I like functional programming.

I simply prefer the way it allows me to think about problem solving. Maybe the world is inherently split into programmers who think functionally and those who think procedurally - someone should fund a psychology research project!

As an example, say I have some data structure and would like to transform it. My thought process is to start writing down a list of actions that I think will be needed. This is an iterative process, as the consequences of each action becomes clearer and extra steps are added. I might also realise that a particular action is really a special case of something that already exists - so I just co-opt that action (or function). In doing so, I may have to change it to make it more generic - but that's fine because it's a pure function and therefore safe.

After a while, I might write some pseudo code which simply sets out the logical action sequence - often the initial list of actions is the pseudo code. And this pseudo code often ends up being the actual function I want. Then there is a detail which is the additional implementation needed to flesh-out some of the functions implied by specific actions in the pseudo code.

If there is iteration needed, that's a single recursive function which can be defined in pseudo code by name at this stage. It is sufficient to know that an iteration is needed and what form the output will take.

If there is a sequence of actions needed, that's a composition which the pseudo code represents with a sequence of functions.

If the logic determines that data needs transforming (e.g. filtering) then that's fine, just state it as an action. The result will be used further down the pseudo code and no need to worry about side-effects because there are none.

So the logical steps to get from initial data to result are quickly clear and little additional infrastructure is needed to just make it happen. This to me is how many people already solve problems and therefore functional programming doesn't necessarily need any change of mindset.

However, the wrinkle is that the incremental state of the data is not immediately obvious - the functional approach almost wants you to not care. And therefore your real mindset switch is to one of imagining data structure transformations as the chain of functions progresses.

Revist the waterfall process but this time with maths

John Kazer — Tue, 20 Aug 2019 21:31:43 +0000

(Cover image by John Westrock on Unsplash)

"Design, maths & framework".

The "you really should read" Pure UI and Pure UI Control articles defined an approach to app development (design and creation) which resolves to these 3 words. If their approach to maths is extended slightly to include testing, then we can say these 3 words constitute a comprehensive but simple waterfall development process.

I find this an attractive simplification, in which maths helps developer's lives become easier, less stressful and less process-dependent.

Design defines the appearance and the core states and transitions. Maths formalises the states and transitions, whilst visually and collaboratively supporting the investigation of missing details and then auto-generating tests. The app framework simply instantiates those states and transitions.

Pure UI design

The essence of 'Pure UI', as eloquently described in the articles at the beginning of this post, is a smooth interaction between design and business logic. UI is really a side-effect of the business logic, or state management. The chosen solution to provide this apparent magic is state charts. It doesn't really matter which framework you use, so long as it's functional and state -> UI is uni-directional!

What are state charts and how do they help?

State charts are a well-described mathematical concept (an intro and some resources) which extend the simpler state machine. The core principles are the definition of states, transitions between states and actions to take in consequence. Optional extras are the management of state and implementation of the actions (I prefer to let the app or UI framework deal with these).

A state chart allows reasoning about function without concern for implementation details - it is therefore easier. In particular, visualisation of a state chart enables easy collaboration with designers and rapid prototyping (or updating) of function. The linked visualisation is from the app example in 'Implementation options' below, and is a simple copy-paste of a state chart from the machines.js file in that app. A state chart designed in this way can therefore be used directly in code.

Automated testing with state charts

Much, of course, is written about unit tests. Functional programming, a natural fit for state machines and React or Hyperapp means unit tests are reasonably straightforward. What is always less so is testing business logic, or the interactions between functions and state which happen when the UI generates related sequences of events.

Cypress can automate business logic testing if state chart 'paths' are provided, which represent automatically generated routes or sequences of transitions. XState has a graph module which does this.

There are less than 10 simple app-specific lines of code, amongst re-usable functions, which drive the Cypress test of each app state and valid 'path'. You can watch the test video 'live' here and the app is hosted here (it is a simple media creation PWA, which doesn't send data anywhere but asks for video, audio and push notification permissions).

Maintenance

New requirements simply filter in from the top, via design updates and logical consequences of state chart changes. In principle the tests will automatically update as a function of the state charts (in practice some small amount of manual input may be required).

Implementation options

There are some details to work out regarding the specific boundary of responsibility between the maths and the app framework (e.g. exactly who hosts the state) but these don't change the principles.

For instance, in this example, the Hyperapp framework is responsible for managing state and executing actions. However in this example React is largely just an interface to the browser's DOM whilst the state chart (from XState) manages state and action execution. Specifically in this case, DOM events are linked to a React Hook which triggers a state transition - the state chart does the rest.

A very useful (but little written about?) state chart feature is that they can automatically provide app use-cases as sequences of state transitions. These sequences, or paths, can be used as the basis for testing - this example drives Cypress in this way. The repository is here - use the 'xstate' branch.

Waterfalls with maths

So there you have it - a waterfall dev process that is much simpler and friendlier than I remember. And some maths that isn't scary and is useful!

An example development process flow might look something like this:

Creating the initial design with Figma and sketch.systems (use this Figma plugin and see the 'Export to clipboard' option in sketch.systems) which informs, and is informed by;
State chart maths in XState.
UI via a framework (such as React or Hyperapp) is a side-effect of state chart behaviour.
Testing can be derived directly from the state chart maths and implemented with Cypress.
Requirement changes can quickly be incorporated into design and state chart maths updates, which (generally) require simple UI framework tweaks.

Use maths not process for reliable web apps

John Kazer — Fri, 16 Aug 2019 18:55:18 +0000

A combination of functional front-end JavaScript and state machines leads to unit and business logic tests which can approach mathematical proof. Watch as Cypress steps through the app, using an auto-generated graph of the state machine that defines the app's possible UI actions and transitions.

Disclaimer - to follow this post you can probably get away with knowing that Hyperapp manages state immutably, uses a virtual DOM and hosts pure functions to handle actions (such as click handlers), effects such as http requests and subscriptions to external events or streams. XState manages state machine nodes and transitions plus actions triggered by transition. The key to this demo is associating Hyperapp and XState actions. Some familiarity with basic Cypress testing will help too.

However, it might help to have some knowledge of Hyperapp actions and events plus state machines, as these are big subjects mostly out of scope of this post.

Why state machines?

De-stress app development.

Not only will you know that your app's pure functions work (existing unit testing) but you will also know that the business logic which wires them together also works!

You can use easily accessible mathematical principles to build reliable apps rather than depending on software process. This article focuses on business logic rather than unit testing.

Using an XState machine means that you can pre-define and test how sequences of user actions and events lead to state change and therefore app behaviour. The logic is clear and easy to reason about (rather than potentially obscured in collections of functions or separate hard-to-maintain flow diagrams). You can visualise the logic with interactive state machine charts and create tests easily. More on tests later but note that the state machine drives UI tests which prove functionality matches the logic.

More - if you update the requirements and change the state machine(s) the tests will automatically update as well.

How?

A Hyperapp demo

Here I introduce a Hyperapp demo, which implements a simple multi-media PWA (Progressive Web App) with XState. You can take a photo, create a recording and simulate uploading them to the cloud. If offline, the PWA will save locally and automatically 'upload' when you're back online. Apologies for the requested browser permissions - video, audio and push notifications - they are necessary for the app's functionality. But nothing leaves your browser!

The repository is here. The master branch just uses Hyperapp, whilst the xstate branch modifies the app to use XState state machines as well. Use 'npm start' to build and run using Parcel. There are two state machines, one to control taking and saving a photo, the other to make a sound recording.

Note that unlike most Hyperapp implementations, this app uses Pug to render HTML. You may prefer to convert to the more usual JSX or hyperscript approach.

How it works - Hyperapp

Hyperapp maintains a central state and message handler which listens for user actions and browser events. When an action or event changes the state, Hyperapp uses a virtual-DOM to update the app. A loop exists, half visible to the developer and half within the framework.

Action -> Event -> [ Listener -> State management -> Virtual-DOM ] -> DOM-change -> Action...

How it works - XState

State machines are a long-standing mathematical tool. Their practical application to apps has some common practical and conceptual features with how Hyperapp defines app behaviour. The main difference is that XState enables the relationships between Actions, Events and State to be defined unambiguously in one place.

A state machine is created in JSON, whilst the XState system provides pure functions for interacting with it. Your app can respond to action and event-driven change from the machine or request state changes directly.

Action -> [ Transition -> Possible Actions too choose from] -> Action...

There are two state machines in the demo, one to manage taking photo's and the other for recordings.

You can visualize and interact with the machines defined in machines.js. I have found it useful to compare the experience of visual logic with that of using the actual app or a mock-up. You can focus on the function without being distracted by form.

How it works - XState within Hyperapp

Two new functions that manage the link. One captures events (e.g. button clicks) and the other converts XState actions into Hyperapp actions. In effect, these functions act as a bridge between Hyperapp actions and state, and XState actions and transitions.

There are some important features of the app implementation which help this process. In particular, I get easy linking between UI and logic by using the same id names for UI elements as for their associated Hyperapp action function names and XState transitions. Conversely, it is a bit fiddly switching between different state machines - a balance between monolithic vs distributed.

For the video handling part of the app, each button has this onclick=[updateVideoState, targetId] tuple as a click handler.

The click handler function updateVideoState receives state and the element's id from Hyperapp, which handles the DOM event. It passes these and state machine details to processNewState shown below. This function calls runActions to link state machine actions associated with the transition to the execution of Hyperapp actions. It must handle the results of Hyperapp actions (new state), events (which return [state, event] tuples) and one-shot effects (no returned object). Finally processNewState returns the updated state to Hyperapp, along with the latest state machines.

const processNewState = (state, { machineState, machineName, machine, id }) => {
    const previousState = pathOr(machineState, [machineState, 'value'], state)
    const newMachineState = machine.transition(previousState, id) // 'id' is synonymous with 'newState'
    const requests = runActions(state, newMachineState, id)
    const videoState = machineName === STATE_MACHINES.VIDEO_MACHINE ? newMachineState : state.videoState
    const audioState = machineName === STATE_MACHINES.AUDIO_MACHINE ? newMachineState : state.audioState
    if (requests.length === 1) { // capture the result of an action
        return { ...requests[0], videoState, audioState }
    } else if (requests.length === 2) { // capture the result of an action-effect tuple
        return [
            { ...requests[0], videoState, audioState },
            requests[1]
        ]
    }
    return { ...state, videoState, audioState } // state machine was updated
}

The runActions function executes the action defined by the machine and allows Hyperapp to process the result. The actions are all regular Hyperapp action functions, the only difference from regular use of Hyperapp is that the state machine determines what action to execute when a valid transition occurs. Note that an action in XState is maintained as an array of action.exec functions.

const runActions = (state, calcState, evtObj) => { // make recursive or map
    let requests = []
    calcState.actions.forEach(action => {
        const stateChangeRequest = action.exec(state,evtObj)
        const isArray = Array.isArray(stateChangeRequest)
        requests = concat(requests, isArray ? stateChangeRequest : [stateChangeRequest])
    });
    return requests
}

Testing!

There are two parts to the testing issue - unit testing of pure functions and testing the app logic generated by sets of functions.

Unit testing Pure functions

Written about by many others...

Testing app logic with state machines

Self-writing tests! What's not to love?

Well, not completely self-writing but not far off. XState provides a graph feature which generates a set of objects and arrays which describe the paths through your app's state machines. I've implemented an approach using Cypress where control of the app tests is managed by these paths. See cypress/integration/tests/tests.js in the xstate branch of the repo.

Here's an example of the Cypress output, showing execution of a 3-stage path to reach 'upload'. Notice again the shared naming between DOM element id and transition.

In the example, there are four states and actions defined in the machine:

const videoMachine = Machine ({
    id: 'videoMachine',
    initial: 'videoState',
    states: {
        videoState: {
            on: {
                captureImage: {
                    target: 'capture',
                    actions: ['captureImage']
                }
            }
        },
        capture: {
            on: {
                success: 'captured',
                fail: 'videoState'
            }
        },
        captured: {
            on: {
                uploadImage: {
                    target: 'upload',
                    actions: ['uploadImage']
                },
                discardImage: {
                    target: 'videoState',
                    actions: ['discardImage']
                }
            }
        },
        upload: {
            on: {
                uploadSuccess: {
                    target: 'videoState',
                    actions: ['discardImage']
                },
                uploadFail: 'captured'
            }
        }
    }
})

Follow through the Cypress results alongside the state machine. Here's a summary, with the transition 'actions' in []

videoState -> [captureImage] -> capture -> [success] -> captured -> [uploadImage] -> upload -> [uploadSuccess] -> videoState

The first transition captureImage takes a picture and displays the result in a div. If this is successful, the second transition doesn't involve an action as Hyperapp resolves the result of an Effect in the background. success of the Effect allows the display of new buttons - to upload or discard. The path gives us an uploadImage action followed by the final step where Hyperapp manages the result of the 'uploadSuccess' effect in the background.

The outcome of the final 'uploadSuccess' step is not immediately obvious, but Cypress 'time-travel' (scrolling back through each test step) shows that we requested upload and received a success response afterwards.

Make the upload request (button click):

Uploaded and ready to take the next picture:

Here is the core of the test code for Cypress (I've not show the XState-graph objects as they're complex, but are essentially lists of transitions).

const checkTransition = ({ state, event }) => {
    cy.log(`checking transition from ${state.value} due to ${event.type}`)
    // if there are paths[x].state.actions[y] then have Hyperapp and XState run the action(s) for us
    // Optionally we could drive the machine from here too
    if (state.actions.length > 0) {
        cy.log(view(machineName, state))
        const machine = currentMachine(state)
        // machine.transition(...)
        return
    }
    // else run paths[x].event
    // add logic to check for DOM elements with id === paths[x].event
    return cy.get(`button[id="${event.type}"]`).click()
}
const listOfPaths = curry((selectPaths, currentState) => {
    cy.log(`List of paths state: ${currentState}`)
    return selectPaths(currentState)
})
const checkPath = (path) => map(checkTransition, path)

// the test
it('tests each state path', () => {
    cy.visit('http://localhost:1234')
    const listOfVideoPaths = listOfPaths(selectSimpleVideoPaths)
    const testVideo = map(compose(appIsReady, reload, checkPath, listOfVideoPaths))
    cy.log('Test video paths')
    testVideo(videoStateList)
})

As such, the core code is pretty reusable for any XState-graph, the only custom tweaking needed is selection of the actual UI action, .click() in this case.

What's next?

Other aspects of the app may be suitable for state machines, including upload/http, online/offline state and PWA installation.

There is an interesting boundary of responsibility between Hyperapp and XState that this demo has only just started to explore - such as state management and action definition.

It would be useful to build a system for defining the relationship between DOM elements being acted upon and the desired action (e.g. click, hover etc.). This is rather manual at present.

Also check out Rosmaro for a state machine implementation - it has a handy graphical UI for creating machines rather than hand-coding JSON.

What role components?

How many state machines do you need and how big should they be?

XState/graph returns a subset of possible paths but not all, so it would be a useful upgrade to ensure full coverage.

Is it a scalable concept?

Please add comments!

Hyperapp with Pug templates

John Kazer — Fri, 28 Jun 2019 09:19:24 +0000

I recently completed an excellent course by James Moore on the basics of functional JavaScript for web apps. The app framework he created is nice but uses hyperscript to define HTML as functions. I find hyperscript to be an interesting approach and it does allow for composable UI components. But I don't really get on with it...

As I was browsing DEV, I noticed a post from @aspittel which mentioned Hyperapp. Turns out this is built along very similar lines with an option to use JSX as well as hyperscript. I imagine the new v2 has moved on quite a bit from when Ali checked it out in early 2018.

Now, the Hyperapp framework is wonderful and has a number of features I like:

Clear functional approach to business logic
State-driven UI
Centralised state and no stateful components (easy 'undo' option and perfect for quick and reliable development of small to medium scale apps)
Events dispatched to update the state which updates the UI using virtualDOM diff
Fast, small and simple but sufficient

However, previously I've used Pug to define my UI templates. I like the retained ability to see the page structure and the clearer separation of logic and UI. Combining HTML with the business logic using hyperscript h functions doesn't sit right with me (yet?) and I find it hard to reason about a page when the layout is so abstract.

Maybe I'll come round eventually, but maybe I don't need to...

Fortunately for me, the project pug-vdom (obviously) brings a virtualDOM to Pug. So what follows is a brief intro to the very simple app I built to demo how Hyperapp can use Pug templates. See the Hyperapp pages to get a better understanding of the full range of what it does, or try their new Udemy/Packt course.

As project setup, here's the package.json. Key items to note being the start script and pug/pug-vdom dependencies (and you need Node.js version 6.4 or above).

{
  "name": "hyperapp-pug",
  "version": "1.0.1",
  "description": "An instance of hyperapp which uses pug and pug-vdom rather than the default h functions",
  "main": "main.js",
  "scripts": {
    "start": "node ./compilePug.js && parcel ./src/index.html"
  },
  "author": "John Kazer",
  "license": "ISC",
  "devDependencies": {
    "parcel": "1.12.4"
  },
  "dependencies": {
    "hyperapp": "2.0.12",
    "pug": "2.0.4",
    "pug-vdom": "1.1.2"
  }
}

And here is the basic project file structure

\dist (parcel output dir)
\src
  app.pug.js (compiled pug template)
  index.html
  main.js
  pug-to-view.js
  pug-vdom.js (added as a local copy to handle Hyperapp approach to textNodes)
\views
  app.pug
compilePug.js (see the start script)
package.json

And that's it.

Note: There's a PR for pug-vdom to include this tweak, so perhaps in future the local copy can go away.

As you can imagine, after running "npm install", using the "npm run start" script runs a compilation process which converts the Pug view into a .js file. The compilation is defined by compilePug.js. The compiled file is included require('./app.pug.js') by main.js and provides the virtual DOM nodes that Hyperapp needs to display the content. Then parcel runs, updates the other files in \src and starts a browser instance of \dist\index.html.

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Hyperapp demo</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
  </head>
  <body>
    <div id='app'></div>
    <script src="./main.js"></script>
  </body>
</html>

The compile process is pretty simple - a list of the templates and destinations (in this case just the one):

// compilePug.js
const vDom = require('pug-vdom')
vDom.generateFile('./views/app.pug', './src/app.pug.js', './views')

It compiles my simple Pug template:

// Need a root div to grab as the start of the view
div
    // receives the variables and functions from the supplied state object
    - var greeting = "Hello " + greet
    p(style={color: "red"}) #{greeting}
    input(size="60" placeholder=placeholder onchange=handler.updateMe)
    button(id='clickMe' onclick=handler.clickMe) Click Me
    p #{userText}

Now let's have a quick look at the main.js which defines the app:

// main.js
import { app, h, text } from 'hyperapp'
import { pugToView } from "./pug-to-view"

const view = pugToView(h, text)

// event handlers
const clickMe = (state, event) => ({
    ...state,
    userText: state.value
})
const updateMe = (state, event) => ({
    ...state,
    value: event.target.value
})

const initialState = {
    greet: 'friends',
    placeholder: 'Write something here first, hit \<enter\> then click the button',
    value: '',
    userText: '',
    handler: {
        clickMe,
        updateMe
    }
}

const node = document.getElementById('app')

app({
    init: initialState,
    view: view,
    node: node
})

Where the helper function pugToView comes from here


import 'pug-vdom/runtime' // runtime library is required and puts 'pugVDOMRuntime' into the global scope
const render = require('./app.pug.js')
export const pugToView = (h, text) => state =>
  render(
    state,
    (name, props, children) => h(name, props.attributes, children),
    text
  )[0] // grabs the root 'div' element whilst adjusting the way pug-vdom deals with props compared to hyperapp

The main.js file uses the three API functions provided by Hyperapp (yes, just three!), h, text and app. These functions provide state management, virtual DOM, diffing and DOM patching when state changes.

This is a static page, so there is only a simple initial state; some initial values and the event handlers to associate with the button and input elements.

We are going to inject the app at the selected node by providing the view as defined by the compiled Pug template using the content function.

The app function pulls all this together and Hyperapp takes care of the rest. Simple declarative, functional, logic coupled to a familiar templating system!

Find the repo here.