DEV Community: Eugene Ghanizadeh

Reactive Programming with React

Eugene Ghanizadeh — Thu, 11 Mar 2021 16:14:17 +0000

Software programs (specifically those with a user interface) typically need to react to stuff. The user clicks on something. They type something into an input. The response for a previously made request arrives. So on.

Handling all these events in an explicit manner (via event handlers, change observers, etc) can be daunting and lead to complicated code:

// this is schematic code, also update handler is simplified
// as in reality it would be more complex.

render(
  <>
    <input type='text' placeholder='type something'
      onInput={e => inputChanged(e.target.value)}/>
    <div ref={resultContainer}/>
  </>
)

on(inputChanged, async input => {
  const response = await fetchData(input)
  const alreadyRendered = []

  //
  // look at rendered children, remove those who
  // are no longer inside our result set
  //
  resultContainer.children.forEach(child => {
    const entry = child.getAttribute('key')

    if (response.entries.includes(entry))
      alreadyRendered.push(entry)
    else
      child.remove()
  })

  //
  // look at the result set, render those who are not
  // already rendered.
  //
  response.entries.forEach(entry => {
    if (!alreadyRendered.includes(entry)) {
      render(
        <div key={entry}>{entry}</div>
      , resultContainer)
    }
  })
})

Fortunately, there are tools out there like React that help with this problem. React, for example, does that by basically masking changes via detecting, propagating and handling them on its own behind the scene, and allowing us to describe our desired UI based on a given snapshot of the program state:

function MyApp() {
  const [input, setInput] = useState('')
  const [entries, setEntries] = useState([])

  //
  // this is sort of an explicit event handler,
  // because in response to input event we need to
  // do something not directly related to UI.
  //
  useEffect(async () => {
    const response = await fetchData(input)
    setEntries(response.entries)
  }, [input])

  return (
    <>
      <input type='text' placeholder='type something'
        onInput={e => setInput(e.target.value)}/>
      <div>
      {
        entries.map(entry =>
          <div key={entry}>{entry}</div>
        )
      }
      </div>
    </>
  )
}

► Playground

Unfortunately, looking only at snapshots of program state is not always enough. For example, in the code above, it might happen that entries displayed do not match the input by the user (check it out). This is because fetchData() takes varying times for responding to different inputs (randomly between 0 and 5 seconds in this case), so the result for some input A might come after the result for some other input B, despite A itself being entered earlier than B. Our component, as described above, will always display the last result it receives from fetchData(), regardless of the order of its corresponding input.

A pure React-based solution to this problem would look like this:

function App() {
  const [input, setInput] = useState('')
  const [entries, setEntries] = useState([])

  // here we maintain a reference to last request
  const lastPromise = useRef()

  useEffect(async () => {
    // keep a reference to our request in `promise`
    // and set it to last request as well
    const promise = lastPromise.current = fetchData(input)

    // wait for the response
    const response = await promise

    // if our request is still the last request,
    // then update the `entries`.
    if (promise === lastPromise.current)
      setEntries(response.entries)
  }, [input])

  return (
    <div>
      <input type='text' placeholder='type something'
        onInput={e => setInput(e.target.value)}/>
      <div>
      {
        entries?.map(entry =>
          <div key={entry}>{entry}</div>
        )
      }
      </div>
    </div>
  )
}

► Playground

For properly ordering responses, we would need a mapping between inputs and their corresponding requests. A single snapshot would not suffice for our case, as we would need to look at the stream of incoming inputs and what happens to them subsequently.

Unfortunately though, we lose that information in snapshot-only style of React. As a result, we need to hack our way by partially maintaining that information (the input stream and its mapping to requests) by holding a reference (useRef()) to the last request and keeping it up to date.

I say it is a hacky solution because it is pretty low-level (most of the information of corresponding code is about implementation detail rather than the intention) and not particularly composable/reusable (imagine a relatively similar situation where we receive our entries from a web-socket and in pieces).

Reactive Streams

Ok so sometimes we actually need to look at the stream instead of snapshots of it. But how would such a stream look like?

Well if you are familiar with RxJS, Observable is another name for a stream. If not, you can think of a stream as a source of data that will emit one or more values at unspecified times in the future (it might be synchronous, it might be one value every 10 seconds, it might be whenever there is a user-click, etc.).

For example, clicks on a button can be modeled with a stream of click events. The values of our input in previous example can be thought of as a stream of strings. A request can even be thought of as a stream, one which emits only a single value (though at a non-specified time in the future). Even a single value can be modeled by a stream that emits that value in a synchronous manner.

Now we can remodel our problem using streams: input is a stream of strings, and we need to map each emitted value to a corresponding request:

pipe(
  input,
  map(i => fetchData(i))
)

What this (☝️) gets us is a stream of requests, but we want a stream of responses. Since each request is a stream as well, we basically have a stream of streams, which we need to flatten down to a stream of values, i.e.:

pipe(
  input,
  map(i => fetchData(i)),
  flatten
)

This is of course callbag lingo (which is a light-weight model for streams, pretty similar to RxJS). Using RxJS our code would look more like this:

input.pipe(
  switchMap(i => fetchData(i)),
)

where switchMap() operator is basically a combination of map() and flatten().

How does this flattening work? Well there are many solutions for turning a stream of streams into a stream of values. For example, you could merge all values emitted by all of the sub-streams (this is what RxJS's mergeMap() does). More often than not, you would want to simply pick (or switch to) the last emitted stream and take values from it, which is what flatten() (or RxJS's switchMap()) does.

All of this means if we could look at input as a stream instead of a snapshot, our request-ordering problem would be solved pretty intuitively:

map each input to a request
flatten the stream

Reactive Streams in React

As we saw in the first example, manually handling changes can be a daunting task and lead to complex code. React masks these changes and allows us to look only at snapshots of our state, however that comes with the trade-off that when we need to look at the stream again (which happens often), our code quickly turns hacky.

What if we could combine the two models? What if we could look at state in snapshots, but also whenever necessary look at the underlying stream and then conveniently turn it back into state snapshots (i.e. normal JavaScript variables)?

Turns out, with a set of pretty simple hooks, we actually can achieve that. The resulting code, would then look like this:

function App() {
  const [input, setInput] = useState('')

  const [entries] = useStream(            // 👉 build a stream
    input,                                // .. from input
    map(i => fromPromise(fetchData(i))),  // 👉 map each to a request
    flatten,                              // 👉 flatten
    map(r => r.entries)                   // 👉 also map responses to `.entries` key
  )

  return (
    <div>
      <input type='text' placeholder='type something'
        onInput={e => setInput(e.target.value)}/>
      <div>
      {
        entries?.map(entry =>
          <div key={entry}>{entry}</div>
        )
      }
      </div>
    </div>
  )
}

► Playground

For this example, I have used react-callbag-streams, which uses callbags to represent reactive streams (it is much like RxJS, but more lightweight and also pretty decentralized).

npm i react-callbag-streams

import { useStream } from 'react-callbag-streams'

// ...

This hybrid approach brings us the best of both worlds: we get the convenience of only dealing with snapshots of the program state (as plain variables), while being able to peek into the underlying stream and run some stream operations on the state (like mapping to request streams and flattening), and then treating the result of those streams as snapshot states again.

This also provides much higher degrees of composability / reusability. For example, imagine if after requesting corresponding entries for some input, our server (or any other source of data) would also push updated entries whenever they change. In that case, fetchData() would need to return an actual stream of responses (i.e. a callbag). But our code would remain basically the same:

  const [entries] = useStream(
    input,
    map(i => fetchData(i)),
    flatten,
  )

► Playground

Other Streaming Operations

Another exceedingly common stream operator is debounce(). It is used for handling back-pressure: you might not want to send a request to the server for every single key the user presses, but rather one request when the user finishes typing. To determine that, you can look at each emission of the input stream, and if a certain amount of time passes without any new emissions, you can assume the user has finished typing and the latest emission is the input you should request your API with (we call this debouncing the stream).

With our hybrid model, adding a debounce (using the standard debounce() operator) can be as simple as this:

  const [entries] = useStream(
    input,
    debounce(200),                       // 👉 debounce the stream by 200ms
    map(i => fromPromise(fetchData(i))),
    flatten,
    map(r => r.entries)
  )

Note that debounce can be implemented by writing a specific debounce hook or using lodash's debounce in combination with useCallback(), but all these solutions would again be hacky:

  // using a hook would require an intermediary variable
  const [input, setInput] = useState('')
  const debouncedInput = useDebounce(input, 200)

  // using useCallback would require wrapping the setter callback
  const [input, setInput] = useState('')
  const debouncedSetInput = useCallback(
    debounce(setInput), 200),
    []
  )

The inherent simplicity in our streaming approach also opens the door for further composability: for example you can easily debounce multiple values together:

//
// this app finds repositories on github
// based on a query and language
//
function App() {
  const [q, setQ] = useState('')
  const [l, setL] = useState('javascript')

  const [repos, loading] = useCombinedStream(
    [q, l],                                             // 👉 a combined stream of query and language
    filter(([q]) => q.length >= 2),                     // 👉 filter out when query is too short
    debounce(1000),                                     // 👉 debounce the combo by a second (github API will block us o.w.)
    map(([q, l]) => fromPromise(search(q, l))),         // 👉 search in github api using query and language
    flatten,                                            // 👉 flatten the stream (preserve order of responses)
    map(res =>                                          // 👉 format the incoming result ...
      res.items.map(item =>                             // .. take each repository ...
        ({ name: item.name, url: item.html_url })       // .. get its name and its url
      )
    ),
  )

  return <>
    <input type='text'
      placeholder='keywords ....'
      value={q}
      onInput={e => setQ((e.target as any).value)}/>
    <input type='text'
      placeholder='language'
      value={l}
      onInput={e => setL((e.target as any).value)}/>
    <br/>
    { q.length >= 2 ?
      (
        loading ? 'loading ...' : (
          <ul>
            { repos?.map(repo => (
              <li key={repo.url}>
                <a href={repo.url}>{repo.name}</a>
              </li>
            ))}
          </ul>
        )
      ) : ''
    }
  </>
}

► Playground

Conclusion

As we saw, there is extreme convenience in React's model of only dealing with snapshots of program state, as explicitly handling changes and updates can result in pretty complicated and hard-to-read code.

However, in many extremely common scenarios (sending requests to an API, debouncing), we would basically need to look at the data stream instead of data snapshots (e.g. we care about order of data emissions or their timing). Subsequently, trying to handle these scenarios in a pure React way would result in hacky, not-so-composable code.

Fortunately though, we can combine the convenience of React's approach with power of reactive stream programming using hook libraries such as react-callbag-streams. This way, we get to handle pretty complicated reactive scenarios in elegant, composable and reusable manners, without sacrificing the convenient provided by React.

Auto Dark Mode w/ JSS

Eugene Ghanizadeh — Tue, 05 Jan 2021 14:45:33 +0000

Recently I've built a little library, called themed-jss for theming my web apps. It enables straightforward and flexible theming, and more importantly it automatically takes care of all additional CSS rules (and JS) needed for proper dark mode support.

In this post I will explain the problem I was trying to solve and then the solution (i.e., the lib, themed-jss). Hope you find it useful in your web apps as well.

Theming

A theme is a set of recurring values used in your web apps style. For example, all buttons might be blue, all text might be black and the general background of the app might be white:

const myTheme = {
  buttons: 'blue',
  text: 'black',
  background: 'white'
}

It is useful to keep all of this in one place. Imagine for example that you decide to change the color of all buttons to green instead, if there is no central theme, you need to go through all of your app and update styles.

Darkmoding

Dark mode support is (or should be) just specifying an additional theme for your app (how the colors should change in dark mode):

const myTheme = {
  button: 'blue',
  background: 'white',
  color: 'black'
}

const myDarkTheme = {
  ...myTheme,
  background: 'black',
  color: 'white'
}

Unfortunately though, in practice you need more work to add dark mode support to your web app. To get started, you need some media queries:

const myStyles = {
  button: {
    background: myTheme.background,
    '@media (prefers-color-scheme)': {
      background: myDarkTheme.background
    }
  }
}

☝️ I typically use JSS for styling my web apps. It provides style isolation, nested styling, etc. out of the box, and I am particularly fond of it alongside JSX (so everything is in JS).

Things become more complicated when you consider the fact that you cannot rely on system settings for dark mode as some operating systems do not have such a setting.

A solution I often use myself is adding a button that allows the user to override dark mode settings. By default, the setting is read from system (media query), but if the user overrides that, then the preference will be stored in local storage and applied as a CSS class:

const myStyles = {
  button: {
    background: myTheme.background,
    '@media (prefers-color-scheme)': {
      background: myDarkTheme.background
    },
    'html.--dark &': {
      background: myDarkTheme.background
    }
  }
}

☝️ This code is actually not complete: if a user whose OS is in dark mode changes preference to light mode, this styling rule would still display in dark mode for them. To fix that, we can add a separate class indicating whether OS dark mode is being overridden or not (regardless of its value):

const myStyles = {
  button: {
    background: myTheme.background,
    '@media (prefers-color-scheme)': {
      'html:not(.--dark-mode-override) &': {
        background: myDarkTheme.background
      }
    },
    'html.--dark &': {
      background: myDarkTheme.background
    }
  }
}

This is a lot of boilerplate for a simple style. The solution? Well if my styles where functions of a theme instead of plain values, i.e.:

const myStyles = theme => {
  button: {
    background: theme.background
  }
}

Then I could run the styles for the two themes I have (light mode and dark mode), and insert additional CSS rules for properties that differ automatically:

const stylesInLight = myStyles(myTheme)
const stylesInDark = myStyles(myDarkTheme)

const D = diff(stylesInLight, stylesInDark)
createAndInjectRules(D)

`themed-jss`

What I described here is basically what themed-jss does. It allows defining styles in terms of functions of themes, and based on that it automatically creates additional styles for dark mode.

Here is a React example:

// my-btn.style.js

import { style } from 'themed-jss'

export default style(theme => ({
  btn: {
    background: theme.primary,
    color: theme.background,
    border: 'none',
    borderRadius: 3
  }
})

// my-btn.jsx

import React from 'react'
import { useThemedStyle } from 'themed-jss/react'

import styles from './my-btn.style'

export default () => {
  const { btn } = useThemedStyle(styles)

  return (
    <button className={btn}>Click ME!</button>
  )
}

This component can be used like this:

// app.jsx

import { theme } from 'themed-jss'
import { Themed } from 'themed-jss/react'

import MyBtn from './my-btn'

const myTheme = theme(
  {
    primary: 'green',
    background: 'white',
    text: 'black'
  },
  // --> dark mode overrides:
  {
    background: 'black',
    text: 'white'
  }
)

export default () => (
  <Themed theme={myTheme}>
    <MyBtn/>
  </Themed> 
)

For manual control of dark mode, I would simply need to import DarkMode from themed-jss/dark-mode and initialize it:

// app.jsx

import { DarkMode } from 'themed-jss/dark-mode'

DarkMode.initialize()

// ...

And now I can have <MyBtn/> to actually switch the dark mode:

// my-btn.jsx

import { DarkMode } from 'themed-jss/dark-mode'

// ...

export default () => {
  const { btn } = useThemedStyle(styles)

  return (
    <button
      className={btn}
      onClick={() => DarkMode.toggle()}>
      Click ME!
    </button>
  )
}

👉 Try it out for yourself!

Anyways, I hope you would find themed-jss useful as well. You can find the complete docs in the readme, and comment here (or create an issue on GitHub, etc) if you've got any further questions. This is a pretty young tool, so any feedback is much appreciated!

How About DX-Driven Architecture?

Eugene Ghanizadeh — Tue, 15 Dec 2020 12:24:18 +0000

People have different, often subjective opinions of what good software architecture is. We want it to be elegant. We want it to be intuitive. Most of the time, we want it to feel good, without knowing what that means in terms of measurable metrics.

IMHO, software architecture is solely a scaling problem. Good architectural design, pragmatically, ensures that multiple developers will be able to conveniently work on the same code base, without screwing things up left and right. In other words, when you have good design:

💡 One can easily find out how to make changes.
🚀 One can make those changes efficiently.
☂️ Performing those changes doesn't cause errors (often).
🧠 The code structure and conventions are memorable.

Of course, a good architectural design has other measurable properties as well, but I am focusing on these for a reason: they perfectly mirror the main pillars of Usability. If you are not familiar with the term, Usability is about ensuring a system (e.g. interface, app, etc.) is usable (people can do stuff with it). It is the cornerstone of UX, so much so that the two terms are often used interchangeably. Usability is mainly composed of the following:

💡Learnability:
How fast/easy is it for users to learn what can they do?

🚀 Efficiency:
How fast can users perform tasks?

☂️ Error-Rate:
How often do users encounter errors?

🧠 Memorability:
How easy is getting back to the system/software?

As you can see, good architecture implies good UX, where our users (the U) are developers working on some code base, and the use-case (the experience, the X) is maintaining / updating the code. Since those users are developers and their usage is developing, it is safe to use the term DX (Developer Experience) here, which means:

Good architecture provides good DX for maintainers.

How does that help? Well, we know how to measure and evaluate UX/DX pretty well, and by looking at software architecture as a UX/DX problem, we can apply the same tools and measurements and make practical and data-driven decisions.

Example: Change Propagation

Imagine we have this function:

export function doSomething(user: User, opts: Options) {
  ...
}

And we need to change the function signature so that it also accepts a boolean flag, e.g.:

export function doSomething(
  user: User,
  opts: Options,
  really: boolean
) {
  ...
}

Beyond the change itself, we need to update every other code module (segment, unit) that is using doSomething(), which might look like this:

Here we would have to modify 8 other modules for making the intended change. We could make this change more efficient if we had less coupling between different modules, for example by introducing intermediary modules/wrappers:

For example, the three modules on the left might all use doSomething() with the same set of options, so our intermediary module could look something like this:

import { doSomething() } from '...'

export function doSomethingWithTheseOpts(user: User) {
  const opts = { ... }
  return doSomething(user, opts)
}

And our intended change would be isolated to this:

  return doSomething(user, opts, false)

Note that in the second architectural design, we have more modules, but making changes such as our example is easier. If the frequency of these changes is far less than frequency of creating new functions similar to doSomething(), then the overhead of the second architecture might be more than its benefits.

Generally speaking, lowering the coupling between different modules limits the propagation of any particular change, which in turns makes it more efficient.

In this design, a change to module A results in 6 other modules being changed. But with lowering the coupling (and increasing cohesion), we can limit it to 2:

This means for good architectural design, generally speaking, we should try to identify most probable changes, and then increase cohesion and reduce coupling in a way that would isolate the propagation of those changes.

Example: Layered Structures

Imagine a frontend project with the following layered structure:

For changes that affect all things related to A (marked in red), we would need to constantly navigate between all A files. This interaction is made particularly difficult by the distance these files have from each other.

Detailed analysis of this navigation interaction depends on the environment setup (e.g. the IDE's file explorer), but to see how the distance affects the difficulty of the interaction, we can assume the simple interaction of opening two of the files subsequently, i.e. clicking on one and then clicking on the other. The difficulty of this interaction is governed by Fitts's law, i.e.:

ID = log(2 * D / W)

👉 ID is index of difficulty, which is proportional to time

👉 D is the distance that the cursor needs to travel

👉 W is the width of the clicked element

In our case, W is determined by the environment (e.g. the width of the IDE file explorer), so the only parameter that can be optimized is D, i.e. the distance of the files. Since index of difficulty scales logarithmically based on distance, the only meaningful optimization here would be to have really small Ds, i.e. if the files were adjacent, for example in a structure like this:

Both structures do have cohesion: the first one has logical cohesion (stuff that do the same logical thing are put alongside each other), the latter eventually will arrive at functional cohesion (stuff that are about the same task are put alongside each other).

👉 Functional cohesion is generally costlier than logical cohesion to achieve, as the latter only requires logically bundling units (e.g. everything related to DB, everything related to HTML) while the former requires a proper conceptual model of the project.

DX-Driven approach allows for a cost-efficient, data-driven, iterative way of reaching functional cohesion: observe which parts of the code change together (under the umbrella of one task) more often, and reduce their relative distance.

Example: Mental Models

So far, we've only focused on the efficiency(🚀) aspect of
DX. Mental models and design patterns start to play a role when we start to consider learnability(💡) and memorability(🧠) aspects as well.

What is a mental model / design pattern? MVC is a famous example, where you imagine some modules as responsible for data (i.e. Model), some others for how the data looks (i.e. View), and others for how data is updated (i.e. Controllers). Instead of controllers, you might assume modules whose job is to translate business data to presentable data and vice-versa, which would give you MVVM, and so on.

An important function of these mental models is to facilitate learning and recalling the design. A common pitfall is to try to evaluate them by how intuitive they are (or conversely, pursuing intuitive mental models and design patterns). The exact same pitfall exists in UX land, as designers pursue the mythical intuitive design, where the interface is so learnable that users know it by heart.

The solution here is also similar: the best practical way for minimizing the learning curve is to draw on things that people already have learned. In other words, a pragmatic approach to intuitiveness is pursuing familiarity instead, both in UX, DX and in architectural design.

For example: the best mental model for a particular project might be an actor model. However, if your developers are familiar with MVC, and if the overhead of modeling the project with MVC is not that higher than using the actor model, you would be better off using MVC.

Conclusion

DX-Driven Architecture (DXDA, if I may 😎) is NOT a ground-breaking, flashy, new design pattern or anything. It is a method of pragmatically contextualizing already existing patterns (e.g. MVC), techniques (e.g. access abstraction) and metrics (e.g. coupling or cohesion), by considering the experience of the maintainers. This allows for making quantitative and data-driven decisions about software architecture and evaluating those decisions empirically.

As mentioned earlier, it is notable that DXDA does not offer an all-encompassing perspective. Good design has properties that are not covered by translating UX principles to developers maintaining a project (i.e., DX), at least not trivially. This means blindly optimizing DX will also not lead to the best architecture, as other goals and concerns should also be considered.

All-in-all though, I feel DXDA provides a really solid basis for thinking about software architecture practically and pragmatically. Whatever our subjective notion of ideal architecture and design is, I think it is universally agreeable that it MUST reduce maintenance/improvement cost, which logically entails that optimizing DX is always a key component of good architectural design, whether we actively consider it or not.

Type Definitions as API Specification

Eugene Ghanizadeh — Tue, 08 Dec 2020 13:49:31 +0000

I hate writing client/server code. Why? Look at this example:

export interface User {
  name: string
}

export async function greet(user: User) {
  await doSomeStuff(user)
  return `Hellow ${user.name}`
}

And this client code using it:

const jack: User = { name: 'Jack' }
greet(jack).then(console.log)

If my greet() function is executed in the client environment, then I just need to import it and I'm done. Everything works perfectly, I get proper type-checking, etc. But if it has to be executed on the server environment, then I would need to add this network layer boilerplate-ish to the server to make it work:

import express from 'express'
import cors from 'cors'

import { greet } from './my-func'

const app = express()
app.use(cors())

app.post('/greet', async (req, res) => {
  const user = JSON.parse(req.query.user)
  const response = await greet(user)
  res.status(200).send(response)
})

app.listen(4000)

And this boilerplate-ish to the client:

export interface User {
  name: string
}

export async function greet(user: User) {
  const stringified = JSON.stringify(user)
  const encoded = encodeURIComponent(stringified)
  const response = await fetch(
    `https://my-server:4000/greet?user=${encoded}`,
    { method: 'POST' }
  )

  return await response.text()
}

The problem is not just that this is a lot of boilerplate, but also:

It is boilerplate that also introduces (and depends on) TONs of arbitrary decisions. The http method is one example (is it / was it / should it be POST / GET / PUT?), the URL is another one, as is the place where the parameters are put (the body, the query parameters, the URL itself, etc).
I've lost any meaningful type-checking now. I am maintaining two versions of User interface and two definitions of greet() function that I need to keep in sync manually.

Sharing Types

If we look at the main server code (where greet() and User are defined) and frontend boilerplate code, we can see that these two files have identical types. The body of greet() differs between them, but all type declarations are exactly the same (TypeScript would generate the same .d.ts file for both):

export interface User {
  name: string
}

export declare function greet(user: User): Promise<string>

What if we could share this type definition between server and client? This way, we would have a single source of truth for our types, while the client would get seamless type-checking.

How could we share type definitions? Well, people coding in TypeScript need to use lots of pure JavaScript libraries without loosing type checking, so TypeScript allows adding independent type definitions alongside JavaScript code. This means we can have different versions of our functions (e.g. greet()) to actually share their type definitions, which is exactly what we need since our functions, though identical in type, need to behave differently on the server and on the client.

This would mean that we would need to write the frontend network layer code in JavaScript, then extract type definition files from backend code and set them alongside each other. It would resolve the type checking issue, but introduce the problem of manually maintaining a JavaScript code that needs to be in sync with those type definitions.

Auto-Generating Boilerplates

Well what if we could auto-generate the JavaScript code of the frontend boilerplate as well? If written in pure JavaScript, this boilerplate would look like this:

function greet(user) {
  const stringified = JSON.stringify(user)
  const encoded = encodeURIComponent(stringified)
  const response = await fetch(
    `https://my-server:4000/greet?user=${encoded}`,
    { method: 'POST' }
  )

  return await response.text()
}

module.exports = { greet }

To write this code, we would need to know the following (and nothing more):

The name of the function greet()
The URL of the corresponding endpoint
The http method of the corresponding endpoint
Where parameters should be injected (request body, header, url, query parameters, etc)

Note that the last 3 are the exact same problematic arbitrary choices we encountered in problem #1. Since the choices are (mostly) arbitrary, we could just decide on them based on the only non-arbitrary parameter here, i.e. the function name. For example, we could follow this convention:

👉 If the function name is "getX()": 
  - the URL would be "/x"
  - the method would be GET
  - parameters would be in query params


👉 If the function name is "updateX()":
  - the URL would be "/x"
  - the method would be PUT
  - parameters would be in request body


👉 If the function name is "createX()":
  - the URL would be "/x"
  - the method would be POST
  - parameters would be in request body


👉 If the function name is "x()":
  - the URL would be "/x"
  - the method would be POST
  - parameters would be in request body

This means that knowing only the names of the functions and assuming they follow this convention, we could fully auto-generate the client-side boilerplate.

The backend boilerplate would also need to strictly follow this convention. Fortunately, that code can also be fully auto-generated knowing the names of the functions and following the same convention:

import express from 'express'
import cors from 'cors'

import { greet } from './my-func'

const app = express()
app.use(cors())

app.post(               // --> from the convention
  '/greet',             // --> from the convention
  async (req, res) => {
    const user = JSON.parse(req.body.user) // --> from the convention
    const response = await greet(user)     // --> from function name
    res.status(200).send(response)
  }
)

app.listen(4000)

Putting Everything Together

Let's recap a bit:
👉 Typical client/server code is problematic because:

It has lots of boilerplate code with arbitrary decisions in it
It takes type-checking away

👉 To fix that:

We can share type definitions
We can auto-generate client network layer boilerplate knowing function names and following some convention
We can auto-generate server network layer boilerplate knowing function names and following some convention

All of these fixes rely on knowing the name of the server functions we want to use on the client-side. To fix that issue, lets add another rule: we will export all such functions from index.ts on our server-side code, and our client/server code will be reduced to the following:

// server/index.ts
export interface User {
  name: string
}

export async function greet(user: User) {
  await doStuff(user)
  return `Hellow ${user.name}`
}

// client code
import { greet } from '<auto-generated-code>'

const jack: User = { name: 'Jack' }
greet(jack).then(console.log)

Will this really work? Well I have actually built a CLI tool that does exactly what I've described here to find out. You can try it out for yourself:

👉 Install the CLI tool:

npm i -g tyfon

👉 Create a folder for server-side code:

mkdir test-server
cd test-server
npm init

👉 Add the server code:

// test-server/index.ts
export interface User {
  name: string
}

export async function greet(user: User) {
  return `Hellow ${user.name}`
}

👉 Run the server:

tyfon serve

🚀 You can already try out your server:

curl -d '{"0":{"name":"Jack"}}' -H "Content-Type: application/json" -X POST localhost:8000/greet

👉 Create a folder for the client-side code (in another terminal, keep the server running):

mkdir test-client
cd test-client
npm init
npm i -g ts-node # --> if you don't have ts-node

👉 Autogenerate network boilerplate:

tyfon i localhost:8000

👉 Add the client code:

// test-client/index.ts
import { User, greet } from '@api/test-server'

const jack: User = { name: 'Jack' }
greet(jack).then(console.log)

🚀 Try it out:

ts-node .

Observations

Although the TyFON CLI tool is pretty young and the concept of using type definitions as API specification is new (at least to me), I've been using it in real-life projects for some time now, and like everything else, there are pros and cons to this approach:

Pros

Cleaner Code: I write simple functions on the server, and I call them on the client-side. The network layer (and all its boilerplate and hassle) completely vanishes.
Strong Type Checking: When I make changes to server code that require changes in the client, my IDE will tell me, or when I want to call some server function, I don't need to go check a list of API URLs, the IDE suggests to me all the functions at my disposal.
Single Source of Truth: All my data types are now defined in the server and seamlessly used in the client.

Cons / Oddities

No Network Layer Access: The down-side of completely masking the network layer is that you won't have access to the network layer. This means right now I cannot put stuff in request headers or handle file uploads, though I've got some ideas of for tackling that issue
No Middleware: I was used to Express middlewares that worked in tandem with Angular interceptors, to for example make authentication happen behind the scenes. Without any access to the network layer, all of that is gone as well, which means I have to explicitly pass auth tokens around now.
New Security Concepts: Now I need to consider whether a server function is to be used internally by other functions or can it be safely used over network as well.

All in all, I am pretty happy with the early results of this approach. Of course as with anything new there are downsides and stuff that I would need to get used to, but the increase in development speed (and my confidence in the generated code) is so much that I will happily make that exchange for all possible future projects.

From Callback Hell to Callback Heaven

Eugene Ghanizadeh — Fri, 04 Dec 2020 17:09:16 +0000

Remember the time when a lot of JavaScript code looked like this?

router.put('/some-url', (req, res) => {
  fs.readFile(filePath(req), (err, file) => {
    if (err) res.status(500).send()
    else {
      parse(file, (err, parsed) => {
        if (err) res.status(500).send()
        else db.insert(parsed, err => {
          if (err) res.status(500).send()
          else res.status(200).send()
        })
      })
    }
  })
})

For those fortunate enough to not remember those days, this was called a callback hell, for obvious reasons. Fortunately, we have moved on, and these days equivalent code most likely looks something like this:

router.put('/some-url', async (req, res) => {
  try {
    const file = await fs.readFile(filePath(req));
    const value = await parse(file);
    await db.insert(value);
    response.status(200).send();
  } catch {
    response.status(500).send();
  }
})

Of course now that we have async/await and Promises, it is easy to blame the callback hell era on lack of syntactic features of JS at the time and move on. But I do think there is value in reflecting back, analyzing the core issue, how it was solved, and what's to learn from it all.

The Issue

Lets look back at the overall structure of the hellish example above:

doX(args, (err, res) => {
  if (err) { ... }
  else {
    doY(args, (err, res) => {
      if (err) { ... }
      ...
    })
  }
})

A glaring issue here is that most of whats on the screen is dedicated to not-really-important stuff:

doX(args /*, (err, res) => {
  if (err) { ... }
  else {*/
    doY(args /*, (err, res) => {
      if (err) { ... } */
      ...
    /*}*/)
  /*}*/
/*}*/)

For comparison, this would be the equivalent structure for the modern not-hellish version:

/* try { */
  /*await*/ doX(args)
  /*await*/ doY(args)
  ...
/*} catch { }*/

The commented bits in both versions are indicative of the same things: doX() and doY() are async functions, and also there might be some errors. In the hellish version though, you need to spend much more space for these side notes, which results in a far less readable code.

👉 Note that we could trim the boilerplate and restructure the code into a more readable format without additional syntax too. Historically speaking, that IS what happened, in the form of Promise libraries (which then got standardized and a bit of more love with some syntactic support):

doX(args)
.then(() => doY(args))
.then(() => ...)
.catch(() => { ... })

doX(args)
/*.then(() =>*/doY(args)/*)*/
/*.then(() =>*/.../*)*/
/*.catch(() => { ... })*/

An important difference between this code and the hellish code is that in the hellish code, important stuff and boilerplate stuff are extremely intertwined, while with promise libraries, they are neatly separated, making the code easier to read even in cases where amount of boilerplate is almost the same:

// without promises:
doX(args/*, (err, res) => { ... }*/)

// with promises:
doX(args)/*.then(() => { ... })*/

// with async/await:
/*await*/ doX(args)

Promises also provide other important features that help with async programming ergonomics, most importantly:

Promises are automatically flattened when chained.
Promises are shared.

However, I think these properties, although beneficial, aren't as important as the aforementioned separation. To illustrate that, lets create an experimental promise library that just does the separation and nothing else, and see how it fares.

The Experiment

So initially, we started with functions looking like this:

doX(args, (err, res) => {...})

The callback here is the main boilerplate (and namesake of our hell), so easiest separation is to take it out of argument list of doX(), and put it in a deferred function instead:

doX(args)((err, res) => {...})

☝️ This is basically a change in how doX is implemented, from this:

function doX(args, callback) {
  // do stuff
  // maybe do more
  callback(undefined, 42)
}

To this:

function doX(args) {
  // do stuff
  return callback => {
    // maybe do more
    callback(undefined, 42)
  }
}

In other words, we just changed the convention from:

accept a callback as last argument

to:

return a function that accepts a callback as argument

Our separation convention seems to not have helped much on its own, as we still have the same amount of boilerplate. However, it did open the door for simplistic utilities that help us take away the boilerplate. To see that, let me first introduce the pipe() utility:

function pipe(...cbs) {
  let res = cbs[0];
  for (let i = 1; i < cbs.length; i++) res = cbs[i](res);
  return res;
}

Simply put, this:

pipe(a, b, c, d)

equals to this:

let x = a
x = b(x)
x = c(x)
x = d(x)

In the not so distant future, pipe() might even get incorporated into JavaScript itself, which would look like this:

a |> b |> c |> d

Anyways, the pipe() operator allows us to neatly transform the function returned by the (new convention) doX() (which remember, is a function accepting a standard callback), without having to write callbacks manually. For example, I could create a then() utility as follows:

export function then(f) {
  return src => {
    src((err, res) => {
      if (!err) f(res)
    })

    return src
  }
}

and with these utilities, my async code would transform from this:

doX(args)((err, res) => { ... })

to this:

pipe(
  doX(args),
  then(() => { ... })
)

or better yet (with pipeline operator incorporated):

doX(args) |> then(() => { ... })

which looks a lot like a standard promise library:

doX(args).then(() => { ... })

I could also create a simple catch() utility:

function catch(f) {
  return src => {
    src((err) => {
      if (err) f(err)
    })

    return src
  }
}

Which would give me async codes like this:

doX(args)
|> then(() => doY(args))
|> then(() => ...)
|> catch(() => { ... })

doX(args)
/*|> then(() =>*/ doY(args) /*)*/
/*|> then(() =>*/ ... /*)*/
/*|> catch(() => { ... })*/

Which is as succinct as promise libraries, with almost no effort. Better yet, this method gives us extensibility as well, as we are not bound to a set Promise object and can create / use a much wider range of utility functions:

function map(f) {
  return src => cb => src((err, res) => {
    if (err) cb(err, undefined)
    else cb(undefined, f(res))
  })
}

function delay(n) {
  return src => cb => src((err, res) => {
    if (err) cb(err, undefined)
    else setTimeout(() => cb(undefined, res), n)
  })
}

and start getting a bit wild:

doX(args)
|> then(() => doY(args))
|> map(yRes => yRes * 2)
|> delay(200)
|> then(console.log)

Real-life Examples

Ok so it seems like a simple change of convention allowed us to create utilities and libraries that provide the same convenience provided by promise libraries (and almost similar to async/await syntax). To get a better perspective, let's look at real-life examples. For this purpose (and mostly out of curiosity), I have created an online playground with an implementation of our experimental lib.

First off, let's take a look at our original example, which looked like this in its most hellish version:

router.put('/some-url', (req, res) => {
  fs.readFile(filePath(req), (err, file) => {
    if (err) res.status(500).send()
    else {
      parse(file, (err, parsed) => {
        if (err) res.status(500).send()
        else db.insert(parsed, err => {
          if (err) res.status(500).send()
          else res.status(200).send()
        })
      })
    }
  })
})

This is how the modern JavaScript version looks like:

router.put('/some-url', async (req, res) => {
  try {
    const file = await fs.readFile(filePath(req));
    const value = await parse(file);
    await db.insert(value);
    response.status(200).send();
  } catch {
    response.status(500).send();
  }
})

And this is how our new callback convention code looks like:

router.put('/some-url', (req, res) => {
  fs.readFile(filePath(req))
  |> map(parse)
  |> flatten
  |> map(db.insert)
  |> flatten
  |> then(() => res.status(200).send())
  |> catch(() => res.status(500).send())
})

The convention gets us pretty close to the convenience of async/await. There is a little nuance though: see the flatten utility used twice in the middle? That is because unlike promises, our callbacks are not flattened while chaining. We assumed that parse() is also async, i.e. it also returns a promise-ish. map(parse) then maps the result of readFile() to a new promise-ish, which should be flattened to resolved values before being passed to db.insert(). In the async/await code, this is done by the await keyword before parse(), and here we need to do it with the flatten utility.

P.s., the flatten() utility is also pretty simplistic in nature:

function flatten(src) {
  return cb => src((err, res) => {
    if (err) cb(err, undefined)
    else res((err, res) => {
      if (err) cb(err, undefined)
      else cb(undefined, res)
    })
  })
}

Let's also take a look at another example: here, we want to fetch some Pokémon info from PokéAPI and log its abilities:

fetch('https://pokeapi.co/api/v2/pokemon/ditto')
|> map(res => res.json())
|> flatten
|> then(res => console.log(res.abilities))

async(() => {
  let res = await fetch('https://pokeapi.co/api/v2/pokemon/ditto')
  res = await res.json()
  console.log(res.abilities)
})()

Conclusion

So to recap, these seem to have been the main issues resulting in callback hells:

Lots of boilerplate code
Boilerplate code severely intertwined with important code

As per our little experiment, addressing the second issue in the simplest manner (just separating boilerplate code and important code with no other change) was pretty key: it allowed us to bundle boilerplate code into small utility functions and reduce the ratio of boilerplate code and important code, making it (almost) as convenience as a heavy-handed solution such as adding new syntax to the language itself.

This notion is particularly important: you might have ugly implementation details and boilerplate that you cannot get rid of, but you can always bundle it together and separate it from actual important code, and doing this even in the simplest manner opens the door for turning a hellish situation into a heavenly one.

It is also notable that the same methodology is applicable to other, similar problems we are facing today. While we have (mostly) solved the issues of asynchronous functions, newer constructs such as asynchronous streams (which are like async functions but with many, possibly infinite outputs instead of one) keep creeping into our toolbox and demand similar problem solving.

P.s. the name callback heaven actually comes from the callbag specification, which is like our new callback convention but for streams instead of promises. If you enjoyed this post, be sure to check it out as well.

What are Callbags?

Eugene Ghanizadeh — Wed, 02 Dec 2020 17:29:39 +0000

TLDR, A callback is a function we pass to another, source function, in order to give the source function a way to talk back to us (call us back). Callbag is a standard for callbacks that enables working with streams. A callbag is any callback that follows that standard.

Callbacks

Take this code:

console.log(source());

When we run this code:

source() is called. We wait for its output.
We log the output from source().

What if source() takes some time to produce data? Instead of waiting for it, we could tell it to "give us data" (by calling it), and give it a way to call us back when it has some data:

source(data => console.log(data));

Here, data => console.log(data) is a callback, as its the method we provide source() to call us back when it has data. source is basically a source of data, and we can communicate with it as follows:

-> [GREETING] "Give me data when you have it" # --> us to source
<- [DATA]     "Here is some data"             # --> source to us

Streams

Now what if our source (e.g. source()) produces an indeterminate number of data entries? For example, our source might be a function responsible for calculating the position of the cursor on X-axis, or it might be a function who is supposed to give us messages coming from a web-socket.

👉 A source that produces an indeterminate number of data entries at indeterminate time intervals is called a stream.

In this case our simplistic callback (or the communication scheme) will be rather limiting:

We might want to tell the source to stop sending more data.
The source might want to tell us that it won't send more data (maybe due to some error).
Some sources push data whenever possible. Others might wait for us to ask them explicitly for more data, in which case we would need to be able to ask for more data as well.

None of these are available under our previous communication scheme, and we need an expanded communication scheme to be able to work properly with streams:

-> [GREETING] "Give me data whenever you have some"                              # --> us to source
<- [GREETING] "I will give you data whenever I have some. Tell me when to stop"  # --> source to us

<- [DATA]     "Here is some data"                                                # --> source to us
-> [DATA]     "Give me more data"                                                # --> us to source, when it needs to be pulled

-> [END]      "Stop sending more data"                                           # --> us to source
<- [END]      "I won't be sending more data (because of X)"                      # --> source to us

To accomodate it we can have our callback accept two arguments instead of one: The first argument denoting the type of the message, the second one denoting the content (or payload):

source((type, payload) => {
  if (type === GREET) console.log('Listening ...');
  if (type === DATA) console.log(payload);
  if (type === END) console.log('Source ended!');
});

Callbags

Callbag is just a term to denote any callback (or function) that looks like the callback we just designed for talking with source(). In other words, any function with the following signature is a callbag:

(type: GREET | DATA | END, payload?: any) => void;

👉 In the callbag spec, message types are denoted by numbers:

0 stands for GREET (also called START)

1 stands for DATA

2 stands for END.

Now lets look at the above example again:

source((type, payload) => {
  if (type === GREET) console.log('Listening ...');
  if (type === DATA) console.log(payload);
  if (type === END) console.log('Source ended!');
});

Here, source is NOT a callbag, since it only accepts one argument. We can fix that by making source accept two
arguments as well, in which case our code would change like this:

source(GREET, (type, payload) => {
  if (type === GREET) console.log('Listeing ...');
  if (type === DATA) console.log(payload);
  if (type === END) console.log('Source ended!');
});

Now what if we want to receive a limited number of data entries (say 5) from source? We greeted source by calling it with GREET alongside a callbag. According to our communication scheme, source needs to also greet us by sending us GREET alongside a way to tell it to stop, i.e. another callbag:

let talkback;
let N = 0;

source(GREET, (type, payload) => {
  if (type === GREET) {
    talkback = payload;                // --> when type is GREET, payload is a callbag
    console.log('Listening ...');
  }

  if (type === DATA) {
    console.log(payload);             // --> when type is DATA, payload is the data sent by source
    N++;
    if (N >= 5) talkback(END);        // --> telling the source to stop
  }

  if (type === END) console.log('Source ended!');
});

👉 So whenever someone greets someone (us greeting the source, the source greeting us), the payload should be another callbag, acting as a way to talk back to the greeter. In this example, talkback plays that role.

Callbag Sources

So far we've just worked with source() as a stream, without looking inside it. But how would a callbag source actually look like? To see that, lets build a simple callbag source that outputs an increasing number every second:

const source = (type, payload) => {
  if (type === GREET) {                           // --> everything starts with a GREET
    let talkback = payload;                       // --> when greeted, the payload is a way to talk back to the greeter
    let i = 0;

    setInterval(() => talkback(DATA, i++), 1000); // --> lets tell the greeter about our increasing number every second
  }
}

☝️ Here, we are not giving the caller any method of telling the source to stop sending more data. This is because we are not following the communication protocol properly: the source MUST greet back and provide a way of talking back (i.e. another callbg):

const source = (type, payload) => {
  if (type === GREET) {
    let talkback = payload;
    let i = 0;

    const interval = setInterval(() => talkback(DATA, i++), 1000);

    talkback(GREET, (_type, _payload) => {
      if (_type === END) clearInterval(interval);
    });
  }
}

► Try It!

Callbags in Practice

In practice, you rarely need to greet sources or handle talkbacks manually. Utilities such as those provided in callbag-common take care of that for you:

import { interval, pipe, map, filter, subscribe } from 'callbag-common'

const source = interval(1000) // --> emits every second
pipe(
  source,
  map(x => x * 3),            // --> multiply by 3
  filter(x => x % 2),         // --> only allow odd numbers
  subscribe(console.log)      // --> log any incoming number
)

> 3
> 9
> 15
> 21
> 27

► Try It!

The workflow is typically like this:

👉 You create some callbag sources, using source factories:

import { interval } from 'callbag-common';

const source = interval(1000);

👉 You then transform these sources using operators.
For example, you might want to multiply each received number by 3:

import { interval, map } from 'callbag-common';

let source = interval(1000);
source = map(n => n * 3)(source);

Or you might want to only pick odd numbers:

import { interval, map, filter } from 'callbag-common';

let source = interval(1000);
source = map(n => n * 3)(source);
source = filter(n => n % 2)(source);

👉 Finally, you start listening to your transformed source by subscribing to it:

import { interval, map, filter, subscribe } from 'callbag-common';

let source = interval(1000);
source = map(n => n * 3)(source);
source = filter(n => n % 2)(source);

subscribe(console.log)(source);

👉 It is also highly recommended to use the pipe() utility for transforming your sources and subscribing to them, as it makes the code much easier to read:

import { interval, map, filter, subscribe, pipe } from 'callbag-common';

pipe(
  interval(1000),
  map(n => n * 3),
  filter(n => n % 2),
  subscribe(console.log)
)

Why Medium Doesn't Work for Programmers

Eugene Ghanizadeh — Mon, 08 Jun 2020 14:51:11 +0000

https://eugene.coding.blog/why-medium-doesnt-work-for-programmers

Codedoc: Easily create beautiful and modern docs/wiki for your software projects

Eugene Ghanizadeh — Wed, 22 Apr 2020 16:39:29 +0000

https://codedoc.cc

Looking for Contributors on CONNECTIVE HTML

Eugene Ghanizadeh — Wed, 05 Feb 2020 13:50:50 +0000

CONNECTIVE HTML is a simple and thin library for creating HTML user interfaces without reliance on (and hence without overhead of) techniques such as Virtual DOM / Change Detection / etc. The idea is that with advent of reactive programming libraries such as RxJS, not only there is no longer any need these techniques, but also relying on them limits our interfaces in terms of performance and capabilities.

The main body of the work on CONNECTIVE HTML is already done and it is already usable for enthusiasts. However there still remains some work to make it widely usable and production-ready. If you are interested, checkout the project's page and contribute, as any help is much appreciated.

A Pragmatic Approach to Software Architecture

Eugene Ghanizadeh — Wed, 06 Nov 2019 14:10:06 +0000

Or on the virtues of measuring good architecture instead of passionately fighting about it https://medium.com/connect-platform/a-pragmatic-approach-to-software-architecture-ea51df8a0dd3

Taking RxJS to next level

Eugene Ghanizadeh — Wed, 28 Aug 2019 10:10:34 +0000

Hey fellas, I have recently started to work on a thin layer on top of RxJS, that helps organize your reactive codes and adds abstractions for reusability. Check it out and tell me what you think!

DEV Community: Eugene Ghanizadeh

Reactive Programming with React

Reactive Streams

Reactive Streams in React

Other Streaming Operations

Conclusion

Further Reading

Auto Dark Mode w/ JSS

Theming

Darkmoding

themed-jss

How About DX-Driven Architecture?

Example: Change Propagation

Example: Layered Structures

Example: Mental Models

Conclusion

Type Definitions as API Specification

Sharing Types

Auto-Generating Boilerplates

Putting Everything Together

Observations

Pros

Cons / Oddities

From Callback Hell to Callback Heaven

The Issue

The Experiment

Real-life Examples

Conclusion

What are Callbags?

Callbacks

Streams

Callbags

Callbag Sources

Callbags in Practice

Why Medium Doesn't Work for Programmers

Codedoc: Easily create beautiful and modern docs/wiki for your software projects

Looking for Contributors on CONNECTIVE HTML

A Pragmatic Approach to Software Architecture

Taking RxJS to next level

`themed-jss`