DEV Community: Varenya Thyagaraj

Async React with Suspense

Varenya Thyagaraj — Sun, 02 Jun 2024 08:02:53 +0000

As React devs we deal with async stuff every day. We know firsthand how awkward and complicated async logic can get.

Now add components into the mix it only gets more complicated.

Suspense was the solution that React came up with to solve for that. Announcement of Suspense happened in 2018. It took a while to stabilize. Finally, with React 19, we will have all the pieces to use it.

The most fascinating thing about Suspense was the use of throw. Yes, you read it right!

Suspense uses throw statements,
intended for error handling, as a control flow mechanism.

Yup, 🤯.

Before we dive into the heavy stuff, let's review what async JS is:

Async JavaScript

Let's take this async function:

function fetchText({ text, delay }: DelayText): Promise<string> {  
  return new Promise<string>((resolve) =>  
    setTimeout(() => {  
      resolve(text);  
    }, delay),  
  );  
}

It takes in two parameters:

a text — which will be returned after a delay
a delay — a specified duration after which the text is returned

How would we go about consuming such a function?

Variant 1:

import { fetchText } from "./sample.ts";  

async function getTexts(): Promise<[string, string]> {  
  const text1 = await fetchText({ text: "hello 1", delay: 1000 });  
  const text2 = await fetchText({ text: "hello 2", delay: 2000 });  
  return [text1, text2];  
}  

console.time("gettingTexts");  
getTexts().then((res) => {  
  console.log({ res });  
  console.timeEnd("gettingTexts");  
});

If you run this on say node, you should see an output like this:

{
  res: [ "hello 1", "hello 2" ]
}
[3.01s] gettingTexts

If you notice here, I have used console.time that prints out the time taken to execute.

Which in the above case is approximately 3 seconds, i.e., (1 sec for text1) + (2 seconds for text2)

Can we do better?

Variant 2:

import { fetchText } from "./sample.ts";  

async function getTexts(): Promise<[string, string]> {  
  const text1 = fetchText({ text: "hello 1", delay: 1000 });  
  const text2 = fetchText({ text: "hello 2", delay: 2000 });  
  return [await text1, await text2];  
}  

console.time("gettingTexts");  
getTexts().then((res) => {  
  console.log({ res });  
  console.timeEnd("gettingTexts");  
});

By making a small change i.e., moving await to the bottom, we have already improved the runtime of the above script:

{
  res: [ "hello 1", "hello 2" ]
}
[2.01s] gettingTexts

Now it takes 2 seconds! i.e., instead of time of request 1 + time of request 2 it has become max (time of request 1, time of request 2)

Ain't that pretty cool?

This is the essence of concurrent programming. We dispatch requests at the same time and wait for them to finish.

So instead of running one thing at a time, i.e., synchronously. We are running things concurrently and achieving net better performance.

Picture worth a thousand words.

Okay, now what about error handling?

We have good old try/catch.

We just have to wrap the async code in it :

async function getTexts(): Promise<[string, string]> {  
  try {  
    const text1 = fetchText({ text: "hello 1", delay: 1000 });  
    const text2 = fetchText({ text: "hello 2", delay: 2000 });  
    return [await text1, await text2];  
  } catch (e) {  
    console.error("Error fetching texts");  
    return ["error fetching text 1", "error fetching text 2"];  
  }  
}

Now that we have gotten an understanding of what async JS is, let's dive into Async React:

Suspend React

Up until now, we dealt with async resources directly. But what about components that have a async dependency?

For example, let's take the same fetchText function. Instead of logging that value on the console, I want to show it in UI.

The common way of doing this in React is via useEffect (Variant 1):

import { useEffect, useState } from "react";  
import { fetchText } from "../async/sample.ts";  
import { Card } from "./Card/Card.tsx";  

function EffectCard() {  
    const [text, setText] = useState("loading..");  
    useEffect(() => {  
        fetchText({ text: "hello, world!", delay: 1000 })  
            .then((res) => {  
                setText(res);  
            })  
            .catch(() => {  
                setText("error");  
            });  
    }, []);  
    return <Card text={text} />;  
}

Sidenote

For more info on why this may not be a good idea, please read this post by core contributor from react-query :

Why You Want React Query | TkDodo's blog

Let's take a look at why you'd want a library like React Query, even if you don't need all the extra features it provides...

tkdodo.eu

Despite looking straightforward, useEffect has too many gotchas. Using it properly is challenging. I would recommend using a library which does it for you and react-query is a great option.

Now, this looks fine and well, but there is a problem with this.

If you think about it, this is similar to what we had in Variant 1 for async handling:

    const text1 = await fetchText({ text: "hello 1", delay: 1000 });  
    const text2 = await fetchText({ text: "hello 2", delay: 2000 });

Why so?

Because, useEffect runs after the render is completed.

So we are running things like this:

This is sequential instead of concurrent. You — fetch after render.

Let's see how we can improve this (Variant 2):


// move promise outside of React!  
const helloTextPromise = fetchText({ text: "hello, world!", delay: 1000 });  

function EffectCard() {  
    const [text, setText] = useState("loading..");  
    useEffect(() => {  
        helloTextPromise  
            .then((res) => {  
                setText(res);  
            })  
            .catch(() => {  
                setText("error");  
            });  
    }, []);  
    return <Card text={text} />;  
}

With this small change, we have them running concurrently!

Now that is much better. It's easy to get caught up in the framework/library and forget we are still using vanilla JS 😀

Okay, let's extend the above example (Variant 3):

import { EffectCard } from "./EffectCard.tsx";

function Input() {  
    const [name, setName] = useState("");  
    return (  
        <input  
            name={"userName"}  
            value={name}  
            onChange={(e) => setName(e.target.value)}  
        />  
    );  
}  
function EffectAndInput() {  
    return (  
        <div>  
            <Input />  
            <EffectCard />  {/*Same as above*/}
        </div>  
    );  
}

Now we have two components loading instead of one. One is theEffectComponent other is an input text element.

The effect can be slow or fast. Since we are emulating things here, let's make it slow.

This means that while the async stuff is still running if the user tries to type, it can create a janky experience. I am sure everyone has experienced this at one time or another.

When we try to type something but nothing shows up after a few moments it shows up at once.

This is because of this.

Here we are trying to unblock UI. i.e., we don't want to “await” on something that can take a lot of time to run. In this case, it's a component.

How can we let React know this?

Enter Suspense.

To signal React that this component is loading something that will take some time.

As I mentioned at the beginning we need to:

throw promise

Yup 🤯 (it still does).

By throwing a promise from a component, you let React know that there is some async activity going on. So React will suspend the component.

React can do other things while suspending the component!

This is Async React!

Let's refactor the above example to use Suspense.

We need to add a little logic to stop throwing the promise when it's finished. For that, we have this small utility:

type Status = "Loading" | "Done" | "Error";  

function createResource<T>(promise: Promise<T>) {  
    let status: Status = "Loading";  
    let result: T | null = null;  
    let error: Error | null = null;  
    promise  
        .then((res) => {  
            status = "Done";  
            result = res;  
        })  
        .catch((err) => {  
            status = "Error";  
            error = err;  
        });  
    return {  
        read(): T {  
            switch (status) {  
                case "Loading":  
                    throw promise; /* throw until promise is still running */  
                case "Error":  
                    throw error;  
                case "Done":  
                    return result!;  /* return once done */
                default: {  
                    return status;  
                }  
            }  
        },  
    };  
}

As you can see, we are throwing the promise till it's in Loading state and returning the result once it's done.

With this, let us convert the EffectCard to a SuspendedCard:

import { fetchText } from "../async/sample.ts";  
import { Card } from "./Card/Card.tsx";  
import { createResource } from "../async/create-resource.ts";  

// Same as before - start the promise outside of React life cycle
const helloTextResource = createResource(  
    fetchText({ text: "hello, world!", delay: 1000 }),  
);  
function SuspendedCard() {  
    const helloText = helloTextResource.read();  
    return <Card text={helloText}></Card>;  
}

⚠️ Don't create the resource inside the Component! Because that would mean that a new resource will be created each time the component re-renders. Which is not something we wish to have (Infinite loops!).

Keep the components idempotent.

Now let's refactor Variant 3 to use the SuspendedCard:


import { Suspense } from "react";  
import { SuspendedCard } from "./SuspendedCard.tsx";  
import { Input } from "./Input.tsx";  

function SuspendedEffectAndInput() {  
    return (  
        <Suspense fallback={<h1>loading..</h1>}>  
            <Input />  
            <SuspendedCard />  
        </Suspense>  
    );  
}  

export { SuspendedEffectAndInput };

Now isn't this cool?

Few things to note here though:

The Suspense the wrapper is called the Suspense Boundary. This is where you provide a fallback i.e., while the component is suspended you can show something else. A loading indicator, for example.
The Suspense boundary is granular. You can put it anywhere you like. Directly above the <SuspendedCard /> or like I have done in the snippet.

The Suspense boundary is so flexible because we are literally throwing the promise!

When you throw something, we can have any number of layers above. We handle it wherever we wish. Now throwing stuff other than errors makes sense. Also, we must handle it!

What about errors? 🤔

If you had noticed in the createResouce function, we had this case too:

case "Loading":  
    throw promise;  
case "Error":  
    throw error; /* error scenario */  
case "Done":  
    return result!;  
default: {  
    return status;

So, in case of an error, we throw the error instead of the original promise.

To handle it, we need to wrap it in a ErrorBoundary:

import { Suspense } from "react";  
import { SuspendedCard } from "./SuspendedCard.tsx";  
import { Input } from "./Input.tsx";  
import { ErrorBoundary } from "react-error-boundary";  /* you have to install this library */

function SuspendedEffectAndInput() {  
    return (  
        <ErrorBoundary fallback={<h1>Some error occurred!</h1>}>  
            <Suspense fallback={<h1>loading..</h1>}>  
                <Input />  
                <SuspendedCard />  
            </Suspense>  
        </ErrorBoundary>  
    );  
}  

export { SuspendedEffectAndInput };

This is similar to wrapping it with try/catch block!

This whole approach is called render as you fetch

Let's push this a bit further.

We can even load the component code dynamically via lazy .

So that means that both code and data can be loaded concurrently!

Let's see how that can be accomplished:


import { lazy, Suspense } from "react";  
import { Input } from "./Input.tsx";  
import { ErrorBoundary } from "react-error-boundary";  
import { createResource } from "../async/create-resource.ts";  
import { fetchText } from "../async/sample.ts";  
const Card = lazy(() =>  
    import("./Card/Card.tsx").then((mod) => ({ default: mod.Card })),  
);  /* async logic to get code */ 

const helloTextResource = createResource(  
    fetchText({ text: "hello, world!", delay: 1000 }),  
); /* async logic to get data */ 

function SuspendedResourceCard({  
    resource: { read },  
}: {  
    resource: { read: () => string };  
}) {  
    const helloText = read();  
    return <Card text={helloText}></Card>;  
}  

function FinalVariant() {  
    return (  
        <ErrorBoundary fallback={<h1>Some error occurred!</h1>}>  
            <Suspense fallback={<h1>loading..</h1>}>  
                <Input />  
                <SuspendedResourceCard resource={helloTextResource} />  
            </Suspense>  
        </ErrorBoundary>  
    );  
}

As you can see, we are loading both code and data concurrently!

Suspense boundaries can handle both of them together without us having to manage the low-level details. The best part is React can do other things like update UI when the user types on the input (no more jank!)

Another cool thing is now with RSC we have a way to start loading promises on the server and then Suspend them on the client!

Some additional resources if you want to deeper into this:

overreacted.io

This Youtube video by Jack details on how to leverage these things on the server:

Also, we don't have to use the createResource method. The React team is shipping with a new hook in React 19 called use :

For a live version of all the code samples you can check them out here:

varenya / react-suspense-blog

Async React

Demo application to showcase how Suspense works.

View on GitHub

All right folks that's it, I hope this was helpful!

Data fetching patterns - balancing client and server with SWR and Next.js

Varenya Thyagaraj — Fri, 13 Jan 2023 05:33:54 +0000

With the advent of modern component JS-based full-stack frameworks, we now have the flexibility to get remote data into our apps either via client or server.

Why does this matter?

Based on how we get data into our system we make different trade-offs with core web vitals.

For example -

If we get all the data from the server we would have to wait for all the data to be available. So the user would have to wait for more time to see something on the screen (TTFB).

So should we shift all the network calls on to the client then 🤔 ?

Lots of loading spinners! 🌀

Well turns out the answer is nuanced - not surprising at all is it 😃.

A good rule of thumb would be if the information is above the fold or critical we shouldn’t make the user see spinners as much as possible. And if something is quite hidden from the initial UI can be potentially loaded “lazily”.

But that means some calls we need to make some calls on the server and some on the client. That requires granular control over which calls need to be executed on the client and which on the server. And hopefully with little or no extra code.

Also what about data changing over time?

Perhaps something like product availability in an e-commerce application that we can load initially at the time from the server but can potentially change while the user is browsing? We would need some ability to re-validate 🤔.

To solve these problems we have some awesome libraries namely:

I have chosen Next.JS as the framework and SWR as the library to showcase different ways to leverage the different options at our disposal.

I have created a demo weather app to showcase different variations of data fetching.

Looks something like this:

Lets dig in!

Pure Client Side

This is the default approach with swr all you need to do is use the hook and provide it with a unique key to identify the remote call.

In this case we will use the location as the key so the code will look something like this:

function Weather() {
  // first param is the key and second argument is custom fetcher.
  // the key is passed to custom fetcher.
  const { data, error } = useSWR(location, getWeatherInfo);
  if (error) {
    return <p>Error!</p>;
  }
  if (!data) {
    return <p>Loading..</p>;
  }
  return (
    <>
      <p>{data.temperature}</p>
      <p>{data.humidity}</p>
    </>
  );
}

I have used the same custom fetch client that I blogged about last time:

Making impossible states impossible ft. Zod and Typescript

You can go through it for more info.

So the initial UI user will see will be something like this (loading indicators..):

For full fledged code for this approach:

GitHub - varenya/swr-blog at pure-client-side

Prefetch on Server

As you can see from the above screenshot I have chosen london as the first location.

We could show the user a loading indicator and make users wait before they can see the weather info.

But why do it when we know that is the first thing user will see?

At the same time doesn’t make sense the get weather info for all cities on server either. Users would stare at a white screen longer than necessary.

So for this application we will only fetch weather info for london on server and will load the rest on the client.

In next.js to make a server side call we need to use the aptly named function getServerSideProps .

And to use it with swr we need to pass the data to fallback prop in the context that swr provides called SWRConfig .

So the page would look something like this:

function Page(props) {
  return (
    <SWRConfig value={{ fallback: props.fallback }}>
      {/* Your App */}
      <Weather />
    </SWRConfig>
  );
}

// london data is prefetched!
export async function getServerSideProps(context: GetServerSidePropsContext) {
  const weatherResponse = await getWeatherInfo("london");
  return {
    props: {
      fallback: {
        london: weatherResponse,
      },
    },
  };
}

Full fledged code:

GitHub - varenya/swr-blog at pure-server-side

Let’s take a moment and talk about this in a bit more detail.

The demo app that I have created has few cities as options and we were able to leverage SWR and Next.JS to shift the remote call from client to server.

Now let’s take a step ahead here and assume that during a peak traffic time the weather API became slow so users would be staring at a white screen.

What if we had the ability to switch over to a pure client side solution in such a situation?

Because in the above case that would be a better option. Since instead of seeing a white screen users would see a loading indicator like in the pure client side approach.

Well for that we just need to remove the getServerSideProps call and we would be good.

So something like this:

function Page(props) {
  return (
    <SWRConfig value={{ fallback: props.fallback }}>
      {/* Your App */}
      <Weather />
    </SWRConfig>
  );
}

// london data is prefetched!
export async function getServerSideProps(context: GetServerSidePropsContext) {
  // add a feature flag - what ever mechanism your org uses.
  const featureFlags = getFeatureFlags();

  if (!featureFlags.clientOnly) {
    const weatherResponse = await getWeatherInfo("london");
  }

  // sending an empty fallback if clientOnly flag is true
  return {
    props: {
      fallback: featureFlags.clientOnly
        ? {}
        : {
            london: weatherResponse,
          },
    },
  };
}

And et voila - we have shifted the call’s happening on the client!

This is quite handy - especially during a pager duty alert 😅 .

Also we are able to reuse lot of code here too:

// on the server - to get weather data:
const weatherResponse = await getWeatherInfo("london");

// on the client
const { data, error } = useSWR(location, getWeatherInfo);

Since we are using same getWeatherInfo function we get type-safety across client and server.

There are plenty more things this enables for us:

If the call fails with an error on server we can send out an empty fallback and client will try the call!
We can set some timeout duration on server and throw an error and have the client make the call!
Apply some retry options via swr on client when if it errors out or times out.

I will let you folks come up with creative solutions here, but I think you all got the gist.

Prefetch on the client

Turns out we can do this via good old link tags on html :

<link rel="preload" href="/api/weather/mumbai" as="fetch" crossorigin="anonymous">

Or like this :

import useSWR, { preload } from "swr";

// ain't that cool!
preload("kolkata", getWeatherInfo);

function Weather() {
  // first param is the key and second argument is custom fetcher.
  // the key is passed to custom fetcher.
  const { data, error } = useSWR(location, getWeatherInfo);
  if (error) {
    return <p>Error!</p>;
  }
  if (!data) {
    return <p>Loading..</p>;
  }
  return (
    <>
      <p>{data.temperature}</p>
      <p>{data.humidity}</p>
    </>
  );
}

You will notice on the network tab that the prefetch query is executed first:

So if user selects kolkata chances of them seeing a indicator will be very less:

https://weather-blog-gpeypw5gj-varen90.vercel.app/

Full fledged code:

GitHub - varenya/swr-blog at prefetch-client-side

Suspense

This is the other part of async coding that can get out of hand. Every time we make a remote call we have to deal with various states i.e.

Trigger the async call
Go into loading state
If successful show data
if error handle the error in a graceful way

Now these states have tendency to go out of hand. Even you even a few remote data dependencies the code complexity grows.

The above states can be modelled as a union to alleviate some of the pain:

function WeatherLoader({ location }: WeatherProps) {
  const weatherData = useWeather(location);
  switch (weatherData.status) {
    case "error":
      return <WeatherErrror />;
    case "loading":
      return <WeatherContentLoader />;
    case "success":
      return <Weather weatherInfo={weatherData.data} />;
    default:
      const _exceptionCase: never = weatherData;
      return _exceptionCase;
  }
}

But this can get out of hand with multiple async data. In fact lot of frontend code will look like this 😃 .

Now bear in mind that we are using same getWeatherInfo so whatever error are thrown there will be handled as part of the “error” condition. We can tune the errors with some additional magic but I won’t go into detail for that here.

With Suspense the above can be refactored into some thing like this:

function WeatherLoader({ location, retryLocation }: WeatherProps) {
  return (
    <ErrorBoundary
      FallbackComponent={WeatherError}
      resetKeys={[location]}
      onReset={() => retryLocation(location)}
    >
      <Suspense fallback={<WeatherContentLoader />}>
        <Weather location={location} />
      </Suspense>
    </ErrorBoundary>
  );
}

To get this working with swr all you need to do is:

const { data } = useSWR<BasicWeatherInfo>(location, getWeatherInfo, {
  suspense: true,
});

Typically this would involve lot more code but that all is abstracted away for us when using libraries like this swr .

To be honest suspense is lot more than just a data fetching convenience. It literally suspends any logic that component is waiting for typically something asynchronous.

To accomplish this we actually have to throw a promise ! (swr does the heavy lifting for us here)

For now its not fully stable and has some gotchas -

https://reactjs.org/blog/2022/03/29/react-v18.html#suspense-in-data-frameworks

Final Code using Suspense :

GitHub - varenya/swr-blog

Also using react-query :

GitHub - varenya/swr-blog at use-react-query

And the final app link:

https://weather-blog.vercel.app/

I hope you all found it useful. Thanks for reading!

Making impossible states impossible ft. Zod and Typescript

Varenya Thyagaraj — Mon, 19 Dec 2022 12:51:48 +0000

There are some patterns in the typed functional world that are quite handy. The most prominent pattern is that of using Union and Product Types to model data to avoid bugs.

The title of this blog post is in fact lifted from a great talk given by Richard Feldman:

Now the above talk showcases a language called Elm.

Can you believe that the app built in Elm hasn't had a single runtime exception? It still blows my mind!

But thanks to TS we have similar abilities that we can leverage!

Let's see some examples with Union Types -

Assume this is an online car seller and as a dev, we modelled the Car type as:

type Car = {
  isElectric: boolean;
  isCombustion: boolean;
  frunkSpace?: number;
  bootSpace: number;
};

Now let's write a function to calculate the total storage space in the Car:

function getTotalCarStorage(car: Car) {
  if (car.isCombustion) {
    return car.bootSpace;
  }
  // for electric car there could be front trunk space!
  const { frunkSpace = 0 } = car;
  return car.bootSpace + frunkSpace;
}

But there is a potential bug in this above code. Can anyone spot that?

Well what if by mistake someone passed the object as:

const porscheTaychan: Car = {
  isCombustion: true,
  isElectric: true,
  frunkSpace: 10,
  bootSpace: 200,
};

The result would be 200 when the answer we are expecting here is 210!

Here the problem is booleans!

Because we allowed this possibility - isCombustion and isElectric can be true!

Now, sure we could add some tests around it and make sure that if both are true we throw an error or handle it some other way.

But by changing the type to use a Union type we can remove the possibility altogether:

type Car =
  | {
      kind: "electric";
      frunkSpace: number;
      bootSpace: number;
    }
  | { kind: "combustion"; bootSpace: number };

function getTotalCarStorage(car: Car) {
  switch (car.kind) {
    case "combustion":
      return car.bootSpace;
    case "electric":
      return car.bootSpace + car.frunkSpace;
  }
}

If you try and access the frunkSpace property inside “combustion” case TS won’t let it compile:

But wait this can get even better, let's say now a new type of Car has launched - a hybrid one! And we need to make sure that the case is handled in the above function!

How can we make sure that the function handles all the scenarios 🤔 ?

We add a never case:

type Car =
  | {
      kind: "electric";
      frunkSpace: number;
      bootSpace: number;
    }
  | { kind: "combustion"; bootSpace: number };

function getTotalCarStorage(car: Car) {
  switch (car.kind) {
    case "combustion":
      return car.bootSpace;
    case "electric":
      return car.bootSpace + car.frunkSpace;
    default:
      const _exceptionCase: never = car;
      return _exceptionCase;
  }
}

Now if you update add another type to the Car type union. TS will give you an error:

How awesome is that 😎 ?

Now some of you reading this might be wondering how can this be helpful if we are talking to a REST API that we don’t control.

Well, that's where Zod has your back!

Let's look at an example.

Assume there is an endpoint that provides weather information like this:

GET https://someweatherapi.com/en/{location}

// Sample Response ->

// if the provided location is supported:

{
  available: 1,
  humidity: 100,
  temperature: 35,
}

// if provided location is not supported

{
    available: 0
}

Now at first glance we can come with a client code to consume the API like this:

type WeatherInfo = {
  available: number;
  humidity?: number;
  temparature?: number;
};
async function getWeatherInfo(location: string) {
  const weatherResponse = await fetch(
    `https://someweatherapi.com/en/${location}`
  );
  if (!weatherResponse.ok) {
    throw new Error(weatherResponse.statusText);
  }

  // Spot 1
  const weatherInfo: WeatherInfo = await weatherResponse.json();

  // We could just as easily missed to check and access the properties directly!
  if (weatherInfo.available === 1) {
    return {
      humidty: weatherInfo.humidity,
      temparature: weatherInfo.temparature,
    };
  }
}

// We can consume it like this

async function main() {
  const currentWeatherInfo = await getWeatherInfo("london");

  // Spot 2
  if (
    currentWeatherInfo &&
    currentWeatherInfo.temparature &&
    currentWeatherInfo.humidty
  ) {
    console.log(`Temperature in London is: ${currentWeatherInfo.temparature}`);
    console.log(`Humidity in London is: ${currentWeatherInfo.humidty}`);
  }
}

The above code looks fine on the surface. But if you observe the consumer code you notice you end up writing a lot of checks to make sure data is available. Like in Spot 2.

Now we can fix the “defensiveness” part of the code with Unions!

Lets refactor:

// We can be strict about availabe being 1 or 0! - we don't have to say it's a
// number!
type WeatherInfo =
  | {
      available: 1;
      humidity: number;
      temparature: number;
    }
  | { available: 0 };

async function getWeatherInfo(location: string) {
  const weatherResponse = await fetch(
    `https://someweatherapi.com/en/${location}`
  );
  if (!weatherResponse.ok) {
    throw new Error(weatherResponse.statusText);
  }
  // Spot 1
  const weatherInfo: WeatherInfo = await weatherResponse.json();
  if (weatherInfo.available === 0) {
    throw new Error("Weather information not available!");
  }
  return {
    humidty: weatherInfo.humidity,
    temparature: weatherInfo.temparature,
  };
}

async function main() {
  const currentWeatherInfo = await getWeatherInfo("london");
  // No checks necessary! 😄 - You need to handle the error though.
  console.log(`Temperature in London is: ${currentWeatherInfo.temparature}`);
  console.log(`Humidity in London is: ${currentWeatherInfo.humidty}`);
}

Tip: If you have a lot of defensive code all over your codebase it's worth reviewing your Type definitions 😄 . With TypeScript this becomes a code smell.

But that’s not all there is to it. If we observe Spot 1 the line of code where we define response as WeatherInfo. That only works at compile time!

After all, it's an API that we don’t control. If the data coming from the network doesn’t fit the Type we have created the App won’t work as expected!

So how do we get around this?

Enter Zod!

Zod at its core a schema validation library when combined with TS it becomes more like Superman instead of General Zod 😄.

Lets refactor again shall we:

import { z } from "zod";

const availableWeatherInfo = z.object({
  available: z.literal(1),
  humidity: z.number(),
  temparature: z.number(),
});

const unavailableWeatherInfo = z.object({
  available: z.literal(0),
});

// https://zod.dev/?id=discriminated-unions
const WeatherInfoResponse = z.discriminatedUnion("available", [
  availableWeatherInfo,
  unavailableWeatherInfo,
]);

// if you hover - you will notice that the model is same as version we created
// earlier.
type WeatherInfo = z.infer<typeof WeatherInfoResponse>;

async function getWeatherInfo(location: string) {
  const weatherResponse = await fetch(
    `https://someweatherapi.com/en/${location}`
  );
  if (!weatherResponse.ok) {
    throw new Error(weatherResponse.statusText);
  }

  // Parsing via Zod instead of the earlier approach.
  const weatherInfo = WeatherInfoResponse.parse(await weatherResponse.json());
  if (weatherInfo.available === 0) {
    throw new Error("Weather information not available!");
  }
  return {
    humidty: weatherInfo.humidity,
    temparature: weatherInfo.temparature,
  };
}

async function main() {
  const currentWeatherInfo = await getWeatherInfo("london");
  console.log(`Temperature in London is: ${currentWeatherInfo.temparature}`);
  console.log(`Humidity in London is: ${currentWeatherInfo.humidty}`);
}

Now tell me - that isn’t cool 😎 !

With this version of code - Its TypeSafe at compile time and at runtime.

At runtime if the API response doesn’t conform to our modelled schema it would throw an error!

Just to showcase how this looks in practice if API response comes like this:

GET https://someweatherapi.com/en/delhi

// Wrong Response ->
{
    available: 1,
}

Zod will throw an error! Which will save you from showing temperature as undefined 😄.

So if you are consuming the API in a React component we can wrap it around Error Boundary and make sure the potential failure is localised and doesn’t crash the whole application!

So to conclude, Zod :

Aligns really well with TS.
Using infer you don’t have to repeat yourself in TS.
Gives you runtime and compile time safety with the help of TS!

Code Samples:

varenya / weather-blog

Thats it, thanks for reading - I hope it was useful!

Art of debugging

Varenya Thyagaraj — Mon, 04 Apr 2022 11:50:14 +0000

Building software is a conundrum of sorts. A computer is a machine that does exactly what we tell it to do and yet we face so many problems. So the problem here is not a computer but us.

Act of building software is that of realising the full potential of our intelligence and also our limitations.

There is a chasm between how we usually solve problems and how computers do it.

In order for us to cross the metaphorical chasm, we need constraints in how we build software.

Meta much? 😃

For more about constraints in software:

API Design 1

API Design 2

Whenever our mental model about how computer operates deviates from how the computer actually operates - we end up with a bug.

Now that we know the source of the bugs let's see how to debug software.

As you might guess there is no hard and fast rule approach to debugging as well. But over time some patterns have emerged in my approach and I wanted to codify them in this post.

Challenge your assumptions

More often than not we struggle to find the issue because we assume that - this is how it’s supposed to work. But of course, if it did we wouldn’t be facing this bug.

Now in practice, this takes different forms.

Example:

If you have ever faced problems with modals showing up in unexpected order even with a style like this:

.modal {
  z-index: 1000000;
}

Here the assumption is that a higher the z-index will result in the DOM element being at the top.

Well, now that you know it's not working as expected.

Our assumption is wrong - so what is it that we are missing in the above case? - stacking contexts!

I won’t go too deep into it but this is a problem that a lot of folks run into when they start off doing CSS. There is more nuance here and I would urge the readers to look for material on this.

And FYI, I too learned about stacking contexts and other nuances involved after the fact.

Another side effect of debugging is that you gain a deeper knowledge of the thing you are working on.

If the bug you found got fixed by some random changes - try and dig deeper into the “why”. It will grow you in more ways than one.

Read the error messages

This one’s straightforward right?

After a while, we take loads of things for granted. Jump to conclusions about what caused the issue. Only to find yourself wasting hours to realize the answer was staring you right in your face.

Debugging lot of times an antidote to hubris 😃

Example:

While working on React app nothing showed up on the UI that I was expecting.

I went through these assumptions:

I didn't return the JSX from the component.
Didn't pass the props.
Applied wrong CSS - white background on white text?
...

Only to see the error message and see that I had misspelled the filename.

Read the docs

I know right? Well, trust me reading docs for a few mins can spare you hours of debugging.

If you open up a popular repo on Github most of the issues reported have answers in documentation. Folks jump to report an issue instead of doing some due diligence.

Some frameworks and tools have very specific semantic way of using them. If the semantics are not followed it can lead to subtle issues that will escape us.

Even after reading issues will appear but we will likely get a signal about what went wrong.

Sometimes the documentation is poor. But it’s still worth giving a cursory look. Paying attention to the possible “gotchas” listed before digging in.

Example:

I tried to use a library for async actions only to realize that the framework I was using was not compatible.

A more nuanced one:

When I used the useEffect in React for the first time to fetch some data I ended up in a infinite loop. Turns out the mental model behind useEffect isn't as simple as it looks.

Gif worth a thousand words:

Example:

I tried to use a library for async actions only to realise that the framework i was using was not compatible.

Context switching

This I found to be one of the sinister ways bugs crept into my code.

Also affected my debug process quite a bit.

The idea here is when a task is in execution one should not switch over to something else while doing so. I found the cost to be massive for even a short switch to something else.

For deep work this can hurt your output.

Don’t do this:

Example:

I was in flow while debugging a hard to reproduce issue.

I got called into a meeting. After the meeting I started off from where I left only to find myself in a mess.

This applies to most tasks in general.

Debugging is where I am the most knee-deep into the guts of the complexity and in deep flow state. So if something else demands your attention. Make sure to take a breather and start from scratch and not assume you can get back to it.

Peel the layers of abstractions

If the above approaches didn’t solve the bug. Most likely it’s something that will need you to dig deeper.

Depending on the issue the "layer" will differ but advice is same.

Example:

A place in UI where the total number items should show up - shows up as NaN.

The layers here could be:

State Management
Parsing
Caching
Query
......

And the above can happen at frontend and backend (web dev perspective).

To isolate where the issue occurred the approach could be:

Bottom-up - starting from where the issue happened and going up the layers.
Top to bottom - starting from where data entered system to where the issue happened.
combination of both - somewhere in the middle.

Tools help here a lot. Setting up a breakpoint and walking you through the code - peeling the layers 😃.

Pair Programming

A lot of times when we are in a rut of debugging something it’s likely that a second set of eyes will reveal the answer. This is where pair programming has helped me a lot.

If you have seen Naruto and how to get out of Gen Jutsu, you know what I mean 😃:

Reproduce the bug consistently

Well, this was the unsaid assumption in all my previous advice that I shared. And a lot of times that is the case.

These sorts of bugs happen less at the app level than at lower layers. The reason is lower-level primitives tend to combine/compose in a very complex mesh.

A good metaphor for this is Chess - each piece has easy to define moves but the combination of them is complex.

Some language primitives which are easy to define but difficult to master:

Pointers - I mean phew right!
Closures - Closures - stale closures anyone? 😄
Async - Well this is most trickiest of all . These issues are hard to reproduce and result in erratic behavior, to say the least.
CSS cascade rules - I mean after a point the complexity is so high it becomes difficult to predict the results.
....

To reproduce issues of such nature we will likely need to setup some instrumentation.

Sometimes it’s as simple as putting in loads of logs to see what the heck is happening. And from that reproduce it in your environment by creating those conditions.

Example:

If its some CSS rule not getting applied as you expect the best way to isolate is:

Create a sample html/css with similarities to the original.
Add the rules one by one to see which one is causing the issue.

If the issue is intermittent. Like some piece of text not showing up every now and then:

Clear the cache.
Isolate the logic where the text is being loaded - run it on a isolated test environment.
If possible get the whole state of the system at that point..
If any async logic is there separate the logic and run it couple of times to see the output.
........

Get some sleep/break

If you keep finding yourself hitting a wall. Then that is a great sign to move away from the problem.

Loads of times the bug that took up the better part of my day got solved first thing in the morning. So get out of your own way by taking a break.

Well, that's it. I hope this helps!.

Core principles of API Design - Part 2

Varenya Thyagaraj — Sat, 12 Mar 2022 07:36:00 +0000

This is the sequel to:

API Design 1

I laid out the principles behind designing an API and wanted to show those principles in play in a non-trivial example.

Turns out its difficult to find the “goldilocks example” i.e. not too hard not too easy. So instead of wasting my time further I figured I would build out an API for TodoList.

Lets think through what requirements would be needed to build such a TodoList:

Create a to-do item.
Update a to-do item.
Add a to-do item.
Get a list of to-do items.
...

I will focus on the Update functionality to step through the process of building out an API.

💡 I will be using JavaScript here to build out the API.

Step 1: Design the Data Structure

Depending on how you design your data structure the code will change accordingly. For example if you choose a list instead of a map all your code will have ripple effects based on that single change.

Lets go with a basic version for todo-item first then iterate on it if need arises:

// We will structure the todo item in object literal with id and title properties

const basicItem = { id: "1", title: "todo something" };

// We will hold all our todo items in a list

const todoList = [];

Step 2: Write a Failing Test

Here I start doing things using Test Driven Development (TDD).

We will start off by assuming that there is a function to add a todo-item to a to-do list.

Function code:

// We will assume this is the function we will be going with.

function updateTodo(todoList = [], todoId, params) {}

Test code:

// Using Jest

test("should update todo based on id", () => {
  const currentList = [
    { id: 1, title: "something" },
    { id: 2, title: "something other than something" },
  ];
  const result = updateTodo(currentList, 2, { title: "updated todo" });
  expect(currentList).toMatchSnapshot();
  const [_, second] = result;
  expect(second).toMatchObject({ id: 2, title: "updated todo" });
});

Well as you would expect the test will fail:

But the point here is to use tests as a client which consumes the API and verifies your requirements along with it. This is how TDD works. I recommend readers unfamiliar with this to read further online.

Step 3: Pass the test

Let’s fill in the blanks for the function we setup in previous setup and make sure the test passes.

I am skipping the part where i fumble around to get the code right 😃 :

function updateTodo(todoList = [], todoId, params) {
  const updatedList = [...todoList];
  const itemIndex = todoList.findIndex(({ id }) => id === todoId);
  const item = todoList[itemIndex];
  const updatedItem = { ...item, ...params };
  updatedList.splice(itemIndex, 1, updatedItem);
  return updatedList;
}

And the test 🎉 :

Step 4: Refactor

Now given the implementation of the update function I set a few constraints:

The function is pure! - if you have been following my posts this shouldn’t come as a surprise 😃
I am not mutating the input data to the function. - Well it won’t be pure if we did!
Using the id to find out the item and updating with the params passed as input to the function.

Now given these requirements the native JS code became quite verbose and we can use a library like immer to make it more concise!

import produce from "immer";

function updateTodo(todoList = [], todoId, params) {
  return produce(todoList, (draftTodo) => {
    let item = draftTodo.find(({ id }) => id === todoId);
    item.id = todoId;
    item.title = params.title;
  });
}

Now lets see how the test looks:

It’s still passing and green 🎉. Dopamine much?

Constraints and Heuristics

See what I meant by saying that API hides away implementation details? We completely changed the code and yet the test remains green!

This implies that any consumer using this function doesn’t have to make any changes!

We haven't talked about Constraints and Heuristics here. Let's see in the context of this example and the approach we took:

Here I set the constraints using a test and made sure whatever code I wrote adhered to those constraints!

In doing so we made it easier to guess how the code needs to be and also verified that our assumptions are right as well.

As per the constraints we need to make sure to use that function to return a new to-do list with an updated item. Also, the original array needs to be as is.

Given this the heuristic here is to:

copy the array
make the changes to copied array
return the copied array with the changes.

The initial version of the code did this as is. Once the test gave us the feedback that it worked I went ahead and used immer library to make the code more succinct! The test remained green!

Some food for thought - what are different way to set constraints in programming:

Types! - psst typescript 😉
Using data structure - Sets, Maps, Lists, Trees!
Encapsulation mechanisms - modules, classes, closures etc.
Tests!
Function contract! - what the input and output would be - can be asserted using types and tests!
.....

I hope this helped. Thanks for reading.

Github Link for codebase:

https://github.com/varenya/todo

More about TDD:

https://www.amazon.in/dp/B095SQ9WP4/ref=dp-kindle-redirect?_encoding=UTF8&btkr=1

Principle's of Automation Testing

Varenya Thyagaraj — Mon, 21 Feb 2022 16:59:37 +0000

Well, this is quite a heavy topic that I have picked on to write about.

It's an age old question on how to build quality software. Over the years testing has become an essential step in building quality software.

The specifics of how to approach testing are still very much in debate and changed over the years. Nonetheless, I believe few principles have emerged over the years which I would like to share.

Let us look at some of the questions one might ask before they start a project:

When is the right time in the project lifecycle to start testing?
Is testing only QA's job?
Does the way a developer builds code affects the testability of software?
Is it okay to mock stuff? If yes how much?
What is the ideal way the tests should look like?
How long should a given suite of tests run?

.... etc.

I hope this give's you an idea about how much difficulty there can be when it comes to testing.

So let's start off the crux of the post and delve into a series of points which will answer the questions above :

Testing cannot be an afterthought

This is an important point that needs to be in everyone's mind while starting off a project.

If this is not followed the result of the project will be hard to predict/buggy and over time hard to grow. Even the use of expensive tools will not change the result if testing starts towards the end.

I understand this will be disappointing to a lot of folks but this has been my experience.

So if I say testing cannot be an afterthought does that mean devs also own this? - The answer is Yes! Building quality software is as much a responsibility of a dev as it is of a QA engineer.

Why so?

If you think about it software is a sum of lot of pieces. Pieces like data structures/functions/classes etc.

Each piece of code can have N different paths of execution. Combine those with other pieces and the complexity increases quite a bit.

I hope that answers the question? Testing should happen right from those individual levels and its combination too. Otherwise, there is no way to have a good level of confidence in the quality of the output.

Developer Approach to Code

Now that we established testing cannot be an afterthought, lets come at it from a dev's perspective as to how to build code that can be tested in the first place. In this area lot of idea's/patterns have emerged the most popular of them being the practice of TDD i.e. Test Driven Development. The basis of the approach is to write a failing a test corresponding to a requirement and then write the code to make the failing test pass and then you can refactor the code to do better all the while having the confidence of having the test be green.

This approach has been incredible for my personal workflow while developing code. It produces small well tested abstractions and grows as you go through more requirements. This way you get tests right from the beginning of the project lifecycle. Although this add's to developers time it saves a ton later in terms of quality. Since bugs in production are lot harder to debug than on your local system.

Other than that few pointers to keep the code testable:

Encapsulate behaviour as much as possible in pure functions.
Keep the API surface minimal.
Make the contracts explicit as much as possible - if you are using a typed language encode that in types to further reduce the possible misuse.
Grow abstractions in layers.
Hide away imperative/complex bits using encapsulation mechanisms and expose a declarative API.
Hoist the parts of the code where side effects are present to the top. And preferably in a singular place.

This is not a exhaustive list but I think its a good place to start from.

E2E Vs Integration Vs Unit

Now these terms are used quite frequently in a testing context and usually along with a term called "Testing Pyramid".

The term "Testing Pyramid" refers to this following diagram:

Source : https://www.browserstack.com/guide/testing-pyramid-for-test-automation

So it basically says:

Unit Tests > Integration Tests > E2E Test

But lets define these types of tests in the first place:

Unit Test

A type of test which tests a "unit" of functionality.

the "unit" above could be lot of things like:

function
class
API route
Module
React Component
....

So based on your context "unit" could mean a lot of things.

Example:

function add(a, b) {
  return a + b;
}

// add.test.js

test("should add two numbers", () => {
  expect(add(1, 2)).toEqual(3);
});

Trade Offs:

Fast feedback loop
High chance of mocking (reduces the reliability of test).

Integration Test

A type of test which usually tests a combination of units.

Example:

function add(x, y) {
  return function (x) {
    return x + y;
  };
}

function multiple(x, y) {
  return function (x) {
    return x * y;
  };
}

function doubleAndAddOne(x) {
  const addOne = add(1);
  const double = multiple(2);
  return addOne(double(x));
}

test("should double and add one", () => {
  expect(doubleAndAddOne(5)).toEqual(11);
});

Trade Offs:

Typically slower feedback loop
Typically lesser mocking

E2E Test:

This is where you test your entire application from a user perspective.

If you are in web dev world, it would look different based on the tools and the language you use to test it.

A sample selenium test using JS:

const By = webdriver.By; // useful Locator utility to describe a query for a WebElement
// open a page, find autocomplete input by CSS selector, then get its value
driver
  .navigate()
  .to("http://path.to.test.app/")
  .then(() => driver.findElement(By.css(".autocomplete")))
  .then((element) => element.getAttribute("value"))
  .then((value) => console.log(value));

Trade offs:

Typically very slow feedback loop
Typically no mocking - more correct.

Let's ponder as to why the pyramid is structured the way it is.

Given the trade offs I have mentioned we can tell that the tests have been structured based on feedback loop time (cost):

Basically unit tests run very fast so you can afford to have many of them and not incur a much cost and anything breaks it can be fixed at relatively high speed - correctness can be low if there is too much mocking.
Integration test are just above the hierarchy and are relatively slower to give feedback so we want them to be lesser - but in terms of correctness they are better since mocking is lesser.
in the same vein - E2E are slower to run but in terms of correctness they are better/best.

The balance to maintain here is correctness and speed.

The pyramid shows the trade-offs involved and gives us a guideline on how to structure our tests.

The point is to look at the trade-offs involved and adapt. Tools like Cypress are good examples of how tools are changing the trade-offs and how we can adapt.

I hope this helps. Thanks for reading.

Complexity of selectors in web development ft React

Varenya Thyagaraj — Sun, 16 Jan 2022 12:35:12 +0000

Selectors essentially let’s us specify which part of HTML that we are trying to target for example:

<div class="parent">
  <div class="child"></div>
</div>

In the above example in order to select the child which is below parent in JS we would have to do something like this:

document.querySelector(".parent .child");

In CSS :

.parent .child {
  background: papayawhip;
}

I have used CSS class names as means to select which is generally the way to select elements.

So, CSS class names and some rules around composing them allow one to pick and choose elements from HTML.

So far so good, problems start to creep in when the app grows.

The complexity increases for selecting elements.

There was a poll on Twitter by Max Stoiber :

// Detect dark theme var iframe = document.getElementById('tweet-1038073603311448064-531'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1038073603311448064&theme=dark" }

Even though the above example is having a simple structure around 57% of 14k people answered it wrong. At scale, it can get out of hand.

The reason being the selectors tie the CSS and HTML in a many-to-many relationship. We can have many CSS classes applied on a single element and a single CSS class can target many elements.

It's not simple but it is easy.

So what does it mean in practice - Looking at HTML and CSS you cannot determine what you will get.

What you see, may not be what you get.

I am sure every one of us has run into issues where we expect the element to have certain styles only to be surprised. We rely on browser tools to determine which class over-wrote what we expected.

Contrast above with the below CSS-IN-JS solution using emotion:

<div
  css={`
    color: blue;
  `}
>
  Hello
</div>

Here it's as close as it can be — What you see is what you will get.

So moving away from selectors in CSS actually reduced a bunch of complexity for us.

Now let’s think of Javascript how do we get around selecting HTML elements this way?

The same complexity will transfer over to JS as well since we use CSS selectors to pick elements.

The answer turns out to be to put HTML-in-JS which is JSX!

Now the only selector you need to give in React is the entry/mount node and that’s it! The way you write JSX is how the HTML shows up.

What you see is what you get!

Now there is actually one other aspect of web development that relies on CSS selectors - unit testing.

To remove complexity from CSS selectors we moved CSS to JS.

To remove the complexity of CSS selectors from JS we moved HTML to JS.

Our unit tests are already in JS 😄 , so what we can we do to reduce/remove the complexity in this case?

Lets take this below HTML form snippet :

<form>
  <label>
    First Name:
    <input type="text" class="text-field" name="firstname" />
  </label>
  <label>
    Last Name:
    <input type="text" class="text-field" name="lastname" />
  </label>
</form>

Here if you had to pick the first field and then the second field and clearly they have both same classnames, how can we do it?

We can do something like this:

document.querySelectorAll("text-field")[0].value = "something";
document.querySelectorAll("text-field")[1].value = "something else";

There are some obvious problems with the above approach:

If there are many elements with a "text-field" class name before the given HTML snippet it would break the test.
If we reorder the fields our test will fail.
If somebody changes the class name the test will fail.

The answer to this problems turns out is to look at the page and think in terms of how a user would find elements. In the above case it’s the label!

Plain and simple we go find text field with particular label text to select the element so we would need something like:

getByLabelText("firstname");
getByLabelText("lastname");

This is so much better! and not to mention lot more semantic very less likely to break even if we change classnames add more html etc etc!

But how do we get selectors like this? look no further Kent C. Dodds has us covered 😃 in his react-testing-library :

https://testing-library.com/

Thanks for reading!

Some references :

Emotion

https://kentcdodds.com/blog/why-i-never-use-shallow-rendering

https://reactjs.org/docs/introducing-jsx.html

Functional JS

Varenya Thyagaraj — Tue, 04 Jan 2022 07:36:14 +0000

There are few functional patterns that have become popular/semi-popular in JS world.

Javascript as a language is extremely malleable to different patterns/preferences and that can be a good thing and a bad thing.

The good thing is it can align with any of varied sensibilities in developer community. Bad thing is within a team, individuals with different preferences can write code completely different way within the same app which hurts readability considerably.

So I wanted to tackle this topic with regards to functional patterns which I am particularly familiar with and have experienced its pro's and con's in the JS world.

So let’s get started then.

Point Free Pattern

This is basically a famous pattern in FP world where in you basically avoid passing function arguments when they basically pass through.

An example:

function addOne(x) {
  return x + 1
}

/*  I want to add one to a array of integers */

/* Way One */

const resultOne = [1, 2, 3].map(function (x) {
  return addOne(x)
})

/* Way Two */

const resultTwo = [1, 2, 3].map(addOne)

Basically instead of explicitly calling the function and adding another layer of anonymous function we are using the function name itself and the arguments get passed implicitly.

So is this a good to have? My Take No.

Reason being its very easy to shoot your self in the foot with this pattern. Lets see how:

/* convert string of numbers to integers using parseInt */

/* Using pointfree */

const pointFree = ['1', '2', '3'].map(parseInt)

/* Expected result : [1,2,3] *

/* Actual Result : [1,NaN,NaN] */

Why the NaN's? Well parseInt take's two arguments basically the string and the base to which it needs to convert to. And map actually sends in the current index along with current item in the list so the result is messed up.

So how to fix the above:

const fixedWay = ['1', '2', '3'].map(function (x) {
  return parseInt(x, 10)
})

See explicit is better than implicit! you end up writing few more lines of code but you get the expected result.

Pure Functions / Immutability

The idea is quite well known in the community. I won’t dig too deep here since it's been discussed/documented quite well now.

This is a great thing to simplify your code.

So my personal approach is:

To break down a particular functionality into pure functions.
Use immutable data structures to maintain state.

This approach works well with TDD. Makes code a lot more compositional to build other functionality in general.

But there are some nuances here that are worth discussing in the context of Javascript.

Javascript is a language that doesn’t enforce these rules at the language level. It's up to developers to make sure the rules are enforced. You can enforce it in the form of tests or other mechanisms but you cannot take it for granted.

The above fact has consequences. For example, some built-in functions in JS won’t adhere to those rules. A good example is the sort function and splice function which mutates the input you provide them. So when we are using them together with other parts of your software that is pure. The software as a whole won’t be pure anymore.

So you can run into subtle issues that can be hard to nail down and it's not just built-in JS functions that can cause this. It can be any piece of JS code - external libraries.

So should we give up on the idea? Well no - purism is good to have but trying to achieve perfection - not so much. Real life is rarely ever so pure 😉.

Local mutations are okay:

/* Here both variables i and total are */
function sum(array) {
  let total = 0
  for (let i = 0; i < array.length; i++) {
    total += array[i]
  }
  return total
}

Keep your state mutations in a single place to so that you can easily know where to look when something breaks:

/* Pure functions */
function increment(currentCount) {
  return currentCount + 1
}

function decrement(currentCount) {
  return currentCount - 1
}

/* impure functions with state updates and mutations */

function updateCount(initialCount = 0) {
  let currentCount = initialCount
  return {
    updater(updateFn) {
      currentCount = updateFn(currentCount)
    },
    getCount() {
      return currentCount
    },
  }
}

// Usage

const counterMethods = updateCount(10)

counterMethods.updater(increment)

counterMethods.getCount() // 11

counterMethods.updater(decrement)

counterMethods.getCount() // 10

If the above feels familiar then your right - its basically what Redux does!

Before I wrap this section up its important to know that immutability and pure functions can be costly when it comes to performance. If you are building a performance sensitive application updating objects through spread operator can be costly operation and can be source of problem.

But in my experience those instances have been rare. This is good advice to follow in general:

make it work, make it right, make it fast - Kent Beck

Reduce the use of reduce

Although reduce is quite a handy higher order function and very flexible it can be taken to an extreme.

A good example:

function getProp(obj, path) {
  return path.reduce((acc, pathItem) => {
    if (typeof acc === 'string') {
      return acc
    }
    if (acc.hasOwnProperty(pathItem)) {
      return acc[pathItem]
    }
    return 'Path not found'
  }, obj)
}

getProp({ person: { firstName: 'test' } }, ['person', 'firstName']) // returns "test"

Here we are having to go through all the items on the list even if the path is not present early on. This could be avoided if we use a simple for loop:

function getProp(obj, path) {
  let currentObj = obj
  for (let pathItem of path) {
    if (currentObj.hasOwnProperty(pathItem)) {
      currentObj = currentObj[pathItem]
    } else {
      return 'path not found'
    }
  }
  return currentObj || 'path not found'
}

The thing about reduce compared to map and filter is that reduce is lot more flexible than the others. So it’s tempting to use it in places where it may not really be a good fit.

This is a blurry line and takes some experience and intuition to say when one is good and when one is bad.

Function Composition/Pipe

Here the idea is that we can send a piece of data through series of operations and modify the data in someway to get a desired result.

This too is very good pattern in general but comes with a few drawbacks in the context of JS:

If any of the operations are having side effects or mutations this can cause really hard to find issues.
Debugging can be tricky if you are composing multiple functions together - determine where something went wrong can be hard.
Order of arguments is important and can be easily missed which can be a source of confusion.

There is a proposal currently in stage-2 to make this built into JS - the syntax looks like this:

const num = 5;
const double = num => num * 2;
const addOne = num => num + 1;
const num
                |> double
                |> addOne // output 11

This is very concise and cool thing to have. The above mentioned drawbacks can be avoided by using few constraints:

Always use small pure functions to pipe through.
In general keep the number of operations at 1-4 to avoid troubles.
Each function should take at most 1-2 arguments.

Conclusion

The goal here is to strike a balance between functional patterns and JS primitives.

Some other takeaways from my experience:

If there is a use case where mutating a variable is simplifying your code - go for it. Instead of coming up with a convoluted way of doing the same thing for the sake of purity.
When it comes to asynchrony separate the effect part from the processing part. Imperative shell and functional core.
Some libraries make some of the patterns I mentioned earlier turn bad to good. A good example is Ramda: https://ramdajs.com/. It’s built on point-free, currying, composition principles.
Don’t go for fancy Dependency Injection frameworks. Good old higher-order functions are than good enough to get the job done.

Thats it. Thanks for reading.

State, Time and Concurrrency

Varenya Thyagaraj — Mon, 12 Apr 2021 06:01:21 +0000

State, time, and concurrency are critical to building scalable software. Yet I had very little context over what they were and how they relate with each other when I started.

So this is the post that I would have wanted when I started.

State and Time

First, let's understand what state is.

It's a piece of data that changes over time.

Some examples of state in programming:

Current time
Location of a character in a game.
Bank Balance
No of. employees at an org
....

I hope you get the drift?

Now you might be wondering how they are related? Well, if you read the definition it's right there "changes over time".

Lets take a code sample:

let balance = 4; /* State */

function incBalance(amt) {
        balance += amt;
}

incBalance(4); /* Output 8 */
incBalance(4); /* Output 12 */

Here the variable balance is a piece of state which is updated by the function incBalance.

I called the function incBalance twice and each time the output was different. Let's look at how the execution went down:

The first time it was run:

incBalance(4);

/* let's execute the function by 
replacing the argument amt with the value i.e. 4*/

balance += 4;

/* the current value of variable balance is 4, so the result is */

balance = 8;

The second time it was run:

incBalance(4);

/* let's execute the function by 
replacing the argument amt with the value i.e. 4*/

balance += 4;

/* the current value of variable balance is 8, so the result is */

balance = 12;

So the function incBalance was updating a piece of state called balance and each time it was run, the output was dependent on the past value.

Another way of looking at this is whenever there is a stateful function (a function dependent on a piece of state) the output is not just dependent on the inputs but also when the function is run. In other terms, it is dependent on time.

A function that doesn't depend on the state is called a "pure" function.

TLDR:

When we introduce state we introduce dependency on time.

Time and Concurrency

Definition of concurrency from a popular StackOverflow answer:

Concurrency is when two or more tasks can start, run, and complete in overlapping time periods

Again from the definition itself, we can see "overlapping time periods".

Let's look at some code again:

let balance = 10;

function incBalance(amt){
    balance += amt;
}

setTimeout(() => incBalance(8), 1000* 3);
setTimeout(() => incBalance(8), 1000* 1);
setTimeout(() => incBalance(8), 1000* 2);

console.log({ balance }); /* Output 10 */

When you execute the above the output should be the value 10, which should be confusing?

In the above code sample, we are trying to achieve concurrency by using the setTimeout function. Basically, incBalance will be called concurrently three times at different time intervals so when all of them are done the final balance value should be 34 instead of 10.

So what is going on here and how do we get the expected result?

We need to ensure that all the concurrent calls are executed before we print the result.

How do we keep track of the fact whether all that needs executing is executed?

By using - State!

Let's update the above code and include some state to make sure all the concurrent stuff is completed at a given time:

let balance = 10;

/* state */
let task_completed = {
  task1: false,
  task2: false,
  task3: false,
};

function incBalance(amt) {
  balance += amt;
}

function concurrentIncBalance(taskName) {
  incBalance(8);
  task_completed[taskName] = true;
  /* time */
  if (
    task_completed["task1"] &&
    task_completed["task2"] &&
    task_completed["task3"]
  ) {
    console.log("Finished running all three tasks and the result is:");
    console.log({ balance });
  }
}

/* concurrency */
setTimeout(() => concurrentIncBalance("task1"), 1000);
setTimeout(() => concurrentIncBalance("task2"), 1000);
setTimeout(() => concurrentIncBalance("task3"), 1000);

Phew! See just adding a few requirements can make the code so much more complex!

The above callback based approach is a good segue to learning about Promises.

I hope this helps, thanks for reading.

Separation of concept and code

Varenya Thyagaraj — Tue, 06 Apr 2021 11:55:56 +0000

This is post is a deep dive into one of the points of my last blog post.

Links for the previous post:

https://varenya.dev/underappreciated-skills-of-dev/

https://dev.to/varenya/under-appreciated-skills-of-a-software-developer-39jn

This sub-title from the previous post is :

"Understand what the code does instead of what it looks like".

In tech and esp in the JS world, there is a lot of FOMO around frameworks and new languages. The temptation to get on with the latest and popular stuff is strong and can get overwhelming at times. Especially for beginners.

Clickbaity articles like "Why Vue is better than React?", "Why Kotlin is the next big thing?" further adds to the problems. (These titles are samples)

Having been in the same boat and sometimes still do feel the temptation. I wanted to understand why over time this need has reduced and put into words so that others might feel at ease too.

The key to dealing with FOMO, I figured is to deep dive into a syntax (one level below the top-level API). Understand what the concepts involved are and how it's represented in code.

I have observed that there is usually an overlap between the underlying concepts. Not necessarily the syntax.

So how does this help?

Learning curve reduces a lot.
It's easier to become a polyglot.
Reduces FOMO.

Lets take a few concepts to demonstrate what I mean:

Asynchronicity

Now within the JS ecosystem, there are now multiple ways of achieving concurrency/async:

Using Callbacks

The code for it looks something like this:

/* Example of a piece of code executing after a set time */

/* The first argument is the callback and the second being delay in ms */

setTimeout(() => console.log('done!'), 1000)

I hope this is example communicates the use of callbacks. Here the concept we achieve is concurrency.

Using Promises

Again, this is a fairly recent syntax introduced into the language which looks something like this:

/* Converting the setTimeout function into a promise  */
const delay = (duration) =>  
        new Promise((resolve,reject) => setTimeout(resolve, duration));

/* Usage */

delay(1000).then(() => console.log('done'));

The above example achieves the same thing we did in the previous section using callbacks but the syntax has drastically changed.

Using async/await

This is the latest way of achieving asynchronicity:


await delay(1000);

console.log('hello');

The same thing as before i.e. same underlying concept but different syntax - do you get my drift?

To gain a deeper understanding of how the Async code is executed in JS, I highly recommend this talk:

And this concept of async can be extending to different languages too where the syntax is drastically different but the concept is the same, some links to async syntax in different languages:

using Java:

Async Await in Java - DZone Java

using C#:

Asynchronous programming in C#

using Kotlin:

https://kotlinlang.org/docs/async-programming.html#futures-promises-and-others

These are just some examples. I hope folks are able to see the parallels between the JS examples and other languages.

TLDR:

grokking the underlying concept can help reduce the learning curve by a good margin

Reactivity

This is specific to JS frameworks.

To understand reactivity usually, the best example is excel sheets. How so?

Well, you can add value to a cell (let's call it A) and then add a formula based on that cell (let's call it B) and show the result in it. Every time the value of cell A changes the value of cell B changes automatically based on the formula.

Why is it important?

This is the core thing that all the frameworks try to achieve i.e. syncing state with view.

React:

In case of React it is achieved using the VDOM and reconciliation algorithm each time setState is called:

function DisplayCount() {
  const [count, setCount] = useState(0);
  return (
    <div>
      <h1> Count : {count} </h1>
      <button onClick={() => setCount((prevCount) => prevCount + 1)}>
        Inc
      </button>
    </div>
  );
}
/* 
    Once setCount is called React triggers the reconciliation algorithm 
  and figures out what the changes are and updates the UI.
*/

VueJS:

In the case of VueJS it is achieved using property accessors i.e. getters/setters. Whenever a variable is accessed or changed Vue automatically triggers a UI update thereby reacting to the changes.

var vm = new Vue({
  el: '#example',
  data: {
    message: 'Hello'
  },
  computed: {
    // a computed getter
    reversedMessage: function () {
      // `this` points to the vm instance
      return this.message.split('').reverse().join('')
    }
  }
})

/* Usage */

/*
    So every time message is updated you see the reversed string below. 
  Basically, it's reacting to the changes.
*/ 
<div id="example">
  <p>Original message: "{{ message }}"</p>
  <p>Computed reversed message: "{{ reversedMessage }}"</p>
</div>

For more info:

https://vuejs.org/v2/guide/reactivity.html

Svelte:

Lets look at the code directly:

<script>
    let count = 0;

    function handleClick() {
         count += 1;
    }
</script>

<button on:click={handleClick}>
    Clicked {count} {count === 1 ? 'time' : 'times'}
</button>

For more info:

https://svelte.dev/tutorial/basics

I hope my point is clear the above examples are drastically different in terms of their syntax but conceptually achieve similar things.

Some links to talk that has greatly helped me grok these concepts:

Talk by Evan You creator of VueJS:

Talk by Rich Harris creator of Svelte:

And FYI, there are different trade-offs to the different API's/Syntax we just saw that I leave up the reader to go deeper into. I wrote another post to help understand the different trade-offs:

https://varenya.dev/api-design/

https://dev.to/varenya/core-principles-of-api-design-part-1-4g37

I hope this helps, thanks for reading!

Under-Appreciated Skills of a Software Developer

Varenya Thyagaraj — Sun, 14 Mar 2021 00:00:00 +0000

This time I wanted to talk about certain skills I have noticed within myself and fellow devs that I think are crucial yet very much underappreciated.

Let's go over them one by one:

Empathy for legacy

Well, this seems obvious, doesn't it?

As a beginner, I used to have a knee-jerk reaction when exposed to a new codebase. I would judge it and think something along these lines:

Why did they use this? ugh
Why use this framework here?
This looks overly complicated?

Do you get the drift? But the truth is it's very likely that if I had been in the place of the original devs who built it, I may not have done a better job. The constraints of the time and place might have resulted in such a codebase (could be bad or good or somewhere in b/w).

Software development has a huge human component. That is often missed while reading/understanding code. Things like:

Clients with budget and time constraints
Devs with different preferences
Teams with different perspectives on how to approach a problem
Trade-off's b/w different tech stacks
Tools used while building the project

And probably many more that go into a software development project.

TLDR :

Don't judge too quickly.

Reading and Understanding Code

I spend most of my time reading and understanding code rather than actually writing it.

And to be able to write it well, I need to be good at reading and understanding the codebase. You might be wondering why?

Well, every codebase has a rhythm/style to it. And one must make sure to adhere to the style already present in the existing codebase.

Lets take a look at an example:

function createUser({ username, password }) {
  return {
    getUserName: function() {
      return username;
    },
    getPassword: function() {
      return password;
    },
  };
}

class User {
  constructor(username, password) {
    this.username = username;
    this.password = password;
  }

  getUserName() {
    return this.username;
  }
  getPassword() {
    return this.password;
  }
}

Both achieve similar goals with different trade-offs. The initial/lead developer might prefer either one of the above. We need to keep the big picture in mind and adhere to the style.

The difference in readability is massive if you don't.

TLDR:

Measure twice, cut once.

Understand what the code does instead of what it looks like

A lot of the time in programming what you see is not what you get.

A good example of this would be in JS:

function User(username) {
    this.username = username;
}

If you are not familiar with JS one would assume this is a function declaration. When in reality this is how we used to define the class structure before ES6 syntax came along.

It looks like a function but actually is a constructor. This kind of misdirection can come at the language level and also at the implementation level. One must be adept at understanding both.

TLDR:

if it looks like a duck, doesn't mean it is a duck.

Knowing when to act

It's very easy to get stuck in the analysis phase. Given that to solve a problem there are multiple ways with different trade-offs, it's very easy to fall into this trap.

I think this is where leadership kicks in - since somebody needs to decide on things and also take the blame for things when things will eventually go wrong!

“You can never know everything, and part of what you know is always wrong. Perhaps even the most important part. A portion of wisdom lies in knowing that. A portion of courage lies in going on anyway.”
—The Eye of the World, Robert Jordan

Simplicity is hard

Chaos is the natural state of things. If you consider all the things I mentioned earlier you can imagine just how much complexity can accrue over time and especially in software where the cost of making a change seems innocently less.

What could wrong if we just add one more variable to this function?

Ans:

So anything in your codebase that you are able to read, understand quickly and it just works? - It's boring, but it's the coolest thing you have just witnessed

“Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. And to make matters worse: complexity sells better. - Edsger W. Dijkstra”

And whenever simplicity is mentioned in software I try and include this talk from Rich Hickey:

Simple Made Easy

Care more about the artifact than the construct

This again I learned from Rich Hickey in the above talk. There was a Twitter thread started by Mattias Peter :

// Detect dark theme var iframe = document.getElementById('tweet-855691565460848640-696'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=855691565460848640&theme=dark" }

I replied with this:

// Detect dark theme var iframe = document.getElementById('tweet-855741548738338816-211'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=855741548738338816&theme=dark" }

He chose my reply, to do a full video which I think does a better job than I ever could:

Thanks for reading! I hope there are more folks who can relate to these.

I always wonder how does one test for these qualities? - Food for thought 😀.

Why React

Varenya Thyagaraj — Sun, 28 Feb 2021 20:00:03 +0000

React has been there long enough that there are people now using it who don't have context over what it brought to the table.

This blog post is for those who have started off UI development with React. Since history is somewhat lost to them I wanted to do a before/after version of UI dev to showcase how React is special.

Just FYI, This post is completely biased 😀.

Giving Frontend Development some well-deserved love.

What do I mean by this? I mean before React came along I felt the state of the industry was stagnant. I mean sure there were plenty of UI libraries like jQuery, Mootools, Bootstrap, etc.

Some approaches to writing CSS like BEM and utility-based frameworks were emerging. Bundling/build tools like browserify, requirejs, gruntjs, gulpjs were coming up in the JS world.

All these made things easier but not better. We were solving the same problems again and again. We were in a rut.

So how did React bring in the love? By Rethinking the best practices :

Watch this talk and witness history!

Now I will go through my take on things:

Making State Change Explicit

I mean this is so important! Before React came along making state changes was such a pain and required so much planning. Even with the best of constraints, the chances of things going wrong are high without React's API.

Making state changes via declarative API is something we take for granted today. Thanks to React.

A fun exercise for beginners would be to build a small non-trivial app without React. Let the code grow to observe how things can go wrong.

Let's take a look at some code here:

Declarative and beautiful to look at! How did we get by without these?

By glancing at this code itself I know at a high-level what's going on!

Code is for humans not computers and declarative API is a huge boost to readability.

Webpack

Okay, now you might be wondering how does this come into my love letter for React? Well, bundlers like Webpack got a huge boost thanks to React.

It solved some nasty problems :

Imports were clear and the dependency version was easier to manage.
A single bundled script at the end.
Bundling different asset types like css/images/js together.
Ease of cache-busting using hashes.
....

TLDR = gave us a module system (look up the history if you want to learn more)

My point is it solved a very crucial problem in web dev and React helped make it mainstream.

Components

Before React there was no concept of components. This basic abstraction now adopted by all the frameworks didn't exist before React.

Think about front-end development without them now

You can't reuse a piece of UI. You can't import your fancy CTA to any part of a page/app.
Repetitive HTML structures all over the place.
CSS can't be co-located.
Context switching between CSS/HTML/JS is exhausting without this unifying abstraction.
Do you want to add a click handler? Use addEventListener and attach a function and don't forget to removeEventListener.
....

I mean a whole list of conveniences go out the window.

Escape from MVC

This was huge for me. The model-view-controller pattern was the law of the land. Web Dev/UI Dev begins with the introduction of this pattern.

On the surface, it seems an organized and clean approach to build UI's. But it turns ugly if the constraints are not set right. Again the uncontrolled state changes reared its head.

Complexity everywhere.

Making FP mainstream

I have read books like Structure and Interpretation of Computer Programs (SICP). In it, the author proposes that computer science is about controlling complexity. I couldn't agree more.

This reinforces my earlier point - Code is for humans, not computers. Any constraint that helps do this is great. React provides those right constraints :

State changes happen in one direction
At a given time you are only looking at one component.
State change can happen only through the React API.

How is this all related to FP (Functional Programming)? Well, this what FP advocates for - minimizing state changes and providing an API to do it when you must. Limiting the number of things a developer needs to keep in mind - a single component.

FP has a rep for being hard to learn and use. React made that hard-to-use idea's into simple abstractions - thereby making FP mainstream.

The constraints imposed in building large software systems are the limitations of our own minds.
-Harold Abelson

Letting JS be JS(X)

By introducing one step to the build process. We open up a world of possibilities. The best part you don't have to learn another language altogether.

You can grow your skills with JS itself and as the language grows so does your ability to extend JSX which is such a powerful thing we take for granted now.

I can use template strings, observables, iterables, promises, generators. All these new JS features now at our disposal without resorting to convoluted hacks.

No more learning an additional templating language.

Composition

Well, I wrote whole another blog post on it:

https://varenya.dev/composition-react/

Unit Testing

Now, this is feasible thanks to components. The feedback loop from this is so good and I can't recommend this library enough : https://testing-library.com/docs/react-testing-library/intro/

You can TDD your UI now - this is powerful.

Example code straight from the docs:

Again a declarative API.

CSS-In-JS

Well, this is controversial but again I am a fan of this approach. Things are changing in this landscape but this again broke a lot of conventions that I think React triggered. Basically, it eliminated the complexity of CSS selectors.

It looks something like this:

Straight out of a example in css-in-js library called emotion. Here, what you see is what you get.

Contrast this with good old html/css:

Now can you tell me what color the text "hello" will be? Basically, you need to understand the specificity rules around CSS and determine what it is you get finally.

Making state management a thing

Well as soon as React introduced these new abstractions folks started thinking of state management as a thing and it kicked off a whole new set of libraries and ecosystems.

React initially recommended the flux pattern and few libraries emerged around the recommended pattern the most popular among them being Redux.

But now we have a whole sets of libraries with unique approaches:

https://xstate.js.org/ (Finite State Machines 🤯)
https://mobx-state-tree.js.org/intro/welcome (FRP)
https://github.com/pmndrs/valtio
https://github.com/pmndrs/jotai
https://mobx.js.org/README.html
https://recoiljs.org/
...

And probably so many more I am not aware of. They are not directly related to React but I think it triggered it.