DEV Community: Jack Herrington

100% Free Vector Search with OpenLlama, Postgres, NodeJS and NextJS

Jack Herrington — Tue, 02 Jul 2024 01:37:40 +0000

So you want to try out vector search but you don’t want to pay OpenAI, or use Huggingface, and you don’t want to pay a vector database company. I’m here for you. Let’s get vector search going, on your own machine, for free.

What Are We Doing?

Let’s take a quick step back and talk about what we are doing and why we are doing it. Vector search of AI embeddings is a way to create a search based on concepts. For example, searching for ‘pet’ might yield results for both dogs and cats. This is super valuable because it means that your customers can get better search results

To accomplish this we first take the text we want to search and send it to an AI for it to create an “embedding”. An embedding is a lengthy array of floating point values, usually between ~300 and ~1500 numbers.

The embedding value for cat, dog, and pet would be similar. So if you were to compare cat and dog they would be close, where dog and pizza would not be close.

What a vector database allows you to do is to store these vectors, along with their associated data (probably the original text of the data). Once the data is stored you can then query the database with a new vector to get any nearby results. For example, if we stored cat and dog with their embeddings in the database, we could then take a input text of “pet”, create the embedding for that, then use that to query the database and we would likely get back cat and dog.

Why Postgres and OpenLlama?

Postgres is a fantastic database that you can easily install and run locally. And with the pgvector extension to Postgres you can create vector fields that you can then use in your SQL queries.

There are multiple ways to install Postgres on your machine. On my Mac I used the Postgres.app to install Postgres.

OpenLlama is a very easy way to install and run AI models locally. I used Homebrew to install OpenLlama using brew install ollama.

For our simple test application we’ll load all the lines from the 1986 horror film Aliens.

Getting Set Up

There are lots of models to choose from with OpenLlama. For this application I chose snowflake-arctic-embed because its ideal for fast creation of embeddings. To install it I used the command ollama pull snowflake-arctic-embed .

The last part of the setup is to create a local Postgres database. You can name it whatever you like, I chose lines because we are searching lines from a movie.

With the database created we can use the psql command to run some commands. The first is to add the vector extension to the database. That enables the vector field type. To do that I use the create extension command:

CREATE EXTENSION vector;

Now we need to create a table to hold the line text as well as the vectors, here are the commands to create the table along with an index on the position value which is the position of the line in the script.

CREATE TABLE lines (
  id bigserial PRIMARY KEY,
  position INT,
  text TEXT,
  embedding VECTOR(1024)
);
CREATE UNIQUE INDEX position_idx ON lines (position);

The important thing to note here is the size of the vector. Different models create different sizes of vector. In the case of our snowflake model the embedding size is 1,024 numbers, so we set the vector size to that.

You will want to use the same embedding AI for both storage and query. If you use different models then the numbers won’t line up.

Creating The Vector Indexes

As you can imagine, comparing two 1,024 value floating point arrays could be costly. And comparing lots of them could be very costly. So these new vector databases have come up with different indexing models to make that more effecient. The Postgres vector support has different types of indexes, we will use the Hierarchical Navigable Small Worlds (HNSW) type to create three different indexes:

CREATE INDEX ON lines USING hnsw (embedding vector_ip_ops);
CREATE INDEX ON lines USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON lines USING hnsw (embedding vector_l1_ops);

Why three? Because there are multiple ways to compare two vectors. There is Cosine, which is often the default, and is good for doing more concept comparison. There is also uclidean, and dot-product comparison. Postgres supports all of these methods (and more).

Whichever you use you will want to make sure the indexes are enabled for that so that you can get high speed queries.

Loading The Database

With the model downloaded, and Postgres setup we can now start loading the database with our movie lines and their embeddings. I’ve posted the complete project that also includes a NextJS App Router UI on github. The script is located in the load-embeddings directory. The original data is from this script page.

Before you can load the data you’ll need to copy the .env.example files to .env and then change the values to match whatever your Postgres connection details.

To load the embedding into Postgres run node loader.mjs with Node 20 or higher.

The key parts of the script are the embedding generation:

import ollama from "ollama";
...
  const response = await ollama.embeddings({
    model: "snowflake-arctic-embed",
    prompt: text,
  });

Where we use the ollama library to invoke the snowflake embedding model with each line of text one-by-one.

We then insert the line into the database using an INSERT statement:

  await sql`INSERT INTO lines
    (position, text, embedding)
  VALUES
    (${position}, ${text}, ${`[${response.embedding.join(",")}]`})
  `;

The only tricky thing here is how we format the embedding which is by joining all the numbers together into a string and wrapping it in brackets.

With all the data loaded into the database it’s time we make a query.

Making Our First Query

To make sure this works there is a test-query.mjs file in the load-embeddings directory. To make a vector query we first run the model to turn the query into a vector, like so:

const response = await ollama.embeddings({
  model: "snowflake-arctic-embed",
  prompt: "food",
});

In this case the prompt is food and we use exactly the same process as we did on the loader script to turn that into an embedding.

We then use a SQL SELECT statement to query the database with that vector:

const query = await sql`SELECT
  position, text
FROM
  lines
ORDER BY
  embedding <#> ${`[${response.embedding.join(",")}]`} 
LIMIT 10`;

We are using ORDER BY to order the records in the database by their similarity to the given embedding then using LIMIT to get back just the top 10 most similar.

The <#> syntax in the ORDER BY is important because it defines which comparison algorithm to use. From the documentation our options are:

<-> - L2 distance
<#> - (negative) inner product
<=> - cosine distance
<+> - L1 distance (added in 0.7.0)

You can decide for yourself which comparison provides the best output for your application, but be sure to index the table properly based on that comparison method.

On my machine this test query yielded, amongst other things:

317 Guess she don't like the corn bread, either.

Which is a classic line from the movie that indeed references a type of food (corn bread).

Putting A User Interface On It

With a little extra effort I put a NextJS App Router interface on it that you can play with by running pnpm dev in the root directory of the project after the database has been loaded and the .env file set up properly.

This NextJS app uses exactly the same SELECT operation to query the lines from the database.

Conclusions

Obviously you’re not going to take an Aliens script searching application production. But from what I’ve shown you here you could search text content, product descriptions, comments, almost any kind of text.

Enjoy!

NextJS’s Amazing New Streaming Server Actions

Jack Herrington — Mon, 04 Dec 2023 23:57:05 +0000

One of the most exciting new advancements in NextJS/React architecture isn’t in NextJS. It’s in Vercel’s AI library. Even there, it’s tucked away in a little corner. It’s the experimental streaming API and what it can do could allow a revolution in application architecture.

The example in the documentation is a little terse, but the idea is simple. The library provides two key elements: experimental_StreamingReactResponse that allows for API routes or server actions to stream back responses, as well as the useChat hook that invokes the API route or server action and handles the streamed response.

This streaming functionality is exciting for two reasons. First, streaming used to be limited to just page routes. Now, we can stream data back from server actions and API routes, which can be super handy for longer running transactions.

However, the second reason is the real mind blower. Here is the server action code from my recent video on this topic:

  return new experimental_StreamingReactResponse(stream, {
    ui: async ({ content }) => {
      const albums = await getSpotifyAlbums(content);
      return (
        <div className="grid grid-cols-2">
          {albums.map((album) => (
            <div key={`${album.artist}-${album.name}`} className="p-1">
              <Album {...album} />
            </div>
          ))}
        </div>
      );
    },
  });

In this code, we are passing the AI response stream to experimental_StreamingReactResponse which invokes the ui method each time the AI comes back with new content. Within that UI, we can parse that AI response and turn it into any React UI we want. In this case, the div tags are rendered on the server and the Album component is a client component that renders on both the client and the server.

Think about how we would have had to do this before. The server action or API route would have sent back JSON that the receiving component would have to turn into UI. The more complex the layout and data, the more complex the logic would have to be on the client to manage laying out the components and giving them the right data.

With this new functionality in the AI library, we can just send back a React Server Component (RSC) stream that the React code on the client can decode back into component instances using the same mechanisms that we use to hydrate the component tree after a page navigation. It’s super slick!

Unfortunately, all of this awesome component streaming functionality is linked to their AI wrapper, and I don’t know about you but I don’t see those two things as linked. You can stream all kinds of responses, not just from AIs. But hey, this is what we have, and it’s amazing.

Let’s try it out with a server action. That could be as simple as this:

"use server";
import OpenAI from "openai";
import { OpenAIStream, experimental_StreamingReactResponse, Message } from "ai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY!,
});

export async function handler({ messages }: { messages: Message[] }) {
  const response = await openai.chat.completions.create({
    model: "gpt-3.5-turbo",
    stream: true,
    messages: messages.map((m) => ({
      role: m.role,
      content: m.content,
    })),
  });

  const stream = OpenAIStream(response);

  return new experimental_StreamingReactResponse(stream, {
    ui({ content }) {
      return <div className="italic text-red-800"> {content} </div>;
    },
  });
}

We start by importing the AI library from openai as well as the experimental_StreamingReactResponse function that we’ll use to stream the response back to the client.

Next, we export a function named handler, though you can name it anything you want. The only rule is that it has to be an async function. To use this with the useChat hook, this server action function needs to take an object containing an array of messages as the first argument.

This async server action function then sends the messages to openai (or whatever AI you want) to start the AI request and to open the stream. For OpenAI, in particular, we need to send the name of the model. There are lots to choose from. We also need to specify that we want a streaming response by setting the stream key to true. Finally, we need to send the list of messages. This list of messages is a set of prompts and responses to and from the AI. And we persist those messages so that the AI has the context of the previous user prompts to apply to the new prompt.

Once we’ve started the request, we turn that into an OpenAIStream stream from Vercel’s AI library. If you are using a different AI library, you’ll want to use a different stream adapter.

Finally, we return the experimental_StreamingReactResponse which takes the stream as input and calls the ui function each time a new streamed content message is received. It’s this function that turns that content into a set of tags, React Server Components, and client components, using JSX. In this example, we are simply wrapping the content in a div formatted with Tailwind.

Using Our Server Action

Once we have the server action created, we need to import it into our page. In the example provided by Vercel, we import the handler and then pass it to the Chat component as shown below:

import { handler } from "./action";
import { Chat } from "./chat";

export const runtime = "edge";

export default function Page() {
  return <Chat handler={handler} />;
}

This Page component is an RSC, and that’s good because only an RSC can import a server action. So, the whole job of this component is to import the handler and send it to the Chat component, which is a client component.

It’s the Chat client component that is going to talk to the server action handler that the component sends to it. Let’s take it apart in sections.

In the first part, we import the useChat hook that will drive the interface and we define our Chat component with the handler property that has the server action.

"use client";
import { useChat } from "ai/react";

export function Chat({ handler }: { handler: any }) {

Inside the Chat client component, we call the useChat hook from the AI library.

  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: handler,
  });

This gives us some handy data and functions that we can use:

messages — This is the array of messages to and from the user and the AI. Each message is an object containing role, content, and ui. Role is a string that tells us whether the message comes from the user or the AI. Content is the text version of the content. And ui has the rendered components from the ui method in the server action.
input — This is the text string of the input prompt.
handleInputChange — This is an event handler that you assign to the onChange of the input control.
handleSubmit — This is a form submit function that you assign to the onSubmit of the input form.

We then take that data and the functions and use them in our JSX return.

  return (
    <div className="container mx-auto p-4">       
      <ul>
        {messages.map((m, index) => (
          <li key={index}>
             {m.role === "user" ? "User: " : "AI: "} 
            {m.role === "user" ? m.content : m.ui} 
          </li>
        ))}
      </ul>
      <form
        className="flex gap-2 fixed bottom-0 left-0 w-full p-4 border-t"
        onSubmit={handleSubmit}
      >

        <input
          className="border border-gray-500 rounded p-2 w-full"
          placeholder="what is Next.js…"
          value={input}
          onChange={handleInputChange}
          autoFocus
        />

        <button type="submit" className="bg-black text-white rounded px-4">
           Send  
        </button>

      </form>

    </div>
  );
}

The code above takes the messages and formats them into a simple unordered list. You can, of course, lay this out and style it however you like.

It’s the ui portion that’s the real magic here. Of course, you can do whatever you want to layout this ui component tree wherever you want it. But the contents of the ui component tree are defined by the server action and that’s the beauty of this whole thing.

Finally, we need a form and an input to interact with AI.

      <form
        className="flex gap-2 fixed bottom-0 left-0 w-full p-4 border-t"
        onSubmit={handleSubmit}
      >

        <input
          className="border border-gray-500 rounded p-2 w-full"
          placeholder="what is Next.js…"
          value={input}
          onChange={handleInputChange}
          autoFocus
        />

        <button type="submit" className="bg-black text-white rounded px-4">
           Send  
        </button>

      </form>

    </div>
  );
}

The way the hook is set up, the handleSubmit function needs to be on a form tag, so we wrap our text input in a form tag and put the handleSubmit on the onSubmit.

Within the form we have the input tag that we connect with the input text by setting the value to input and the onChange to handleInputChange.

And finally, we have a submit button that automatically calls the onSubmit of the parent form when it’s clicked or when the user hits return in the text field.

All in all, it’s a combination of the mundane and the magical. This is just a simple client component that renders some data it gets back from a hook, and then has an input form that can send data to a server action. But wow, what we can do with this ui component tree that is returned from the server action (or API route)!

React Streamed Components Video Thumbnail

Conclusions

This nifty little experimental function in Vercel’s AI library opens up a world of possible dynamic server rendered UI. We’ve been talking about HTMX and how it puts rendering back into the server. Well, this implements a similar model but gives you even more power because the components returned from the server can be client components that are interactive. The client components returned from the server action can also access context defined in the application. That means that the dynamically returned client components can act just like any other component on the page! It’s very exciting!

Converting Figma To React the Fast and Easy Way

Jack Herrington — Wed, 29 Nov 2023 16:01:37 +0000

If you think that AI is going to replace web developers, look no further than trying to turn a Figma prototype into a working React application. Figma is a great tool for designers to build the UI of their dreams. But to turn those dreams into reality requires a lot of work.

Step one of converting a mockup is to create the HTML. Thankfully, the folks at Figma added a “developer mode” earlier this year to help out. Developer mode allows for 3rd party plugins to turn Figma mockups into code. Those plugins have improved the quality of those exports immeasurably.

There are a lot of plugins out there and they are continually improving. The best I’ve found so far is Visual Copilot from Builder.io (the folks behind the Builder CMS, Qwik, Partytown, and more.) These are smart folks and this is a smart tool. It uses a custom built LLM to do the initial conversion and the results, in my opinion, get you about 80% of where you need to go with the HTML/CSS.

Let’s take an example mockup for a home page and a product detail page:

Two page eCommerce mockup in Figma

The mockup looks pretty good. There is the home page mockup on the right hand side and the detail page on the left.

To start converting, we first switch the developer mode.

The developer mode toggle and Visual Copilot plugin in Figma

The big toggle switch at the top with the tag symbol moves us between designer and developer mode. Once we are in developer mode, we select the Plugins tab and search for the Builder.io plugin.

Once we select the plugin, we then select a section of the mockup we want to export.

Selecting a portion of the Figma mockup to start the code generation process

You can select as much or as little as you like. For small pages, you could select the whole thing. For larger pages, you can select each section and export them individually to build up a set of components.

I’ll just select the shoes hero section and then press the Generate code button.

The generated React/Tailwind code in the Builder UI

After just a few seconds of processing, I go into the Builder CMS UI that shows me the portion of the mockup I exported and the code at the bottom of the page.

Don’t worry if you aren’t using the Builder CMS in your application as you can just copy and paste the code from here into your own application.

Before we get into the generated code, it’s worth noting a few things.

At the top of the preview section is a selector where you can toggle between desktop, tablet, and mobile preview sizes. Visual Copilot generating responsive code that works at different breakpoints and the Builder CMS previewing the common breakpoints is a massive time saver.

In the code section, there are selectors for Framework and Styling. You can choose between all the common Frameworks: React, Vue, Svelte, Angular, Qwik, etc. Then you can choose between different Styling options like Tailwind, Emotion, Styled Components and more.

There is also a Fast versus Quality toggle. The Fast variant is free… and very fast. While the Quality version (which is a pay-for option) runs an extra GPT pass to try and further refine the code to produce more semantically structured results.

The Quality version also allows for AI refinement.

The AI refinement section presented in the “Quality” mode of the Visual Copilot

At the bottom of the display, you can have GPT further refine the code based on whatever prompt you give it. They even have some handy suggestions, like using NextJS’s Image component instead of using tags directly.

Finally, there is the copy button that you’ll want to use to get the code into your application.

In the video associated with this article, I’ll walk you through the entire process of converting this set of pages into a NextJS application–including extracting all of the images from the Builder CDN to local files in public and converting the tags to Image components. I’ve built a little script for this that you are welcome to use.

As I mentioned at the start of this article, this process will prove to you that AI is never going to do all the work required to turn mockups into production applications. Which seems odd, given that I then went on to cover an AI tool that does just that. Or does it? Turns out there is a lot left to be done by web developers to get to production.

It’s Not Always This Easy

The first issue is that these Figma-mocked UIs are not always this easy to work with. The example I used was a reasonable design. However, the reason that web developers have a love/hate relationship with Figma is that Figma allows designers to do anything they want, including creating very hard to implement designs.

AI tools will get you really far with reasonable designs, but for the weirder stuff they will get you a certain distance and then it’s all on you to not just implement, but also to negotiate with designers on what’s reasonably achievable and what’s not.

Responsiveness

As I mentioned, Visual Copilot does a decent job of doing responsive layout of a well-designed mockup. But there are often times when the mobile version of a UI will vary widely from the desktop version, and web developers will need to make up that gap. Take the navigation menu in the header:

The navigation portion of the header in both desktop and mobile mode

We only had the desktop version of the navigation menu in the mockup, so the AI did what it could with that and created a menu that scales really nicely down to mobile. But maybe the customer doesn’t want that, maybe they want a hamburger menu for mobile. It’s going to be up to the designer, product manager, and developer to work all that out.

Component Reuse

Probably the biggest issue I’ve seen with Figma mockups is that they rarely point out when we intend to reuse a component or what that reused component is supposed to be. Take these two sections of the mockup.

The same carousel on both the home page and the product detail page

One version says “Featured Products”, the other section on the detail page says “You may also like”, but the carousel style UI underneath certainly looks the same in both cases. So, can these sections be reused? Should they be reused? These are choices that can only be made in conversation with the product manager and designer.

Full Implementation

Of course, the 800lb gorilla in the room is to make it actually functional. Obviously, this means wiring up the data to make it all data driven. As well as integrating the eCommerce functionality required to allow the customer to actually buy the shoes. And on top of that, there is accessibility and internationalization.

That’s just to implement what we see in the mockup. There is often hidden functionality we don’t see, like low data states, or functionality that is implied but not rendered in the mockup, like the carousel for “Featured Products”. Does that always take four products? Does it scroll on the desktop? Does it flick on mobile?

Thumbnail for the video associated with this article

Conclusions

The great part about what’s left over after Visual Copilot has given you a fantastic start on the code is the fun part of engineering. That’s the great part about these AI tools. They aren’t meant to replace you. They are meant to take the tedious jobs, like converting a Figma mock into some reasonable HTML, and do those for you so that as a developer, you can focus on what you do best.

Personally, I’m endlessly fascinated by every part of UI, from design through to implementation. So, I’ve been working with Figma mocks and designers for years. I’m always on the lookout for tools that will help accelerate the process of going from a mockup to an app because I love seeing projects come to life. I strongly recommend trying out Figma plugins to help accelerate your workflow. And if you are looking for a plugin to go from Figma to React/Tailwind (and lots of other variations), Visual Copilot is the best plugin I’ve seen for doing that work.

Jack Herrington is a Principal Full Stack Engineer and YouTuber.

He is currently writing a course on Professional NextJS. Sign up for the free newsletter and get a full free tutorial on state management in the NextJS App Router.

How Do I Obscure API Keys On The Client?

Jack Herrington — Thu, 03 Aug 2023 16:39:31 +0000

Question : How do I obscure API keys on the client?

Answer : So you have a super powerful key to a 3rd party API that allows you to read/write any data you want from any customer, and you don’t want that falling into anyone else’s hands. What do you do? Gotcha. First off, you never send that to the client, not in any way, not obscured, not encrypted, not anything. Because try as you might to hide it, the key will end up showing clear as day in the Network tab of the Inspector as soon as you make a call on the client.

React + NextJS Questions And Answers Banner

What you should do instead is create a proxy API endpoint on your NextJS server. NextJS has always had great support for making API routes. You can use a route handler to create a proxy endpoint that will take the request from the client, and then make the request to the 3rd party API on the server. This way, the client never sees the API key, and you can do whatever you want with the response before sending it back to the client.

This proxy API also doesn’t need to be a 1-to-1 proxy, you can create an abstraction layer that allows you to do things like caching, or rate limiting, or even just to make the API easier to use. For example, if you have a 3rd party API that requires you to make 3 separate calls to get the data you need, you can create a single endpoint that makes those 3 calls and returns the data in a single response. In addition this new API ensures that if you want to change out that 3rd party API layer later that you only have to change it in one place.

To be honest, learning how to make the calls from a NextJS route handler will teach you how to make the calls to the API during Server Side Rendering, which is probably what you want to do anyway. So learning this technique of building proxy APIs is a two-fer!

Before you do any of this, make sure that the API key is indeed dangerous to be on client. Not all API keys are meant to be kept secret. Some keys, like Firebase keys, are meant to be used on the client. If you’re not sure, check the documentation for the API you’re using.

Jack Herrington is currently working on a course on NextJS, subscribe to the newsletter to get updates as well as React and NextJS tips, tricks and tutorials.

Why Not Just Fetch All The Data From The Client?

Jack Herrington — Thu, 03 Aug 2023 00:50:00 +0000

Question : With my NextJS application, why not just fetch all the data from the client? Why do we need to fetch data on the server?

React + NextJS Questions and Answers banner

Answer : There are several key advantages to fetching data on the server:

It’s Usually Faster — NextJS servers are usually deployed in the same Virtual Private Cloud (VPC) or cluster as the microservices they call. This means that the network latency between the NextJS server and the microservices is very low. This is not the case for requests from the client. A customer web client can be anywhere, and the network latency between the client and the microservices is much higher. This means that fetching data on the server is usually faster than fetching it on the client.
Microservices Stay Behind the Firewall — If only the NextJS server (or other microservices) can talk to the microservices then those microservices can stay behind the firewall. This means the surface area of attacks is limited to just the NextJS server itself. To make the API calls from the client we would have to expose the microservices to the public internet, which expands the security risk.
API Keys Are Hidden — If we are using third party serivces as part of our application then those probably have API keys. Those keys should never go out to the client. Making the requests only from the NextJS server ensures that those keys are hidden.
Search Engine Optimization — Server side rendering, where the server renders HTML in response to a request, has long been the preferred choice of Search Engine Optimization (SEO) professionals. It is an arguable point that nowadays search engines can index client side rendered pages, but if you have a site where SEO is an important point then most likely they will prefer the SEO relevant portions of the site to be server side rendered.

This is not to say that client side fetching is a bad thing, it’s not. There are many cases where client side fetching is the right choice. But if you are building a NextJS application then you should consider server side fetching as the default choice.

Jack Herrington is currently working on a course on NextJS, subscribe to the newsletter to get updates as well as React and NextJS tips, tricks and tutorials.

Does NextJS SSR Render Just the VDOM Nodes?

Jack Herrington — Thu, 03 Aug 2023 00:45:09 +0000

Does NextJS SSR Render Just The VDOM Nodes?

One of the great things about running a coding YouTube channel are the amazing questions that I get that give me some insights into how folks are thinking about React and NextJS. One recent question was about the NextJS App Router and whether it renders only VDOM or if it renders the entire page.

React + Next Questions and Answers banner

Answer : The App Router renders the requested route into both the Virtual DOM (VDOM) and the HTML. The HTML is what the customer sees on the page, and the VDOM is used to initialize the React Virtual DOM on the client.

There are three ways you can see exactly what’s coming out of your NextJS server for a given route. You can curl it on the command line using a command like this:

curl http://localhost:3000

Or if you don’t like the command line you can right click on the page in the browser and select View Page Source. The page source is exactly what came back from the server. It is not a live view of the current DOM of the page.

A third way is to bring up the inspector in your browser, select the Network tab, and then refresh the page. The first item in the list will be a GET request for the current route and the response is the HTML that came back from the server. You can click on that item and then select the Preview tab to see the HTML.

The response from the server looks pretty bad because all of the whitespace is removed between the tags, so it will all be on one line. In the browser if you are viewing the page source you can click on the Line Wrap option to at least see all the HTML.

At the top you will see the rendered HTML of the components. Below that is a large set of script tags that set up the Virtual DOM on the client. Those aren’t really VDOM nodes per se, they are a combination of component references, props, rendered DOM nodes, and other information that is used to initialize the client side VDOM.

If you want you can analyze that VDOM section further using this RSC Parser tool.

I can understand why you might think the server only generates VDOM nodes since React does so much client side rendering. But believe me, if the server only generated VDOM nodes then the page would be blank, and search engines would have nothing to index if they only looked at the HTML. That would give Search Engine Optimization folks a heart attack.

Jack Herrington is currently working on a course on NextJS, subscribe to the newsletter to get updates as well as React and NextJS tips, tricks and tutorials.

App Router Secret: Promises for Client Component Props

Jack Herrington — Tue, 01 Aug 2023 01:49:03 +0000

We know that we can send objects, arrays, strings, numbers, dates and more value types as properties from a React Server Component (RSC) to a client component, but did you know you can send a promise? You can use the new use hook to block on the promise inside the client component!

To see this in action be sure to check out the YouTube video:

In the video we demonstrate not just using the use hook but also using the popular React-Query library because with React-Query (or SWR) you can track a promise.

Let’s dig a little deeper into the React-Query use case by showing you also how to refetch data from the client component on the browser! You can go through the code yourself in this GitHub repository.

Setting up the API

Our simple NextJS App Router application is going to be a timer. When it’s completed it will look like this:

We’ll get that timestamp from an API endpoint on /api/timer . The code for the endpoint is very simple:

import { NextResponse } from "next/server";

export function GET() {
  return NextResponse.json(new Date().toISOString());
}

The home route page component is what makes the request to the API:

export default function Home() {
  const timerPromise = fetch("http://localhost:3000/api/timer", {
    cache: "no-cache",
  }).then((res) => res.json());

  return (
    <main className="max-w-3xl mx-auto mt-5">
      <RQTimer timerPromise={timerPromise} />
    </main>
  );
}

At the top we make the request to /api/timer but we don’t await it, and that’s important because we can pass that promise to the RQTimer client component.

Using The Use Hook For Promise Data

Let’s start our client component off by just using the use hook to get the data.

"use client";
import { use } from "react";

export default function FoodList({
  timerPromise,
}: {
  timerPromise: Promise<string>;
}) {
  const time = use(timerPromise);
  return (
    <>
      <div className="font-light text-xl mb-2">RQ/RSC Timer</div>
      <div className="font-extrabold text-3xl">{time?.toString()}</div>
    </>
  );
}

The use hook is a new hook that, given a promise, will monitor that promise once it’s fulfilled will force a re-render of the component and then return the time which we format into some decent looking Tailwind.

Didn’t know that you could send a promise as a property? Keep up with these kinds of NextJS tips and tricks by subscribing to the ProNextJS newsletter.

Now that’s all well and good, but let’s say we are React-Query fans, can we use that instead? Sure we can!

Using React Query Instead of Use

To use React-Query we need to wrap the page in a QueryProvider (shown in the repo code). Then we can use useQuery in our RQTimer component like this:

"use client";
import { useQuery } from "@tanstack/react-query";

export default function FoodList({
  timerPromise,
}: {
  timerPromise: Promise<string>;
}) {
  const { data: time } = useQuery(["timer"], () => timerPromise);

  return (
    <>
      <div className="font-light text-xl mb-2">RQ/RSC Timer</div>
      <div className="font-extrabold text-3xl">{time?.toString()}</div>
    </>
  );
}

The useQuery hook requires two parameters. First the query key, which is just a unique key to describe the request, in this case we use timer . And then a function that makes the request, and for that we can just give it a function that returns the timerPromise that we got as a property.

So easy!

But… what if we want to re-fetch that API endpoint once a second? To do that we can use React-Query’s fetchInterval option!

Using FetchInterval To Refetch The API On The Client

So first off we need to only run the refetch of the /api/time API on the client. So we’ll need to use a useEffect to set a flag that we can use to tell if we are on the client.

  const clientLoaded = useRef(false);
  const { data } = useQuery(["timer"], () => timerPromise);

  useEffect(() => {
    clientLoaded.current = true;
  }, []);

So we declare a clientLoaded ref and then set it in the useEffect. Then in our useQuery query function we check that clientLoaded ref.

  const { data } = useQuery(
    ["timer"],
    () => {
      if (clientLoaded.current) {
        return fetch("http://localhost:3000/api/timer", {
          cache: "no-cache",
        }).then((res) => res.json());
      }
      return timerPromise;
    },
    {
      refetchInterval: 1000,
    }
  );

If the clientLoaded ref is true then we know we are on the client and we get the data from the API, otherwise we are on the server (or in the first render on the client) and we return the original timerPromise.

Manually Refetching

In this case we are using the refetchInterval to make a new fetch to the API every second. But if you want you can force the refetch manually by getting the refetch function in the return of the useQuery hook. Then, whenever you want to refetch on the client just call that function.

Five Clever Hacks for React-Query or SWR

Jack Herrington — Tue, 05 Jul 2022 14:52:36 +0000

So dealing with the double-render issue in React 18 has finally gotten you to use a API handling library like react-query or swr. Awesome! But did you know you can get more out of that 12Kb (or 4Kb in the case of swr) than just API fetching? Here are five pretty novel uses for these awesome libraries.

Prefer a Video?

If you prefer to watch your technical story then have a watch over on YouTube.

Simplified Multiple Fetches

We tend to think of a useQuery hook as one hook per fetch. But let’s say you have two fetches to make. For example you have a login system where you first fetch to do the login and then fetch again to get the user information once you have their user ID.

You might start with something like this:

import { useQuery } from "react-query";

const fetchLogin = () => fetch("/login.json").json();
const fetchUser = (id) => fetch(`/${id}.json`).json();

const MyComponent = () => {  
 const { data: login } = useQuery("login", fetchLogin);
 const { data: user } = useQuery(
    "user", () => fetchUser(login.id),
    { enabled: login?.id }
  );

  return <div>{JSON.stringify(user)}</div>
}

In this model we cascade these two useQueryhooks. First we get the login, and then once the login is returned with a non-zero id then we enable the second query. Now… this works. But such pain! And imagine if it were more complex with three or more requests. There has to be a better way!

There is of course, we can just make a login function, like so:

const login = async () => {
  const resp = await fetch("/login.json");
  const { id } = await resp.json();

  const userResp = await fetch(`/${id}.json`);
  const user = await userResp.json();
  return user;
};

And use that instead in our component.

const MyComponent = () => {  
 const {data: user} = useQuery("login", login);
  return <div>{JSON.stringify(user)}</div>
}

You see, useQuery monitors any function, it could be a single fetch or it could be a function like this that makes multiple fetches with logic and such. Or it might not be a fetch at all (as we will soon see.) The point here is to start thinking outside the fetch box.

But before we leave the topic of fetch lets look at just two more handy variants.

For example, if you have an array of fetches to make in series you could do something like this:

const getTextData = async () => {
  const out = [];
  for (const name of ["a", "b", "c"]) {
    const resp = await fetch(`/data_${name}.json`);
    out.push(await resp.json());
  }
  return out;
};
...
const {data: textData} = useQuery("textData", getTextData);

In this case we are using a for loop to iterate through an array of values and then requesting the JSON for each of them before returning it all. BTW, if you like this example but don’t like for and you replace it with forEach it won’t work and that’s because forEach isn’t compatible with async/await , but hey, try it for yourself and enjoy.

If you wanted to do this in parallel you might try something like this:

const getTextData = async () => Promise.all(
  ["a", "b", "c"].map(async (name) => {
    const resp = await fetch(`/data_${name}.json`);
    return await resp.json();
  })
);

This will also work, but I don’t think the order of results is guaranteed, it will depend on how fast the individual fetches resolve.

I hear you screaming: “Enough with fetching! Show me something new!” Fine, fine!

Keeping Track Of Time

Let’s make a stopwatch using SWR. No, I’m not kidding!

We’ll start by create a functor (a function that makes functions) and this functor will make use a function that knows the time at which it was created. And then when we call it, it will return the delta between that start time and the current time, in seconds.

const createStopwatch = () => {
  const startTime = Date.now();
  return () => {
    return Math.round((Date.now() - startTime) / 1000);
  };
};

Now when we call createStopwatch we will get a function back that knows its start time and will give us the elapsed time since then. And we can use that in a component with that uses the useSWR hook, like so:

import useSWR from "swr";

const Stopwatch = () => {
  const stopwatchRef = useRef(createStopwatch());
  const { data } = useSWR("stopwatch", stopwatchRef.current, {
    refreshInterval: 100,
    dedupingInterval: 100,
  });

  return <div>Time: {data}</div>;
};

We start by creating a ref to hold the function, which, because we use useRef will only get called once on mount. Then we use that function (by getting it from stopwatchRef.current) in the useSWR hook, which calls that function every 100 milliseconds because of the refreshInterval.

That’s it! Boom! A stopwatch! We are using the refresh interval built into SWR to, instead of fetching data every hundred milliseconds, to instead call this synchronous function.

Now this is cool and all, but not really practical, let’s try something related but more practical.

Monitor Those Logs!

Let’s say you want part of the UI to monitor a log. And the log updates a lot , like easily every 100 milliseconds. But you don’t want to update the UI that often because, let’s face it, the log isn’t that important. So can we use react-query (or SWR) to throttle the update speed? Sure we can!

First, let’s simulate a log:

const subscribeToLog = () => {
  let log = [];
  let logIndex = 0;

  setInterval(() => {
    log.push(`${logIndex}: ${Date.now()}`);
    log = log.slice(-3);
    logIndex++;
  }, 100);

  return () => log;
};

const logListener = subscribeToLog();

Now we have a logListener global that is a function that returns the log messages that are continuously being built by the interval function. Every 100 milliseconds that interval adds a new log message and then trims the log down to the most recent three events (just to keep the display size small.)

Now we use react-query to fetch the log, but only once every second:

import { useQuery } from "react-query";

const Logger = () => {
  const { data } = useQuery("log", logListener, {
    refetchInterval: 1000,
  });

  return (
    <div>
      {data?.map((line, index) => (
        <div key={line}>{line}</div>
      ))}
    </div>
  );
};

In this case we are using the useQuery hook to poll the logListener (which returns the last three items in the log) every 1000 milliseconds (1 second). And that throttles the display so that we don’t update it too often.

Of course, the swr code is dramatically different. You have to change refetchInterval to refreshInterval and add that dedupingInterval . It’s crazy, I know, the differences are staggering.

Ok, so that really was a different use for something like react-query or swr, but what else have I got? How about getting GPS coordinates!

Five Clever Hacks for React-Query and SWR image

Going Home With GPS

Anything you can wrap in a promise you can monitor with these awesome libraries. Take getting your GPS coordinates for example. Here we wrap the browsers built-in getCurrentPosition in a promise:

const getGPSCoordinates = async () =>
  new Promise((resolve, reject) => {
    navigator.geolocation.getCurrentPosition(
      (position) => {
        resolve({
          latitude: position.coords.latitude,
          longitude: position.coords.longitude,
        });
      },
      (error) => {
        reject(error);
      }
    );
  });

And then we can call it, with… lemme just pick one… swr this time:

import useSWR from "swr";

const GPS = () => {
  const { data } = useSWR("gps", getGPSCoordinates);
  return <div>Location: {JSON.stringify(data)}</div>;
};

And there you go, GPS coordinates in your component.

The key point here is that; anything you can turn into a synchronous function, or a promise based async function, is going to work with these libraries. Anything. At all.

Parallelize With Web Workers

Which brings me to Web Workers, which are really handy bits of code that you can run in a different thread on the page. Take a simple one like this:

export const multiplyNumbers = (a, b) => {
  postMessage({ type: "result", result: a * b });
};

This little guy can multiple two numbers and send back the result. Such a good little function! Anyway, we can integrate it into our code super simple using react-query (or swr). We first need to load it:

import worker from "workerize-loader!./worker";

const workerInstance = worker();

Now we have an instance of the worker that we have loaded using the workerize-loader Webpack loader. We can then wrap that in a promise based function which calls it, waits for the result, and then resolves the promise with the output.

const multiplyNumbers = async (args) =>
  new Promise((resolve) => {
    workerInstance.addEventListener("message", (message) => {
      if (message.data.type === "result") {
        resolve(message.data.result);
      }
    });

    workerInstance.multiplyNumbers(args.a, args.b);
  });

All we do is create a promise, register a listener on the instance, then make the call. Once the listener fires we have our result. And here is the component code that uses this function, this time using react-query.

import { useMutation } from "react-query";

const WebWorker = () => {
  const { data: result, mutate } = useMutation(
    "multiply", multiplyNumbers);

  const [valueA, setValueA] = useState("10");
  const [valueB, setValueB] = useState("20");

  return (
    <div>
      <input
        value={valueA}
        onChange={(evt) => setValueA(evt.target.value)}
      />
      <input
        value={valueB}
        onChange={(evt) => setValueB(evt.target.value)}
      />
      <button onClick={
        () => mutate({ a: valueA, b: valueB })
      }>Multiply</button>
      <div>{result}</div>
    </div>
  );
};

In this case I’m using the useMutation hook from react-query because it makes a little more sense in that it’s actively executing something. And that’s kind of important as you look to perhaps use some of these patterns; make sure your queries are modeled as useQuery and that actions that potentially change things use the useMutation hook.

Of course that doesn’t help you with swr, that doesn’t have a mutation hook, but there is still a way to do this with swr as well.

Now, let’s finish this off in grand style, by answering the age old question; if you have react-query or swr, do you need a state manager?

Built-In State Manager?!?

Both swr and react-query manage caches, right? They can both make sure that if you access the same query key from two different spots you’ll get the same data.

Which means that you can use that cache to store bits of data you want, globally, and when you update them, they will update everywhere they are “subscribed”. Which is like … 80%? of what a state manager does?

So we can create a custom hook called useSWRGlobalState that does exactly this global shared stuff, check it out.

import useSWR from "swr";

const useSWRGlobalState = (key, initialData) => {
  const { data, mutate } = useSWR(key, () => initialData);
  return [
    data ?? initialData,
    (value) =>
      mutate(value, {
        revalidate: false,
      }),
  ];
};

You give this hook a key , which is the query key we’ve been using all of over the place, and whatever you want for the initial data. And it in turn uses useSWR to get the current data as well as the mutate function.

The hook then returns an array that looks like the return from useState . It is an array where the first item is the current value, and the second is a setter function.

The setter function is where the magic happens. We call that mutate function we got back and give it the new value but we tell swr not to re-fetch the value. Which basically means; set the cache, but that’s all.

Now we can wrap this in some components!

const StateEditor = () => {
  const [value, setValue] = useSWRGlobalState("sharedText", "");

  return (
    <input value={value}
       onChange={(evt) => setValue(evt.target.value)} />
  );
};

const StateViewer = () => {
  const [value] = useSWRGlobalState("sharedText", "");

  return <div>{value}</div>;
};

const GlobalStateDemo = () => {
  return (
    <div>
      <StateEditor />
      <StateViewer />
    </div>
  );
};

Here we have two separate components, one that edits the state, that’s the StateEditor component, and one that views the shared state, that being the StateViewer. When you type into the StateEditor the change shows up immediately in StateViewer .

No kidding, really. No context. No Redux. No atoms. Just that one little hook, and the “fetch library” you already have.💥 Crazy, right?

Now, would I use this for realsies? In a big application that maybe already has a state manager, then for sure not. But if all I needed to share around my component hierarchy was a single piece of state, like maybe the user ID, and a JWT, then yeah, I might just do this.

BTW, this is possible with React-Query as well.

const useRQGlobalState = (key, initialData) => [
  useQuery(key, () => initialData, {
    enabled: false,
    initialData,
  }).data ?? initialData,
  (value) => client.setQueryData(key, value),
];

This hook returns an array, just like before, where the first item in the array is the current value, that we get with useQuery and then second value is a setter function that sets the cache data for the query directly on the react-query client.

Wrapping It Up

I hope you’ve had a fun ride looking at a bunch of different ways you can wring more value out of the kilobytes you are adding to your app code by bringing in these awesome libraries. They really are an invaluable addition to the React ecosystem.

React 18 useEffect Double Call for APIs: Emergency Fix

Jack Herrington — Wed, 01 Jun 2022 02:37:18 +0000

So you’ve upgraded to React 18, enabled strict mode, and now all of your useEffects are getting called twice.

React 18 API Calls need an Emergency Fix!

Which would normally be fine, but you have API calls in your useEffects so you’re seeing double traffic in development mode. Sound familiar? No problem, I’ve got your back with a bunch of potential fixes.

Fix #1: Live With It

A legitimate option is simply to live with it, it’s dev-mode behavior only. It’s also trying to help you by stress testing your components to ensure they are compatible with future features in React. But, hey, I get it, you are here, you don’t like it, so … let’s just move on.

Fix #2: Remove Strict Mode

It is strict mode that is causing the double render, so another option is just to remove it. Out of the box the StrictMode component is used in index.js and it’s here:



root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

So simply remove it, like so:



root.render(<App />);

That being said, I don’t recommend this route since strict mode does a lot of good checking on your app code so you really consider keeping it around.

Fix #3: Use An Abort Controller

Another fix is to use an AbortController to terminate the request from the first useEffect . Let’s say this is your code:



  const [people, setPeople] = useState([]);
  useEffect(() => {
    fetch("/people")
      .then((res) => res.json())
      .then(setPeople);
  }, []);

This code was fine (sort-of) in React 17, but strict mode in 18 is showing an issue by mounting, unmounting, and re-mounting your component in development mode. And this is showing off that you aren’t aborting the fetch if it hasn’t been completed before component un-mount. So let’s add that AbortController logic.



  useEffect(() => {
    const controller = new AbortController();
    fetch("/people", **{  
      signal: controller.signal,  
    } )
      .then((res) => res.json())
      .then(setPeople);
    return () => controller.abort(); 
  }, []);

The code is pretty simple. We create a new AbortController then we pass its signal to the fetch and in our cleanup function we call the abort method.

Now what’s going to happen is that the first request will be aborted but the second mount will not abort and the fetch will finish successfully.

I think most folks would use this approach if it weren’t for one thing, that in the Inspector you see two requests where the first one is in red because it has been cancelled, which is just ugly.

Fix #4: Create a Custom Fetcher

A cool aspect of a JavaScript promise is that you can use it like a cache. Once a promise has been resolved (or rejected) you can keep calling then or catch on it and you’ll get back the resolved (or rejected) value. It will not make a subsequent request on a fulfilled promise, it will just return the fulfilled result.

Because of that behavior you can build a function that create custom cached fetch functions, like so:



const createFetch = () => {
  // Create a cache of fetches by URL
  const fetchMap = {};

  return (url, options) => {
    // Check to see if its not in the cache otherwise fetch it  
    if (!fetchMap[url]) {
      fetchMap[url] = fetch(url, options).then((res) => res.json());
    }

    // Return the cached promise  
    return fetchMap[url];
  };
};

This createFetch function will create a cached fetch for you. If you call it with the same URL twice, it will return the same promise both times. So you can make a new fetch like so:



const myFetch = createFetch();

And then use it in your useEffect instead of fetch with a simple replace:



  const [people, setPeople] = useState([]);
  useEffect(() => {
    myFetch("/people").then(setPeople);
  }, []);

Here is why this works. The first time the useEffect is called the myFetch starts the fetch and stores the promise in the fetchMap . Then the second time the useEffect function is called it the myFetch function returns the cached promise instead of calling fetch again.

The only thing you need to figure out here is cache invalidation if you choose to use this approach.

Fix #5: Use React-Query

None of this is a problem if you use React-Query. React-Query is an amazing library that you should honestly be using anyway. To start with React-Query first install the react-query NPM package.

From there create a query client and wrap your application in a QueryProvider component:



import { QueryClient , QueryClientProvider } from "react-query";

...

const AppWithProvider = () => (
  <QueryClientProvider client={new QueryClient()}>  
    <App />
  </QueryClientProvider>  
);

Then in your component use the useQuery hook, like so:



  const { data: people } = useQuery("people", () =>
    fetch("/people").then((res) => res.json())
  );

Doesn’t that just look better anyway? And it doesn’t do the double fetch.

This is just the tiniest fraction of what React-Query can do. And folks are using React-Query for more than just fetch, you can use it to monitor any promise-based asynchronous work you do.

Fix #6: Use a State Manager

I’m not going to go into code detail on this one since it depends a lot on the state manager you use. But if you use Redux then use then if you use the RTK Query functionality in Redux Toolkit you won’t be effected by this double-mount behavior.

What You Shouldn’t Do

I strongly recommend against using useRef to try and defeat this behavior. There is no guarantee that the component that gets called on the first useEffect is the same one that gets called the second time around. So if you do things like use useRef to do tracking between mounts then… it’s unclear if that is going to work.

Also the code that is currently going around to create a useEffectOnce doesn’t work. It does not call the cleanup function. Which is far worse behavior than having useEffect called twice.

What You Should Do

If you like this content then you should check out my YouTube channel. I cover topics like this all the time. In fact I’ve already covered the useEffect topic already over there, but I haven’t covered the API call aspect specifically … yet.

React Apps with Plugins Using ScandiPWA

Jack Herrington — Tue, 09 Feb 2021 15:41:02 +0000

With the move to NodeJS we lost the architectural pattern of creating pluggable applications. Java application servers can be configured at runtime to override pluggable elements, like database drivers, etc. Can we get there with NodeJS? Yep! With the webpack plugin from ScandiPWA we can have our cake and eat it too!

Banner Image

Setting Up An App

The first job is to set up a Create React App. Let’s do this in a directory called scandi-test.

% mkdir scandi-test
% cd scandi-test
% yarn create react-app my-host-app

Once that’s done we can go into the host application and update it to use scandipwa-scripts

% cd my-host-app
% yarn add @scandipwa/scandipwa-scripts

With that installed we can change the package.json to change the start and build scripts to be:

"scripts": {
  "start": "scandipwa-scripts start",
  "build": "scandipwa-scripts build",
  "test": "react-scripts test",
  "eject": "react-scripts eject"
},

There are other tools that work like this; craco, and react-app-rewired.

We then add a scandipwa key to the package.json like so:

  "scandipwa": {
    "type": "theme",
    "extensions": {
    }
  },

This registers the application as a host application. In ScandiPWA parlance this is called a “theme” because ScandiPWA is a company that produces a theme for Magento, and this plugin extensibility layer allows ScandiPWA customers the ability to customize that theme down to the function, class, member variable, and more, level.

Making The App Extensible

Now that our Create React App based application is enabled for ScandiPWA plugin it’s time to make it extensible. Over in App.js let’s create a new function called getData and define it like so:

/** @namespace my-host-app/getData */
const getData = () => [1,2,3,4];

And lets re-define our App React component like so:

function App() {
  return (
    <div>{JSON.stringify(getData())}</div>
  );
}

Is this going to be pretty? Heck no! But we aren’t concerned with looks, we want that yummy functionality. So let’s get to building out our extension.

Building The Extension

In the parent directory of my-host-app we create a peer directory called extension and initialize it as an NPM repo with some directories like so:

% mkdir extension
% cd extension
% yarn init -y
% mkdir -p src/plugin
% touch src/plugin/extension.plugin.js

This is where it starts getting good! Next let’s define our plugin by updating the contents of extension.plugin.js to:

export default {
  "my-host-app/getData": {
    function: () => [10,20,30,40,50]
  }
}

Enabling the Extension

Now we can go back to the package.json of my-host-app and enable the extension:

  "scandipwa": {
    "type": "theme",
    "extensions": {
      "extension": true 
    }
  },

This will enable the extension. And while we are in package.json let’s go to dependencies and add:

"extension": "link:../extension",

So let’s start up your application and see where we’ve gotten to:

% cd my-host-app
% yarn
% yarn start

That first yarn establishes the link with the extension project and the second one launches the application.

If everything works you should see:

The override data shown in the host application

I know it doesn’t look like much. But think about the potential. You can mark anything in your application is extensible and have one or more extensions installed that override the functionality.

Right now with this tooling you can override:

Functions (as we did)
Classes
Static member variables
Member variables
Member functions

Where To Go Next

One the Blue Collar Coder channel we have a video that walks through getting this all working using a far more interesting example. Go have a look!

Conclusions

Pluggable architectures are a great way to move your highly verticalized application to something that is an extensible platform that folks can customize to match their requirements. This ScandiPWA webpack plugin (which can be used outside of CRA) allows you to specify extension points and override their functionality. That gives you a great new architectural tool for NodeJS applications.

Mastering Typescript for React Hooks

Jack Herrington — Wed, 03 Feb 2021 15:43:05 +0000

Mastering TypeScript for React Hooks

So you want to use TypeScript in your React application, but even the hooks are giving you grief. Well, let’s get you comfortable with how to use TypeScript typing with those hooks and get you on your way.

Banner image

This article is meant to supplement the excellent React TypeScript Cheat Sheet which you should definitely take a look at.

useState

useState is a fun one because we use it all the time and most of the time it’s fine, until it’s not. Take this example:

const [myNumber, myNumberSet] = useState(10);

const onClick = () => myNumberSet(20);

TypeScript is totally fine with this because the typing on useState looks at the initial value, sees that it’s a number and sets this type to this:

const [myNumber, myNumberSet] = useState<number>(10);

So any number is going to be fine.

The problem comes up when you have something like this:

const [myAccount, myAccountSet] = useState(null);

const onAuthResponse = () => myAccountSet({ user: "foo", ... });

TypeScript has no idea that what you initially set to null could potentially be an account record. So what you need to do is tell it that:

interface IAccount {
  user: string;
  ...
}
const [myAccount, myAccountSet] = useState<IAccount | null>(null);

const onAuthResponse = () => myAccountSet({ user: "foo", ... });

Now TypeScript understands that your myAccount value can either be null or an object that matches the typing of IAccount.

A similar issue happens with arrays. Take this example:

const [myNumbers, myNumbersSet] = useState([]);

const onClick = () => myNumbersSet([10, 20, 30]);

TypeScript is going to give you a really odd error about trying to use a number[] when a never[] is expected. Which actually makes sense because, as far as TypeScript knows the only valid value is an empty array (i.e. never[]). It has no idea you intend to store numbers in there.

So the fix for this is to type it

const [myNumbers, myNumbersSet] = useState<number[]>([]);

const onClick = () => myNumbersSet([10, 20, 30]);

And now TypeScript will be happy again because even an empty array is a valid type of number[].

useEffect

The great thing about useEffect that that it doesn’t take any types. So if you are looking to make sure you are typing it correctly, fear not, you are.

If you want to check that for yourself then right click on the word useEffect in your VS Code and use the Go to Type Definition command to go to where useEffect is defined in the React source.

useEffect takes two arguments, the first is a function with no parameters that either returns void, or returns another function (the cleanup function), which takes no arguments and returns a void.

IMHO, using Go to Type Definition should be your first stop any time you run into an issue in TypeScript.

useContext

Getting useContext typed properly really comes down to getting the createContext call typed properly. For example, you might have something like this:

const MyContext = createContext(null);

Which basically leaves TypeScript with no clue about what could potentially be in the context and so it leaves it at; the context must always contain null. Which is probably not what you want.

The easiest way to handle this would be, if you want either null or some data, to define it like this:

interface IMyContextState {
  userID: string;
}
const MyContext = createContext<IMyContextState | null>(null);

Which tells TypeScript that the context must either contain an object that matches IMyContextState or null.

If you have a default state it gets a lot easier:

const myDefaultState = {
  userID: "";
}

export type MyContextType = typeof myDefaultState;

const MyContext = createContext(myDefaultState);

export default MyContext;

In this case we don’t need to tell TypeScript that the context has the types in myDefaultState it already knows that, but we now export the schema of the default state as MyContextType. So that we can then use it when we call useContext like so:

import MyContext, { MyContextType } from './store';
...
const ctx:MyContextType = useContext(MyContext);

The typing of ctx is a bit of overkill in this case because useContext already knows the types from MyContext and you can just get away with:

import MyContext from './store';
...
const ctx = useContext(MyContext);

useReducer

Typing useReducer is a lot like typing Redux, so it’s a two-fer, if you get this right, you are that much closer to Redux typing. So useReducer takes two things, the reducer function and the initial state. Let’s start off with the initial state:

const initialState = {
  counter: 0,
};

Next we need some actions. Now in Javascript we wouldn’t type these at all but in TypeScript we can and we should type them, and that would look like this:

type ACTIONTYPES =
  | { type: "increment"; payload: number; }
  | { type: "decrement"; payload: number; };

And then the reducer is going to look something like this:

function myReducer(state: typeof initialState, action: ACTIONTYPES) {
  ...
}

const [state, dispatch] = useReducer(myReducer, initialState);

And this will give you hinting on the state and also ensure that any call to dispatch will need to match one of the variants in ACTIONTYPES.

useRef

Typing useRef, particularly when it comes to use refs with DOM elements, which is a pretty common use case is straightforward. Let’s say you have something like this:

return (<input ref={inputRef} />);

In your code then the corresponding useRef would look like this:

const inputRef = useRef<HTMLInputElement | null>(null);

And specifying the types isn’t 100% necessary here either. The only trick is to make sure you get the right type for the corresponding DOM element.

If you are going to use a ref to hold data then you can do something like this:

const intervalRef = useRef<number | null>(null);

If you are, for example, holding a reference to an interval.

useMemo

The typing on useMemo is all about what is produced by the factory function that you put in there. For example:

const [numbers] = useState([1,2,3,4]);
const filteredNums = useMemo(
  () => numbers.filter(n => n > 2),
  [numbers]
);

In this case the typing on filteredNums is inferred by TypeScript to be number[] because of the output of the factory function. If you wanted to type it you could do:

const filteredNums: number[] = useMemo(
  () => numbers.filter(n => n > 2),
  [numbers]
);

But you really don’t need to. TypeScript is very, very good at figuring out the return type of a function. In fact, if you want to you can use the ReturnType utility type to get the return type from a function like so:

type MyFunctionReturnType = ReturnType<typeof myFunction>;

You can find more information on the amazing array of utility types on the TypeScript language site.

Video Version

If you want to see an in-depth walk through of a lot of this information and much more check out the associated YouTube video:

Conclusions

The more I work with TypeScript and React the more that I am convinced that it’s worth the investment. You get the benefits of hinting as you code. You are communicating your intent through the types. And you get the benefits of a type safety check at compile time.

Hopefully this article will help you realize these benefits as you try out using TypeScript in your React projects and you learn to master the typing of your React hooks.

Are Typescript Types Alone Enough Documentation?

Jack Herrington — Mon, 25 Jan 2021 16:36:44 +0000

Here is an interesting question; are the types in your Typescript project enough documentation, on their own, to allow someone to learn the library?

That was the question I asked myself when I saw the Amazon had released a React UI library for AWS. They only released the code and the Typescript types. No documentation. No storybook or styleguidist. Just the code and the types.

Now I love digging into UI libraries. And this was a fun challenge. Can I learn the library from just the source? So I started a Create React App using the typescript template, then installed the @awsui/components-react library and started my sleuthing.

The first thing to realize is that VS Code loves Typescript, and in this journey it will be your invaluable partner. For example, you can right click on the import and ask for the definitions, as shown in the screenshot below.

Using VS Code to navigate to the definitions

And that takes you to the central definitions file that lists all the components and their properties.

Just a few of the fascinating components in the AWS UI toolkit

Jackpot! Now I know what’s in there. But how to use it?

Getting The Layout Right

Right away what captured my eye was AppLayout because, really, there are two types of toys in a UI library; layout components and interactive components. And job number one is creating the layout, and job two is putting stuff in that layout.

Naming is important, right? AppLayout. Sounds good! I’m making an App, check. And I need a Layout, double check. AppLayout it is! It also helped that it was alphabetical and in the A's.

So I dropped an AppLayout on the page and right away I got a three column layout with a collapsible navigation panel to the left and a tools panel to the right, with a content section between. How did I know that? Because of the types!

The types for content, navigation and tools properties on the AppLayout component

So this is not only telling me what sections are available, but also the kind of content they are expecting, in this case ReactNode types which are, basically, anything renderable in React.

I replaced the entire contents of the canned App component with this code:

function App() {
  return (
    <AppLayout
      navigation={<div>Navigation</div>}
      content={<div>Content</div>}
      tools={<div>Tools</div>}
    />
  );
}

Then I ran yarn start and right away I got a pretty decent looking responsive layout with three columns and collapsible sections. I was off to the races!

As Expected, With Exceptions

I’m not one to just play with components, I want to build something real-ish. So I created a Pokemon filter application. Some things, like Button were very straightforward.

<Button onClick={() => alert("hello!")}>Say Hello</Button>

This worked out of the box, but the button itself wasn’t very visually appealing by default. Once again, Typescript to the rescue!

The potential variants of Button styling

Using Go To Definition in VSCode I was able to select a Button component and drill until I found the ButtonProps which showed me that there is a variant prop that I could use to select different styles of button. I ended up going with link.

Another moment came when I was working with the Input component, which, as you might guess is a text input. Everything was going great until I got to onChange where the event structure was a little different from what we expect.

Hinting on the event data from onChange

In this case event.target.value wasn’t there. Once again, VS Code to the rescue and I was able to find event.detail.value was the droid I was looking for instead.

Finding a New Pattern

Eventually I needed to put up a list of the Pokemon. I not only found a great Cards component to do that, I also found a great way of defining generic React components. Check this out!

Typescript generics for React components

So you give Cards an array of items and then you provide content functions that render the content for each item.

Anyway, because Typescript, the AWS team defined Card as a generic and when you pass an array in for that items property Typescript automatically infers the type of the array and then applies it to the other parts of the property definitions as required. In your content function you get type safety and hinting around the type from the array of items. And you don’t, as a consumer, need to do any additional work to make that happen. It just works, for free, just the way you’d expect it.

Seeing this was the most exciting moment of discovery on this journey. For some reason or another I hadn’t really thought about using generics for React components. So it was a genuine “Ah ha!” moment that I got from reading the code. This is not something I would have gotten from the documentation (most likely).

In addition to generics, seeing this pattern where you give the component some data and a render function (which is a pattern throughout the toolkit) was great to see. I’d seen it before in toolkits like Apple’s iOS List component. But I’ve rarely seen it on the web and I think it’s a good architectural pattern that’s worthy of more extensive use.

Video Version

If you want to see my exploration of the AWS UI toolkit for yourself check out the YouTube version.

Conclusion

The two big takeaways I learned from this sleuthing experience was:

Learn To Read Code — Code is truth. And learning how to read someone else’s code can be a source of information, inspiration, education, all the good stuff. And what’s even better is that when you build that code reading muscle you can start to look at Github not just as a place to “grab code” but as the best code reading library the Universe has ever known.
Typescript Is Documentation — Sure typing doesn’t answer all the questions. But it answers a lot of them very quickly, and it’s free. So the cost/benefit ratio is outstanding. Typescript is worth it, y’all.
Bonus: Naming Is Really Important — I only scratched the surface on this library. There are tons of components. The only moments that I really struggled was with the naming on some of these components. For example; what’s a “Tile”? (Turns out it’s a radio button). Now that’s not really a fault of the folks at AWS. Naming is tough. But as frontend developers it would probably benefit us, at some point, to normalize what we name our UI elements, beyond Button.