DEV Community: Sidney Alcantara

How to add hotlink protection to your web fonts with Netlify Edge Functions and Deno

Sidney Alcantara — Fri, 05 May 2023 01:33:22 +0000

Because the people crafting quality typefaces deserve to be paid

While working on my redesigned personal website sidney.me, I chose to use a paid font, Klim Type Foundry’s The Future Mono. It was my first time using a paid web font for a project, so I read (skimmed) the web font licence agreement. Curiously, it included this clause (emphasis mine):

3d. Web Fonts File Protection

You agree to use reasonable measures to ensure the Web Fonts are available only for the process of styling text on Your Website. At a minimum, and by way of illustration not limitation, reasonable measures include a.) preventing unlicensed third-party access, i.e. hotlinking and b.) not allowing direct download of the Fonts unrelated to the process of styling text for Your Website.

In simpler terms, you must make sure you deploy the web font files so that no one can paste the URL in their browser to download it and that other sites can’t use the web font by linking to that URL.

I’ve been using Netlify for years to host my websites, so I searched for a way to do it natively. I came across this Netlify support forum thread that unfortunately concluded with, “We don’t really have a solution for this particular problem yet, but it’s one that might be covered in the future. Stay tuned!” In March 2020.

The replies did have some suggestions, like:

using Netlify Functions (overkill for this use case and potentially slow) to check the Referer header in the HTTP request,
using a more performant Cloudflare Worker to do that check (but I don’t want to sign up for another service), or
serving the fonts from an S3 bucket or separate Apache server (breaking the whole point of the serverless stack) and using .htaccess.

Since the last reply on that thread, Netlify has released Edge Functions. They run JavaScript or TypeScript code using Deno, the hot new JS runtime meant to succeed Node, on CDN servers that are much closer to users than where typical serverless functions run. In other words: fast, fast, fast.

The code

Setting up Edge Functions on your Netlify project is incredibly simple. Start by creating a netlify/edge-functions directory and making a JS or TS file in that. Then, export a default function for Netlify to run. Let’s call it hotlink-protection.ts and return “Hello world” as a basic response:

// netlify/edge-functions/hotlink-protection.ts

export default () => new Response("Hello world");

Then, we need to specify which routes this function should run on. In this case, we can limit it to font files. I only serve fonts in the WOFF2 format, as browser support is excellent. In the root of your project, create a file called netlify.toml, and set the path and the function name to be the same as the file name you chose.

# netlify.toml

[[edge_functions]]
    path = "/*.woff2"
    function = "hotlink-protection"

While we’re here, let’s add a cache header for our fonts. They’re immutable, static assets that won’t change: we need to specify that to the browser.

# netlify.toml

[[headers]]
    for = "/*.woff2"
    [headers.values]
        Cache-Control = "public, max-age=31536000, immutable"

Now let’s get it working. We’re effectively creating middleware that modifies and returns the response—either the font file itself or a 403 Forbidden HTTP error. Edge Functions receive two arguments, request and context. We’ll need to await the response (the font data) using context, so let’s change it to an async function. Here’s the definition in TypeScript:

import type { Context } from "https://edge.netlify.com";

export default async (
    request: Request,
    context: Context
): Promise<Response> => {
    // ...
};

Then, let’s get the Referer header from request and use a regular expression to test that it’s coming from our website. If there’s no Referer or it doesn’t match, return a 403.

import type { Context } from "https://edge.netlify.com";

export default async (
    request: Request,
    context: Context
): Promise<Response> => {
    const referer = request.headers.get("referer");

    const regex = /^https?:\/\/(.*\.)?sidney\.me(\/.*)?$/;

    if (!referer || !regex.test(referer)) {
        return new Response("Forbidden", { status: 403 });
    }

    // ...
};

That regex looks gnarly, but most of its special characters are used to escape the special characters / and ., which are also part of URLs. It matches sidney.me and any subdomains and directories, so make sure to change those characters (excluding the \. in between) when copy-pasting this snippet. For an explanation of the regex, check it out on regex101.com. The top-right panel highlights what each character does.

Then to return the font file as-is, we can use await context.next() to get the next HTTP response. Here’s the final code:

// netlify/edge-functions/hotlink-protection.ts

import type { Context } from "https://edge.netlify.com";

export default async (
    request: Request,
    context: Context
): Promise<Response> => {
    const referer = request.headers.get("referer");

    const regex = /^https?:\/\/(.*\.)?sidney\.me(\/.*)?$/;

    if (!referer || !regex.test(referer)) {
        return new Response("Forbidden", { status: 403 });
    }

    const response = await context.next();
    return response;
};

Now you can commit your changes and deploy your project to Netlify as usual. Once deployed, navigating to the font URL should display “Forbidden”.

Note: if you want your fonts to work in branch deploys, you’ll need a separate check for your Netlify domain name. In my case, it’s sidney-me.netlify.app and the branch name and -- is prefixed: branch-name--sidney-me.netlify.app. Here’s my regex:

    const prodRegex = /^https?:\/\/(.*\.)?sidney\.me(\/.*)?$/;
    const devRegex = /^https?:\/\/(.*\--)?sidney-me\.netlify\.app(\/.*)?$/;

    if (
        !referer ||
        !(prodRegex.test(referer) || devRegex.test(referer))
    ) {
        return new Response("Forbidden", { status: 403 });
    }

Subset your font files for additional protection and a performance boost

Of course, this protection itself isn’t bulletproof. Someone can easily spoof the Referer header or use DevTools to download the font files. The font data must be sent to users for their browsers to display the font. Once it’s on the user’s machine, there will always be a way for them to extract the font file.

One way to mitigate this is to subset the font files only to include the characters you use on your site, excluding characters for other languages or writing systems. Also, fonts typically have OpenType features like alternate characters, primarily for stylistic purposes, and we can follow Google Fonts’ example in removing them from the files we serve.

For my site, I used pyftsubset from FontTools to subset. Follow these instructions to install (note that you’ll need Python installed too). Here’s an example:

pyftsubset the-future-mono-regular.woff2 \
--output-file="the-future-mono-regular--subset.woff2" \
--flavor=woff2 \
--unicodes="U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+2000-206F,U+2074,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD"

This command subsets to the Latin character set and some punctuation symbols for a total of 385 characters. It also removes discretionary OpenType features as described in the subset docs. In my case, this more than halved the size of the font file, from 30.1 kB to 14.7 kB. This can add up and improve your Lighthouse performance scores.

Then specify the Unicode range in your CSS to help the browser use an appropriate fallback font if necessary.

@font-face {
    font-family: "The Future Mono";
    src: url("the-future-mono-regular--subset.woff2") format("woff2");
    font-display: swap;
    font-weight: 400;
    unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA,
        U+02DC, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215,
        U+FEFF, U+FFFD;
}

Serve fonts from a different project

If you’re publishing your site’s code as open-source, make sure not to include your font files in that repo. You don’t want to be part of the problem of GitHub becoming the web’s largest font piracy site.

Add your font files to your .gitignore file with *.woff2.
If you have already publicly published the repo, permanently remove the files from history (instructions).
Then, deploy your font files in a new private repository as a separate Netlify project.
You can keep the files in your local repository for testing and leave your CSS unchanged, pointing to files in the same domain name. Then use a rewrite to point to the other project’s domain name like so:
```
# netlify.toml

[[redirects]]
  from = "/fonts/the-future-mono-*"
  to = "https://your-project.netlify.app/the-future-mono/the-future-mono-:splat"
  status = 200
  force = true
```

Important: you must have the hotlink protection Edge Function on both projects for the matching URLs. If someone goes to your primary URL to get the font, the Edge Function on the private project will get the correct Referer header and let anyone download or hotlink it.

You can copy the same function into your primary repository and set it to the appropriate paths in netlify.toml:

# netlify.toml

[[edge_functions]]
  path = "/fonts/the-future-mono-*"
  function = "hotlink-protection"

Now you have the best of both worlds: the font is available for local development while protected when deployed.

Additional resources

Here are some suggestions from TypeKit on additional protections for web fonts.
Alternatively, you can use glyphhanger to subset web fonts based on what characters are used on your website.
You can see which characters are in a Unicode range and do maths on ranges using Unicode Range Interchange.
You can see which OpenType features and all the characters are included in your font file using Wakamai Fondue.
If you’re curious about web font performance, Zach Leatherman wrote an excellent guide on its implementation on CSS-Tricks.

And, of course, check out my personal site, sidney.me, and let me know what you think on Mastodon, @notsidney@indieweb.social.

How and why you should store React UI state in the URL

Sidney Alcantara — Fri, 10 Dec 2021 02:50:19 +0000

Deep linking in React, as simple as useState

Have you ever used a complex web app with many features, modal windows, or side panels? Where you get to the perfect state with just the right information on the screen after a few clicks through different screens, but then you accidentally close the tab? (Or Windows decides to update?)

It would be great if there were a way to return to this state without going through the same tedious process. Or be able to share that state so a teammate can work on the same thing you are.

This problem could be solved with deep linking, which is used today in mobile apps to open the app to a specific page or UI state. But why does this not exist in many web apps?

⏭ Click here to skip to the solution and code snippets.

Bring back deep linking on the web

The emergence of single-page applications (SPAs) has allowed us to craft new user experiences that are instantly interactive on the web. By doing more on the client side using JavaScript, we can respond to user events immediately, from opening custom dialog windows to live text editors like Google Docs.

Traditional server-rendered websites send a request to get a new HTML page every single time. An excellent example is Google, which sends a request to its servers with the user’s search query in the URL: https://www.google.com/search?q=your+query+here. What’s great about this model is that if I filter by results from the past week, I can share the same search query by simply sharing the URL: https://www.google.com/search?q=react+js&tbs=qdr:w. And this paradigm is entirely natural for web users—sharing links has been part of the world wide web ever since it was invented!

When SPAs came along, we didn’t need to store this data in the URL since we no longer need to make a server request to change what is displayed on the screen (hence single-page). But this made it easy to lose a unique experience of the web, the shareable link.

Desktop and mobile apps never really had a standardized way to link to specific parts of the app, and modern implementations of deep linking rely on URLs on the web. So when we build web apps that function more like native apps, why would we throw away the deep linking functionality of URLs that we’ve had for decades?

Dead-simple deep linking

When building a web app that has multiple pages, the minimum you should do is change the URL when a different page is displayed, such as /login and /home. In the React ecosystem, React Router is perfect for client-side routing like this, and Next.js is an excellent fully-featured React framework that also supports server-side rendering.

But I’m talking about deep linking, right down to the UI state after a few clicks and keyboard inputs. This is a killer feature for productivity-focused web apps, as it allows users to return right to the exact spot they were at even after closing the app or sharing it with someone else so they can start work without any friction.

Notice how the URL updates to add `#modal="webhooks"` as the modal opens.

You could use npm packages like query-string and write a basic React Hook to sync URL query parameters to your state, and there are plenty of tutorials for this, but there’s a more straightforward solution.

While exploring modern state management libraries for React for an architecture rewrite of our React app Rowy, I came across Jotai, a tiny atom-based state library inspired by the React team’s Recoil library.

The main benefit of this model is that state atoms are declared independent from the component hierarchy and can be manipulated from anywhere in the app. This solves the issue with React Context causing unnecessary re-renders, which I previously worked around with useRef. You can read more about the atomic state concept in Jotai’s docs and a more technical version in Recoil’s.

The code

Jotai has a type of atom called atomWithHash, which syncs the state atom to the URL hash.

Suppose we want a modal’s open state stored in the URL. Let’s start by creating an atom:

Then in the modal component itself, we can use this atom just like useState:

And here’s how it looks:

And that’s it! It’s that simple.

What’s fantastic about Jotai’s atomWithHash is that it can store any data that useState can, and it automatically stringifies objects to be stored in the URL. So I can store a more complex state in the URL, making it sharable.

In Rowy, we used this technique to implement a UI for cloud logs. We’re building an open-source platform that makes backend development easier and eliminates friction for common workflows. So, reducing friction for sharing logs was perfect for us. You can see this in action on our demo, where I can link you to a specific deploy log: https://demo.rowy.io/table/roadmap#modal="cloudLogs"&cloudLogFilters={"type"%3A"build"%2C"timeRange"%3A{"type"%3A"days"%2C"value"%3A7}%2C"buildLogExpanded"%3A1}

Decoding the URL component reveals the exact state used in React:

A side effect of atomWithHash is that it pushes the state to the browser history by default, so the user can click the back and forward buttons to go between UI states.

This behavior is optional and can be disabled using the replaceState option:

Thanks for reading! I hope this has convinced you to expose more of your UI state in the URL, making it easily shareable and reducing friction for your users—especially since it’s effortless to implement.

You can follow me on Twitter @nots_dney for more articles and Tweet threads about front-end engineering.

rowyio / rowy

Low-code backend platform. Manage database on spreadsheet-like UI and build cloud functions workflows in JS/TS, all in your browser.

✨ Airtable-like UI for managing database ✨ Build any automation, with or without code ✨

Connect to your database and create Cloud Functions in low-code - without leaving your browser.
Focus on building your apps Low-code for Firebase and Google Cloud

Website • Documentation • Chat with us • Twitter

Live Demo 🛝

💥 Explore Rowy on live demo playground 💥

Features ✨

20211004-RowyWebsite.mp4

Powerful spreadsheet interface for Firestore

CMS for Firestore
CRUD operations
Bulk import or export data - csv, json, tsv
Sort and filter by row values
Lock, Freeze, Resize, Hide and Rename columns
Multiple views for the same collection

Automate with cloud functions and ready made extensions

Build cloud functions workflows on field level data changes
- Use any NPM modules or APIs
Connect to your favourite tool with pre-built code blocks or create your own
- SendGrid, Algolia, Twilio, Bigquery and more

Rich and flexible data fields

30+…

View on GitHub

UI design trends of today and how to apply them in your apps

Sidney Alcantara — Tue, 23 Nov 2021 15:02:42 +0000

From outline icons to switch designs, the UIs of major OSes are all starting to look similar.

If you’re like me, you get excited by major new software releases and enjoy reading roundups of every new feature and UI tweak, comparing them to the previous version. With the release of Windows 11, Android 12, and iOS 15 this year (and last year’s macOS Big Sur), I’ve noticed quite a few similarities in the designs of the major operating systems. And in some cases, what appears to be convergence. These range from the use of outline icons to variable UI fonts and inset elements.

These similarities show that designers are coming up with new standards for digital user interfaces and offer hints as to where UIs are headed over the next decade and how the thinking behind UI design is shifting.

I’m a front-end engineer at Rowy, and I recently redesigned its UI with the primary goal of improving the information-dense desktop experience. I also wanted to move away from the 2015-era, mobile-first Material Design we were following to something more modern and better complements the UIs we see today. The trends I noticed helped inform this redesign and will hopefully assist you when designing your apps.

Outline icons by default, complementing text
Personalization based on the wallpaper
Rounded corners are harmonious
Variable UI fonts with optical sizes
Focus on content
Inset everything
Differentiation beyond color
And a standard switch design

Outline icons by default, complementing text

Icons play a significant part in UIs and have proven to be emblematic of the trends in UI design over the past few decades. When graphical user interfaces were novel, icons mimicked real-world objects to as great detail as possible within the limitations of display technology; they were made to bridge the user experience from the physical to the digital. This can be seen from Susan Kare’s icons for the original Macintosh to Microsoft’s photorealistic set in Windows Vista and 7.

When UIs moved from this skeuomorphic style to a “flat” style with Windows 8 and iOS 7, icon design also shifted to a monochromatic, outline style. And with the release of Android 12 and Google’s new version of Material Design called Material You (or Material Design 3), all major OSes now use this style.

But why does everyone use this style for icons? Designers aren’t just investing more into creating cohesive and complementary icon sets for design systems—designers are making icons just like typographers make type. This style recognizes how integral icons have become to the user experience, as important as text itself.

Icons are designed with similar considerations to typefaces, and often appear alongside text.
— Material Design 3 guidelines

Iconography [is] designed to integrate seamlessly with San Francisco, the system font.
— Apple Human Interface Guidelines

Apple’s SF Symbols is the prime example of this: it’s designed in the same nine weights as their system font and aligns with the text’s cap height. This also means icons can respect the user’s accessibility settings for bolder UI text.

Apple’s SF Symbols icon set is designed with the same nine weights as their system font. Source: Apple

They also align to the system font’s cap height to better complement text. Source: Apple

Uber’s design team similarly made icons to complement their UI font Uber Move, set in three weights for different degrees of emphasis.

Source: Uber

Icons are designed to mimic our writing systems

So it becomes clear why system icons are monochromatic and drawn with outlines: instead of mimicking physical objects, they mimic our writing systems. Modern alphabets, Latin in particular, are fundamentally composed of lines instead of filled shapes. They made a similar transition from pictographs that resemble physical objects (like Egyptian hieroglyphs) to more abstract representations of physical objects. For example, you don’t need to draw a bird to talk about a bird; you just write a set of letters or symbols that mean ‘bird’.

By designing icons that resemble text, designers recognize that icons are a vital medium to communicate information to users. Icons can break language barriers, while text remains essential for those unfamiliar with icons. And designing both to appear similar can reduce cognitive load when reading icons alongside text.

Another benefit of defaulting to outline icons is that filled icons can now be used to represent state or emphasis, like bold text. For instance, “active states are represented with filled icons” in Material Design 3 and Apple suggests to “use the fill variant to indicate selection” or “to give a symbol more visual emphasis” in their Human Interface Guidelines. Twitter’s 2021 redesign takes this a step further, with the active page being differentiated only by a filled icon and bold text, without any color change.

Twitter only uses filled icons and bold text to differentiate the current page. The jury is still out on whether this is accessible enough.

Predicting the future is a fool’s errand, but looking at the history of writing systems, they’ve seen convergence on what symbols represent what sounds and ideas: the symbol ‘a’ always means ‘a’ and the letters ‘ant’ mean the insect named ant. There remain variations in how these symbols are drawn (the capital A can have serifs or not) but they all fundamentally look the same across different typefaces.

Iconography seems to be heading this way too—the “add folder” icon above has the same basic design across Apple’s, Google’s, and Microsoft’s icon sets: an outline of a closed Manila folder with a ‘+’ symbol. As icons become more commonplace in GUIs, people tend to agree on what symbols represent what ideas and differences become purely stylistic, just like text.

How can I achieve this? When using icons in your apps, you should use the icon set designed with the font you’re using: this is trivial if you’re using system fonts, which now all have system icon fonts. But if you’re using another UI font, there probably isn’t an icon set designed specifically for that font. Using an outline icon set is sufficient, such as the open-source Feather icons or Iconic.app. And if you were using filled Material Icons like us, you can easily switch to the outline style. (It seems Google has now made this style the default as part of Material Design 3.)

Personalization based on the wallpaper

Personalization is becoming an increasingly important element of UI design. While customization has been present since the ’90s with custom wallpapers and themes, designers are now finding more ways to personalize UIs throughout the system, primarily from the user-selected wallpaper. Google’s Material Design blog says it best: “Users customize their desktops in the physical and digital worlds with images that are personal and provide comfort and joy.” The wallpaper is the first thing a user sees and serves as a backdrop for the UI. So it’s a no-brainer to adapt the rest of the UI to this element.

The technique of blending the wallpaper into the UI has been around for a while, from the semi-transparent dock and menus in Mac OS X’s Aqua interface to the translucent window title bars in Windows Vista and 7’s Aero Glass theme. iOS 7 famously brought blurred backgrounds to many parts of its UI, with entire screens like the Notification Center and Control Center using a blurred version of the user’s wallpaper.

More recently, the wallpaper has started to influence almost the entirety of the UI. When Apple introduced dark mode in macOS Mojave, they incorporated a tint in window backgrounds based on the wallpaper, called Desktop Tinting. According to the Human Interface Guidelines, it “helps windows blend more harmoniously with their surrounding content.”

Source: Apple

They later expanded on this in their major redesign in Big Sur by applying Desktop Tinting to light mode as well:

On the left, the window background is tinted purple to match the wallpaper. On the right, tinting is disabled, and the window background is pure gray.

Windows 11 introduces a similar element with its Mica material, which their design guidelines describe as “an opaque material that incorporates the user’s theme and desktop wallpaper to create its highly personalized appearance.” It also uses this tint as a signifier for the currently active window.

Taking it to the next level is Material You in Android 12, which tints the app’s background, the bright accent colors for buttons and other controls, and down to the more neutral text colors. An entire color palette is generated from each user’s unique wallpaper.

It’s a critical element of their new design philosophy, “Instead of form following function, what if form followed feeling?” It’s a radical rejection of the status quo in UI design, which seeks a “universal” design with the most technically superior interface to fulfill user needs. It will be interesting to see if others follow suit in this philosophy.

Source: Google

Material Design built a color system

What the Material Design team left out of their announcements is how they achieved this, especially since they say they “had to find a way for any color combination to also have accessible contrast […] without testing each one.” Diving into the recently-published Material color utilities repo on GitHub unveils all the mystery:

The Material team built their own color system: ‘hue, chroma, tone’, or ‘HCT’, based on the CAM16 and CIELAB (or LAB) color appearance models. CAM16 is a successor of LAB, designed to match how humans perceive color.
Key to these models is the ‘tone’ or L* value, which describes a color’s perceived luminance or lightness, with an L* value of 0 being black and 100 being white. This is very useful when creating accessible color palettes, ensuring colors have enough contrast based on perceived luminance.
On the web, the WCAG 2 guidelines prescribe a minimum contrast of 4.5:1 for body text. Directly using perceived luminance as a value to describe a color makes this much more straightforward, as the Material team explains: “Unlike contrast ratio, measuring contrast in L* is linear, and simple to calculate […] a difference of 50 guarantees a contrast ratio >= 4.5.”
With this knowledge, all that’s left is to generate a palette of colors with different tones or L* values and apply any hue to it. Then use sufficiently contrasting pairs for UI elements. For example, a button can have a background color with L* = 40 and white text (L* = 100), and it would easily pass the minimum contrast requirement (L* difference > 50).

Source: Google

LCH for the rest of us

This is a powerful technique to effortlessly generate accessible colors, but it doesn’t require Material color utilities. LAB can be represented as LCH (luminosity, chroma, hue), similar to Material’s HCT, where the L value can be used to calculate contrast. (This is a better representation than existing ones like HSL, which can vary in perceived lightness even if the lightness value is the same.) You can learn more about LCH in this article.

(Also, LCH is making its way into web standards as part of CSS Color Level 4! So you can write lch(40% 44 49) in CSS without needing to convert it to HSL or RGB, but this is currently only supported in Safari. Lea Verou, who wrote the article linked above, is part of the W3C CSS Working Group developing this very standard.)

So, all you need is a starting color, convert it to LCH, and modify the L value to make a palette. Then use a pair of colors that have a difference of 50 or more in luminosity to ensure accessible contrast.

We can see how this technique is used in the Material color system below: the tones in the palette match the LCH luminosity value. (They also modify the chroma (similar to saturation) and hue slightly across tones.) I made a small web app that displays the palettes generated by the Material color utilities to get the LCH values below.

Source: Google

Before Material’s HCT, the design teams of Lyft and Stripe explored the same issue, and Stripe also ended up using LCH. I’ve linked both of their explorations, which are excellent reads.

How can I achieve this? There are a few tools to convert to and from LCH:

Lea Verou made an LCH color picker,
the Colord website lets you convert from LCH to hex or RGB,
if you have a Mac, Sindre Sorhus’ System Color Picker is excellent, and
Accessible Palette generates a palette using LAB/LCH.

I used this technique to add theming to Rowy, allowing users to choose their own accent color while maintaining accessibility and tinting key UI elements. The source code has the exact LCH values I used.

Rounded corners are harmonious

When you have a box with rounded corners in a design and want to nest an inner box with a small gap, the inner element should have a smaller corner radius, with the difference being the gap size. If you make the corner radii equal for both, the inner box looks out of place. This technique has been known for a while in digital design and is even part of the CSS3 spec.

// Detect dark theme var iframe = document.getElementById('tweet-1349782080021028865-440'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1349782080021028865&theme=dark" }

This is also present in hardware design: the screen’s rounded corners match the device frame on the iPad Pro and iPhone X designs. On iPhones with rounded screens, the dock matches the screen’s curvature—Apple even provides the exact point size in the software. You can also see other elements in the video player matching the screen curvature.

These values were measured using the image provided in Sketch’s built-in iOS App Icon template and the display corner radius for the iPhone 12 from Kyle Bashour’s ScreenCorners library. The app icon radius remains consistent across the system, so it does not match the screen’s curvature.

Oddly, this principle was not applied to UI elements, which used a single corner radius. In Material Design 2, both dialogs and inner buttons have a corner radius of 4dp, despite having a gap of 8dp in between. macOS was similar in the Yosemite-era design, and Windows 10 used square corners for almost all UI elements.

However, with Big Sur and Windows 11, this effect is approximated by increasing the corner radii of larger UI elements. In macOS, buttons now have a corner radius of 5pt and dialog windows 10pt, plus they all use ‘smooth corners’ to match the corners of Apple’s hardware. Meanwhile, Windows 11 famously rounds previously sharp corners, with buttons at 4px and windows 8px.

It also seems that UIs are becoming more rounded in general:

Big Sur increases corner radii and uses smooth corners, which look more rounded;
iOS 15 introduces button styles with fully-rounded corners;
Windows 11 removed sharp corners on most UI elements; and
Android 12 increased the radii for dialogs, navigation drawers, and fully round the corners of buttons, in stark contrast to all previous versions of Material Design.

How can I achieve this? Set the smallest UI elements to some base corner radius, then set the larger containing elements like dialogs to a larger corner radius. Try to make them proportional to the distance between the smaller element, or double the smaller corner radius to simplify. Here’s how it looks in our app, with the corner radii doubled:

Variable UI fonts with optical sizes

When typefaces were first developed, they were physical designs etched in metal, with fixed font sizes. When a typographer designed the same typeface for a different size, they modified the design to be optimal by altering facets like spacing and proportion: this is known as optical size. You can learn more about optical size in this article.

Variable fonts are a new font format based on OpenType that allows the designer to customize specific ‘variation axes’ (or variables) of a type design, such as a non-fixed weight, slant, and optical size. You can learn more about variable fonts in this excellent Variable Fonts Primer, which uses Roboto Flex, a variable font extension of Google’s Roboto typeface.

In 2021, all major OSes now use this variable font technology to implement optical sizes in UI typography:

Apple’s system font San Francisco was released in 2015 with two optical sizes: ‘Display’ for sizes 20 points and larger, and ‘Text’ for everything smaller. In 2020, Apple released these fonts as a single variable font, SF Pro, with optical size as a variation axis. Apple’s system icons, SF Symbols, also use variable font techniques.
For Windows 11, Microsoft redesigned its system font Segoe UI as Segoe UI Variable with its own optical size axis.
And as part of Material Design 3, Google introduces GS Text and GS Variable, an evolution of its corporate font Google Sans.

Also of note: these typefaces are all different styles of sans-serif. San Francisco is neo-grotesque, Segoe is humanist, and Google Sans is geometric.

How can I achieve this? Variable fonts are a relatively new technology, and producing them is expensive, so there aren’t as many of them around, especially in the free and open source space. So far, the only open source variable font with optical sizes that I’ve found is Roboto Flex, but it doesn’t seem to be finished yet. Rasmus Andersson’s increasingly ubiquitous Inter typeface has a display size in beta. In the meantime, using a more expressive typeface for prominent titles can elevate your design. For Rowy, I used Inter as the typeface in small text and Space Grotesk in headings for brand expression.

Focus on content

Flat design has been with us for over a decade, and its primary goal is to focus on the content by stripping away the clutter and ornamentation of UI elements. The iOS 15 design guidelines state, “a crisp, beautiful interface help[s] people understand and interact with content while never competing with it.”

The most recent OS releases iterate on this concept by having even less “chrome”. Navigation bars are transparent in iOS 15 and Android 12 and blend into the background until you scroll.

On the desktop, macOS Big Sur blends the title bar and toolbar until you scroll or hover over the bar for a few seconds. And some apps in Windows 11 don’t distinguish the title bar at all but house the content in a distinct card-like layer.

These elements all achieve the same goal: to reduce the amount of visual clutter around and elevate the visual prominence of the content.

How can I achieve this? If you have a navigation bar docked to an edge, blend it into the background when it doesn’t need to be distinguished, such as when the user hasn’t scrolled. And if you have primary content, house it in a subtle layer distinct from the background. We use a React UI library, MUI, which makes it easy for us to achieve this effect where the navigation bar is only distinguished on scroll:

Inset everything

In a similar vein, more UI elements are inset, no longer taking the entire width of their containers. When the iPhone X introduced an on-screen home indicator in place of the home button, Apple changed their guidelines to prescribe inset buttons with rounded corners, eschewing the full-width buttons introduced in iOS 7. Best practices changed for many other elements fixed to the bottom of the screen to adapt to the new iPhone design.

With iOS 15, Apple is insetting ‘table views’ in more apps like Settings and Mail. It seems to be in response to the growing size of iPhone screens, with the guidelines noting, “in a compact environment, an inset grouped table can cause text wrapping, especially when content is localized.”

In macOS Big Sur, they extend this design to the lists in Mail, consistent with the iPadOS design. It’s also present in menus throughout the system, including the menu bar. Note the click targets extend to the edge of the menus, like the previous full-width design. Windows 11 shares the same style in its menus and navigation items. Android 12’s system UI and apps generally follow this style too.

This style could improve accessibility as the separation between elements and their containers now extends to all four sides, but I haven’t found any research supporting this. When paired with harmoniously rounded corners, it can make menus look more modern.

Differentiation beyond color

Designers are adding more ways to display states without relying on color, which is inaccessible for people with color blindness. Here are some examples I noticed:

Spotify added dots below the shuffle and repeat buttons instead of relying solely on changing their color in 2017.
Material Design 3 displays a pill-shaped indicator and uses filled icons for active pages in the navigation bar.
Windows 11 adds a consistent, distinct line to selected items in lists and navigation panes.

This informed the decision to redesign toggle buttons for Rowy:

And a standard switch design

With Android 12, switches now have the same fundamental design across the major OSes. This makes it easier for a user to switch between these platforms and reduces cognitive load.

Where is UI design headed?

The overarching theme I notice in all these design decisions is that designers put user interface design into perspective. They’re acutely aware of where digital interfaces fit in the human experience and interact with the physical world.

The convergence on outline icons to mimic text recognizes the importance of icons in communication.
Increasingly personalized interface elements—especially the direction Material You is taking—acknowledge how people like to make the things they use their own, including the tech they use every day.
Harmoniously rounded corners and inset elements are inspired by physical objects and industrial design, so our software matches the hardware more closely.
The use of variable fonts with optical sizes hearkens back to typography’s origins, and along with differentiating elements beyond color, they improve usability for all, especially those with disabilities.
The more minor things help, too: reducing visual clutter to elevate content allows users to focus on what they want to do. And using a standard switch design eliminates any cognitive load required to figure out what a UI element does.

Photo by Vinicius "amnx" Amano on Unsplash

This kind of thinking behind UI design sets us up nicely for the next generation of computing centered on ~~AR/VR~~ the metaverse, where metaverse-first experiences will have to answer how they improve the human experience and interact with the physical world. Designers are already looking at how to adapt design systems for this change. Looking at the decisions above, applying color science (Material’s HCT takes viewing conditions into account) and inset elements that elevate content help these UI elements transition from the 2D world to the 3D metaverse.

Hey 👋 if you enjoyed this, you can follow me on Twitter @nots_dney, where I talk more about my work as a front-end engineer. Also, check out Rowy, where we’re building an improved user experience for working on cloud platforms, starting with Google Cloud and Firebase.

rowyio / rowy

Low-code backend platform. Manage database on spreadsheet-like UI and build cloud functions workflows in JS/TS, all in your browser.

✨ Airtable-like UI for managing database ✨ Build any automation, with or without code ✨

Connect to your database and create Cloud Functions in low-code - without leaving your browser.
Focus on building your apps Low-code for Firebase and Google Cloud

Website • Documentation • Chat with us • Twitter

Live Demo 🛝

💥 Explore Rowy on live demo playground 💥

Features ✨

20211004-RowyWebsite.mp4

Powerful spreadsheet interface for Firestore

CMS for Firestore
CRUD operations
Bulk import or export data - csv, json, tsv
Sort and filter by row values
Lock, Freeze, Resize, Hide and Rename columns
Multiple views for the same collection

Automate with cloud functions and ready made extensions

Build cloud functions workflows on field level data changes
- Use any NPM modules or APIs
Connect to your favourite tool with pre-built code blocks or create your own
- SendGrid, Algolia, Twilio, Bigquery and more

Rich and flexible data fields

30+…

View on GitHub

How we built our virtual live event platform with Firestore and Firetable

Sidney Alcantara — Thu, 18 Feb 2021 01:07:37 +0000

Behind the scenes from Antler’s virtual Demo Day

As in-person events continue to be held online amid the ongoing Covid-19 pandemic, many events are finding new ways to connect with their audiences and deliver more personal, engaging experiences. It’s no different at Antler — we used to run physical Demo Day events to exhibit our portfolio companies, and now, we need to adapt the format for a decentralised, virtual audience.

I’ve previously written about our first virtual event while explaining why we chose Gatsby over Next.js to achieve excellent performance. Now we wanted to build on top of this foundation to deliver an even better live experience.

We launched this new platform for our virtual Demo Day Rewired event in Sydney. For the first time, viewers could have their questions answered live, we actively surfaced useful information about each startup as they presented, and we made it even easier to get in touch with each startup’s founders.

But Antler is present in 12 locations, each of which runs its own Demo Day, and we wanted to enable each site to deliver the same live experience we had in Sydney in one easy-to-use and customisable platform.

Here’s how we did it.

Enhancing interactivity with Firestore Listeners

From the start, we envisioned this new virtual event experience would augment the live viewing experience by updating the page with relevant information as the live stream progresses, without the user ever having to reload the page.

Specifically, we wanted to make it a lot easier for viewers to learn more about each startup as they presented by showing

more information about what they do,
background on who the founders are, and
a link to their slide deck that the viewer can read and download.

All we needed was a way to say which startup was currently presenting.

We initially used Algolia to get a restricted public subset of our startups’ data and reduce the number of bytes downloaded by the user with its small JavaScript library (at only 7.5 kB gzipped). Unfortunately, by design, Algolia only fetches data once, and we can’t easily update the front-end whenever data changes. So if we were to continue using it, we would need to periodically fetch new data — a very inefficient method, especially when there are no changes to data between each fetch.

But since we already stored all our data on a Firestore database, we could use Listeners to get realtime updates to our data effortlessly. Then, we could store which startup was currently presenting in a single Firestore document, listen to that doc’s updates, and update the page accordingly. And we don’t even have to do any particular configuration or write new code thanks to community-supported libraries like react-firebase-hooks.

With this setup, we could also make it much easier for viewers to contact each startup through a specialised pop-up form. This experience is a marked improvement from the previous in-person one, which asked viewers to physically divert attention from the presenters and open a specific URL on their phone. Now they could do that without even switching the tab — that’s a lot less work required.

Additionally, we partnered with Slido, which provides easy tools to add Q&A and polls for live events, allowing viewers’ questions to be answered by presenters live on air.

Adding these features enhances the level of interactivity in the live experience. It shows the viewer that we truly rethought the event format for an online virtual audience and not just a rudimentary port of the original.

Enabling customisation with Firetable

Now that we’d settled on using Firestore to show the currently presenting startup in realtime, we could also use the same document to store the configuration for each event, such as the event title, time, and live stream URL.

We wanted our global teams to configure their Demo Day as they see fit, so we needed a user-friendly UI to expose this config document to them. Thankfully, we didn’t have to build an entirely new UI to facilitate this, and we avoided the additional baggage of having to update the code when we add a new setting or creating a new UI element to configure a specific field.

We were already using Firetable, our open-source project that combines a spreadsheet UI with the full power of Firestore. At Antler, it allows our team to easily manage and update our internal database and automate day-to-day tasks that involve it.

We could continue using Firetable to expose those configuration fields directly — from text fields to toggles to dropdowns that link to other documents in the database — with minimal extra work on our part and little additional training for our team. Now we just had to decide what can be configured and write the code to enable that in our Demo Day web app.

While we initially used this setup for configuring basic settings for each event, we realised we could also use it to give our team full control over the viewing experience. Our Demo Day app has four pages:

a registration page to collect info on the viewer,
a pre-event page so those who just registered can preview the startups,
the live stream page with interactivity, and
a post-event page so viewers can rewatch individual pitches.

Instead of setting timers to switch between states, we could now allow our team to change the page displayed through toggles.

Enabling this is especially useful if, for example, the live stream was running late and they weren’t ready to switch over from the pre-event page. And since it directly updates the Firestore document, it would also trigger the Firestore listener on the front-end, so again there would be zero page refreshes required. We were even able to extend this by adding small tweaks requested by one event as toggles, so we don’t modify other events and to let future events opt-in to these tweaks.

Ensuring performance with Gatsby

While we were willing to accept the small performance cost of switching from the lean Algolia library to the beefier Firestore one, I wanted to continue to improve the performance of the app, especially during the first load.

As detailed in the previous article, we had minimal use of static site generation: we only used it to render the page skeleton while we waited for the Algolia query to finish. We wanted to eliminate this by including a snapshot of the config document as part of Gatsby’s static build. Then, when the Firestore Listener first loads, it will update the page data with the latest (mostly minor) updates.

Also, embedding configs in the static build became a necessity since we allow our team to set each event’s meta tags, which Facebook, LinkedIn, and Google use to display on their sites. These platforms’ crawlers perform a single HTTP request on the main webpage and don’t run any JavaScript (such as the Firestore Listener), so we need to include this in the static build.

To retrieve the Firestore document during Gatsby’s build process, we used @deanc/gatsby-source-firestorer so the doc can be accessible in Gatsby’s GraphQL layer. Now I know what you’re thinking: this seems like unnecessary extra work to achieve this in Gatsby and looks a lot simpler to implement in something like… Next.js. Unfortunately, we didn’t have enough time to build and test a Next.js implementation, and the current Gatsby implementation achieved the same result for our viewers anyway.

Now that we cached our configs for the static build, we could rebuild the site at any time so that the viewer gets the latest data right as they load the page. But the question was now: when do we rebuild the site? We couldn’t do this every time the config doc was updated — this would be every time a new startup presents, or every few minutes — and each rebuild would only update a small portion of the page. Rebuilding every time would be very inefficient and cost unnecessary build minutes from Netlify.

We knew we had specific situations where a rebuild is necessary: when our team updates the social media meta tags and when they switch the current page. If the static-generated site displays another page to the one set in the config doc, it will flash to the new page when the Listener loads. This flashing is a poor and potentially confusing user experience, especially if a previously-registered user logs on to the live stream page, but has to see a flash of the registration page.

Luckily, we could use Netlify’s Build Hooks feature to trigger a new build via a Cloud Function. Then, our team could activate it right in Firetable with the single click of a button, again providing full control of the virtual event to our team.

More performance wins with thumbnails

At the end of the previous article, I wrote about how we were displaying uncompressed images uploaded directly by our founders: this meant we were loading potentially lossless images, thousands of pixels wide, for an area that was only 80px wide.

I also complained about the lack of WebP support in Safari (i.e. all iOS devices). Thankfully, the next major version, Safari 14, will support this. Unfortunately for WebP, I came across an article via Hacker News that details why WebP isn’t any better than a well-compressed JPEG.

Considering these two factors, I decided to stick with JPEG and PNG when writing a Cloud Function that generates multiple, lossy-compressed thumbnails when images are upload. (I first wrote it for displaying thumbnails on Firetable and reused it here.) These thumbnails reduced the number of bytes loaded significantly, down from multiple megabytes to just hundreds of kilobytes!

Now, most viewers don’t have to look at whitespace when scrolling down the page or when new startups appear on-screen — those bytes should be downloading the live stream anyway. Our team can now also upload any image without worrying about its size. Plus, we won’t have to ask for images to be uploaded at specific sizes, and they won’t have to resize it in an image editor — or even learn how to use one.

Thanks for reading! While I still can’t link the source code, you can have a look at our virtual Demo Day events here.

You can follow me on Twitter @nots_dney as I write more about what we’re building with Firetable.

rowyio / rowy

Low-code backend platform. Manage database on spreadsheet-like UI and build cloud functions workflows in JS/TS, all in your browser.

✨ Airtable-like UI for managing database ✨ Build any automation, with or without code ✨

Connect to your database and create Cloud Functions in low-code - without leaving your browser.
Focus on building your apps Low-code for Firebase and Google Cloud

Website • Documentation • Chat with us • Twitter

Live Demo 🛝

💥 Explore Rowy on live demo playground 💥

Features ✨

20211004-RowyWebsite.mp4

Powerful spreadsheet interface for Firestore

CMS for Firestore
CRUD operations
Bulk import or export data - csv, json, tsv
Sort and filter by row values
Lock, Freeze, Resize, Hide and Rename columns
Multiple views for the same collection

Automate with cloud functions and ready made extensions

Build cloud functions workflows on field level data changes
- Use any NPM modules or APIs
Connect to your favourite tool with pre-built code blocks or create your own
- SendGrid, Algolia, Twilio, Bigquery and more

Rich and flexible data fields

30+…

View on GitHub

We refactored 10K lines of code in our open-source React project

Sidney Alcantara — Tue, 09 Feb 2021 02:15:46 +0000

Know the how, the when, and the why behind our refactoring

When working on any project, especially in the MVP stage, we as developers often prioritise one thing above all else when writing code: making sure it works. Unfortunately, this can mean we write code hyperfocused on the MVP’s requirements, so we end up with code that is hard to maintain or cumbersome to expand. Of course, this isn’t a problem one can easily avoid since we don’t live in an ideal world. The forces of time are always against us — sometimes we just need to push something out.

I’m a software engineer building Rowy, an open-source React app that combines a spreadsheet UI with Firestore and Firebase’s full power. We ran into this exact issue with some fundamental code: the code for all the different field types, from the simple ShortText to the complex ConnectTable field.

After refactoring, we now have a more solid foundation to build more features, we squashed a few hard-to-find bugs, and we now even a guide on how our contributors can write new field types.

When code smells and tech debt became big problems

When we first started building Rowy, the idea was to build a spreadsheet interface, and naturally, the resulting product closely matched that. Looking at old screenshots, it’s remarkable how closely it resembles spreadsheet programs like Excel and Google Sheets:

A screenshot of an old build from February 2020

We used React Data Grid to implement this. It accepts “formatter” components used to render cells and “editor” components used to edit cells when a user double-clicks the cell. We structured our code around this, with formatters and editors becoming folders alongside the code for Table.

A few months later, we added the SideDrawer, a form-like UI that slides over the main table. It was designed to make it easier to edit all the fields of a single row at a time, which we found was an everyday workflow for our users. At the time, it seemed like the most logical way to structure this new code was similar to how we structured the Table, so we created a Fields folder in the SideDrawer folder.

But as we maintained this code, cracks began to show.

One of our distinctive field types is Action, which displays a button on the table that lets the user run code based on the row’s data using Firebase Cloud Functions and show the results in the very same cell. We’ve used it for novel applications such as setting our database’s access controls from right within Rowy using Firebase Auth custom roles.

We had a bug where the Cloud Function wasn’t receiving the right parameters when called by Action cells. But to update the code, we had to do it in two separate locations — the Table formatter and the SideDrawer field. Not only that, it turns out we had duplicated the code calling the Cloud Function due to time constraints. There was simply no clear location for that shared code, and the bug was too high-priority for us to have time to answer that question correctly.

The final straw was when we noticed we had inconsistently implemented the column lock feature. Some fields remained editable in the SideDrawer but not the Table or vice versa, or we didn’t implement it at all for that field. This was a result of adding this feature after we had implemented the minimum requirements for each field type, so we had to go through each Table formatter and each SideDrawer field — double the number of field types we had. This tedious manual process was clearly prone to errors.

At this point, we knew it was time to refactor.

Refactoring for success

We identified the main problem: we didn’t have a single place to store code for each field type. It was scattered throughout the codebase: Table formatters and editors, SideDrawer fields, column settings, and more. This scattering rapidly inflated the cost for adding new features for field types and weeding out bugs.

The first thing we did was invert our approach to code structure entirely — instead of grouping code by each feature that would use the field types, we grouped the code by the field types themselves.

The new approach translates to a new top-level component folder called fields, comprising subfolders for each field type, and within each, we have files such as TableCell.tsx and SideDrawerField.tsx. Then we could export these features in a config object, so all this code would only need to be imported once by the consumer. This is similar to a problem solved by React Hooks: grouping related code and not having to think about lifecycle methods.

This approach also simplifies how we import a field’s code throughout the codebase. Previously in the Table and SideDrawer, we would rely on switch statements that looped through each field type until we could fetch the correct component and import each field one by one. So whenever we added a new field type, we would also have to add a new entry to these switch blocks — again ballooning the cost of development. Instead, we could create a single array with every field config, then share it across the codebase. So we only need to define a new field type once.

Additionally, the config object lets us quickly implement new features and ensure all fields do so correctly. Now we could simply check if a field’s config has a property. And since we’re using TypeScript, each config object must implement our interface, which can enforce certain features (properties of the interface) to be of a particular type, such as a React component accepting specific props. This new functionality allowed us to fix column locking implementation and made it much easier to develop a new feature, default values for columns. All we had to do was add a new property to the interface.

With this in mind, our refactor not only made our code easier to maintain and fix bugs — but it also provided a much more solid foundation on which we can build advanced features for fields and removing extra costs to development.

Lessons for the future

Of course, we could have avoided all this pain and extra work had we initially gone with this approach. But we don’t live in an ideal world. All of the non-ideal solutions I mentioned above were the result of time constraints on our end, especially when we were working on other projects simultaneously, which directly impacted day-to-day work.

Many of us work for a business that doesn’t have excellent code quality as its primary goal. As developers, we are hired to build tech solutions that meet business requirements, and the “how” is abstracted away. In this case, however, our poorly-structured code and the amount of accrued tech debt were directly impacting our ability to work.

And whilst writing this article, I came across Refactoring.Guru, an excellent guide on refactoring. We clearly satisfied their first recommendation on when to refactor: “When you’re doing something for the third time, start refactoring.”

This experience has taught us many valuable lessons on code structure and when a refactor is necessary. I hope you’ve gained some insights by reading about our journey.

Thanks for reading! You can find out more about Rowy below and follow me on Twitter @nots_dney.

rowyio / rowy

Low-code backend platform. Manage database on spreadsheet-like UI and build cloud functions workflows in JS/TS, all in your browser.

✨ Airtable-like UI for managing database ✨ Build any automation, with or without code ✨

Connect to your database and create Cloud Functions in low-code - without leaving your browser.
Focus on building your apps Low-code for Firebase and Google Cloud

Website • Documentation • Chat with us • Twitter

Live Demo 🛝

💥 Explore Rowy on live demo playground 💥

Features ✨

20211004-RowyWebsite.mp4

Powerful spreadsheet interface for Firestore

CMS for Firestore
CRUD operations
Bulk import or export data - csv, json, tsv
Sort and filter by row values
Lock, Freeze, Resize, Hide and Rename columns
Multiple views for the same collection

Automate with cloud functions and ready made extensions

Build cloud functions workflows on field level data changes
- Use any NPM modules or APIs
Connect to your favourite tool with pre-built code blocks or create your own
- SendGrid, Algolia, Twilio, Bigquery and more

Rich and flexible data fields

30+…

View on GitHub

How to useRef to fix React performance issues

Sidney Alcantara — Tue, 24 Nov 2020 00:59:18 +0000

And how we stopped our React Context re-rendering everything

Refs are a seldom-used feature in React. If you’ve read the official React guide, they’re introduced as an “escape hatch” out of the typical React data flow, with a warning to use them sparingly, and they’re primarily billed as the correct way to access a component’s underlying DOM element.

But alongside the concept of Hooks, the React team introduced the useRef Hook, which extends this functionality:

useRef() is useful for more than the ref attribute. It’s handy for keeping any mutable value around similar to how you’d use instance fields in classes.

While I overlooked this point when the new Hook APIs launched, it proved to be surprisingly useful.

👉 Click here to skip to the solution and code snippets

The Problem

I’m a software engineer working on Rowy, an open-source React app that combines a spreadsheet UI with the full power of Firestore and Firebase. One of its key features is the side drawer, a form-like UI to edit a single row, that slides over the main table.

When the user clicks on a cell in the table, the side drawer can be opened to edit that cell’s corresponding row. In other words, what we render in the side drawer is dependent on the currently selected row — this should be stored in state.

The most logical place to put this state is within the side drawer component itself because when the user selects a different cell, it should only affect the side drawer. However:

We need to set this state from the table component. We’re using react-data-grid to render the table itself, and it accepts a callback prop that’s called whenever the user selects a cell. Currently, it’s the only way to respond to that event.
But the side drawer and table components are siblings, so they can’t directly access each other’s state.

React’s recommendation is to lift this state to the components’ closest common ancestor, in this case, TablePage. But we decided against moving the state here because:

TablePage didn’t contain any state and was primarily a container for the table and side drawer components, neither of which received any props. We preferred to keep it this way.
We were already sharing a lot of “global” data via a context located close to the root of the component tree, and we felt it made sense to add this state to that central data store.

Side note: even if we put the state in TablePage, we would have run into the same problem below anyway.

The problem was whenever the user selected a cell or opened the side drawer, the update to this global context would cause the entire app to re-render. This included the main table component, which could have dozens of cells displayed at a time, each with its own editor component. This would result in a render time of around 650 ms(!), long enough to see a visible delay in the side drawer’s open animation.

Notice the delay between clicking the open button and when the side drawer animates to open

The reason behind this is a key feature of context — the very reason why it’s better to use in React as opposed to global JavaScript variables:

All consumers that are descendants of a Provider will re-render whenever the Provider’s value prop changes.

While this Hook into React’s state and lifecycle has served us well so far, it seems we had now shot ourselves in the foot.

The Aha Moment

We first explored a few different solutions (from Dan Abramov’s post on the issue) before settling on useRef:

Split the context, i.e. create a new SideDrawerContext.
The table would still need to consume the new context, which still updates when the side drawer opens, causing the table to re-render unnecessarily.
Wrap the table component in React.memo or useMemo.
The table would still need to call useContext to access the side drawer’s state and neither API prevents it from causing re-renders.
Memoize the react-data-grid component used to render the table.
This would have introduced more verbosity to our code. We also found it prevented necessary re-renders, requiring us to spend more time fixing or restructuring our code entirely, solely to implement the side drawer.

While reading through the Hook APIs and useMemo a few more times, I finally came across that point about useRef:

useRef() is useful for more than the ref attribute. It’s handy for keeping any mutable value around similar to how you’d use instance fields in classes.

And more importantly:

useRef doesn’t notify you when its content changes. Mutating the .current property doesn’t cause a re-render.

And that’s when it hit me:

We didn’t need to store the side drawer’s state — we only needed a reference to the function that sets that state.

The Solution

Keep the open and cell states in the side drawer.
Create a ref to those states and store it in the context.
Call the set state functions (inside the side drawer) using the ref from the table when the user clicks on a cell.

The code below is an abbreviated version of the code used on Rowy and includes the TypeScript types for the ref:

Side note: since function components run the entire function body on re-render, whenever the cell or open state updates (and causes a re-render), sideDrawerRef always has the latest value in .current.

This solution proved to be the best since:

The current cell and open states are stored inside the side drawer component itself, the most logical place to put it.
The table component has access to its sibling’s state when it needs it.
When either the current cell or open states are updated, it only triggers a re-render for the side drawer component and not any other component throughout the app.

You can see how this is used in Rowy here and here.

When to useRef

This doesn’t mean you should go ahead and use this pattern for everything you build, though. It’s best used for when you need to access or update another component’s state at specific times, but your component doesn’t depend or render based on that state. React’s core concepts of lifting state up and one-way data flow are enough to cover most app architectures anyway.

Thanks for reading! You can find out more about Rowy below and follow me on Twitter @nots_dney.

rowyio / rowy

Low-code backend platform. Manage database on spreadsheet-like UI and build cloud functions workflows in JS/TS, all in your browser.

✨ Airtable-like UI for managing database ✨ Build any automation, with or without code ✨

Connect to your database and create Cloud Functions in low-code - without leaving your browser.
Focus on building your apps Low-code for Firebase and Google Cloud

Website • Documentation • Chat with us • Twitter

Live Demo 🛝

💥 Explore Rowy on live demo playground 💥

Features ✨

20211004-RowyWebsite.mp4

Powerful spreadsheet interface for Firestore

CMS for Firestore
CRUD operations
Bulk import or export data - csv, json, tsv
Sort and filter by row values
Lock, Freeze, Resize, Hide and Rename columns
Multiple views for the same collection

Automate with cloud functions and ready made extensions

Build cloud functions workflows on field level data changes
- Use any NPM modules or APIs
Connect to your favourite tool with pre-built code blocks or create your own
- SendGrid, Algolia, Twilio, Bigquery and more

Rich and flexible data fields

30+…

View on GitHub

Gatsby won against Next.js in this head-to-head

Sidney Alcantara — Fri, 24 Apr 2020 14:47:31 +0000

I made the same web app in Gatsby and Next.js and found Gatsby performed better

With the ongoing Covid-19 pandemic and social distancing measures, many events have been forced to migrate to online virtual events. I’m a software engineer at Antler, which runs a global startup generator program that usually runs multiple in-person Demo Day events a year that showcase around a dozen new startups, and we faced the same situation.

We wanted to deliver a solid online experience that puts the focus on the content — our portfolio companies’ pitches. With the wider audience of this event and the fact that it may be a user’s first exposure to Antler’s online presence, we needed to put our best foot forward and ensure it loads fast. This was a great case for a highly-performant progressive web app (PWA).

TL;DR

Displaying a skeleton while the data loaded made the app seem faster than just a blank page while the server loaded the data.
Gatsby’s static output was only slightly faster than Next.js, but Gatsby’s plugins and documentation made for a better developer experience.

Server-side rendering or static site generation?

For some background: all of our web products are built with React and the Material-UI library, so we stuck with that stack to keep development fast and ensure the new code is compatible with our other projects. The key difference is all our other React apps were bootstrapped with create-react-app and are rendered entirely on the client side (CSR), so users would be faced with a blank white screen while the initial JavaScript is parsed and executed.

Because we wanted top-notch performance, we were looking into leveraging either server-side rendering (SSR) or static site generation (SSG) to improve this initial load experience.

Our data will be sourced from Cloud Firestore via Algolia to have more granular, field-level control over public data access with restricted API keys. This also improves query performance: anecdotally, Algolia queries are faster and the Firestore JavaScript SDK is 86 KB gzipped compared to Algolia’s, which is 7.5 KB.

We also wanted to make sure the data we serve is as fresh as possible in case any errors are published live. While the standard SSG practice is to perform these data queries at compile time, we expected frequent writes to our database from both our admin-facing interface, firetable, and our web portal for founders, causing multiple builds to run concurrently. Plus, our database structure may cause irrelevant updates to trigger new builds, making our CI/CD pipeline incredibly inefficient, so we needed the data to be queried whenever a user requests the page. Unfortunately, this means it could not be a “pure” SSG web app.

Initially, the app was built with Gatsby since we had already been maintaining landing pages built in Gatsby and one of them was already bootstrapped with Material-UI. This initial version produced a page that initially displays a skeleton while the data was being loaded and achieved a first contentful paint time of around 1 second. 🎉

But since the data was being loaded on the client side:

Users would have to wait after the initial page load to view the actual content and wait for four network requests to Algolia to finish.
There’s more work for the browser’s JavaScript engine as React needs to switch out the skeletons for the content. That’s extra DOM manipulation!
Search engine crawlers might not be able to load the content and they generally prefer static sites.

So over a public holiday long weekend, I decided to experiment with a server-rendered version with Next.js. Lucky for me, Material-UI already had an example project for Next.js so I didn’t have to learn the framework from the start — I just had to look through specific parts of the tutorial and documentation. Converting the app and querying the data on the server side on each request solved all three points I raised above and the end result was…

Results from running Google PageSpeed Insights

Roughly triple the time for the first contentful paint.
Plus, the Lighthouse speed index quadrupled and the time to first byte rose from 10–20 ms to 2.56 seconds.

While it is noteworthy that the Next.js version is hosted on a different service (ZEIT Now vs Firebase Hosting — this may have also contributed to the higher TTFB), it was clear that pushing the data fetching step to the server produced a seemingly slower result, even if the content loaded at around the same time, because the user only sees a blank white page.

Screen recording of the two versions loading. They were not run simultaneously; the recordings were synced to the moment the Enter key was pressed.

This highlights an important lesson in front-end development: give your users visual feedback. A study found that apps that used skeleton screens are perceived to load faster.

The results also go against a sentiment you might have noticed if you’ve been reading articles about web development for the past few years:

The client side isn’t evil.

SSR is not the catch-all solution to performance issues.

Gatsby vs Next.js: static site generation performance

While the two frameworks have been known exclusively for static site generation and server-side rendered apps respectively, Next.js 9.3 overhauled its SSR implementation to rival Gatsby.

At the time of writing, this update was just over a month old and was still featured on the main Next.js landing page and there weren’t many — if any — comparisons of the frameworks’ SSG implementations. So I decided to run an experiment myself.

I reverted the changes made to the Gatsby version back to client-side data fetching and made sure both versions had the exact same feature set: I had to disable the SEO features, favicon generation, and PWA manifest, which were handled by Gatsby plugins. To compare just the JavaScript bundles produced by the frameworks, there were no images or other content loaded from external sources and both versions were deployed to Firebase Hosting. For reference, the two versions were built on Gatsby 2.20.9 and Next.js 9.3.4.

I ran Lighthouse six times for each version on my local machine.

The results very slightly favour Gatsby:

Average scores and timings of the six Lighthouse runs per framework

The Next.js version was just slightly behind Gatsby in the overall performance score, first contentful paint, and speed index. It also registered a higher max potential first input delay.

Diving into the Chrome DevTools Network panel to find an answer, the Next.js version split the JavaScript payload into three more chunks (ignoring the manifest files generated), but resulted in a 20 KB smaller compressed payload. Could these extra requests have outweighed the gains made by the smaller bundle size so much that they hurt performance?

Looking into JavaScript performance, DevTools show the Next.js version took 300 ms longer to achieve first paint and spent a long time evaluating the runtime scripts. DevTools even flagged it as a “long task”.

I compared the two branches of the project to see if there were any implementation differences that could have caused the performance hit. Other than removing unused code and fixing missing TypeScript types, the only change was the implementation of smooth scrolling when navigating to specific parts of the page. This was previously in the gatsby-browser.js file and was moved to a dynamically-imported component so it would only ever be run in the browser. (The npm package we’re using, smooth-scroll, requires the window object at the time it’s imported.) This may very well be the culprit, but I’m just not familiar with how Next.js handles this feature.

Gatsby has a superior developer experience

Ultimately, I decided to stick with the Gatsby version. Disregarding the very minor performance benefits over SSG Next.js (am I really going to nitpick over a 0.6 second difference?), the Gatsby version had more PWA features already implemented and it would not have been worth the time to re-implement it.

When initially building the Gatsby version, I was able to quickly add the final touches to make a more complete PWA experience. To implement page-specific SEO meta tags, I just had to read their guide. To add a PWA manifest, I just had to use their plugin. And to properly implement favicons that support all the different platforms, which remains a convoluted mess to this day, well, that’s already a part of the manifest plugin I just installed. Huzzah!

Implementing those features in the Next.js version would have required more work Googling tutorials and best practices and would not have provided any benefit, especially since the Next.js version did not improve performance anyway. This was also the reason I decided to just disable these features when comparing against the Gatsby version. While the Next.js documentation is more succinct (likely since it’s leaner than Gatsby) and I really like their gamified tutorial page, Gatsby’s more expansive documentation and guides provided more value in actually building a PWA, even if it looks overwhelming at first.

There is a lot to be appreciated about Next.js, though:

Its learning curve feels smaller thanks to its tutorial and shorter documentation.
Its primary data fetching architecture revolves around async functions and fetch, so you don’t feel like you need to learn GraphQL to fully utilise the framework.
It has TypeScript support out of the box, while Gatsby requires a separate plugin and it doesn’t even do type checking — that requires its own plugin. (When converting the app to Next.js, this caused some issues as I didn’t even realise I had incorrect types, causing the compile to fail.)

With its overhauled SSG support, Next.js has become a powerful framework to easily choose between SSR, SSG, and CSR on a page-by-page basis.

In fact, had I been able to fully statically generate this app, Next.js would be a better fit since I’d be able to use Algolia’s default JavaScript API and keep the data fetching code in the same file alongside the component. As Algolia does not have a built-in GraphQL API and there is no Gatsby source plugin for Algolia, implementing this in Gatsby would require adding this code to a new file and goes against the more intuitive declarative way of specifying pages.

There are always more performance improvements

With that out of the way, there were even more performance improvements to be made to get ever closer to that 100 performance score in Lighthouse.

Algolia’s March 2020 newsletter recommended adding a preconnect hint to further improve query performance. (Unfortunately, the email had the wrong code snippet; here’s the correct one.)
Static files should be cached forever. These include the JS and CSS files generated by Gatsby’s webpack config. Gatsby has a great documentation page on this and even has plugins to generate the files for Netlify and Amazon S3. Unfortunately, we have to write our own for Firebase Hosting.
The images we were serving are all JPEGs or PNGs uploaded by our founders and aren’t compressed or optimised. Improving this would require more complicated work and is beyond the scope of this project. Also: it would be really nice to just convert all these images to WebP and store just one very efficient image format. Unfortunately, as with many PWA features, the Safari WebKit team continues dragging their feet on this and it is now the only major browser without WebP support.

Thanks for reading! Normally, I would post a link to view the final project but due to legal reasons, it cannot be shared publicly.

You can follow me on Twitter @nots_dney to get updates as I’ll be writing and sharing more about my experiences as a front-end engineer.