DEV Community: Ronald Rey

[Boost]

Ronald Rey — Sun, 23 Feb 2025 04:58:54 +0000

Dealing with open database transactions in Prisma

Ronald Rey ・ Feb 23

Dealing with open database transactions in Prisma

Ronald Rey — Sun, 23 Feb 2025 04:58:32 +0000

There’s an issue that has been affecting one of the applications I maintain: runtime errors due to timed-out database transactions. The specific constraints that I was dealing with and the way I ended up resolving the issue were a bit interesting, so I felt like writing about it.

For context, this happened in a Node.js v22 application maintained by three fast-paced engineering teams, with a total of over a dozen contributors. We use Prisma v6 as our ORM against a MySQL database. However, everything I mention here is also applicable to Drizzle, another ORM for JavaScript and TypeScript applications.

The error

The exact error I’m referring to is the one below:

PrismaClientKnownRequestError: Transaction API error: Transaction
already closed: A commit cannot be executed on an expired
transaction. The timeout for this transaction was 5000 ms, however
5012 ms passed since the start of the transaction. Consider
increasing the interactive transaction timeout or doing less work
in the transaction.
  code: 'P2028',
  clientVersion: '6.3.1',
  meta: {
    error: 'Transaction already closed: A commit cannot be executed
    on an expired transaction. The timeout for this transaction was
    5000 ms, however 5012 ms passed since the start of the
    transaction. Consider increasing the interactive transaction
    timeout or doing less work in the transaction.'
  }

The error message is self-explanatory but it’s also misleading at the same time. You’ll indeed see this if you open a transaction with Prisma and don’t finish it or close it before the timeout interval, but what it doesn’t say is that open transactions can get stuck and never be closed by Prisma itself when certain conditions are met.

This can be minimally reproduced with the code snippet below:

const prisma = new PrismaClient()

await prisma.$transaction(async (tx) => {
  await tx.entity.update(/*...*/)
  await prisma.entity.update(/*...*/)
})

What’s happening here is that on the first update call, Prisma will open a transaction that is supposed to be committed when you exit the scope of the current callback. However, if you do another database write using the Prisma client instead of the available transaction before that exit, Prisma will never commit that open transaction and you end up getting the thrown exception above.

I can’t think of legitimate use cases for the code snippet above. Doing database mutations that way is almost certainly a mistake by the developer, not an intentional choice.

In a sense, we can say this is a bug in Prisma. I would not expect this code snippet to work as expected but at the very least, I feel that it could be handled slightly better by the ORM and provide a more useful error message that is less of a red herring.

The intended use of this API by the Prisma team is that you always use the transaction client inside the scope of an open transaction, but sadly, the design of the API does not help the consumer avoid this potential issue.

One additional hurdle here is that $transaction has an overload that takes in an array of queries instead of a callback like the example above, and you can also run into the same error in situations where you mix both APIs.

Our problem

This misuse is exactly what was happening to us.

We were aware of what caused the error and why, but we have a very large and complex code base with long function calls. To orchestrate database operations that happen in multiple modules, we pass the transaction client as an argument to functions through many levels. When endpoints get too big, this passing of arguments becomes an issue very similar to the “prop drilling” problem you get in React apps.

This becomes more than just annoying when those functions are re-used in different contexts where a transaction is not always needed, so that argument is then marked as optional by the developer working on a new feature.

Our initial approach was to remove all optional transaction client arguments and make them all required instead. This worked well in a few places but we quickly realized that it did not scale well with the size of our code base and we had way too many instances of this issue that would’ve required too many risky changes to make safely, not to mention it would’ve taken us a long time to complete.

Our second approach was to try to track with logs whenever we accessed the Prisma client instead of the transaction client when a transaction was active so that we would surgically fix those few instances instead of doing a big massive re-write.

That second approach would’ve fixed our existing offending call-sites but it did nothing to prevent this from happening again in the future without a change in technical direction in how we dealt with transactions, so it didn’t sit too well with me.

The solution

I was looking for a solution that:

Was minimal and did not require a massive refactor
Fixed the existing offending issues
Prevented the issue from happening again in the future altogether

What I realized was that I had already implemented a context-tracking functionality in our app a long time ago based on AsyncLocalStorage, to store data like trace ids and user ids throughout the lifetime of a request. We had been battle-testing it for a long time already so I am 100% confident that it works everywhere, especially after we squashed a few edge cases like this one.

Our code base is private, but I extracted that bit to an OSS package, so you can view those sources here if you’re curious: https://github.com/reyronald/async-local-storage. We don’t use that package in our app but its implementation is very similar.

My solution consists of storing the active transaction in that asynchronous context, and instead of accessing the Prisma client, access the transaction client if available anytime a database query is attempted.

We also had OpenTelemetry already, which provides a context concept that’s internally built on top of AsyncLocalStorage. I could’ve used that instead if we didn’t already have our own.

To be able to do this though, I needed to be able to control all access to those clients. We had something in place already for this but it wasn’t used everywhere. Thankfully the changes required to address that were small and I was okay with pursuing that.

In essence, we wanted to go from:

// db.ts
export const prisma = new PrismaClient()

// another-file.ts
import { prisma } from "@/db"

await prisma.entity.update()

await prisma.$transaction(async (tx) => {
  /*...*/
})

To this:

// db.ts
const _prisma = new PrismaClient()

export const getDbInstance = () => _prisma

export async function runInDbTransaction<T>(
  fn: (tx: TransactionClient) => Promise<T>,
) {
  // ...
}

// another-file.ts
import { getDbInstance, runInDbTransaction } from "@/db"

const db = getDbInstance()

await db.entity.update()

await runInDbTransaction(async (tx) => {
  /*...*/
})

This also has the additional benefit that it completely prevents the usage of the second overload of $transaction which is another surface area that can trigger the error.

With this in place, I can now leverage our asynchronous context to implement the solution. First, I make sure to store the active transaction in the store:

import { ctx } from "./asyncContext"

export async function runInDbTransaction<T>(
  fn: (tx: TransactionClient) => Promise<T>,
) {
  const prevTx = ctx.tx
  try {
    const result = await prisma.$transaction(async (tx) => {
      ctx.tx = tx
      const result = await fn(tx)
      return result
    })
    return result
  } finally {
    ctx.tx = prevTx
  }
}

Now, the magic. We have to make sure whenever prisma from getDbInstance() is used anywhere, we use tx instead if it exists in the context.

export function getDbInstance(): TransactionClient {
  return new Proxy(_prisma, {
    get(target, prop: keyof typeof _prisma, receiver) {
      const realTarget = ctx.tx || target
      return Reflect.get<typeof realTarget, keyof typeof realTarget>(
        realTarget,
        prop,
        receiver,
      )
    },
  })
}

A Proxy is a great feature of the language that allows us to intercept reads on objects. We can use it to wrap it around the return value of our utility and do our checks there.

This minimal change immediately addresses and fixes any existing instance of the problem in the entire code base and makes it impossible for a consumer to accidentally get into this situation again. It also allows us to remove transaction clients from arguments everywhere as we were doing before, reducing the complexity of the signature of our functions.

Something else that's really nice is that it mimics the semantics of transactions in SQL itself. In SQL, a transaction is started simply by executing the BEGIN statement. After that, any and all subsequent statements will happen inside that transaction scope until a COMMIT or ROLLBACK statement is found. There's no need to specify any form of transaction identifier in future statements, and there's no additional reference to the open transaction either.

In my solution, calling runInDbTransaction represents that BEGIN statement, and closing the scope of its callback argument represents the COMMIT phase to end the transaction.

Although I understand Prisma's decision for its current transaction API, I feel we would be better off with an API that more closely resembles how it works in SQL.

I also maintain codebases in Go where we use GORM. They have an API that is exactly the same as Prisma's and Drizzle's callback version, but on top of that, they also offer a manual transaction control option. Take a look:

// begin a transaction
tx := db.Begin()

// do some database operations in the transaction (use 'tx' from this point, not 'db')
tx.Create(...)

// ...

// rollback the transaction in case of error
tx.Rollback()

// Or commit the transaction
tx.Commit()

This makes more sense to me, especially after our experience with this issue. An API like this could be replicated in user-land using context and proxies, just like I did above.

The callback API feels like too much of a foot-gun. You only run into issues if you use it wrong, in an unintended way, but the fact that a wrong way to use it exists at all is a design flaw, in my opinion. Either way, offering options is the way to go, so kudos to the GORM team here.

Performance

A common concern and criticism that you see related to the usage of proxies is that they have an impact on performance. This impact comes from the additional overhead associated with the fact that each property access has to go through the Proxy handler.

To understand how much of an impact we’re talking about, I ran a local benchmark test with mitata. This is what I got.

import { PrismaClient } from "@prisma/client"
import { run, bench, boxplot, summary, do_not_optimize } from "mitata"

const prisma = new PrismaClient()

const getDbInstance = () => {
  return prisma
}

const getDbInstanceWithProxy = () => {
  return new Proxy(prisma, {
    get(target, prop, receiver) {
      return Reflect.get(target, prop, receiver)
    },
  })
}

boxplot(() => {
  summary(() => {
    bench("no proxy", function () {
      do_not_optimize(getDbInstance().user)
    })

    bench("with proxy", function () {
      do_not_optimize(getDbInstanceWithProxy().user)
    })
  })
})

await run()

This shows that wrapping the client with a proxy slows down access by a factor of 6. I also ran tests where the proxied client was cached instead of re-wrapped on every iteration of the benchmark, and the result was the same, which means that the instantiation of the proxy doesn’t have a measurable impact.

This might seem concerning but if we look at the durations, we’re talking about a difference of 31 nanoseconds, versus 185 nanoseconds. For context, there are 1,000,000 nanoseconds in a millisecond. It would take roughly 6,500 property access calls before this adds 1ms to the duration of the request.

In our application, we only access the database client a handful of times per request, so this is certainly insignificant for us.

It’s safe to say that this won’t be an issue for most apps. If someone has a use case where a couple of hundred nanoseconds makes a difference, they probably shouldn’t be using Node.js or Prisma to begin with.

What about Drizzle?

Drizzle is a competing option to Prisma in the ORM space for JS and TS runtimes. It’s a bit younger but preferred by many not only because of the similarity of its API to raw SQL, but also because it is said to be more performant and provides a few features that Prisma doesn’t have. For example, it supports transaction save points, which are not available at the time of this writing in Prisma v6.

I wanted to see how Drizzle would handle this same scenario. Turns out that it behaves almost the same, and you do get a similar runtime error thrown. See below.

Error: Lock wait timeout exceeded; try restarting transaction
    at PromisePool.query (...)
    at MySql2PreparedQuery.execute (...)
    at QueryPromise.execute (...)
    at QueryPromise.then (...)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
  code: 'ER_LOCK_WAIT_TIMEOUT',
  errno: 1205,
  sql: '...',
  sqlState: 'HY000',
  sqlMessage: 'Lock wait timeout exceeded; try restarting transaction'
}

There’s one big difference though. The delay is much longer, 50 seconds to be exact, before the transaction times out and throws. This timeout is not configurable with Drizzle.

This experience is much worse compared to that in Prisma which has a default timeout of 5 seconds that can be changed per transaction. Imagine introducing this bug by accident on production and your users have to wait 50 seconds before they get a failed response from the server! The longer timeout also makes it more difficult to troubleshoot and fix. So I’ll have to give Prisma the win on this one.

Conclusion

This solution has been in production for a while. I tested it across instances of previous incidents we had already fixed manually before. The results are as expected and no issues have been found.

Develoepr experience has also improved dramatically for implementation of new features now that our engineers don't have to worry about transaction management, argument drilling and making sure they're using the correct client. Personally I feel the release of this cognitive burden makes me more relaxed.

How we built a better search experience

Ronald Rey — Sat, 19 Sep 2020 22:46:21 +0000

Photo by Andrew Ridley on Unsplash

Recently I was tasked with improving the existing search functionality of a web application, as part of a much greater long-term effort to improve the overall user experience of the product.

The app in question is a Software-as-a-Service (SaaS) platform targeted to small businesses and medium enterprises. The specifics of the application are not relevant to this post, only that each client gets their own "portal" in our cloud-hosted environment and can manage users scoped to their organization.

The existing search functionality works exclusively as a way to find and navigate to the profile of other users in the portal. However, there were several drawbacks that customers complained about and that our product team recognized could be improved with redesign and re-implementation. Simply put, those were:

Lack of flexibility. The logic for finding entries was straight-forward and didn't capture very common use-cases. The search capabilities lacked in comparison to other products and did not meet user expectations.
Lack of functionality. Much more could be baked into the search functionality. Not just finding users, but site navigation in general. It could and should be a feature capable of answering as many questions a user could have about the app.
Out-dated design. Since it was one of the first features ever built, its looks did not match the design language used more recently elsewhere in the app.
Performance. It was unacceptably slow and users noticed. Its speed was considerably slower than what one would expect for this type of feature.

The goal of the project was to address all those items and release a more intuitive and capable new search experience that users would want to use more often, reduce the number of support cases asking simple questions, and naturally help our customers be more productive on their own.

An entire re-write made sense given the conditions, rather than a simple fix or changes on top of the existing code. Besides the user-facing goals of the project, this was also an opportunity for us to remove legacy code that relied on old frameworks and libraries in the client-side, and replace it with a modern component written with React and carefully tested.

New Functionality

The app in question is really big and complicated. Over time our team had received feedback on the difficulties users had navigating it.

This is when the product team recognized that we could do something to address that with an improved search. The existing search functionality could only find other registered users in the portal and you would use it to navigate to their profiles. However, the way it was built was very simplistic and not very helpful.

First, we improved the user search by factoring in some other data in the filtering logic instead of just the usernames or full names; like connections, identification numbers, and anything else that made sense that was associated with the user entity in the database.

Beyond that, we also enabled it to search through the entire site map so that results would show up when keywords related to specific pages or tools were searched for. If you searched for "settings", a result would show up for the Settings page and you could just click to get to it, instead of manually relying on the regular navigation menu. This is advantageous since some of the parts in the app are hard to find and deeply nested within other menus or routes.

To achieve this we had to build a massive object that contained all of the necessary metadata of all the routes in the site. That metadata would contain properties like tool or page name, associated search keywords, and URL path, and also had to account for logged-in user permissions since not all routes are visible to everyone depending on their role.

This object had to be manually crafted and maintained since the metadata cannot be automatically derived. This means that when adding new routes to the app we had to remember to go back and update that object or otherwise it wouldn't show up in the new search tool.

To avoid this, I refactored the way our routes were defined throughout the app and created a single function that would return all the route definitions instead. I then added a check at the end of that function that would compare the collection of routes with the search tool metadata object. If there are any discrepancies, I render a full-screen error overlay in the app during development mode with instructions on how to proceed. It looks like this:

This was extremely important for us because there are four development teams with about five engineers each contributing to this repository daily in a very fast-paced environment. Unless we have an automatic way to make sure it is kept up-to-date, we would not have been able to keep the search tool working as expected over time. It is not feasible for us as an organization to review every single pull request that is merged in.

There were a few other things that the product team wanted to include in the search results that did not match the "navigation" category. We have some widgets like real-time chat and help desk support that can be used anywhere. If we wanted to promote this new search tool as an all-in-one place to find everything you need, a way to trigger those from it had to be included.

This was not particularly difficult, but the fact that the search results could be anything meant that the API design, filtering logic, and UI had to be flexible enough to support this. Beyond that, the possibility of adding different types of results in the future required an additional level of thought effort as well.

Another very subtle detail was added. At first, I did not think anything of it when I saw it on the designs, but it ended up becoming my overall favorite feature after implementation and release: a list of recently selected search results every time you focus the search input and open up the search panel. This can save the user many clicks and navigations, notably speeding-up the process of moving around the app. This alone accelerates productivity and enhances the user experience tremendously.

Improving user search performance

The existing search functionality was built using Backbone.js and relied on jQuery UI Autocomplete. Its UI did not look very different than the vanilla example hosted on that site. It had a "typeahead" or "autocomplete" behavior that would suggest entries to the user as they typed into the textbox. Those entries would be the names of other users in the portal.

Behind the scenes, the technical approach was the usual associated with this type of component. There is a debounced change event listener that only triggers after the user has stopped typing for an arbitrary short amount of time chosen by the developer. When that debounce timer is cleared, a callback is executed with the logic to compute the suggestions. This callback was mostly an asynchronous network call to a server that would query a database and do some logic based on the input.

The debounce aspect is an optimization that aims to reduce the amount of unnecessary work as much as possible. It does not make much sense to compute suggestions for every single keystroke on the text input, since the user is most interested in those pertaining to the already complete or semi-complete search term.

What I have described so far is practically the de-facto way of building typeahead or autocomplete components and almost every site out there with a search functionality behaves this way.

What makes the most sense as an approach to improve the performance consists of optimizing the server code that accesses the database and computes the suggestions as much as possible. After analyzing the endpoint used I noticed a lot of low-hanging fruits that would have a noticeable positive impact without much effort.

The endpoint in place was a general-purpose resource controller action and used in several other places of the application. It had a lot of code in it that was irrelevant to the search. This meant that not only the execution duration was longer, but also the returned payload from the server was much bigger than necessary as it contained an excessive amount of data that the search didn't use. This resulted in an overall longer network round-trip, and a higher memory footprint.

Let's look at some real production metrics:

Legacy Search - Past 7 Days
(min: 158.21ms, max: 9.95s, avg: 562.47ms)

This shows the duration of network round-trips for this endpoint when used specifically for the legacy search functionality. The unusual random peaks obfuscate the visual information a little bit. I tried to find a significant period that did not have one but could not, so left it in as it represents the real nature of the behavior of the endpoint anyway.

We can focus on the averages and minimums. Even when looking at longer periods, the average of ~500ms (half a second) is maintained. However, the reality is that the performance differs per portal.

Organizations with fewer users will experience a duration much closer to the minimum of 150 - 200 ms, whereas our biggest portals so far experience a consistent 1 - 1.1 seconds, with some peaks of up to 5 or 10 seconds occasionally.

So, if you are unlucky enough to be part of one of the biggest organizations, you would have to wait at a minimum 1.5 seconds before the search displayed suggestions when we account for the debounce time and DOM rendering duration in the browser. This would be an awful user experience.

Generally, I am a huge advocate for standard and spec-compliant RESTful APIs and very much against single-purpose endpoints in most cases. For this scenario, however, doing just that makes total technical sense given the constraints, the goal, and the return of investment.

If we create a new endpoint that only does and returns the bare minimum the same metrics would look considerably different. This was discussed with the rest of the development team and we all agreed. Now we had a plan to move forward.

Nevertheless, after sleeping on it, it occurred to me that although that approach makes sense in general, for our particular case a filtering logic happening on the client-side rather than on the server could potentially yield drastically better performance improvements, as the number of records to be searched through for each portal is in the order of magnitude of thousands in the worst-case scenario, rather than millions.

In other words, if you have to perform a search over millions and millions of records, without a doubt you need to execute this logic on the server and have an optimized database or search engine to do that heavy lifting. But if you are only searching through hundreds or thousands of records, up to a certain limit it makes sense to not involve a server at all and let the user's device do it.

This is our case because our haystack is the users that belong to a certain organization, and not only do we know exactly that number, we also have an established business target that caps that number to a limit that we control.

With that hypothesis in place, I needed to confirm that it was indeed a good idea. Using this approach would mean that we would have to return a payload to the browser with a set of ALL users registered so that when they used the search bar, we already had them in memory and ready to be filtered through. This brings up a few questions that would concern any experienced front-end engineer:

What would the total size of that payload be?
How long would it take to download that payload?
Are there significant memory implications of having this big data set in the browser instance?
When performing the search, wouldn't this heavy computation of filtering through thousands of array items in the client potentially freeze the browser's tab?
How fast can the browser filter through thousands of records?

To make a technical decision we need to take into account business variables too. When dimensioning, it is wise and common to discuss worst-case scenarios, e.g. how big is the total size of the payload for our theoretically biggest organization, but we also have to recognize that that scenario might only account for 0.01% or less of the user population and that we can have a 99% percentile or above with completely more reasonable numbers.

Take the payload download duration, for instance. It is true that under a 2G/EDGE or low bandwidth connection this approach could fail to meet an acceptable user experience when the haystack is big enough but is it not true that every application out there is meant to or will be used with this type of connection.

This is when having good reliable data about your users and your business audience pays off. Just as an example, it makes no sense to rule out a technical solution because it does not work in low-end mobile devices if none of your users are relying on mobile to access the application in the first place. I believe this is where a lot of optimization-oriented engineers drop the ball. When they fail to recognize or to account for the demographics of their users.

With this in mind, I turned to our analytics and databases to scoop out all the information necessary to answer the questions above using sensitive percentiles. In other words, what would the answer be for 80%, 90%, 95%, 99%, 99.5% of our users, and so on? With this data, I put together low effort proofs of concept in our testing servers that could illustrate the problem in practice and started doing some experiments.

The results were extremely positive. The browser was much faster than I had anticipated even in environments of low computational power, and I started to get excited at how much of a perceived difference it would be in the user experience after we completed the project. It was time to start building the real thing.

Typeahead component

In the legacy implementation, I mentioned that jQuery UI's Autocomplete plugin was used in a component built with BackboneJS. For the new one, we wanted to re-write it in React. We could have still relied on jQuery UI, but the truth is that the plugin itself had a few bugs associated with race-conditions so it was not perfect by any means.

We also wanted more flexibility and potentially remove any jQuery dependency in the app altogether in the future, so parting ways, and doing it from scratch was a better option. Thanks to the ergonomic design of React's API it is not that hard to build an autocomplete or typeahead anyway, so it was a no-brainer.

The component can be summarized as "a textbox that displays suggestions to the user as they type in it". As for technical acceptance criteria, we can establish:

The suggestions are not computed on every keystroke.
The suggestions should be computed after the user has stopped typing.
Should be fast.
If there are more suggestions than what can be displayed, the suggestions panel should be scrollable.
Should support mouse and keyboard interactions.
- Arrow keys highlight the suggestion below or above.
- Home and end keys take the user to the first or last suggestion result.
- Page up and down keys scroll the suggestions panel.
- Mouse wheel scrolls the suggestions panel.
- Enter key on a highlighted suggestion selects it.
- Escape key closes the suggestions panel and clears the text in the input.
Should be fully accessible and conform to the "listbox" role requirements as established by the Accessible Rich Internet Applications (WAI-ARIA) 1.1 specification (see https://www.w3.org/TR/wai-aria-1.1/#listbox and https://www.w3.org/TR/wai-aria-practices-1.1/#Listbox).

As far as the asynchronous nature of the interactions on the input and the suggestions computation, the Observer pattern paradigm fits perfectly well with the problem domain, so I built a solution using RxJS. The reason why it fits so well becomes clear if you try to compare the code that achieves the same visible behavior with and without it.

This is not meant to be an RxJS tutorial so I will not spend too much time focusing on the reactive details. A simple version of the subscription that achieves what we want could look like this:



import { BehaviorSubject } from 'rxjs'
import {
  debounceTime,
  distinctUntilChanged,
  filter,
  switchMap,
  retry,
} from 'rxjs/operators'

import { computeSuggestions } from './computeSuggestions'

const minLength = 2
const debounceDueTime = 200
const behaviorSubject = new BehaviorSubject('')

// ...

const subscription = behaviorSubject
  .pipe(
    debounceTime(debounceDueTime),
    distinctUntilChanged(),
    filter((query: string) => query.length >= minLength),
    switchMap((query: string, _: number) => {
      return computeSuggestions(query)
    }),
    retry(0)
  )
  .subscribe(
    value => {
      // set suggestions
    },
    error => {
      // handle errors
    }
  )

// ...

input.addEventListener('click', e => {
  behaviorSubject.next(e.currentTarget.value)
})

If we pass through the input value to the behavior subject every time the input changes, the operators piped to it guarantee that this subscription will execute the first callback passed to .subscribe() if:

a) the value is 2 or more characters long,
b) the user has stopped typing for 200 milliseconds, and
c) the last value that triggered the callback execution is not the same as the current one.

This could be easily integrated into a React component and we would have a very elegant and concise way of handling a stream of input change events in the way we need for our typeahead. Add the keyboard events handling logic, and we have all we need.

However, instead of doing that we can offer a more flexible solution if this is packed into a "headless" React hook with no UI concerns and shift that responsibility to the consumer. This way, we achieve a true separation between logic and view that allows us to re-use this hook in any situation without any changes no matter what design we have to adhere to.

This CodeSandbox has a complete and very similar implementation of the "useTypeahead" hook that I wrote for the feature, but with a completely different UI treatment, which demonstrates the flexibility of the API design.

Blocking the Main Thread

JavaScript is a single-threaded programming language. The fact that we would be doing the filtering in the browser instead of the server implies that the computation would not be an asynchronous operation anymore.

This is problematic because it means that as long as JavaScript is busy running our filtering logic and iterating through thousands of items the browser cannot do anything else, which results in a literal freeze of the tab. In this scenario, many interactions like JS-based animations, typing in inputs, selecting text, and others, become completely unresponsive. You have most likely experienced this before, and we usually refer to this as "blocking the Main Thread".

MDN has a much better definition of what's going on:

The main thread is where a browser processes user events and paints. By default, the browser uses a single thread to run all the JavaScript on your page, as well as to perform layout, reflows, and garbage collection. This means that long-running JavaScript functions can block the thread, leading to an unresponsive page and a bad user experience.

— MDN

Thankfully though, the browser is extremely fast. Even when filtering through thousands of records it only takes a few dozen milliseconds at worst on medium-end devices, which is not long enough for a user to notice any frozen or blocked behavior.

I wanted to be responsible and professional anyway and not block the main thread if possible. Thankfully (again), it is possible to do so by using a browser feature called "Web Workers".

Web Workers have been around for over 10 years but for some reason, they haven't gone mainstream yet. I blame it on how difficult they are to integrate into your development and deployment flow ergonomically. If you haven't heard of them, they're essentially an escape hatch that browsers provide to run code in a separate thread different from the Main Thread, as to not cause any blocking. There are certain caveats to using them but nothing that represented a deal-breaker for my use-case. The only real challenge was being able to integrate them seamlessly into our architecture and having them work with our infrastructure.

Web Workers are a little bit awkward to use in the sense that you have to pass in a path to a JavaScript file where your threaded code lives in, then you use asynchronous event messages to pass information back and forth.



// main.js

const worker = new WebWorker('../my-worker-file.js')

worker.postMessage('hello world')



// ../my-worker-file.js

onmessage = function(msg) {
  console.log(msg)
}

Just like any modern big scope single-page application, we bundle all our code together into a few processed files that we then statically serve to the browser at runtime, so there is never a one-to-one relationship between the file that lives in our source code and the file that is served to a user. Meaning, although we might have a file in our repo located at src/my-worker-file.js, that does not mean there is going to be a my-worker-file.js hosted in a server, since it is going to be prepackaged into our production bundle, with the rest of the codebase.

We could simply just opt to not bundle it and serve it directly as-is so that the code snippet above would work, but that means we would have to be manually editing our bundling configuration every time we wanted to rename, add or remove worker files. With the added risk that there would be a disconnect between our main thread code and those files at compile-time. We would have to remember to keep these changes in sync and do that manually, without any automated help from the build tooling. Needless to say, this is very brittle and not a good developer experience at all.

Ideally, it would be great to have an abstraction that allowed us to instantiate Web Workers anywhere in the codebase without having to update bundling configuration at all, while at the same time allowing usage of dependencies, share code across threads and keep all our compile-time checks in place like linting, import and exports checks, and type-safety.

The goal would be to have something similar to this work as expected, even when bundling is involved:



// main.js

import worker from '../my-worker-file'

worker.postMessage('hello world')



// ../my-worker-file.js

onmessage = function(msg) {
  console.log(msg)
}

Of course, one can build tooling to achieve this, but there are great ones already available in the community, like Comlink by Surma and Workerize by Jason Miller.

I used workerize since it fit my use-case better, and along with workerize-loader, it provided exactly what I wanted and even more. I replicated the configuration used in this minimal set-up repo which even includes test setups for both Jest and Mocha: https://github.com/reyronald/minimal-workerize-setup.

You can see an online demo here, which also demonstrates the Main Thread problem that I stated before pretty clearly.

No web worker	Using web worker

I used that same set-up and located the filtering logic in a separate thread, which guaranteed the browser's responsiveness even when heavily throttling down the CPU.

There is something else in the set-up that is included in the sample repo that I want to bring attention to. While working on this part of the project I started thinking of other places in the app that could benefit from moving code into a separate thread, but I did not want to spawn a new thread each time for every different piece of logic because in some instances there could be multiple needed in the same page.

Instead, I wanted to have a simple easy-to-use mechanism that could be leveraged to share Web Worker instances across the entire application, while making sure they were always terminated when no longer needed. This is the API I went with:



function ComponentA() {
  const [
    requestWorkerInstance,
    releaseWorkerInstance,
    getWorkerInstance,
  ] = workerManager()

  React.useEffect(() => {
    requestWorkerInstance()
    return () => {
      releaseWorkerInstance()
    }
  }, [requestWorkerInstance, releaseWorkerInstance])

  // ...

  const instance = getWorkerInstance()

  instance.doSomeHeavyAsyncWork()
}

In any component, you can get an instance to a single Web Worker thread by calling getWorkerInstance(). However, you have to make sure to call requestWorkerInstance() before so that a new one is spawned for you if it does not exist yet. If one is already available, you will get that instead.

When you are done and not going to need access to the thread anymore, then you call releaseWorkerInstance(), which will terminate it as long as no other consumer is depending on it.

The references of requestWorkerInstance and requestWorkerInstance never change so it is safe to include them as React.useEffect's dependencies, which makes it easy to integrate this system into any component. The most common flow would be requesting an instance when the component mounts and releasing it when it unmounts.

Internally, those functions keep track of how many consumers are depending on those instances at any given time so that they know when to instantiate a new one or terminate the current one. It is a singleton pattern applied to Web Worker threads.

The "worker manager"'s code is very simple and looks a little bit like this:



import workerizeFactory from './my-worker.worker'

let instance
let instanceCreated = false
let consumers = 0

const requestInstance = () => {
  if (!instanceCreated) {
    instance = workerizeFactory()
    instanceCreated = true
  }

  consumers++
}

const releaseInstance = () => {
  if (--consumers === 0) {
    instance.terminate()
    instanceCreated = false
  }
}

const getWorkerInstance = () => instance

export function workerManager() {
  return [requestInstance, releaseInstance, getWorkerInstance]
}

The actual version that I used is a little more complicated to accommodate for correct and proper type checks with TypeScript. You can see the full version in the CodeSandbox and repo posted above.

Smart Search logic

I mentioned earlier that we wanted this new search to be more flexible and smarter. I thought it would be cool if the matching algorithm worked similarly to other tools we developers use every day. I am talking about the approximate or fuzzy matching baked into the navigation search bar that apps like VSCode, Sublime Text, and even Chrome's DevTools have.

If you are not familiar, the logic will match any results that have the same input characters in the same order of appearance, but without the requirement that those characters appear consecutively. For example, the input "shnet" will match "Show Network". See the screenshot below.

Personally, I completely abuse and adore this feature of every software I use that has it. To me, it was a no brainer that this would improve the user experience. I went with it.

We released a version of the search with this matching logic, and to my surprise, users did not like it at all. A lot of them were very confused when they saw results that did not obviously resemble what they searched for, and instead of ignoring it or accepting it, they got concerned and even reached out to the support team to report them as bugs.

After getting overwhelmed with this type of feedback, we decided to remove the fuzzy matching aspect and go with exact matches. But product managers still wanted some level of tolerance to typos, and they also wanted results to be prioritized in their order of appearance in a "smarter" way, but they could not articulate properly how they wanted this to happen.

It was up to me to come up with a logic that was not just filtering out items that did not match the query, but that also had sensitive ordering and less aggressive approximate matching.

This was going to be a nightmare to deliver because we had to please the "gut feeling" that the results were good, without having explicit acceptance criteria items or clear requirements. It was obvious that it would require numerous iterations of design, development, release, then back to the drawing board to refine whatever heuristics were in place until the product managers and stakeholders were satisfied.

Instead of doing that, I decided to have a more unconventional approach to what we usually have in our team when it comes to new features. I built a CodeSandbox with about 2 or 3 different filtering strategies and some sample data, that would display the results of all of them side by side on the same screen, and sent it to our product manager. He would play around with it and give me feedback on what he liked, disliked, and what he would expect. I used this feedback to build unit tests, improved the heuristics, added a new iteration of the search logic, and repeated the process.

Ultimately we ended up with about 9 different strategies before we settled on one we were comfortable with. Many different libraries were used including Fuse.js, match-sorter, fuzzladrin-plus, and others. Some approaches were completely zero-dependencies, and some others were hybrids.

The one that took the cake worked something like this:

For user search...

Use Regex to find exact partial or complete matches of different words separately. Input terms have to be properly sanitized since the regular expression is built dynamically.
Sort the results that matched based on the index of the match. Matches that are closer to the start of the word should show up first. E.g., for the term "ron", "RONald" should show up before "byRON".
Break sort ties to the above alphabetically, so that if several results had the same match index, they show up A-Z in the UI, making it easier for the user to find what they want.

For non-user search (questions, tools, commands, pages, etc.)...

This is a little more complex since those items have search keywords associated with them in the metadata that user entities do not need to have, and these need to be factored into the logic.

Use Regex to compare the search term with a computed string that contains both the entity's primary name or string representation, and its search tags. If the regular expression matches, we then do a direct comparison of the search term only with the name. If both match, it is pushed to the results collection with a priority of 0. In this algorithm the lower the priority score the better. If just the regular expression matches, and not the direct equal comparison, it is pushed with a priority of 1. For example, if there's an item called "Settings" and the user searches for "settings", it would be a match with a score of 0. If they searched for "setti", it would be a match with a score of 1.
If the previous step failed, the user most likely made a typo. In this case, we cannot use a regular expression anymore. Instead, I iterate over all the separate words of the search term that are 5 characters or longer and compute the Levenshtein distance between them and all the search tags associated with each result individually. The 5 character limitation is there because the fewer characters you have in a word, the many more other words it resembles by just changing 1 or 2 characters. In other words, there were too many mismatches otherwise.

If for all cases there is an acceptable distance, we decide that it is a match. Before we push it though, we check if the term that matched also equals the item's primary name. If it does, it is pushed with a priority of 2, otherwise 3.
Finally, we sort these results based on the aforementioned "priority" so that ones with a lower score show up first.

This produces a set of results for each search term that is very intuitive, feels organic, almost hand-picked, and is very easy to navigate through.

End Result

As with every release, we always try to gather as much data and feedback as possible so that we can gauge the success of every project. On this one, we included many statistical metrics to help us understand how our users were employing the new search and how we could improve either the implementation or the metadata associated with each result to bump their visibility appropriately.

A good one to discuss is usage duration. It measures how long it takes the user from the moment they focus the search input to the moment they select a search result or exit the search. This helps us know if they are finding what they need quickly enough. If it is too long, it means that the users are struggling.

The image above shows that in the last 30 days, in 73.4% of the instances a user result was selected within 0 to 5 seconds. The next runner-up is 5-10 seconds with 20.8%. Both of these account for 94.2% of the searches, and the biggest percentile corresponds to the shortest amount of time, so I consider this a positive outcome.

We also include a survey box in the app itself via Appcues. On a scale from 1-6, with one being the worst and six being the best, the new search functionality was well received with an average of 5.2 out of 6. Some quotes from participants:

I love this enhancement! This is very helpful when we work on reports.

and

I can speak to it as exciting to have such a great update.

Now let us look at the most interesting metric to me, performance. This graph is over a longer period than the legacy one, two weeks instead of just one.

New Search - 2 weeks
(min: 3.25ms, max: 121.13ms, avg: 17.11ms)

	Legacy	New
min	158.21ms	3.25ms
avg	562.47ms	17.11ms
max	9,950.00ms	121.13ms

The difference is astounding across the board. On average, it is 30 times faster than the legacy implementation. Not only that, but this duration is much more consistent across different portals regardless of size and it's not dependent on network conditions, meaning that our bigger portals are experiencing up to 80 times the performance, maybe even more.

This validates all of the hypotheses I made at the grooming stage of the process, so I was very satisfied to see that my predictions came true. I closely monitored this metric following the formal release to make sure there were no exceptions and everyone was having a smooth experience. No surprises were found.

Conclusion

The biggest conclusion I want to draw attention to is that even though something may sound sub-optimal in theory and does not fit already established best practices, it does not mean that it will be in the real world when we factor in actual business variables and data.

A client-side approach like this would never work in the majority of cases of search functionalities. This scenario usually makes it more difficult to think outside of the box and come up with alternate solutions. The nature of our problem specifically was different and we failed to recognize that as a team in our first discussions about the project, but thankfully, we recognized that before investing any significant effort.

Another success of the process was writing down the questions and concerns we had with the approach, and answering them experimentally with real data and low-effort proofs of concept in a spike early in the project. This gave us the confidence we needed before formally committing to any technical decisions, and above everything, real, not just theoretical technical arguments to back-up those decisions. This in particular is something that our team was not used to doing and has struggled with in the past, and we have had to pay a big price as a result.

Just for completeness sake, the CodeSandbox below is an oversimplified visual representation of what I built. It is lacking many of the details I described in the post and some others that I did not mention. For instance, it just searches for one entity type, users, does not rely on Web Workers, is lacking a lot of code we included to gather metrics, and has no automatic tests.

Why I avoid `get`-like functions in JS

Ronald Rey — Tue, 10 Mar 2020 11:30:56 +0000

Photo by Brazil Topno on Unsplash

Because they are not statically analyzable.

Let's unpack.

First, let me clarify what I mean with that title. I'm referring to the type of functions that receive a path to an object property in the form a string, and return the value at that property or a fallback default one. For example:

const typeName = get(response, 'data.type.name', null)
// ☝ instead of doing `const typeName = response.data.type.name`

There are many implementations of this pattern in very popular JavaScript libraries out there and I've seen it all over the place, including very high-profile projects, despite me considering it a very bad idea. You might remember it from lodash's get. Immutable also has its own version of the same concept with getIn.

These functions exist as a convenient way of reaching a value in a nested property of an object in a safe manner. In many cases it is common to have an object that is only partially defined, and trying to access any given property in it may cause the program to fail.

const response = {}
const typeName = response.data.type.name
// ❌ Uncaught TypeError: Cannot read property 'type' of undefined

// 👆 That would be a runtime error happening in the app
//    when that code is executed. Crashing it.

To avoid that, the developer should make sure that all the properties in the path are defined before actually trying to access them. The vanilla way of achieving this would be something like:

let typeName = null
if (response && response.data && response.data.type) {
   typeName = response.data.type.name
}

So yeah, needless to say, a utility function that abstracts away all the redundant ugliness is much welcomed. So what's the problem with this type of get function, as I defined it above?

It is not type-safe.

With a type system in place, like TypeScript or Flow, we would have a type alias or interface that defines the shape of the object we are working with. The compiler uses that metadata to find bugs in your code when you are accessing and manipulating those objects, so it would be able to warn us when we try to do something that would end-up in a TypeError like the one we saw above.

type MyResponseType = {
  data?: {
    type?: {
      name: string
    }
  }
}

function main(response: MyResponseType) {
  const typeName = response.data.type.name
  //                         👆
  // TypeScript: ❌ Object is possibly 'undefined'.

  // Compilation error happening at build or development time,
  // not when the app is running.
  return typeName
}

However, when you do that property access through a string path, you are killing the compiler's ability to analyze your code, understand your intention and provide helpful advice BEFORE your app is deployed and running. The real problem arises when we start considering the implications of that beyond our immediate example from above.

If we rewrite that snippet to use the vanilla approach our compilation error is gone and we can now build and run our app. Let's see what happens if we introduce a type alias update.

type MyResponseType = {
  info?: { // 👈 Rename `data` -> `info`
    type?: {
      name: string
    }
  }
}

// ...

let typeName = null
if (response && response.data && response.data.type) {
   typeName = response.data.type.name
   // TypeScript: ❌ Property 'data' does not exist on type 'MyResponseType'.
}

TypeScript can recognize that the properties we are trying to access do not match the contract we've defined for this object and therefore this would undoubtedly fail at runtime, but we are getting this very informational heads up from the type system.

Had we been using a more dynamic approach like the one suggested by the utility functions we're discussing, this mistake would've been completely invisible to the compiler and our app would've built like there's no problem at all, when in fact we've unknowingly introduced a bug, or worse, several bugs all over the place.

type MyResponseType = {
  info?: { // 👈 Rename `data` -> `info`
    type?: {
      name: string
    }
  }
}

// ...

const typeName = get(response, 'data.type.name', null)
// TypeScript: Everything looking good chief!

If you are working in a large organization with multiple development teams contributing to the same codebase this is an occurrence that could happen surprisingly frequently. Even if you are the single developer of an app, this will still happen eventually to any non-trivial codebase.

This is a terrible mistake that could cause very serious production crashes that your users would end-up being the victims of. The reputation of your product would be harmed and the engineering team would be the one to blame.

But most importantly, this also makes refactoring a nightmare and a very stressful endeavor to a developer or a team. Rewriting code that is not statically analyzable will have you introducing regressions all over the place and slowing down the whole process dramatically since every line of code changed will require a much more thorough review and manual testing.

This is fatal for a product since, in practice, this will freeze your codebase in time, tying it up to accumulating technical debt given that continuous improvement through refactoring becomes very dangerous, risky and intentionally avoided by both the development team and the business team.

Then, given enough time, codebase becomes such an untouchable mess that it requires an entire re-write if any thought of sustainable progress is expected, causing the organization considerable and preventable losses.

The root of the problem

I blame the dynamic nature of the JS language that made this type of API design common-place throughout its maturing process. In other more strict languages working on the implementation of this get-like function would've been more tricky, motivating developers to come up with a more robust type-safe approach instead.

Had this function been designed with a more functional mindset, it could've been avoided easily. Just for illustration purposes, take a look at this alternative API that achieves the same goal, without losing type-safety.

function get<T>(fn: () => T, defaultValue: T): T {
  try {
    const result = fn()
    return result
  } catch (error) {
    return defaultValue
  }
}

// ...

const typeName = get(() => response.data.type.name, null)

What I recommend

Use the optional chaining operator.

It is available in TypeScript, Babel, even plain JS in Node.js 12 and above and all the latest versions of the most popular browsers. So you can now just do:

const typeName = response?.data?.type.name ?? null

No libraries. No superfluous functions. No plugins. Just plain JavaScript.

Do it even if you are not using any type system. Some code editors and IDEs can still provide rudimentary type-safe support to plain JS files, and if you eventually do integrate a type system, you'll get that coverage for free.

If for some reason you are working in an environment where you can't use the optional chaining (can't upgrade TypeScript/Babel, an old version of Node, have to support old browsers and have no compilation tools, etc.), then maybe opt to use the functional get alternative I used as an example above, but I would argue you have bigger problems to take care of!

My Favorite Web DevTools Features of 2019

Ronald Rey — Mon, 02 Mar 2020 14:17:24 +0000

Photo by Jo Szczepanska on Unsplash

A few weeks ago I published "My Favorite Web Features of 2019". There, I promised I'd follow up with a DevTools version of the same concept. Well, here we are!

The structure of the post will be the same. I'll start from my least favorite (but still good) to my most favorite DevTools features that have been introduced to the Firefox and Chrome developers tools in 2019. This is not a list of all introduced features, just the ones that stood out to me.

Good New Features

Chrome: Visualize layout shifts

https://developers.google.com/web/updates/2019/07/devtools#layoutshifts

A tool that will help you visualize how content jumps around in the page as assets load. I don't have much else to say about this. Although I haven't used it yet, I know this feature will be invaluable whenever I have to do this kind of optimization, so I'm keeping it in the back of my mind.

Firefox: CSS Flexbox Inspector

https://developer.mozilla.org/en-US/docs/Tools/Page_Inspector/How_to/Examine_Flexbox_layouts

The Flexbox Inspector allows you to examine CSS Flexbox Layouts using the Firefox DevTools, discovering flex containers on a page, examining and modifying them, debugging layout issues, and more.

– Mozilla

This feature won't have too much impact on me personally since I've been using flex for years and know my way around it by now, but I can imagine this tool being amazing for developers not very familiar with flexbox yet.

Flex can be tricky to grasp and there are a lot of standalone sites out there that aim to make it easier to understand, but having a tool for this built into the browser is still really helpful.

In general, I love it when browsers make it easier for people to get into front-end development, given how tricky and chaotic such a journey can be!

Firefox: Accessibility-related tooling

Check out these other links that go more in-depth on all Accessibility related tooling in Firefox:

It's really great to see so many accessibility-related features land in Firefox in a single year! It's clear that the Firefox team has been prioritizing accessibility; they're taking it very seriously.

In general, I would say that accessibility tools across browsers leave a lot to be desired. There's a lot of room for improvement. This is a huge problem because it makes for a high barrier of entry to writing accessible applications.

I think we can partially blame the current state of developer tooling around accessibility as the primary culprit for not having more accessible apps on the web today.

What I like about Firefox is that all accessibility-related tooling is grouped under the Accessibility Inspector tab, making it easier to find everything related to accessibility in a single place. It also has some features that Chrome doesn't have.

Chrome's equivalent of the "Check for issues" feature is Audits > Accessibility > Run Audit, tucked away in a different part of the DevTools interface. It's difficult to discover, particularly if you are expecting the Accessibility tab to have everything you need. It hasn't. There's also no mention of color contrast in it, for example.

In my opinion, Chrome needs to redesign the UI/UX around accessibility features in their DevTools. Firefox's UI/UX is better here. I'd probably switch to it when I have some serious accessibility auditing to do.

Chrome: Code Folding in the Sources and Network tab

https://developers.google.com/web/updates/2019/01/devtools#folding

A super simple feature that's very useful. There's not much to elaborate on here, other than saying that it was long overdue.

Firefox: Multi-line mode console

https://developer.mozilla.org/en-US/docs/Tools/Web_Console/The_command_line_interpreter#Multi-line_mode

The fact that Chrome hasn't implemented something like this yet is astonishing because this is just so useful! It's generally very annoying when you have to Shift + Enter something that's more than one line long. I accidentally hit Enter without pressing Shift all the time, and then I have to start over 🤦‍♂️. Ugh!

Great New Features

Chrome: Highlight all nodes affected by CSS property

https://developers.google.com/web/updates/2019/03/devtools#highlight

Hover over a CSS property that affects a node's box model, such as padding or margin, to highlight all nodes affected by that declaration.

– Google Developers

This is another very simple feature that adds a lot of value. Being able to directly visualize this in the browser can make a big difference when working with layouts. If this feature were taken out tomorrow, I'm sure that a lot of us would struggle to adjust back.

Firefox: Inactive CSS

The Inspector now grays out CSS declarations that don’t affect the selected element and shows a tooltip explaining why -- and even how to fix it.

– mozilla.org

The "how to fix it" part is what I love the most about this feature. CSS declarations aren't always applied, because the developer made some wrong assumptions. It can be hard to figure out what's going on, and it usually involves some time-consuming experimentation, investigation, and context switching.

Having this feature available can be more valuable than you think, and it can make a big difference for beginners learning CSS. Check out this video demo:

Fantastic New Features

Firefox: New "Changes" tab for CSS

https://developer.mozilla.org/en-US/docs/Tools/Page_Inspector/How_to/Examine_and_edit_CSS#Track_changes

Firefox added a new "Changes" tab to their developer tools that allow the user to see a diff of CSS changes made through the Inspector in the current session. This is great when you're tweaking a design in the browser and want to know what you've changed so that you can take back those changes to your stylesheets.

This is an extremely common practice for front-end developers and it's very annoying having to keep track of those changes mentally. You'd often find yourself trying not to make too many changes so that you don't forget them when you return to your code. I've wasted so much time in the past taking care of this manually. Because of this feature, not anymore!

Here's a video demonstration of how it behaves:

Chrome has a similar feature, but it's not as user-friendly, discoverable, and easy to use as it is in Firefox, so I'm going to be whipping out a Firefox instance when I have a lot of CSS changes to make.

Chrome: Inline breakpoints in the breakpoint pane

https://developers.google.com/web/updates/2019/04/devtools#inline

Finally! This has been a pet peeve of mine for years, especially since Promise-based asynchronous code and inline arrow functions became so popular in recent years.

Not being able to surgically insert breakpoints at specific parts of a multiple call single statement makes it so difficult to debug these kinds of programs. So much so that I developed the habit of splitting up these calls as much as possible and always favoring explicit over implicit returns when writing arrow functions, just for the sake of making it easier to insert breakpoints and debug later.

This is something that I use every day now and that's why it is so high on my list.

Chrome: Detailed tooltips in Inspect Mode

https://developers.google.com/web/updates/2019/01/devtools#inspect

When my instance of Chrome auto-updated, I noticed this within the first minute. This is such a simple change and I feel like it brings so much value. Having this information pop up right away, when hovering over elements, has been more useful to me than I thought.

This being said, I appreciate that there's a certain development workflow that wouldn't benefit from this much. It all depends on what you are working on and how you are working on it.

Chrome: Autocomplete with CSS values

https://developers.google.com/web/updates/2019/05/devtools#values

A feature that allows you to only type the CSS value of any given rule instead of the property name and DevTools will provide you with auto-complete. This takes another load of your mind when styling in the Inspector.

After years of doing front-end development work, I still don't remember the box-sizing: border-box; combination on the spot. I usually have to do some context switching and quick Googling before coming back to my app, so the first time I tried this after this feature was introduced I was ecstatic. It's the simple things...

Chrome: Logpoints

https://developers.google.com/web/updates/2019/01/devtools#logpoints

Oh boy, I saw this feature in the release notes for Chrome 73 in January and knew 2019 was going to be a good year. No more console.log all over the place in your codebase (and then having to remove them all). Not only that, but it also eliminates having to do changes to your app, waiting for it to recompile, and then refreshing the browser for those changes to take effect.

Not to mention when you had to do logging-related debugging in production apps where you just couldn't change the source that is running to add console.log yourself.

This is of great value to me. It makes me wonder why it took the Chrome team so many years to implement this, especially since they've had the breakpoint editor (thanks to conditional breakpoints) already designed and working for many years.

Conclusion

So there you have it. I'm genuinely curious about what your favorite DevTools picks would be. Is there a feature that you rely on in your day-to-day that was introduced last year that I haven't mentioned? Or that I mentioned but that's not as high as you might expect? Maybe there was a really good new feature that I missed entirely. Let me know in the comments!

My Favorite Web Features of 2019

Ronald Rey — Mon, 10 Feb 2020 14:35:55 +0000

Photo by Lavi Perchik on Unsplash

With 2019 behind us, I thought it would be a fun idea to list out all my favorite features that came to the web in 2019. We saw a lot of great functionality come to fruition, both from the developer's and the user's perspective. This article serves in part to bring more attention to features that I believe are worth a look.

These are my personal picks, starting from least favorite to most favorite. This is not a list of all web features introduced in 2019, just the ones that stood out to me.

I won't be going into too much detail of what the individual features do or how they work, as that has already been thoroughly explained in their official release channels or in other documentation. Click on the links provided at the start of each item for that. My focus will be on why I picked them and what impact I think they'll have on myself and the web.

I'm going to follow up with a post about my favorite DevTool features of 2019 in a few days, so stay tuned for more if that sounds interesting to you.

Honorable Mentions

KV Storage

A proposal for an async key/value storage API for the web.

– http://wicg.github.io/kv-storage

This might've slipped under your radar. An async alternative to localStorage has been under discussion for quite some time. The main idea is to improve the terrible performance and degraded user experience that comes with the synchronous nature of localStorage, because it blocks the main thread.

If you were unaware of this (let's face it, not all localStorage usages are intensive enough to cause noticeable performance differences for everyone), check out Chris Heilmann's old, but still relevant article: "There is no simple solution for local storage".

The proposed alternative to the above-mentioned problem is "KV Storage". A working version finally landed in Chrome 74. Read about it in the links provided above. I'm really looking forward to this API reaching stability and being implemented everywhere. It should bring significant improvements to our web apps!

CSS Properties and Values API

The CSS Properties and Values API — part of the CSS Houdini umbrella of APIs — allows developers to explicitly define their CSS custom properties, allowing for property type checking, default values, and properties that do or do not inherit their value.

– MDN

This is a feature that looked pretty exciting to me, until I actually tried it. Misusing a CSS variable just results in its value being ignored. It doesn't result in any runtime errors or warnings. Even if you use this API and register the property, you still don't get any error or warnings, even though the default value kicks in.

This is fundamentally different from how we usually experience default values in other ecosystems. If you were to pass the wrong value type to a property with a default value in a statically or dynamically typed language, you would get a type error at compile time or runtime, instead of getting the default value applied there and the execution continuing.

For those reasons, I think the term "type checking" doesn't protect you against much. It's a little bit misleading in this context too, and I probably would've suggested different terminology if I were part of the team that wrote the spec.

Also, registering the property with JS after your CSS has loaded will override whatever value you defined for it in :root or html. The CSS way of doing it with the @property syntax isn't implemented yet, so you would need to refactor some of your existing code for it to play well with this functionality. You need to be extra careful to get this right.

I also don't feel that the "default values" feature is that useful either, as I don't see it being such a problem in an organized codebase. Even if the codebase wasn't organized, this is the kind of thing you would immediately catch when testing your changes in the browser.

To me, the most practical application of this addition right now is transitions. Without this API, if you tried to use an animation that relies on a custom CSS property, it just wouldn't work. The browser wouldn't know how to interpret those values in that context. By registering the property with this API, it provides the browser with the necessary information to interpolate correctly.

As of right now, I can't think of many instances where this would improve my developer experience or the code quality of my stylesheets and design systems. On another note, this was only added to a very recent version of Chrome and nowhere else yet, so it doesn't seem very reasonable to introduce this in your workflow yet.

Still, it is very exciting to finally see more of Houdini being implemented in some browsers. I can't wait to see the rest of the APIs land!

The `rendersubtree` attribute

Adds the rendersubtree attribute to all HTML elements, which locks a DOM element for display. When rendersubtree is set to "invisible", the element's content is not drawn or hit-tested, allowing for rendering optimizations. The rendersubtree "activatable" token allows the browser to remove the invisible attribute, rendering the content, and making it visible.

– Chrome Platform Status

I am immediately drawn to any feature that improves the performance of my apps. When I saw the addition of the rendersubtree HTML attribute, I was intrigued. Because this is extremely recent, I haven't used it yet, but I'm looking forward to using it in production. I'll definitely do some benchmarking when I do.

I don't expect the differences to be that significant for small to medium trees, and huge trees are not very common in the apps I've been working with recently. So it'll be a while until I can directly benefit from this. Still, it's nice to know it exists and that I can reach for it if necessary.

Largest Contentful Paint (LCP)

Largest Contentful Paint (LCP) is an important, user-centric metric for measuring perceived load speed because it marks the point in the page load timeline when the page's main content has likely loaded—a fast LCP helps reassure the user that the page is useful.

– Philip Walton

A new API that allows us developers to better measure the performance of our sites. This bridges a gap in performance measurement that other existing APIs like load, DOMContentLoaded, First Paint, First Contentful Paint (FCP), and others didn't cover.

Sometimes it is very difficult to justify efforts for optimizing our code and our app's performance. Tools like this allow us to easily lay out objective measurements that can make this investment decision much easier not just for us, but also for our business and product management teams.

It is much easier to get budget for performance improvements and technical debt when you have numbers to back it all up, and that's why I'm happy this exists.

Several utility functions added to JavaScript

Object.fromEntries. https://v8.dev/blog/v8-release-73#object.fromentries.
String.prototype.matchAll(). https://v8.dev/blog/v8-release-73#string.prototype.matchall.
Promise.allSettled. https://v8.dev/features/promise-combinators#promise.allsettled.

I'm grouping all these under one section because there's not much to say about them other than what has already being said in the links above and all around the web.

They're all great to have official native support for, but they're not higher on my list of favorites because we've been able to use some of them for a while already, thanks to compilers like Babel and TypeScript.

There have been okay workarounds for some of these with plain JS as well. For instance, Object.fromEntries can be polyfilled with a one-liner using Array.prototype.reduce, and Promise.allSettled's behavior can be simulated with some clever promise wrapping.

In other words, their entry into the language won't make such a meaningful impact on a developer's life or the web. Nonetheless, they're nice to have.

Public & private class fields in JavaScript

We've been writing class fields in JavaScript for many years now, thanks to tools like Babel and TypeScript, so some people may be surprised to know that, until very recently, all that was non-standard JavaScript. The proposal hadn't yet formally reached Stage 3 of the TC39 process, which is why there hadn't been many official implementations.

But that changed in 2019, when we saw it implemented natively, with no compilation necessary in Node 12 and Chrome 72 & 74!

This didn't make it to my list of absolute favorites, because we've had tools that allowed us to use this syntax for a long time now. Even though this is already implemented in several platforms, Babel's plugin currently still tries to compile the private fields even when using loose mode. It offers no configuration options to skip it.

The TS implementation (which was just released in TypeScript 3.8 Beta, not stable yet as of this writing) will do the same, so you won't be able to benefit from the engine's native implementation of the private variant if you use any of these tools (at least for now). This means that there will be no significant improvements to bundle size and no significant improvements to performance.

Really happy about

`backdrop-filter` CSS property

The backdrop-filter CSS property lets you apply graphical effects such as blurring or color shifting to the area behind an element. Because it applies to everything behind the element, to see the effect you must make the element or its background at least partially transparent.

– MDN

A new CSS property that allows us to be more creative with our designs and UIs! Check out this CodePen by Robin Rendle to see a demo of what it is about.

Here's a more common use case of what this property now lets us implement natively in the browser, without ugly hacks:

Credits to Renan Barco

To be perfectly fair, this was already implemented in Safari and Edge years ago, but 2019 saw it land on Chrome 76. Only Firefox is missing!

Compute img/video aspect ratio from width and height HTML attributes

This was an extremely easy one to miss, as it wasn't promoted much at all through the usual release channels. However, if you do a lot of front-end work, this will make you very happy! In my opinion, this should've been made more public, as I think it's a great feature (or fix, depending on how you look at it). Since there isn't much documentation about it out there, I'll go into a little bit more detail about this particular item.

Previously, if you had an image tag with explicitly defined width and height HTML attributes, and you wanted this image to be as wide as the parent's element, you would naturally use CSS width: 100%. The problem was that the browser wasn't able to correctly compute the aspect ratio before the image loaded. So you'd end up with the page content/layout jumping around and being repositioned after the image had loaded. This problem is called layout shifting.

Before:

After:

Now, as long as the image has height: auto, the content won't jump around anymore:

Here's an online demo: https://codesandbox.io/s/github/reyronald/compute-img-video-aspect-ratio-demo

This feature/fix only landed in Firefox and Chrome as of this writing, so you might still see the old behavior in other browsers. Kudos to the Firefox team for leading this effort!

However, it's worth mentioning that this won't work on iframes (which I don't think will affect many people).

Here's a video by Jen Simmons from Mozilla where she demonstrates the issue and the fix herself in more detail:

Desktop PWAs

https://developers.google.com/web/progressive-web-apps/desktop

I don't develop or work with Progressive Web Applications (PWAs), so any PWA-related feature is irrelevant to my day-to-day work at the moment. But I still recognize their value and always stand in admiration whenever I see some new advancements made on that front.

When I originally came across PWAs, I perceived them as a mobile-only feature. It never occurred to me that we could benefit from them in the desktop world, but here we are! I highly recommend that you install the PWAs of the websites you frequently use, as the user experience is much snappier than through the browser.

I use them myself and can happily recommend them to anyone. It also makes me believe that desktop PWAs could even replace some simple (and maybe even not so simple) Electron apps. This would further decrease the barrier of entry to native development.

PWAs in general are very exciting. If you haven't had the chance to research and experiment with them yet, give them a shot. The coolest thing about PWAs is that it requires little effort to turn your existing web app into a PWA. All you need is a manifest.json file, a minimal service worker, and you're good to go!

My absolute favorites

`prefers-color-scheme` media query

"The prefers-color-scheme CSS media feature is used to detect if the user has requested the system use a light or dark color theme."

– MDN

Example:

@media (prefers-color-scheme: dark) {
  body { background: black; color: #ddd; }
}

@media (prefers-color-scheme: light) {
  body { background: #eee; color: black; }
}

As a developer, I haven't had the opportunity to leverage this feature yet. But I have built theme-able sites, and you might end up including styles for themes in your bundle that are never actually used if you're not careful. So if your themes are exclusively dark/light (which I assume is the majority of use cases), having this feature available as a media query allows you to more easily optimize. Just write something like:

<link href="/light.css" rel="stylesheet" media="(prefers-color-scheme: light)">
<link href="/dark.css" rel="stylesheet" media="(prefers-color-scheme: dark)">

Note that you might be able to get away with just one stylesheet for both themes if you rely on custom CSS properties. It'll just be a little bit more challenging and might require some more clever architecting.

This feature gets extra points in my book because my preferred color scheme is dark and I benefit from it a lot as a user. I'll benefit even more as websites adopt this new media query in the future. Having the web adapt to that automatically as I browse will improve my user experience and quality of life.

Hardware Media Keys support

https://developers.google.com/web/updates/2019/02/chrome-73-media-updates#media-keys

Desktop Chrome users can now use the media keys on their keyboards and headsets, like play/pause, previous and next track to control the audio, and video playback in the browser.

This is a very niche feature that will only have an impact in a very small subset of developers, since not all of us work with media-related apps. Regardless, it's a very welcomed feature from the user's perspective that will further improve our experience on the web!

I personally love this one, as I consume a lot of media on the web and have a headset with very accessible and easy-to-use media keys. I bet that developers working on these kinds of apps are very happy to see this become reality as well.

Additions and improvements to the `Intl` API

There have been several additions to the internationalization API in late 2018 and 2019:

Intl.RelativeTimeFormat https://v8.dev/features/intl-relativetimeformat
Intl.DateTimeFormat improvements https://v8.dev/blog/v8-release-76#intl-datetimeformat
Intl.ListFormat https://v8.dev/features/intl-listformat
Intl.Locale https://v8.dev/blog/v8-release-74#intl.locale
Intl.NumberFormat added capabilities https://v8.dev/features/intl-numberformat

Any improvement or addition on Intl is an enormous win for the web, because it allows us developers to rely on the platform for anything related to internalization, instead of having to pull in third-party libraries, bloat the size of our bundles, and increase the surface area of bugs and complexity in our codebase.

I'm particularly excited about the date- and time-related ones. Currently, a very popular library for that is moment, which is a surprising 231.7
kB minified and 65.9 kB minified + gzipped. You can write entire apps smaller than 65 kBs!

Although the improvements on the Intl API are not a replacement for everything that moment can do, I have worked with websites where all of moment's usages could be replaced with Intl.RelativeTimeFormat and Intl.DateTimeFormat. I'm fairly certain that such a scenario is not a rare occurrence.

Not all apps require internationalization though, so this is not something that absolutely everyone can benefit from. Unlike my next pick.

Native Lazy Loading

Lazy loading is a strategy that delays the loading of some assets until they are needed by the user based on the user's activity or navigation pattern. If correctly implemented, this delay in asset loading is seamless to the user experience and might help improve initial load performance, including time to interactive, as fewer assets are required for the page to start working.

– MDN

Lazy loading can greatly improve the user experience of any app in all situations, and the benefits are significantly greater for users with low-end devices and poor network conditions.

So the fact that lazy loading images and iframes are now natively supported in Chrome is huge news! Specifically because of how easy and plug-and-play the whole functionality is. Just add the loading HTML attribute:

<img src="..." alt="" loading="lazy" />

Although it is not yet supported across all major browsers and MDN still considers this an "experimental" feature, it is perfectly safe to use, as other platforms will simply ignore it and default to their existing behavior anyway.

It is also very easy to polyfill. Check some of the links above for a super simple polyfill implementation or check out this link for a more robust one.

I already went ahead and added loading="lazy" in all my image tags for some of my projects, and I've seen improved performance. It's probably the feature with the highest benefit to cost ratio that I can think of in recent memory.

Conclusion

So that's it. It was difficult to rank them and you can come up with arguments that will rank any of these higher than other ones, but don't get too lost on that.

What about you? Do you have any particular favorite picks? Did I miss anything relevant? Would you like to elaborate on the importance or lack of importance of any particular pick? Let me know in the comments below.

Like I said in the beginning, I'm going to follow up with a post about my favorite 2019 DevTools features in a few days, so stay tuned for that!

TypeScript Types or Interfaces for React component props

Ronald Rey — Mon, 03 Feb 2020 15:08:02 +0000

Photo by Simone Hutsch on Unsplash

When writing React components with TypeScript, you have two options when it comes to typing its props. You can use either type aliases or interfaces. Which one would you choose? Does it matter? Is there really an optimal choice? Are there any drawbacks to one or the other? Let's explore the relevant differences between them before we conclude.

This is not a deep case study of how types aliases and interfaces differ in TypeScript, but I'll provide just a brief overview of some of the differences that are relevant to React props so we're on the same page. Later we will explore how those differences can come into play in this context.

Type aliases vs. Interfaces

Type aliases and interfaces in TypeScript are equivalent in the majority of cases. Things that you can do in one you can also do with the other with just syntax changes, and of course, there are exceptions.

Let's take a look at some examples:

Regular object with properties

✔ Equivalent

type User = {
  name: string
}
// ...
interface User {
  name: string
}

Arrays or indexers

✔ Equivalent

Arrays or indexers:

type Users = User[]
// ...
interface Users {
  [index: number]: User
}

👆 In this case, though, the interface would be missing all array methods like .push, .map, etc. so both definitions won't exactly be equivalent, and the interface would be less useful unless that's precisely what you are aiming for.

To remediate this, you would have to explicitly extend from the array type like so:

type Users = User[]
// ...
interface Users extends Array<User> {
  [index: number]: User
}

Functions

✔ Equivalent

type GetUserFn = (name: string) => User
// ...
interface GetUserFn {
  (name: string): User
}

Function overloading with added properties

✔ Equivalent

Let's use a real-world example, this is the it: TestFunction type definition from mocha, see the source here.

type TestFunction = 
  & ((fn: Func) => Test)
  & ((fn: AsyncFunc) => Test)
  & ((title: string, fn?: Func) => Test)
  & ((title: string, fn?: AsyncFunc) => Test)
  & {
    only: ExclusiveTestFunction;
    skip: PendingTestFunction;
    retries(n: number): void;
  };
// ...
interface TestFunction {
    (fn: Func): Test
    (fn: AsyncFunc): Test
    (title: string, fn?: Func): Test
    (title: string, fn?: AsyncFunc): Test
    only: ExclusiveTestFunction
    skip: PendingTestFunction
    retries(n: number): void
}

Although you can achieve this with both, I would recommend sticking to an interface in this case because of the clearer semantics and cleaner syntax.

Merging

✔ Equivalent

Merging properties from different types into one, referred to as intersections when using types aliases, or extensions when using interfaces.

type SuperUser = User & { super: true }
// ...
interface SuperUser extends User {
  super: true
}

type Z = A & B & C & D & E
// ...
interface Z extends A, B, C, D, E {}

There's one significant difference here that is not obvious from just looking at those examples. When extending interfaces, you absolutely need to declare a new one with the extension result, whereas with a type alias, you can inline the intersection type, for example:

function(_: A & B) {}
//...
interface Z extends A, B {}
function(_: Z) {}

Class implementation

✔ Equivalent (!)

This might seem counter-intuitive, but you can implement both type aliases and interfaces in classes!

type AnimalType = {}
interface IAnimal = {}

class Dog implements AnimalType {} // ✔ Works
class Cat implements IAnimal {}    // ✔ Works

However, although possible with both, this use-case is more commonly attributed to interfaces due to classic object-oriented language design, and it's safe to say you will rarely see types used this way in real-world codebases.

Union types

❌ NOT Equivalent

It is possible to define a type that is either one thing, or the other when declaring it as a type alias by using the union type syntax, but this is not possible with an interface:

type Z = A | B
//...
interface IZ extends A | B {} // <- ❌ INVALID SYNTAX, not possible to achieve this

It is also not possible to extend from a type that is declared as a union type.

type Z = A | B

interface IZ extends Z {} // ❌ Compilation error:
// "An interface can only extend an object type or intersection
// of object types with statically known members."

Redeclaration

❌ NOT Equivalent

There's another way of extending an interface definition. By redeclaring it, whatever is defined in the latest declaration will be merged with the properties of all previous declarations. So you can say that an interface's behavior is very similar to the cascading nature of CSS.

interface User {
  name: string
}

interface User {
  gender: string
}

const user: User = { name: 'Ronald', gender: 'male' }

This is not possible to achieve with a type alias though:

type User = { name: string }
type User = { gender: string } // ❌ Compilation error
// "Duplicate identifier 'User'."

This is particularly useful if you need to extend an existing object's definition whose type is declared outside of your reach, i.e. it's coming from a third-party package, or it's part of the standard library.

Imagine your web app adds a few properties to the window object. You won't be able to use your added properties without getting a compilation error because they won't be part of the original definition of the Window type. But since Window is declared as an interface, you can do this somewhere near the entry point of your client app:

declare global {
  interface Window {
    $: jQuery
  }
}

// ...

// and now you use $ globally without type errors
window.$; // 👍

NOTE: This is not an encouragement to use jQuery.

Using in React props

With all those considerations in mind, which one would you say is the best choice for typing a React component's props? Is there a unique best practice? Can we say that using one or the other is an anti-pattern or should be avoided? Let's unpack.

When I see props declared with an interface, I immediately stop in my tracks and think: "Is it declared as an interface because the developer is going to be implementing it in a class later?", "Is it declared as an interface because the developer is going to redeclare it later, or is the possibility of redeclaration an intended feature of this component? If it is, how does this affect the usage of the component, if at all?"

I then proceed to start looking for the answer to these questions before I continue doing what I was doing, most times to little fruition because those were not factors involved in the decision to use an interface, but at this point, I've already wasted development time and more importantly precious scarce cognitive resources that I will never get back.

I don't ask myself those questions when I see a type alias though. A type alias feels like a more appropriate language construct for plainly defining what the shape of an object should look like and is more akin to functional-style programming, so it feels more at home with React given how React itself is a functional stab at designing user interfaces. An interface, on the other hand, has a lot of object-oriented baggage associated with it that is irrelevant when we specifically talk about React component props, and object-oriented programming is not React's paradigm.

Also, as you could see from the previous examples, types declarations are almost always more concise than their interface counterparts because of their syntax, and they can also be more composable thanks to the possibility of unions. If the prop object you are typing is really small you can get away with inlining it in the function declaration too, which you wouldn't be able to do if you are strictly sticking to interfaces.

Cool, does this mean I would always use a type for props, rather than interfaces? If you go and explore the type definition files for the most popular React re-usable component libraries you will notice that most of them use interfaces for props instead, so you could conclude that this is the globally accepted community approach for some reason.

When it comes to re-usable libraries, using interfaces instead is a very good and pragmatic choice as it allows the library itself to be more flexible because now each consumer can re-declare each of those interfaces as needed to add properties. This is useful because many OSS libraries separately maintain their type definitions from their sources so it's common for those definitions to get out of date, and when they do, users can easily get around it by leveraging the interfaces, and maintainers themselves are not bombarded with compilation related issue reports from the community.

But let's imagine a different scenario now. Imagine you are working in a multi-team company where many different teams work independently in their own front-end apps, but all depend on a private/internal re-usable component library that your team owns but everybody else contributes to. By nature, humans will always strive to find the path of least resistance to their goals. If you decided to use interfaces because of the reasons I stated above, it's very likely that when another team encounters a typing inconsistency issue they decide to quickly fix it in their codebases by leveraging the flexibility of extension points rather than contributing a fix upstream, and further fragmenting the consistency of the development experience across the company as a result.

In this case, I want to avoid providing too much extension or flexibility, and an interface's characteristics would be harmful.

Conclusion

So, what's my definitive answer? Type aliases or interfaces? My answer is: "I don't care" and "it depends".

Both types and interfaces are almost the same, and their fundamental differences are not that relevant for the super-simplistic case of React component props. Use whichever one you feel comfortable with unless there's a specific valid reason to use one over the other, like in the examples I laid out above.

The only thing I ask of you is that you don't mislead others into thinking that "you should always use types for React props" or "React props should always be declared with interfaces" and that one or the other is a "best practice" or "anti-pattern". All "best practices" are best practices because of several reasons that are situational and conditional and may not apply to all cases. Based on my experience, many engineers won't be brave or confident enough to challenge those assumptions and will go on with their lives living a lie that can affect their careers.

If you take away anything from this blog post is this:

Always challenge preconceived notions, assumptions and established "best practices".
Don't forget the reasoning behind best practices. And if you do, look them up before you use it in an argument or make a decision based on it.
If the line that divides many options is too blurry, the factors involved are too difficult to spot or are very trivial, don't waste the youth of your brain and go with whichever.

Accessibility will make your code better

Ronald Rey — Sun, 20 Oct 2019 23:23:21 +0000

(Photo by Bundo Kim on Unsplash)

Before diving into this post, you should know what accessibility is. A good place to start could be this "What is accessibility" article by MDN.

Usually the biggest and most common selling point that I see for writing accessible web applications is, in short, making your apps usable to users that rely on assistive technologies. That short statement alone can be split into multiple other very elaborate reasons, like the ones you'll see listed in the article I linked above. They are all true, but they all revolve around the user-facing benefits of accessibility, and this will be the case as well with most other documentation you stumble upon online.

This can be a problem because my professional experience shows that most companies and enterprises will bail out of investing in engineering efforts for accessibility claiming that the user-base percentage that will actually need it is too small to justify the expense. They will probably not use those harsh words though, or probably don't even address the issue in the first place. And you know what, although it might sound cruel, it can make total business sense in many scenarios, i.e. a software that is only used internally, and the company is 100% certain that none of its employees are handicapped in some way, therefore, will not need it.

However, despite this, I've always tried to write my code as accessible and semantic as possible within the budget my teams are allowed, as I feel it is my ethical duty as a professional of the web to not only deliver the highest quality code to my employers, but also the highest quality apps to its users. I like to think of it as an unofficial unspoken oath that I've made, similar to those doctors do on movies and T.V. shows, if you know what I mean.

In doing so, I've noticed certain unexpected developer facing benefits that are hardly ever discussed and might shift the mentality of development teams and drive them to do the same. Let's go through some examples to illustrate my point.

Case One

In many teams and OSS projects that I've worked on I see this style of UI tests, or similar:



const submitBtn = document.querySelector('.btn-primary')
Simulate.click(submitBtn)
expect(submitBtn.classList).toInclude('btn-pimrary__disabled')
expect(submitBtn.classList).toInclude('btn-pimrary__loading')

// ...

In short, using CSS class names or selectors to find elements and write the assertions of the tests. To some of you reading it might be obvious that this is an anti-pattern and not the best of practices, but I assure you, it is not so obvious to everyone. Just this week alone I changed a class name that broke a multitude of tests unnecessarily that I later wasted the rest of my day fixing, that incident alone was enough motivation for me to write this post.

The HTML standard is rich enough that you can do all of this and more, more semantically and resiliently, without relying on any style related attributes or rules at all? Heck, if you are using a CSS-in-JS solution or similar that scrambles your class names this might not even be possible to you in the first place, and in that case people fall back to rely in implementation details of their UI components to achieve the same thing, which is also a bad practice.

Let's look at my proposed alternative:



const submitBtn = getByText('Submit')
Simulate.click(submitBtn)
expect(submitBtn.hasAttribute('disabled')).toBe(true)
expect(submitBtn.hasAttribute('aria-busy')).toBe(true)

With WAI-ARIA and regular HTML attributes, you can represent almost any possible (if not all) state that your elements can be in, including active/inactive tabs, expanded/collapsed panels, loading/ready elements, disabled/enabled inputs or buttons, valid/invalid forms, visibility... you name it. You will not only be making your tests much more easier to write, but also much more robust, readable and semantic, not to mention that you would be making your app more accessible in the process, it's a win-win scenario in my book. I'm usually hesitant to talk about "readability" because I've noticed that it is hugely sensitive and subjective, but I think I'm confident with using it in this case. Click here for a full list of state related ARIA attributes.

If you use Jest and the Testing Library suite of testing tools, you can get even higher quality tests by updating the above to:



const submitBtn = getByText('Submit')
Simulate.click(submitBtn)
expect(submitBtn).toBeDisabled()
expect(submitBtn).toHaveAttribute('aria-busy', 'true')

And if your assertions fail, you'll get errors like:



Received element is not disabled:
 <button>Submit</button>

and



Expected the element to have attribute:  
  aria-busy="true"  
Received:  
  aria-busy="false"

Which I think we can all agree is better than just Expected false to be true.

Case Two

Let's say you have to implement a table with checkboxes that looked like this:

The checkboxes are kind of "floating" around in this table with no immediate indication as to what might be their purpose. By looking at the entire picture though, you can probably infer that each checkbox value is associated with the combination of the column and the row names. Just for the sake of an example, let's say we replace the column names with days of the week, let's go with Monday, Wednesday and Friday, and the rows with activities or chores, if we see a checkbox checked in the "Wednesday" and "Mow lawn" intersection, we could say that that is an activity that either has to be done on that day, or was done on that day.

But what if you had to only rely on the contents of the markup to figure that out, without seeing any layout? Regardless of whether this is a good design and representation for that type of data or not, let's use it for this exercise. Minimalistically speaking this could be the HTML behind it:



<table>
  <thead>
    <tr>
      <th></th>
      <th>Col1</th>
      <th>Col2</th>
      <th>Col3</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Row1</td>
      <td><input type="checkbox" /></td>
      <td><input type="checkbox" /></td>
      <td><input type="checkbox" /></td>
    </tr>
    <!-- Row2, Row3... -->
  </tbody>
</table>

Would you be able to figure out the purposes of this table and the checkboxes from that markup as quickly and easily? What if you are a developer coming into this screen for the first time to fix a bug, and maybe you are looking at this markup directly in the code or in a failed test, would it be immediately obvious to you how this UI component works? In a real scenario this could be a table rendering several dozen columns and rows, and have a lot of added markup for styling, making it even more difficult to inspect. As a side note, although we already stated that this is not a user-facing oriented post, imagine being a blind user relying on a screen reader to decipher this UI... it wouldn't go smoothly, to say the least.

We can improve this greatly by just adding:



<table>
  <thead>
    <tr>
      <th></th>
      <th>Col1</th>
      <th>Col2</th>
      <th>Col3</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Row1</td>
      <td><input type="checkbox" aria-label="Col1 + Row1" /></td>
      <td><input type="checkbox" aria-label="Col2 + Row1" /></td>
      <td><input type="checkbox" aria-label="Col3 + Row1" /></td>
    </tr>
    <!-- Row2, Row3... -->
  </tbody>
</table>

Feel free to format or phrase the label anyway you like, but now, in a huge wall of HTML it is perfectly clear what the purpose of the checkbox is, without the need to see the visual layout of the elements. This small detail can save a developer a lot of time and headaches when working with this component in the future, debugging an issue or adding new functionality.

What about writing tests?



const checkbox = getByLabelText('Col2 + Row1') as HTMLInputElement
expect(checkbox.checked).toBe(true)

Without this label, you would have to rely on very flaky CSS selectors that would leak implementation details of your component into your tests, and end up breaking with the smallest change to the markup, when refactoring or changing only styling. Not going to bother providing a snippet for how that would look like since there could be a million ways to do it, and they would all be bad.

You can go one step further to improve these inputs by also providing a tooltip of some form to the input element. A quick-fix there would be reaching out for the title attribute as well and mirroring the value of the label in it. Keep in mind though that title attributes have certain limitations that are clearly described in this article by Heydon Pickering: Tooltips & Toggletips. Or check out Reach UI's Tooltip component, even if you're not using React, you can learn a lot from its implementation if you'd like to roll out your own. You will notice it's not trivial.

Final Thoughts

Although it may not seem like much, this approach evolves into robust and readable tests that serve as not only as bug barriers but more importantly as easy to digest coded documentation on how the components work in a way that other types of test don't, which greatly increases the productivity of developers on the team. The ones that are going to notice the most are unfamiliar developers coming into sections of the codebase and quickly being able to get up to speed.

This is extremely valuable in companies with dozens of developers that contribute across the whole platform. And that's without mentioning the implementation code itself, that will be a clearer reflection of the intent of the developer who wrote it.

DEV Community: Ronald Rey

[Boost]

Dealing with open database transactions in Prisma

Ronald Rey ・ Feb 23

Dealing with open database transactions in Prisma

The error

Our problem

The solution

Performance

What about Drizzle?

Conclusion

How we built a better search experience

New Functionality

Improving user search performance

Typeahead component

Blocking the Main Thread

Smart Search logic

End Result

Conclusion

Why I avoid `get`-like functions in JS

The root of the problem

What I recommend

My Favorite Web DevTools Features of 2019

Good New Features

Chrome: Visualize layout shifts

Firefox: CSS Flexbox Inspector

Firefox: Accessibility-related tooling

Chrome: Code Folding in the Sources and Network tab

Firefox: Multi-line mode console

Great New Features

Chrome: Highlight all nodes affected by CSS property

Firefox: Inactive CSS

Fantastic New Features

Firefox: New "Changes" tab for CSS

Chrome: Inline breakpoints in the breakpoint pane

Chrome: Detailed tooltips in Inspect Mode

Chrome: Autocomplete with CSS values

Chrome: Logpoints

Conclusion

My Favorite Web Features of 2019

Honorable Mentions

KV Storage

CSS Properties and Values API

The rendersubtree attribute

Largest Contentful Paint (LCP)

Several utility functions added to JavaScript

Public & private class fields in JavaScript

Really happy about

backdrop-filter CSS property

Compute img/video aspect ratio from width and height HTML attributes

Desktop PWAs

My absolute favorites

prefers-color-scheme media query

Hardware Media Keys support

Additions and improvements to the Intl API

Native Lazy Loading

Conclusion

TypeScript Types or Interfaces for React component props

Type aliases vs. Interfaces

Regular object with properties

Arrays or indexers

Functions

Function overloading with added properties

Merging

Class implementation

Union types

Redeclaration

Using in React props

Conclusion

Accessibility will make your code better

Case One

Case Two

Final Thoughts

Links

The `rendersubtree` attribute

`backdrop-filter` CSS property

`prefers-color-scheme` media query

Additions and improvements to the `Intl` API