DEV Community: rem

A guide to frontend migrations

rem — Sun, 17 Jul 2022 00:00:00 +0000

Introduction

The frontend world innovates fast. In the new wave of React state management, we saw how new approaches to a problem space let us tackle common pain points, in new ways that make our life easier.

Despite the cycle of new tools and patterns becoming available. Working with "legacy" technologies and patterns is the day-to-day norm as developers.

The adoption of new tools and coding patterns is gradual for larger projects.

For projects that have been around a while, it's common to have collected multiple overlapping approaches, and tools, that have been introduced over time.

When we start collecting more than one tool or pattern to solve the one specific problem, complexity starts to multiply.

Performance also suffers because we require users to download and run code that solves the same problem multiple times unnecessarily.

Actively managing these migrations between tools, patterns, and frameworks, is key to maintaining simple and fast frontends at scale.

So what are the proven best practices for migrating between frontend tools and frameworks?

This guide will first go over the strategies for tackling frontend framework migrations. We'll then flesh out the principles for managing the adoption of new libaries and tools in our frontends.

The types of frontend migrations

If you stay in the frontend world long enough, it's likely at some point you'll work on a migration type project. Let's go over the two most common ones.

Framework migrations

These are a large, macro style migration and often the most complex. Some of the ways one may encounter these:

Through acquisitions where the company that got acquired has a different legacy frontend tech stack, and we now need to integrate it in our existing frontend.
A legacy application needs to be modernized. Sometimes a legacy application that hasn't been touched in years, suddenly needs a set of new features. That now makes sense to build in the new stack. Or the experience needs to align with existing products.
Scaling issues can crop up. Organizational ones, like hiring developers. It also includes performance and reliability issues. This occurs when working on a legacy code base that has turned into spaghetti, that relies on a few people who understand it. We'll touch on the topic of rewrites in a bit.

Library and tool migrations

These come when we introduce a new tool or pattern that makes our job easier in some way. For smaller projects, quick migrations tend to be easily manageable.

For larger projects, we need strategies. Because the adoption often has second order knock on effects, like introducing new concepts, coding patterns, or what folder structures now make sense with the new approach.

Here are some common examples of when adoption makes sense:

Solves the problem in a simpler way. Simple here means it solves the specific problem without any extraneous kilobytes or concepts. A lot of times we adopt tools that are much more powerful than what the problem demands.
Solves the problem more performantly. One example here is date formatting. A common frontend problem. Solved with moment at 290kb and date-fns at 89kb.
Has wider community support. Sometimes the new approach may not necessarily solve it better than the previous one. But has a wider community around it.

Making it easier to hire, find answers to common issues, and receive future upgrades. A common example is migrating from Flow to Typescript.

Migration strategies: gradual vs big bang

The analogy of waterfall versus agile is a good framing to understand the trade-offs between big bang migrations and incremental ones.

Waterfall methods can work well for projects that are relatively predictable (like the Flow to Typescript example above).

Big bang approaches batch up the changes, and update things in a single fell swoop. This means the change is "atomic" in the sense that the codebase avoids having a prolonged transitional state.

This is risky though, because we can end up going dark for lengthy periods of time. It assumes things won't change during the time we spend migrating in the dark, before pushing it out.

In contrast, an agile approach deals with unknowns through fast feedback loops and smaller iterations, that allow us to learn as we go. This is also the benefit of going gradual.

The risk here is staying in a transitional period indefinitely. Where we need to manage the split brain between the two different approaches. A major source of complexity.

In practice different priorities often come up, which can pause migration efforts.
Over time this leads to stale code no on wants to touch, that becomes risky to change.

A gradual approach is effective, but also has strong negatives if not managed proactively.

For very large code bases under active feature development, where stopping the world for a migration is not feasible. Defaulting to a gradual migration will often be the pragmatic choice.

Frontend framework migrations

Sometimes we may find ourselves needing to migrate to an entirely different framework. Or integrate a legacy framework into our existing one. The example we'll use is migrating a Backbone SPA to an existing tech stack in React.

Should you rewrite?

There’s lots of interesting debate around this. Some famously taking the position that you should never rewrite your software.

Where others disagree pointing out that sometimes a rewriting from scratch is good.

We won't go into the debate here. But here are some common pitfalls to be aware of when migrating to a new frontend framework:

Not considering potential trade-offs. We might adopt a new framework thinking all our problems will be solved. Unfortunately there's no guarantee the second time will be better than the first.

This is because first draft of anything is usually bad. Often times it takes a while to find the patterns for your app when adopting a new framework.

And these may change during the migration process (think class based components to functions with hooks). Leaving you in multiple states of transition at once.

Under-estimating the challenge. A big part of underestimating the challenge is knowing that it's been done before.

Legacy code has lots of small bug fixes that developers have accumulated over the years. Maybe without tests, comments, or any context.

That tribal knowledge gets lost to the sands of time. It's easy to miss the edge cases when starting in anew without understanding how the old way works.

Making updates to the UX at the same time. Scope creep is common in these migrations. This contributes to prolonged timelines because we end up replacing the engine while the plane is still flying, and also building new features at the same time.
Not fully committing. Large frontend migrations take time. Sometimes they are multi-year projects. It's difficult when momentum is lost, and other priorities take place. As long as we have at least one user on the old stack, it means we can't delete the old code, and need to maintain a split brain between the old and new.

Progressive strategies for migrating frameworks

Let's take a look at two common progressive migration strategies:

Migrating "outside in"

This is a top down approach. We can think about this as an adaption of the strangler pattern, often used with backend service migrations.

The idea here is that we first migrate the application's shell to the new stack. And migrate the legacy pages one by one at the route level.

The "shell" in this example is referring to the container that handles the rendering of pages.

Which means the router, and top level page layout components. It can also extend to include the shared data layer (if applicable) and things like analytics and monitoring etc.

Let's take an example migrating our legacy Backbone SPA to React.

Assuming we have our new (or existing) React application set up. We now need to create an abstraction that allows us to render Backbone pages as React components.

Here's some pseudoish code on what this might look like:

  const LegacyBackbonePage = ({ pageKey }) => {
    // run any effects that may be needed
    // e.g subscribe to events the legacy can send to the React world
    // ..
    // can also render any common infrastructure components and set up analytics etc
    // ..
    // render element container the Backbone SPA takes over when rendered on the page
    // in the legacy Backbone app this is usually the <body>
    return <div id="legacy-backbone-root" />
  }

From the outside looking in this is just a React component. The component does some set-up and renders an element container for the Backbone page to render into.

To keep everything together, it's often easier to copy the legacy code into a directory in the same repo as the React repo.

Getting set up is the easy part. If you're lucky things may "just work" depending on how the legacy pages work. But probably not.

The hard part of legacy migrations like this, is once the legacy code has taken over that element and rendered itself, there's usually a series of weird bugs to address one by one.

A common challenge is intermediating the client side routers if they conflict. In addition to managing any global styles or scripts that have side effects, that run as a result of loading the legacy code into the React SPA.

We'll hand-wave over these hairy bits for now.

This approach allows you to start getting the benefits of the new stack once the shell has been migrated. And creates clear boundaries between legacy and new.

It allows you to release newly migrated pages progressively at the route level which can be controlled via feature flags.

Migrating "inside out"

This is a bottom up approach. Where the apps shell and entry point stays the same, where legacy pieces are replaced piecemeal from the "inside out".

Going back to our Backbone to React example. This would involve creating a way for Backbone code to create new react roots and render them into sections on the page. That React can then take over.

// .. other imports
// import our React component into the Backbone world
const ComponentWrittenInReact = require('/path/to/cool-new-component')
// ...
// a bunch of legacy Backbone app code 
// ...
// render the new component into an element container
ReactDOM.render(
  // no JSX so we use the imperative API
  React.createElement(
    ComponentWrittenInReact,
    // component props and handler functions passed from legacy
    { someData, someCallback } 
  ),
  containerElement // the root HTML element React will render into and take over
)

Here we pass the initial data and function references to methods the React component can invoke.

In practice orchestrating communication between components in the legacy world and the new world is a challenge.

One simple approach is using custom events, which keeps things decoupled. But doesn't scale that well, and can be a pain to debug.

What approach is best to take?

Outside in is the often better approach if we know for certain we want to migrate to the new framework. It requires us to bite the bullet upfront, and set up the initial shell and infrastructure. Which may not always be feasible.

However this pays off once done, because you can reap the benefits of the new framework early. Which builds strong momentum. And you have clear boundaries at the route level between what is old and what is new.

Inside out is good for exploring whether or not you want to commit to a full migration. It's easier to get started with because you don’t have to touch any of the surrounding shell or infrastructure like routing. It allows replacing sections piecemeal to experiment. In addition to going route by route.

You can still switch to a top-down, outside-in model once you decide that the framework is worth investing in further.

Testing and releasing incrementally

In either case, testing can be tricky. It's often effective to rely on end to end tests to ensure the pages or components load correctly, and are interactable on a real instance.

Using a feature flag tool also allows us to gate newly migrated pieces that can be turned on and off during runtime. This allows for progressively releasing migrated pieces to avoid going dark for long periods of time.

Frontend library and tool migration strategies

We introduce a new tool or pattern that will make our lives easier in the long term. The cost of this in the short to medium term (and often times indefinitely) is the transitional period.

Some common pitfalls in large projects:

Old patterns keep replicating. Especially as a new starters, we often look around to see how other people did something and take inspiration from those approaches. This effect often tends to multiply quickly when deprecated patterns are replicated. Extending the transition and making it harder to switch to newer approaches.
Performance degradation. As different tools and approaches accumulate for the same problem, we end up shipping that code down to users.

As a somewhat contrived extreme example. We can image an app that started with Redux, later adopted react-query with another other team using apollo-client somewhere else.

Multiply this for each problem domain and throw a framework migration in the mix, and that's how some web apps end up sending megabytes of code down the wire.

This phenomenon of so much Javascript is a main driver for the trend of pushing more back to the server, and only sending what is absolutely necessary.

Decreased velocity. When these transitional states linger, it creates stale parts of the code. This creates a reluctance to touch old portions of the code base with the increased risk of causing issues.

It also becomes harder to onboard new developer's, as tribal knowledge usually becomes necessary to understand the older parts that become untouched.

Strategies for mitigating these pain points

Break transitions into distinct phases. Often times a migration will need several intermediary stages. Knowing what stage you are in, and what comes next can help you track progress and maintain momentum with clear goals.
Get buy in on the migration plan. In the 3'ds of frontend feature leading we highlighted the fact that frontend software engineering is a team sport, and building consensus on what's important is a huge part of that.
Track progress through charts showing the percentage of remaining code required migrate can help maintain momentum. Having this clearly laid out helps assign owners who can be accountable for driving the work to eventual completion.
Investing in static code analysis helps when leveraging automated change tools like code mods. Writing code that is statically analyzable is a key part of this. This means being consistent across the board. Which helps us find and update common patterns in bulk. Some strategies to help with this:
- Avoid import aliasing - renaming imports like { Button as RenamedButton } can make things hard to find at consumption points because you lose the context.
- Dynamic behaviour such as spreading props into components can be hard to search for usages, and statically analyze.
- Rename with overly explicit names for things that deprecated and shouldn't be used anymore. See this variable name in the React code base as an example. Can also apply to package names and directories.
Lean on automation. There are regex based tools you can leverage that detect deprecated usages and automatically leave comments on pull requests.

You can have bots automatically comment to use the new pattern in instead. It helps if you have documentation around your choices that you can point to.

Custom ES lint rules Following the point above, you can write custom es-lint rules to help enforce the migration approach you are committed to.
Documentation if it actually exists, and gets updated, is very useful. Especially for new starters who join who can read up and get up to speed with the specific tools, technologies and patterns that are used.

Recap

For large frontend code bases that have been around a while, it's easy to accumulate multiple transitions between different tools, coding patterns and even frameworks.

This is an inevitable part of starting with one approach and over time introducing newer ways that attempt to make our lives easier in the longer term.

For large frontend code bases, managing these migrations is a key part in taming the complexity of the frontend architecture and avoiding performance nightmares.

Today's cool approach is tomorrow's legacy. Whenever introducing a new pattern or tool, it's useful to include a plan for both migrating to and also away from the previous way.

For larger framework migrations - it's often better to go top down, "outside in" if you know for certain of the framework you want to commit to.

A bottom-up "inside out" approach works better for exploration on whether or not you want to commit to a migration by replacing smaller sections of your app with the new framework.

There's a lot more to say on this topic than has been covered in this guide. One aspect of the micro frontend tend can potentially be useful for handling these types of larger framework migrations at scale.

References

Understanding micro frontends

rem — Sat, 16 Jul 2022 00:00:00 +0000

What are micro frontends?

Micro frontends are a set of principles for structuring large frontend applications. Originally inspired by the backend micro-services trend.

While micro frontends are a trending topic of interest, there's something about them that's hard to pin down precisely. When understanding a new trend or technology, it's often useful to start understanding the problem space first.

There are many resources online that go over the potential benefits of micro frontends that we won't cover here. Instead we'll start by understanding the problems they aim to solve. And the trade-offs they make in the pursuit of solving those problems.

What problems do micro frontends try to solve?

The simplest way to understand micro frontends is seeing them as a technical solution to organizational problems that occur at scale.

For large organizations, with many frontend teams wanting to ship things fast, we'd want those teams to be decoupled. So they can work independently without getting in each others way.

What this means in practice on the frontend:

Independently buildable frontend assets allows teams to own their own builds. This avoids lengthy build times, and prevents individual teams breaking the build for the rest of the organization. Blocking others while it gets fixed.
Independently deployable frontend assets allows decoupled these teams to ship their code to production regularly, with without needing to have complex deployment sequences.

This also means when bugs crop up, and incidents occur, things can be rolled back automatically, without disrupting other teams and having their code rolled back as well.

Clear areas of ownership is necessary at scale. In large organizations if no one owns something, it's almost like it doesn't exist. Because no one is accountable for keeping it up to date, it often becomes stale and hard to change.

These are some of the core problems micro frontends hope to address. They are legitimate problems, quite difficult to solve.

If solved correctly, it can really accelerate the development speed of teams. This might explain the hype around the potential of the micro frontend trend.

Defining micro frontends

Just as "micro services" is a kind of guiding principle in the backend world, on how to structure services, micro frontends can be seen in the same way.

As a set of guiding principles enabled by specific technologies.

There are underlying technologies like webpack module federation that make a micro frontend approach possible. And popular frameworks like single SPA that provide batteries included set up.

The following principles of micro frontends taken from micro-frontends.org are paraphrased here:

Be technology agnostic means teams can chose whatever frontend tech stack they want. The recommendation seems standardize on web components as common shared primitives.
Isolate team code means focussing on decoupling teams through independently built, self contained apps that are deployed separately.
Establish team prefixes when you have to share stuff. This means a name-spacing for custom event names and things like locale storage and cookies to avoid collisions.
Favor native browser features over custom APIs.
This often means using custom elements and web components and custom events for inter micro-frontend communication.
Build a resilient site means build your micro frontends with progressive enhancement in mind.

Some of these are somewhat open to interpretation. Which is probably by design, as micro frontends don't point to a specific universal implementation. That is partly one of the reasons why there's confusion around what micro frontends are used for.

A comparison with the islands architecture

Web frameworks like Fresh, Astro and Marko promote the "component islands pattern".

Micro frontends on the surface look similar, due to rendering a page build from independent sections of components.

But the underlying philosophies are quite different.

The guiding principle for the island architecture pattern is:

To default to server side rendering over client side rendering
Only send what is absolutely necessary to the client for interactivity.

The focus is on minimizing any unnecessary overhead of Javascript.

It's a almost a throwback to traditional multi page architectures, where Javascript was "sprinkled" on on top of server rendered HTML pages to give them a bit interactivity.

In this cycle of innovation, the "sprinkles" are instead component based islands authored using modern technologies like Preact rather than something like jQuery.

Micro frontends, on the other hand, are all about solving organizational issues, rather than performance ones. And are used in the context of large SPAs worked on by many teams.

Potential issues with micro frontends

Micro frontends have noble goals, and aim to solve real organizational issues.

However based on their philosophy outlined above, we can see some aspects that would need strict governance in practice in order to not fall into the falling pitfalls:

Potential performance pitfalls. At large scale, giving teams the freedom to chose whatever tech stack they want on the frontend has some significant trade-offs.

In the context of micro-services on the the backend. It makes sense for teams to chose their stack. Because the end user doesn't need to download and run that code.

On the frontend though, things are different. You end up forcing users to download and run code that solves all sorts of common problems like styling, fetching and mutating data, date formatting etc. Many times over, in different ways, with different tools and frameworks. That's a lot of overhead.

Can't we just code split everything? Each micro frontend can be selectively code-split on a page load. Code-splitting isn't a silver bullet. And it's often the network waterfall of components loading, then fetching data, which results in more async loading, which is the real culprit of slow page loads. Especially so on mobile, that often has more sensitivity to network latencies.

Difficult dependency management. Even if the same library is shared to solve a particular problem, at scale there is always the problem of different versions of that library needing to be kept up to date. Especially for things like high priority security updates.

Multiple duplicated transitive dependencies lead to users re-downloading the same code (in different versions) multiple times. This is not a trivial problem to manage with lots of teams, even when using a single bundler, let alone one for each independent team.

Inconsistent look and feel. Decoupled teams are often necessary at scale. However often times with completely isolated teams, it's easy to reinvent the wheel many times over. Leading the overall whole of the application feeling disjointed and janky.
Requires strong governance, oversight and guidelines. Similar to backend micro services. A key question here, is how micro should a micro frontend be?

If we break things down too much we run the risk of creating dependencies that require us to deploy multiple frontends together. Which invalidates a major problem we originally set out to solve.

These points are not to say micro frontends don't have their use cases. But their adoption should take into consideration mitigation strategies for these potential things to go wrong.

A use-case for frontend framework migrations

In a guide to managing frontend migrations we looked at the "outside in" approach to managing frontend framework migrations.

There is a good potential use case for micro frontends here when you have no choice but to integrate two different frontend tech stacks and adopt a transitional architecture.

Using a framework like single SPA as the application shell to manage the transitional architecture.

That said, there are often simpler approaches that don't require the adoption of a framework that can work as well.

Recap

We can understand the rise of micro frontends by looking at the organizational issues they aim to solve.

These are real issues, that if solved correctly, can have a great benefit to frontend teams.

If you don't have these organizational problems, then you probably don't need, or want, to adopt a micro frontend architecture.

If you do have these problems, micro frontends are worth checking out. But as we've seen they are not without significant trade-offs, often at the expense of the end user, that need to be actively managed.

It's good to keep in mind micro frontends are not the only way to solve those organizational issues, but that's beyond the scope of this article for now.

References

The new wave of React state management

rem — Thu, 30 Jun 2022 00:00:00 +0000

Introduction

As React applications grow in size and complexity, managing shared global state is a challenge. The general advice is to only reach for global state management solutions when you need it.

This post will flesh out the core problems global state management libraries need to solve.

Understanding the underlying problems will help us assess the trade-offs made the "new wave" of state management approaches. For everything else, it's often better to start local and scale up only as needed.

React itself does not provide any strong guidelines for how to solve this for shared global application state. As such the React ecosystem has collected numerous approaches and libraries to solve this problem over time.

This can make it confusing when assessing which library or pattern to adopt.

The common approach is to outsource this and use whatever is most popular. Which as we'll see was the case with the widespread adoption of Redux early on, with many applications not needing it.

By understanding the problem space state management libraries operate in, it allows us to better understand why there are so many different libraries taking different approaches.

Each makes different tradeoffs against different problems, leading to numerous variations in API's, patterns and conceptual models on how to think about state.

We'll take a look at modern approaches and patterns that can be found in libraries like Recoil, Jotai, Zustand, Valtio and how others like React tracked and React query and how fit into the ever evolving landscape.

By the end we should be more equipped to accurately assess the trade-offs libraries make when we need to chose one that makes sense for our applications needs.

The problems global state management libraries need to solve

Ability to read stored state from anywhere in the component tree. This is the most basic function of a state management library.

It allows developers to persist their state in memory, and avoid the issues prop drilling has at scale. Early on in the React ecosystem we often reached for Redux unnecessarily to solve this pain point.

In practice there are two main approaches when it comes to actually storing the state.

The first is inside the React runtime. This often means leveraging API's React provides like useState, useRef or useReducer combined with React context to propagate a shared value around. The main challenge here is optimizing re-renders correctly.

The second is outside of React's knowledge, in module state. Module state allows for singleton-like state to be stored. It's often easier to optimize re-renders through subscriptions that opt-in to re-rendering when the state changes. However because it's a single value in memory, you can't have different states for different subtrees.
Ability to write to stored state. A library should to provide an intuitive API for both reading and writing data to the store.

An intuitive API is often one that fits ones existing mental models. So this can be somewhat subjective depending on who the consumer of the library is.

Often times clashes in mental models can cause friction in adoption or increase a learning curve.
A common clashing of mental models in React is mutable versus immutable state.

React's model of UI as a function of state lends itself to concepts that rely on referential equality and immutable updates to detect when things changes so it can re-render correctly. But Javascript is a mutable language.

When using React we have to keep things like reference equality in mind. This can be a source of confusion for Javascript developers not used to functional concepts and forms part of the learning curve when using React.

Redux follows this model and requires all state updates be done that in an immutable way. There are trade-offs with choices like this, in this case a common gripe is the amount of boilerplate you have to write to make updates for those used to mutable style updates.

That's is why libraries like Immer are popular that allow developers to write mutable style code (even if under the hood updates are immutable).

There are other libraries in the new wave of "post-redux" global state management solutions such as Valtio that allow developers to use a mutable style API.
Provide mechanisms to optimize rendering. The model of UI as a function of state is both incredibly simple and productive.

However the process of reconciliation when that state changes is expensive at scale. And often leads to poor runtime performance for large apps.

With this model, a global state management library needs to both detect when to re-render when it's state gets updated, and only re-render what is necessary.

Optimizing this process is one of the biggest challenges a state management library needs to solve.

There are two main approaches often taken. The first is allowing consumers to manually optimize this process.

An example of a manual optimization would be subscribing to a piece of stored state through a selector function. Components that read state through a selector will only re-render when that specific piece of state updates.

The second is handling this automatically for consumers so they don't have to think about manual optimizations.

Valtio is another example library that use Proxy's under the hood to automatically track when things get updated and automatically manage when a component should re-render.
Provide mechanisms to optimize memory usage. For very large frontend applications, not managing memory properly can silently lead to issues at scale.

Especially if you have customers that access these large applications from lower spec devices.

Hooking into the React lifecycle to store state means it's easier to take advantage of automatic garbage collection when the component unmounts.

For libraries like Redux that promote the pattern of a single global store, you will need manage this yourself. As it'll continue to hold a reference to your data so that it won't automatically get garbage collected.

Similarly, using a state management library that stores state outside the React runtime in module state means it's not tied to any specific components and may need to be managed manually.

More problems to solve:
In addition to the foundational problems above, there are some other common problems to consider when integrating with React:

Compatibility with concurrent mode. Concurrent mode allows React to "pause" and switch priorities within a render pass. Previously this process was completely synchronous.

Introducing concurrency to anything usually introduces edge cases. For state management libraries there is the potential for two components to read different values from an external store, if the value read is changed during that render pass.

This is known as "tearing". This problem lead to the React team creating the useSyncExternalStore hook for library creators to solve this problem.
Serialization of data. It can be useful to have fully serializable state so you can save and restore application state from storage somewhere. Some libraries handle this for you while others may require additional effort on the consumers side to enable this.
The context loss problem. This is a problem for applications that mix multiple react-renderers together. For example you may have an application that utilizes both react-dom and a library like react-three-fiber. Where React can't reconcile the two separate contexts.
The stale props problem. Hooks solved a lot of issues with traditional class components. The trade off for this was a new set of problems that come with embracing closures.

One common issue is data inside a closure no longer being "fresh" in the current render cycle. Leading to the data that is rendered out to the screen not being the latest value. This can be a problem when using selector functions that rely on props to calculate the state.
The zombie child problem. This refers to an old issue with Redux where child components that mount themselves first and connect to the store before the parent can cause inconsistencies if that state is updated before the parent component mounts.

A brief history of the state management ecosystem

As we've seen there's a lot of problems and edge cases global state management libraries need to take into account.

To better understand all the modern approaches to React state management. We can take a trip down memory lane to see the how the pain-points of the past have lead to lessons that we call "best practices" today.

Often times these best practices are discovered through trial and error and from find that certain solutions don't end up scaling well.

From the beginning, React's original tagline when it was first released was the "view" in Model View Controller.

It came without opinions on how to structure or manage state. This meant developers were sort of on their own when it came to dealing with the most complicated part of developing frontend applications.

Internally at Facebook a pattern was used called "Flux", that lent itself to uni-directional data flow and predictable updates that aligned with React's model of "always re-render" the world.

This pattern fitted React's mental model nicely, and caught on early in the React ecosystem.

The original rise of Redux

Redux was one of the first implementations of the Flux pattern that got widespread adoption.

It promoted the use of a single store, partly inspired by the Elm architecture, as opposed to many stores that was common with other Flux implementations.

You wouldn't get fired for choosing Redux as your state management library of choice when spinning up a new project. It also had cool demoable features like ease of implementing undo / redo functionality and time travel debugging.

The overall model was, and still is, simple and elegant. Especially compared to the previous generation of MVC style frameworks like Backbone (at scale) that had preceded the React model.

While Redux is still a great state management library that has real use cases for specific apps. Over time there were a few common gripes with Redux that surfaced that lead it to fall out of favour as we learnt more as a community:

Issues in smaller apps

For a lot of applications early on it solved the first problem. Accessing stored state from anywhere in the tree to avoid the pains of prop-drilling both data and functions to update that data down multiple levels.

It was often overkill for simple applications that fetched a few endpoints and had little interactivity.
Issues in larger apps

Over time our smaller applications grew into larger ones. And as we discovered that in practice there are many different types of state in a frontend application. Each with their own set of sub-problems.

We can count local UI state, remote server cache state, url state, and global shared state, and probably more distinct types of state.

For example with local UI state, prop drilling both data and methods to update that data often becomes a probably relatively quickly as things grow. To solve this, using component composition patterns in combination with lifting state up can get you pretty far.

For remote server cache state there are common problems like request de-duplication, retries, polling, handling mutations and the list goes on.

As applications grow Redux tends to want to suck up all the state regardless of it's type, as it promotes a single store.

This commonly lead to storing all the things in a big monolithic store. Which often times exacerbated the second problem of optimizing run-time performance.

Because Redux handles the global shared state generically, a lot of these sub problems needed to be repeatedly solved (or often times just left un-attended).

This lead to big monolithic stores holding everything between UI and remote entity state being managed in a single place.

This of course becomes very difficult to manage as things grow. Especially on teams where frontend developers need to ship fast. Where working on decoupled independent complex components becomes necessary.

The de-emphasis of Redux

As we encountered more of these pain points, over time defaulting to Redux when spinning up a new project became discouraged.

In reality a lot of web applications are CRUD (create, read, update and delete) style applications that mainly need to synchronize the frontend with remote state data.

In other words, the main problems worth spending time on is the set of remote server cache problems. These problems include how to fetch, cache and synchronize with server state.

It also includes many other problems like handling race conditions, invalidating and refetching of stale data, de-duplicating requests, retries, refetching on component re-focus, and ease in mutating remote data compared to the boilerplate usually associated with Redux.

The boilerplate for this use-case was unnecessary and overly complex. Especially so when commonly combined with middleware libraries like redux-saga and redux-observable.

This toolchain was overkill for these types of applications. Both in terms of the overhead sent down to the client for fetching and mutations but in complexity of the model being used for relatively simple operations.

The pendulum swing to simpler approaches

Along came hooks and the new context API. For a time being the pendulum swang back from heavy abstractions like Redux to utilizing native context with the new hooks APIs. This often involved simple useContext combined with useState or useReducer.

This is a fine approach for simple applications. And a lot of smaller applications can get away with this. However as things grow, this lead to two problems:

Re-inventing Redux. And often times falling into the many problems we defined before. And either not solving them, or solving them poorly compared to a library dedicated to solving those specific edge cases. Leading many feeling the need to the promote the idea that React context has nothing to do with state management.
Optimizing runtime performance. The other core problem is optimizing re-renders. Which can be difficult to get right as things scale when using native context.

It's worth noting modern user-land libraries such as useContextSelector designed to help with this problem. With the React team starting to look at addressing this pain point automatically in the future as part of React.

The rise of purpose built libraries to solve the remote state management problem

For most web applications that are CRUD style applications, local state combined with a dedicated remote state management library can get you very far.

Some example libraries in this trend include React query, SWR, Apollo and Relay. Also in a "reformed" Redux with Redux Toolkit and RTK Query.

These are purpose built to solve the problems in the remote data problem space that often times were too cumbersome to implement solely using Redux.

While these libraries are great abstractions for single page apps. They still require a hefty overhead in terms of Javascript needed over the wire. Required for fetching and data mutation. And as a community of web builders the real cost of Javascript is becoming more fore-front of mind.

It's worth noting newer meta-frameworks like Remix address this, by providing abstractions for server-first data loading and declarative mutations that don't require downloaded a dedicated library. Extending the "UI as a function of state" concept beyond the just the client to include the backend remote state data.

The new wave of global state management libraries and patterns

For large applications there is often no avoiding needing to have shared global state that is distinct from remote server state.

The rise of bottom up patterns

We can see previous state management solutions like Redux as somewhat "top down" in their approach. That over time tends to want to suck up all the state at the top of the component tree. State lives up high in the tree, and components below pull down the state they need through selectors.

In Building future facing frontend architectures we saw the usefulness of the bottom-up view to constructing components with composition patterns.

Hooks both afford and promote the same principle of composable pieces put together to form a larger whole. With hooks we can mark a shift from monolithic state management approaches with a giant global store. Towards a bottom-up "micro" state management with an emphasis of smaller state slices consumed via hooks.

Popular libraries like Recoil and Jotai exemplify this bottom-up approach with their concepts of "atomic" state.

An atom is a minimal, but complete unit of state. They are small pieces of state that can connect together to form new derived states. That ends up forming a graph.

This model allows you to build up state incrementally bottom up. And optimizes re-renders by only invalidating atoms in the graph that have been updated.

This in contrast to having one large monolithic ball of state that you subscribe to and try to avoid unnecessary re-renders.

How modern libraries address the core problems of state management

Below is a simplified summary of the different approaches each "new wave" library takes to solve each of the core problems of state management. These are the same problems we defined at the start of the article.

Ability to read stored state from anywhere within a subtree

Library	Description	Simplified API example
React-Redux	React lifecycle	`useSelector(state => state.foo)`
Recoil	React lifecycle	`const todos = atom({ key: 'todos', default: [] }) const todoList = useRecoilValue(todos)`
Jotai	React lifecycle	`const countAtom = atom(0) const [count, setCount] = useAtom(countAtom)`
Valtio	Module state	`const state = proxy({ count: 0 }) const snap = useSnapshot(state) state.count++`

Ability to write and update stored state

Library	Update API
React-Redux	Immutable
Recoil	Immutable
Jotai	Immutable
Zustand	Immutable
Valtio	Mutable style

Runtime performance re-render optimizations

Manual optimizations often mean the creation of selector functions that subscribe to a specific piece of state. The advantage here is that consumers can have fine-grained control of how to subscribe and optimize how components that subscribe to that state will re-render. A disadvantage is that this is a manual process, that can be error prone, and one might argue requires an unnecessary overhead that shouldn't be part of the API.

Automatic optimizations is where the library optimizes this process of only re-rendering what is necessary, automatically, for you as a consumer. The advantage here of course is the ease of use, and the ability for consumers to focus on developing features without needing to worry about manual optimizations. A disadvantage of this is that as a consumer the optimization process is a black box, and without escape hatches to manually optimize some parts may feel a bit too magic.

Library	Description
React-Redux	Manual via selectors
Recoil	Semi-manual through subscriptions to atoms
Jotai	Semi-manual through subscriptions to atoms
Zustand	Manual via selectors
Valtio	Automatic via `Proxy` snapshots

Memory optimizations

Memory optimizations tend to only be issues on very large applications. A big part of this will depend on whether or not the library stores state at the module level or within the React runtime. It also depends how you structure the store.

The benefit of smaller independent stores compared to large monolithic ones is they can be garbage collected automatically when all subscribing components unmount. Whereas large monolithic stores are more prone to memory leaks without proper memory management.

Library	Description
Redux	Needs to be managed manually
Recoil	Automatic - as of v0.3.0
Jotai	Automatic - atoms are stored as keys in a `WeakMap` under the hood
Zustand	Semi-automatic - API's are available to aid in manually unsubscribing components
Valtio	Semi-automatic - Garbage collected when subscribing components unmount

Concluding thoughts

There's no right answer as to what is the best global state management library. A lot will depend on the needs of your specific application and who is building it.

Understanding the underlying unchanging problems state management libraries need to solve can help us assess both the libraries of today and the ones that will be developed in the future.

Going into depth on specific implementations is outside the scope of this article. If you're interested to dig deeper I can recommend Daishi Kato's React state management book, which is a good resource to go deeper into specific side-by-side comparisons of some of the newer libraries and approaches mentioned in this post.

References

Building future facing frontend architectures

rem — Mon, 13 Jun 2022 00:00:00 +0000

Building frontend architectures that are performant and easy to change is hard at scale.

In this guide we'll explore the main ways complexity can rapidly and silently compound in frontend projects worked on by many developers and teams.

We'll also look at effective ways to avoid getting overwhelmed in that complexity. Both before it's a problem, and after if you find yourself thinking "oh shit, how did this end up getting so complicated?" when you're tasked to add or change a feature.

Frontend architecture is a broad topic with many various aspects. This guide will focus specifically on component code structure that results in resilient frontends that can easily adapt to changes.

The examples given in this guide use React. But the underlying principles can apply to any component based framework.

We'll start from the very beginning. On how the structure of our code is influenced, even before any code is written.

The influence of common mental models

The mental models we have, how we think about things, end up influencing our decisions to a large extent.

In large codebases, it's the culmination of these many decisions being constantly made that result in it's overall structure.

When we build stuff as a team, it’s important to make explicit the models we have, and expect others to have. Because everyone usually has their own implicit ones.

That's why teams end up needing things like shared style guides and tools like prettier. So as a group, we have a shared model of how things should be consistent, what things are, and where things should go.

This makes life much easier. It allows us to avoid the descent into an unmaintainable code base over time with everyone going down their own path.

If you've experienced a project under rapid development by many developers eager to ship, you might have seen how fast things can get out of hand without proper guidelines. And how over time the frontend can get slower and slower as more code is added and runtime performance deteriorates.

In the next few sections we'll look answering the following questions:

What are the most common mental models to have when developing frontend applications using a component based model frameworks like React use?
How do they influence how we structure our components?
What trade-offs are implicit in them, that we can make explicit, that lead to the rapid rise of complexity?

Thinking in components

React is the most popular component based frontend framework. "Thinking in react" is usually the first article you read when first getting started.

It lays out the key mental models on how to think when building frontend applications "the React way". It's a good article because the advice can also apply to any component based framework.

The main principles it lays out allow you to ask the following questions, whenever you need to build a component.

What is the one responsibility of this component? Good component API design naturally follows the single responsibility principle, which is important for composition patterns. It’s easy to conflate something simple as easy. As requirements come in and change, keeping things simple is often quite hard as we’ll explore later in the guide.
What’s the absolute minimum, but complete, representation of its state? The idea is that it’s better to start with the smallest but complete source of truth for your state, that you can derive variations from.
This is flexible, simple, and avoids common data synchronization mistakes such updating one piece of state but not the other.
Where should the state live? State management is a broad topic outside the scope of this guide. But generally, if a state can be made local to a component, then it should be. The more components depend on global state internally the less reusable they become. Asking this question is useful to identify what components should depend on what state.

Some more wisdom from the article:

A component should ideally only do one thing. If it ends up growing, it should be decomposed into smaller sub components.

The principles are outlined here are simple, battle-tested, and they work for taming complexity. They form the basis for the most common mental model when creating components.

Simple doesn't mean easy though. In practice this is much easier said than done in the context of large projects with multiple teams and developers.

Successful projects often come about from sticking to basic principles well, and consistently. And not making too many costly mistakes.

This brings up two questions we'll explore.

What are the circumstances that prevent the application of these simple principles?
How can we mitigate those circumstances as best as possible?

Below we'll see why over time maintaining simplicity is not always so straight forward in practice.

Top down vs bottom up

Components are the core unit of abstraction in modern frameworks like React. There's two main ways to think about creating them. Here's what thinking in React has to say:

You can build top-down or bottom-up. That is, you can either start with building the components higher up in the hierarchy. In simpler examples, it’s usually easier to go top-down, and on larger projects, it’s easier to go bottom-up and write tests as you build.

More solid advice. At first glance it sounds simple. Like reading "single responsibility is good" it's easy to nod along and move on.

But the distinction between a top-down mental model and bottom-up one, is much more significant than it seems on the surface. When applied at scale, both modes of thinking lead to very different outcomes when one is shared widely as an implicit way of building components.

Building top down

Implied in the quote above is a trade-off between ease in making progress by taking a top down approach for simpler examples, versus a slower more scalable bottom up approach for large projects.

Top down is generally the most intuitive and straight forward approach. In my experience it's the most common mental model developers working on feature development tend to have when structuring components.

What does a top down approach look like? The common advice when given a design to build is to “draw boxes around the UI, these will become your components”.

This forms the basis for the top level components we end up creating. With this approach we often create a coarse grained component to begin with. With what seems like the right boundaries to get started with.

Let’s say we get a design for a new admin admin dashboard we need to build. We go ahead and look at the designs to see what components we'll need to make.

It has a new sidebar nav in the design. We draw a box around the sidebar, and create a story that tells the developers to create the new <SideNavigation /> component.

Following this top down approach, we may think about what props it takes, and how it renders. Let's assume we get the list of the nav items from a backend API. Following our implicit top down model, it wouldn't be surprising to see an initial design something like in the pseudo code below:

    // get list from API call somewhere up here 
    // and then transform into a list we pass to our nav component
    const navItems = [
        { label: 'Home', to: '/home' },
        { label: 'Dashboards', to: '/dashboards' },
        { label: 'Settings', to: '/settings' },
    ]
    ...
    <SideNavigation items={navItems} />

Our top down approach so far seems fairly straight forward and intuitive. Our intention is to make things easy and reusable, consumers just need to pass in the items they want rendered and our SideNavigation will handle it for them.

Some things to note that are common in top down approaches:

We started building at the top level boundary we initially identified as the component we'll need. From the box we drew around in the design.
It's a singular abstraction that handles all the things related to the side navigation bar.
It's API is often "top down" in the sense that consumers pass down the data it needs to work through the top and it handles everything under the hood.

Often times our components render data directly from a backend data source, so this fits that same model of passing the data "down" into the components to render.

For smaller projects, there's nothing necessarily wrong with this approach. For large codebases with many developers trying to ship fast, we'll see how a top down mental model gets quickly problematic at scale.

Where top down goes wrong

A top down mode of thinking tends to fix itself on a particular abstraction out of the gate to solve the immediate problem at hand.

It's intuitive. It often feels like the most straight forward approach to building components. It also often leads to APIs that optimize for initial ease of consumption.

Here’s a somewhat common scenario. You're on a team, on a project that is under rapid development. You've drawn your boxes and created the story, and now you've merged your new component. A new requirement comes along that requires you to update the side navigation component.

Here's when things can start to get hairy fast. It's a common set of circumstances which can lead to the creation of large, monolithic components.

A developer picks up the story to make the change. They arrive at the scene, ready to code. They’re in the context of the abstraction and API having already been decided.

Do they:

A - Think about whether or not this is the right abstraction. If not, undo it by actively decomposing it before doing the work outlined in their story.

B - Add an additional property. Add the new functionality behind a simple conditional that checks for that property. Write a few tests that pass the new props. It works and is tested. And as a bonus it was done fast.

As Sandy Mets puts it:

Existing code exerts a powerful influence. Its very presence argues that it is both correct and necessary. We know that code represents effort expended, and we are very motivated to preserve the value of this effort. And, unfortunately, the sad truth is that the more complicated and incomprehensible the code, i.e. the deeper the investment in creating it, the more we feel pressure to retain it (the "sunk cost fallacy")

The sunk cost fallacy exists because we are naturally more acute to avoiding loss. When you add time pressure, either from a deadline, or just simply "the story point is a 1". The odds likely are against you (or your team mates) from chosing A.

At scale it's these rapid culmination of these smaller decisions that add up quickly and start to compound the complexity of our components.

Unfortunately we've now failed one of the foundational principles outlined in “Thinking in React”. The easy thing to do, does not often lead to simplicity. And the thing that leads us to simplicity is not easy to do, compared with the alternatives.

Caveats

Again context matters here, if you’re in a rush to ship an MVP that was due yesterday, do what you have to do to keep the business or project alive. Technical debt is a trade off and situations call for taking it on.
But if you’re working on a product with many teams contributing to it, that has a long term plan, thinking about effective decomposition through continual refactoring is critically important to longevity.

Let's apply this common scenario on to our simple navigation sidebar example.

The first design change comes around. And we need to add the requirement for nav items to have icons, different size texts and for some of them to be links rather than SPA page transitions.

In practice UI holds a lot of visual state. we also want to have things like separators, opening links in a new tab, some to have selected default state, and so on and so forth.

Because we pass down the list of nav items as an array to the side bar component, for each of these new requirements, we need to add some additional properties on those objects to distinguish between the new types of nav items and their various states.

So our type for our now might look something like with type corresponding to whether its a link or a regular nav item:
{ id, to, label, icon, size, type, separator, isSelected } etc.

And then inside the <SideNavigation /> we'll have to check the type and render the nav items based on that. A small change like this is already starting to get a bit of a smell.

The problem here is top down components with APIs like this, have to respond to changes in requirements by adding to the API, and forking logic internally based on what is passed in.

From little things big things grow

A few weeks later a new feature is being requested and there is the requirement for being able to click on a nav item and to transition into a nested sub navigation under that item, with a back button to go back to main navigation list. We also want the ability for admins to be able to reorder the navigation items via drag and drop.

We now need to have the concept of nesting lists and associating sub lists with parent ones, and some items being draggable or not.

A few requirements changes and you can see how things start to get complicated.

What started as a relatively simple component with a simple API quickly grows into something else within a few quick iterations. Let's say our developer manages to get things working in time.

At this point the next developer or team who needs to use or adapt this component is dealing with a monolithic component that requires a complex configuration, that is (let's be real) most likely poorly documented if at all.

Our initial intention of "just pass down the list and the component will take care of the rest" has back fired at this point, and the component is both slow and risky to make changes to.

A common scenario at this point is considering throwing everything away and rewriting the component from scratch. Now that we understand the problems and use-cases it needs to solve from the first round of iterations.

The organic growth of monolithic components

Everything should be built top-down, except the first time.

As we've seen monolithic components are components that try to do too much. They take in too much data, or configuration options through props, manage too much state, and output too much UI.

They often start as simple components, and through the organic growth of complexity as described above, which is more common, end up doing too much over time.

What started as a simple component, within a few iterations (even within the same sprint) as you build the new features can be on the way to becoming a monolithic component.

When this happens to multiple components as teams work on the same codebase under rapid development, the frontend quickly becomes harder to change and slower end for users.

Here are some other ways monolithic components can lead to things silently imploding.

They arise through premature abstraction. There is one other subtle gotcha that leads to monolithic components. Related to some common models that get instilled early on as software developers. Particularly the adherence to DRY (don't repeat yourself).

The fact that DRY is engrained early, and we see a small amount of duplication at the sites where components are being composed. It’s easy to think "that's getting duplicated a lot, it would be good to abstract that into a single component" and we rush into a premature abstraction.

Everything's a trade-off, but it's far easier to recover from no abstraction than the wrong abstraction. And as we’ll discuss further below starting with a bottom up model allows us to arrive at those abstractions organically, allowing us to avoid creating them prematurely.
They prevent code re-use across teams. You’ll often discover another team has implemented, or is working on, something similar to
what your team needs.

In most cases it’ll do 90% of what you want, but you want some slight variation.
Or you just want to re-use a specific part of it’s functionality without having to take the whole thing on.

If it's a monolithic “all or nothing” component like our <SideNavigation /> it will be harder to leverage that existing work. Rather than taking on the risk of refactoring or decomposing someone else's package. It often becomes easier to just re-implement and fork it into the safety of your own package. Leading to multiple duplicated components all with slight variations and suffering from the same problems.
They bloat bundle sizes. How do we allow only the code that needs to be loaded, parsed and ran at the right time?

In practice there are some components that are more important to show users first. A key performance strategy for larger applications is the coordination of async loaded code in "phases" based on priority.

In addition to giving components the ability to opt-in and out of being rendered on the server (because ideally we perform the server side rendering as fast as possible only with components that will actually be seen by the user on first paint). The idea here is to defer when possible.

Monolithic components prevent these efforts from happening because you have to load everything as one big chunky component. Rather than having independent components that can be optimized and only loaded when truly needed by the user. Where consumers only pay the performance price of what they actually use.
They lead to poor runtime performance. Frameworks like React that have a simple functional model of state -> UI are incredibly productive. But the reconciliation process to see what has changed in the virtual DOM is expensive at scale. Monolithic components make it very difficult to ensure only the minimal amount of things are re-rendering when that state changes.

One of the simplest ways to achieve better rendering performance in a framework like React that as a virtual DOM is to separate the components that change from the ones that do change.

So you when state changes you only re-renders only what is strictly necessary. If you use a declarative data fetching framework like Relay, this technique becomes more and more important to prevent expensive re-rendering of sub-trees when data updates happen.

Within monolithic components and top down approaches in general, finding this split is difficult, error prone, and often leads to over use of memo().

Building bottom up

Compared to a top down approach, going bottom up is often less intuitive and can be initially slower. It leads to multiple smaller components whose APIs are reusable. Instead of big kitchen sink style components.

When you're trying to ship fast this is an unintuitive approach because not every component needs to be reusable in practice.

However creating components whose APIs could be reusable even if they aren't, generally leads to much more readable, testable, changeable and deletable component structures.

There's no one correct answer on how far things should be broken down. The key to managing this is use the single responsibility principle as a general guideline.

How is a bottom up mental model different to top down?

Going back to our example. With a bottom up approach we are still likely to create a top level <SideNavigation /> but it's how we build up to it that makes all the difference.

We identify the top level <SideNavigation /> but the difference is our work doesn't begin there.

It begins by cataloging all the underlying elements that make up the functionality of the <SideNavigation /> as a whole, and constructing those smaller pieces that can then be composed together. In this way it's slightly less intuitive when getting started.

The total complexity is distributed among many smaller single responsibility components, rather than a single monolithic component.

What does a bottom up approach look like?

Let's go back to the side navigation example. Here's an example of what the simple case might look like:

    <SideNavigation>
        <NavItem to="/home">Home</NavItem>
        <NavItem to="/settings">Settings</NavItem>
    </SideNavigation>

Nothing remarkable there in the simple case. What would the API look like to support nested groups?

    <SideNavigation>
        <Section>
            <NavItem to="/home">Home</NavItem>
            <NavItem to="/projects">Projects</NavItem>
            <Separator />
            <NavItem to="/settings">Settings</NavItem>
            <LinkItem to="/foo">Foo</NavItem>
        </Section>
        <NestedGroup>
            <NestedSection title="My projects">
                <NavItem to="/project-1">Project 1</NavItem>
                <NavItem to="/project-2">Project 2</NavItem>
                <NavItem to="/project-3">Project 3</NavItem>
                <LinkItem to="/foo.com">See documentation</LinkItem>
            </NestedSection>
        </NestedGroup>
    </SideNavigation>

The end result of a bottom up approach is intuitive. It takes more upfront effort as the complexity of the simpler API is encapsulated behind the individual components. But that's what makes it a more consumable and adaptable long term approach.

The benefits compared to our top down approach are many:

Different teams that use the component only pay for the components they actually import and use.
We can also easily code split and async load elements that are not an immediate priority for the user.
Rendering performance is better and easier to manage because the only the sub-trees that change due to an update need to re-render.
We can create and optimize individual components that have a specific responsibility within the nav. It's also more scalable from a code structure point of view, because each component can be worked on and optimized in isolation.

What's the catch?

Bottom-up is initially slower, but in the long term faster, because it’s more adaptable. You can more easily avoid hasty abstractions and instead ride the wave of changes over time until the right abstraction becomes obvious. It’s the best way to prevent the spread of monolithic components.

If it's a shared component used across the codebase like our sidebar nav, building bottom up often requires slightly more effort for the consumer side of things to assemble the pieces. But as we've seen this is a trade off worth making in large projects with many shared components.

The power of a bottom-up approach is that your model starts with the premise “what are the simple primitives I can compose together to achieve what I want” versus starting out with a particular abstraction already in mind.

"One of the most important lessons of Agile software development is the value of iteration; this holds true at all levels of software development, including architecture"

A bottom up approach allows you to iterate better in the long term.

Next let's recap some useful principles to keep in mind that make build this way easier:

Strategies for avoiding monolithic components

Balancing single responsibility vs DRY.

Thinking bottom up often means embracing composition patterns. Which often means at the points of consumption there can be some duplication.

DRY is the first thing we learn as developers and it feels good to DRY up code. But it’s often better to wait and see if it’s needed before making everything DRY.

But this approach lets you “ride the wave of complexity” as the project grows and requirements change, and allows abstract things for easier consumption at the time it makes sense to.
Inversion of control

A simple example to understand this principle is the difference between callbacks and promises.

With callbacks you won’t necessarily know where that function is going, how many times it will be called, or with what.

Promises invert the control back to the consumer so you can start composing your logic and pretend as if the value was already there.
```
    // may not know onLoaded will do with the callback we pass it
    onLoaded((stuff) => {
        doSomethingWith(stuff);
    })

    // control stays with us to start composing logic as if the
    // value was already there
    onLoaded.then(stuff => {
        doSomethingWith(stuff);
    })
```
In the context of React, we can see this achieved through component API design.

We can expose “slots” through children, or render style props that maintain the inversion of control on the consumers side.

Sometimes there is an aversion to inversion on control in this regard, because there is the feeling consumers will have to do more work. But this is both about giving up the idea you can predict future, and opting to empower consumers with flexibility.
```
    // A "top down" approach to a simple button API
    <Button isLoading={loading} />

    // with inversion of control
    // provide a slot consumers can utilize how they see fit
    <Button before={loading ? <LoadingSpinner /> : null} />
```
The second example is more both more flexible to changing requirements and more performant, because the <LoadingSpinner /> no longer needs to be a dependency inside the Button package.

You can see the subtle differences in top down versus bottom up here. In the first example we pass down data and let the component handle it. In the second example we have to do a bit more work but ultimately it's a more flexible and performant approach.

It's also interesting to note that <Button /> itself could be composed from smaller primitives under the hood. Sometimes a particular abstraction has many different sub behavioral elements underneath that can be made explicit.

For example we could break it down further into things like Pressable that apply to both buttons and things like Link components, that can combine to create things like a LinkButton. This finer grained breakdown is usually left for the domain of design system libraries, but worth keeping in mind as product focussed engineers.
Open for extension

Even when using composition patterns to build bottom up. You’ll still want to export specialized components with a consumable API, but built up from smaller primitives. For flexibility, you can also expose those smaller building blocks that make up that specialized component from your package as well.

Ideally your components do one thing. So in the case of a pre-made abstraction, consumers can take that one thing they need and wrap it to extend with their own functionality. Alternatively they can just take a few primitives that make up that existing abstraction and construct what they need.
Leveraging storybook driven development

There is usually a ton of discrete state that ends up getting managed in our components. State machine libraries are becoming increasingly popular for good reasons.

We can adopt the models behind their thinking when building out our UI components in isolation with storybook and have stories for each type of possible state the component can be in.

Doing it upfront like this can avoid you realizing that in production you forgot to implement a good error state.

It also helps to identify all the sub components that will be needed to build up to the component that you are working on.
- Is it accessible?
- What does this look like when it's loading?
- What data does it depend on?
- How does it handle errors?
- What happens when only a partial amount of data is available?
- What happens if you mount this component multiple times? In other words what kind of side effects does it have, and if it manages internal state would we expect that state to be consistent?
- How does it handle “impossible states” and the transitions between those states. E.g if it has a loading and error props what happens if they are both true? (In this example it’s probably an opportunity to rethink the component API)
- How composable is it? Thinking about its API.
- Are there any opportunities for delight here? E.g subtle animations done well.

Here are some more common situations to avoid that prevent building resilient components:

Name components based on what they actually do. Comes back to the single responsibility principle. Don't be afraid of long names if they make sense.

It’s also easy to name a component slightly more generic than what is actually does. When things are named more generically than what they actually do, it indicates to other developers that it is the abstraction that handle everything related to X.

So naturally when new requirements comes it stands out as the obvious place to do the change. Even when it might not make sense to do so.
Avoid prop names that contain implementation details. Especially so with UI style "leaf" components. As much as you can it's good to avoid adding props like isSomething where something is related to internal state or a domain specific thing. And then have that component do something different when that prop is passed in.

If you need to do this, it’s clearer if the prop name reflects what it actually does in the context of that component consuming it.

As an example, if the isSomething prop ends up controlling something like padding, the prop name should reflect that instead, rather than have the component be aware of something seemingly unrelated.
Be cautious of configuration via props. Comes back to inversion of control.

Components like <SideNavigation navItems={items} /> can work out fine if you know you’ll only have one type of child (and you know for certain this definitely won't change!) as they can also be typed safely.

But as we've seen it's a pattern that's hard to scale across different teams and developers trying to ship fast. And in practice tend to be less resilient to change and tend to grow in complexity quickly.

As you’ll often end up wanting to extend the component to have a different, or additional type of child. Which means you’ll add more stuff into those configuration options, or props, and add forking logic.

Rather than have consumers arrange and pass in objects, a more flexible approach is to export the internal child component as well, and have consumers compose and pass components.
Avoid defining components in the render method. Sometimes it might be common to have "helper" components within a component. These end up getting remounted on every render and can lead to some weird bugs.

Additionally having multiple internal renderX, renderY methods tend to be a smell. These are usually a sign a component is becoming monolithic and is a good candidate for
decomposition.

Breaking down monolithic components

If possible refactor often and early. Identifying components likely to change and actively decomposing them is a good strategy to bake into your estimates.

What do you do when you find yourself in a situation where the frontend has grown overly complex?

There's usually two options:

Rewrite things and incrementally migrate to the new component
Break down things down incrementally

Going into component refactoring strategies is outside the scope of this guide for now. But there are a bunch of existing battle-tested refactoring patterns you can utilize.

In frameworks like React, "components" are really just functions in disguise. Sp you can replace the word "function" with component in all the existing tried and true refactoring techniques.

To give a few relevant examples:

Concluding thoughts

We covered a lot of ground here. Let's recap the main takeaways from this guide.

The models we have affect the many micro-decisions we make when designing and building frontend components. Making these explicit is useful because they accumulate pretty rapidly. The accumulation of these decisions ultimately determine what becomes possible - either increasing or reducing the friction to add new features or adopt new architectures that allow us to scale further (not sure about this point or merge it below).
Going top down versus bottom up when constructing components can lead to vastly different outcomes at scale. A top down mental model is usually the most intuitive when building components. The most common model when it comes to decomposing UI, is to draw boxes around areas of functionality which then become your components. This process of functional decomposition is top down and often leads to the creation of specialized components with a particular abstraction straight away. Requirements will change. And within a few iterations it’s very easy for these components to rapidly become monolithic components.
Designing and building top down can lead to monolithic components. A codebase full of monolithic components results in an end frontend architecture that is slow and not resilient to change. Monolithic components are bad because:
- They are expensive to change and maintain.
- They are risky to change.
- It’s hard to leverage existing work across teams.
- They suffer poor performance.
- They increase the friction when adopting future facing techniques and architectures that are important to continue scaling frontends such as effective code-splitting, code-reuse across teams, loading phases, rendering performance etc.
We can avoid the creation of monolithic components
by understanding the underlying models and circumstances that often lead to the creation premature abstractions or the continued extension of them.

React lends itself more effectively to a bottom up model when designing components. This more effectively allows you to avoid premature abstractions. Such that we can "ride the wave of complexity" and abstract when the time is right. Building this way affords more possibilities for component composition patterns to be realized. Being aware of how costly monolithic components truly are, we can apply standard refactoring practices to decompose them regularly as part of everyday product development.

Frontend system design interviews - the definitive guide to approaching them

rem — Fri, 27 May 2022 00:00:00 +0000

Learn how to approach the frontend system design interview in 2022. A behind the scenes look at what big tech companies are looking for.

Introduction

Top tech companies are in desperate need of experienced frontend engineers. These companies are looking for frontenders that have knowledge in the architectural and software engineering principles, that years ago were mainly associated with backend developers.

It's becoming more common for these companies to have system design rounds focussed on the frontend domain as part of their hiring process. These types of interviews are common for both experienced and entry-level roles.

A lack of resources for frontenders

The frontend system design interview shares many similarities with the traditional backend system design interview. Specifically in how they are structured. While there many great resources that focus on the backend domain, there seems to be few resources that go into depth on how to approach frontend system design interviews. In particular, resources that are grounded in what companies are actually looking for in candidates.

The frontend world moves fast. Frameworks come and go, and patterns fall in and out of style as we learn together as a community. Despite the rapidly evolving frontend landscape, companies from startups to big tech companies are looking for a particular set of common attributes in candidates.

These are the attributes that go beyond the current frontend technology trends. And make you an effective engineer whatever tech stack you find yourself using. Understanding how you can demonstrate these attributes can go a long way when practicing and preparing for the frontend system design interview.

About this guide

This guide aims to provide a framework for the frontend system design interview, grounded in the specific attributes companies are looking for.

The first half goes into detail on what each of these attributes are, and why they are important for frontend software engineers to have. Not surprisingly these attributes transcend just the interview process, and the frontend domain. They are also ones you can continually focus on developing as you progress towards more seniority as a frontend developer.

The second half then provides the framework for approaching the frontend system design interview. And outlines how you can demonstrate each of these attributes as you move through the different stages of the interview.

By the end of this guide you will have the knowledge on how to tackle and prepare for the frontend system design interview, and be equipped with the following:

Be aware of what kind of questions to expect in frontend system design interviews.
Know what specific signals an interviewer is actively looking for, that go beyond specific technologies, and how to prove you have them throughout the interview.
Learn how to take ambiguous problem statements and break them down in to manageable chunks.
Understand the heuristics for generating good questions.
Confidence approaching your next frontend system design interview knowing you can work through it step by step in a methodical way.

The attributes interviewers are looking for are necessary for frontend engineers to have at all levels, but especially so as you reach more senior levels in your career. The framework outlined in this guide also applies to approaching ambiguous problems more generally.

With that out the way, let’s get into it and have you nailing your next frontend system design interview.

What companies are looking for behind the scenes

We're about to dig into all the attributes top tech companies are looking for when hiring frontend engineers. They are attributes companies would expect an experienced frontend engineer to have.

You might be stronger or weaker in some depending on what level you are interviewing for. But you’ll want to have these in the back of your mind when practicing, so you can communicate in such a way that shows you possess these attributes when going through the interview.

As we will see further on, the interview can be broken down into distinct stages, and in each stage you'll have a chance to show these attributes to the interviewer. These attributes also tend to overlap with each other somewhat.

These are the foundational "hard to fake" qualities of top engineers that companies naturally screen for as part of their hiring process. So let's see how they play out in the context of the frontend system design interview.

Problem solving ability

Alright captain obvious. Let's break this down concretely. What this means in practice:

Your ability to deeply understand the problem/s enough that you can clearly articulate a specific problem statement/s to be solved.
Your ability break down the problem into smaller distinct pieces. Such that they can be solved individually such that you can piece them back together to solve the larger specific problem you have articulated.

Humans usually don’t solve vague problems all at once in a big chunk. So we'll need to start with a solid understanding the problem we are solving first. So we can then begin to identify how to break it down.

We don't want to waste time creating solutions for problems that don't exist, or solutions to the wrong problem.

In the context of the frontend system design interview, knowing what questions to ask in each phase of the interview is a core part of demonstrating this attribute. Understanding the problem space in detail begins with asking the right questions. Which then allows us to break down the problem further into smaller pieces.

This also helps us know what things to optimize for when the time comes to make a difficult decision or trade off. Later on we will expand more on some guidelines on how to ask good questions.

If I had an hour to solve a problem and my life depended on it, I would use the first 55 minutes determining the proper question to ask, for once I know the proper question, I could solve the problem in less than five minutes.

Technical proficiency and knowledge

Frontend system design interviews are very open ended, and the initial question is often super vague. This is usually true for high level questions, but also true for lower level component design exercises. This is intentional because interviewers are assessing your ability to explore a problem space, as much as your ability to produce a solution.

Before going into the interview, it’s a helpful framing to understand that there are no solutions, only trade-offs within a given context. Even when arriving at a specific solution, you'll want to understand its strengths and weaknesses. The use-cases it supports, and maintain an awareness of what it doesn't currently support.

As you begin to break down the problem into separate smaller problems - the solutions to those smaller problems will in practice be solved by specific frontend patterns, technologies, protocols and techniques.

As you identify solutions to problems, what’s important here is your ability to discuss, or at the very least mention, trade-offs upfront with multiple different approaches. Your technical knowledge and proficiency in the frontend domain allows you to make informed decisions regarding patterns, technologies and various architectures. All while understanding their limitations and strengths for the given problem at hand.

Ability to drive a solution to completion

At what granularity should we break things down to? How many possible solutions should we enumerate and analyze?

While there is no "right" answer to those questions. A pragmatic answer is "enough so you can move forward confidently with the information you have available". Because there's always time constraints.

In real world projects at tech companies there's usually tight deadlines. In frontend system design interviews it's around one hour. So as you break things down and enumerate potential solutions, you’ll need to make decisions and drive the interview to a working solution. For each sub-problem, there will be multiple options with different trade-offs. In practice we move things forward by making informed decisions and committing to a course of action.

There’s a delicate balance here. Interviewers are assessing your ability to make progress through the design question without getting stuck in paralysis by analysis. But on the other hand, charging ahead with the first solution that comes to mind and not mentioning other potential options is also a red flag.

Often in real projects you’ll likely need to move forward and make decisions based on limited information. So understanding the trade-offs with a given approach is important. Knowing how the current design can evolve over time to support future requirements that aren't an immediate priority now, is even better. This leads us into the next attribute.

Adaptability

This attribute touches on both "soft skills" as well as the hard skills in designing systems and frontend components that are adaptable to change. It's your ability to respond to changes in requirements. In practice this relies on two things:

Not being dogmatic about a particular approach, pattern or technology and being open to discussion. Also feeling comfortable being challenged without getting defensive. It's helpful to view the interview as collaboration between you and the interviewer.
Understanding the trade-offs and limitations of a given approach (a common theme in this guide). Without this you are effectively blind to how the system can evolve over time. Understanding the use-cases a particular design supports is important, but for large complex systems, understanding the use-cases it doesn't currently, or can't support is just as important.

For both low level component API design and high level frontend system design, a lot can be said on ways to achieve long term adaptability. This is a topic unto itself (so stay tuned and subscribe to the newsletter for future upcoming posts on this topic 😅).

Operational awareness

This is your ability to consider all the things outside the immediate working solution. It includes things like resiliency, performance, accessibility, testing, observability, security, scale etc. in your design. And calling out any concerns in relation to these as you progress through the interview. These are useful questions to have in the back of your mind so you can proactively call things out as you go along even if you don't end up digging into them in depth.

A some questions to keep in your back pocket as you design

What can possibly go wrong here?
What are the specific failure modes?
What does the user experience when things fail?
What does the user experience when backend is slow ?
How will we know if things have gone wrong?
How will we know if what we built is successful?
How will we handle 100X more data on this screen or component?

You can probably think of many more questions whose answers are not without trade offs. There is a lot to consider here. Too much for a one hour interview to go in depth on all of them. But it’s useful to maintain an operational awareness that goes beyond the immediate solution. Your ability to think in terms of the wider context surrounding the solution, and not use just solution itself is a good way to demonstrate your experience as a frontend software engineer.

Product and design sense

Frontend engineering is a highly collaborative role. We work closely with designers, product managers, and backend engineers. Being able to view a particular problem through the lens of each of these disciplines is empowering. It allows you to view a problem space with many different lenses. And avoids the "if all you have is a hammer, everything looks like a nail" issue highly specialized people tend to have.

Having general knowledge around UX trends and patterns is also useful too because they also open up different ways of solving a particular problem. It goes a long way if you can proactively identify potentially poor user experiences, bottle-necks, or issues with the UI design.

The frontend applications we are building end up being expressed in a particular style of common user interface elements. Having knowledge of general UI patterns and their trade-offs from a UX perspective can help you think of different solutions and weigh them up.

As one simple example, if you’re asked a scaling question, such as how would we handle 1000X of whatever it is we are rendering. One way of thinking about this is to only show what is relevant to the user, and to defer and hide what is not immediately relevant. There are many established patterns you can utilize whenever we can’t fit something into the screen at once.

Some example UX patterns for fitting more stuff into the screen

virtualized list
paginated tables
tabs
scroll view
drawer
master-detail pattern
full page navigation
modal
combo box
collapsible section

The more broad your knowledge is in this area the more likely you can think up of solutions to problems. Or even better, avoid whole classes of problems by taking a different approach to the end user experience if possible.

In practice this attribute is weighted less in frontend system design interviews compared to the previous ones. But nonetheless one to keep in mind due to the highly collaborative nature of frontend engineering.

Things to avoid

We now have a good grasp of the different attributes to focus on demonstrating concretely. On the flip side, these are red flags you definitely want to avoid during the interview.

It’s good to be aware of them so you can course correct if you catch yourself unintentionally exhibiting any of these, either from nervousness from the pressure of the interview or lack of real world experience to draw from if you are going for a junior role.

Jumping straight into solution mode without probing clarifying questions

This is probably the most common mistake candidates make for both coding and system design interviews.

Not being collaborative

Keep in mind interviewers are assessing your technical communication and collaboration skills in conjunction with your technical proficiency.

In frontend system design interviews this is weighted more than in coding interviews. The last thing you want to do is be dismissive of a question or challenge from the interviewer without providing technical reasoning.

While it’s a collaboration, you’re in the drivers seat, and its your job to drive the interview forward towards a reasonable solution. But at the same time you need to be receptive and open to feedback, as they will likely give you helpful hints along the way.

Being all over the place

Without the right framework on how to approach system design interviews it can be common to jump from one thing to do next in an unstructured way. This is about sticking to one thing at a time and move through each step in a methodical manner.

Later on in this guide, we’ll go over how to structure your overall approach the the interview, and how to break the interview itself down into sections to work through. And how much time you should be roughly spending on each section. This provides structure to the interview and can help you avoid this.

Don’t bullshit if you're not sure

Sometimes you might simply be stumped by a question. If you don’t have knowledge of a particular area, it's better to be open about your lack of experience, rather than trying to pretend you do.

If you get caught in this situation, showing vulnerability is better than pretending you have knowledge in an area where you don't. It shows humility, and shows you are open to trusting others opinions which is critical for working in teams.

Not playing to your strengths

The frontend domain has many sub-specializations within it. You can go incredibly deep on any number of topics. With only around an hour in the interview, you want to try and show case your technical proficiency and knowledge in areas where you are strongest. Unless the interviewer explicitly asked to go deeper into a particular topic it's better to direct the conversation to areas you are most comfortable.

For example it would be challenging to do a deep dive and talk through the different problems you might face when building a responsive site that also needs to support left-to-right, and right-to-left languages if you’ve never worked on an app that requires mobile responsiveness or internationalization.

What to expect in the interview

Now we have a good idea of what interviewers are looking for when running these interviews and what to avoid. Let's switch our attention to the interview itself.

Most frontend system design interviews will be centered around either a high level design question, like “how would you go about building instagram?”, or a lower level design question like “how would you go about designing and infinitely scrolling newsfeed?”

Generally the format for a one hour interview will look roughly something like:

Initial intro and question ~ 5 minutes
Understanding the problem and requirements gathering ~ 5 minutes
The design phase ~ 25 minutes
Optimizations, adaptability, deep dive questions. ~ 20 minutes
Ad-hoc questions. ~ 5 minutes

As you move through the stages of the interview outlined below, you’ll want to be aware of both the amount of time being spent in one section and the concrete design artifacts you need to create for each section.

Creating design artifacts and building momentum

The main tool used in frontend system design interviews is the whiteboard. Utilizing it effectively means you’ll want to be able to show a clear thought process by taking what’s in your head and laying it out in structured sections.

If the input to a frontend system design interview is a vague question. The output is a clearly defined set of requirements, prioritized use cases, a high level plan, and design on the whiteboard. It's also a shared understanding between you and the interviewer of what the overall solution is along with its pros and cons.

This ability is an important skill to have when working on a team when designing features. Providing this structure for the team allows others to chime in with ideas, versus just having it in your head. If you’re a technical lead on a team providing this space for input is essential.

At the end of the interview, ideally you could hand over the whiteboard to someone else and they would have a high level understanding of what is required and a good idea the solution explored. Creating design artifacts on the whiteboard as you move through each section of the interview will help you build momentum and confidence throughout the interview.

Next up we'll dig into what the distinct stages are in the frontend system design interview. Along with what tangible design artifacts we should be focussed on creating in each stage.

The interview format

General problem statement and overview

The initial introduction to the interview usually goes for a few minutes, where you will introduce yourselves and the interviewer will give you your question.

Your goal here is to listen closely to understand the the problem being presented. The question will either be a high level design question, or a lower level design of a frontend component. It's likely the the question will relate some how to the product the company builds. Or a specific highly used component within the product. So doing some initial practice runs using the company's product you are applying for can help get you in the right frame of mind before the interview.

Examples of high level design questions

“How would you go about designing a chat application like Slack?”
“How would you build a photo sharing application like Instagram?”
“How would you design a rich text editor ?”

Examples of low level design questions

“How would you go about designing an infinite scrolling newsfeed?”
“How would you design a combobox / typeahead component ?”
"How would you design a loading progress bar for an API?"

Requirements gathering

Your goal during this stage of the interview is to explore the problem space with the interviewer. You want to collect a list of functional and non-functional requirements. As you dig for requirements, make sure to listen for any hints. In most cases they will deliberately leave the initial question vague. In other cases they may explicitly tell you what area to focus on.

Guidelines for gathering requirements

The best way to get a solid set of requirements is to ask good questions. The best way to generate good questions, is to start with the users of the component or system in mind.

Thinking about who the users are, how they will access the system and what they will do once they can use the system can help you get started generating many various considerations.

One way to frame it is if you are tasked with designing this feature of system as part of your job, and the interviewer is a product manager that you work with.

Starting with these questions from the end-users perspective will help you find the main use-cases to design for, and can lead you to explore a wide range of factors that will need consideration.

The example questions below are definitely not exhaustive. But hopefully give you an idea on how asking about the users can lead to identifying what the main functional and non-functional requirements will be.

Who are the expected users of this system?

For example there is a big difference if we expect the users to employees accessing an internal system from company hardware and networks. Versus a anyone with a computer and internet connection.

How will users access this system?

This is a broad question that can go down lots of different paths. What devices are they most likely to access it with? This question brings up a wide range of considerations. E.g accessibility, mobile responsive, low power CPU devices.

Once they’ve accessed the system, what are they most likely to do?

A web application may have hundreds of different features available to users. This is a good question that helps prioritize the top two or three use-cases to design for in the next phase of the interview.

What are the main use cases they will use the system for? Use this question to focus your main efforts into in the interview.

From where are they likely to access it?

Brings considerations in networking strategy and offline support. Will they be in different counties? On the subway with spotty connections? How will we handle internationalization?

How will our frontend application be delivered to users?

Seems simple on the surface but there are a number of considerations here. From how the application is deployed - bundling and build pipelines, touching on approaches to different rendering architectures.

How many expected users are there?

This opens many considerations around scale both for high level design questions and low level component API design.

With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.

How will we know users can successfully use the system as we intend?

Can explore approaches to testing, observability, analytics and error monitoring and resiliency through graceful degradation.

By just asking simple questions about the user, we have already encountered a large number of considerations.

There may not be enough time to mention all the considerations in a one hour interview let alone discuss them all in depth.

So you’ll need to scope things down, proactively call out considerations as they come up, and focus diving deep into particular areas where you are strongest.

Recapping the requirements gathering phase

You'll have three main goals in this phase before moving into the design phase:

Define the functional and non-functional requirements. Start asking about the end user. This will likely generate more potential considerations than is possible to cover in the time of the interview.
Define the scope of the interview. Work with the interviewer to identify the top 2-3 core use-cases of the component or system to focus on for the design phase.
You can also optionally propose a course of action and outline of how you plan to structure the time available in the interview and what you will focus on.

The artifacts you create on the whiteboard in this phase will be a list of requirements. Mostly likely a list of bullet points, with the 2-3 main use-cases highlighted as top focus.

For specific use-cases it might be helpful to phrase them as user stories like “as a user I want to upload a photo and see it displayed on my feed”, or “as a user I want to type my search query into this box and see a list of suggestions”. These act as generic problem statements you can then start to break down.
Bullet list of non functional-requirements and considerations.
Numbered list on how you plan to approach the interview in stages. Shows your plan of attack and gives the interviewer confidence you can break down an ambiguous question and drive it towards a solution. Also provides a chance to get early feedback on what areas the interviewer may be looking to explore.

High level architecture design

You'll have the top use-cases of the system or component as you move into this phase of the interview.

There are no hard and fast rules here. But one way to think about structuring this phase of the interview is taking a top down approach. It can be helpful to start high level from the top with basic wire-frames, and then depending on how you prefer to think about things, work from a top down perspective to flesh out those details, or work from the bottom up.

Starting with basic wire-frames can help you understand the problem of the particular use-case you are solving and get on the same page as the interviewer. A good approach from there is identifying the abstract data entities that power the specific use-case.

The design artifacts you’ll create for both high and low level designs typically include but not limited to:

High level wire-frame to define the problem and to confirm you on the same page as the interviewer.
Data entities - e.g pseudo code of type definitions for data entities.
Break down of wire-frame into individual components, identifying their hierarchy and composition.
The higher level APIs those components rely on.
Both user and system initiated events use that interact with the data entities that power the functionality of the use-case.

Guidelines for high level designs

As you being to design solutions to use-cases it’s helpful to be constantly asking - “what is the user trying to do here?” to help frame your solution. Asking this helps prevent creating solutions to problems that aren’t relevant and helps you stay focussed on driving a solution forward that keeps things simple.

As you practice on real examples it’s good to keep things simple. No need to over-complicate things. The majority of times solutions won’t be ground breaking new designs. A “good enough” solution is one where it solves the specific use-case in front of you, and you have analyzed it’s trade-offs and called out any potential limitations. This is much better than an over-engineered solution that attempts to solve everything at once.

It can be helpful to apply different lenses to look at the problem from different view points. Especially if you get stuck or not sure how to move things forward. Switching lenses can help you enumerate different potential solutions and considerations that apply to your solution.

The users viewpoint

Thinking in terms of how the user will actually use the system, and being able to visualize that, can help uncover edge cases and different user experiences you may need to take into consideration.

For example when navigating away from an infinite scrolling list, when the user comes back would they expect the scroll position to be in the same spot of the list when they went back?

Thinking about the small details like this from the users first person experience can help uncover implementation details you'd need for a high quality solution. While you are not expected to solve every single problem or consideration that comes up, identifying potential problems ahead of time goes a long way.

The architects viewpoint

In the frontend system design interview it's useful to work with your interviewer and make a few assumptions about your users so you know what trade-offs to make as the architect of the system. It's always useful to base things on what users actually do.

As a real world example, we may implicitly make the assumption users will be using our frontend application in long lived sessions. And so when deciding on a appropriate rendering architecture we land on a client side rendered SPA. Trading off initial page load time for a faster post loading experience.

But what if that initial assumption isn't correct. For example, perhaps the main entry point into the app is via link in an email notification that something has updated or requires action. And in practice users often do full page loads, and have relatively short lived sessions. In that case you might want to consider optimizing for a faster initial page load based on how the users actually use the system.

This is a random example, but remember everything is a trade-off. And what is the "right" trade-off will come back to who is using the component or system you are designing and how they actually use it.

Non-exhaustive list of some additional considerations:

Security - How do we make the app secure for end users? Where are the potential vulnerabilities?
Accessibility - How do we make the system accessible for all types of users?
Performance - How do we ensure the app is fast? How do keep it fast as more features get added and changed?
Delivery - How do we efficiently bundle and send the code down to the user?
Testability - How do we know the application doesn't have critical bugs and can be changed with confidence?
Resiliency - What are the failure modes? How can we gracefully handle errors when the unexpected happens?
Observability - How do we know the application is working as expected?

The maintainers viewpoint

As you design in this phase it's good to proactively call out how things can fail or break, or potentially be a bad user experience.

For example if you're talking about network communication, you can bring up things like retries, error states, loading skeletons for slow communication etc. How will we know if something is broken? What kinds of monitoring can we put in place to ensure our experience is working correctly and in a performant way? What does fault tolerance mean for this user interface? You don't necessarily have to go dive deep into them unless the interviewer asks, but just mentioning them as you go helps demonstrate your operational awareness.

For lower level design questions, this means thinking about how the API can be adapted over time. What happens when we want to reuse, copy or adapt specific aspects of this component with another in a separate part of the app? Does the abstraction allow for that? How easy is the component to adopt and use as a consumer?

Recapping the high level architecture phase

Your goal here is design solution for the main use-cases identified in the requirements gathering phase. To recap:

Solve the problem in chunks. With the specific use-cases acting as the entry point for further technical breakdown.
To get started, you can frame the use-case by drawing simple wire frame so you and the interviewer are on the same page.
Before committing to a particular solution you can ask yourself "what is the user trying to do here?" And "what does it look like for this use-case to be solved?" Keep it simple.
Identify the underlying entities that powers the use-case.
Identify the individual pieces and components that will utilize those entities.
Identify the higher level APIs those pieces will need, in order to make the use case possible (in practice typically CRUD interfaces).

These are just some guidelines to help you get started. The best way to feel confident about this phase is to go work through real problems you are likely to encounter. If you are interested in seeing this framework applied to real questions, stay tuned by signing up to the newsletter at the end of this guide.

Optimizations, deep dives and adaptability questions

Alrighty, we're in the final stages of the interview now. Let’s finish strong.

In this phase the interview will usually switch to a phase where the interviewer asks more probing questions. They will typically end up being how you would adapt the design to meet a new set requirements. Either functional, like “how would we go about supporting real time collaboration?” or non-functional, like “how would we go about supporting 100x more data for larger customers?”.

This is a good chance to demonstrate your technical depth on a particular subject and your adaptability. There's no hard and fast rules here. What you discuss will depend on what the interviewer wants to probe more on. But it's very likely the topic of frontend performance will come up. So you'll want a solid understanding of performance fundamentals if you lack real experience in this area, to ensure you can discus this aspect in depth.

Here are some example questions you might expect to see in this phase of the interview:

"A customer has raised a support ticket about bad performance and you are tasked to investigate, what do you do ?"

Similar to the initial question they will usually be open ended to see how you take an ambiguous problem statement and drive towards a solution. In this case they’re looking for general debugging ability and performance knowledge. Which can roughly be broken down to:

Reproducing reliably.
Narrow down. Identify the type of slowness e.g browser rendering performance or an I/O operation being slow.
Identifying the root cause.

The principles are the same, narrow down until you find the root cause, enumerate options to solve the issue, identify their pros and cons and make an informed decision on how to move forward.

"A backend API is slow, however the backend team is flat out and does not have the bandwidth to fix the problem this quarter, what can you do on the frontend to alleviate the pain of this?"

Again a good way to explore a problem space is to ask things about the user. Based on this, you could ask things like "How important is it for the user to see the latest data immediately?"

In this case you wouldn't go wrong to discuss different strategies that can be used to improve perceived performance. High lighting different strategies like pre-fetching upfront at the appropriate time, and various caching strategies.

"We now need to support real-time collaboration, what needs to change on the frontend?"

Sometimes you'll get a question that requires you to adapt your existing design to new requirements. These types of questions will also be pretty open ended. So like we did with the initial question, we'll want to identify the foundational changes required, and focus our attention on those.

In this case, switching to a real-time system would very likely change both how and what data is sent back and forth between the client and server. So we could focus our attention on the network communication and discuss different potential approaches such as web sockets, server sent events vs long polling and the pros and cons of each as a starting point.

"We have a requirement that customers need to support third party plugins - how would the design need to change in order to support this?"

Similar to the question above, you'll want to narrow things down to the core challenge posed by this new requirement and solve that first.

In this case because third party plugins would involve running other peoples untrusted code, so you wouldn't go wrong starting with how to make things secure, and how specific technologies like iframes etc can provide the properties an end solution would need.

To recap this phase of the interview

In this phase of the interview you will be asked some form of optimization question that either deep dives into a particular aspect in more detail. Or how the system can be changed to support new use-cases. Performance is a popular topic in this phase.

In the case of changes to requirements, for each requirement change, you can suggest multiple methods and enumerate their pros and con, and estimate the impact this change would have on the rest of the system. What else would need to change? What assumptions have now be broken?

Putting it all into practice

Well done if you’ve made it this far. This guide was written to help you think about the frontend system design interview at a higher level. Breaking down what tech companies are actually looking for, and how to structure your approach to the interview.

If you would like to see examples of applying this framework to a set of real world problems you can sign up to the newsletter Frontend Mastery monthly newsletter.

The 3 D's of frontend feature leading

rem — Thu, 26 May 2022 00:00:00 +0000

Learn the foundational principles of feature leading you can always rely on in the midst of any chaotic project.

Introduction

A feature lead refers to a person in a team that's responsible for the technical planning and delivery of a new feature or project. It's different to the job title "tech lead".

It’s common for individual engineers to lead projects at high growth start-ups or big tech companies. It's one of the best ways to level up and gain more experience and seniority.

Feature leads act as the go to person for everything related to the planning, implementation and delivery of that project or feature. At some point you may find yourself needing to fulfil this role within your team.

For those completely new to feature leading, it can be a source of confusion and stress. As your primary focus expands from specific coding tasks, to also being responsible for the delivery of a project.

So what do those non coding related things involve?

The goal of this post is to try and distill all those fuzzy parts. Some may call "soft skills" into a few fundamental principles.

These principles come from observations when working with other great feature leads. While trying to understand what it is specifically they do that leads a project to success. Also from my own experience acting as a frontend technical lead on a team. Despite the title, these principles transcend specific domains like the frontend and apply more generally to software projects.

The aim of this post is to arm up and coming feature leads with a framework to make sense of those fuzzy parts when leading a team to deliver a feature. In practice there is much more to be said on the topic of feature leading that is outside the scope of this post.

But these are the fundamental principles you can always rely on. Do each of these well, and you’ll have a pretty good time. On flip the side, in my experience, if i've started neglecting one of these, it usually doesn’t end up great.

So let’s dig in and understand each one in turn.

1. Discover

Product managers and designers will typically have a good idea of the value they want to ship to users by the time a project starts. This is where you come in as the team's feature lead. You'll need to discover what needs to be done at a technical level to make that a reality.

There are three main aspects to the process of discovery:

Understanding the problem you are solving. Looking at this another way, what new value will be created when everything shipped to the end user?
Understanding the current technical terrain.
Charting a clear path forward for the team on how to get there.

What discovery looks like in practice

What are the concrete tasks a feature lead will be doing when they are in the discovery phase?

Generally at the start of a project:

Researching similar prior art from other teams.
Searching for similar problems that have been solved before. You should avoid re-inventing the wheel if you can.
Reading through any existing relevant documentation if it exists.
Speaking with other relevant teams and feature leads.

Gathering the high-level details

Discussing your project with relevant subject matter experts.
Getting the high-level architectural details. And identifying potential dragons before digging in the low-level details is helpful.

Gathering the low-level details

Reading through existing code and making notes.
Doing this first without having a picture of the higher level makes it much harder.

Ball parking high level estimates

I always try to remember to add extra buffer. This helps to set correct expectations, which is critical at the beginning of a project. It’s better to under promise and over deliver

The hard parts of discovery

No plan survives first contact with the enemy"

In the early stages of a project you’ll spend a lot of time in discovery mode. But discovery is also an ongoing process throughout the lifecycle of a project. In practice unknown things will come up half-way and even towards the end of a project.

This is because in a fast paced environments things change often. Especially requirements. You’ll find yourself cycling back to this phase often, for various sub-streams of the feature. This can make it hard to stick to to detailed plans upfront.

This phase is about working towards a completed view of the project. On the one hand you need to understand the core parts, so the project can actually begin. On the other, you'll need to be okay moving things forward often with incomplete information.

You'll also want to understand the things that have the potential to go wrong or slow things down. Both at a technical level, but also at an organizational level. There will likely be many questions that come up during this phase.
For example, "does this feature depend on other teams?"

"It will be much faster to ship this in the existing tech stack, but we want to move to the new stack. How should we balance this?"

"The platform doesn't currently support X, do we build something bespoke, wait for support, or build it ourselves into the platform first?"

In practice some of these questions may be answered easily. While others may be more open ended, leading to further questions and exploration.

Keep in mind your goal here is understand the current technical reality as much as possible. This happens through asking the right questions and seeking their answers. So it's expected to have a bunch of unanswered questions in this phase. It's a good sign you are starting to develop a clearer picture on what needs to happen. And also what further discovery is needed.

Practical tips for discovery

Validate early

After you have an initial idea of your approach, you'll want to confirm you are on the right track. Organizations sometimes have architectural sparring sessions, or dedicated office hours with architects and principle engineers that you can utilize.

If there is no formal process for this, you can try reaching out to other senior engineers to bounce ideas off. This helps you get early feedback. And also spread awareness across the organization of upcoming changes and features.

Technical spiking

If you’re making a change to an existing system. It’s often super insightful to do a 1-2 day time-boxed spike to get a sense of the actual work involved.

As you hard code values, mock out data, comment out code and generally leave a trail of destruction on a spike branch. You’ll end up with a useful set of todo tasks like “TODO: need to get the data here somehow”, “TODO: we’ll need to refactor and extend this” etc.

This is a useful starting point for breaking down concrete tasks that developers can pick up and work on. Translating an idea into code tends to reveal the hidden technical decisions you need to make. That simply having a high level conceptual understanding won't reveal.

Communicating progress

Sometimes in the discovery phase it’s hard to show you are making progress. It can be useful to create tasks in whatever issue tracker you use, dedicated to investigations. This is both to show progress externally, and to also help keep track of all the investigations that often go on in parallel so you can refer to them later. It's good to time-box these to 1-2 days and provide regular updates on your findings. This helps avoid spending too much time down rabbit holes.

Things to watch out for

Breaking down tasks too early

Sometimes a developer will pick up a task, and discover that the approach outlined is much more difficult than originally thought. Or not currently possible.
This can lead to confusion and increased context switching, and if it happens often can reduce morale.

The Dunning-Kruger effect

This is a kind of cognitive bias we all have that leads to underestimating things. Or put differently, overestimating our ability to do something. We often feel certain about a number of things we don’t actually have direct experience with.

This is often true when planning work. It can bite you if you have a high level conceptual understanding of something and think “should be pretty straight forward".

The devil is always in the details:

This is often the cause for overly optimistic estimates, which can lead to setting the wrong expectations early on.
These overly optimistic estimates can also contribute to a sense of time-pressure for developers who pick up stories that have these estimates. Especially in scrum environments, there is an unspoken sense of “the story is estimated as a 1, I will just merge now and do all the extra X in a follow up later”.
This can lead to a pile of tech debt over time. The thing to watch out here is that there is also a kind of Lindy effect for stuffed chucked in the backlog. In my experience the longer something is put off in the backlog, the longer it is likely to be continued to be put off or forgotten about.
It’s also useful to understand this bias likely exists in external stakeholders. It’s the classic “why does it take 2 weeks to make that button work?”.
You don’t want to bombard stakeholders with all the technical limitations and challenges. In fact you want to shield them from this to some extent. Being able to communicate the technical landscape in a language external stakeholders can understand is a useful skill to develop on its own.

Having a clear understanding of what the team is building, and how it will built is a critical part of being a feature lead.
As part of this process you are likely to discover multiple possible approaches. Each with their own trade-offs and open ended questions that need to be answered. As a feature lead you'll want to be continually pushing to keep a forward momentum without getting blocked.

The next step is taking this context and making decisions that allow you to move things forward.

2. Decide

This is all about building consensus.

Building products is said to be a series of trade-offs. As a feature lead you are going to be the key person driving these decisions. Along with design, product managers and other teams who may be impacted by your project or feature.

In an ideal world, everything is discovered, and all decisions around the user experience and implementation details have been decided. As a team, that's a good state to be in. Assuming solid discovery, and no spanners are thrown in the mix, from there it's a straight forward matter of execution.

It's unlikely you'll get to this state early on for every aspect of the feature. So you'll want to work towards this state for the various sub-streams.

The hard parts of deciding

Things that matter most should never be at the mercy of things that matter least."
"Deciding what not to do is as important as deciding what to do

There is a bit of a balancing act here.

While you want to have clarity on what needs to happen, deciding too early means you could be building the wrong thing, the wrong way. This can result in your decisions being undone down the road, leading to re-work and loss of team morale and momentum.

This can be especially painful if you decide on a course of action that effects other teams unknowingly.

Often times it is best to defer making engineering decisions until all information is available, if it ever comes at all. There really is no silver bullet here. The best you can do is break things down and start building consensus on the smaller independent sub-streams. So you can get started, while more information is then gathered.

Often times what is proposed is not workable in the time frames given. Speed, quality and scope - you can usually pick 2. In the majority of cases picking speed and quality is the right move. This follows the principle that it is generally better to do less work to a high standard, rather than to try and do everything at once poorly. At a higher level this maps to the lean way of incremental delivery.

Sometimes you'll find yourself between two or more competing interests across teams. It’ll be your job as the feature lead to offer multiple solutions or ways forward. Ideally with high level estimates, that balance many sides. And then to ensure you’re driving those decisions to an outcome one way or another.

Your ability to come up with a good set of options comes back to knowing the why behind the feature, and understanding the customer pain you are solving. Knowing this allows you to focus on what matters most.

What driving decisions looks like in practice

Creating, updating and driving decision register documents. These are documents that capture the problem statement, propose a bunch of solutions and provide a format for building consensus in writing.
Collaborating with product to understand what is in scope and by when.
Updating and driving smaller, less impact decisions and updating decision registers.
Talking with other teams and people to engage in the decision making process and garner alignment.
Communicating the outcome of your decisions via whatever status reporting format your team uses, weekly syncs, stand-ups, async posts etc.

Practical tips

Run a project kick-off

A project kick-off is used to build initial consensus on what a successful outcome looks like. These are helpful for aligning everyone involved on the value you plan to ship and roughly by when.

It's useful to get everyone on the same page early on in this regard. Because as you make trade-offs down the line, you can optimize and generate solutions based on what the customer truly cares about in the feature being delivered. Establishing what that is as a group before work begins is very useful.

Keep track of your decisions and the reasons why they were made

It’s useful to record each decision and why it was made. This helps you own the decisions you make and move forward with confidence. People will question some decisions (not necessarily maliciously but out of curiosity). Having a decision register helps you come back to the why as time passes.

Quantify decisions if you can

By passing through the discovery phase, it's likely you would have encountered some issues that need extra work to be done. There will come a point where you will need to convince others that some action needs to be made, or avoided.

It's helpful to articulate the value something will provide whenever you put forward a proposal. Or alternatively the issues you won't have to deal with down the line if you do the work now. Strive to get metrics if you can. For example refactoring something to make it faster. How much faster? by what %, or correlate it to existing support tickets if available. Sometimes even having a ballpark estimate is useful for making a stronger case.

Things to watch out for

Not being explicit about the trade offs you're making

At some point you will need to decide between a trade off. When recording decisions it’s useful to be explicit about the trade off you are making. This serves as both a way to leave a paper trail for your work and serve as documentation for the future.

Decisions leading to rapid accumulation of tech debt

Depending on the project you’ll likely find situations where you simply have to cut-corners and introduce tech-debt due to time constraints. Avoid this if you can, but in my experience it’s quite hard to completely avoid.

In these situations it's useful to make this clear to external people "we are borrowing time from the future" so we can provide customer value faster, but at some point this will come back to bite us.

Own this as your responsibility as the person closest to the implementation. Communicate this back out to the team and relevant stakeholders, and ideally get commitment to cleaning things up in a following iteration if possible.

Having your decisions undone

This leads to re-work and inefficiency. Also loss of momentum and morale.
In practice usually this will come from not assessing the impact on other teams and other dependencies. Or from not understanding the current technical landscape in detail enough, and having to back-track out of a particular approach. When locking in decisions make sure to include every relevant party. Good discovery is key in avoiding this situation.

3. Delegate

This is all about not trying to play what is a team sport, solo.

As you discover what needs to be done and decide how you will do it, as feature lead you'll need to delegate. As you discover and decide, you’ll need to regularly download what's in your brain to the rest of the team.

Delegation is not about palming off work to others. It comes back again to consensus building. It's also about creating a sense of ownership in the rest of your team. And as much as possible having each member "act as a principle" by having clear areas of responsibility. You want to eliminate as many single points of failure in the team as possible - aka the "bus factor".

The hard parts of delegating

If you want to go fast, go alone. If you want to go far, go together.

Sometimes you might find that there aren't enough people to delegate to, depending on your team size and team composition. In these cases it’s best to speak to your manager or project sponsor to get either get more help, or reduce scope / increase the timeline.

Often times it will be faster for you to "just do it", rather than share the context and delegate to someone else. There is a trade-off to balance here, in the majority of cases it's better to share whatever is in your brain with the team and delegate as needed.

If you're new to feature leading it can sometimes be difficult to feel comfortable delegating to others. This is a pattern to identify and break as early as possible to avoid piling everything on yourself and burning out.

What delegating looks like in practice

Breaking down tasks and fleshing them out with all the necessary information

This also means delegating to your future self. Providing all the context needed and links to files in source code where you need to make changes. As with all the context switching it's so easy to lose or forget context, where you need to make the changes.

Running estimation meetings

Or any meeting where the engineers sync up to talk about the work they are going to do. This involves walking other engineers through tasks to provide technical context. So the others are on the same page and are in a position to work on them independently. It also includes regularly organizing the backlog based on priorities. And being available for ad-hoc meetings and answering questions for those taking on the work.

Practical tips for delegating

Have developers work and own small subsets of the feature

This makes development more efficient and creates a sense of ownership. Pieces are ideally small enough to not create knowledge silos.

Delegation can apply to more areas than coding

It’s not just about delegating the development work. For larger projects you’ll likely need to delegate the work related to discovery. This also applies to driving decisions to a decided state when multiple decisions are needed.

Things to watch out for

Trying to take everything on yourself

You have all the context and it often just feels like it will be quicker and easier to do it yourself. Most likely you'll just put yourself on the path towards burn out, it's also inefficient to try and do everything yourself. Remember it's a team sport.

Not shielding the team

Ideally when you download your brain to the team, it is with information that won’t later become irrelevant. As you go through the process of discovering and driving decisions, there may be a churn in technical direction.

It’s useful to be mindful of this as you share context with the team. You’ll want to strike a balance between shielding them from that versus bombarding them with details at every step of the way, which usually leads to confusion.

Over delegating

Sometimes over-delegation can happen. It’s much rarer compared to feature leads piling everything on top of themselves, but something to watch out for none the less.

The most important D

Chances are at some point you will be called upon to lead a project as a technical feature lead. To take what is just an idea, and drive that through to something concrete that provides value for other people.

When this happens, don't panic!

Remember the 3 Ds and you'll do fine.

Good luck.

DEV Community: rem

A guide to frontend migrations

Introduction

The types of frontend migrations

Framework migrations

Library and tool migrations

Migration strategies: gradual vs big bang

Frontend framework migrations

Should you rewrite?

Progressive strategies for migrating frameworks

Migrating "outside in"

Migrating "inside out"

What approach is best to take?

Testing and releasing incrementally

Frontend library and tool migration strategies

Strategies for mitigating these pain points

Recap

References

Understanding micro frontends

What are micro frontends?

What problems do micro frontends try to solve?

Defining micro frontends

The following principles of micro frontends taken from micro-frontends.org are paraphrased here:

A comparison with the islands architecture

Potential issues with micro frontends

A use-case for frontend framework migrations

Recap

References

The new wave of React state management

Introduction

The problems global state management libraries need to solve

A brief history of the state management ecosystem

The original rise of Redux

The de-emphasis of Redux

The pendulum swing to simpler approaches

The rise of purpose built libraries to solve the remote state management problem

The new wave of global state management libraries and patterns

How modern libraries address the core problems of state management

Ability to read stored state from anywhere within a subtree

Ability to write and update stored state

Runtime performance re-render optimizations

Memory optimizations

Concluding thoughts

References

Building future facing frontend architectures

The influence of common mental models

Thinking in components

Top down vs bottom up

Building top down

Where top down goes wrong

The organic growth of monolithic components

Building bottom up

Strategies for avoiding monolithic components

Breaking down monolithic components

Concluding thoughts

Related readings

Frontend system design interviews - the definitive guide to approaching them

Introduction

A lack of resources for frontenders

About this guide

What companies are looking for behind the scenes

Problem solving ability

Technical proficiency and knowledge

Ability to drive a solution to completion

Adaptability

Operational awareness

Product and design sense

Things to avoid

What to expect in the interview

Creating design artifacts and building momentum

The interview format

General problem statement and overview

Requirements gathering

Guidelines for gathering requirements

High level architecture design

Guidelines for high level designs

Optimizations, deep dives and adaptability questions

Putting it all into practice

The 3 D's of frontend feature leading

Introduction