DEV Community: metamn

On Design Systems

metamn — Mon, 29 Nov 2021 10:41:18 +0000

Design systems bring a novel approach to UI/UX design and development — to solve its inherent consistency and scalability problems.

As of today, dedicated companies — with resources — can afford their own custom design system. Others copy or innovate.

The series walks-through the problem in-depth, from the viewpoint of all stakeholders.

And proposes a solution — of recent origin — to help companies with less resources to roll their own.

1. Workflow, practices and deliverables

The first article takes a look at existing systems and defines the workflow, practices and deliverables for all stakeholders.

2. Design tools and processes

The second article examines design systems from the designer viewpoint.

Offers novel ideas — design systems as database (code), pair designing — to reduce the cost of rolling out design systems.

It presents an ideal and working use case where a rare species — a developer who designs — creates a design system, and more, in the browser.

3. Consistent user interfaces in web context

The third article examines design systems from the developer viewpoint.

It presents past and current front-end development paradigms and examines why they lead, or not, to consistent user interfaces.

And comes to the conclusion: to end fragility in front-end development — standard software development patterns and practices should replace the current hacking mindset and non-programming languages.

4. Implementing a design system in React and Typescript

A work in progress. A complete example for developers where the findings above merge into a real-life project.

On design systems — 2

metamn — Sun, 07 Nov 2021 00:00:00 +0000

Tools and processes.

TLDR;

Design systems are a paradigm shift in UX/UI design and front-end development for native and web applications, sites.

With a systematic approach to the design process, design systems produce consistent and scalable user interfaces.

Companies with resources lead the adoption. They do it by augmenting old tools and processes with extensive mind power.

Smaller companies with less resources must innovate to keep up.

The current landscape

Design systems are new, yet old.

They’ve been around since 2015 when Google rolled out Material Design. Still, only big companies can afford to have their own design system today.

The rest builds on open source, freely available component libraries and design systems by employing heavy customizations. We all know the result: similar looking websites and apps all around the internet.

If one goes further — and tries to roll out unique looking, yet consistent and scalable user interfaces for a progressive audience — hits the same obstacles as their predecessors, the big companies.

Same obstacles, less resources. For Airbnb, it took several years and iterations to come up with a solution which scales.

Design systems are hard.

They introduce a systematic approach to a domain previously ruled by human-driven, ad-hoc processes.

Before design systems

Google is a company with dozens of digital assets — email, maps, search, ads, video, and many more — across all platforms.

Before employing a design system these assets were looking more-or-less the same, in the best case, but many times completely different.

Material Design made the entire Google portfolio to look consistent, following unconditionally and perfectly the brand design guidelines.

Whatever the asset, whenever the platform might be.

What a design system does?

It offers a systematic approach to the design process to reduce the cognitive load on designers.

To understand the amount of the cognitive load required to design, let’s take for example the buttons. In a simple use case, we have 20 variations.

Aside to buttons, a design deals with dozens of other, more complex patterns and components like layout, spacing, typography, navigation, responsiveness, images, animations, accessibility and more.

A designer — to be able to reason about, modify, or extend a design — has to keep in her mental model all this information. The details, the nuances, the settings, and the states.

An impossible task even at Google level with an infinite pool of designers.

Design systems solve this challenge via the divide et impera method.

They separate design into small, independent and manageable parts / layers — style tokens, components, sections, pages, sites, apps — and enforce consistency on the entire path, across the entire portfolio.

Each layer is independent, self-contained, and acts as a single source of truth for its own domain.

Each layer builds on the previous layer — without the ability to modify it.

For example, tokens — the base layer — are atomic, and their single responsibility is to describe style via pure data.

Components build on tokens, but they don’t interfere with tokens. They use tokens for styling, but they can’t add new, custom styles. If a new style element is required — first it is added to the tokens, then the component becomes able to use it.

This separation of concerns, augmented with strong human discipline, makes consistency and scalability achievable across hundreds of tokens, components, sites and apps.

Tools and processes

The systematic approach brought by design systems left untouched the tools and processes used in creating digital assets.

A typical workflow is the same today as it was before design systems.

Designers use visual, non-programmable tools like Figma, Sketch, Adobe Illustrator etc. to create the layers of a design system.

These tools are visualization tools, not decision-making tools.

When a designer has to make a decision — to change, extend a design — again, she has to build a mental model first, even if the model is now smaller due to the design system approach.

The decision made cannot be proven correct in an objective way like we used to have in software development, or in other exact science.

Designing with mental models is error-prone. Introduces continuous uncertainty. And requires human resources to scale.

The other drawback of the visual, non-programmable design tools is the deliverables they produce.

The designer’s work needs to be further translated to the target medium by developers. This process is communication-heavy, error-prone again, and far from being real-time.

Better tools and processes

It’s easy to observe: the tools used by designers produce side effects related to scalability, communication latency and cost.

They are the bottleneck in the process of building consistent user interfaces in an affordable way, with support for companies with less resources.

What if the tools in the design process — in the same way as in the development process — would be programmable?

What if the outcome of a design process would be a program / data instead of the current visuals-to-be-translated-into-code mess?

That would solve the mental model capacity problem and reduce the design and development process to real-time.

Design systems as a database

Imagine the design system as a database.

Tokens, components, sections, pages, apps, sites — all quantifiable, thus correct. All queryable, to an arbitrarily small level, to fit comfortably into a single person’s mental model.

Databases handle billions of entries and help us reason about infinite relationships. They help us make correct decisions and ensure changes happen in real-time.

Designing with the help of databases would offer the same benefits.

It sounds like utopia — but it can be done today and requires only a minor cultural shift.

Pair designing

Pair programming is an age-old paradigm.

Two developers sit in the front of a single machine. They collaborate instantly and produce a single output.

Pair designing would be similar. A designer and a developer would sit at a single desk and produce a single output.

The designer would still use her own favourite visual tool like Figma. But the deliverable would not be official — just a temporary result.

The developer would instantly input the Figma deliverables into a design system built on code, like a database.

That would be the final, official deliverable — a living design system.

A case study

I’m fortunate enough to both have a computer science degree and design works featured in online galleries.

I’m a developer who designs. An ideal pair designer.

In 2015, I’ve rolled out my portfolio powered by a living styleguide. There were no design systems at that time — by the term coined. The rest was / is still the same.

Since then, I’m working in this novel domain.

I produce unique looking websites and apps for progressive brands with the help of a handcrafted, tailor made, built-from-scratch, completely customizable design system.

With a considerable investment yielding high rewards.

I’ve analysed other design systems; wrote articles; a mini-book / technical guide; learnt other languages and stacks to better understand the current theory and common practices; done four complete iterations on the code base; employed the result in a couple of real-life applications.

Today, in short, this custom-made design system / component library offers:

Wireframing — To sketch up the requirement specifications and get immediate feedback.
Mockuping — To show real-life-like components, helping to finalize the requirements, the information architecture and the data structure.
Inline content editing — To let editors add content at an early stage, to find out missing parts and inconsistencies in the original requirement specification.
Theming — To support design trials and errors, iterations and variations, rollbacks. In a fully responsive way, with annotated design decisions.
Living documentation and playground — Generated from code, to assure the design system driving the final product is tested, and all its parts are documented.

All these live, inside the browser.

No 3rd parties like Figma, Sketch, Storybook and co.

What you see is what you’ve already got.

Production-ready code with React, Typescript and CSS-in-JS. Statically generated pages using a headless CMS and GraphQL.

Simple, code-based tools. Real-time production process with verifiable, scalable deliverables.

In less time, cost and hustle.

On design systems — 1

metamn — Wed, 06 Jan 2021 00:00:00 +0000

Context and deliverables.

Update

This post was heavily edited on 10th of November 2021, when another take on the subject was published.

The posts are complementary.

This is a high level overview, focusing on deliverables. The second take is more technical, focusing on tools and processes.

A short history

Design systems have been around since 2013-2014.

Lonely Planet’s Rizzo was the early bird, then followed by the well-known Material UI implementing Google’s Material Design.

2020 was the year when design systems popped up every week. Adele of UXPin lists over a hundred systems and libraries—yet the list is incomplete.

Purpose and audience

Why companies create design systems? A quick analysis reveals the main goals:

Internal: To build apps for their brand and/or platform.
External: To build general purpose apps.
Foundational: To help build design systems and component libraries.

The majority builds internal systems—no wonder—design systems shine when used to create uniform looking products across a portfolio.

A handful of companies create general purpose systems. More precisely, they create an internal system and share to the public.

This generosity links to the company size: Google, IBM, Ant, Adobe can afford to open-source code worth millions:

This represents millions of dollars of investment for each company to duplicate work that many other companies are also doing. — Adobe Spectrum

Foundational systems represent a new business model—Modulz helps teams create design systems without writing code—or institutions sharing their work for the common good.

Type

The terms design system and component library are often interchanged. The difference is subtle and far-reaching.

The difference between a component library and a design system is whether or not your components have ’className’ and ’style’ props. — Mark Dalgleish

Design systems are strict. They form a complete system. They don’t allow on-the-fly customisation. Modifying a system is possible at a well-defined entry point, then changes reflect across the site automatically.

Component libraries are loose. They offer the basics and let customisation happen at any point, any time.

Design systems are more expensive to create, and easier to use later. Braid’s Playroom shows even non-devs can create with a design system.

Features

Purpose and audience defines a design system’s features.

While purpose, audience and type overlap, features differentiate and define the unique characteristics of a design system.

Material Design goes full circle. From top-bottom connects designers and their tools with developers of all platforms and technologies. From Figma to web and native apps everything is in a system.

Radix from Modulz is bare bones to that level of not offering any styling.

Spectrum from Adobe offers support for server-side rendering, virtual lists, state management and accessibility for anybody building design systems.

Common practices

This quick design systems analysis reveals common practices covering theory, technology and usability.

Theory

Theory is important. Drives design decisions. It’s good to have a solid foundation and principles enduring years.

Design systems have no common theoretical foundation yet. Everybody rolls their own, following—or not—design patterns.

The patterns listed below come compiled from the analysed systems.

Shopify Polaris and Adobe Spectrum are implementing all these patterns to a certain extent. The rest implements none, or part of them.

Single Source of Truth (SST)

Design systems collect settings under a common place. Any later change goes into this single source of truth and reflects across the system automatically.

This reduces the cognitive load to manage and extend the system.

Single Responsibility Principle (SRP)

Design systems contain dozens, hundreds of tokens and components. Wiring them into a system is not a trivial task.

Functional programming advocates the idea of composing up a system from smaller parts is the best possible when the underlying components behave predictably.

Independent components—self-contained, with stable interfaces— compose better and provide modular architecture.

The base / variant pattern (BEM)

In Building (and Re-Building) the Airbnb Design System an old pattern, BEM, emerges as a solution for scalability.

Keeping a simple base and following simple rules for extensions scales up the source code and reduces its complexity.

Technology

In contrast to theory, technology is well-defined on the design systems scene.

The majority of the solutions goes with Typescript and CSS-in-JS. All solutions embrace packaging and publishing to NPM. Half of them via a monorepo.

Testing isn’t fully satisfactory. The majority does unit testing using ts-jest and React Testing Library with questionable coverage.

Often times, Storybook complements missing unit tests and mocks integration tests with visual tests.

Storybook mocks also documentation.

Bold alone managed to come up with a good-looking Storybook for their API docs. The rest uses Storybook to complement their documentation suite, which results in a scattered user experience—two separate apps with different look-and-feel.

The big players—again—managed to come up with an in-house, integrated documentation tool reaching UX excellence.

What’s missing from the majority of solutions is generated documentation (JSDoc). Manual, handwritten documentation—no matter what—breaks from the source code’s reality.

Usability

Design systems have users who build solutions with the system without extending it, and have devs who build / adapt / tweak new design systems from the existing system.

The first group values User Experience (UX) while the second group longs for Developer Experience (DX).

The groups overlap. Important is to examine design systems from these two perspectives instead of a single one.

Documentation

For users, the significant feature is the documentation. This is the entry point where they meet the design system.

From the analysis, IBM’s Carbon turned out the most complete, offering the following features:

Impeccable UX of the documentation site.
Integrated: All docs (API, Guide) integrated into a single application.
Generated: Entries generated from JSDoc, MDX, tests instead of written separately.
Searchable.
Well-explained props.
Every prop comes with its own example.
Live editor for examples.

API

For users, another significant feature is the usability of the API. Namely:

import ../../... won’t do it. import @package/ will do it.
<Card as='thumbnail'> is better than <Card image={true} title='H3' excerpt={true}>
<Button as='link'> and <Heading level={3}> won’t do it. <Heading as='H3'> is better.

Having a clean and consistent API is art. It takes iterations to achieve simplicity. Braid offers this feature as a unique selling point:

We’re aggressively focused on the simplicity and composability of its API.

Example apps

Show, don’t tell. An example worth thousands of words.

After browsing the docs, a common practice is to check the examples to get a glimpse about the quality of the code.

Yet half of the examined systems have no example apps. Or the code quality rings alarms.

Naming conventions

For developers, the It Just Works! factor is perhaps the most important.

Finding parts of the system should be intuitive with as less cognitive load as possible. From the big picture—tokens, hooks, components, themes—to the details—button variations.

The vertical integration of the layers should be clear and well-thought. The token -> theme -> hook -> component path should be easy to follow when extending the system.

Guidelines for design principles

Design decisions make or break a product. Yet, documenting these decisions is often missing.

Guidelines explaining design decisions reduce development time. Once developers understand the theory behind, they’ll use the system as their own.

Summing up

Creating a design system or a component library starts with analysis and planning. The goal of the process is to prepare the implementation.

Input	Output
Purpose

Audience

Common practices | Features

Technologies

Deliverables

Theory |

Deliverables are standard. They follow a common practice.

Deliverable	Features
Documentation	Integrated

Standard theory is missing. However it should be built using existing software design patterns.

Theory	Features	Notes
Single Source of Truth (SST)	All settings in one single place.	Specific for design systems
Single Responsibility Principle (SRP)	Functional compositions
Modular architecture	Functional Reactive Programming
The base / variant pattern (BEM)	Scalability and simplicity	A common practice
Clean API	Minimal props with uniform naming	The hardest to achieve
Folder structure	Reduce cognitive load	Find anything in no time

Resources

How to write React code in 2019

metamn — Sun, 20 Oct 2019 00:00:00 +0000

Cory House nails it.

Even if there is no official style guide to write React code — there are official guidelines and rules to follow, and a common sense emerging from the leaders.

In Lessons learned from 5 years in React Cory House collects his wisdom into one single document. I found it deeply resonating with my practice — and the common sense.

(With his hopefully mutual permission) I'm replicating the document as checkboxes to see those points I've met, and the ones where I might catch up.

You can do the same exercise too by forking the list from the source code of this article.

Getting familiar with a new project

[x] Review package.json first to understand deps, scripts, and config.
[x] Draw tree on the whiteboard, or use React dev tools. Helps to visualize state.

Dev workflow

[ ] Quickly nav to component or func: CMD click in JSX (VSCode)
[ ] Quickly nav to parent: CMD+SHIFT+F to Search for <ComponentName (VSCode)
[ ] Quickly view list of parents: Use React dev tools
[ ] Create a component state checklist. Use it for every component. (error, no data, lots of data, long values, full list here)
[ ] Debugging? type debugger. console.assert also handy.
[ ] Work against mock data and mock API (I like json-server, json-graphql-server)
[ ] Centralize mock data for Storybook, tests, and mock API.
[x] Pass an object to a function and destructure to create named parameters. Reads more clearly. And can destructure in the function signature too. This keeps the calls in the func short and documents the expected object properties.
[x] Storybook driven development – Build and test each component in isolation. Document each state in a separate story. Then use Percy or Chromatic to snapshot.
- [x] Knobs
- [x] Responsive design

JSX

[x] You can only write expressions within a return. This limits what you can do in JSX. Options:
- [x] Return early. (good for loaders and errors)
- [x] Extract a separate function from render when you want the full power of JavaScript (if/else/switch)

Performance

[x] Do the most convenient thing. It’ll probably be fast enough. Inline func? Fine. Worried about renders? Don’t be. Worried about context performance? Okay, then maybe you’re misusing context (should rarely change). Sure, perf test (set Chrome perf to 6x), but don’t speculate. Degrade ergonomics after establishing a perf issue.
[x] Remember, a render != DOM change. With virtual DOM, the diff is in-memory. Flow: render -> reconcile -> commit. If the DOM doesn’t change, there’s likely no perf issue. So stop worrying about needless re-renders. React is smart enough to only change the DOM when needed, so it’s typically fast enough.
[x] Don’t slap useMemo, shouldComponentUpdate, PureComponent everywhere. Only where needed. They have overhead because it’s an extra diff. If they were typically faster, they’d be the default!

State management

[x] Keep state as low as you can. Lift when needed.
[x] Avoid state that can be derived. Calc on the fly. Reference objects by id instead of duplicating.
[ ] Use _myVar convention to resolve state naming conflicts.
[x] Don’t sync state, derive it. Example, calculate full name on the fly by concatenating firstName and lastName in render. Don’t store fullName separately. Doing so risks out of sync issues and requires extra code to keep it in sync.
[x] State that changes together, should live together. Reducers help. So does grouping via useState. Consider state machines – they describe valid states, which makes invalid state impossible (as new customer with 5 previous purchases, or an admin with no rights shouldn’t be possible. If separate states, they can get outta sync)
[x] Probably don’t need Redux. Lifting state scales nicely and is easy to understand. Prop drilling pain is overblown. Keep prop names the same. Spread props. Pass child. Memoize. Use context and useReducer cover the rare global needs well. Show slides of diff data approaches from my updated Redux course.
[ ] Context isn’t just useful for global data. Useful for compound components. Can be useful for performance.
[ ] setLoading(false) in finally to assure it’s called

Props

[x] Require all props at first
[x] Destructure props in func signature to shorten calls below. Useful on event handler funcs too. But what about props with dashes in name like aria-label? Well, don’t destructure that by using spread: …otherProps
[x] Make your props as specific as possible
[x] Standardize naming. onX for eventHandler props. handleX for the func.
[x] Centralize your propTypes
[x] Document propTypes via JSDoc style comments = autocomplete and docs in Storybook. Can even use markdown!
[x] Spread props or pass children to reduce the pain of prop drilling
[ ] Prop existence conveys truth. So <Input required /> is sufficient.
[x] Honor the native API in your reusable component designs. Pass the full event to event handlers, not merely the value. Then you can use a centralized change handler. Honor the native names (onBlur, onChange, etc). Doing so maximizes flexibility and minimizes the learning curve.

Styling

Note: Here I have a completely different approach based on styled-components best practices and inspired by Material UI.

[ ] Mix styling approaches.
- [ ] Inline styles for dynamic styles.
- [ ] Namespace via CSS modules.
- [ ] Use plain Sass for global styles.
- [ ] CSS in JS remains a hard sell – too many horses in the race.
[ ] Use classnames to apply multiple styles
[x] Use flexbox and CSS Grid over floating styles
[ ] Create abstraction over flexbox to abstract breakpoints for consistency (bootstrap gives ya this)

Reusable components

[x] 3 keys to easy reuse
[ ] Consider dedicating a person/team to this. Why? Speed. Less decision fatigue. Smaller bundles. Consistency = better UX. Less code = fewer bugs.
[x] Look for repeated code – opportunity for reuse. Every reuse is a perf enhancement.
[ ] DRY out your forms by combining custom hooks, context, and reusable components to create an opinionated custom approach that encapsulates your app’s business rules. These tools are the foundation.
[ ] Accept both a simple string and an element. Use React.isValidElement to tell which you’re getting.
[ ] Create an “as” prop to specify the top-level element.
[ ] Create a reusable AppLayout using the slot pattern.
[ ] Centralize alerts in AppLayout and provide function for showing the alerts via context.
[x] Gen custom docs via react-docgen
[ ] Consider creating separate mobile and desktop components if they differ significantly. Lazy load the relevant size.

Testing

[ ] Prefer RTL over Enzyme. Simpler API = pit of success. Encourages a11y. Easy to debug. Can use same queries for Cypress.
[ ] JSDOM doesn’t render, so can’t test responsive design there. Use Cypress to test responsive design behavior.
[ ] Avoid Jest snapshot tests. They’re brittle, they test implementation details, they’re often poorly named, they all fail when a single line changes, and they’re hard to fix when they fail. Instead, prefer Percy or Chromatic to test visuals
[ ] Use the scenario selector pattern to run your app against different data. Automate these tests via Cypress/Selenium
[ ] Use Cypress testing library so your Cy selectors match your React Testing library selectors = No need to change code to support Cypress tests!
[ ] Cypress driven development – TDD for integration testing. Use Cypress to navigate to the spot you need to test. Use cy.only to call a single test. It should fail first. Make it pass.

Dev env

[ ] Consider customizing create-react-app (CRA).
- [ ] Use react-app-rewired to tweak the config without ejecting
- [ ] Customize linting rules.
- [ ] Add webpack-bundle-analyzer. Know what’s in your bundle.
- [ ] Consider forking at least react scripts. Consider forking CRA. Create a company framework that generates a project with a single dependency: Your react-scripts fork that includes your company’s components, tweaks, dependencies, linting rules, webpack.config, etc.
[x] Use Prettier. Convert in one big commit. You’ll look like the hero!
[x] Lean on ESLint. Use as a teaching tool. Object shorthand. No var. Disallow certain imports (jquery, lodash, moment). Require triple equals. Don’t form a committee. Assign someone you trust and enable a lot of good stuff. Can always back off later. Add plugins like jsx-a11y/recommended.
[x] Require strict propTypes (or TS). I don’t get many type issues. (see link for list)
[ ] Use .vsextensions to encourage extensions.
[ ] Keep client and server separate. If embedding React in a server-side tech, use Storybook to develop components in isolation.

Consider a monorepo

[ ] Why? Rapid feedback. Continuous integration.
[ ] Easy reuse
[ ] CI integration tests projects on every PR
[ ] Use Lerna, Bolt, Yarn Workspaces, or even simply a relative file reference to your reusable components to manage. I typically use Lerna.

Learning

[x] Have a system for organizing your knowledge. Find a new tool? Technique? Document it. I use GitHub issues here on my reactjsconsulting repo.

Resources

A Big Picture

metamn — Sun, 20 Oct 2019 00:00:00 +0000

React and the ecosystem mapped up.

A higher resolution image and detailed information about the building blocks is available in Miro.

All resources based upon the chart was drawn are available in Putting React in context

We're gonna program like it's 1999

metamn — Sat, 19 Oct 2019 00:00:00 +0000

How joy and simplicity was brought back into our lives by the Facebook crew.

Personally

With React and its ecosystem — functional reactive programming, microservices, domain-driven design, data-driven applications — I've got back my mojo.

Now I write code as I've written twenty or thirty years ago.

Now I design and develop user interfaces and applications like a software engineer.

No more hacking, plumbing, massaging HTML, CSS, Javascript with Gulp, Grunt or Webpack — it's programming again.

On GraphQL

metamn — Thu, 10 Oct 2019 00:00:00 +0000

Easy to use on the front-end. More complicated on the back-end.

Definition

GraphQL, according to GraphQL.org is three things:

A query language
A server-side runtime
A type system

Query language

We all know query languages. SQL — to query relational databases. REST API — to query data on the backend.

GraphQL is in the same way a query language. It is like REST built on the more advanced principles of functional and reactive programming.

Server-side runtime

The UNIX philosophy of

Do one thing and do it well

is built into GraphQL making it a super simple layer on the server.

The GraphQL runtime does only one thing: returns results for queries. How results are computed, put together, collected from other services — the business logic — is outside its scope.

(As a compensation) GraphQL offers extensive connectivity to various backend services like databases, storage engines, serverless functions, authentication, caching to be used in any combination to define how the application works.

Type system

What glues together the client-side queries and server-side responses is the GraphQL Schema — a place where:

All types are defined together with
All fields for all types, and
All single purpose functions (resolvers) associated with each and every field

In practice:

/* A GraphQL Schema */

/**
 * Data type
 * - Defines a data entity
 */
type Book {
    id: ID
    title: String  /* A field */
    author: Author
}

/**
 * Data type
 * - Defines a data entity
 */
type Author {
    id: ID
    firstName: String  /* A field */
    lastName: String
}

/**
 * Query type
 * - Defines operations on data
 */
type Query {
    book(id: ID): Book  /* A field */
    author(id: ID): Author
}

/**
 * Server-side, single purpose functions (resolvers)
 */
const resolvers = {
  Query: {
    author: (root, { id }) => find(authors, { id: id })
  },
  Author: {
    books: author => filter(books, { authorId: author.id })
  }
};

# Client-side query
#
GET /graphql?query={
    book(id: "1") {
        title,
        author
        {
            firstName
        }
    }
}

/**
 * The result
 */
{
  "title": "Black Hole Blues",
  "author": {
    "firstName": "Janna",
  }
}

The Facebook way

GraphQL was created by Facebook and later open sourced for the community. Together with the other parts of the stack — React, Relay — they power one of the largest web apps today, Facebook.com.

It's good to be aware of the Facebook way. To learn about the best practices on large scale.

Facebook has been using GraphQL in production for almost four years; today, it serves over 300 billion queries a day and its schema has nearly 10,000 types.

In building this API, we’ve developed a set of best practices for designing an understandable and scalable GraphQL schema. — Dan Schafer at react-europe

Facebook defines GraphQL using the following concepts:

The underlying database and business model is a graph
There is a single source of truth
The API is a thin layer

Graph databases

Comparing Database Types: How Database Types Evolved to Meet Different Needs has a great overview and definition for graph databases:

Graph databases are most useful when working with data where the relationships or connections are highly important.

In contrast, the relational database paradigm is best used to organize well-structured data:

In general, relational databases are often a good fit for any data that is regular, predictable.

In other words graph databases focus on interactions in an unpredictable environment while relational databases focus on structure in a well-known context.

In graph databases entities have flexible shapes and more importantly they can form relationships freely, on the fly.

In relational databases the business domain is well known a priori and what's left is to create a well performing model upon.

No wonder Facebook chose the graph approach. It handles better the use case of interaction-heavy user interfaces.

Domain-driven design — DDD

Dynamic contexts needs a new kind of design thinking to be able to provide solutions.

In a rigid environment, where there are no moving parts and everything is under control one could easily model how things work using an imperative approach.

In dynamic environments the only (relatively) sure thing is the existence of an entity. The capabilities an entity offers can change over time. Therefore the most important thing an entity can do is to declare what are its capabilities. Then the other parts of the system will be able to understand it and interact with.

For such evolving models where an entity is:

An object that is not defined by its attributes, but rather by a thread of continuity and its identity.

a suitable design approach is called Domain-driven design.

via Martin Fowler

Microservices

The Facebook way implies Domain-driven design which is closely related to the microservices architecture.

Ville Touronen from University of Helsinki wrote a well-worth-reading master thesis about how GraphQL connects to DDD and microservices.

In short — this new context, array of technologies, and paradigms requires the business domain to be split into different functional domains (services) which are highly isolated, independent and loosely coupled (micro).

via Apollo / Expedia GraphQL architecture

Microservices complete the big picture. The Facebook way is a full bet on the Functional Reactive Programming paradigm from design (DDD), data (GraphQL and graph databases), implementation (React) to servers (microservices).

Single source of truth

In a dynamic context it is very important to establish a single source of truth from where all other parts of the stack provision themselves.

The creators of GraphQL are always eager to emphasize the importance of such a truth layer.

In Thinking in Graphs / Business Logic Layer chapter there is a clear definition and a diagram describing the use case:

Your business logic layer should act as the single source of truth for enforcing business domain rules

In the Facebook approach the truth gatekeeper role is given to GraphQL.

GraphQL's type system / schema is suitable to declare and define the capabilities of an entity. And it is extendable through Smart Data Objects / GraphQLObjectType to connect with the business logic layer.

/**
 * From Ville Touronen's master thesis
 * 
 * See: https://helda.helsinki.fi/bitstream/handle/10138/304677/Touronen_Ville_Pro_gradu_2019.pdf
 */



/**
 * - The business logic is held in a separate layer
 * - Each type (`Book`) has an associated `model` where
 * ... data fetching, business logic, or security is solved 
 * ... exactly once for this type across the application
 * ... providing the single source of truth
 * 
 * See: https://blog.apollographql.com/graphql-at-facebook-by-dan-schafer-38d65ef075af 
 */ 
import { getBook } from './models/book'


/**
 * Bindings to the business logic layer
 */
const bookQuery = new GraphQLSchema({
    query: new GraphQLObjectType({
        name: `Query`,
        fields: {
            book: {
                type: bookType ,
                args: {
                    id: {
                        description: 'internal id of the book',
                        type: GraphQLNonNull ( GraphQLString ) ,
                    },
                },
                /**
                 * Resolvers **always** map to the business logic
                 */
                resolve: ( root, { id } ) => getBook( id ),
            }
        }
    })
});

/**
 * The capabilities of an entity aka the types
 */
const bookType = new GraphQLObjectType({
    name: 'Book',
    description: 'A book with an ISBN code',
    fields: () => ({
        id: {
            type: GraphQLNonNull(GraphQLString) ,
            description: 'The internal identifier of the book',
        },
        /* ... The other fields ... */
    })
})

/**
 * All wrapped together
 */
export const BookSchema = new GraphQLSchema({
    query: bookQuery,
    types: [ bookType ],
});

Thin API Layer

The most important takeaway up to this point is the:

type → field → resolver → business logic pattern.

Types have fields and every field has an associated server-side function which returns results and connects to the business logic layer.

The first three items constitute the thin API layer of GraphQL, the last one is the separated business logic layer.

|------------------|                 |----------------------|
| GraphQL Thin API |                 | Business Logic Layer |
|---------------------------|        |--------------------------------|
| Type -> Field -> Resolver |   ->   | Model / Single source of truth |
|---------------------------|        |--------------------------------|

This pattern is a double-edged sword. It makes design and development easier but scaling on the server-side harder.

The N+1 problem

The N+1 selects problem is a basic design and development constraint in older paradigms like relational databases. It makes the business / data / component model to follow certain strict technical guidelines which are not natural to default human thinking.

In GraphQL this issue is automatically solved.

The original N+1 problem is related to database design. Improperly designed database tables can lead to more database queries than optimal reducing considerably the app response time. To circumvent this issue in the object-relational paradigm various normalization techniques are used.

In GraphQL there is no N+1 problem. One can design freely the types in the schema and a middle-layer — the Dataloader — takes care of eliminating the N+1 performance issues.

In practice this means fields can be freely added to types without worrying about normalization. Components can be modeled in a less rigid, more human friendly way using graphs which let directly store the relationships between records.

Writing the associated resolvers to fields is again free thinking: just focus on the single purpose of the function of returning the results and forget about redundancy, caching and performance.

The chatty server-side functions (resolvers) which might repaetedly load data from the database are collected, optimized into a single request, and their results cached — by the GraphQL middle-layer.

Challenges are mounting on the back-end

Around two third of all talks from the 2019 GraphQL conference is about the schema.

How to build it from fragments to make it scalable; how to design it in a way to properly handle error messages; a dozen of opinions on how to manage the growth of the schema. From Github, Facebook to Twitter, Coursera and Visa everybody is facing the schema scaling issue.

The GraphQL / Domain-driven design / Microservices patterns — API Gateway, Integration Database, Data Federation, Backend for Front End — are new concepts and all subject of scaling.

Conclusion

GraphQL is no silver bullet. It's not better or worse than other paradigms.

It makes app design and user interface development more human by empowering the architects, designers and front-end developers. What is gained here has to be solved on the back-end in new ways with new efforts.

Resources

The reactive, functional nature of React

metamn — Tue, 08 Oct 2019 00:00:00 +0000

React follows the Functional Reactive Programming paradigm. That's a shift from the old proactive, imperative approach.

The context

Both paradigms solve the same problem — to build modular, interactive applications — in different ways.

The old way implemented modularity through tight-coupling and interaction synchronously. The new way implements modularity with loose-coupling and interaction asynchronously.

Modularity and interaction abstracted to composition and coupling can further elucidate the picture. Composition is about how the components of the system can add up to form a whole, while coupling is about how they communicate with each other.

Functional reactive programming deals with composition the functional programming way and with coupling the reactive programming way.

Composition

Composing up a system from smaller parts is best possible when the underlying components behave predictably:

They have clear and stable interfaces — input parameters and return values — which can be easily combined.
During execution, while input parameters are transformed into return values

2.1. The process doesn't modify the input parameters

2.2. No external information is used beside input parameters to produce the return value

2.3. No additional operations are performed beside the planned functionality

Pure functions

Such components in functional programming are called pure functions.

/**
 * A pure component example
 */
const PureComponent = props => {
  /**
   * ✅ Clear interfaces:
   * - props are input params, JSX is the return value
   * ✅ Input params are not modified
   * ✅ No external information is used beside input params to produce the return value
   * ✅ No additional operations are performed beside the planned functionality
   */
  return <div>Pure component props: {JSON.stringify(props)}</div>;
};

Impure functions

Having only pure functions in a complex system is almost impossible. Therefore the concept of impure functions and side effects is introduced.

/**
 * An impure component example with side effects
 */
const ImpureComponent = props => {
  /**
   * ✅ Clear interfaces:
   * - props are input params, JSX is the return value
   * ✅ Input params are not modified
   * - props are protected by React so they can't be easily distorted
   */

  /**
   * Side effect #1
   *
   * ❌ External information is used beside input params to produce the return value
   */
  const external = "External info";

  /**
   * Side effect #2
   *
   * ❌ Additional operations are performed beside the planned functionality
   */
  console.log("props:" + JSON.stringify(props));

  return (
    <ul>
      <li>Impure component props: {JSON.stringify(props)}</li>
      <li>External info in return value: {external}</li>
    </ul>
  );
};

Side effects, managed

When side effects are not avoidable they have to be made resilient — not distorting the original application flow.

/**
 * Side effects managed through useEffect
 */
const SideEffectsWithUseEffect = props => {
  /**
   * ✅ The `external` side effect is wrapped into a state
   * ✅ Later it can be managed via `useEffect` which makes it resilient 
   */
  const [external, setExternal] = useState("Initial value ...");

  /**
   * ✅ By using `useEffect` the rendering of the component is not suspended, the flow is not distorted.
   * ✅ First the component is rendered with the default value
   * ✅ After 3 seconds React automatically updates the component with the new value.
   * (The 3s delay added makes the effect visible on UI)
   */
  useEffect(() => {
    /**
     * Manages the side effect
     */
    const timer = setTimeout(function() {
      setExternal("Initial value replaced with useEffect");
    }, 3000);

    /**
     * Cleans up the side effect
     *
     * @link https://reactjs.org/docs/hooks-effect.html
     */
    return () => clearTimeout(timer);
  }, []);

  /**
   * ✅ The `console.log` additional operation side effect is wrapped into the `useEffect` hook
   * ✅ The Effect Hook lets you perform side effects in function components
   */
  useEffect(() => {
      console.log("props:" + JSON.stringify(props))
    }, [props]);

  return (
    <ul>
      <li>Component props: {JSON.stringify(props)}</li>
      <li>External info in return value with useEffect: {external}</li>
    </ul>
  );
};

Coupling

Coupling deals with the communication between the components of a system. To make a system scale communication has to be in the same effective as composition.

Reactive programming achieves this through:

Isolation:

1.1 Components are aware of, and care about their own problems only

1.2 Components does not interfere with each other's inner workings
Asynchronous communication:

2.1 Components subscribe to events

2.2 They react when an event happens

2.3 They return an observable — which emits events other components can subscribe to.

Isolation

In React isolation is about the local state — components can update only their own state and can't modify other component's state.

Asynchronous communication

In React asynchronous communication is all about hooks. Components communicate with each other and the surrounding ecosystem — the local data store, the local cache store, the database — via the hooks mechanism.

|-----------------|-------------------|
| Hook            | Ecosystem         |
|-----------------| ------------------|
| useContext      | Local data store  |
| useMemo         | Local cache store |
| useCallback     | Other components  |
| useQuery,       |                   |
| useMutation,    |                   |
| useSubscription | GraphQL database  |
|-------------------------------------|

Wrapping up

Using reactive and functional programming concepts React and its building blocks — props, state, hooks, side effects, pure components — can be put in a general context.

A general context which is a great help when mapping up the big picture.

|================|                            |================|
| Component 1    |                            | Component 2    |
|================|                            |================|
| Props          |                            | Props          |
|----------------|                            |----------------|
| Local state    |                            | Local state    |
|----------------|                            |----------------|
|                | <-------- Hooks -------->  |                |
|                | <-------- Hooks -------->  |                |
| Side effects   | <-------- Hooks -------->  | Side effects   |
|                | <-------- Hooks -------->  |                |
|                | <-------- Hooks -------->  |                |
|----------------|                            |----------------|
| Return JSX     |                            | Return JSX     |
|================|                            |================|

     ^                 ^                ^                ^
     |                 |                |                |
     | Hooks           | Hooks          | Hooks          | Hooks
     |                 |                |                |
|----------|     |-----------|     |----------|     |-----------|
| State    |     | Data      |     | Cache    |     | UI        |
| store    |     | store     |     | store    |     | Events    |
|----------|     |-----------|     |----------|     |-----------|

Resources

Ontology, taxonomy, choreography

metamn — Sun, 06 Oct 2019 00:00:00 +0000

A way to think about Information Architecture in React terms.

What is Information Architecture, and what is not?

We all think we know what Information Architecture is — so why the question?

Wikipedia has 9 definitions for this term. Choosing the ones related to web / app design we have:

The art and science of organizing and labeling web sites, intranets, online communities, and software to support findability and usability.
The combination of organization, labeling, search and navigation systems within websites and intranets.
The practice of organizing the information / content / functionality of a web site so that it presents the best user experience it can, with information and services being easily usable and findable (as applied to web design and development).

These definitions relate to keywords specific to UX design — usability, experience — yet there is a clear differentiation between IA and UX:

Information Architecture concerns structure.
User Experience concerns emotion.

Let's analyze further Information Architecture by the means of structure.

Architects of understanding

The term IA was born together with the web and evolved with mobile devices and apps.

Peter Morville, an Information Architecture icon, captures perfectly these moments:

... and concludes with a statement where — again — UX is removed:

Here we are in the business of understanding: understanding the problem and understanding how people can best solve the problem — in an interactive space.

Information Architecture now has a shape. It's about modeling problem structure and modeling user behavior with the help of user interfaces.

Structure and behavior

Dan Klyn has a clear take on how to model structure and behavior in user interfaces: with ontology, taxonomy and choreography.

explaining information architecture from Dan Klyn on Vimeo.

Simplified to this point — the tasks of an Information Architect are clear:

Reduce, limit the complexity of the problem.
Organize the information about the problem into data.
Define data entities, the properties of entities, and the relations between.
Organize entities into hierarchies, maps and views.
Design the sequences of movement between the elements above.

Ontology, taxonomy, choreography and React

Finally we can put React in context with Information Architecture giving developers and UI/UX designers a bigger picture and a shared vocabulary.

Ontology, taxonomy and choreography can act as the boundary between these two different disciplines.

|------------|     |------------|     |--------------|
| Ontology   |     | Taxonomy   |     | Choreography |    UI/UX ▲
|------------|     |------------|     |--------------|    -----------
| Data       |     | Sitemap    |     | Events       |    UI/UX + Dev
|------------|     |------------|     |--------------|    -----------
     |                   |                   |            Dev ▼
     |                   |                   |
     ▼                   ▼                   ▼
|------------|     |------------|     |--------------|
| GraphQL    |     | Router     |     | State        |
|------------|     |------------|     |--------------|
     |                   |                   |
     |                   |                   |
     ▼                   ▼                   ▼
|----------------------------------------------------|
|                 React Components                   |
|----------------------------------------------------|

Resources

On state

metamn — Sat, 05 Oct 2019 00:00:00 +0000

Everything in React rolls around the concept of state.

The concept

State was unknown in proactive programming. Now it is surfacing as the central element of reactive programming.

In Dan Lew's seminal article about Functional Reactive Programming the concept of state is fully explained and put further in context. Please read it first. It's pure joy.

Proactive programming — No state

In proactive programming a component directly calls public methods of other components — and the passive components are not aware of these calls. They don't do accounting for what is happening inside them — they have no state.

The caller component is responsible to interpret the results of the call. The passive component has one single responsibility: to make the publicly callable methods available to the external world.

Reactive programming — With state

In reactive programming everything is different. Each component owns its own state and modifies it on request. No component can directly modify the state of another component.

The caller component has to ask the receiver component to perform an operation which might modify the receiver component's internal state, then wait for — subscribe to — the results.

This kind of isolation brings greater composability — essential to build large scale systems and complex, interactive user interfaces.

How it works

|============|==================|=================|==============|
| Paradigm   | Step 1           | Step 2          | Step 3       |
|============|==================|=================|==============|
| Reactive   | The switcher     | The bulb        | The bulb     |
|            | emits request    | receives        | changes its  |
|            | for state change | the request     | own state    |
|------------|------------------|-----------------|--------------|
| Imperative | The bulb         | The switcher    |              |
|            | exposes a state  | calls the state |              |
|            | change function  | change function |              |
|            |                  | of the bulb     |              |
|================================================================|

State in React

State is so important in React it is marked first in the list of features:

React makes it painless to create interactive UIs. Design simple views for each state in your application, and React will efficiently update and render just the right components when your data changes. — https://reactjs.org/

More, when hooks were introduced they were immediately related to state:

Hooks are a new addition in React 16.8. They let you use state and other React features without writing a class. — Introducing Hooks

React has to offer built-in mechanisms to deal with state — to support one of its main core features. And yes it does with useState and useReducer.

None of these are powerful enough to handle complex scenarios like global shared state — leaving to the ecosystem to come up with alternatives like:

Redux, MobX — Global, shared state services for large applications.
Apollo GraphQL — State management bundled with data management from a proven provider.
XState — Finite state machines and state charts to manage state and make it error prone.

Every day a new state library or approach is popping up. There is no holy grail which one to use — everybody makes its own bet.

When in doubt important is to follow the React way: 'Don’t overthink it`. Which in this case translates to: 'Use only when you need it'.

Learn the different techniques and combine them by need.

|================|=============================|=========================================== | Implementation | When to use | State type | Notes | |================|=============================|=========================================== | useState | Simple states | Local | | |----------------|-----------------------------|------------|-----------------------------| | useReducer | Complex state logic | Local | WithuseContext~= Redux | |----------------|-----------------------------|------------|-----------------------------| | useMachine | Complex state logic | Local | - Use for component design | | (XState) | Fool-proof states | | - Autogenerate tests | |----------------|-----------------------------|------------|-----------------------------| | Redux, MobX | When none above are enough | Global | | |================|=============================|============|=============================|

Resources

Rapid prototyping for GraphQL APIs?

metamn — Thu, 03 Oct 2019 03:00:00 +0000

That's nearly impossible today.

GraphQL is young

According to its creator GraphQL is expected to become ubiquitous in 2020-2021. Today is in the phase of being accepted by the large companies.

The official GraphQL page lists only a handful of services ready to build upon. Of them a few are complete solutions — backends as a service — and of them a few are production ready.

Graphcool - is self hosted
Reindex - looks empty
Scaphold - closed down
Tipe - is in beta
Hasura - has no pricing plans

What's left is Apollo, GraphCMS and AWS AppSync.

Before checking them let's see how an ideal GraphQL API backend should look like.

The ideal GraphQL API service

An ideal service would offer the following features:

Visual Schema designer
Automatic database generation based on the schema
Automatic resolver generation
Automatic lambda function generation for the resolvers
Automatic Type / Schema code generation and sync for clients

In other words:

One should be able to visually design the data structure
From that design all else — possible — is automatically generated: database tables, resolvers, lambda functions, schema, types, boilerplate client code
On client side, during development, when the schema is modified it is automatically synced with the backend.

That would mean rapid prototyping.

Apollo and AWS AppSync

Both of them support all requirements except 1.) the visual scheme designer.

In Apollo all tasks imply (mostly) manual work. In AWS AppSync + Amplify everything is automated.

No wonder. Their description and distinctive features are put pretty clear on GraphQL.org.

Apollo Engine: A service for monitoring the performance and usage of your GraphQL backend.

AWS AppSync: Fully managed GraphQL service with realtime subscriptions, offline programming & synchronization, and enterprise security features as well as fine grained authorization controls.

One has to choose AWS AppSync then, or dive deep into Apollo to find hidden gems unsurfaced during this quick research.

However in our collective mindset AWS is for large scale production apps and enterprise use not for rapid prototyping. All plans are paid and every operation is billed.

GraphCMS to the rescue?

Headless CMS is young (and not enough)

From the list of developer friendly headless CMS services on JAMStack — including Contentful, GraphCMS, Sanity, TakeShape, Prismic, Kentico — only a few experiences worth it.

Setting up an account and trying to get a GraphQL endpoint on a free plan was tedious or impossible in many times:

Contentful - No GraphQL on the free plan
GraphCMS - Except 5.) Automatic code generation and sync — it offers all other features. However these features seems to be undocumented and very fragile.
Sanity - A strange, download first approach. Skipped for now.
TakeShape - The GraphQL endpoint works out of the box. Worth experiencing with it further.
Prismic - GraphQL support has to be asked for / enabled manually by the support team. Worth experiencing with it further.
WPGraphQL - Works perfect however is tailored to the WordPress data structure. It can be extended of course but WordPress is a specific, not a general purpose CMS.

Conclusion

Wait for the Headless CMS scene to take off.

Otherwise, if needed right now, go with AWS. All services can be used for free for a year.

Is AWS expensive? It's a Tier 1 service. Probably all other Headless CMS providers on upper tiers are building upon its infrastructure — increasing the bill.

Resources

Identify the right leaders

metamn — Thu, 03 Oct 2019 00:00:00 +0000

Or else.

The buzz

React is famous about its steep learning curve. No wonder — starting from the programming model through data to user interaction introduced changes which are often paradigm shifts.

Want the buzzwords?

Reactive programming, Functional programming, Pure functions, Side effects, State, Hooks, Functional components, Referential equality, Finite State Machines / XState, Data driven applications, Code splitting, Pre-caching assets, Pre-rendering static HTML, Memoization, Static type checking — none of these were present in your old framework, perhaps.

Learning React is not like it was learning Ruby on Rails, Yii, or MVC. It’s more like a fresh start with unlearning first, then re-learning everything from information architecture design to event-based user interfaces.

The problem

Identify the right leaders.

A surprisingly compact ecosystem around React makes this process enjoyable. There are a handful of opinion leaders who often share their thoughts which are in line.

The list goes:

Robin Wieruch
Kent C. Dodds
Dan Abramov
Dave Ceddia
(sorry if I skipped you …)

Once you’ve found the line your learning process will be smooth unless you leave the safe waters. At the edge there is nobody to guide you.

An example

All the buzz is around the state. It’s React’s cornerstone and it’s not yet solved.

Go with Redux? Forget Redux after hooks? Dive into finite state machines? What about MobX? Many job posts require it — it must be popular and useful.

Apollo started to introduce local state management with GraphQL. What that means for all the other libraries? Will they push it together with Apollo Client?

RWieruch just published a tutorial for state management with RxJS. Should I read it?

There is a new library React Tracked: Super fast global/shared state with context and hooks. Should I read it? Can I trust the author? Is he in the line?

The solution

I went with finite state machines / Xstate. Why?

Its leader is trustworthy. Checked videos, blog posts, chatted on twitter, created a test project, advanced it, found a bug, filed a bug in Github, got a fix for it — and all the time every interaction felt so natural.

Will this choice stand the test of time? I don’t know. But this new piece of knowledge perfectly fits the bigger picture. Good UI/UX should be flawless and this tool helps achieve that.

I’m adding David K. Piano to the list.