DEV Community: Laurin Quast

GraphQL Live Queries backed by the Relay Specification?

Laurin Quast — Sat, 20 Mar 2021 13:29:24 +0000

Photo by Ricardo Gomez Angel on Unsplash

Note: This post requires some basic knowledge of what GraphQL live queries are and how relay works. Therefore I recommend reading my previous articles about live queries GraphQL Live Queries with Socket.io and Collecting GraphQL Live Query Resource Identifier with GraphQL Tools and this awesome series Relay: the GraphQL client that wants to do the dirty work for you first.

The Relay GraphQL specification has some nice implications that could potentially greatly benefit live query implementations.

Unique global identifiers
The Query.node field

As each Node.id field should resolve to a global unique identifier that can be passed to the Query.node(id:) for querying for the given resource a live query engine can leverage that for only re executing partials of a query document.

Example Schema

interface Node {
  id: ID!
}

type Position2D {
  x: Float!
  y: Float!
}

type Token implements Node {
  id: ID!
  label: String!
  position: Position2D!
}

type MapGrid implements Node {
  id: ID!
  position: Position2D!
  columnWidth: Float!
  columnHeight: Float!
}

type Map implements Node {
  id: ID!
  grid: MapGrid
  tokens: [Token!]!
}

type Query {
  node(id: ID!): Node
}

Example Live Query

query map($id: ID) @live {
  map: node(id: $id) {
    ... on Map {
      id
      grid {
        id
        position {
          x
          y
        }
        columnWidth
        columnHeight
      }
      tokens {
        id
        label
        position {
          x
          y
        }
      }
    }
  }
}

The live query engine could then build the following queries for efficiently re-executing partials instead of the full query document, after a global unique ID has been invalidated:

Token

query node($id: ID) {
  node(id: $id) {
    ... on Token {
      id
      label
      position {
        x
        y
      }
    }
  }
}

Given a token has a global unique id (Token.id) of Token:1 a invalidation and execution of the ad-hoc query could be scheduled via liveQueryStore.invalidate("Token:1").

MapGrid

query node($id: ID) {
  node(id: $id) {
    ... on MapGrid {
      id
      position {
        x
        y
      }
      columnWidth
      columnHeight
    }
  }
}

And then publish the result in some patch format:

Token Sample JSON payload

{
  "data": {
    "id": "Token:1",
    "label": "Some Orc",
    "position": {
      "x": 10,
      "y": 10
    }
  },
  "path": ["map", "tokens", 0],
  "hasNext": true
}

MapGrid Sample JSON payload

{
  "data": {
    "id": "Map:1:MapGrid",
    "position": {
      "x": 10,
      "y": 10
    },
    "columnWidth": 50,
    "columnHeight": 50
  },
  "path": ["map", "grid"],
  "hasNext": true
}

On the client we definitely need some middleware for applying the deltas similar to @n1ru4l/graphql-live-query-patch.

For bigger queries this can drastically reduce the payload that must be sent over the wire.

Furthermore, a JSON patch (or similar) middleware could furthermore optimize the payload, so it is only necessary to send deltas.

E.g. if a Token position would have changed the delta could look similar to this:

{
  "patch": [
    { "op": "replace", "path": "/position/x", "value": 5 },
    { "op": "replace", "path": "/position/y", "value": 5 }
  ],
  "path": ["map", "tokens", 0],
  "hasNext": true
}

What about lists?

As always lists are a huge pain point of real-time and are currently still not properly addressed by the InMemoryLiveQueryStore implementation.

The connection specification of relay, however, might help building a proper abstraction for invalidating pagination.

First of all one should clarify whether pagination is actually needed. In the example above one could argue whether it is necessary.

On the one hand we could have a small map with only 10-20 token objects. Having pagination does not make sense for that. But we could also have a list of like millions of items (imagine Google maps). There a connection might be handy and the connection args might include some information about the visible area and the zoom level, so you can return the most important items to display based on that.

But that might not really the issue for us right now. The most important thing that should be relevant to us right now is: How can we efficiently add and remove items?

Let's first take a look on how I tackled this in the past using GraphQL subscriptions with a Subscription.notesUpdates field that publishes payloads for manually updating the existing connection in the client cache.

type Query {
  notes(first: Int, after: String): NoteConnection!
}

type NoteConnection {
  edges: [NoteEdge!]!
  pageInfo: PageInfo!
}

type NoteEdge {
  cursor: String!
  node: Note!
}

type Note implements Node {
  id: ID!
  documentId: ID!
  title: String!
  content: String!
  contentPreview: String!
  createdAt: Int!
  viewerCanEdit: Boolean!
  viewerCanShare: Boolean!
  access: String!
  isEntryPoint: Boolean!
  updatedAt: Int!
}

type NotesUpdates {
  """
  A node that was added to the connection.
  """
  addedNode: NotesConnectionEdgeInsertionUpdate
  """
  A note that was updated.
  """
  updatedNote: Note
  """
  A note that was removed.
  """
  removedNoteId: ID
}

type NotesConnectionEdgeInsertionUpdate {
  """
  The cursor of the item before which the node should be inserted.
  """
  previousCursor: String
  """
  The edge that should be inserted.
  """
  edge: NoteEdge
}

type Subscription {
  notesUpdates(endCursor: String!, hasNextPage: Boolean!): NotesUpdates!
}

The corresponding client code has been implemented like this:

const subscription = requestSubscription<tokenInfoSideBar_NotesUpdatesSubscription>(
  environment,
  {
    subscription: TokenInfoSideBar_NotesUpdatesSubscription,
    variables: {
      endCursor: data.notes.pageInfo.endCursor,
      hasNextPage: data.notes.pageInfo.hasNextPage,
    },
    updater: (store, payload) => {
      if (payload.notesUpdates.removedNoteId) {
        const connection = store.get(data.notes.__id);
        if (connection) {
          ConnectionHandler.deleteNode(
            connection,
            payload.notesUpdates.removedNoteId
          );
        }
      }
      if (payload.notesUpdates.addedNode) {
        const connection = store.get(data.notes.__id);
        if (connection) {
          const edge = store
            .getRootField("notesUpdates")
            ?.getLinkedRecord("addedNode")
            ?.getLinkedRecord("edge");
          // we need to copy the fields at the other Subscription.notesUpdates.addedNode.edge field
          // will be mutated when the next subscription result is arriving
          const record = store.create(
            // prettier-ignore
            `${data.notes.__id}-${edge.getValue("cursor")}-${++newEdgeIdCounter.current}`,
            "NoteEdge"
          );

          record.copyFieldsFrom(edge);

          if (payload.notesUpdates.addedNode.previousCursor) {
            ConnectionHandler.insertEdgeBefore(
              connection,
              record,
              payload.notesUpdates.addedNode.previousCursor
            );
          } else if (
            // in case we don't have a previous cursor and there is no nextPage the edge must be added the last list item.
            connection?.getLinkedRecord("pageInfo")?.getValue("hasNextPage") ===
            false
          ) {
            ConnectionHandler.insertEdgeAfter(connection, record);
          }
        }
      }
    },
  }
);

const TokenInfoSideBar_NotesUpdatesSubscription = graphql`
  subscription tokenInfoSideBar_NotesUpdatesSubscription(
    $endCursor: String!
    $hasNextPage: Boolean!
  ) {
    notesUpdates(endCursor: $endCursor, hasNextPage: $hasNextPage) {
      removedNoteId
      updatedNote {
        id
        title
        isEntryPoint
      }
      addedNode {
        previousCursor
        edge {
          cursor
          node {
            id
            documentId
            title
          }
        }
      }
    }
  }
`;

So the three important events are

A node got added
A node got deleted
A node got updated

The latter can actually already be addressed by simple invalidation via the global unique note id. (e.g. Note:1).

liveQueryStore.invalidate("Note:1")

{
  "data": {
    "id": "Token:1",
    "label": "Some Orc",
    "position": {
      "x": 33,
      "y": 33
    }
  },
  "path": ["map", "paginatedTokens", "edges", 0, "node"],
  "hasNext": true
}

So it must not necessarily be covered by our live connection abstraction. The only crucial thing is that we always need to know the index if the item in the edge array. I am not sure if we could guarantee this. Any recommendations are welcome!

So if we want to implement this with live queries we will have to come up with a solution for (1) a node got added and (2) a node got deleted.

Let's jump back to our Token example and let's model it as a connection.

Token modeled with the connection spec

type TokenEdge {
  cursor: String!
  node: Token!
}

type TokenConnection {
  pageInfo: PageInfo!
  edges: [TokenEdge!]!
}

extend type Map {
  paginatedTokens(first: Int, after: String): TokenConnection!
}

Maybe the TokenEdge.cursor field might be the source of truth for this?

If we can identify where an item must be added or deleted based on the cursor that might make sense.

If we want to add a new item we can do this by adding an item to the list AFTER an item with a specific cursor.

If we want to remove an item we can do this by removing an item WITH a specific cursor.

Other things that one might need is re-sorting items. This could be achieved by having a list of remove and add instructions for all affected items.

The question now is: How can we model this abstraction in live query land?

Live query but with connection instead of a simple list

query map($id: ID) @live {
  map: node(id: $id) {
    ... on Map {
      id
      grid {
        id
        position {
          x
          y
        }
        columnWidth
        columnHeight
      }
      paginatedTokens {
        edges {
          cursor
          node {
            id
            label
            position {
              x
              y
            }
          }
        }
      }
    }
  }
}

Diffing the whole connection might be super expensive. So the easiest solution might be to add some kind of imperative API for notifying that an item got added/removed from a connection.

// The cursor string is combined out of three parts.
// 1. Connection name
// 2. Edge resource type name
// 3. Edge node resource id
// We could also obscurify this for the client. For simplicity I kept it a plain string :)
const cursor = "TokenConnection|TokenEdge|Token:1";
liveQueryStore.triggerEdgeRemoval(cursor);

const afterEdgeCursor = cursor;
const newEdgeCursor = "TokenConnection|TokenEdge|Token:2";
liveQueryStore.triggerEdgeInsertion(afterEdgeCursor, newEdgeCursor);

If the live query store is aware of the cursor format and can do stuff based on its contents, it can then generate the patches that should be sent to the client.

E.g. for the edge removal flow via the "PaginatedTokens|TokenEdge|Token:1" cursor can first look for all operations that select the TokenConnection type. Then check which of those connections includes the TokenEdge that has a node with the id Token:1 and send a patch for this items removal to the affected clients.

Patch for removing a token:

{
  "connectionPatch": {
    "type": "removeEdge",
    "cursor": "TokenConnection|TokenEdge|Token:1"
  },
  "path": ["map", "paginatedTokens"],
  "hasNext": true
}

For the edge insertion task it can do the steps above for the afterEdgeCursor ("PaginatedTokens|TokenEdge|Token:1"), and then additionally load the new resource edge node ("TokenConnection|TokenEdge|Token:2") via our partial operation we generated earlier:

query node($id: ID) {
  node(id: $id) {
    ... on Token {
      id
      label
      position {
        x
        y
      }
    }
  }
}

Patch for adding a new token after another token:

{
  "connectionPatch": {
    "type": "insertEdge",
    "afterCursor": "TokenConnection|TokenEdge|Token:1",
    "edge": {
      "cursor": "TokenConnection|TokenEdge|Token:2",
      "node": {
        "id": "Token:2",
        "label": "foo bars",
        "position": {
          "x": 20,
          "y": 20
        }
      }
    }
  },
  "path": ["map", "paginatedTokens"],
  "hasNext": true
}

If the list is initially empty we don't have a afterCursor, so we might need to use null instead to indicate that.

But also what if the connection returns different data based on the connection arguments or even viewer scope? E.g. an admin user might see all tokens and a normal user might only see tokens that are marked as visible.

If we encode this information in the cursor that might work. I will update this once I gather some more thoughts about this.

Another thing is actual pagination. How does this scale if we have to fetch more items? Another thing I need to think more of.

If you have any ideas regarding any of the mentioned above. Please contact me!

Right now this is all just theory, but I might take a spin at trying to implement this soon over here: https://github.com/n1ru4l/graphql-live-query

Collecting GraphQL Live Query Resource Identifier with GraphQL Tools

Laurin Quast — Mon, 28 Sep 2020 17:55:32 +0000

Photo by Łukasz Nieścioruk on Unsplash

GraphQL live queries can solve real-time updates in a more elegant way than GraphQL subscriptions.

Instead of subscribing to events live queries primarily subscribe to data changes.

Instead of updating the client store manually a live query updates the client store magically without any redundant cache update logic.

You can learn more about the differences here

All those benefits, however, come with the drawback of the server having to become stateful, in particular, being aware of all the data the client operation consumes and re-executing those query operations for a specific client once the underlying data changes.

As I first started experimenting with GraphQL live queries the easiest solution was to simply trigger live query re-executions based on the Query object type root fields. E.g. a query with a selection set selection on the Query.viewer field could be re-executed by emitting the Query.viewer event through the live query store event emitter. However, the viewer could be a completely different record/resource for each client that consumes the given query operation.

To be more clear here is the corresponding schema:

type User {
  id: ID!
  login: String!
}

type Query {
  """
  Returns the authenticated user. Returns null in case the user is not authenticated.
  """
  viewer: User
  """
  List of the users that are currently online.
  """
  onlineUsers: [User!]!
}

type Mutation {
  updateLogin(newLogin: String!): Boolean!
}

query viewer @live {
  viewer {
    id
    login
  }
}

Let's see how the implementation for this could look like:

const Query = {
  viewer: (source, args, context) => {
    return context.viewer;
  },
};

const Mutation = {
  updateLogin: async (source, args, context) => {
    await context.db.updateUser(
      context.viewer.id,
      args.newLogin
    );

    context.liveQueryStore.invalidate(
      `Query.viewer`
    );
    return true;
  },
};

If a specific user updates his login we shouldn't invalidate and re-execute any live query operation that has a viewer selection set for any connected user who might not even be affected by that change!

At the same time, the user could also be referenced in another operation e.g. a list of all available users (Query.onlineUsers). The Query.viewer event would not cover and schedule a re-execution for operations that select the user via that field.

There must be a better solution for uniquely identifying the selection set data

As you probably noticed the user has an id field of the ID! (nonnull id) type. This is a commonly used field for uniquely identifying a resource on the client-side. Apollo-client uses the __typename field in combination with the id field as the default resource cache key (User:1), Relay goes a step further and already assumes that the resource type is already encoded (e.g. base64("User:1") Note: You are not forced to use base64 🤔) inside the id and therefore only uses the id field.

What if we could also use such an identifier on the server-side in our live query store implementation?

My current implementation just traversed the AST of the query operation and extracted the schema coordinates on the root query type. E.g. Query.viewer for the viewer live query operation from above.

However, in case we would want to identify the user via the id we must also add something like User:1 to the set of resources the live query operation selects. This requires schema knowledge as the live query store needs to know which type has an id field and if included in the selection set, gather the corresponding resource identifier.

As mentioned above this allows more granular query invalidations.

The first drawback I had in mind is that if an operation does not specify the id field on the selection set, the resource cannot be tracked by the live query store.

However, most operations will probably select the id field as it is most likely used on the client for the cache keys.

Furthermore, it could be possible to simply transform the query in such a way that the id field is added to the selection set (similar to how apollo-client is by default adding a __typename selection to each object type).

For keeping things simple I decided to push the responsibility for selecting the id field to the client that sends the live query operation. I also could not find a use-case in my existing application where there was no id selection for a resource 👍.

Implementing the resource identifier collector

The next obstacle is to decide how the ids are extracted and I had two options in mind.

1. Traversing the GraphQL execution result tree

This simply seemed complicated to me as I would need to traverse the whole result while somehow guessing/checking the type of each leaf based on the operation AST and the schema. I quickly dropped that idea.

2. Manually register the resource identifier by calling a function that is injected via the context

The goal of my live query store implementation is to add live query support to any schema with minimal effort. Passing something along-side the context that a library user must call inside a query resolver seemed wrong and all this should be an implementation detail the library user should not care about.

Imagine if we had to register a resource manually in each resolver that returns an object type.

const Query = {
  viewer: (source, args, context) => {
    const viewer = context.viewer;
    context.registerResource(`User:${viewer.id}`);
    return viewer;
  },
};

It might seem quite simple for a single resolver, however, it can quickly clutter and lead to bugs if we have to manually do that for any resource in any resolver.

Ideally a library user will just have to add a context.liveQueryStore.invalidate("User:1") line to the updateLogin mutation field resolver in order to magically schedule an operation re-execution, without the overhead of adding an additional function call to each resolver.

const Query = {
  viewer: (source, args, context) => {
    // No tracking registration code here.
    return context.viewer;
  },
};

const Mutation = {
  updateLogin: async (source, args, context) => {
    await context.db.updateUser(
      context.viewer.id,
      args.newLogin
    );

    context.liveQueryStore.invalidate(
      `User:${context.viewer.id}`
    );
    return true;
  },
};

So, I thought more about how this could be implemented in a less verbose way.

As any other field, the id field has a resolver (either the default resolver provided by GraphQL or a user-defined resolver), so if there was a way to wrap each id field resolver with a function that could solve the issue. The wrapper could call the actual resolver, register the resource, and then return the value. The user won't have to care about anything (besides adding the id field to the selection set of the query).

The best library for transforming and modifying GraphQL Schemas is graphql-tools. Fortunately, it is now maintained by The Guild, as apollo abandoned it and was maintained pretty poorly.

So I dug a bit into the fancy documentation and quickly found what I needed: @graphql-tools/wrap.

A quick excerpt from the documentation:

Schema wrapping is a method of making modified copies of GraphQLSchema objects, without changing the original schema implementation.

As the schema is also used for "normal" queries/mutations/subscription operations. I didn't want the overhead of wrapping all id fields for non-live query operations.

With the TransformObjectFields transform, it is pretty straight forward to wrap the schema fields.

import {
  GraphQLSchema,
  isScalarType,
  isNonNullType,
  GraphQLOutputType,
  GraphQLScalarType,
  execute,
} from "graphql";
import { wrapSchema, TransformObjectFields } from "@graphql-tools/wrap";

const isNonNullIDScalarType = (
  type: GraphQLOutputType
): type is GraphQLScalarType => {
  if (isNonNullType(type)) {
    return isScalarType(type.ofType) && type.ofType.name === "ID";
  }
  return false;
};

const addResourceIdentifierCollectorToSchema = (
  schema: GraphQLSchema
): GraphQLSchema =>
  wrapSchema(schema, [
    new TransformObjectFields((typename, fieldName, fieldConfig) => {
      let isIDField = fieldName === "id" && isNonNullIDScalarType(fieldConfig.type);

      let resolve = fieldConfig.resolve;
      fieldConfig.resolve = (src, args, context, info) => {
        if (!context || !context[ORIGINAL_CONTEXT_SYMBOL]) {
          return resolve(src, args, context, info);
        }

        const collectResourceIdentifier = context.collectResourceIdentifier;
        context = context[ORIGINAL_CONTEXT_SYMBOL];
        const result = resolve(src, args, context, info);
        if (isIDField) {
          if (isPromise(result)) {
            result.then(
              (value) => collectResourceIdentifier({ typename, value }),
              () => undefined
            );
          } else {
            collectResourceIdentifier({ typename, result });
          }
        }
        return result;
      };

      return fieldConfig;
    }),
  ]);

The implementation for executing the operation is similar to the following:

const newIdentifier = new Set(rootFieldIdentifier);
const collectResourceIdentifier: ResourceGatherFunction = ({ typename, id }) =>
  // for a relay spec conform server the typename could even be omitted :)
  newIdentifier.add(`${typename}:${id}`);

// You definitely wanna cache the wrapped schema as you don't want to re-create it for each operation :)
const wrappedSchema = addResourceIdentifierCollectorToSchema(schema);

const result = execute({
  schema: wrappedSchema,
  document: operationDocument,
  operationName,
  rootValue,
  contextValue: {
    [ORIGINAL_CONTEXT_SYMBOL]: contextValue,
    collectResourceIdentifier,
  },
  variableValues: operationVariables,
});

I had to wrap the "user" context in a context (context-ception 🤯) on which I also attached the function for adding the resource identifier to the resource identifier set. I got inspired for this by the apollo-server source code, as I knew it has a way for measuring resolver execution time, which must be done on a request/operation basis similar to the resource identifier collection. This method allows using a new function/context for each execution. Inside the field resolver, the correct user context is then passed into the actual (user) field resolver.

Now after the operation has been executed against the schema the newIdentifier Set should contain the identifiers of all the resources that were resolved during the operation execution.

The live query store can now use that information for re-executing queries once a resource identifier event is emitted 👌.

Conclusion

Identifying resources and invalidating queries based on a resource basis rather than a query root field basis allows more efficient query re-executions and can avoid pushing unnecessary updates to clients.

GraphQL Tools is a super handy library that can be used for solving a huge variety of problems. I am glad it got such a huge update and good documentation!

The implementation probably won't cover all use-cases. What if a client is not authenticated and the Query.viewer resolver returns null. There is no User:ID string available on the live query store operation context once the user has authenticated. Either a Query.viewer update must be emitted through the live query store emitter (which will affect ANY client operation that selects the viewer), the client must re-execute the operation after login or the live query store must somehow be notified to re-execute all operations of the user that just authenticated.

In case you are interested in the source code for the implementation check out https://github.com/n1ru4l/graphql-live-queries/pull/94

There is still more to discover and build in live query land!

We still need to manually notify the live query store that a resource must be invalidated. An abstraction for doing this behind the scenes could vastly differ for different stacks.

Maybe the ORM/database store layer could emit the events or a proxy could emit those events based on database operations such as INSERT, DELETE, and UPDATE.

Re-executing a query operation is nice and smart, but not the most efficient solution. What if we could only re-execute certain resolvers? I already have some ideas in mind and I will probably write about that as well!

Check out this super cool talk about live queries @ Facebook!

Check out this super cool talk about live queries @ Samsara!

I also wrote an article about my Socket.io GraphQL Server Engine implementation!

If you are interested in discussing live queries or GraphQL in general feel free to contact me on Twitter or by writing a comment below 🙂. Please also consider sharing the article with people that might be interested in it 😉.

GraphQL Live Queries with Socket.io

Laurin Quast — Mon, 17 Aug 2020 20:35:26 +0000

Photo by Luca Campioni on Unsplash

Note: For a better understanding I recommend reading Subscriptions and Live Queries - Real-Time with GraphQL first.

I have been using a GraphQL schema served over Socket.io for a long time. Finally, I abstracted the protocol for both client and server into a library for easier re-use across projects.

In addition to the usual GraphQL operations, I also added support for executing live queries.

TL;DR:

@n1ru4l/socket-io-graphql-server: A layer for serving a GraphQL schema via a socket.io server. Supports Queries, Mutations, Subscriptions, and Live Queries.
@n1ru4l/socket-io-graphql-client. A network interface for consuming a GraphQL schema that is served via @n1ru4l/socket-io-graphql-server. Can be used with all major GraphQL clients such as Relay, Apollo Client, or Urql.
@n1ru4l/graphql-live-query. Utilities for adding live queries to ANY GraphQL schema.
@n1ru4l/in-memory-live-query-store. GraphQL live query implementation.

All the packages can be found in this repository:

https://github.com/n1ru4l/graphql-live-queries

For showcasing the libraries I created a todo example app that syncs its state across all clients using the above packages:

Server: todo app server implementation with graphql-js, @n1ru4l/socket-io-graphql-server and @n1ru4l/in-memory-live-query-store.
Client Relay: Todo app client implementation with create-react-app, relay and @n1ru4l/socket-io-graphql-client.
Client Apollo: Todo app client implementation with create-react-app, @apollo/client and @n1ru4l/socket-io-graphql-client.
Client Urql: Todo app client implementation with create-react-app, urql and @n1ru4l/socket-io-graphql-client.

Motivation

It seems like real-time for GraphQL is not pushed as much as it should by all the big players out there.

The most popular Node.js implementation for subscriptions is poorly maintained as apollo focuses more on different fields.

There a few live query implementations and NONE that are not tied to a specific database out there.

A few cool ideas popped up in the community (such as graphql-live-subscriptions. But none of those are maintained or have some major flaws such as not being compatible with interface or union types.

The implementations in @n1ru4l/graphql-live-query and @n1ru4l/in-memory-live-query-store should serve as an example of how it could be done without being tied to any specific (reactive) database or data structure. The implementation will hopefully mature with time as people report new use-cases and start adopting live queries.

In addition to that, I created both @n1ru4l/socket-io-graphql-server and @n1ru4l/socket-io-graphql-client as I was already heavily using GraphQL over Socket.io in a project that required real-time updates.

GraphQL already has a solution for real-time

Subscriptions are the perfect tool for responding to events. An example of this would be triggering a sound or showing a toast message once a new message has been received.

Subscriptions are also often used for updating existing query results on the client. Depending on the complexity cache update code can eventually become pretty bloated. Often it is more straight-forward to simply refetch the query once a subscription event is received.

Live Query Magic

Live queries however should feel magical and update the UI with the latest data from the server without having to do any cache update wizardry. This moves the complexity from the client to the server.

Concept

The current definition of a live query for my project is a query operation that is annotated with a @live directive.

query users @live {
  users(first: 10) {
    id
    login
  }
}

A live query is sent to the server (via WebSocket or HTTP) and stored there until the client disconnects or notifies the server he is no longer interested in the live query operation (and hence the server disposing of it).

On the server, the query is re-executed once the data selected by the live query operation changes. The result is then streamed to the client. A re-execution can be scheduled by invalidating a selected resource of the result of the previous execution. The invalidation can either be triggered by using schema coordinates for the root field (e.g. Query.todos) or by a resource identifier (e.g. Todo:1, todo with the id 1). UPDATE: You can learn more how the library collects the resource identifiers here..

LiveQueryStore Implementation

The InMemoryLiveQueryStore is a class provided by the @n1ru4l/in-memory-live-query-store package InMemoryLiveQueryStore which stores all the information about the active live queries in memory.

Registering the LiveQueryStore

In order to enable live queries for the @n1ru4l/socket-io-graphql-server implementation a excute function that returns a AsyncIterator must be passed to the registerSocketIOGraphQLServer function.

In addition to the default execute function from the graphql-js reference implementation, the new execute function can also return a AsyncIterableIterator<ExecutionResult>. Note: As @defer and @stream is added to the graphql-js reference implementation it now also can return AsyncIterators.

The InMemoryLiveQueryStore has the execute property which must be used for executing the live queries (but also non-live query operations by falling back to the default execute from graphql-js if the operation is not identified as a live query operation).

import socketIO from "socket.io";
import { InMemoryLiveQueryStore } from "@n1ru4l/in-memory-live-query-store";
import { registerSocketIOGraphQLServer } from "@n1ru4l/socket-io-graphql-server";
import { schema } from "./schema";

const liveQueryStore = new InMemoryLiveQueryStore();
const server = socketIO(httpServer);

registerSocketIOGraphQLServer({
  socketServer,
  // getExecutionParameter is invoked for each incoming operation.
  // a different context or even schema based on the connection can be returned
  // and will be used for that operation.
  getExecutionParameter: () => ({
    execute: liveQueryStore.execute,
    graphQLExecutionParameter: {
      schema,
      contextValue: {
        liveQueryStore,
      },
    },
  }),
});

There is a WIP pull request on the graphql-js reference implementation for adding AsyncIterableIterator<ExecutionResult> as a valid return result from execute for @defer and @stream directives. As a side effect, this would also help live queries.

Notifying the live query store for changes

In order to re-execute the registered live queries and stream the new results to the connected clients, we must invalidate the live query operations, that select the specific data.

For the given query:

query messages @live {
  messages(first: 10) {
    id
    content
    author {
      id
      name
    }
  }
}

That would look like this:

// Mutation.createMessage resolver
const createMessage = async (root, args, context) => {
  await context.messageStore.createMessage({
    content: args.content,
    userId: context.viewer.id,
  });
  // notify liveQueryStore via query schema coordinate that all live queries that select Queries.messages must be re-executed and sent to the clients.
  context.liveQueryStore.invalidate("Query.messages");
};

const editMessage = async (root, args, context) => {
  await context.messageStore.updateMessage({
    messageId: args.messageId,
    content: args.content,
  });
  // notify liveQueryStore via resource identifier
  context.liveQueryStore.invalidate(`Message:${args.messageId}`);
}

Note: It might be possible to do some kind of abstraction here. Imagine the live query store living on your mesh entry point keeping track of the live queries and mutations occurring and then automatically triggering the invalidations based on mutations instead of hard coding it into resolvers. Apollo recently build something similar but quite a bit different.

Adding the `@live` directive to your GraphQL schema

@n1ru4l/graphql-live-query exports a directive that can easily be added to an existing schema. You must add it, otherwise, the GraphQL server will complain about unknown directives in your GraphQL operations.

import { GraphQLLiveDirective } from "@n1ru4l/graphql-live-query";

export const schema = new gql.GraphQLSchema({
  query,
  mutation,
  subscription,
  directives: [GraphQLLiveDirective],
});

For people that are using a GraphQL SDL driven development flow, you must add the following to your type definitions.

directive @live on QUERY

Consuming live queries on the client

The @n1ru4l/socket-io-graphql-client package can be used to execute (live) Query, Mutation and Subscription operations on to the server setup by @n1ru4l/socket-io-graphql-server. It implements the underlying GraphQL over Socket.io protocol.

I also created a PR for supporting live queries with graphql-transport-ws.

Basic client creation

import io from "socket.io-client";
import { createSocketIOGraphQLClient } from "@n1ru4l/socket-io-graphql-client";

const socket = io();

const socketIOGraphQLClient = createSocketIOGraphQLClient(socket);

Executing GraphQL operations

The SocketIOGraphQLClient provides an execute method that will return an Observable which can be used to subscribe to the response.

A simple query or mutation operation will only publish one value. However, a live query or subscription will publish multiple values, therefore a Promise is not the perfect data structure for this.

The observable returned is compatible with the proposal spec and is easily consumable by libraries such as apollo-client and relay. But also by tools like GraphiQL.

socketIOGraphQLClient.execute({
  operation: /* GraphQL */ `
    query messages {
      id
      content
    }
  `
}, {
  next: console.log,
  error: console.log,
  complete: console.log
});

socketIOGraphQLClient.execute({
  operation: /* GraphQL */ `
    query messages @live {
      id
      content
    }
  `
}, {
  next: console.log,
  error: console.log,
  complete: console.log
});

socketIOGraphQLClient.execute({
  operation: /* GraphQL */ `
    subscription onNewMessage {
      onNewMessage {
        id
        content
      }
    }
  `
}, {
  next: console.log,
  error: console.log,
  complete: console.log
});

GraphiQL fetcher

It is possible to easily consume and display the results of the GraphQL API with our SocketIOGraphQLClient. We just need to pass a custom fetcher to the GraphiQL component.


const fetcher = ({ query: operation, ...restGraphQLParams }) =>
  ({
    subscribe: (
      sinkOrNext,
      ...args
    ) => {
      const sink: Sink =
        typeof sinkOrNext === "function"
          ? { next: sinkOrNext, error: args[0], complete: args[1] }
          : sinkOrNext;

      const unsubscribe = socketIOGraphQLClient.execute(
        {
          operation,
          ...restGraphQLParams,
        },
        sink
      );

      return { unsubscribe };
    },
  });

const CustomGraphiQL = () => (
  <GraphiQL
    fetcher={({ query: operation, ...execRest }) =>
      socketIOGraphQLClient.execute({ operation, ...execRest })
    }
  />
);

Implementation with Relay

Relay is a powerful library for managing the cache client-side. A relay environment (which holds information about the cache and how the data is fetched from the server) can easily be built around a SocketIOGraphQLClient instance.

import { SocketIOGraphQLClient } from "@n1ru4l/socket-io-graphql-client";
import {
  Environment,
  Network,
  RecordSource,
  Store,
  Observable,
  GraphQLResponse,
  RequestParameters,
  Variables,
} from "relay-runtime";

export const createRelayEnvironment = (
  networkInterface: SocketIOGraphQLClient<GraphQLResponse, Error>
) => {
  const execute = (request: RequestParameters, variables: Variables) => {
    if (!request.text) throw new Error("Missing document.");
    const { text: operation, name } = request;

    return Observable.create<GraphQLResponse>((sink) =>
      networkInterface.execute(
        {
          operation,
          variables,
          operationName: name,
        },
        sink
      )
    );
  };

  const network = Network.create(execute, execute);
  const store = attachNotifyGarbageCollectionBehaviourToStore(
    new Store(new RecordSource())
  );

  return new Environment({
    network,
    store,
  });
};

Consuming live data then becomes straight forward:

const ChatApplicationMessagesQuery = graphql`
  query ChatApplication_MessagesQuery @live {
    messages(limit: 10) {
      id
      ...ChatApplication_message
    }
  }
`;

const ChatApplicationMessageRenderer = React.memo(
  ({ message }: { message: ChatApplication_message }) => {
    return (
      <div>
        <div>{message.author.name}</div>
        <div>{message.content}</div>
      </div>
    );
  }
);

const ChatApplicationMessage = createFragmentContainer(
  ChatApplicationMessageRenderer,
  {
    message: graphql`
      fragment ChatApplication_message on Message {
        id
        content
        author {
          id
          name
        }
      }
    `,
  }
);

export const ChatApplication: React.FunctionComponent<{
  relayEnvironment: RelayEnvironment;
}> = (props) => {
  return (
    <QueryRenderer<ChatApplication_MessagesQuery>
      environment={props.relayEnvironment}
      query={ChatApplicationMessagesQuery}
      variables={{}}
      render={({ props }) => {
        if (!props) {
          return null;
        }

        return props.messages.map((message) => (
          <ChatApplicationMessage key={message.id} message={message} />
        ));
      }}
    />
  );
};

See complete example app

Usage with Apollo Client

import { SocketIOGraphQLClient } from "@n1ru4l/socket-io-graphql-client";
import {
  ApolloClient,
  InMemoryCache,
  ApolloLink,
  Operation,
  Observable,
  FetchResult,
  Observable,
} from "@apollo/client";
import { print } from "graphql";

class SocketIOGraphQLApolloLink extends ApolloLink {
  private networkLayer: SocketIOGraphQLClient;
  constructor(networkLayer: SocketIOGraphQLClient) {
    super();
    this.networkLayer = networkLayer;
  }

  public request(operation: Operation): Observable<FetchResult> | null {
    return new Observable((sink) =>
      this.networkLayer.execute({
        operationName: operation.operationName,
        operation: print(operation.query),
        variables: operation.variables,
      })
    );
  }
}

export const createApolloClient = (networkInterface: SocketIOGraphQLClient) => {
  return new ApolloClient({
    link: new SocketIOGraphQLApolloLink(networkInterface),
    cache: new InMemoryCache(),
  });
};

See complete example app

Usage with Urql

import { SocketIOGraphQLClient } from "@n1ru4l/socket-io-graphql-client";
import {
  Client,
  dedupExchange,
  cacheExchange,
  subscriptionExchange,
  ExecutionResult,
} from "urql";

export const createUrqlClient = (
  networkInterface: SocketIOGraphQLClient<ExecutionResult>
) => {
  return new Client({
    url: "noop",
    exchanges: [
      dedupExchange,
      cacheExchange,
      subscriptionExchange({
        forwardSubscription: (operation) => ({
          subscribe: (sink) => ({
            unsubscribe: networkInterface.execute(
              {
                operation: operation.query,
                variables: operation.variables,
              },
              sink
            ),
          }),
        }),
        enableAllOperations: true,
      }),
    ],
  });
};

See complete example app

The Future

This is the first implementation of the live query library. As more people try it out and build projects with it the API can become more mature and flexible for different use-cases.

New LiveQueryStore implementations for distributed systems could be built (e.g. based on Redis PubSub).

The network layer could furthermore be optimized to only transport the updated patch instructions to the client in order to decrease the payload size.

Other people might not use Socket.io. (Hybrid) implementations for other server libraries out there e.g. (apollo-server or express-graphql) must be built.

Note: GraphQL live queries can now be executed with graphql-ws and even express-graphql experimental. Check out GraphQL Bleeding Edge Playground!

Do you have an idea? Feel free to contact me per Twitter, open a GitHub issues or write a comment down below 😊.

Homebrew React Hooks: useIsKeyPressed

Laurin Quast — Fri, 03 Apr 2020 06:22:36 +0000

Cover Art by Enrique Grisales

It has been a while in this series...

But now I am picking it up again ☺️.

The first two entries in this series were a bit complex, so in order to get things rolling again, I will start with a simple, but also a bit tricky hook (Spoiler: Cross-Platform issues 😖)!

Ever wanted to know whether a key is pressed? useIsKeyPressed will be your friend 👌.

Let's start with defining the interface of our hook:

The input should be a string that identifies the key, such as a, Alt, Enter.

The return value should be a boolean that indicates whether the key is currently pressed or not.

Key is pressed: return true
Key is not pressed: return false

Okay, let's check the APIs we are going to use!

We will use the keydown and keyup events for our event handlers.

Usually, we attach event listeners in React by passing a Function to the HTML primitive JSX element.

import React from "react";

const MyComponent = () => {
  const handler = () => {
    console.log("hey")
  }
  return <div onKeyDown={handler} />
}

However, this will only trigger the event listener in case the events are triggered within the element. For our use-case, we gonna implement a global event listener that is registered on the global Window object.

Let's build our hook:

import React from "react";

const useIsKeyPressed = (key) => {
  const [isKeyPressed, setIsKeyPressed] = React.useState(false);

  React.useEffect(() => {
    setIsKeyPressed(false);
    const onKeyDown = (ev) => {
      if (ev.key === key) setIsKeyPressed(true);
    };
    const onKeyUp = (ev) => {
      if (ev.key === key) setIsKeyPressed(false);
    };
    window.addEventListener("keydown", onKeyDown);
    window.addEventListener("keyup", onKeyUp);

    return () => {
      window.removeEventListener("keyup", onKeyUp);
      window.removeEventListener("keydown", onKeyDown);
    };
  }, [key]);

  return isKeyPressed;
}

We ensure that the isKeyPressed value is set to false in case the key parameter has changed by calling setIsKeyPressed at the start of our hook. In case the setState (setIsKeyPressed) function is called with the same value as the state (isKeyPressed), this will not trigger any unnecessary re-renders, because it is strict-equal, which is cool as we don't need to add any "if-statement" noise 👌.

After using this hook in production for some time I experienced a bug which was reported by a Windows user:

I used the hook for tracking whether the Alt key is pressed. On Windows you can tab between windows with the key shortcut Alt + Tab.

This combination resulted in the isKeyPressed value being updated to true, but not back to false as the keyup event was not triggered on the window object.

After he pressed the Alt key again when switching back to the browser window, everything worked again.

I did some research on how to solve this issue and first thought about setting up an interval timer that checks whether a key is still pressed after some time.

While doing that research I also learned that there is no API for checking whether a key is pressed or not. It is only possible by setting up listeners for the keydown and keyup events 😔.

So my next idea was to somehow detect when the browser window is unfocused. I found some crazy methods like running requestAnimationFrame and checking whether the delay between calls is around one second as it is throttled when not focused.

Fortunately, there is a simpler solution which just requires us to setup one more event listener, the blur event.

It seems like the blur event is fired on the window element when minimizing the window, pushing it to the background or any similar action.

We can adjust our hook to just set the isKeyPressed value back to false upon blur.

import { useState, useEffect } from "react";

export const useIsKeyPressed = (key) => {
  const [isKeyPressed, setIsKeyPressed] = useState(false);

  useEffect(() => {
    setIsKeyPressed(false)
    const onKeyDown = (ev) => {
      if (ev.key === key) setIsKeyPressed(true);
    };
    const onKeyUp = (ev) => {
      if (ev.key === key) setIsKeyPressed(false);
    };
    const onBlur = () => {
      setIsKeyPressed(false);
    };

    window.addEventListener("keydown", onKeyDown);
    window.addEventListener("keyup", onKeyUp);
    window.addEventListener("blur", onBlur);

    return () => {
      window.removeEventListener("keyup", onKeyUp);
      window.removeEventListener("keydown", onKeyDown);
      window.removeEventListener("blur", onBlur);
    };
  }, [key]);

  return isPressed;
};

Another thing I realized quickly afterward is that a keyup event is not triggered in case you press the cmd (Meta) key in addition the any other key on MacOS.

The workaround for this is always set isKeyPressed to false when the keyup key is the Meta key. This will result in a true -> false -> true state change, which is not optimal but still better than being stuck in the isKeyPressed true and having to press the key again to leave it.

I am curious if there are other methods to prevent this, let me know your thoughts in the comments below ⬇️.

Let's take a look at our final hook:

import React from "react";

const useIsKeyPressed = (key) => {
  const [isKeyPressed, setIsKeyPressed] = React.useState(false);

  React.useEffect(() => {
    setIsKeyPressed(false);
    const onKeyDown = (ev) => {
      if (ev.key === key) setIsKeyPressed(true);
    };
    const onKeyUp = (ev) => {
      if (ev.key === key || ev.key === "Meta") setIsKeyPressed(false);
    };
    const onBlur = (ev) => {
      setIsKeyPressed(false);
    };
    window.addEventListener("keydown", onKeyDown);
    window.addEventListener("keyup", onKeyUp);
    window.addEventListener("blur", onBlur);

    return () => {
      window.removeEventListener("keyup", onKeyUp);
      window.removeEventListener("keydown", onKeyDown);
      window.removeEventListener("blur", onBlur);
    };
  }, [key]);

  return isKeyPressed;
};

**Demo-Time Bonus 🎉:

Thank you for reading ☺️

Configure the create-react-app public URL post-build with Node.js and express

Laurin Quast — Sun, 29 Mar 2020 11:56:54 +0000

Cover Art by Joshua Welch

For a project, I had the requirement of making the public URL of the application configurable via an environment variable that can be set before starting the Node.js express server.

The frontend of that application is built with create-react-app which is very opinionated and has some limitations.

I will share with you those limitations and the workarounds for implementing such a feature.

The Defaults

create-react-app assumes that your application is hosted on the server root. E.g. the URL to your favicon in the build output index.html file would look similar to this the following:

<link rel="shortcut icon" href="/favicon.ico"/>

In case you want to host your website under a relative part that is different from the server root there is an option for specifying the base URL either via the homepage key inside your package.json or the PUBLIC_URL environment variable that must be set before building the project. When running the react-scripts build script, the %PUBLIC_URL% placeholders inside the index.html file are replaced with the environment variable string.

In case we want to serve our application under a different public URL, such as https://my-site.com/app, we can build the project like that:

PUBLIC_URL=https://my-site.com/app yarn react-scripts build

The contents of the build artifact index.html have now changed:

<link rel="shortcut icon" href="https://my-site.com/app/favicon.ico"/>

The limitations

This method, however, has the drawback of requiring us to already know the public URL when building the frontend application.

As mentioned earlier our use-case requires setting the public URL dynamically, as the express server that is bundled as a binary and each user should be able to run that web server under a domain/path they specify.

The solution

The initial idea was to set PUBLIC_URL to some string that could get replaced by the express web server. The Node.js script loads the index.html file and replaces all the occurrences of the placeholder string:

"use strict";

const express = require("express");
const app = express();
const path = require("path");
const fs = require("fs");

const PUBLIC_PATH = path.resolve(__dirname, "build");
const PORT = parseInt(process.env.PORT || "3000", 10)
const PUBLIC_URL = process.env.PUBLIC_URL || `http://localhost:${PORT}`;

const indexHtml = path.join(PUBLIC_PATH, "index.html");
const indexHtmlContent = fs
  .readFileSync(indexHtml, "utf-8")
  .replace(/__PUBLIC_URL_PLACEHOLDER__/g, PUBLIC_URL);

app.get("/", (req, res) => {
  res.send(indexHtmlContent);
});

app.use(express.static(path.join(PUBLIC_PATH)));

app.listen(PORT);

Now we can build our app like this:

PUBLIC_URL=__PUBLIC_URL_PLACEHOLDER__ yarn react-scripts build

However, this only solves linking the assets correctly. From an application point of view, we also need to figure out the application root path. Here is a quick example of our Image component:

const Image = () =>
  <img src={`${process.env.PUBLIC_URL}images/me.jpeg`} />

Because we specified PUBLIC_URL being set to __PUBLIC_URL_PLACEHOLDER__ and the environment variable is also embedded inside the JavaScript bundles (and used for resolving asset paths) the server will now send requests to __PUBLIC_URL_PLACEHOLDER__/images/me.jpeg 😅.

If we search for the string __PUBLIC_URL_PLACEHOLDER__ inside the build assets located at build/static/js we can find multiple occurrences.

create-react-app injects an environment object inside the bundle that is similar to the Node.js process.env object.

process.env = {
  NODE_ENV: "production",
  PUBLIC_URL: "__PUBLIC_URL_PLACEHOLDER__/"
}

In order to have a viable solution we also need to replace those occurrences on that object with the correct URL.

But parsing those .js files while serving them and replacing the string with express is not a good option as we now need to do it either on each request or cache the file contents in memory or in a separate file.

After some thinking, I realized there is a better option available that would allow us to only replace the .js content once post-build.

First, we add the following to our index.html file:

<script>
  window.__PUBLIC_URL__ = "";
</script>

Make sure to add it into the head of the document to ensure it is loaded/evaluated before our application .js bundles.

Up next we must transform the process.env definition to the following:

process.env = {
  NODE_ENV: "production",
  PUBLIC_URL: window.__PUBLIC_URL__ + "/"
}

We can achieve that by writing a script that will replace the occurrence of the __PUBLIC_URL_PLACEHOLDER__ string inside our build/static/js/*.js files with window.__PUBLIC_URL__. That script can be executed immediately after running yarn react-scripts build.

I found a cool library replacestream, that allows replacing file contents while streaming it. This keeps the memory footprint low for bigger application bundles.

// scripts/patch-public-url.js
"use strict";

const fs = require("fs");
const path = require("path");
const replaceStream = require("replacestream");

const main = async () => {
  const directory = path.join(__dirname, "..", "build", "static", "js");
  const files = fs
    .readdirSync(directory)
    .filter(file => file.endsWith(".js"))
    .map(fileName => path.join(directory, fileName));

  for (const file of files) {
    const tmpFile = `${file}.tmp`;
    await new Promise((resolve, reject) => {
      const stream = fs
        .createReadStream(file)
        .pipe(
          replaceStream(
            '"__PUBLIC_URL_PLACEHOLDER__"',
            // the whitespace is added in order to prevent invalid code:
            // returnwindow.__PUBLIC_URL__
            " window.__PUBLIC_URL__ "
          )
        )
        .pipe(
          replaceStream(
            '"__PUBLIC_URL_PLACEHOLDER__/"',
            // the whitespace is added in order to prevent invalid code:
            // returnwindow.__PUBLIC_URL__+"/"
            ' window.__PUBLIC_URL__+"/"'
          )
        )
        .pipe(fs.createWriteStream(tmpFile));
      stream.on("finish", resolve);
      stream.on("error", reject);
    });
    fs.unlinkSync(file);
    fs.copyFileSync(tmpFile, file);
    fs.unlinkSync(tmpFile);
  }
};

main().catch(err => {
  console.error(err);
  process.exitCode = 1;
});

Let's also replace the window.__PUBLIC_URL__ assignment inside our index.html within the Node.js code.

"use strict";

const express = require("express");
const app = express();
const path = require("path");
const fs = require("fs-extra");

const PUBLIC_PATH = path.resolve(__dirname, "build");
const PORT = parseInt(process.env.PORT || "3000", 10)
const PUBLIC_URL = process.env.PUBLIC_URL || `http://localhost:${PORT}`;

const indexHtml = path.join(PUBLIC_PATH, "index.html");
const indexHtmlContent = fs
  .readFileSync(indexHtml, "utf-8")
-  .replace(/__PUBLIC_URL_PLACEHOLDER__/g, PUBLIC_URL);
+  .replace(/__PUBLIC_URL_PLACEHOLDER__/g, PUBLIC_URL)
+  .replace(/window\.__PUBLIC_URL__=""/, `window.__PUBLIC_URL__="${PUBLIC_URL}"`);

app.get("/", (req, res) => {
  res.send(indexHtmlContent);
});

app.use(express.static(path.join(PUBLIC_PATH)));

app.listen(PORT);

Let's also adjust our build script inside the package.json:

PUBLIC_URL=__PUBLIC_URL_PLACEHOLDER__ react-scripts build && node scripts/patch-public-url.js

Post-build, we can start our server like this:

PUBLIC_URL=http://my-site.com/app node server.js

Bonus 🎁: NGINX Reverse Proxy Configuration

upstream app {
    server localhost:3000;
}
server {
    listen       80;
    server_name  my-site.com;

    location /app {
        rewrite ^/app(/.*)$ $1 break;
        proxy_pass http://app/;
        # We also sue WebSockets :)
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
    }
}

Further note regarding service workers:

In case you inspected the build folder and searched for __PUBLIC_URL_PLACEHOLDER__, you probably noticed that there are also service-worker .js files and also an asset-manifest.json file that includes the given string. I currently do not care about those, because our application has no offline mode. If you consider this, you will probably have to make some more string replacements.

Furthermore, since we are exclusively using CSS in JS I did not do any CSS string replacements. If you do so and use the url() you might also need to adjust your CSS files.

We have finished 🎉.

Do you have anything to add to that method, found a typo or got a better method for doing the same thing? Drop your comment and start a discussion below ⬇

Thank you so much for reading!

Incrementally migrating a CRA application to TypeScript without ejecting

Laurin Quast — Wed, 26 Feb 2020 19:45:58 +0000

Cover Art by 周康

Update (29.02.2020):

I was notified by whatwhatwhatwhatwhut on Reddit
that it is way easier to add TypeScript support! We will just have to follow these steps mentioned in the documentation.

TL;DR:

yarn add typescript @types/node @types/react @types/react-dom @types/jest

Rename any file from .js to .ts or .tsx (and adjust types).

Restart the development server.

This is definitely an improvement of the somehow cumbersome method below. So far I did not notice any drawbacks!

Original Content:

JavaScript Tooling is constantly improving. The power of CRA makes scaffolding a new React project easier than ever before. Just a few years ago the modern JavaScript developer had to fiddle around with Webpack configs and whatnot. Nowadays higher-level abstractions such as CRA made all the configuring obsolete.

Another tooling that has made a buss in the community is TypeScript. Inevitable, CRA added TypeScript support without having to eject (and therefore avoid dealing with Webpack configuration again).

There is nothing wrong with learning how to configure Webpack (it even got way easier by providing better defaults in more recent versions), but still, each second spent on worrying about Webpack is stealing our precious time that could rather be spent building the product. But let's get back to the topic 😅.

Unfortunately, it is still hard to incrementally adopt TypeScript in an existing "vanilla" CRA project without having to eject or renaming all the files which will result in a giant pull request and conflict with everything other people are working on right now.

For an existing project, I tried a new approach that I want to share with you!

Bootstrapping the TypeScript Config

The basic idea is to have a new folder ts_src that holds all our new TypeScript code. When we compile that code we want the output files built into our src directory so that our CRA development server will pick up the changes and reload our application running inside the browser.

mkdir -p ts_src
touch ts_src/tsconfig.json
echo '
{
  "compilerOptions": {
    "module": "ESNext",
    "target": "esnext",
    "lib": [
      "dom",
      "dom.iterable",
      "esnext"
    ],
    "noImplicitAny": false,
    "outDir": "./../src",
    "rootDir": "./",
    "sourceMap": true,
    "declaration": true,
    "strict": true,
    "allowSyntheticDefaultImports": true,
    "esModuleInterop": true,
    "strictNullChecks": true,
    "jsx": "react",
    "skipLibCheck": true,
    "moduleResolution": "node",
    "noEmit": true,
  }
}' > ts_src/tsconfig.json

Add TypeScript Build Script to `package.json`

We add one for building the whole application as well as one for development.

-  "build": "react-scripts build",
+  "build": "yarn build:ts && react-scripts build",
+  "build:ts": "yarn tsc --build src_ts/tsconfig.json",
+  "build:ts:watch": "yarn build:ts --watch",

Moving our file to `ts_src`

Then we can start moving existing files over to our new ts_src folder. We will keep the same folder structure as inside the src directory.

# ensure directory exists
mkdir -p $(dirname src_ts/path/to/file.js)
# move file to new path
mv src/path/to/file.js src_ts/path/to/file.ts

Add path `src/path/to/file.js` to `.gitignore` and `.eslintignore`

The file is now being built to src/path/to/file.js and therefore a build artifact, so there is no reason left for tracking the build artifact inside git.

We also must add the file to the .eslintignore because the react-scripts does also apply eslint on files listed in the .gitignore and the build artifacts could trigger eslint warnings/errors that will let our CI build fail.

echo 'src/path/to/file.js' >> .gitignore
echo 'src/path/to/file.js' >> .eslintignore
git rm --cached src/path/to/file.js

Repeat step for every file that is imported by `src_ts/path/to/file.ts`

Now we need to repeat the steps above for every file that is imported by the file we just moved over, otherwise, the TypeScript compiler will complain 😅. Ideally, we would start with migrating smaller files, that do not have many imports.

Add typings

Then we can finally add our types! I already caught some nasty bugs in some of the code I migrated by just adding types! Let me know what you encountered along the way in the comments 😉

Committing the changes changes

Nice, we just migrated a part of our application over to TypeScript.

For faster development iteration we can use the TypeScript compiler in watch mode (besides the yarn start command that spins up the CRA development server):

yarn build:ts:watch

Adding new functionality

We can now add new features in TypeScript inside our src_ts folder.

Before we create a new file we should also make sure the corresponding path inside src is not already taken by an existing file!

Someday in the future: the `src` folder is empty 🎉

Depending on the project size this could take a while, but hopefully, that day will finally come!

The migration is now almost over, there are no more JavaScript source files available and everything is now typed!

We can now delete the empty src folder and rename the ts_src to src.

rmdir src # ensure it is empty ;)
mv ts_src src

Next, we can move the TypeScript config to the root of the repository.

mv src/tsconfig.json tsconfig.json

And also apply some adjustments:

  {
    "compilerOptions": {
      "module": "ESNext",
      "target": "esnext",
      "lib": [
        "dom",
        "dom.iterable",
        "esnext"
      ],
       "noImplicitAny": false,
-      "outDir": "./../src",
-      "rootDir": "./",
       "declaration": true,
       "strict": true,
       "allowSyntheticDefaultImports": true,
       "esModuleInterop": true,
       "strictNullChecks": true,
       "jsx": "react",
       "skipLibCheck": true,
       "moduleResolution": "node",
       "noEmit": true,
-    }
+    },
+    "include": [
+     "src"
+    ]
  }

Then we can also finally remove all this .eslintignore and .gitignore entries we collected while migrating existing .js files (or adding new .ts files).

We are done 👌

This alternative approach might have some drawbacks such as adding entries to .eslintignore and .gitignore or having to migrate all files that depend on each other at once, but I think this is a great alternative towards traditional ejecting.

Many of the tasks could be enhanced further e.g. by creating a handy CLI that will automatically run the commands required for moving a file from src to src_ts (along with its imported files) while also adding the corresponding entries to the .eslintignore and .gitignore files.

In case I further explore such ideas I will definitely write about it or even publish it as an npm package.

Do you know a better method for migrating the project? Any other ideas on how the migration path can be improved further? Are you currently planning to migrate an application to TypeScript?

Let me know, down in the comments ⬇ and thank you so much for reading!

(How) do you test the synergy of multiple services?

Laurin Quast — Tue, 18 Feb 2020 18:51:17 +0000

Hey there! I just wanna hear some more opinions on testing services in order to ensure they are communicating with each other as expected.

For the foundation of the discussion let's assume we have three separate services.

Service A: Handles HTTP Requests, proxies requests to Service B; sends job requests to Service C
Service B: Returns a result based on data that has been processed by Service C
Service C: Processes some data and writes it somewhere else; receives job requests from Service A

Let's first talk through how a scenario could look like:

User sends HTTP request to Service A
Service A tries to load data from Service B
Service B tries to load data from a database
Data is non-existing in the database
Service B informs Service A that the data is not available
Service A notifies Service C that it should fetch the data
Service A informs User that the data is currently not available and he should retry later

Testing each service in isolation

We can (and probably should) test the public API of each service in isolation as a single unit.

We need to mock all the other services

If services are directly dependent e.g. by HTTP requests we need to spin up a server that sends back expected responses.

Will the service run in production?

How can we ensure that a Node.js application is still working when put inside a docker container for production deployment, e.g. we could have forgotten to install a dependency and it will fail on startup?

In order to guarantee this, we actually have to spin up the container and test its functionality before deploying it.

How can we ensure that two services are actually able to communicate with each other?

The tests could get out of sync because one person changed Service B but forgot to apply the changes to the test of Service A. The Service B-mock for the Service A-test is now wrong.

Both are no longer able to communicate with each other.

Testing each service in isolation actually gives us no guarantee that both services will be able to communicate with each other in production.

Testing the services together

In addition to writing tests for each service, we could also have some tests that spin up all our services and check whether they are able to deliver a result.

We now have to orchestrate different services and make sure they are all up and running (and healthy) before our tests start.

Luckily there are libraries like dockest (jest + docker containers) that help spinning up containers and to make sure the containers are ready before running the tests and other stuff like automatically failing tests in case a container unexpectedly dies.

However, CI duration can now drastically increase, because besides testing each container in isolation we now test them as a whole stack. It could increase even further in case the containers do some very complex and time-intensive tasks, like loading something from a data source, transforming it and writing it to a different data source.

One solution to that problem would be using smaller datasets than in production.

My Conclusion

Complex service interaction seems to be a very hard thing to test. Maybe we should try to avoid separating our products into different services, but sometimes there is no other option available.

We still need to do the testing in order to be sure our production system will not fail, due every service working fine in isolation but not in synergy with the other services.

I am curious about your input on this topic!

How do you ensure separate services are able to communicate with each other?

What tools do you use to do so?

Optimizing your GraphQL Operations with GraphQL Code Generator and the Relay Compiler

Laurin Quast — Mon, 15 Jul 2019 18:30:42 +0000

This week I have published a guest post on the The Guild Medium Publication.

The Relay Compiler is a great tool developed by Facebook for applying transforms to you GraphQL operations.

Are you curious on how the Relay Compiler Works?

I built this interactive REPL for you!

https://relay-compiler-repl.netlify.com

Learn more of the benefits and how you can use the Relay Compiler with the GraphQL Code Generator on The Guild Medium Publication:

https://medium.com/the-guild/optimizing-queries-with-the-graphql-code-generator-b8c37d692857

Homebrew React Hooks: useAsyncEffect Or How to Handle Async Operations with useEffect

Laurin Quast — Fri, 05 Jul 2019 11:41:23 +0000

TLDR

Async functions lack cancelability. We can use generator functions for mimicking cancelable async functions. I created a library for writing async effects: useAsyncEffect on Github

The Problem

Most of us love working with the async-await syntax!

Some of you (including me) might have tried executing the following piece of code

import { useState, useEffect } from "react";

const [state, setState] = useState()
// do not try this at home
useEffect(async () => {
  const data = await fetchSomeData()
  setState(data);
}, []);

And those who did so might also have noticed that this piece of code will print a big error message into the developer console:

Warning: An Effect function must not return anything besides a function, which is used for clean-up.

It looks like you wrote useEffect(async () => ...) or returned a Promise. Instead, you may write an async function separately and then call it from inside the effect:

async function fetchComment(commentId) {
  // You can await here
}

useEffect(() => {
  fetchComment(commentId);
}, [commentId]);

In the future, React will provide a more idiomatic solution for data fetching that doesn't involve writing effects manually.

Why does `useEffect` not accept my async functions?

The error message actually gives a clear explanation 😅. Let's break it down!

An async function always returns a Promise, thus you cannot synchronously return a cleanup function.
React calls the cleanup function when one of the dependencies of useEffect changes or the component unmounts.

Even if useEffect would support resolving cleanup functions from a Promise, that change could happen before the Promise has resolved (or even worse, rejected). As a result, the cleanup function would either be called too late or never.

Why would I even need a cleanup function anyways?

// Detect dark theme var iframe = document.getElementById('tweet-1146879490347470848-469'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1146879490347470848&theme=dark" }

Given this valid react useEffect usage:

const [data, setData] = useState();
useEffect(() => {
  const runEffect = async () => {
    const data = await fetchSomeData(filter);
    setData(data);
  };
  runEffect();
}, [setData, filter]);

Let's assume that the component unmounts while the fetchSomeData promise is still unresolved. That would mean setData is called despite the component already being unmounted.

You might remember the Can't call setState (or forceUpdate) on an unmounted component. warning from Class Components, this still applies to hooks.

Even worse, when the filter dependency changes before fetchSomeData resolves we have two race conditions colliding. What if for some reason the second fetchSomeData promise resolves before the first fetchSomeData promise? In that case, the "newer" data will be overwritten by the "old" data once the delayed promise has resolved 😲.

How exactly do we prevent such issues?

Async-Await is not perfect

In an ideal world, we would not have to care about such things, but unfortunately, it is not possible to cancel an async function. Which means we have to check whether the current useEffect cycle has ended after each async operation (Promise).

const [data, setData] = useState();
useEffect(() => {
  let cancel = false;
  const runEffect = async () => {
    const data = await fetchSomeData(filter);
    if (cancel) {
      return;
    }
    setData(data);
  };
  runEffect();

  // Cleanup function that will be called on
  // 1. Unmount
  // 2. Dependency Array Change
  return () => {
    cancel = true;
  }
}, [setData, filter]);

This can become very tedious in an async function that does many awaits in sequence:

const [data1, setData1] = useState();
const [data2, setData2] = useState();
const [data3, setData3] = useState();
useEffect(() => {
  let cancel = false;

  const runEffect = async () => {
    const data1 = await fetchSomeData(filter);
    if (cancel) {
      return;
    }
    setData1(data);

    const data2 = await fetch(data1.url);
    if (cancel) {
      return;
    }
    setData2(data);

    const data3 = await fetch(data2.url);
    if (cancel) {
      return;
    }
    setData3(data);
  };
  runEffect();

  // Cleanup function that will be called on
  // 1. Unmount
  // 2. Dependency Array Change
  return () => {
    cancel = true;
  }
}, [setData1, setData2, setData3, filter]);

This is the only way we can ensure setState is not called after the cleanup function has been called, nevertheless, the async operation aka the network request (initiated through fetch) is still being executed.

Cancelling Pending Async Operations

Modern Browers come with a new API called AbortController which can be used for aborting pending fetch requests.

const [data, setData] = useState();
useEffect(() => {
  const controller = new AbortController();
  const runEffect = async () => {
    try {
      const data = await fetch(
        "https://foo.bars/api?filter=" + filter,
        { signal: controller.signal }
      );
      setData(data);
    } catch (err) {
      if (err.name === 'AbortError') {
        console.log("Request was canceled via controller.abort");
        return;
      }
      // handle other errors here
    }
  };
  runEffect();

  return () => {
    controller.abort();
  }
}, [setData, filter]);

Now every time filter changes or the component is updated the pending network request is aborted. Instead of resolving, the fetch Promise will reject with an error 👌.

You can learn about browser support for AbortController here (of course IE does not support AbortController 😖): https://caniuse.com/#feat=abortcontroller

There is a polyfill available. It does not actually implement canceling since it must be done natively in the browser. Instead, it mimics the behavior by throwing an abort error after the fetch call has resolved/rejected.

Furthermore, this solution only works for fetch calls 😕.
Some API's provide ways of canceling async operations, others do not.

For instance, this is how you can cancel loading an Image with a useEffect hook today:

export const loadImage = src => {
  const image = new Image();
  const done = false;

  const cancel = () => {
    if (done) {
      // do not change the image instance once it has been loaded
      return;
    }
    // this will abort the request and trigger the error event
    image.src = "";
  };

  const promise = new Promise((resolve, reject) => {
    image.src = src;
    const removeEventListeners = () => {
      image.removeEventListener("load", loadListener);
      image.removeEventListener("error", errorListener);
    };
    const loadListener = () => {
      removeEventListeners();
      done = true;
      resolve(image);
    };
    const errorListener = err => {
      removeEventListeners();
      reject(err);
    };
    image.addEventListener("load", loadListener);
    image.addEventListener("error", errorListener);
  });

  return { promise, cancel };
};

useEffect(() => {
  const task = loadImage(url)
  const runEffect = async () => {
    try {
      const image = await task.promise;
      // do sth with image
    } catch (err) {
      // handle cancel error
    }

  };
  runEffect();

  return () => {
    task.cancel();
  }
}, [url])

In an environment where you are working with other uncancelable async API's, you will still have to set and check a boolean variable.

Hopefully, all async based APIs will someday support using the AbortController.

For now, we have to handle a mix of boolean checks and try catches.

But what if we could have some abstraction over both canceling requests and stopping function execution after an await keyword?

Introducing `useAsyncEffect`

Have you heard about Generator Functions before?

const generator = function *() {
  yield "bars";
  yield "foo";
  return "fizz"
}

A generator function is a pausable function. The yield keyword indicates a pause of the function. Let's run this generator!

// create instance of generator
const instance = generator();
// call next to run the generator until the next yield keyword
let result = instance.next();
console.log(result); // {value: "bars", done: false}
// continue calling
result = instance.next();
console.log(result); // {value: "foo", done: false}
// we can continue calling next until done is true
result = instance.next();
console.log(result); // {value: "fizz", done: true}

Besides passing values out of the generator, we can also pass in values as an argument of the next method:

const generator = function *() {
  const echo = yield "hello";
  console.log(echo);
}

// create instance of generator
const instance = generator();
let result = instance.next();
console.log(result); // {value: "hello", done: false}
// pass string into generator that will be assigned to the echo variable
instance.next("hello generator");

This is pretty cool! But how can this help us with the async-await issue?

In the past generators have been used to simulate async-await behaviour

Generators have been around since ECMAScript 2015 (6th Edition, ECMA-262)

Async functions were not part of the spec until ECMAScript 2017 (ECMA-262)

During the period between EcmaScript 2015 and 2017 various libraries that mimicked the behaviour of async-await with generators popped up.

One of the most popular ones being co

import co from 'co';

// wrap generator into function that returns a promise
const asyncFunction = co.wrap(function * () {
  const result = yield fetch(url);
  console.log(result);
  return 1
});

asyncFunction().then((res) => {
  assert.equal(res, 1);
})

Co does basically run the generator until a promise is yield-ed, then waits for the promise resolving and continues running the generator with the resolved value of the promise (get.next(resolvedPromiseValue)) until the generator is done (gen.next(resolvedPromiseValue).done === true).

One thing that distinguishes async-await and generators (besides their syntax), is that generators are not forced into resolving a Promise or even continuing execution of the generator function after it has paused.

Which basically means we can use a generator as a "cancelable" async-await.

Let's built that `useAsyncEffect` hook

Implementation

import { useEffect } from "react";

const noop = () => {}

const useAsyncEffect = (generator, deps = []) => {
  // store latest generator reference
  const generatorRef = useRef(generator);
  generatorRef.current = generator;

  useEffect(() => {
    let ignore = false;
    let onCancel = noop;

    const runGenerator = async () => {
      // create generator instance
      const instance = generatorRef.current(_onCancel => {
        // allow specifying a onCancel handler
        // that can be used for aborting async operations
        // e.g. with AbortController
        // or simple side effects like logging
        // For usage: see example below
        onCancel = _onCancel || noop;
      });

      // generator result
      let res = { value: undefined, done: false };
      do {
        res = instance.next(res.value);
        try {
          // resolve promise
          res.value = await res.value;
        } catch (err) {
          try {
            // generator also allow triggering a throw
            // instance.throw will throw if there is no
            // try/catch block inside the generator function
            res = instance.throw(err);
          } catch (err) {
            // in case there is no try catch around the yield
            // inside the generator function
            // we propagate the error to the console
            console.error("Unhandeled Error in useAsyncEffect: ", err);
          }
        }

        // abort further generator invocation on
        // 1. Unmount
        // 2. Dependency Array Change
        if (ignore) {
          return;
        }
      } while (res.done === false);
    };
    runGenerator();

    // Cleanup function that will be called on
    // 1. Unmount
    // 2. Dependency Array Change
    return () => {
      ignore = true;
      onCancel();
    };
  }, deps);
};

Usage

const [data, setData] = useState();
useAsyncEffect(function * (onCancel) {
  const controller = new AbortController();

  // handle error 
  onCancel(() => {
    console.log("cancel while fetch is still executed, use controller for aborting the request.");
    controller.abort();
  });
  try {
    const data = yield fetch(
      "https://foo.bars/api?filter=" + filter,
      { signal: controller.signal }
    )
    setData(data);
  } catch (err) {
    if (err.name === 'AbortError') {
      console.log("Request was canceled via controller.abort")
      // we know that an 'AbortError' occurs when the request is
      // cancelled this means that the next promise returned by yield
      // will be created but not actively used, thus, we return in
      // order to avoid the promise being created.
      return;
    }
  }

  // set new cancel handler
  onCancel(() => {
    console.log("cancel while doSthAsyncThatIsNotCancelable is still being executed");
  });
  const newData = yield doSthAsyncThatIsNotCancelable();
  setData(newData);

  // all our async operations have finished
  // we do not need to react to anything on unmount/dependency change anymore
  onCancel(() => {
    console.log("everything ok");
  })
}, [setData, filter]);

This hook now allows us to omit all the boolean checks (ignore === true) in our component while still giving us the power to cancel async operations (that are cancelable) or handling other side-effects by registering a handler function with onCancel.

I hope you enjoyed reading this!

Have you used generators before? How do you handle async operations with useEffect today? Will you use the useAsyncEffect hook in your code? Do you have any feedback or spotted a bug?

Let's discuss in the comments!

Also, feel free to follow me on these platforms, if you enjoyed this article I ensure you that a lot more awesome content will follow. I write about JavaScript, Node, React and GraphQL.

Have an awesome and productive day!

Mapping JavaScript Objects with Modern APIs

Laurin Quast — Fri, 21 Jun 2019 12:26:36 +0000

Object.entries allows you to convert an object into an array structure:

console.log(Object.entries({prop1: 1, prop2: 2 }))
// [ ["prop1", 1], ["prop2", 2] ]

This method is part of the ECMAScript 2017 Spec and already heavily used by many developers (including me).

Until recently, there was no convenient method available to transform this structure back into an object.

Now, we finally have Object.fromEntries which allows constructing an object from the structure returned by Object.entries.

This makes it super convenient to map objects:

Implementation:

const mapValues = (input, mapper) =>
  Object.fromEntries(
    Object.entries(input).map(([key, value]) => [
      key,
      mapper(value, key, input)
    ])
  );

Usage example:

const input = {
  prop1: 1,
  prop2: 4,
};

const output = mapValues(input, value => value * 2);

expect(output).toEqual({
  prop1: 2,
  prop2: 8,
});

Support

Browser Support: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/fromEntries#Browser_compatibility
Node Support: https://node.green/#ES2019-features--Object-fromEntries

Homebrew React Hooks: useCurrent

Laurin Quast — Sun, 16 Jun 2019 15:37:54 +0000

You might probably have heard of React Suspense.

In 2018 Dan Abramov presented this new feature which is yet to be released. As part of the talk he showcased how suspense can make data fetching easier.

I was really fascinated by his words about loading indicators and how removing them can lead to better user experience.

Some of you might now think: Why should I show a blank page to my users?

Well, he actually meant not completely removing them, but only showing them when they are necessary.

Let's say you have a request that only takes 100 milliseconds. You would show a loading spinner for a very short time frame. The user might not even have enough time to realize he did just enter a loading state. He might just notice the page flashing.

On the other hand, if you do not have a loading indicator at all and the request takes a few seconds (e.g. due to bad internet connection) and all the user sees is a blank page, the situation becomes even worse.

Such "janky" behavior can confuse the user and make him assume that the page is buggy or broken, in the worst case he could even become angry (and we all know that only a happy user is a loyal customer).

In order to provide the best user experience we need to solve both problems at the same time!

So there are two big questions to answer, the first of them being:

What do we render while the request for new content is still pending?

Correct! We render The old content™️

Of course, in case we do not have any old content we still need to show a loading spinner.

Here is the second question:

What do we render when the request for new content takes forever?

Yes! We show a loading spinner.

Although suspense might be the silver bullet to solve this problem in the future, I asked myself: Can we achieve the same user experience with hooks today?

Let's take a look on existing data fetching libraries

In the following examples, I am going to use a hook that simulates a network request for the data fetching part. It uses setTimeout internally.

const {data, loading} = useFakeFetch("/endpoint")

The code should look familiar to folks that have worked with react-apollo-hooks or react-fetch-hook before.

Almost all of those data fetching hooks work the same, once a hook parameter changes, the data is re-fetched with the new parameters.

Try clicking the change endpoint button in the following example.

Did you notice that immediately after clicking the button data becomes null?

This is exactly the kind of behavior that we want to change!

So where do we start?

We could, of course, change the existing data fetching hook.

However, my first rule regarding hooks is the power of hooks lies in the composability of multiple hooks.

So instead of altering the existing hook we will now step by step build a new hook that is compatible with every data fetching hook that returns a data and loading value.

So let's start by storing the old data until the new data is available.

The perfect use-case for useState combined with useEffect

const {data, loading} = useFakeFetch("/endpoint")
const [currentData, setCurrentData] = React.useState(data);

React.useEffect(() => {
 // condition for updating the stored data
 if (loading === false && data !== currentData) {
   setCurrentData(data)
 }
}, [loading, data, setCurrentData, currentData]);

// use currentData instead of data

Check out the following example:

Did you notice the new value currentData which now holds the old value until the new data was fetched?

Half of the problem is already solved!

In the next step, we will ensure that the loading indicator should only be shown after a certain threshold has been exceeded (aka the request is taking longer than expected).

Reintroducing our old friends setTimeout and clearTimeout

const {data, loading} = useFakeFetch(endpoint);
const [currentData, setCurrentData] = React.useState(data);

React.useEffect(() => {
  if (loading === false && data !== currentData) {
    setCurrentData(data);
  }
}, [loading, data, setCurrentData, currentData]);

// NEW STUFF STARTS HERE
const previousLoadingRef = React.useRef(loading);
const [
  shouldShowLoadingIndicator,
  setShouldShowLoadingIndicator
] = React.useState(loading);

React.useEffect(() => {
  let timeout = undefined;
  // only update in case loading has changed between renders
  if (previousLoadingRef.current !== loading) {
    if (loading) {
      // should show loading indicator if request time
      // exceeds one second
      timeout = setTimeout(() => {
        setShouldShowLoadingIndicator(true);
      }, 1000);
    } else {
      setShouldShowLoadingIndicator(false);
    }
  }
  previousLoadingRef.current = loading;

  // cancel the timeout in case the data is available 
  // before one second has passed
  return () => timeout && clearTimeout(timeout);
}, [loading, setShouldShowLoadingIndicator]);

// use currentData instead of data
// use shouldShowLoadingIndicator instead of loading

In this example, we now have a long request, after one second we want to show a loading indicator!

Almost done! Now we have a working proof of concept implementation. Let's convert that code into a reusable hook:

const useCurrent = (data, loading, showLoadingIndicatorThereshold = 300) => {
  const [currentData, setCurrentData] = React.useState(data);
  const previousLoadingRef = React.useRef(loading);
  const [
    shouldShowLoadingIndicator,
    setShouldShowLoadingIndicator
  ] = React.useState(loading);

  React.useEffect(() => {
    if (loading === false && data !== currentData) {
      setCurrentData(data);
    }
  }, [loading, data, setCurrentData, currentData]);

  React.useEffect(() => {
    let timeout = undefined;
    if (previousLoadingRef.current !== loading) {
      if (loading) {
        timeout = setTimeout(() => {
          setShouldShowLoadingIndicator(true);
        }, showLoadingIndicatorThereshold);
      } else {
        setShouldShowLoadingIndicator(false);
      }
    }
    previousLoadingRef.current = loading;
    return () => timeout && clearTimeout(timeout);
  }, [loading, setShouldShowLoadingIndicator, showLoadingIndicatorThereshold]);

  return [shouldShowLoadingIndicator, currentData];
};

Here's a usage example example

const { data, loading } = useFakeFetch(endpoint);
const [shouldShowLoadingIndicator, currentData] = useCurrent(
  data,
  loading,
  300
);

And of course, there is also a live example

We are done! This hook is now officially compatible with the following libraries:

react-apollo (with the HOC API)
react-fetch-hook
react-apollo-hooks
insert every hook data fetching library here

Let's make the web a better place for users by using this hook until React Suspense finally lands!

Bonus: Decrease amount of rerenders (possible performance optimization)

It is further possible to even decrease the amount of rerenders.

Open the console of the following example, wit until the initial data is loaded, then click the button Change endpoint button once.

The last output should be render 8. Which means 8 renders occurred during the whole process. We can reduce the total amount of necessary renders for this procedure to 6, having 2 less rerenders.

Before we optimize the example I want to mention, that we probably do not need to do such an optimization. However, when we have a large component tree and use this hook is used on top of it, you might feel the impact of the two additional rerenders!

Let's start with the currentData state. We do not need an actual useState hook for storing it. That's because of every time data or shouldShowLoadingIndicator changes a rerender is already triggered.

We can, therefore, store currentData using useRef. As a result, we can spare also the useEffect for updating currentData.

Furthermore, shouldShowLoadingIndicator is only changed under two conditions:

timeout was not cancelled (request duration was exceeded)
during useEffect when loading has changed and is now false

The latter is actually unnecessary, we can refactor shouldShowLoadingIndicatorR to a useRef as well. That's because shouldShowLoadingIndicator is automatically false when loading is false, which means we can update shouldShowLoadingIndicator at the same time currentData is updated.

But how do we update shouldShowLoadingIndicator to true? Mutating a ref is not triggering any rerenders. The solution is the introduction a new state which soles purpose is to trigger a rerender.

Let's take a look at the optimized version:

const useCurrent = (data, loading, showLoadingIndicatorThereshold = 300) => {
  const currentDataRef = React.useRef(data);
  const previousLoadingRef = React.useRef(loading);
  const shouldShowLoadingIndicatorRef = React.useRef(loading);
  const [, triggerStateUpdate] = React.useState(0);

  // those values should always reflect the input when loading is false 
  if (!loading) {
    currentDataRef.current = data;
    shouldShowLoadingIndicatorRef.current = false;
  }

  React.useEffect(() => {
    let timeout = undefined;
    // only when loading has changed from false to true
    if (previousLoadingRef.current !== loading && loading) {
      timeout = setTimeout(() => {
        // mutate ref
        shouldShowLoadingIndicatorRef.current = true;
        // trigger rerender
        triggerStateUpdate(i => i + 1);
      }, showLoadingIndicatorThereshold);
    }
    previousLoadingRef.current = loading;
    return () => timeout && clearTimeout(timeout);
  }, [loading, triggerStateUpdate, showLoadingIndicatorThereshold]);

  return [shouldShowLoadingIndicatorRef.current, currentDataRef.current];
};

Again, open the console in the following example! Wait until the initial loading has happened, click the button and observer the console logs.

We have successfully removed two unnecessary rerenders!

I hope you enjoyed reading this blog post!

I wanna thank @mfpiccolo and @sseraphini which helped my by proofreading this article!

This is only the first of many blog posts about utility hooks and other topics (including react and GraphQL).

Stay updated by following me here on DEV or Twitter and Github.

DEV Community: Laurin Quast

GraphQL Live Queries backed by the Relay Specification?

Collecting GraphQL Live Query Resource Identifier with GraphQL Tools

There must be a better solution for uniquely identifying the selection set data

Implementing the resource identifier collector

1. Traversing the GraphQL execution result tree

2. Manually register the resource identifier by calling a function that is injected via the context

Conclusion

GraphQL Live Queries with Socket.io

Motivation

GraphQL already has a solution for real-time

Live Query Magic

Concept

LiveQueryStore Implementation

Registering the LiveQueryStore

Notifying the live query store for changes

Adding the @live directive to your GraphQL schema

Consuming live queries on the client

Basic client creation

Executing GraphQL operations

GraphiQL fetcher

Implementation with Relay

Usage with Apollo Client

Usage with Urql

The Future

Homebrew React Hooks: useIsKeyPressed

Configure the create-react-app public URL post-build with Node.js and express

Incrementally migrating a CRA application to TypeScript without ejecting

Bootstrapping the TypeScript Config

Add TypeScript Build Script to package.json

Moving our file to ts_src

Add path src/path/to/file.js to .gitignore and .eslintignore

Repeat step for every file that is imported by src_ts/path/to/file.ts

Add typings

Committing the changes changes

Adding new functionality

Someday in the future: the src folder is empty 🎉

We are done 👌

(How) do you test the synergy of multiple services?

Testing each service in isolation

Testing the services together

My Conclusion

Optimizing your GraphQL Operations with GraphQL Code Generator and the Relay Compiler

Homebrew React Hooks: useAsyncEffect Or How to Handle Async Operations with useEffect

TLDR

The Problem

Why does useEffect not accept my async functions?

Why would I even need a cleanup function anyways?

Async-Await is not perfect

Cancelling Pending Async Operations

Introducing useAsyncEffect

In the past generators have been used to simulate async-await behaviour

Let's built that useAsyncEffect hook

Implementation

Usage

Mapping JavaScript Objects with Modern APIs

Homebrew React Hooks: useCurrent

What do we render while the request for new content is still pending?

What do we render when the request for new content takes forever?

Let's take a look on existing data fetching libraries

Bonus: Decrease amount of rerenders (possible performance optimization)

Adding the `@live` directive to your GraphQL schema

Add TypeScript Build Script to `package.json`

Moving our file to `ts_src`

Add path `src/path/to/file.js` to `.gitignore` and `.eslintignore`

Repeat step for every file that is imported by `src_ts/path/to/file.ts`

Someday in the future: the `src` folder is empty 🎉

Why does `useEffect` not accept my async functions?

Introducing `useAsyncEffect`

Let's built that `useAsyncEffect` hook