DEV Community: Kyle Johnson

Five Open Source tools to track your PostgreSQL database performance

Kyle Johnson — Tue, 28 Jan 2025 14:34:32 +0000

Written by Eddie Jaoude (see outro)

How is your PostgreSQL (aka Postgres) database performing in production after releasing that latest shiny new feature? If you don’t measure its performance then you will never know and the best you can hope for is an educated guess. Do you want to rely on guesswork to decide whether you should be rolling out this new feature to your users?

In this blog post I will take you through the steps to achieve/create a dashboard that monitors a PostgreSQL database. The dashboard will also be annotated when we enable/disable features. This is an effective way of monitoring the impact that enabling a new feature has on a database.

All tools that I refer to in this blog post are Open Source and free:

NextJS for our app https://nextjs.org/
PostgreSQL as our database https://www.postgresql.org/
PostgreSQL Exporter to expose metrics -https://github.com/prometheus-community/postgres_exporter
Prometheus to collect metrics https://prometheus.io/
Grafana to display metrics https://grafana.com/
Flagsmith for feature flag management https://www.flagsmith.com/

Prerequisites

So that you can follow along please make sure that you have the following:

Docker
Flagsmith (I will use their free cloud account, but you can also host it yourself on Docker).

Architecture

I will be using the following architecture setup, however you can move any items between any of the layers to better suit your situation:

Host machine
- Our Application (NextJS)
- K6 to simulate users
Docker
- PostgreSQL database
- PostgreSQL Exporter
- Prometheus
- Grafana
Cloud
- Flagsmith (you can run this in Docker if you prefer)

PostgreSQL

For this blog I will be using the SQL database Postgres. Postgres is a popular Open Source relational database that is super robust and stable with decades of experience. I find it has a great balance of the latest trending features without compromising stability. I have used Postgres in projects for startups to UK banks, and it handles a variety of different data and performance situations without skipping a beat.

If you already have a PostgreSQL database then please feel free to skip this section.

This Docker command will create a PostgreSQL 14 database. Set the username and password as well as expose the port on our local machine:

docker run --name postgres14 -e POSTGRES_USER=user -e POSTGRES_PASSWORD=password -p 5432:5432 -d postgres:14

Create a small private network for all our Docker containers to communicate together:

docker network create postgres-network

Then add the new PostgreSQL database to this private network. This is achieved by using the container name we created before:

docker network connect postgres-network postgres14

Then you can inspect the network to get the PostgreSQL database’s IP with this command and specify the name of the network to inspect:

docker network inspect postgres-network

In the JSON output there will be a “Container” section that lists the containers in the network and also their respective IP. You will need the PostgreSQL IP in the next section.

"Containers": {
            "4469e0602d83ac7fa2bd70ecbbcb34c41911cc342506c0f0a67f71365a61d54a": {
                "Name": "postgres14",
                "EndpointID": "5c8a26405db3598f472e9dcb4217b7b0f1d117974a5e2981e301bcda850a99e1",
                "MacAddress": "02:42:ac:12:00:04",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            }
        }
}

PostgreSQL Exporter

Now we have a database, we need to get the performance metrics out from it. The PostgreSQL Exporter is the industry standard way to gather PostgreSQL metrics for Prometheus consumption. It is a stable project that has been around for many years and is still getting new features added all the time. At the time of writing this “Multi-Target Support” has been added as a beta feature. With plenty of content and use cases out there, you will not be short of examples to match your situation.

To get the Docker container running we need to pass some more parameters. In addition to the Docker container name and exposing the port, specify the PostgreSQL database IP (you can get this from the end of the previous section using the Docker inspect command), the PostgreSQL database username and password.

docker run --name postgres-exporter -p 9187:9187 -e DATA_SOURCE_URI="172.18.0.2:5432/postgres?sslmode=disable" -e DATA_SOURCE_USER=user -e DATA_SOURCE_PASS=password -d quay.io/prometheuscommunity/postgres-exporter

As before, add the new container to the private Docker network with:

docker network connect postgres-network postgres-exporter

Test the PostgreSQL Exporter is working by visiting this url in the browser or using curl on your host machine:

curl "http://localhost:9187/metrics"

You can learn more about the PostgreSQL Exporter in their GitHub repository https://github.com/prometheus-community/postgres_exporter

Next inspect the private network again for the PostgreSQL Exporter IP, as this will be needed in the following section:

docker network inspect postgres-network

Prometheus

At this stage we have a database and we have the performance metrics, but we need to store the collected data over time. Prometheus is the go to Open Source tool for “metrics to insight” built with monitoring and alerting in mind. It stores the data in a time series format and has many integrations with popular tools and platforms. With great documentation, an active GitHub repo (50k+ Stars), and a large community, it is no surprise that Prometheus is the go to tool.

Although Prometheus has a big ecosystem it is straightforward to get started. All we will need is a small YAML file and one Docker command.

Create a YAML file called “prometheus.yml”. You can keep this anywhere you prefer, but I have chosen to do so in my “Downloads” folder (full path “~/Downloads/prometheus.yml”).

In this file, first set global configs for “scrape_interval” and “evaluation_interval”. Note these global configs can be overwritten at the job level if required when having multiple jobs.

scrape_interval: defines how often the monitor is scraped - you can change this to your preference, I have selected 15 seconds here.
evaluation_interval: defines how often queries are evaluated for alerting.

It is recommended in most cases to have both these global configs set at the same value.

Below the global configs, we specify the jobs under “scrape_configs”, this is made up of the following:

job_name: you can call this the name you prefer, it will show in your Prometheus web page of targets (screenshot below)
static_configs
- targets: this is the IP and port to scrape of the “Postgres Exporter” in this example. Use the IP from the last Docker command in the previous section.

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "postgres-exporter"
    static_configs:
      - targets: ["172.18.0.3:9187"]

Now to run the Prometheus Docker container to use the YAML config file just created:

docker run --rm --name prometheus -p 9090:9090 -v ~/Downloads/prometheus.yml:/etc/prometheus/prometheus.yml -d prom/prometheus

The last step in this section is to add our new Prometheus container to the private network we created. Use the following command to do this is:

docker network connect postgres-network prometheus

Then visit http://localhost:9090/targets and you will see the job and the state of “UP”:

Note: If you stop the Prometheus container, it will no longer have access to the private network. Therefore you will need to add it back to the network created so it has access to the other containers. If you have restarted other containers also check none of their container IPs have changed, as these might need updating in the config files and also network.
You can also learn more about Prometheus on their official website: https://prometheus.io/docs/introduction/overview/

Grafana

Now we have our Postgres performance metrics stored, we need to visualise the data. Grafana visualises, queries and alerts on our data from multiple sources including Prometheus as well as NoSQL/SQL databases. With Grafana being so popular and customizable, it has a large community with pre-existing templates to kickstart our Grafana dashboard and then we can customize as needed.

Grafana can run on Docker with one command, which uses port 3000. I will map this externally to port 3001 on my host machine because my NextJS is already using port 3000:

docker run -d -p 3001:3000 --name=grafana grafana/grafana-enterprise

Add the new container to our network:

docker network connect postgres-network grafana

Visit the Grafana Dashboard http://localhost:3001 in the browser. You can log in with the username “admin” and the password “admin” - you will be prompted to update the password, but if running Grafana locally you can click the “skip” button for now.

Firstly open the side menu and expand “Connections” and click “Data source”. Then you can click “Add data source” for Prometheus.

Select Prometheus and you will need to add the Prometheus URL. You can get the IP by inspecting the Docker network again:

docker network inspect postgres-network

Only the Prometheus URL is required (don’t forget to add the port). The other fields are optional and can be updated later if required:

Now there is a data source, a great place to start a Grafana Dashboard is from an existing template and customise it further from there.

Visit your Grafana Dashboard on http://localhost:3001/dashboard/import and paste in the templated Grafana Dashboard URL https://grafana.com/grafana/dashboards/455-postgres-overview/ and then click “load”. A new form will appear and you can rename the dashboard, but the important part is to select a Prometheus source.

My imported Grafana Dashboard looks like this:

List your running Docker containers with this command and you should see the following containers running:

docker ps

If you want to read more about Grafana then check out their website: https://grafana.com/docs/grafana/latest/

Flagsmith

We have all been there where our project needs some feature flags and we think we are being smart by moving them to a config file or environment variable. In reality we still need to deploy to make changes and the developers/devops in the team will need to make those changes. Plus they are not instant and usually require a full redeploy.

To manage my feature flags, I don’t like to reinvent the wheel, so I use the Open Source project Flagsmith and their cloud platform. Flagsmith has more features than I could ever create, plus it allows me to ship faster and more confidently by decoupling my deployments and releases as well as other great features that allow me to release to specific users or groups of users before everyone.

With Flagsmith we can not only toggle this feature on/off but we can also expose it to specific people or groups of people.

Sign up

First start by signing up to Flagsmith's free account https://www.flagsmith.com/.

Once you sign up / log in, you will create an “Organisation” and then a “Project”:

Create a feature flag

Click the “Create Feature” button at the top right.
Give the feature a name. One thing to take into consideration is that you cannot change the name later.
Leave “Enabled by default” off.

“Value” is not needed for our feature as we will use enable/disable only. Value would be used for controlling the content. For example when to show a banner and the content of the banner.
You can also give a “Description”, but note that you can change this later.

Create a feature flag

Flagsmith has many integrations so do make sure that you check out their great documentation.

To use Flagsmith with NextJS, add it as an npm dependency with the command:

npm i flagsmith --save

Create a NextJS provider in file “src/app/providers.js”:

"use client";

import { FlagsmithProvider } from "flagsmith/react";
import { createFlagsmithInstance } from "flagsmith/isomorphic";
import { useRef } from "react";

export default function Providers({ serverState, children }) {
  const flagsmithInstance = useRef(createFlagsmithInstance());

  return (
    <FlagsmithProvider
      flagsmith={flagsmithInstance.current}
      serverState={serverState}
    >
      <>{children}</>
    </FlagsmithProvider>
  );
}

Open the “src/app/layout.js” file and in the import section add:

import { createFlagsmithInstance } from "flagsmith/isomorphic";
import Providers from "./Providers";

Then in the RootLayout function add (don’t forget to make the function async):

const flagsmith = createFlagsmithInstance();
await flagsmith.init({
  environmentID: process.env.NEXT_PUBLIC_FLAGSMITH_ID,
});
const serverState = flagsmith.getState();
console.log(serverState); // just to check the data is coming through

Wrap the “body” in the provider:

<Providers serverState={serverState}>
  <body className={inter.className}>{children}</body>
</Providers>

Lastly add your “NEXT_PUBLIC_FLAGSMITH_ID” to the “.env” file, which you can find in the “SDK Keys” menu on the left of your Flagsmith Dashboard.

Your “.env” file should look something like this:

NEXT_PUBLIC_FLAGSMITH_ID=3C5QvbtgZ3C5Qvbxyz

Let’s run the app and check it all works together as expected:

npm run dev

There should be no visual change in the UI, but in the terminal there will be a further output from Flagsmith because of our console.log.

In my terminal I have the following output which contains the feature flags I created in Flagsmith. (Currently I only have one feature flag, if there were more you would see them in the output also.)

 ✓ Compiled in 101ms (496 modules)
{
  api: 'https://edge.api.flagsmith.com/api/v1/',
  environmentID: '3C5QvbtgZh3C5QvbtgZh',
  flags: {
    stats: {
      id: 98466,
      enabled: false,
      value: ''
    }
  },
  identity: undefined,
  ts: null,
  traits: {},
  evaluationEvent: null
}

Use Feature Flag

On my homepage, I am thinking of showing some near real-time statistics but I am concerned about the impact. By wrapping it in a Flagsmith Feature Flag, I can enable it for my user, a group of users or for everyone, as well as disable it if required. I have created a Stats component that I will include on the homepage, and in the Stats component it will check for the Feature Flag “stats” status.

"use client";

import { useFlags } from "flagsmith/react";

export default function Stats() {
  const { stats } = useFlags(["stats"]);
  if (!stats.enabled) {
    return null;
  }

  return (
    // ...
  );
}

Now we can show/hide these stats from the Flagsmith Dashboard.

Integrate with Grafana

It is important to know when we toggle on/off features in our app what the direct impact on the application’s performance is. Yes we could try and guess what happened when, but this is neither efficient nor accurate. It is much better to integrate Flagsmith with Grafana, so that it actually puts a marker (called a “Grafana Annotation”) on the graph for us.

The integration is straightforward. However we need to expose our Grafana instance to the internet so Flagsmith in the cloud can communicate with it.

If you are not using Flagsmith cloud and your Flagsmith instance is local to your Grafana or your Grafana is on the internet already (because you are using Grafana cloud), then you can skip this section.

There are multiple ways to achieve this, but my preferred way is to use an npm package which results in a total of using two commands:

Install the npm package “localtunnel” globally with the command:

npm install -g localtunnel

Tell localtunnel what port to expose to the internet and forward to my local instance:

lt --port 3001

This command will return us a URL that can be used for the integration on Flagsmith.It is therefore important to remember this URL as it will be needed for the next step on the Flagsmith’s Dashboard.

Get a “service account token” from your Grafana.This can be done by opening the side menu and clicking “Users and access”, then “Service accounts”. Then give the service account a name.

In the newly created service account, create a token.

Copy this token to your clipboard as you will need it in the next step on the Flagsmith Dashboard.

Go to the Flagsmith Dashboard and click on “Integrations”:

Then scroll down to “Grafana” and click on “Add Integration”. Add the URL that was generated in the terminal and the “service account token” previously copied.

Now you will see the integration is complete.

The final step is to display the Flagsmith feature flag changes on the Grafana Dashboard. We will need to edit the Grafana Dashboard we imported earlier to add the annotation as it occurs.

Go back to the Grafana Dashboard created earlier and press “Edit”.

In the Grafana Dashboard settings click the “Annotations” tab and then “Add annotation query”.

Give the annotation a name and select “Grafana” for the data source. For the filter drop down select “Tags” and select the tags you are interested in - I selected my whole Flagsmith project.

Save the annotation changes to the Grafana Dashboard and exit edit mode, returning to the main Grafana Dashboard view. You will now have a “Flagsmith” toggle at the top of the Grafana Dashboard to show/hide the annotations.

With this toggle enabled, each time changes are made on the Flagsmith Dashboard they will appear on the Grafana Dashboard. I will demonstrate this shortly but first, I will generate some user load to make the graphs more interesting.

k6

Simulating and creating user traffic can be tricky. There are many tools out there and I have tried and used many over the years. Recently I came across k6. This is a load testing tool created by Grafana. It is designed to be developer friendly, from browser performance testing to chaos and resilience testing. k6 has packages for the popular operating systems as well as a Docker container and a standalone binary.

Installation

I will install k6 on my Mac with brew:

brew install k6

Script

In k6’s Getting Started guide, they have some great example scripts to get started. I have taken their “Ramp VUs up and down in stage” and modified them a little for my needs.

The script has the imports at the top as you would expect, followed by the test configuration. In the configuration there are an array of “stages” where each item is an object of “duration” and “target”. In this “stages” array you can ramp up and down the users at specific times.

After exporting the config options, export a default function with the URL k6 will make the request against and then validate the response, in this case we are expecting a status code of 200.

I will add the following k6 script to a file called “script.js” in the root of the project:

import http from "k6/http";
import { check, sleep } from "k6";

// Test configuration
export const options = {
  thresholds: {
    // Assert that 99% of requests finish within 3000ms.
    http_req_duration: ["p(99) < 3000"],
  },
  // Ramp the number of virtual users up and down
  stages: [
    { duration: "30s", target: 100 },
    { duration: "1m", target: 50 },
    { duration: "3m", target: 30 },
    { duration: "20s", target: 0 },
  ],
};

// Simulated user behavior
export default function () {
  let res = http.get("http://localhost:3000");
  // Validate response status
  check(res, { "status was 200": (r) => r.status == 200 });
  sleep(1);
}

To run the above k6 script, use the command:

k6 run script.js

The output is:

While k6 is running, we can toggle our Flagsmith feature on/off and see the effect it has on the application. Each time changes on the Flagsmith’s Dashboard are made the annotation appears and we can hover the annotating line to get more information.

Now you can test in production and make an informed decision as to whether you should roll out the new feature to everyone.

Troubleshooting

You might find yourself having connection issues between the containers during setup. Or the setup initially worked but after restarting one or more containers or your host machine, it no longer works. I found that this is usually due to IP changes, which caught me out multiple times before.

Note that you might not get any visible errors in the logs, but no data will be coming through the workflow. Check the IP of each container and make sure it is in the private network and the relative YAML config files are updated.

Conclusion

What effect does your latest feature have on your app’s health? If you can’t quantify the impact, how do you know if you should or should not roll it out to all your users? There is no need to build scripts and tooling to capture then measure the performance of your app before and after changes. This is complex and time consuming, especially when you could be using this time to focus on your app. There are so many Open Source tools that you can use and integrate together to help you measure performance and improve your product, allowing you to focus on what you can deliver and the benefits to your users. Plus if you don’t want to host and manage these tools most of them have cloud versions you can also use.

Having a strong foundation for your project will help you make more informed decisions rather than guessing.

When you find yourself in the situation when something happened to your production app and you are trying to understand the “how” and “why” while debugging the situation, if you have not captured the historic information, then this becomes very difficult.

Key takeaways:

1: You can’t improve what you don’t measure. Measure before and after the changes so you can compare the impact these changes have on your project.
2: Be proactive. Simulate user load and see how your product performs; one user is different to 1000 users.
3: Use feature flags to safely and confidently test in production without affecting your users, which will allow you to make more accurate decisions.

Be proactive with all your changes, know the impact, and be confident to deploy on a Friday!

About the Author

Eddie is passionate about technologies and products which contribute towards a developer's work becoming more efficient and high quality.

Eddie uses his 15+ years of Fullstack expertise to offer personalised consulting on Open Source best practices, as well as creating technical content to bring awareness, review and teach his clients' latest products and services, to the tech community. To name a few areas, Eddie has a keen interest in Javascript/Typescript, API, databases and DevOps.

https://www.eddiejaoude.io/

http://eddiejaoude.substack.com/links

Implementing Feature Flags with Next.js and App Router

Kyle Johnson — Thu, 02 Nov 2023 16:33:00 +0000

Introduction

In this article we will discuss how we can combine the power of Vercel and feature flags (Flagsmith - open source) to build and deploy a config controlled application from scratch in less than 15 minutes using Next.Js feature flags. Written with Abhishek Agarawal🚀!

Through this blog, we will learn:

The importance of feature flags
How to create a basic Next.js application with Vercel Approuter
Setting up Vercel and push the project to Vercel
Deploy the application
Add feature flags to it
See real time changes on the website as we update feature flags from Flagsmith console

Background

Vercel is developers’ favourite platform to develop, preview and ship any application. It makes building complex applications easier and much more efficient.

Next.js is a popular open-source framework provided by Vercel and built on top of Node.js for building server-side rendered (SSR) React applications. Its core advantages include faster page loads, better SEO, and improved developer experience.

Feature flags offer the power of managing features in production out of the box. We chose to integrate with Flagsmith (https://github.com/Flagsmith/flagsmith). The set-up and integration is seamless and provides enormous power with just the click of a few buttons.

Understanding Feature Flagging and Its Importance

Feature flagging, also known as feature toggling or feature flipping, is a powerful software development technique that enables developers to deploy new features and changes to production in a controlled and gradual manner. Instead of releasing all changes at once, feature flags allow you to selectively enable or disable specific features for different segments of your user base.

But why is this approach so valuable? There are several reasons why feature flagging has become an essential part of modern software development:

Reduced Deployment Risks: Feature flagging reduces the risks associated with deploying new features. By gradually rolling out changes to a small subset of users, developers can closely monitor the impact and performance of the new features before making them available to a broader audience. This helps catch potential issues early and allows for quick adjustments if needed.
Continuous Delivery and Experimentation: With feature flagging, you can continuously deliver updates and experiment with new features without causing disruptions to the entire user base. This enables agile development practices and encourages a culture of experimentation, where developers can test hypotheses and gather real-world data to inform their decisions.
Personalized User Experiences: By segmenting users based on various traits (e.g., profession, location, subscription type), you can provide personalized experiences tailored to different user groups. This allows you to deliver targeted content, A/B test different variations, and optimize user engagement.
Hotfixes and Rollbacks: Feature flags provide a safety net for quick hotfixes and rollbacks. If a new feature causes unexpected issues or receives negative feedback, you can easily turn it off with a flag, avoiding the need for a full redeployment.

Integrating Flagsmith with Next.js

In this guide, we've chosen Flagsmith as our feature flagging solution. Flagsmith is an open-source platform that offers a user-friendly interface for managing feature flags, A/B testing, and segment overrides. You can control feature behaviour without modifying code, making it a powerful tool for managing complex projects with multiple feature variations.

By integrating Flagsmith into our Next.js project deployed on Vercel, we gain the ability to perform phased rollouts, control feature visibility based on user segments, and dynamically change feature settings in real-time without redeploying the application.

Let’s start building!

Flagsmith Account Setup

If you do not have an account already, visit https://app.flagsmith.com/signup and create a new account. Flagsmith also offers simple sign in with Google or Github options.
Create an organisation. For the purpose of this tutorial I’ll name it “Flagsmith <-> Vercel”. Feel free to pick your own.
Create a project inside this organisation by clicking “Create A Project''. I’ll just name it “my-nextjs-project”.
Congrats! You have already completed the Flagsmith setup process. You should see a screen with instructions on how to create new features.

We will come back to this screen when we want to add Next.js feature flags to our application.

Github repo setup

Create a new repo on https://github.com/. I have named it “vercel-flagsmith-integration”. Feel free to rename it.

Vercel account setup

Visit https://vercel.com/signup and you can easily signup with your Github account
Create a New project by Clicking Add New -> Project. You can simply visit https://vercel.com/new
On the home page, all the Github repos associated with the account should be visible. Make sure the access to read repos is granted. The recently created project should also be visible in the list. Vercel automatically detects that it is a Next.js project.
We will import this project once we have added some code

Now, let’s start coding 😀

Create a starter Next.js project

We will be using the create-next-app CLI tool to build us a starter project. Reference - https://nextjs.org/docs/api-reference/create-next-app . On the top of this, I have added a basic login page, so that we can take some user input and toggle behaviour based on that in the next steps.

We will be using the Vercel AppRouter paradigm for building our application.

Clone this repo to view the code https://github.com/abhishekag03/vercel-flagsmith-integration. Refer to the pull requests to understand the changes step by step. Specifically, this state of the repo describes the current expectation.

Let’s deploy!

Time to visit our Vercel dashboard again - https://vercel.com/new. Click on the “Import” button next to the repository where you just pushed.
Hit the “Deploy” button now

After the deployment has completed (typically it should not take more than a couple of minutes), click on the preview screen which will take you to the deployed website. The DNS in this case is https://vercel-flagsmith-integration.vercel.app/

You can visit the dashboard for your project (by clicking “Continue to Dashboard” in the above section) and view the domains and various other settings

Now, any change that we push to the “main” branch of our repo will get deployed here in near real time!

That’s how simple deploying applications with Vercel is!

Combining the power of Flagsmith with Vercel

We already saw how easy it is to set up and deploy a Next.js project. This is good for a basic project - however, in most cases we want more control towards the application behaviour.

For instance, imagine we want to add a new card to the below page. Currently it contains 4 cards - “Docs”, “Learn”, ‘Templates” and “Deploy”. Adding a new card is trivial, but for large applications, such changes are always rolled out incrementally. They are rolled out either based on geography, type or percentage basis. Building this incremental rollout functionality can be tricky as it involves managing a lot of variables with real time updates.

That’s where Flagsmith comes in. It offers a solution which allows you to manage all your Next.js feature flags, A/B testing and experimentation at one place with just the click of a button.

We will see how easy it is to rollout a feature to a certain segment of users with Flagsmith.

What we want is:

On visiting the site, user should have an option to type their name and choose their Profession
Upon submitting, based on the user’s chosen profession, they should see the next screen.
The screen behaviour based on professions should be controlled from the Flagsmith console.
Change the list of professions dynamically on Flagsmith console and see realtime updates on our website

Integrating Flagsmith with Next.js Application

Add dependencies

We need to first add the flagsmith package. Run the below command: \



npm i flagsmith --save

Initialise the Flagsmith client

Now, we need to add code to initialise the flagsmith client.

TL;DR - Pull Request

Since we want the flagsmith features to be loaded and present in the state, we will create a provider file which handles loading the flag state and supplying it to our app pages. This allows us to use the FlagsmithProvider.

Thus, create a file named app/provider.tsx. The content inside it should be as follows:



"use client";

import flagsmith from "flagsmith/isomorphic";
import { FlagsmithProvider } from "flagsmith/react";
import { IState } from "flagsmith/types";
import { ReactElement } from "react";

export default function Provider({
  children,
  flagsmithState,
}: {
  children: React.ReactNode;
  flagsmithState?: IState<string, string>;
}) {
  return (
    <FlagsmithProvider flagsmith={flagsmith} serverState={flagsmithState}>
      {children as ReactElement}
    </FlagsmithProvider>
  );
}

The Flagsmith provider will allow all of its child components to access feature flags via the useFlags hook. In our example, we will fetch these flags from the server and pass them to the provider via the serverState property.

We will be importing this provider inside our app/layout.tsx file. This is also the place where we call the init function for getting Flagsmith feature flag state.

Add the 2 imports as shown below:



import Provider from "./provider";
import flagsmith from "flagsmith/isomorphic";

The RootLayout in the app/layout.tsx file needs to updated as shown below:



export default async function RootLayout({
  children,
}: {
  children: React.ReactNode;
}) {
  const flagsmithState = await flagsmith
    .init({
      // fetches flags on the server
      environmentID: "heiEFz5x78igSLA8fGRgiP", // substitute your env ID
      identity: "my_user_id", // specify the identity of the user to get their specific flags
    })
    .then(() => {
      return flagsmith.getState();
    });
  return (
    <html lang="en">
      <body>
        <Provider flagsmithState={flagsmithState}>
          {children as ReactElement}
        </Provider>
      </body>
    </html>
  );
}

Remember to substitute your env specific environmentID in the layout.tsx file.

To get the environmentID, navigate back to the flagsmith website.

Click on “Settings” for your environment, then click “Keys”.

Ensure that you are still able to successfully run the application by running:



yarn dev

Create a new card for the about page

As described in the agenda, we will add a new Card. This is the card that we intend rolling out in a phased manner based on user profession / segment.

TL;DR: Pull request

The below code can be added to the Page() function inside app/about/Page.tsx file to get this new card.



<a
  href="https://vercel.com/new?utm_source=create-next-app&utm_medium=default-template&utm_campaign=create-next-app"
  className={styles.card}
  target="_blank"
  rel="noopener noreferrer"
>
  <h2 className={inter.className}>
    Configure <span>-&gt;</span>
  </h2>
  <p className={inter.className}>
    Configure your feature rollout with&nbsp;Flagsmith
  </p>
</a>

Once this code is merged and deployed, this change will be visible to all the users who visit our website.

However, that’s not what we want. As a principal, new features are added incrementally. What if the Configure link was broken? What if there was an issue in the flow inside the configure? What if user’s have started hating the Configure card and we want to remove it?

What if I told you all this control is possible with just a single click?

Let’s add a phased rollout mechanism, so that only users with the profession “Freelancer” are shown this card first.

For this, we need to add a new feature on the Flagsmith dashboard.

Creating a Feature

Click on “Create Your First Feature” option below:

I’m naming the feature “configure_card_controller” since it controls the card named “Configure”

Create Segment override

Let’s edit this feature and create a Segment override. A segment override essentially allows you to configure the feature flag behaviour based on traits of the user. We will use the trait as “profession” in our example.

Click on the feature and navigate to the “Segment Overrides” tab. Click on “Create Feature-Specific Segment”

Adding an override for users with profession = freelancer.

Segments can also be created globally for the project but in this case we just wanted the segment override for this one feature.

Segment overrides let you specify the enabled state and remote configuration of the feature. You can add as many segment overrides to a feature as you want, the priority of the overrides are dictated by their order.

Now that the segment override is added, Turn it on and save the override. Note that by default the feature is turned off, and only for this segment we have turned it on.

Control card with Flagsmith

TL;DR - Refer to this Pull request for the code changes - https://github.com/abhishekag03/vercel-flagsmith-integration/pull/11

Add the below import to app/about/page.tsx



import { useFlags } from "flagsmith/react";

Initialise the flag variable by telling it which feature flag key needs to be read



// only causes re-render if specified flag values change
const flags = useFlags(["configure_card_controller"]);

Now, wrap the newly added card (“Configure” card) inside a condition, which checks for this flag.

The way we read the value of our flag is:



flags.configure_card_controller.enabled

To conditionally view the configure card, we will just enclose that card markup inside the following:



{flags.configure_card_controller.enabled &&
    (
      <a 
        ... // the card segment code
      </a>
    )}

In order to get the appropriate screen when the user reaches this screen, we need to make sure that the flag trait gets set appropriately.

We will modify the handleOnSubmit function to set the trait.

Add the below import and the line highlighted in green to the handleOnSubmit function inside app/page.tsx file.

The following code sets a Flagsmith trait value based on user selection. Setting a trait is async, it will fetch the latest flags, considering any new segment overrides the user may have as a result of the trait.



import flagsmith from "flagsmith/isomorphic";

const handleOnSubmit = async(e: FormEvent<HTMLFormElement>) => {
    e.preventDefault();
    await flagsmith.setTrait("profession", user.position); // setting the "profession" trait as the one selected by user

 router.push("/about?name=" + user.name + "&position=" +   user.position);
  };

We need useRouter to get the information passed from the Login page.

We need to specify what flags and traits we are going to use in the code. As shown above, we are using the “configure_card_controller” flag and “profession” trait. This makes the flags variable tightly coupled with these names, thereby reducing our chances of error while coding.

Testing and Experimentation

Deploy this code to Vercel and start testing.

Notice that when you choose the profession as Freelancer, you will see 5 cards. The “Configure” card should be present.

On choosing any other profession, only 4 cards are visible:

You might be thinking that this could have been achieved with a trivial if/else condition - why do we need Flagsmith?

Flagsmith allows you to control these configurations on the fly without needing any code change or any redeployment.

For instance, if we see a good response from this new card - we would want to extend it to another Profession. To do that, we just need to go to the flagsmith UI and add the new profession name.

Let’s try that - update the segment to also allow the “designer” profession to see this new card.

Edit the segment and add a condition for profession = “designer”.

On redeploying, you can see that both Freelancer and Designers are able to see this new card. The third profession (Frontend Developer) is still seeing the 4 cards.

Let’s push this code and get it deployed to our main site.

The code is now live, and we are free to update the user segment and rollout as we need. You can test it running on https://vercel-flagsmith-integration.vercel.app/

Without any pull requests, code change, redeployment or testing, we were able to alter the behaviour of our website. This is a very powerful integration and can be used extensively for all small to large scale projects.

Conclusion

In conclusion, through this article, we demonstrated the practical application and significance of feature flags in a Next.js project hosted on Vercel. By integrating Flagsmith, we showcased how Next.js feature flags empower developers to reduce deployment risks, experiment safely, and deliver personalized user experiences. The ease of dynamic updates and segment overrides highlights the agility and control feature flags provide without code changes or redeployment. Through this hands-on approach, you can fine-tune feature rollouts and adapt your application based on user feedback, making feature flags a valuable tool for iterative and user-centric development. Embrace feature flags to unlock new possibilities and deliver exceptional user experiences in your software projects.

Hope you found this article useful and this integration helps you in building out applications!

Improving your React Native application performance with react-navigation-focus-render

Kyle Johnson — Sun, 20 Feb 2022 15:25:46 +0000

Performance in React Native has always been a bit of a battle, great performance is achievable but is a lot more sensitive to unoptimized code than traditional web development.

Background

I'd recently discovered an issue in my application where my home tab screen had a collection of components that contained multiple re-renders.

The problem was quite straightforward to resolve, but during this, I found out that this non-performant screen was slowing down the other tabs within my application.

A simple example of why this happened

Let's make a simple example that replicates this issue.

ExpensiveComponent

Here's our component that's causing the issue, every time this renders we'll see an obvious performance hit. It's connected to redux and will re-render whenever the state changes for the count.

const ExpensiveComponent = () => {
  const {count} = useSelector((state) => ({
    count: state.count,
  }));
  return  (
    <>
      {!!count && <Text>Count is {count}</Text>}
      {new Array(5000).fill(0).map((v, k) => (
        <Text key={k}>{v}</Text>
      ))}
    </>
  );
};

HomeScreen

Our home screen renders the expensive component and lets us go to screen 2. As long as ExpensiveComponent is re-rendering we will see an obvious performance hit.

const HomeScreen = () => {
    const navigation = useNavigation();
    const goScreen2 = ()=>{
        navigation.navigate('Screen2')
    }
    return (
        <>
            <Button
                title={'Go to Screen 2'}
                onPress={goScreen2}
            />
            <ExpensiveComponent />
        </>
    );
};

Screen2

Our second screen has no performance issues by itself, it contains a button that dispatches an action to update the count. It doesn't render much and you'd expect pressing the button and displaying an updated count to be immediate.

const Screen2: React.FC<ComponentType> = ({}) => {
    const {count} = useSelector((state) => ({
        count: state.count,
    }));
    const dispatch = useDispatch();
    const setCount = useCallback(
        (data: number) => {
            return dispatch(AppActions.setCount(data));
        },
        [dispatch],
    );
    const onPress = ()=> {
        setCount((count || 0) + 1)
    }
  return (
      <Button
        onPress={onPress}
        title={`Update Count (${count || 0})`}/>
  );
};

You would expect Screen2 to have no performance issues right? Wrong. Pressing the update count button was consistently blocking the UI by around 250ms, this can be seen by using a tool I made, react-native-performance-monitor.

So why did this happen?

The reason made sense, other tabs were using updating the state that was also used on the HomeTab and as it turned out, inactive tabs will re-render even if they aren't shown.

Even with optimized components, the fact that this happens is useful to be aware of. If you have 5 tabs in your app that have been visited in a session, any state updates can trigger re-renders on all of them.

Preventing this behavior with react-navigation-focus-render

This is where my new npm package comes in. By wrapping the render our ExpensiveComponent in <FocusRender, children are not re-rendered until the screen is focused

const ExpensiveComponent = () => {
  const {count} = useSelector((state) => ({
    count: state.count,
  }));
  return  (
    **<FocusRender>**
      {!!count && <Text>Count is {count}</Text>}
      {new Array(5000).fill(0).map((v, k) => (
        <Text key={k}>{v}</Text>
      ))}
    **</FocusRender>**
  );
};

Simply by adding this, our example is more performant. Here's the comparison to prove it:

This shows an average render time of around 6ms vs the original 250ms.

How does this work?

It's quite simple, This module works by preventing screen re-renders of inactive screens until that screen is focused.

The entire code can be found here. It uses the useIsFocused() hook provided by react-navigation combined with a classic shouldComponentUpdate, returning true only if the screen is focused.

To illustrate this clearly, here's how the above example behaves.

As you can see, it's noticeable when this re-render occurs due to how expensive the component is. However, in cases less extreme, it's more likely to behave like the following:

Conclusion

Although this library should not replace optimizing components, I believe with big applications this library can introduce big performance benefits.

Let me know if you find this useful, you can check it out on GitHub :)

Happy Hacking!

Using Feature Flags to troll our competitor without a line of code 😃

Kyle Johnson — Mon, 14 Feb 2022 20:40:26 +0000

The back story

📖 When we built Flagsmith, we decided early on to power the admin dashboard with our own Feature Flags. As well as following a methodology we truly believed in, this meant we could be super flexible in how our admin panel behaves.

This proved very useful in the story I'm about to share.

How it began

🕒 It's Monday 6 pm and I'm just about to wrap up for the day.

Early on during product development, we wrote a crude SlackBot that alerted us to signups from new email domains we haven't seen before. At 6:38 pm UK Time, we saw a new domain, it was our biggest feature flag competitor.

Laying the foundations

The first step was to make sure this prank wasn't going to affect anyone else, I may have been caffeine depleted but I wasn't going to go full-on cowboy.

💡 I created a segment to match any user from the company's domain, users are given a trait of their email address when they log in so I was easily able to target that trait with a regular expression.

😇 In the interest of privacy I've hidden said domain from the post. If someone from there is reading this, you're welcome.

🕵️‍♂️ I also added in my email so I could see what they see, think of it as testing in production.

🤔 At this point I had quite a few options.

A simple butter bar

I felt like off starting off lightly, just a simple hello to let them know we know.

Thankfully this was really easy, we have a butter bar messaging feature to send out suitable messaging to segments of users.

😃 Our cheeky user segment should definitely get a personalised message.

That was easy, now they see a lovely message on every page they take a look at!

A more bespoke experience

This was nice but I felt like I could do more for them.

⭐ I decided they deserved their very own SDK integration on our platform. I'm sure this is something they'd enjoy.

The result

This bit of remote config leads to a much more personalised user experience for our new users.

👌 In the end there can only be one feature flag platform.

What I've learned creating a React / React Native performance monitor

Kyle Johnson — Fri, 20 Mar 2020 18:01:44 +0000

This post covers an introduction to react-native-performance monitor, a realtime performance monitor for React Native I've just made (which would also work for React). The tool is completely open-source, published on npm and can be checked out here.

(original post here).

Motivation

Even for the most experienced React Native developer, maintaining and improving performance in a large React Native app can be far from trivial. Also, the act of implementing performance improvements is an art in itself. Based on experience here's my checklist for a starting such an approach:

1 - Don't jump straight in, get a benchmark!

This is probably the most important point, and above all, is the main motivator for this tool.

Although it's very tempting to jump in and chop away at ugly lines of code, without a real measurement of a before and after a lot of the times you're pretty much none the wiser how big of an impact you made (the odd change here and there may even introduce a performance deficit)

2 - Make sure there's a good cost-benefit

Simply put, if you've reduced the maintainability and readability of the code for a couple of milliseconds improvement, it's probably not worth it. To reiterate the first point, this is made a lot easier if you have numbers to back this up, don't introduce bias and the phrase "it feels faster".

3 - Record your findings

You've reached a mind-melting conclusion that will improve your React Native knowledge forever, great! Record your finding and document them / share it with your team.

4 - Periodically retest

Without a good way to test performance again at a later date, you could find that your hard work is ruined by regression.

Profiler in React Native

As of React Native 0.61.0, the React Profiler is now officially stable. Profiler component exposes the following callback as a prop:

function onRenderCallback(
  id, // the "id" prop of the Profiler tree that has just committed
  phase, // either "mount" (if the tree just mounted) or "update" (if it re-rendered)
  actualDuration, // time spent rendering the committed update
  baseDuration, // estimated time to render the entire subtree without memoization
  startTime, // when React began rendering this update
  commitTime, // when React committed this update
  interactions // the Set of interactions belonging to this update
)

This release allows you to also profile interactions much like you'd do in chrome on the web.

Although useful, I have a few problems with using this to test performance and iterate.

The feedback loop is slow. This requires a process of recording, writing down the numbers, making a change and then recording again.
The information can kind of become a bit overwhelming and confusing to work with, this is especially true with larger applications with many nested components.
The output of the tests depends on how quickly you hit record and performed the interactions.

I get the best use out of this by using it to figure out a starting place of what to work on. But in my mind, I feel like making performance improvements needs a different tool, one that provides immediate, a/b comparison based feedback.

Introducing react-native-performance-monitor

Considering the above I decided to get to work in making this tool. I've been using it for a few weeks and believe the best approach is to:

1 - Identify an area of the application you feel needs improving (maybe via the React Profiler).
2 - Record your baseline by either triggering a component mount or force update, I added x5 and x10 buttons to get a better average baseline. Whether you choose to test remounting or force updates depends on the nature of what you are trying to test, if your component receives frequent updates somewhere in the tree you may see a lot of benefits just concentrating solely on updates.
3 - Whilst you are developing pause the recorder, then when you want to out your changes add a variant and click resume.
4 - Repeat 3 as many times as you need to to reach an improvement, to reduce the noise you can always clear tests that you wish to discard.

I thought I knew React Native inside and out having worked on it since 2015, but seeing metrics for every little change opens up a rabbit hole of things you thought you knew. It's strangely addicting to play around with.

The amount of subtle changes with components and their impact has surprised me, for example:

This difference was entirely down to the following code. I, of course, could understand why this would be the case however I'd not predicted the effect this change would have.

Before:

<Text style={[this.props.style]}>
    {this.props.children}
</Text>

After:

<Text style={this.props.style}>
    {this.props.children}
</Text>

How it works

The overall implementation is quite straight forward and simply involved passing the onRenderCallback values via a WebSocket server to finally render them in a fancy graph.

There are 3 main components:

A React Native component that sends profile data to a server via REST and listens to messages (remount and force update) to trigger renders.
A Web socket server responsible for passing messages between the graph and the react-native component
A Web application that receives WebSocket values and renders them to a graph

The following diagram is a brief explanation of the data flow:

Next steps

Hopefully this has encouraged you to also jump into the rabbit hole of optimising your app. This project is still in its infancy but I'm open for feedback and how it could further help everyone out. Check it out and let me know!

Achieving a perfect 100% Google Lighthouse audit score with Next and Redux

Kyle Johnson — Mon, 26 Aug 2019 11:10:17 +0000

(Moved to here)

This post covers how we can build a React/NextJS app with Redux that achieves a 100% audit score with server-rendering, localisation support and can be installed as a PWA and navigated whilst offline.

next.js

next.js is my new favourite thing. Built specifically for react, NextJS lets you server render your react application with little compromise to how you would normally build your app.

Developing a React app will be pretty familiar, you'll have to switch out react-router with their built-in router, and be aware that your components will have to be executable in NodeJS (just like if you were unit testing them).

The main difference is this bit of magic which we can add to our pages:

// Calls before the page is mounted, the call will happen on the server if it's the first page we visit
static async getInitialProps({ ctx: { store } }) {
  await store.dispatch(AppActions.getWidgets());
  return {};
}

Any asynchronous tasks or fetching can occur here on our pages.

Rather than regurgitate all of the power of next, I'd recommend just stepping through their getting started guide. This post details how I added redux, sagas and achieved a 100% score on Lighthouse.

I'm bored, just send me the code.

Fine. The project is also hosted at https://nextjs-redux.kyle-ssg.now.sh/. But read on if you're interested.

1. next.js with Redux

Rather than defining routes within JavaScript, routes in next are based on what's in your /pages directory.
Next.js defines how pages are rendered with an App component, which we can customise by making our very own _app.js. Great, that means we can create our store and give it our root app component just like any other app.

import App, { Container } from 'next/app';
import Head from 'next/head';
import React from 'react';
import { Provider } from 'react-redux';
import createStore from '../common/store';
import withRedux from 'next-redux-wrapper';
class MyApp extends App {
    static async getInitialProps({ Component, ctx }) {
        let pageProps;
        // Ensure getInitialProps gets called on our child pages
        if (Component.getInitialProps) {
            pageProps = await Component.getInitialProps({ ctx });
        }

        return { pageProps };
    }

    render() {
        const { Component, pageProps, store } = this.props;
        return (
            <Container>
                <Provider store={store}>
                    <>
                        <Head>
                            {/*...script and meta tags*/}
                            <title>TheProject</title>
                        </Head>
                        <Header/>
                        <Component {...pageProps} />
                    </>
                </Provider>
            </Container>
        );
    }
}

export default withRedux(createStore)(MyApp);

Some of this will probably look familiar to you, the main differences being:

In our route app, we need to make sure our pages getInitialProps functions are being called before rendering
Next.js provides a Head component that lets us render out any standard tags that live inside the head, this can even be done per page. This is useful for adding opengraph/meta tags/titles per page.
next-redux-wrapper is an out of box library that lets us use createStore.

The outcome

Adding a simple get widgets action we can see the following differences depending on if we loaded the page from landing straight on it vs navigating to it from another page.

This happens because getInitialProps is called on the server during the initial page load, it knows which page to call it on based on the route.

2. Achieving a 100% Lighthouse score

Even locally, I noticed how fast everything felt. This leads me to wonder how performant I could get the page. Within chrome dev tools there's a great tool called L that rates your site based on several recognised best practices and meets the progressive web app standard.

Baseline score

The baseline score was not too bad, with performance not being a problem for a redux page hitting an API.

Accessibility

Most of these items are trivial to solve and involve employing best practices such as image alt tags, input roles and aria attributes.

Appropriate colour contrast

Lighthouse is clever enough to know which of your elements are not meeting the WCAG 2 AA contrast ratio thresholds, stating that your foreground and background should have a contrast ratio of at least 4.5:1 for small text or 3:1 for large text. You can run tools such as Web AIM's contrast checker. A quick CSS change fixed this but obviously, this will mean a good amount of refactoring for content-rich sites.

Localisation

This one was a little more tricky. To do a good job of this I wanted the serverside render to detect the user's preferred locale and set the lang attribute as well as serve localised content. Searching around I did come across next-i18next, however, I noticed that it doesn't support serverless and it's difficult to share locale strings with react-native-localization.

I wanted something that would work with react-localization, so my approach was as follows:

1: When the document attempts to render on the server, we want to get the preferred locale and set the lang attribute to the HTML tag. This info comes from the server, either from a cookie which we could set or by parsing the Accept-Language Header. A code snippet for how I did this can be found here.

    // _document.js
    static async getInitialProps(ctx) {
        const initialProps = await Document.getInitialProps(ctx);
        const locale = API.getStoredLocale(ctx.req);
        return { ...initialProps, locale };
    }
    ...
    render() {
        return (
            <html lang={this.props.locale}>
                ...
            </html>
        )
    }

2: I define some localised strings

// localization.js
import LocalizedStrings from 'react-localization';

const Strings = new LocalizedStrings({
    en: {
        title: 'Hello EN',
    },
    'en-US': {
        title: 'Hello US',
    },
});

export default Strings;

3: I want my app to know what the locale is in a store so that I can use that information later.

    // _app.js
    static async getInitialProps({ Component, ctx }) {
        let pageProps;
        const locale = API.getStoredLocale(ctx.req); // Retrieve the locale from cookie or headers
        await ctx.store.dispatch(AppActions.startup({ locale })); // Post startup action with token and locale
        ...
    }

4: I set the language once in my app on the initial client and server render.

// _app.js
render(){
        if (!initialRender) {
            initialRender = true;
            const locale = store.getState().locale;
            if (locale) {
                Strings.setLanguage(locale);
            }
        }
    ...
}

5: In my pages, I am now free to use localised strings.

    // pages/index.js
     render() {
            return (
                <div className="container">
                    <h1>Home</h1>
                    {Strings.title}
                </div>
            );
      }

Best practices

Since the project had pretty up to date libraries and didn't do anything unruly, this already had a good score. The only thing we had to do was use http2 and SSL, which is more down to how you're hosting the application. Using Zeit covered both of these.

SEO

Thanks to nextJS you can easily add meta tags on a per-page basis, even using dynamic data from getInitialProps.

Progressive web app

PWAs make our web apps installable, combined with service workers we can serve content whilst the user is offline.

The first step was to add a simple manifest, this lets us configure how it should behave when installed.

/static/manifest.json
{
  "short_name": "Project Name",
  "name": "Project Name",
  "icons": [
    {
      "src": "/static/images/icons-192.png",
      "type": "image/png",
      "sizes": "192x192"
    },
    {
      "src": "/static/images/icons-512.png",
      "type": "image/png",
      "sizes": "512x512"
    }
  ],
  "start_url": "/?source=pwa",
  "background_color": "#3367D6",
  "display": "standalone",
  "scope": "/",
  "theme_color": "#3367D6"
}

//_app.js
<link rel="manifest" href="/static/manifest.json"/>

Offline support with service workers

Thanks to next-offline, adding service worker support was simple. Getting the service worker to work with serverless and hosted on Zeit however was a bit fiddly, we had to add a route for our server to serve the correct content header.

// now.json
{
  "version": 2,
  "routes": [
    {
      "src": "^/service-worker.js$",
      "dest": "/_next/static/service-worker.js",
      "headers": {
        "Service-Worker-Allowed": "/"
      }
    }
    ...
  ]
}

And then configure next-offline to serve the service worker from static.

next.config.js
{
    target: 'serverless',
    // next-offline options
    workboxOpts: {
        swDest: 'static/service-worker.js',

The result

As a result of this, we now have a solid base project with a 100% audit score, server-rendered, localised and can be installed and navigated whilst offline. Feel free to Clone it and hack around!

Our Tools - 2019 Edition

Kyle Johnson — Mon, 24 Jun 2019 09:09:12 +0000

Here's what we are using to build Bullet Train, our Feature Flag platform. With some careful VPS purchases, most of this stuff can be run completely free of charge!

(Moved to here)

GitLab All The Things

GitLab is pretty awesome, and it gets more awesome with every release. The big thing it has going for it is that it integrates a bunch of different tools, and makes them work together to help drive productivity. This also means that a single GitLab instance can manage a large part of the development process.

We self-host GitLab. It is a fairly big application so we're running on a 3GB VM on Google's Cloud Platform, but you could easily use a cheaper VPS provider if you wanted to. The open source version of GitLab is called their "Community Edition", but don't be fooled, it's fully featured and is only missing a couple of items from their paid product. It installs pretty easily via their Omnibus packages, and updating is pretty much just

apt-get update; apt-get upgrade

You can also use their hosted product as we do, it has a great free tier.

Source Code

Well, yeah obviously. All our code is pushed to GitLab. Being open source and self-hosted there are no limits to the number of developers or private projects you can host.

CI/CD

GitLab has a first class CI/CD platform built in. You can get push-to-deploy working quickly and easily. There's only a couple of things you need to do to get up and running.

Add a .gitlab-ci.yml file to the root path of your project. This file contains the commands you need to build your product. 2. Register a GitLab Runner, basically, a service that runs somewhere that can receive jobs from GitLab, run them, and send the results back to GitLab. You can even run the runner on the same machine as the GitLab Instance.

Agile Project Management

We still think Trello is the best of the bunch. It's fast, simple and gets out of the way. Having said that, they have recently been making some changes to their free tier, and so the one other option that might be worth checking out is, again, back in Gitlab.

You can use the Gitlab issue tracker to manage requirements, and view them in a Kanban board, similar to Trello. A big advantage of doing this is, again, the tight integration between the Gitlab components. For example, if you push a commit with the message

Fixes #252

, issue number 252 will be referenced by that commit. When you merge that commit into the master branch, Gitlab will even mark the issue as resolved! There's a bunch of other integrations that use this pattern.

Dev/Staging Builds

We're using the amazing Dokku to run dev and feature branches. Dokku is an amazing bit of software that basically turns your VPS into a mini-Heroku platform. Its pretty simple to create and destroy feature branches, as well as build and deploy development builds. We use this Docker image to integrate Gitlab CI with Dokku.

Production Hosting

This really depends on the application you are working on. For Bullet Train, our Feature Flag platform, we actually split out the hosting:

Our Web front-end is written in Node/JS and deployed to Standard AppEngine. It's mega-cheap (as in practically free) and rock solid.
Our API is written in Django/DRF and uses a Postgres database as its data store. We host this on AWS and Elastic Beanstalk. This gives us decent automated scaling options and a simple deployment process.

Feature Flags

Well, we definitely use our Feature Flags to help build our Feature Flags! To ensure a smooth CI/CD process, and to reduce the amount of code that is committed but not deployed, we use our Bullet Train Feature Flags platform within Bullet Train itself. Meta. We have a decent free tier too, so we don't have to pay ourselves to run our feature flags.

Support, Email Etc

All free! We're using FreshDesk for our customer issue tracker, FreshStatus for our status page and Uptimely for our uptime monitoring/outage alerts.

For email, we're using the pretty awesome Migadu for inbound email, and SendGrid for our outbound email.

For marketing, MailerLite to manage drip campaigns, and the free tier of Intercom to do our in-app chat.

Overcoming bottlenecks between design, frontend and backend developers

Kyle Johnson — Wed, 29 May 2019 19:13:10 +0000

(Moved to here)

During my job as a frontend developer for Solid State Group, I've seen several scenarios which block work for specific areas of the development team. Be it design, markup/style implementation, technical frontend or backend, at one point or another they reach a point where they are relying on someone to complete some work before they can get on with their task.

Over the past few years, I've been quite interested (rather selfishly) in finding as many ways as possible to get to a place all these people can work happily in parallel as much as possible without being ground down to a halt.

Some scenarios

To set the scene, here are some examples of the sorts of things I'm referring to:

A frontend developer wants to start building out a feature but the API isn't ready yet.
A backend developer wants to start building out a feature but is not sure what frontend needs in terms of data.
A backend developer is making some changes (e.g. removing some fields from a DTO) but is not sure if changes will break frontend.
A technical frontend developer (who does not generally touch markup/CSS) is developing a feature but not sure what markup /styles are available to use.
A design based frontend developer (who does purely markup/CSS) is ready to replicate designs but the technical frontend implementation or data isn’t ready.
A designer adjusting markup but is unsure of how to get to the page required.
A developer is validating a particular scenario issue or feature but has to go through tedious app state(onboarding etc) to get to the place they want.
A developer new to a project needs onboarding from another developer to understand the codebase.

Here are some solutions...

Developing with pact-js

My recent experience with Pact is probably worth a post of its own, it has changed how we approach development on larger projects for the better and has saved an immense amount of time.

Pact, by itself, is a contract testing tool. A consumer of data (in this case the frontend) defines what it wants from a data provider and publishes this to a broker before doing any work. Developers of the provider (in this case the API) reference this and use it to develop against.

Both frontend and backend developers can work against this single source of truth once it is defined, using it as a basis to discuss changes if they arise. Expanding on Pact a bit, we've built on top of it to automatically generate a local mock server on the frontend.

As a result, often we've found that we can develop a large chunk of an application only then to flip the switch from localhost to a development API to start integration testing. During this time both the frontend and backend developers aren't waiting for each other to complete bits of work before knowing where they stand.

Before Pact

We'd either have to wait for backend or frontend to be built first then wait for each other as we eventually deemed things weren't suitable.
Optionally we could use something like Apiary. This albeit a great tool will not help you track changes and test that the current API is doing what the frontend needs.

After Pact!

Pact files are used as the single source of truth, tests can be written to ensure changes to the API payload won't affect the frontend.
A frontend developer can use the Pact JSON as a mock server, this is really useful for developing when you can't reach the API or when endpoints don't exist/haven't been changed yet. This is much better than working against a static JSON variable because when the API is ready you can simply switch the endpoint from localhost.

Component based development and a style guide page

Recently in our web and mobile projects, we've kept a page that contains every single component the application presents. In particular, I've found this works really well with React and React Native.

Non-technical developers on the design side can implement the designs without getting bogged down in application logic, while technical developers have a reference to what markup to use when implementing new features.

Not only this, but it often means that any bugs or tweaks to do with appearance can be worked on once in a single place rather than tracking down a certain screen or workflow.

Before style guide pages

Technical developers would have to remember what CSS classes and components correspond to certain designs.
Non-technical developers would get bogged down trying to find where to implement markup.
To implement a change, developers would have to navigate the appropriate page, often including tedious workflow such as user registration/onboarding.

After style guide pages!

Technical developers can easily find and reuse components already developed by designers.
Improves overall codebase by making developers develop simple, reusable components.
Non-technical developers have a simple page that isn't bloated by code to implement the markup for their designs.

Add teardown endpoints

Being able to efficiently test workflows and clear down the data saves a lot of time and prevents the need from relying on backend developers to clear down databases.

Before teardown endpoints

Development environments would get filled with garbage test data.
Testers or developers may need to rely on data being cleared down before they can continue testing.

After teardown endpoints!

Workflows can be tested over and over and cleared down without any repercussions.

Develop your application so that you can simulate complicated scenarios

Having covered this in a bit more detail in my post here, this idea is more about developers not being throttled by the complexity of a large app.

I must have spent days on old projects constantly registering as a new user to test out onboarding or messing around with integrations with hardware just to get to the part I'm working on.

The solution is to build your app in a way that it can be manipulated into certain states via a constants file, or better yet a remotely configured feature flags

Before simulation flags

Diagnosing and fixing bugs/features took a lot longer to replicate.
Checking what happens in certain scenarios requires a lot of manual effort.

After simulation flags

Demonstration/replicability of scenarios takes a lot less time.
Bugs and features in obscure parts of your application become a lot easier to develop on.

Standardising project structure and setup

I won't go into how much developers should rely on boilerplate magic after all every project is different. In our case, we build 10s of projects per year, so having a standardised setup for development has been crucial.

Previously, I'd seen developers sit together for half a day or more getting people set up and aware of the oddities of a particular project. Having a base layout project makes developers new on a project feel like they've worked on it before.

If you are interested, here's a lite example of what our frontend project is based on.

Before Base layouts

We'd have to invest a lot more time into initial project setup and infrastructure.
Working on new projects was sometimes very painful to get going.
New developers needed more support from the existing team for every project.

After Base layouts!

Developers can start developing on a project a lot quicker and they are familiar with where to begin based on previous projects.
Previous gotchas on projects are already covered based on previous iterations.
Developers can more easily be switched in and out of projects without there being too much reliance on a select few.

Hopefully, people have found some of these tips useful! Let me know what practices you use to help work better together.

Step by Step - Writing end to end tests for your web project

Kyle Johnson — Mon, 10 Dec 2018 23:25:04 +0000

(Moved to Here)

Based on my previous post Hacking our e2e tests to make them more useful, there seemed to be little resources available for writing e2e tests. This is something I've become a strong advocate of since helping write Bullet Train.

This guide starts with a simple project and step by step adds e2e testing, each step links to a git diff so you can see exactly what I did.

If you find this useful or would maybe prefer me to do a video covering this let me know.

The project

End to end tests are super useful for regression testing time-consuming workflows, this example will go over perhaps the prime use-case for this, user registration and confirming a user's email address.

The project is a simple react web application with a node backend, with it you can log in, register and receive a confirmation email address email using SendGrid. Our test will go through all of this workflow so we don't have to keep manually testing it in the future.

The scope of this tutorial isn’t to go over how the application is built, however as a brief overview:

There is a Node API that has endpoints to log in, register and confirm email.
Then on the frontend side we have a few pages written in React that host our login/register/confirm email address fields and functionality.

1. Project Setup

The Project: https://github.com/kyle-ssg/e2e-tutorial/

1.1. Installing nightwatch, selenium and chromedriver

npm i nightwatch selenium-server chromedriver --save

We are going to need to install 3 things to get started:

Nightwatch - a nice API for interacting with web drivers and selenium.
selenium-server - needed in order to run our web driver.
chromedriver - communicates to the selenium server via WebDriver’s wire protocol.

Installing selenium-server and chrome driver as npm modules mean that you don't need to worry about global dependencies to run the tests. You can run this on any machine with different operating systems, and more importantly as part of CI pipelines without having to worry about if and where these binaries come preinstalled.

1.2. Writing a nightwatch.conf.js file

Nightwatch comes with a lot of configuration options which can be a bit overwhelming.

Based on my experience here's the minimum config you'll need. The two key difference here to their default config is:

We allow our chromedriver and selenium-server node modules to define where the binaries are located.
We set end_session_on_fail to false, allowing us to interact with the browser if errors occur. More on how this can be useful here

const chromedriver = require('chromedriver');
const seleniumServer = require('selenium-server');
const os = require('os');

const browserSize = 'window-size=1024,768'; // define how big to make the browser screen

module.exports = {

    'selenium': {
        'start_process': true, // tells nightwatch to start/stop the selenium process
        'server_path': seleniumServer.path,
        'cli_args': {
            'webdriver.chrome.driver': chromedriver.path, // chromedriver from our npm module
        },
    },

    'test_settings': {
        'end_session_on_fail': false, // don't close the browser straight away on fail in case we want to check the state
        'default': {
            'desiredCapabilities': {
                'browserName': 'chrome', // Run the e2e test in chrome
                'chromeOptions': {
                    // In linux we pass a few more arguments
                    'args': os.platform() === 'linux' ? ['headless', 'no-sandbox', browserSize] : [browserSize],
                },
            },
            'globals': {
                'waitForConditionTimeout': 5000, // global default time to wait for element to exist.
            },
        },
    },
};

Rather than go into a lengthy explanation, see the comments above on exactly what each property is used for.

1.3. adding a script in package.json to run tests

In our package.json we just need to write an npm script to run nightwatch.

    "test:e2e": "nightwatch ./e2e/index.js"

1.4. Writing our test entry file

The entry file exports an object containing our tests and a before and after hook:

module.exports = Object.assign(
    {
        before: (browser, done) => { // Runs before tests start
            // runs before all of the tests run
            done();  // tell nightwatch we're done after we have done all of our bootstrapping
        },
        after: (browser, done) => { // Runs after the tests have finished
            // runs after all of the tests run
            browser.end(); // kill the browser
            done(); // tell nightwatch we're done
        },
    },
    // the main tests
    require('./register.test')(),
    require('./login.test')(),
);

1.5. Our skeleton tests

Before we go into writing our e2e tests, the following is some example syntax on how our tests are structured:

module.exports = () => ({
    'Registration - test page loads': function (browser) {
        browser.url('https://google.com')
            .waitForElementVisible('body') // page load
    },
});

We give each assertion a title, then we execute functions using the browser object. This may be

Waiting for an element to be visible
Clicking a button
Setting the value of an input
Switching to an iframe

These functions are called one after another and wait for the previous to complete, they can be chained so that you can write whole workflows whilst keeping tests small e.g:

browser.url('http://localhost:8080')
    .waitForElementVisible("#my-input")
    .setValue("#my-input", "test")
    .click("#my-button")
    .waitForElementVisible("#confirmation-button")
    .click("#confirmation-button");
    .waitForElementVisible(...)

At this point, we are ready to start integrating Nightwatch with our code. We can currently run npm run test:e2e* which launches Chrome, navigates to Google and validates that the body tag is visible.

2. Writing our first test

Code for this section

2.1 Running our application locally with Nightwatch

So now that we have Nightwatch configured and we can run tests on external pages, we now want to do the same but on localhost as if we ran the application locally. The only tricky thing about this is that we need to know our server is ready before we progress with our tests. With a bit of trickery, we can do exactly that with Node's child_process.fork() which spawns a child process that can communicate via IPC.


const fork = require('child_process').fork;

const server = fork('./server'); // start our server

Once we start the server with process.fork() we listen for it to tell our parent process that it's done with process.send({ done: true });. Revisiting the before hook Nightwatch provides we wait for this acknowledgement message before starting the tests. Similarly, when the tests finish we want to kill the server.

 before: (browser, done) => {
            // runs before all of the tests run, call done() when you're finished
            server.on('message', () => { // boot up the server which sends process.send({ done: true }); when ready
                done();
            });
        },
 after: (browser, done) => {
            // runs before all of the tests run, call done() when you're finished
            browser.end(); // kill the browser
            done(); // tell nightwatch we're done
            server.kill('SIGINT'); // kill the server
        },

2.2 Targeting DOM elements sensibly

While we could target our elements via their class name's or id's, I personally feel it's better to target via data-test properties. This way, you have a better idea of whether you're going to affect your tests. To add a bit of syntactic sugar I created the following util:

byTestID: id => `[data-test="${id}"]`,

With this setup, we add data-test="your_id" to the key elements and write our test.

const { byTestID } = require('./util');

module.exports = (email, password) => ({
    'Registration - test page loads': function (browser) {
        browser.url('http://localhost:8080')
            .waitForElementVisible(byTestID('login-page')) // page load
            .click(byTestID('toggle-login')) // click toggle login
            .waitForElementVisible(byTestID('registration-email')) // wait for registration form
            .setValue(byTestID('registration-email'), email) // set fields
            .setValue(byTestID('registration-password'), password)
            .click(byTestID('registration-submit')) // submit form
            .waitForElementVisible(byTestID('check-email-page')); // wait for confirm email page
    },
});

3. Finishing our confirm email tests with mailinator

Code for this section

Most of the tests from here on are generally quite simple to write. The only tricky part was writing a few utility functions that deal with mailinator. This sort of thing you want to pull out from the main tests as you may want reuse to use/maintain them from one place.

 gotoMailinator(browser, email) {
        const target = email.replace('@mailinator.com', ''); // get the mailinator username
        // goto the inbox and wait for the content to be ready
        return browser.url(`https://www.mailinator.com/v3/index.jsp?zone=public&query=${target}#/#inboxpane`)
            .waitForElementVisible('#inboxpane');
    },
    clickFirstMailinatorMessage(browser) {
        // click the latest message in the inbox pane
        browser.waitForElementVisible('#inboxpane table tr td:nth-child(3n)')
            .click('#inboxpane table tr td:nth-child(3n)');
    },
    getMailinatorMessage(browser) {
        return browser.waitForElementVisible('#msg_body') // wait for the content to be ready
            .frame('msg_body') // switch to the message content's iframe
            .pause(1000); // the most reliable way I've found to ensure the content has loaded
    },

Using this and writing some very similar tests to our first we end up with the above code and our end-to-end workflow of user registration, confirm email address and login.

Expanding on this we would start writing tests that check that validation all checks out for invalid data and add test for any new major workflows.

Feel free to checkout the example on GitHub, adding a .env file with SENDGRID_API_KEY="YOUR_KEY" should give you everything you need to run the tests!

Happy hacking!

Hacking our e2e tests to make them more useful

Kyle Johnson — Thu, 22 Nov 2018 16:24:01 +0000

In this post, we're going to run through an overview of my experience using e2e tests for the feature flag platform, Bullet Train. I'll discuss the pros and cons of adding e2e coverage to your project, and how I eventually improved their usefulness with a bit of clever JavaScript. I'll also provide a code example that's used in production for all of you to play with.

What are end-to-end tests

Unlike unit tests which verify individual segments of code are working, end-to-end (e2e) testing is a methodology that is more of a high-level run-through of your project, which tests whether automated interactions against the UI work as expected.

In the case of a web project, we write code that launches a browser and tests the actual application as if we were a real user, interacting with elements and making sure the app behaves as e with both good and bad data.

There are a bunch of frameworks that make this fairly easy to implement, in my case I've found that Nightwatch provides a very terse API and has been easy to work with. There are many alternatives in this area so it's mainly down to personal preference.

The benefits

Increases confidence in the application. We can write all the unit tests in the world, but there's no substitute for clicking around and verifying it all works together.
Tests a lot of the component parts in one swoop with the least amount of effort.
Great for regression. Actually running the application as our test touches everything: the UI, frontend business logic, contracts with API and even the API itself. If any of these things break it can be caught with E2E.

The drawbacks

Can introduce annoying false positives. If the tests are written in a brittle fashion (e.g. looking for li>span>.myClass>input) it's easy to break them by changing the UI.
If the UI of your project is constantly changing, the tests can be costly to maintain. Unit tests can often go untouched as they are isolated, however constant UI changes may require e2e tests to be maintained and regularly updated.
Sometimes tests failure provide poor visibility of what the root cause of the error actually is.

This last point brings us to our topic at hand.

Better Root Cause Identification

In the strive for achieving fault tolerance and redundancy, we recently we migrated the Bullet Train API over to AWS. The move went pretty well, however we hit a brief issue where users were unable to create organisations. Straight away the E2E tests started shouting at us:

Ok, great. But what does that actually mean? Thankfully, in this case, I had a pretty good hunch. We never reached the project selection page, due to us not being able to register. I clicked around on the site and figured out there was an API issue. This took a little while, but eventually, we fixed the API and our tests started to pass again. Although our tests caught the error, it took us quite a while to gather all the information we needed together to fix it. Clearly, the current level of error reporting wasn't good enough, so we set out to improve this.

1. Sending screenshots of e2e tests to slack

This part was quite straightforward. Any selenium framework (even if it uses headless PhantomJS) has the ability to take screenshots of the browser in its current state. And fortunately Slack has a great API for uploading images to a channel.


const Slack = require('node-slack-upload');
const slack = new Slack(process.env.SLACK_TOKEN); 
const uri = path.join(__dirname, 'screenshot.png');
...
        browser.saveScreenshot(uri, ()=> {
            slack.uploadFile({
                file: fs.createReadStream(uri),
                filetype: 'auto',
                title: "Screenshot",
                channels: Process.env.E2E_SLACK_CHANNEL},
            }, function (err, data) {
              ...
            });
        });

This is our basic starting point. Nightwatch provides a hook called after that gets called after our test finishes (either by error or from finishing successfully). We just needed to make sure the browser doesn't close automatically when tests finish, so we can check what was left after the tests had run.

Since we host this publicly on GitHub we make sure to always hide our sensitive tokens behind env variables!


//nightwatch.conf:

"test_settings": {
        "default": {
            "end_session_on_fail": false,
            ...
        }
        ...
}


//index.test.js:
module.exports = Object.assign(
    require('./test1.js'),
    require('./test2.js'),
    {
        after: (browser, done) => {
           uploadScreenshot(browser)
           ...

           server.kill('SIGINT');         
           browser.end();                 
           done();
       }
   }
)

And voila, we get our screenshot sent to slack when our test finishes!

2. Reporting API errors

This was where things get a bit clever. A common problem of end-to-end testing is the visibility of what's actually going on under the hood. After all, we're only really checking the state of DOM elements. Errors at the API or database level are a world away.

So in order to report 'deeper' applications errors, our solution is to have our site write any relevant logging info to the DOM that we can then use later.

We want to ensure this only happens when end-to-end tests are running, otherwise, we might accidentally leak out sensitive information to regular users.

Tell the frontend we are running E2E


//package.json:
    "test": "cross-env E2E=true nightwatch ./tests/index.test.js",

``

We set the environment variable E2E to true so we can tell WebPack to build the application in E2E mode.

``

plugins: [
    new webpack.DefinePlugin({
        E2E: !!process.env.E2E
    }),
...
]

Webpack's DefinePlugin allows us to set global variables for our site to access. In this case, window.E2E will now match our environment variable.

Writing debug information to the DOM


 //Handle all requests
if (E2E) {
    const payload = {
        url,
        options,
    };
    document.getElementById('e2e-request').innerText = JSON.stringify(payload);
}

fetch(url, options)

...
//Handle all responses
(response, err) => { // handling api errors
    req = fetch(url, options);
    if (E2E) {
        const error = {
            url: response.url,
            status: response.status,
            error: err,
        };
        document.getElementById('e2e-error').innerText = JSON.stringify(error);
    }
}

We then use this E2E variable to write our debug info to DOM elements. Which we send over to slack.

A real-world example

If you're curious to how this actually gets used in production, here's the commit which is now running in our gitlab pipelines https://github.com/SolidStateGroup/bullet-train-frontend/commit/4a1d41b3ea103a3c2b823803d3fa273eae8bd49f.

Happy hacking!

A sensible approach to Cross platform development with React and React Native

Kyle Johnson — Mon, 27 Aug 2018 15:11:31 +0000

The ability to develop simultaneously for Android, iOS and recently Windows Phone is often advertised as the biggest selling point for businesses considering adopting React Native. Arguably an even bigger and often overlooked opportunity this tech stack brings is the ability to extend this cross platform ecosystem to web with React.

The end goal here is to be in a position where you can achieve and sustain feature parity with your web and mobile apps, written by a single team and have the ability to write new features and fix bugs simultaneously with a single codebase.

This guide sets out an approach to building and maintaining a codebase that sensibly maximises shared functionality across all platforms. It concentrates solely on React/React Native so that you don't get bogged down with dependencies and state management systems like Redux and Mobx.

Project structure

This may seem like an overly obvious place to start but it's important to set out a clear project structure with cross platform in mind so that you and your team know where everything lives.

/common

As the name states, any code that can be shared between web and mobile goes here. Whilst we want to maximise code sharing, it's important to be pragmatic and to not overfit code just to get a bit of reuse. Above all, our applications must be easy to understand and follow. By our rule of thumb, a module can be here if all the following requirements are met:

It does not depend on any modules which are not supported by web, mobile or node.
It is platform agnostic and never contains code that checks which platform is executing it.
It does not contain any markup anything presentational.
Its functionality does not make assumptions that can hinder UX differences between web and mobile.

With these rules in mind, some examples of of common modules include:

Your application's API calls.
State management (Redux, MobX etc).
Environment config / constants.
Common utility functions.
Application business logic.
Higher order components.

/mobile

This is the root of your React Native project, to keep things tidy we store our app structure under /mobile/app

/styles: contains all your project styles with a single file to store our config / style variables such as colours and font sizes.
/components: contains all your project components, generally these components mostly just care about presentation and are stateless.
/pages: keeping pages separate from the component folder makes them easier to find.
/project: contains any none-presentational but specific config to mobile e.g. our routes, polyfills and mobile constants.

/web
This contains anything to do with the web frontend and will in no way touch anything to do with React Native. For example, the top level of web may look like:

/images, /fonts: and all your other static asset folders
/styles: contains all your project styles, most likely from a preprocessor such as Sass or LESS.
/components: contains all your project components, generally these components mostly just care about presentation and are stateless.
/pages: keeping pages separate from the component folder makes them easier to find.
/project: contains any none-presentational but specific config to web e.g. our routes, polyfills and web constants.

/webpack

Webpack takes our js, styles and other resources and bundles them into static assets, all of our build config is self contained in this folder.

Rather than going too much into detail here, our webpack configs are used in our package.json scripts to either bundle our app for development or deploy minified/cachebusted files to /build to be used in production.

Syncing common code between web and mobile

For this approach to work a key part is being able to sync common code so that they are usable in web and mobile and can be updated from a single location.

Since the React Native packager doesn’t support symlinks we need an alternative. Even if it did support symlinks, we needed a more reliable way keep the mobile common folder up to date without bloating web/mobile with dependencies such as gulp-watch. Making use of wix's wml library we wrote a simple bash script that launches with xCode and ensures only one instance of this is running.

#!/bin/sh

LINE=$(ps aux | grep -w "common-watch.sh" | grep -v grep | wc -l)
echo $LINE

if [ "$LINE" -le "3" ]
then
  npm install -g wml
  watchman watch ../../common
  wml add ../../common ../common-mobile
  wml start
 else
     echo "Process already running"
     exit

fi

We add a script phase to our xcode build

PROJECTDIR="${PROJECT_DIR}"
osascript -e "tell app \"Terminal\" to do script \"cd $PROJECTDIR/../bin && sh ../bin/common-watch.sh && exit\""

Web and mobile Polyfilling

To help maximise the code we reuse in our common layer, there are certain web and mobile features we might want to make available on each platform so that we don't have to worry where we use them.

On web we added the following react native features to our existing pollyfil:

AppState - sometimes we may want to know whether the browser tabs are active in the same way that apps are active.
AsyncStorage - Without worrying about syntax differences we can easily local storage app state and bootstrap it to our application on a shared layer.
Clipboard - having this allows us to manage the clipboard in a common place such as a higher order component.
NetInfo - this allows us to write cross platform functionality that utilises the user's internet connectivity.

With the help of react-native-web we wrapped up these modules in a simple package which can be found on our here.

Dumb and smart components

When looking solely at our app components, using Dan Abramov's pattern of smart and dumb components is generally a good way to separate our dumb /mobile, /web components and smart /common components.

Our dumb components are primarily concerned with how things look and rarely deal with state unless it's to do with UI rather than data. These components shouldn't need to care about dependencies such as flux actions or stores.

Our smart components on the other hand don't usually contain markup, they deal with data and how things work and pass those results down to dumb components as props.

Client APIs

In order to increase what we can do in our common layer, both web and mobile expose a global API that perform similar tasks in their own way. Common code doesn't have to worry about how each platform deals with the requests, examples of this may include:

Recording analytic events when flux actions are called

API.recordEvent(Constants.FOO_CLICKED)

Social auth via a higher order component

API.auth.google().then(this.onLogin)

Even in cases that are not used by common, this is quite a good idea to keep syntax familiar between platforms.

Base modules

As well as sharing modules across platforms we also split some of our modules into parts that are unlikely to change, for example our main mobile stylesheet looks like this:

import baseStyles from './base/baseStyleVariables'
export default Object.assign(
    {}, baseStyles,
    {
        //    Project specific styles and overrides
    }
);

Tying it all together

Our github project shared-react provides a solid example of the above principles in an easy to understand way.

Web entry flow

The entry point main.js which kicks off the following
1 - Polyfills React Native APIs we wish to use, in our case AppState, AsyncStorage, Clipboard and NetInfo.
2 - Our API is assigned to global scope and exposes a single function recordEvent(eventName).
3 - Imports our main style.scss, in development this supports scss hot reloading. In production this is extracted and inserted as a css link in the html head.
4 - Initialises the project routes defined in and renders the results within the main application div.

Mobile entry flow

The entry point index.js which kicks off the following
1 - Polyfills any web APIs we wish to use, in our case we don't need any but in other instances we may use something such as react-native-fetch-blob.
2 - Our API is assigned to global scope and exposes a single function recordEvent(eventName).
3 - Uses react-native-globals to assign components like View to global scope such as View.
4 - Exposes a few base components such H1-H6 and Flex and initialises a global stylesheet.
5 - Registers our ExampleScreen using AppRegistry. We don't use a router in our example like on web as there are quite a few different and popular options in React Native.

The example application

Our web and mobile include 1 page/screen that store and retrieve a single field.

They use an example higher order component which uses AsyncStorage to do the heavy lifting and passes down the following props:


   <WrappedComponent instructions={instructions} //Instructions to the user
                    isLoading={isLoading} //Whether the storage is loading
                    isSaving={isSaving} //Whether the storage is saving
                    value={value} //The current value entered for the storage
                    save={save} //A function to save the current value
                    reset={reset} //A function to reset the stored value
                    onChange={onChange} //A function to call when the text value changes
                    success={success} //Determines whether saving was successful
                    {...this.props} //Pass all other props
                />

With this we create a simple form on web and mobile, our higher order component uses our cross platform api to record events,

I've released over 100 apps in React Native since 2015, Ask Me Anything!

Kyle Johnson — Tue, 21 Aug 2018 09:22:47 +0000