DEV Community: Alex Luong

Comprehensive Guide to Twitter Webhook

Alex Luong — Fri, 25 Oct 2019 20:27:06 +0000

For the past 3 months, I've done a few development explorations with Twitter API. The results were (shameless plugs) my two projects, Tweet Scheduler and thank u, next. My experience has been less than stellar, to say the least. In my experience, the problem is that Twitter Developer documentation left a lot to desire. The biggest issue is Account Activity API, the new webhook concept that is quite challenging to understand and work with. In this post, I hope to explain how it works and build a fun Twitter bot to put these concepts in practice.

Webhook and Subscription

Here is how Twitter webhook works.

When you use Twitter webhook to follow a user, you get notified whenever new events related to that user happen. Here is the list of all events that you can subscribe to.

Simple enough, right?

In many products, there is a nice and pretty UI where you can register the webhook URL. With Twitter you have to do everything programmatically. On top of that, Twitter also adds in the concept of subscription.

Let me elaborate with this diagram about Twitter subscriptions.

In many cases, in order to let a Twitter user to subscribe to your app, you need to handle OAuth to ask for permissions. Here are 2 use cases, one where you need to ask for permission and one where you don't.

Use case 1

You're building a Twitter bot where it subscribes to its own activities. For example, @this_vid, a Twitter bot that will download the video for users on mention, subscribes to the mention event.

In this case, when creating the Twitter app from that bot account, Twitter will give you an access token and secret that work for that account only. It is good enough for this use case, so you don't have to worry about OAuth. You will still need to use those token and secret to add a subscription to your webhook.

Use case 2

You're building a Twitter bot where it subscribes to others activities. For example, in thank u, next, the app subscribes to many different users direct messages events instead of its own Twitter account.

In this cases, you need to ask for permission to gain access to their activities. This involves Twitter authentication using OAuth 1. After a user signs in to your app, you will receive an access token and a secret token. Using those tokens, you can then add that user's subscription to your webhook.

To sum up

Register webhook to Twitter with an API call
For each account whose activities you want to subscribe to, first ask for permission with OAuth to retrieve an access token and a secret token
Then add a new subscription with another API call

Practice

So, I'm gonna show you how Twitter webhook works by building @GimmeDadJoke. Whenever you mention the bot in your tweet, it will reply with a dad joke.

The jokes are going to be fetched from icanhazdadjoke.

I will write everything in JavaScript/NodeJS and deploy it on Zeit Now.

Let's do it.

Create Twitter App

First, create a new Twitter account with the desired handle; in my case: GimmeDadJoke.

Then, fill in a Twitter form to apply for access to Twitter API. It may look scary, but it seems like an automated process, so don't worry.

Once applied and given access, you can create a new Twitter app from this form.

And lastly, navigate to the Keys and tokens tab to get access to your app API keys.

From there, click on the button that will create access token and access token secret.

Let me explain what these tokens do:

Consumer API Keys, including API key and API secret key, are tokens that let Twitter know which app is calling the API. They are needed in every API call your app will perform.
Access token and Access token secret are tokens that identify which user is performing the action. For every user that gives your app permission, you will receive their access token and access token secret. In this case, these two tokens belongs to the app's owner user. In my case, the account @GimmeDadJoke.

Next, go to the Permissions tab and change your access permission to Read, write, and Direct Messages. Even if you don't need to access DM, you still need to use this permission because the Account Activity API automatically sends the DM data to your webhook.

Finally need to create a Twitter development environment to use subscription APIs. Navigate to the environment dashboard and click the "Set up dev environment" button in the Account Activity API / Sandbox section. You'll then be prompted to name your environment and select which app it belongs to. In my case, I name it development and select the app that I just created.

That's it. We're finally done with Twitter set up and ready to write some code.

Set up

Navigate to a folder of your choice and start setting up our code:

mkdir gimmedadjoke && cd gimmedadjoke

Webhook URL

If you haven't used Zeit Now before, you also need to install their CLI and login or create an account:

npm i -g now
now login

Now, you're ready to set up the API serverless function:

mkdir api && cd api
yarn init -y

Inside package.json file, add in the start and build scripts:

// gimmedadjoke/api/package.json

{
  "scripts": {
    "start": "now dev",
    "build": "now --prod"
  }
}

Now, let's create the webhook function:

// gimmedadjoke/api/webhook.js

const crypto = require("crypto");

function createCrcResponseToken(crcToken) {
  const hmac = crypto
    .createHmac("sha256", process.env.TWITTER_CONSUMER_SECRET)
    .update(crcToken)
    .digest("base64");

  return `sha256=${hmac}`;
}

function getHandler(req, res) {
  const crcToken = req.query.crc_token;

  if (crcToken) {
    res.status(200).send({
      response_token: createCrcResponseToken(crcToken)
    });
  } else {
    res.status(400).send({
      message: "Error: crc_token missing from request."
    });
  }
}

function postHandler(req, res) {
  const body = req.body;
  console.log(body);
  res.status(200).json(body);
}

module.exports = (req, res) => {
  try {
    switch (req.method) {
      case "GET":
        return getHandler(req, res);
      case "POST":
        return postHandler(req, res);
      default:
        return res.status(410).json({ message: "Unsupported Request Method" });
    }
  } catch (error) {
    console.log(error.message);
    res.status(500).send();
  }
};

Some brief explanation of the purpose of the code:

This route expects 2 types of request, a GET and a POST.
A GET route is for authentication purpose. When you register the webhook, Twitter will send a test request to make sure that you're the one who controls the webhook URL.
A POST route is for the actual events. Whenever a new event happens, Twitter will send a POST request to this route. Currently, we're not doing anything yet.

To run function in development mode, you can run yarn start. You can try to make a POST request to http//localhost:3000/webhook.js to confirm your function is working.

Deploy to Now

To set up deployment, create a now.json file:

// gimmedadjoke/api/now.json

{
  "name": "gimmedadjoke",
  "version": 2,
  "builds": [{ "src": "webhook.js", "use": "@now/node" }],
  "env": {
    "TWITTER_CONSUMER_KEY": "@gimmedadjoke-consumer-key",
    "TWITTER_CONSUMER_SECRET": "@gimmedadjoke-consumer-secret",
    "TWITTER_ACCESS_TOKEN": "@gimmedadjoke-access-token",
    "TWITTER_ACCESS_TOKEN_SECRET": "@gimmedadjoke-access-token-secret"
  }
}

Next, you need to set up the environment variables:

now secrets add gimmedadjoke-consumer-key TWITTER_CONSUMER_KEY
now secrets add gimmedadjoke-consumer-secret TWITTER_CONSUMER_SECRET
now secrets add gimmedadjoke-access-token TWITTER_ACCESS_TOKEN
now secrets add gimmedadjoke-access-token-secret TWITTER_ACCESS_TOKEN_SECRET

Don't forget to change gimmedadjoke to your bot name and use the correct tokens for their values.

After this, you are ready to deploy your function that powers the Twitter bot. Run yarn deploy.

Now, to test if your deployment is successful:

Scripts

Once your Now deployment is ready to go, you can start writing a few Node scripts to set up webhook and subscriptions.

pwd # make sure you're at your bot root directory
mkdir scripts && cd scripts
yarn init -y
yarn add dotenv request request-promise

In your .env file inside scripts directory:

TWITTER_API_URL=https://api.twitter.com/1.1
TWITTER_CONSUMER_KEY=your token from app dashboard
TWITTER_CONSUMER_SECRET=your token from app dashboard
TWITTER_ACCESS_TOKEN=your token from app dashboard
TWITTER_ACCESS_TOKEN_SECRET=your token from app dashboard
TWITTER_BEARER_TOKEN=
TWITTER_WEBHOOK_ENV=development (or whatever you name it when creating your dev environment)
WEBHOOK_URL=https://gimmedadjoke.now.sh/webhook.js (your Now webhook function)

Wait, what is that mysterious Bearer Token?

Bearer Token is yet another way for your app to authenticate with Twitter. Quite confusing, I know. Don't worry, I'll walk you through the code we're about to write.

To make the scripts files simpler, you will write a big api file that takes care of the Twitter API interaction. Here are everything we're going to do with our scripts:

// gimmedadjoke/scripts/src/api.js

require("dotenv").config();
const request = require("request-promise");

const TWITTER_API_URL = process.env.TWITTER_API_URL;
const TWITTER_CONSUMER_KEY = process.env.TWITTER_CONSUMER_KEY;
const TWITTER_CONSUMER_SECRET = process.env.TWITTER_CONSUMER_SECRET;
const TWITTER_ACCESS_TOKEN = process.env.TWITTER_ACCESS_TOKEN;
const TWITTER_ACCESS_TOKEN_SECRET = process.env.TWITTER_ACCESS_TOKEN_SECRET;
const TWITTER_BEARER_TOKEN = process.env.TWITTER_BEARER_TOKEN;
const TWITTER_WEBHOOK_ENV = process.env.TWITTER_WEBHOOK_ENV;

const oauth = {
  consumer_key: TWITTER_CONSUMER_KEY,
  consumer_secret: TWITTER_CONSUMER_SECRET,
  token: TWITTER_ACCESS_TOKEN,
  token_secret: TWITTER_ACCESS_TOKEN_SECRET
};

const authorizationHeaders = {
  authorization: `Bearer ${TWITTER_BEARER_TOKEN}`
};

exports.getBearerToken = function() {
 return [request.post](http://request.post)({
    url: "[https://api.twitter.com/oauth2/token?grant_type=client_credentials](https://api.twitter.com/oauth2/token?grant_type=client_credentials)",
    auth: {
      user: process.env.TWITTER_CONSUMER_KEY,
      pass: process.env.TWITTER_CONSUMER_SECRET
    },
    json: true
  });
};

exports.getWebhook = function() {
  return request.get({
    url: `${TWITTER_API_URL}/account_activity/all/${TWITTER_WEBHOOK_ENV}/webhooks.json`,
    headers: authorizationHeaders,
    json: true
  });
};

exports.createWebhook = function(webhookUrl) {
  return [request.post](http://request.post)({
    url: `${TWITTER_API_URL}/account_activity/all/${TWITTER_WEBHOOK_ENV}/webhooks.json`,
    oauth,
    form: {
      url: webhookUrl
    },
    json: true
  });
};

exports.deleteWebhook = function(webhookId) {
  return request.delete({
    url: `${TWITTER_API_URL}/account_activity/all/${TWITTER_WEBHOOK_ENV}/webhooks/${webhookId}.json`,
    oauth
  });
};

exports.getSubscription = function() {
  return request.get({
    url: `${TWITTER_API_URL}/account_activity/all/${this.webhookEnv}/subscriptions.json`,
    oauth,
    json: true
  });
};

exports.createSubscription = function() {
  return request.post({
    url: `${TWITTER_API_URL}/account_activity/all/${TWITTER_WEBHOOK_ENV}/subscriptions.json`,
    oauth,
    json: true
  });
};

exports.deleteSubscription = function(userId) {
  return request.delete({
    url: `${TWITTER_API_URL}/account_activity/all/${this.webhookEnv}/subscriptions/${userId}.json`,
    headers: authorizationHeaders,
    json: true
  });
};

Here are all the functions we've written:

getBearerToken
getWebhook
createWebhook
deleteWebhook
getSubscription
createSubscription
deleteSubscription

We'll create 7 scripts that directly correlate with these functions later.

In the mean time:

Take a break

Hey, well done for following along. I know that has been a lot of code, but that is the most of the code for this section.

When you come back, I'll explain what these functions are really doing, and hopefully you can gain a better and more practical understanding of how Twitter webhook actually works.

Now, go get a snack or grab a cup of tea. You totally deserve it.

Bearer token

First of all, let's write a script to retrieve your app bearer token:

// gimmedadjoke/scripts/src/bearerToken.js

const api = require("./api");

function run() {
  api
    .getBearerToken()
    .then(response => {
      console.log(response);
    })
    .catch(error => {
      console.log(error.message);
    });
}

run();

Then, in the terminal:

pwd # make sure you're inside the scripts directory
node src/bearerToken.js

If things go well, you should see something like this:

{
  token_type: 'bearer',
  access_token: 'some_token'
}

Copy that token and put it into your .env file. You're ready to write some webhook goodness.

Webhook

Let's warm up by writing a script that retrieve all current webhooks associate with our app.

// gimmedadjoke/scripts/src/webhook.get.js

const api = require("./api");

function run() {
  api
    .getWebhook()
    .then(response => {
      console.log(response);
    })
    .catch(error => {
      console.log(error.message);
    });
}

run();

Then, run node src/webhook.get.js. If the response is [], you're on the right track.

To add webhook to your app:

// gimmedadjoke/scripts/src/webhook.post.js

const api = require("./api");

const WEBHOOK_URL = process.env.WEBHOOK_URL;

function run() {
  api
    .createWebhook(WEBHOOK_URL)
    .then(response => {
      console.log(response);
    })
    .catch(error => {
      console.log(error.message);
    });
}

run();

When you run it, you may notice it takes slightly longer than other commands you've run. That is because a lot is going on here:

You asked Twitter to register your webhook URL.
Twitter sent a test request to your function on Now.
Once it's successfully tested, Twitter answered back to you with your new webhook information.

And lastly, let's create a script that remove the webhook:

// gimmedadjoke/scripts/src/webhook.delete.js

const api = require("./api");

function run() {
  api
    .getWebhook()
    .then(response => {
      const webhookId = response[0].id;
      api
        .deleteWebhook(webhookId)
        .then(response => {
          console.log("Successfully delete webhook");
        })
        .catch(error => {
          console.log(error.message);
        });
    })
    .catch(error => {
      console.log(error.message);
    });
}

run();

Now, with these 3 scripts, you can register your webhook, get its information, and remove it when you want to.

Although you can register your webhook, it is not doing anything just yet. You also need to use subscription to make your webhook functional.

Subscription

Similar to the scripts you wrote for webhooks, you will now write another 3 scripts for subscriptions. I'm going to show you the code, and we can talk about it afterwards:

// gimmedadjoke/scripts/src/subscription.get.js

const api = require("./api");

function run() {
  api
    .createSubscription()
    .then(response => {
      console.log("Successfully subscribe the app owner user to webhook.");
    })
    .catch(error => {
      console.log(error.message);
    });
}

run();

// gimmedadjoke/scripts/src/subscription.post.js

const api = require("./api");

function run() {
  api
    .createSubscription()
    .then(response => {
      console.log("Successfully subscribe the app owner user to webhook.");
    })
    .catch(error => {
      console.log(error.message);
    });
}

run();

// gimmedadjoke/scripts/src/subscription.delete.js

const api = require("./api");

function run() {
  const userId = process.env.TWITTER_ACCESS_TOKEN.split("-")[0];

  api
    .deleteSubscription(userId)
    .then(response => {
      console.log("Successfully remove subscription.");
    })
    .catch(error => {
      console.log(error.message);
    });
}

run();

Try subscribing the app owner Twitter user (the user associated with the developer account -- i.e in this case @gimmedadjoke) to your webhook.

node src/subscription.post.js

If your script runs without error, congratulations, your webhook is now in action.

In my case, whenever the Twitter user @GimmeDadJoke receives any new event, I will know about it.

You can confirm by sending a direct message to your Twitter bot and check the function's log.

🎉🎉

Respond with a dad joke

Because this is not the focus of this guide, you can check out the actual handling of the webhook request in the source code.

In general, this is how you can handle it:

function postHandler(req, res) {
  const body = req.body;

  if (body[THE_EVENT_YOU_CARE_ABOUT]) {
    // do stuff

    return res.status(200).send();
  } else {
    return res.status(200).send();
  }
}

The end result:

🎉🎉

Extend Twitter bot to subscribe to other users' events

As of right now, the bot is listening to its own Twitter account event. You can extend it to subscribe to other users as well. This use case is beyond the scope of this guide, but I'll give you a general approach to implement it.

Maybe it can be a fun challenge for you.

Here is the general approach:

Implement "sign in with Twitter" functionality using OAuth. One way you can do that is by using Firebase.
Once signed in, your app will receive the access token and access token secret.
Using those tokens to add a new subscription to your bot. This is exactly the same as when you add the subscription to your own bot user:

request.post({
  url: `${TWITTER_API_URL}/account_activity/all/${TWITTER_WEBHOOK_ENV}/subscriptions.json`,
  oauth: {
    consumer_key: process.env.TWITTER_CONSUMER_KEY,
    consumer_secret: process.env.TWITTER_CONSUMER_SECRET,
    token: ----->YOUR_USER_ACCESS_TOKEN_THAT_YOU_JUST_ACQUIRED,
    token_secret: ----->YOUR_USER_ACCESS_TOKEN_SECRET_THAT_YOU_JUST_ACQUIRED,
  },
  json: true
});

Once your user is subscribed, your webhook function will receive a POST request every time a new event occurs.

Limitation

When using the free tier of Twitter API, there are many limitations. Here are a few that directly relates to webhooks:

You can only have 1 development environment for subscription.
Within that environment, you can only have 15 subscriptions. That means if you app wants to subscribes to many users (use case 2), you can only have 15 users on the free tier.

CLI

I'm actively working on an open source command line tool to smoothen this process of registering webhook and managing subscriptions. Stay tuned for updates, and I'd love your collaboration if you want to get involved.

Resources

Twitter API Documentation
Twitter Sample Webhook on GitHub: The Twitter dev team created a sample dashboard for you to handle this process. This is super helpful, and I'd highly recommend you check this out. This project was how I got started with Twitter webhook.

Thank you for checking out my guide on Twitter webhook.

Don't hesitate to ping me if you run into any problem, and please let me know when you build something from this. I look forward to seeing all the cool stuff you build on top of Twitter.

Tweet Scheduler - A Technical Retro

Alex Luong — Wed, 02 Oct 2019 22:14:23 +0000

I recently "launched" Tweet Scheduler, my first finished side project. I got a chance to play with new technologies and enjoyed working on it. Here is a reflection on what I learned and my decision making process.

Requirements

Here is how the app works:

User can write the tweets and choose when they will be posted
At said time, the tweets will be posted

Simple enough.

On a technical perspective, here are what's needed:

The UI for users to schedule the tweets
The database to save these data
A CRON job that runs every minute, checking if there are any tweet to be posted
A server to post tweets

Minimal Viable Product

In the MVP, I chose technologies optimized for speed. For that, Firebase was my top choice because at its core, it's Database as a Service along with other features, including easy Twitter authentication. That, my friend, is a front end developer dream, especially for rapid prototype.

To put everything together with Firebase, here is the application architecture:

A brief explanation of the architecture:

Users use the front end to write scheduled tweets.
These tweets are saved to Firebase.
Every minute, EasyCron, a CRON service, sends a request to a Netlify function.
That function checks if any there are any tweets to be posted and post them to Twitter.

Using this architecture, I was able to build the first version of Tweet Scheduler in a day or two. The MVP was functional but has a big issue: Security.

Because I'm posting the tweet for users, I have to save their access token and secret in the database. In my Netlify function, I will then fetch those tokens and send the request. However, there is currently some issues using Netlify function and firebase-admin, so the authentication is more complicated that I desire. Coupled that with some minor Firebase issue on the front end and my wish to try out other technologies, I decide to rip out the Firebase layer and try out something else.

Replace Firebase

Without Firebase, I needed to come up with solutions for database and authentication. I decided to try out some technologies that I've known for a while but hadn't had a chance to use yet: FaunaDB and Auth0.

Of the two, I'm particularly excited about FaunaDB. Besides Dynamo, Aurora, and Cosmos provided by AWS and Azure, Fauna seems to be the only other serverless database option. It's supposed to work great with Netlify functions because of the serverless architecture.

Here is how the updated architecture would look like:

This change in architecture put a lot of weight and complexity on the Netlify functions layer. Before, it was just 1 function that runs every minute. With this change, it would become the server layer that takes request from the front end and talks to the database.

Excited as I were, I did not find much success after a few hours trying out these technologies. Here comes the recap of my experiences, but please note that this is just my opinion based on my own personal bias and preference only. You may have different opinions, and I highly recommend that you try them out yourself.

The main problem I ran into with Fauna is Fauna Query Language (FQL). I have some experiences with SQL and MongoDB, and FQL is certainly a new concept to me. After a few hours, I was still not productive and ran into some issues with their querying syntax. As Fauna is not as popular as other databases, I could not find many good educational resources on it.

With that said, a few days after I finished Tweet Scheduler, Chris Biscardi dropped a collection of FaunaDB lessons on Egghead. Had this existed 2 weeks earlier, I would probably have a much better experience.

As for Auth0, there is no specific flaws or shortcomings on their side. I'm using Gatsby to build the app, and there were some problems that came with server side rendering. I was able to resolve it, but the end result was not as streamlined as I'd like.

VPS - Final Solution

In the end, I decided to roll out my own back end on a VPS. In doing so, I hoped to gained more experience using server and Linux. With that, I would also run my own CRON job from my server without relying on an external service.

Here is how it looks:

I know, not as exciting in todays' world where serverless is taking over everything. Maybe as I gain more experiences with traditional servers, I can finally appreciate what serverless can do, as I'm previously just kinda impressed with them but was never a fan.

Until then, I'm quite happy with this set up.

Misc

Besides the overarching architecture of the app, I also made other smaller decisions over which technologies or libraries to use.

For the UI library, I used Chakra UI and enjoyed it a lot. I really like its aesthetic. And as it uses Styled System, this was also my first exposure to the library, and it didn't disappoint. As of right now, this is by far my favorite way of writing "CSS".

For the front end GraphQL client, I decided to try out Urql as I already have some experiences with React Apollo. Because Urql relies heavily on its cache, I gained a better understanding and appreciation to how different GraphQL client libraries handle cache. I'd definitely recommend anybody to try it out as I have a great time using it as well.

In the end, I'm happy with my decisions and the tech stack I chose for Tweet Scheduler. If there's one thing I would do differently, I would also 1. try out Hasura and 2. maybe stick with FaunaDB a bit longer. But overall, the final architecture is going to be my tech stack of choice for my future projects.

A Journey Through Gatsby Build Process via Building a Plugin

Alex Luong — Tue, 30 Jul 2019 12:22:32 +0000

Let me take you through my journey of building a Gatsby plugin. Hopefully, from my experience, you can learn a thing or two about Gatsby and maybe even React Hooks.

The mission

This post attempts to explain what happens when you run gatsby develop and gatsby build in regards to the building and serving HTML step.

This post assumes you have some experiences working with Gatsby and know some Gatsby specific API. Please feel free to ask me to explain further if I lose you somewhere.

The plugin

The plugin that I'm building is gatsby-plugin-firebase. I want to use Firebase to build a web application with Gatsby, but there are some challenges setting things up. Mainly, the Firebase web SDK is client-only, which doesn't sit well with Gatsby server-side rendering process.

I searched for a solution to integrate Firebase with Gatsby, but there doesn't seem to be many. In my search, I came across 2 resources that are very helpful, so you can check them out for better context:

Kyle Shevlin's blog post: Firebase and Gatsby, Together At Last
Muhammad Muhajir's gatsby-starter-firebase

The plugin that I'm gonna build should allow you to register it in gatsby-config.js and have Firebase initialized and ready to go for you.

Attempt #1

The code

Taking inspiration from these 2 resources, I built gatsby-plugin-firebase. I will speed through my code as it is not the main focus of this post. Here is what I did:

Using gatsby-browser.js and gatsby-ssr.js, I wrapped Gatsby root in a React component:

import React from "react"
import Layout from "./src"

export const wrapRootElement = ({ element, props }) => (
  <Layout {...props}>{element}</Layout>
)

In the Layout component at src/index.js, I initialized Firebase and put a firebase instance in a React Context:

import React from "react"
import FirebaseContext from "./components/FirebaseContext"

function Index({ children }) {
  const [firebase, setFirebase] = React.useState(null)

  React.useEffect(() => {
    if (!firebase && typeof window !== "undefined") {
      const app = import("firebase/app")
      const auth = import("firebase/auth")
      const database = import("firebase/database")
      const firestore = import("firebase/firestore")

      Promise.all([app, auth, database, firestore]).then(values => {
        const firebaseInstance = values[0]
        firebaseInstance.initializeApp({
          apiKey: process.env.GATSBY_FIREBASE_API_KEY,
          authDomain: process.env.GATSBY_FIREBASE_AUTH_DOMAIN,
          databaseURL: process.env.GATSBY_FIREBASE_DATABASE_URL,
          projectId: process.env.GATSBY_FIREBASE_PROJECT_ID,
          storageBucket: process.env.GATSBY_FIREBASE_STORAGE_BUCKET,
          messagingSenderId: process.env.GATSBY_FIREBASE_MESSAGING_SENDER_ID,
          appId: process.env.GATSBY_FIREBASE_APP_ID,
        })
        setFirebase(firebaseInstance)
      })
    }
  }, [])

  if (!firebase) {
    return null
  }

  return (
    <FirebaseContext.Provider value={firebase}>
      {children}
    </FirebaseContext.Provider>
  )
}

export default Index

Created FirebaseContext with some helpers to easily access firebase inside src/index.js:

import React from "react"

const FirebaseContext = React.createContext(null)

export function useFirebase() {
  const firebase = React.useContext(FirebaseContext)
  return firebase
}

export const withFirebase = Component => props => (
  <FirebaseContext.Consumer>
    {firebase => <Component {...props} firebase={firebase} />
  </FirebaseContext.Consumer>
)

export default FirebaseContext

And inside the root index.js I exported some helpers:

exports.FirebaseContext = require("./src/components/FirebaseContext").default
exports.useFirebase = require("./src/components/FirebaseContext").useFirebase
exports.withFirebase = require("./src/components/FirebaseContext").withFirebase

Did it work?

It did 🎉🎉. When I wrote some code to consumed the library and ran gatsby develop, it worked beautifully. Here is a sample component showing how I used it:

import React from "react"
import { useFirebase } from "gatsby-plugin-firebase"

export default () => {
  const firebase = useFirebase()
  const [name, setName] = React.useState("there")

  React.useEffect(() => {
    firebase
      .database()
      .ref("/name")
      .once("value")
      .then(snapshot => setName(snapshot.val()))
  }, [firebase])

  return <div>Hi {name}</div>
}

Problems arose when I tried to run gatsby build && gatsby serve. The site still built successfully and worked, but something weird happened.

When visiting a page that doesn't use Firebase, it would render the content, then a flash of white screen, and then render the content again.

When visiting a page that does use Firebase, it would render the default value, flash, default value, and then the value from Firebase.

What is going on?

What happened was that in development phase, Gatsby uses Webpack Dev Server, so everything runs completely on the client. Gatsby is basically a React app at that point (disregarding the GraphQL part). Therefore, everything worked perfectly.

When running gatsby build, it generates HTML files for all of your pages in a Node process. In React components, it didn't run the lifecycles like componentDidMount or useEffect hook. In the end, pages that didn't depend on Firebase were the same. And because Firebase was run inside useEffect, the page that I wrote just used the default name state and rendered "Hi there".

When serving the site, after rendering the HTML, Gatsby will rehydrate the site to a React app. At that point, it would initialize Firebase and do all sort of stuff that it didn't do during the build step.

In my src/index.js file when I set up FirebaseContext, I had these lines:

if (!firebase) {
  return null
}

This is the reason that the white flash appeared. The source of all evil. If you replace return null with return <div style={{ width: "100%", height: "100%", background: "red" }} />, you would have a very red flash instead.

Attemp #2

Well if those 3 lines are the causes of the white flash, maybe we can just remove them, right? Right?

That's what I did. And boy was I wrong.

On first render, firebase = null. Remember in my src/index.js file, I wrap the Firebase initialization code inside a useEffect. Firebase will only exist after the first render. When removing those 3 lines, I receive firebase is undefined error right from the development step.

Solution

To solve the error, I can simply check whether firebase exists before doing anything with it. It works. But I don't like it. I don't want to add an extra cognitive load to the users' brain every time they try to do stuff with Firebase.

Besides, to check whether firebase exists is quite simple in React Hooks:

React.useEffect(() => {
  if (!firebase) {
    return
  }
  doSomething(firebase)
}, [firebase])

Whereas in a class component, it would be a bit more involved:

class Component extends React.Component {
  componentDidUpdate(prevProps) {
    if (!prevProps.firebase && this.props.firebase) {
      doSomething(this.props.firebase)
    }
  }
}

export default withFirebase(Component)

Well, it's not that bad. But it could be better.

Attempt #3

In search for a better API, I just randomly thought of how useEffect works. Since you have to use Firebase in that hook anyway, and it takes a function as its first argument, what if my useFirebase works like that too? In that case, the function in the argument can receive firebase only when it's already initialized so that the end-users would never have to care about it.

The end-users would know that firebase is always there, ready for them.

Here is my rewrite of the helper hook:

function useFirebase(fn, dependencies = []) {
  const firebase = React.useContext(FirebaseContext)
  React.useEffect(() => {
    if (!firebase) {
      return
    }
    return fn(firebase)
  }, [firebase, ...dependencies])
}

With this hook, the users can simply write their component like so:

function Component() {
  const [name, setName] = React.useState("there")

  useFirebase(firebase => {
    firebase
      .database()
      .ref("/name")
      .once("value")
      .then(snapshot => setName(snapshot.val()))
  })

  return <div>Hi {name}</div>
}

Beautiful, if I do say so myself.

What about Classes, bro?

Now that I'm happy with this API, I try to come up with a way to support the same easy-to-use API but for class component as they cannot use hooks.

And quite frankly, I just cannot come up with an API as intuitive as hook. The problem is that class component is too coupling with the render method that it's impossible to defer that aspect to the user the way hooks allow.

Conclusion

Well that's it folks. Some quick recaps:

gatsby develop runs a React app
gatsby build builds HTML pages
When served, after rendering the HTML, Gatsby will rehydrate the site to React. Lifecycles method will run, which may or may not affect how your site looks, potentially causing flickering/flashes.
React Hooks are awesome

And if you use Firebase with Gatsby, consider using my plugin gatsby-plugin-firebase maybe?