DEV Community: Ran Ding

From Transformers to ChatGPT

Ran Ding — Sun, 26 Mar 2023 00:23:50 +0000

You can read the full note here (with better formatting) https://www.dingran.me/from-transformer-to-llm/

Introduction

Large language models such as GPT-3 have shown impressive performance not only in NLP benchmarking tasks but also blew the minds of the public through application interfaces such as ChatGPT.

This note provides a high-level summary of the progress in large language models (LLM) from 2017 (the inception of the Transformer model) to now (the end of 2022), serving as a fast-paced recap for readers to catch up on this field quickly. General familiarity with machine learning/deep learning is assumed.

This note will only cover a small, core set of papers: Transformer, BERT, GPT, GPT-2, GPT-3, and InstructGPT. There are undoubtedly many other notable papers published during the same period of time - I'll leave them to a literature survey/reading list.

This is a somewhat long note - it is broken down into the following sections:

Overview
1. What is NLP
2. Past progress (pre-2017)
3. Recent progress (2017 - 2022)
Model details
1. Transformer
2. GPT
3. BERT
4. GPT-2, GPT-3
5. From GPT-3 to ChatGPT
Conclusions & reflections

1. Overview

1.1 What is NLP?

Natural Language Processing (NLP) involves a wide range of tasks that focus on the processing and understanding of human language. Some of the main tasks in NLP include Text Classification, Named Entity Recognition (NER), Sentiment Analysis, Machine Translation, Information Retrieval, Question Answering, and Text Summarization. A more complete list of typical NLP tasks and progress in each is available here.

Here are a few excellent references:

Linguistic basics: Emily Bender’s Linguistic Fundamentals for Natural Language Processing: 100 Essentials from Morphology and Syntax (2013)
Yoav Goldberg’s book Neural Network Methods for Natural Language Processing (2017)
Chris Manning’s lectures (CS224d) on Natural Language Processing with Deep Learning (Youtube)

1.2 Past progress (pre-2017)

Historically Natural Language Processing (NLP) was mostly based on rule-based approaches or statistical models. Deep learning took over NLP in the mid-2010s, with Recurrent Neural Networks (RNNs) and Long Short-Term Memory networks (LSTMs) being the de-facto models (there are some minor attempts in other types of models but nothing super successful yet).

The field has made a big jump due to the introduction of these models, but at the same time felt a bit stagnant/limited especially compared to Computer Vision (CV). Let's maybe make a comparison between the status of CV and NLP in the table below.

	NLP	CV
Model Scaling	Poor	Good
Dataset Size	Small	Large
Model Transferability	Poor	Good

RNN/LSTM are autoregressive models, and because of that, the training is fundamentally sequential and harder to parallelize (e.g., a piece of text reading from left to right). Model scaling in NLP has been quite behind CV - where model architecture such as Convolutional Neural Networks (CNN) offers much easier parallelization.

Another difficulty scaling in NLP is the lack of large, labeled data set. In CV, we have datasets like ImageNet, with ~1M images labeled with 1000 categories. Although in machine translation, we could potentially construct datasets with ~1M pairs of sentences, the information (and model supervision) we could generate from sentence pairs is probably 1 to 2 orders of magnitude lower than an image.

On model transferability, in NLP, we haven't seen the kind of success we saw in CV, where a pre-trained model (on a supervised learning dataset) achieves strong performance on downstream tasks. This is partly due to the diversity of NLP tasks, but equally important is the lack of large labeled datasets to train a good and large enough model that transfers and generalizes well.

One side effect of that we see a big gap in generation capabilities in NLP vs CV. Photorealistic images and video generation (e.g. DeepFake) have been around for several years (based on VAE, GAN and etc), while text generation capability has been extremely primitive- which is also why the recent capability in ChatGPT is seemingly unbelievable.

1.3 Recent progress (2017 - 2022)

The progress from 2017 to 2022 changed all of the above constraints in NLP and made it leapfrog CV. This period of time is indeed the breakthrough period for NLP. We'll talk through the details in the sections below. Here is a preview of the significant changes.

Transformer

Although initially developed specifically for machine translation, Transfomer quickly became NLP's new standard model architecture, largely replacing RNNs and LSTMs. The model architecture allows modeling sequences without having to be autoregressive. This massively improved our ability to scale up the model. More recently, Transformer has gone beyond NLP and has been used to model images and videos and in multi-modal applications, and it is considered to be one of the most important core model architectures in machine learning generally.

BERT, GPT

Thanks to the foundation laid out by the Transformer model, BERT and the GPT-series models could scale up the model size from 100M parameters (GPT and BERT-base) to 175B parameters (GPT-3) in a short span of 2 years. Researchers also found creative ways to leverage large unlabelled datasets to support model size scaling.

A key innovation from GPT and BERT is the clever structuring of pre-trained language model input/output to allow the pre-trained models to be transferred (fine-tuned) for a wide array of NLP tasks. Thus establishing a familiar pre-training + fine-tuning setup we saw in CV, with compelling model performance and transferability.

GPT-2, GPT-3

In GPT-2 and GPT-3, the authors introduced the new paradigm of not further adjusting the model (i.e., fine-tuning), but instead of they use natural language "prompts" as a way to tell the model to perform new tasks. The prompts can potentially include some examples (aka, demonstrations). This is called a zero-shot or few-shot setting (depending on how many examples are given to the model).

This allows much-improved transferability since we no longer need to collect a labeled dataset for each specific downstream task to do fine-tuning. This is a significant new step towards making a pre-trained model capable of performing previously unseen tasks completely based on the context/prompt user provides and engaging in open-ended tasks such as doing classifications on an unknown set of labels, engaging in dialogs, writing code, etc. - activity starts to resemble reasoning and intelligence.

From GPT-3 to ChatGPT

Fundamentally, GPT-3 is just a language model. When given a piece of text (i.e., prompt), it generates, somewhat randomly, plausible subsequent text that fits the best. This objective is misaligned with “following the user’s instructions helpfully and safely.” InstructGPT focuses on effectively aligning large language models such as GPT-3 with user intent through fine-tuning with human feedback so that the output can be more helpful, truthful, and harmless.

2. Models

2.1 Transformer

...

Read the full note here https://www.dingran.me/from-transformer-to-llm/

Next.js: Firebase Authentication and Middleware for API Routes

Ran Ding — Sun, 28 Feb 2021 03:48:33 +0000

This article was originally published on my blog. Head over there if you like this post and want to read others like it.

Recently I made a small web app that requires user accounts. I learned quite a bit about setting up authentication with Firebase on the client-side and using it on the server-side to protected API routes with a middleware pattern similar to Express.js. This post is a recap of what I learned based on this project for future reference. You can find the code for this project on GitHub here.

Authentication - Client Side

Initialization

Setting up Firebase is easy. You create a project here and enable the sign-in providers you plan to use, along with authorized domains. Grab the credentials from Project Settings in the Firebase console, and we can initialize the Firebase SDK on the client-side like this.

//lib/firebase.js
import firebase from 'firebase/app';
import 'firebase/auth';
import 'firebase/firestore';

const clientCredentials = {
  apiKey: process.env.NEXT_PUBLIC_FIREBASE_API_KEY,
  authDomain: process.env.NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN,
  databaseURL: process.env.NEXT_PUBLIC_FIREBASE_DATABASE_URL,
  projectId: process.env.NEXT_PUBLIC_FIREBASE_PROJECT_ID,
  appId: process.env.NEXT_PUBLIC_FIREBASE_APP_ID,
};

if (!firebase.apps.length) {
  firebase.initializeApp(clientCredentials);
}

export default firebase;

(See file and folder structure here in the actual project)

React Hooks and Context Provider

Since the user's authentication status is a "global" state, we can avoid recursively passing it as a prop through many layers of components by using Context.

To do this, we need a context Provider and a context Consumer. A Provider comes with a Context created by createContext(). The value prop we pass to the Provider will be accessible by its children.

    //lib/auth.js
    const authContext = createContext();

    export function AuthProvider({ children }) {
      const auth = /* something we'll fill in later */;
      return <authContext.Provider value={auth}>{children}</authContext.Provider>;
    }

For the descendant components to use the value, i.e., consume the Context, we can use `Context.Consumer`, or more conveniently, the `useContext` [hook](https://reactjs.org/docs/hooks-reference.html#usecontext).

    //lib/auth.js
    export const useAuth = () => {
      return useContext(authContext);
    };

    //components/SomeComponent.js
    const SomeComponent = () => {
      const { user, loading } = useAuth();
      // later we can use the object user to determine authentication status
      // ...
      }

In Next.js, the AuthProvider we implemented above can be inserted in the _app.js so all the pages in the app can use it. See here.

Implementation Details of `AuthProvider`

In the AuthProvider skeleton above, we passed an auth object as the value prop, and this is the key thing that all the consumers consume. Now we need to figure out what we need to implement this auth object.

The key thing auth need to achieve is subscribing to the changes in the user's login status (and associated user info). These changes can be triggered through the Firebase SDK, specifically the sign-in / sign-out functions such as firebase.auth.GoogleAuthProvider() and authentication state observer function firebase.auth().onAuthStateChanged().

So, our minimal implementation could be the following, mainly pay attention to the new getAuth function. We definitely need to return something from getAuth and that'll be the auth object used by AuthProvider. To do this, we implement the handleUser function to update the state user as follows

    //lib/auth.js
    import React, { useState, useEffect, useContext, createContext } from 'react'
    import firebase from './firebase'

    const authContext = createContext()

    export function AuthProvider({ children }) {
      const auth = getAuth()
      return <authContext.Provider value={auth}>{children}</authContext.Provider>
    }

    export const useAuth = () => {
      return useContext(authContext)
    }

    function getAuth() {
      const [user, setUser] = useState(null)
      const handleUser = (user) => {
        if(user){
          setUser(user)
        }
      }

      useEffect(() => {
        const unsubscribe = firebase.auth().onAuthStateChanged(handleUser);
        return () => unsubscribe();
      }, []);

      /* TBA: some log in and log out function that will also call handleUser */

      return {user}
    }

Since we are calling other React Hooks, e.g. userEffect, getAuth needs to be either a React functional component or a custom hook in order to follow the rules here. Since we are not rendering anything, just returning some info, getAuth is a custom hook and we should thus rename it to something like useFirebaseAuth (i.e the custom hook's name should always start with use, per note here). The main function userFirebaseAuth provides us is to share the user state between components. Actually, across all the components since we used a Context Provider in _app.js.

Below is a fuller implementation of userFirebaseAuth. There are quite a few things we added here:

Exposing sign-in and sign-out logic so that context consumers can use them. Since they would trigger changes in user state similarly to firebase.auth().onAuthStateChanged, it is better to put them here.
We actually need to change firebase.auth().onAuthStateChanged to firebase.auth().onIdTokenChanged to capture the token refresh events and refresh the user state accordingly with the new access token.
Adding some formatting to make the user object only contains our app's necessary info and not everything that Firebase returns.
Add redirect to send user to the right pages after sign-in or sign-out.

    import React, { useState, useEffect, useContext, createContext } from 'react';
    import Router from 'next/router';
    import firebase from './firebase';
    import { createUser } from './db';

    const authContext = createContext();

    export function AuthProvider({ children }) {
      const auth = useFirebaseAuth();
      return <authContext.Provider value={auth}>{children}</authContext.Provider>;
    }

    export const useAuth = () => {
      return useContext(authContext);
    };

    function useFirebaseAuth() {
      const [user, setUser] = useState(null);
      const [loading, setLoading] = useState(true);

      const handleUser = async (rawUser) => {
        if (rawUser) {
          const user = await formatUser(rawUser);
          const { token, ...userWithoutToken } = user;

          createUser(user.uid, userWithoutToken);
          setUser(user);

          setLoading(false);
          return user;
        } else {
          setUser(false);
          setLoading(false);
          return false;
        }
      };

      const signinWithGoogle = (redirect) => {
        setLoading(true);
        return firebase
          .auth()
          .signInWithPopup(new firebase.auth.GoogleAuthProvider())
          .then((response) => {
            handleUser(response.user);

            if (redirect) {
              Router.push(redirect);
            }
          });
      };

      const signout = () => {
        return firebase
          .auth()
          .signOut()
          .then(() => handleUser(false));
      };

      useEffect(() => {
        const unsubscribe = firebase.auth().onIdTokenChanged(handleUser);
        return () => unsubscribe();
      }, []);

      return {
        user,
        loading,
        signinWithGoogle,
        signout,
      };
    }

    const formatUser = async (user) => {
      return {
        uid: user.uid,
        email: user.email,
        name: user.displayName,
        provider: user.providerData[0].providerId,
        photoUrl: user.photoURL,
      };
    };

Authorization - Server Side

The other use case we need with Firebase authentication is to ensure users have proper access to server-side resources, i.e., specific API routes will be only accessible if certain access criteria is met. I guess this called authorization. An example would be, for /api/users/[uid] route, we would only return results the user requesting their own info.

Firestore Security Rules

One pattern to manage access to backend resources (mostly database access) is to use Firestore and Firebase authentication together and use Firestore's security rules to enforce access permissions.

For example, in the example above, to limit access to user info, on the client-side, we attempt to retrieve the user record as usual

    export async function getUser(uid) {
      const doc = await firestore.collection('users').doc(uid).get();
      const user = { id: doc.id, ...doc.data() };
      return user;
    }

But we define the following set of security rules to only allow read/write when the user's uid matches the document's uid.

    rules_version = '2';
    service cloud.firestore {
      match /databases/{database}/documents {
        match /users/{uid} {
          allow read, write: if isUser(uid);
        }
      }
    }
    function isUser(uid) {
      return isSignedIn() && request.auth.uid == uid;
    }
    function isSignedIn() {
      return request.auth.uid != null;
    }

You can actually do a lot with this setup. For example, in order to determine access to a document, you can do some extra queries on other collections and documents. Here are the security rules I used, which involved a bit of that.

With this client-side setup and security rules, there are downsides. Mainly:

We are defining access using this security rule syntax, which is less flexible than just writing arbitrary code on the server-side.
Firestore also limits the number of queries you can do to verify the access permission on each request. This may limit how complex your permission scheme can be.
Some of the database operations can be very heavy, such as recursively deleting a large document collection, and should only be done on the server-side. (See Firestore's documentation here for more details.)
Testing security rules require extra work. (Firebase does have a friendly UI and simulator for this).
Finally, it gets a little scattered that some database access logic lives on the client-side (code pointer) and some on the server-side (code pointer). I probably should consolidate to the server-side.

Using Firebase Admin on Server Side

OK, now the more "classic" way of doing the authorization on the server-side. The general workflow is:

The client-side code should send over an access token along with each request.
The server-side code has an instance of firebase-admin, which can verify and decode the access token and extract user information, such as the uid of the user
Based on that information, the server-side code can do more queries and apply more logic to figure out it should proceed or reject the request. (The firebase-admin will have privileged access to all Firebase resources and will ignore all the security rules, which are only relevant for client-side requests).

This is how I initialized firebase-admin

    //lib/firebase-admin.js

    import * as admin from 'firebase-admin';

    if (!admin.apps.length) {
      admin.initializeApp({
        credential: admin.credential.cert({
          projectId: process.env.NEXT_PUBLIC_FIREBASE_PROJECT_ID,
          clientEmail: process.env.FIREBASE_CLIENT_EMAIL,
          privateKey: process.env.FIREBASE_PRIVATE_KEY.replace(/\\n/g, '\n'),
        }),
        databaseURL: process.env.NEXT_PUBLIC_FIREBASE_DATABASE_URL,
      });
    }

    const firestore = admin.firestore();
    const auth = admin.auth();

    export { firestore, auth }

The documentation here suggests generate a private key JSON file. The file contains many different fields, the three fields above: projectId, clientEmail, and privateKey seem to be enough to get it to work.

Now we can extract uid on each request and verify the user's access

    import { auth } from '@/lib/firebase-admin';

    export default async (req, res) => {
      if (!req.headers.token) {
        return res.status(401).json({ error: 'Please include id token' });
      }

      try {
        const { uid } = await auth.verifyIdToken(req.headers.token);
        req.uid = uid;
      } catch (error) {
        return res.status(401).json({ error: error.message });
      }

      // more authorization checks based on uid 
      // business logic
    }

Authentication Middleware for Next.js API Routes

One small annoyance with the above is that as we have more API routes that need authentication, the code need to be repeated in these API routes functions. I find Next.js out of the box doesn't have as strong a support for server-side development. A couple of things from Express.js I wish Next.js have are: routers and middleware.

In this scenario, making authentication work as a middleware would be convenient. Middleware is things you can plug into the request handling lifecycle; the middleware would enrich the request and/or the response objects and can terminate the request early if errors occur.

It turned out to be pretty straightforward, we just need to create a wrapper for our normal handler function, and in the wrapper we can modify the req and res objects and return early if errors occur.

Here is how I defined a withAuth middleware

    import { auth } from '@/lib/firebase-admin';

    export function withAuth(handler) {
      return async (req, res) => {
        const authHeader = req.headers.authorization;
        if (!authHeader) {
          return res.status(401).end('Not authenticated. No Auth header');
        }

        const token = authHeader.split(' ')[1];
        let decodedToken;
        try {
          decodedToken = await auth.verifyIdToken(token);
          if (!decodedToken || !decodedToken.uid)
            return res.status(401).end('Not authenticated');
          req.uid = decodedToken.uid;
        } catch (error) {
          console.log(error.errorInfo);
          const errorCode = error.errorInfo.code;
          error.status = 401;
          if (errorCode === 'auth/internal-error') {
            error.status = 500;
          }
          //TODO handlle firebase admin errors in more detail
          return res.status(error.status).json({ error: errorCode });
        }

        return handler(req, res);
      };
    }

And this is how we can use it, notice instead of exporting handler we are exporting withAuth(handler)

    // get all sites of a user
    import { withAuth } from '@/lib/middlewares';
    import { getUserSites } from '@/lib/db-admin';

    const handler = async (req, res) => {
      try {
        const { sites } = await getUserSites(req.uid);
        return res.status(200).json({ sites });
      } catch (error) {
        console.log(error);
        return res.status(500).json({ error: error.message });
      }
    };

    export default withAuth(handler);

Here are the relevant files on GitHub: middleware.js and sites route.

That's all I learned about authentication on the client and server side with Next.js and Firebase. Overall it's a great developer experience and pretty painless to figure things out.

This article was originally published on my blog. Head over there if you like this post and want to read others like it.

Takeaways From "State of JS 2020"

Ran Ding — Sun, 14 Feb 2021 03:14:44 +0000

As I'm starting to learn about frontend development (see my plan here), a friend of mine recommended I take a look at the State of JS survey.

This is a pretty cool survey with 20,000 developers to identify current and upcoming trend and seems like a great resource for a quick overview of the landscape.

Knowledge Gaps

The first thing I got out of it are the knowledge gaps I have on various features of Javascript - the language and . Things that seem immediately useful but I didn't know about are:

Syntax
- Nullish coalescing
- Optional chaining
- Private Fields
Language features
- Proxies
- Decorators (didn't know JS has this)
- Promise.allSettled() (a rejected+resolved version of Promise.all())
- Dynamic Import
Data structures
- Typed arrays
- Array.prototype.flat()
Browser APIs
- Service workers
- WebVR
- Shadow DOM

See the full list: State of JS 2020: Features

Technologies

Pretty amazing visualization here, basically

Each line goes from 2016 to 2020, so we can see the trajectory
Upper right corner (1st quadrant) are popular technologies people also enjoy using
Lower right corner (4th quadrant) are things that are great but hasn't become super popular yet.

Overall, I seem to be picking items from the 1st quadrant already with the following exceptions

I haven't really done much testing, it seems Mocha, Jest and Cypress are good to check out. Another super high satisfaction but currently lower usage one is Testing Library.
TypeScript is on my radar, though not adopted yet
I picked up Next.js thought it is great to make static page, and app pages in one place - easy to write and deploy, good to see it's on an up and coming trajectory.
Redux seemed a bit of an overkill for the current level of complexity of my projects, but will keep an eye on it.

A couple of frameworks/tools people really love but I haven't heard or learned much about are:

Svelte (Front-end Framework)
Testing Library (Testing)

See the full list:

State of JS 2020: Technologies

Libraries

Looking over the list of libraries, lots of them are around:

dates
- moment
- date-fns
UI
- material-ui
- styled-components
- classnames
- tailwind css
data fetching
- axios
- got
data fetching with caching
- swr
- react-query
visualization+3D:
- d3.js
- three.js
form handling
- formik
- react-hook-form
utils
- lodash
- underscore
- jquery
misc / haven't looked into
- RxJS (async events management)
- Immer
- Ramda
- Luxon
- yup (schema validation)
dates
UI
visualization+3D:
form handling

More at State of JS 2020: Other Tools

Resources

The survey has a pretty long resources section. I'm particularly curious what developers are reading. Based on personal experience in the last month or so, I'm not surprised by CSS-Tricks ranking #1 :) . It's interesting to see Medium and Dev.to rank so high, I'll consider reading more there and repost my posts perhaps.

Opinions

The opinions section is interesting, in particular I'm glad to see the the "Javascript ecosystem is changing too fast" issue is less severe now...

But somehow a lot of other questions also have more dispersed (evenly distributed) response in 2019 and 2020 compared to prior years, not sure if just the respondent distribution changed, e.g. see graph below. So I'm not very sure whether to trust these.

Conclusion

OK, that was a useful resource to quickly catch up on the state of Javascript, I incorporate some findings in my plan here. Back to coding now ✌️

My Web Frontend Learning Plan - 2021

Ran Ding — Sat, 13 Feb 2021 18:03:22 +0000

You can also read the post at my blog, progress will be updated there

Despite working at a few large tech companies, I know very little about web development and especially frontend development - most of my time is spent on machine learning models. That's the curse of specialization, perhaps.

I have always been quite curious about the whole web frontend development ecosystem. It seems to me technology and developer tooling evolve very fast, and people have done a lot of creative things in this space in the last several years. This year, one of my goals is to get self-sufficient with building web projects end-to-end, and learning frontend development is one of the main items to tackle.

I thought about what I could do to keep myself more motivated and accountable and decided to write down my goals, plan, and updates here as I make progress.

Goals

My main goals are

Knowledge, breadth: become familiar with frontend development tools and process overall.
Skills, depth: establish my own workflow with a set of tools of choice, comfortable making frontend changes/projects.
Practical experience: have a few actual, small projects under my belt to gain practical experience, silly projects are OK/encouraged.

Plan and Progress Updates

Here is a basic breakdown of the key components in the tech stack that I know of so far. I will keep adding sections as I learn more. Under each section, I will also add updates (with dates) later on progress made or adjusted views.

I'll summarize the outcome in Notes and Projects sections at the bottom of this page. I plan to write posts to document my learning - some posts will be "learning notes" documenting something I learned or how I figured something out, other posts will be about some standalone projects. I hope writing help me develop more clear understanding of the subjects and create useful future reference.

HTML

I considered it to be pretty self explanatory and didn't dedicated anytime here, I'll trust Google/StackOverflow moving forward 🤞

Just kidding, I had a bit of experience before using HTML and datatables for putting together small data dashboard sites (all the pages are compiled beforehand). So I feel I knew enough here to get started.

CSS

Here is a quick 1.5-hr crash course that's pretty fast paced, but fairly minimal/basic. I'd expect a ton of Googling later.

Javascript (Browser and Node.js)

This is a new language to me, here is a great and quick summary I read through. I also took two Udemy courses last year: one for the language itself, one for using it on the server side with Node.js. I found these courses much lengthier than necessary, and can't really recommended them to others. I skipped most of the exercises and some topics - I plan leave that experience to actual projects. It was still good to get some context on practical applications and tooling setup.

React

New frontend frameworks show up every year, but Vue and React seem popular and here to stay. More recently Svelte seems to be gaining a lot of popularity. Overall, I feel React is more mature/well-supported, decided to go with it. For an quick comparison of other frameworks see the chart below from State of JS 2020.

Regarding learning resources: I was attempting to go through this course on Udemy but too fed up with the lengthiness and verbosity and decided to just read the docs on reactjs.org, which is excellent.

Some problems with React:

React seems to be completely client-side rendered and is geared towards SPAs (Single Page App). This brings some difficulties.

A complete web app / website will have some app pages and some content pages (like its landing page, blog content, FAQ etc). Client-side rendering these static pages that don't really have much UX/interactivity will be both unnecessary and probably catastrophic for SEO - slow page load speed compared to server-rendered/pre-rendered static pages, and crawler might have trouble executing client side JS.

One solution is to complete separate these static/content pages from the app. But for these content pages, it would be nice to reuse the same set of UI components from the React app (buttons, navbar etc). So completely separating also isn't a great option.

Here is where Next.js comes in.

Next.js (still using React, but better)

After reading about what Next.js offers in terms of hybrid static & server rendering it does, I was so impressed and think this is thing I was looking for.

Also to note, I tried out the deployment tooling at Vercel and really liked it. Previously to make a web app, we'd need to host the backend server somewhere (say AWS or Heroku), and deploy the frontend bundle somewhere (e.g. Netlify or S3 with some CDN). So there are always kind of two places to deploy to and to pay for - a bit of a hassle especially for small projects.

With Vercel, it handles the CDN for all the static assets, and hosts all the server side logic (api routes) as serverless functions executed on AWS Lambda (I think), so you really have zero hosting infra to worry about.

This is getting into a bit of backend, but hard to not talk about it given the issues we face with React alone. Historically Express.js has been the de facto standard here for Javascript developers, but Next.js has become a serious contender. See the chart below from State of JS 2020.

I want to give a big shoutout to Lee Robinson for putting out a lot of great content on Next.js, check out his courses, repos and content. Here is one of his courses that's pretty practical with an actual app.

UI Framework / Design System

After some tinkering with Material UI, it just felt a bit too convoluted. Perhaps I just didn't get it. After watching that React 2025 course videos, I decided to go with Chakra UI, will see how it goes and report back.

TypeScript

I took a very quick look at TypeScript for JavaScript Programmers, and played around with it by converting a small JavaScript project into TypeScript. The comfort and confidence writing with Typescript is excellent, and I can see for a bigger project/bigger team this is going to be super useful/mandatory, something I'll revisit as my project get bigger. The adoption can be incremental anyways.

Webpack

In the process of setting up TypeScript project I also got to learn a bit about Webpack. When we write client side code with lots of dependencies, we need to somehow build them into something (a bundle) the browser can run), there are many browsers to accommodate.

Webpack is a pretty popular and powerful tool for this bundling task (though I haven't really looked at others). It has a lot of flexibilities to customize the build process, so you can add additional steps, like processing TypeScript to JavaScript before transpiling and polyfilling so your code is compatible with older browsers. Webpack also makes it possible to handle non-code dependencies, like images, css or even markdown files.

One reality though is that the modern frameworks such as React or Next already provided build script and you largely don't need to deal with Webpack directly, but in various project if you want to customize things a little bit (like add an post-proccessing step, or copying some files somewhere), knowing a bit about Webpack comes in handy.

I learned everything I knew about Webpack from its documentation and this tutorial on YouTube, and can highly recommend it.

Testing

I haven't really done much here, it seems Mocha, Jest and Cypress are good to check out. Another super high satisfaction but currently lower usage one is Testing Library.