DEV Community: Alex Eagleson

How to Set Up a Fullstack Rust Project with Axum, React, Vite, and Shared Types

Alex Eagleson — Tue, 18 Oct 2022 13:28:33 +0000

Motivation
Project Setup
Rust Server
React Client
Sharing Types
Add Styles
Conclusion

Motivation

In this tutorial, I'll be demonstrating how to create a template for a fullstack web app using Axum as the backend in Rust, and React with Vite for the frontend.

We'll also be taking advantage of the Typescript type generating tool we built in a previous tutorial to generate TS types from Rust code and share types between our frontend and backend. This is a common benefit of working with a Node.js driven backend, but more difficult to achieve when building in another language (like Rust!)

Project Setup

If you're following along after completing the previous tutorial, then I suggest you create your app in a folder right next to your type generation CLI so that you can easily invoke it without dealing with painful file paths.

Alternatively if you published it, you can install the utility with cargo install, or simply install the demo one I created for the tutorial with cargo install typester.

(You can even follow this tutorial simply for the template setup and not use the type sharing utility at all if you choose)

Open your terminal where you'd like to create the project and run this command:



cargo new fullstack-app
cd fullstack-app

Open up /fullstack-app in your IDE and add the following dependencies to your Cargo.toml file:



[package]
name = "fullstack-app"
version = "0.1.0"
edition = "2021"

[dependencies]
axum = "0.5.16"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0.68"
tokio = { version = "1.0", features = ["full"] }
tower-http = { version = "0.3.4", features = ["cors"] }

In addition to axum for the web server, which is built on tokio, we also use tower-http to simplify CORS.

We will be using serde and serde_json for serialization of our Rust data to send to the frontend.

Rust Server

Begin by opening up main.rs and replacing it with this simple template for a web server with returns "Hello, world!" at the root route:

fullstack-app/src/main.rs



use axum::{routing::get, Router};
use std::net::SocketAddr;
use tower_http::cors::{Any, CorsLayer};

#[tokio::main]
async fn main() {
    let cors = CorsLayer::new().allow_origin(Any);

    let app = Router::new()
        .route("/", get(root))
        .layer(cors);

    let addr = SocketAddr::from(([127, 0, 0, 1], 3000));
    println!("listening on {}", addr);

    axum::Server::bind(&addr)
        .serve(app.into_make_service())
        .await
        .unwrap();
}

async fn root() -> &'static str {
    "Hello, World!"
}

If you are familiar with web servers in other languages this should all look pretty familiar. The root() function is the route handler for / which is set here:



let app = Router::new()
    .route("/", get(root))
    .layer(cors);

We'll be using a very permissive set of rules for CORS to avoid any issues testing locally, though you'll likely want to adjust this before you publish the app live.

Let's test it out! Start your server with:



cargo run

You should be able to visit http://localhost:3000 and see the "Hello, World!" response from your root route.

Once you have that working we'll add a route that returns some serialized Rust data.

First we'll create a second file in src called types.rs similar to how we did in the Typester project.

Populate it with the following type:

fullstack-app/src/types.rs



use serde::Serialize;

#[derive(Serialize)]
pub struct Person {
    pub name: String,
    pub age: u32,
    pub favourite_food: Option<String>
}

We're going to keep it fairly simple for this demonstration, just a basic struct. You can add more complex types later if you like.

Now let's update our server code to include a GET route that returns a serialized vector of Person structs.

When this is deserialized in Typescript the front end should expect to receive an array of Person objects.

fullstack-app/src/main.rs



use axum::{http::StatusCode, response::IntoResponse, routing::get, Json, Router};
use std::net::SocketAddr;
use tower_http::cors::{Any, CorsLayer};

// NEW
mod types;
use types::Person;

#[tokio::main]
async fn main() {
    let cors = CorsLayer::new().allow_origin(Any);

    let app = Router::new()
        .route("/", get(root))

        // NEW
        .route("/people", get(get_people))
        .layer(cors);

    let addr = SocketAddr::from(([127, 0, 0, 1], 3000));
    println!("listening on {}", addr);

    axum::Server::bind(&addr)
        .serve(app.into_make_service())
        .await
        .unwrap();
}

async fn root() -> &'static str {
    "Hello, World!"
}

// NEW
async fn get_people() -> impl IntoResponse {
    let people = vec![
        Person {
            name: String::from("Person A"),
            age: 36,
            favourite_food: Some(String::from("Pizza")),
        },
        Person {
            name: String::from("Person B"),
            age: 5,
            favourite_food: Some(String::from("Broccoli")),
        },
        Person {
            name: String::from("Person C"),
            age: 100,
            favourite_food: None,
        },
    ];

    (StatusCode::OK, Json(people))
}

There are three blocks in the above I've annotated as // NEW to indicate the changes we have made to the origin example of main.rs.

We've imported the types from the types.rs file, created a new route function called get_people which returns a vector of example people as JSON (the Json struct takes care of the serialization for us using serde in the background).

React Client

Now let's create the front end client that will access the server we just created.

From your /fullstack-app directory run the following command (make sure you have NodeJs installed first):



npm create vite@latest

For project name choose client and for the template choose React with Typescript.

Open up App.tsx and replace the sample code with the following:

fullstack-app/client/src/App.tsx



import { useEffect, useState } from "react";
import { Person } from "./types";

function App() {
  const [people, setPeople] = useState<Person[]>([]);

  useEffect(() => {
    fetch("http://localhost:3000/people")
      .then((res) => res.json())
      .then((people: Person[]) => setPeople(people));
  }, []);

  return (
    <div>
      {people.map((person) => (
        <p>
          {person.name} is {person.age} years old
        </p>
      ))}
    </div>
  );
}

export default App;

Sharing Types

You'll find that you have an error in the above code. Where is the Person type coming from?

The types file it's trying to import is not defined.

We'll use our typegen utility to create it!

Now the way that you use it is really going to depend on a few factors, such as whether you did the previous tutorial, and whether you published it. If you didn't do it at all, the third option will be to simply use the version I published with the post:

Option 1: Run your local copy

Let's say you have your typegen utility in a folder adjacent to this app like so:



/YOUR_TYPEGEN_PROGRAM
/fullstack-app

Then you will invoke it with the following command from the /YOUR_TYPEGEN_PROGRAM directory (or whatever you named yours):



cargo run -- --input=../fullstack-app/src/types.rs --output=../fullstack-app/client/src/types.d.ts

Option 2: Run your published copy

If you published your copy to crates.io then you can install it on your machine with:



cargo install YOUR_TYPEGEN_PROGRAM

And then from within the /fullstack-app directory:



YOUR_TYPEGEN_PROGRAM --input=./src/types.rs --output=./client/src/types.d.ts

Option 3: Run my Published Copy

If you didn't follow the previous tutorial you can simply use mine.

Just repeat Option 2 with typester in place of YOUR_TYPEGEN_PROGRAM.

Option 4: Run something more feature complete and production ready

If you're looking to take this little boilerplate demo further into something really usable (and not specifically interested in building your own type sharing library) then our little demo from the previous tutorial probably won't get you very far.

Here's a couple of much more feature rich options available out there:

Typeshare by 1Password, the original inspiration for this blog post. It's available on crates.io.
tsync by Wulf created as part of the awesome Create Rust App utility for setting up fullstack Rust apps. It's available on crates.io

If you choose to use one of these make sure you read their documentation as the API will be different from our utility we built.

If all goes well you'll have a generated file with the following filename and content:

client/src/types.d.ts



type HashSet<T extends number | string> = Record<T, undefined>;
type HashMap<T extends number | string, U> = Record<T, U>;
type Vec<T> = Array<T>;
type Option<T> = T | undefined;
type Result<T, U> = T | U;

export interface Person {
  name: string;
  age: number;
  favourite_food: Option<string>;
}

With this type file in place you should no longer see any error in your App.tsx file and person should be correctly typed and identifiable as a Person by your IDE for type checking and intellisense.

Here's a look at how VS Code will interpret it:

Add Styles

Just before we test out the finished version live, let's just it a quick pass with some CSS and turn these people into some basic employee cards.

We'll be straight up pulling it directly from this simple example here:

Create a file called App.css directly beside App.tsx (or override the one that is included with the project by default) and add this CSS:

client/src/App.css



.app {
  display: flex;
  flex-direction: row;
  column-gap: 16px;
}

.card {
  box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2);
  transition: 0.3s;
  max-width: 300px;
}

.card:hover {
  box-shadow: 0 8px 16px 0 rgba(0, 0, 0, 0.2);
}

.container {
  padding: 2px 16px;
}

img {
  width: 300px;
}

Next, update your App.tsx to include the following. You can feel free to change the avatar URLs if you choose. Make sure you don't miss the import "App.css"; in there!

client/src/App.tsx



import { useEffect, useState } from "react";
import { Person } from "./types";
import "./App.css";

const AVATAR_1 =
  "https://res.cloudinary.com/dqse2txyi/image/upload/v1666049372/axum_server/img_avatar_lf92vl.png";

const AVATAR_2 =
  "https://res.cloudinary.com/dqse2txyi/image/upload/v1666049372/axum_server/img_avatar2_erqray.png";

function App() {
  const [people, setPeople] = useState<Person[]>([]);

  useEffect(() => {
    fetch("http://localhost:3000/people")
      .then((res) => res.json())
      .then((people: Person[]) => setPeople(people));
  }, []);

  return (
    <div className="app">
      {people.map((person, index) => (
        <div key={index} className="card">
          <img src={index % 2 == 0 ? AVATAR_1 : AVATAR_2} alt="Avatar" />
          <div className="container">
            <h4>
              <b>{person.name}</b>
            </h4>
            <p>Age: {person.age}</p>
            <p>Favourite Food: {person.favourite_food ?? "Unknown"}</p>
          </div>
        </div>
      ))}
    </div>
  );
}

export default App;

Now, start up your Rust web server:

(Make sure you are in the /fullstack-app directory!)



cargo run

You should see this output:



listening on 127.0.0.1:3000

Next start your front end Vite dev server.

(Make sure you are in the /fullstack-app/client directory!)



npm run dev

If you see any errors on these commands, double check again that you are running them from the directories listed above.

With both running visit the default Vite server URL/port at:

http://localhost:5173/

Enjoy your beautiful new app!

Conclusion

I hope you've learned something new about building a web app using the best of both worlds (Rust and Typescript). I would encourage you to keep developing on this idea and see what additional features you can build!

That said if you are looking for something more production ready, I would highly encourage you to check out Wulf's Create Rust App, a fantastic template built in the same vein of this project, but much more feature complete.

How to Build a Rust CLI Tool to Generate Typescript Types from Rust

Alex Eagleson — Tue, 18 Oct 2022 13:27:13 +0000

Motivation
Introduction
Project Setup
Building the CLI
Working with Files
Basic Types
Enums
Structs
Tuples
Standard Library Types
Tests
Building and Publishing
Future Improvements
Similar Open Source Tools
Conclusion

Motivation

Why would you want to write a tool that converts Rust types to Typescript types? Aren't they entirely separate languages?

There are many reasons you might want to do so. The most common (though far from the only) case that we'll be focusing on is someone who is writing an API (like a web server) in Rust and consuming it with a Typescript frontend.

Being able to automate the sharing types defined on the backend with the frontend has a lot of benefits and and can significantly reduce development time spent re-writing the same definitions in multiple languages.

Introduction

Our very first goal will be to turn this into valid Typescript code by writing the type definitions in Rust and using our program to convert them into Typescript types:

import { NumberAlias } from "./types";

const printNum = (num: NumberAlias) => {
  console.log(num);
};

printNum(10);

If you try and paste this code into a .ts file and run it you will get an error saying NumberAlias is undefined and the types file you are trying to import it from does not exist. We will use the output of our program to create that file.

Type aliases work almost identically in Rust as they do in TS so this first challenge will be very simple in terms of output. Our goal is to generate the file:

types.d.ts

type NumberAlias = number;

From this Rust input file:

types.rs

type NumberAlias = i32;

Project Setup

There are two crates we will be relying on to develop out type conversion utility:

clap short for Command Line Argument Parser is the most popular Rust crate for dealing with command line arguments. We'll be using it to allow our uses to tell us the input/out filenames of the Rust and Typescript files they are using,
syn derived from the word syntax is a crate designed to convert Rust source code into a type-safe syntax tree where each token can be examined and parsed individually.

Open up your favourite terminal. Make sure you have installed Rust on your system. Fresh installations will also include Rust's package manager called Cargo which we'll need for this tutorial.

When you're ready run the following command to create the utility. I've decided to call mine typester, but you can call yours whatever you like.

cargo new typester

This will create a Cargo.toml file (equivalent to package.json in the JS/TS world) and an src directory with a main.rs file with a demo program. Test it out with cargo run and you should see "Hello, world!" printed to your terminal.

Next we will add the two packages (crates) that we need to write our program. Run the following command to automatically add the latest versions to your Cargo.toml file:

cargo add clap
cargo add syn --features=full,extra-traits

The syn crate does not include all features by default, so make sure you include the ones I've listed above. Full gives us access to the parse_file function and extra-traits derives Debug for syn's types to make it easier for us to log values and debug our code.

After adding your Cargo.toml file should include the following:

[dependencies]
clap = "4.0.9"
syn = { version = "1.0.101", features = ["full", "extra-traits"] }

Your version numbers may not match exactly, though if you are seeing different major versions I would recommend you align with the versions shown above for the sake of this tutorial in case either library has introduced breaking API changes since it was written.

Building the CLI

Before diving into the parsing itself, let's take a moment to set up the command line argument parsing with the clap crate. Fortunately for us, clap makes that extremely easy.

Here's some sample code. You can review clap's documentation for the version we are using, or read ahead for a brief overview.

use clap::{Arg, Command};

fn main() {
     let matches = Command::new("Typester")
        .version("0.1.0")
        .author("Alex E")
        .about("Convert Rust types to Typescript types")
        .arg(
            Arg::new("input")
                .short('i')
                .long("input")
                .required(true)
                .help("The Rust file to process (including extension)"),
        )
        .arg(
            Arg::new("output")
                .short('o')
                .long("output")
                .required(true)
                .help("The name of the Typescript file to output (including extension)"),
        )
        .get_matches();

    let input_filename = matches.get_one::<String>("input").expect("input required");
    let output_filename = matches
        .get_one::<String>("output")
        .expect("output required");

    dbg!(input_filename);
    dbg!(output_filename);
}

The above code will provide you with a great baseline to get started. It forces the user to define both a input filename (Rust) and an output filename (Typescript).

clap takes care of some really handy grunt work like providing a nicely formatted --help output and enforcing required arguments.

Before testing the program out, it's good to understand a Rust CLI's (command line interface) argument structure. When compiling a release binary you can simply add the arguments as any other CLI after the name of the program, but for cargo run you need to separate the run and arguments with two hyphens like so:

cargo run -- --input=src/types.rs --output=types.d.ts

The output of this command with our program will be:

[src/main.rs:29] input_filename = "src/types.rs"
[src/main.rs:30] output_filename = "types.d.ts"

Furthermore, as a user who may not be aware of the required arguments can take advantage of the --help flag:

cargo run -- --help

Convert Rust types to Typescript types

Usage: typester --input <input> --output <output>

Options:
  -i, --input <input>    The Rust file to process (including extension)
  -o, --output <output>  The name of the Typescript file to output (including extension)
  -h, --help             Print help information
  -V, --version          Print version information

That nicely formatted help output is thanks to the great clap crate.

Working With Files

We'll use the standard library's File, Read and Path to read the contents of our Rust file, and then parse it into a syn::File where we can iterate over the contents of the file in a typesafe manner.

use clap::{Arg, Command};
use std::{
    fs::File,
    io::{Read, Write},
    path::Path,
};

fn main() {

     ... // Code from previous section

    let input_path = Path::new(input_filename);

    let mut input_file =
        File::open(input_path).expect(&format!("Unable to open file {}", input_path.display()));

    let mut input_file_text = String::new();

    input_file
        .read_to_string(&mut input_file_text)
        .expect("Unable to read file");

    // This is our tokenized version of Rust file ready to process
    let input_syntax: syn::File = syn::parse_file(&input_file_text).expect("Unable to parse file");
}

Since this is a simple utility meant for learning and does not need to be bulletproof and handle every potential bad file input, we'll simply use .expect() in each scenario that doesn't conform to our requirements.

For those unfamiliar, .expect() is the same as .unwrap() which will panic on a Result of Err or an Option of None, but it allows us to provide some information to the user as to where/why that failure occurred.

Now that we have created a syn:File which the contents of our Rust file ready to parse, we can iterate over each token one at a time and start converting them into strings of text that conform to Typescript's syntax rules.

To do this we take our syn file and iterate over it with .items.iter():

use clap::{Arg, Command};
use std::{
    fs::File,
    io::{Read, Write},
    path::Path,
};

fn main() {

    ... // Code from previous sections

    // This string will store the output of the Typescript file that we will
    // continuously append to as we process the Rust file
    let mut output_text = String::new();

    for item in input_syntax.items.iter() {
        match item {
            // This `Item::Type` enum variant matches our type alias
            syn::Item::Type(item_type) => {
                let type_text = parse_item_type(item_type);
                output_text.push_str(&type_text);
            }
            _ => {
                dbg!("Encountered an unimplemented type");
            }
        }
    }

    let mut output_file = File::create(output_filename).unwrap();

    write!(output_file, "{}", output_text).expect("Failed to write to output file");
}

fn parse_item_type(item_type: &syn::ItemType) -> String {
    String::from("todo")
}

At this point we are ready to test it out Create the following types.rs file directly next to your main.rs file:

types.rs

type NumberAlias = i32;

Now run the following command:

cargo run -- --input=src/types.rs --output=types.d.ts

(Note the "--" before the input flag, this is a placeholder for the program name while we are building in development, make sure you include it or your program will not work properly.)

In your root of your project you should see a types.d.ts file appear containing the text todo:

types.d.ts

todo;

Basic Types

Now we are ready to parse the actual tokens. We'll start with the simplest one, the basic type alias type NumberAlias = i32;.

The will match on syn::Item::Type as we have written above and call our parse_item_type function which we need to expand and define. Each of our parsing functions for the different types our crate handles will return a string so we can simply concatenate them all together to build the output.

... // Code from previous sections with `parse_item_type` stub removed

/// Converts a Rust item type to a Typescript type
///
/// ## Examples
///
/// **Input:** type NumberAlias = i32;
///
/// **Output:** export type NumberAlias = number;
fn parse_item_type(item_type: &syn::ItemType) -> String {
    let mut output_text = String::new();

    output_text.push_str("export type ");

    // `ident` is the name of the type alias, `NumberAlias` from the example
    output_text.push_str(&item_type.ident.to_string());
    output_text.push_str(" = ");

    let type_string = parse_type(&item_type.ty);
    output_text.push_str(&type_string);
    output_text.push_str(";");

    output_text
}

The NumberAlias in our type will be captured with the ident value.

The i32 portion however could be any number of type values, and not necessarily a primitive. It would be (i32, i32) for example.

For this we'll create another handler function called parse_type:

/// Converts a Rust type into a Typescript type
///
/// ## Examples
///
/// **Input:** (i32, i32) / Option<String>
///
/// **Output:** \[number, number\] / Option<string>;
fn parse_type(syn_type: &syn::Type) -> String {
    let mut output_text = String::new();

    match syn_type {
        // Primitive types like i32 will match Path
        // We currently do not do anything with full paths
        // so we take only the last() segment (the type name)
        syn::Type::Path(type_path) => {
            let segment = type_path.path.segments.last().unwrap();

            let field_type = segment.ident.to_string();

            let ts_field_type = parse_type_ident(&field_type).to_owned();
            output_text.push_str(&ts_field_type);

            match &segment.arguments {
                // A simple type like i32 matches here as it
                // does not include any arguments
                syn::PathArguments::None => {}
                _ => {
                    dbg!("Encountered an unimplemented token");
                }
            }
        _ => {
            dbg!("Encountered an unimplemented token");
        }
    };

    output_text
}

As you an see above, we are only focusing on our simple type for the time being and ignoring all other matches.

If any type that our utility does not know how to process is encountered, the dbg! macro will print a message along with a convenient line number so user's can easily identify which condition was hit and expand on the utility if they choose.

You may also notice that for this Path type we have matched we are only taking the last() token. That is because currently we are not doing anything with the other parts of the path (for example std:fs:File), just extracting the name of the type.

The final function necessary to finish our prototype we will call parse_type_ident which deals with the most primitive types, or simply return the name of the type if it is a custom type.

All of Rust's many different types of numbers will simply be treated as a number when deserialized as Typescript:

/// Convert a primitive Rust ident to an equivalent Typescript type name
/// Translate primitive types to Typescript equivalent otherwise
/// returns the ident untouched
///
/// ## Examples
///
/// **Input:** i32 / Option / bool;
///
/// **Output:** number / Option / boolean;
fn parse_type_ident(ident: &str) -> &str {
    match ident {
        "i8" | "i16" | "i32" | "i64" | "i128" | "u8" | "u16" | "u32" | "u64" | "f32" | "f64"
        | "isize" | "usize" => "number",
        "str" | "String" | "char" => "string",
        "bool" => "boolean",
        _ => ident,
    }
}

We're finally ready to try it out. Run the command again:

cargo run -- --input=src/types.rs --output=types.d.ts

With the input:

types.rs

type NumberAlias = i32;

We get the output:

types.d.ts

export type NumberAlias = number;

And that's a valid Typescript type!

Congratulations, we have our first minimum prototype for a Rust -> TS conversion utility.

Of course as you can imagine, getting complete support for serialization of all Rust's types is a monumental effort, likely requiring months or years of continued development.

But fortunately like many things, 10-20% coverage of the most common types will take care of probably 80-90% of the most common use cases.

So let's focus on getting a few more of the most common cases covered, most specifically: enums and structs:

Enums

This is the example enum we'll be using to test our support:

#[serde(tag = "t", content = "c")]
enum Colour {
    Red(i32),
    Green(i32),
    Blue(i32),
}

And the expected output we'll be trying to create:

export type Colour =
  | { t: "Red"; c: number }
  | { t: "Green"; c: number }
  | { t: "Blue"; c: number };

You may be wondering what the #[serde(tag = "t", content = "c")] attribute is for?

Well it turns out there are quite a few different ways that an enum can be serialized. If you'd like to learn more about them this page in the serde documentation covers all the standard ones.

The method we'll be using is called adjacently tagged, the reason for this is that it makes it very clean and easy to handle on the Typescript side by using union types to discriminate on the t value and infer the correct type of the content c value.

Technically these t and c values can be any name that you choose, and we could write our utility to parse the attribute and read what the user has set, but since we are focusing on simplicity for now we're going to operate on the assumption that the user is using t and c.

With that preliminary assumption in place, we can now expand our initial match in the main function to include enums, and create a new parse_item_enum function:

    ... // Main function from previous sections

    for item in input_syntax.items.iter() {
        match item {
            // This `Item::Type` enum variant matches our type alias
            syn::Item::Type(item_type) => {
                let type_text = parse_item_type(item_type);
                output_text.push_str(&type_text);
            }
            syn::Item::Enum(item_enum) => {
                let enum_text = parse_item_enum(item_enum);
                output_text.push_str(&enum_text);
            }
            _ => {
                dbg!("Encountered an unimplemented type");
            }
        }
    }

    let mut output_file = File::create(output_filename).unwrap();

    write!(output_file, "{}", output_text).expect("Failed to write to output file");
}

Now we create the parse_item_enum function:


/// Converts a Rust enum to a Typescript type
///
/// ## Examples
///
/// **Input:**
/// enum Colour {
///     Red(i32, i32),
///     Green(i32),
///     Blue(i32),
/// }
///
/// **Output:**
/// export type Colour =
///   | { t: "Red"; c: number }
///   | { t: "Green"; c: number }
///   | { t: "Blue"; c: number };
fn parse_item_enum(item_enum: &syn::ItemEnum) -> String {
    let mut output_text = String::new();

    output_text.push_str("export type");
    output_text.push_str(" ");

    let enum_name = item_enum.ident.to_string();
    output_text.push_str(&enum_name);
    output_text.push_str(" ");
    output_text.push_str("=");
    output_text.push_str(" ");

    for variant in item_enum.variants.iter() {
        // Use the pipe character for union types
        // Typescript also allows it before the first type as valid syntax
        // https://www.typescriptlang.org/docs/handbook/2/everyday-types.html#union-types
        output_text.push_str(" | {");
        output_text.push_str(" ");

        // For simplicity this implementation we are using assumes that enums will be
        // using serde's "Adjacently Tagged" attribute
        // #[serde(tag = "t", content = "c")]
        // https://serde.rs/enum-representations.html#adjacently-tagged
        // As an improvement on this implementation you could parse the attribute
        // and handle the enum differently depending on which attribute the user chose
        output_text.push_str("t: \"");
        let variant_name = variant.ident.to_string();
        output_text.push_str(&variant_name);
        output_text.push_str("\" , c: ");

        match &variant.fields {
            syn::Fields::Named(named_fields) => {
                output_text.push_str("{");
                for field in named_fields.named.iter() {
                    if let Some(ident) = &field.ident {
                        output_text.push_str(&ident.to_string());
                        output_text.push_str(":");

                        let field_type = parse_type(&field.ty);
                        output_text.push_str(&field_type);
                        output_text.push_str(";");
                    }
                }
                output_text.push_str("}");
            }
            syn::Fields::Unnamed(unnamed_fields) => {
                // Currently only support a single unnamed field: e.g the i32 in Blue(i32)
                let unnamed_field = unnamed_fields.unnamed.first().unwrap();
                let field_type = parse_type(&unnamed_field.ty);
                output_text.push_str(&field_type);
            }
            syn::Fields::Unit => {
                output_text.push_str("undefined");
            }
        }

        output_text.push_str("}");
    }
    output_text.push_str(";");

    output_text
}

I've included comments above to annotate some of the sections that may be a bit more difficult to follow.

The great thing is that we are already seeing the benefits of reuse with the previous functions we created like parse_type.

As you continue to developer and improve this utility you'll find that creaking eac individual token parser into its own function allows for a lot of opportunities for reuse when you start handling more complex types.

With this in place we are now ready to test:

Expand our types.rs file to include both types we now support:

types.rs

type NumberAlias = i32;

#[serde(tag = "t", content = "c")]
enum Colour {
    Red(i32),
    Green(i32),
    Blue(i32),
}

Run the command:

cargo run -- --input=src/types.rs --output=types.d.ts

And our output will be:

types.d.ts

export type Colour =
  | { t: "Red"; c: number }
  | { t: "Green"; c: number }
  | { t: "Blue"; c: number };

export type NumberAlias = number;

(Note that I am using a Typescript formatter, specifically prettier, on the output to make it more human friendly. The actual output, while totally valid TS, will appear on a single line)

Structs

Now let's add support for structs. Our example test input will be:

struct Person {
    name: String,
    age: u32,
    enjoys_coffee: bool,
}

And our expected output will be:

export interface Person {
  name: string;
  age: number;
  enjoys_coffee: boolean;
}

Update the match in the main function to match on syn::Item::Struct:

    ... // Main function from previous sections

    for item in input_syntax.items.iter() {
        match item {
            syn::Item::Type(item_type) => {
                let type_text = parse_item_type(item_type);
                output_text.push_str(&type_text);
            }
            syn::Item::Enum(item_enum) => {
                let enum_text = parse_item_enum(item_enum);
                output_text.push_str(&enum_text);
            }
            syn::Item::Struct(item_struct) => {
                let struct_text = parse_item_struct(item_struct);
                output_text.push_str(&struct_text);
            }
            _ => {
                dbg!("Encountered an unimplemented type");
            }
        }
    }

    let mut output_file = File::create(output_filename).unwrap();

    write!(output_file, "{}", output_text).expect("Failed to write to output file");
}

Create the parse_item_struct function:

/// Converts a Rust struct to a Typescript interface
///
/// ## Examples
///
/// **Input:**
/// struct Person {
///     name: String,
///     age: u32,
///     enjoys_coffee: bool,
/// }
///
/// **Output:**
/// export interface Person {
///     name: string;
///     age: number;
///     enjoys_coffee: boolean;
/// }
fn parse_item_struct(item_struct: &syn::ItemStruct) -> String {
    let mut output_text = String::new();

    let struct_name = item_struct.ident.to_string();
    output_text.push_str("export interface");
    output_text.push_str(" ");
    output_text.push_str(&struct_name);
    output_text.push_str(" ");
    output_text.push_str("{");
    match &item_struct.fields {
        syn::Fields::Named(named_fields) => {
            for named_field in named_fields.named.iter() {
                match &named_field.ident {
                    Some(ident) => {
                        let field_name = ident.to_string();
                        output_text.push_str(&field_name);
                        output_text.push_str(":");
                    }
                    None => todo!(),
                }
                let field_type = parse_type(&named_field.ty);
                output_text.push_str(&field_type);
                output_text.push_str(";");
            }
        }
        // For tuple structs we will serialize them as interfaces with
        // fields named for the numerical index to align with serde's
        // default handling of this type
        syn::Fields::Unnamed(fields) => {
            // Example: struct Something (i32, Anything);
            // Output: export interface Something { 0: i32, 1: Anything }
            for (index, field) in fields.unnamed.iter().enumerate() {
                output_text.push_str(&index.to_string());
                output_text.push_str(":");
                output_text.push_str(&parse_type(&field.ty));
                output_text.push_str(";");
            }
        }
        syn::Fields::Unit => (),
    }
    output_text.push_str("}");
    output_text.push_str(";");

    output_text
}

Let's test it out:

types.rs

type NumberAlias = i32;

#[serde(tag = "t", content = "c")]
enum Colour {
    Red(i32),
    Green(i32),
    Blue(i32),
}

struct Person {
    name: String,
    age: u32,
    enjoys_coffee: bool,
}

cargo run -- --input=src/types.rs --output=types.d.ts

And our output:

types.d.ts

export type NumberAlias = number;

export type Colour =
  | { t: "Red"; c: number }
  | { t: "Green"; c: number }
  | { t: "Blue"; c: number };

export interface Person {
  name: string;
  age: number;
  enjoys_coffee: boolean;
}

Tuples

Tuples are a common data type in Rust that roughly correspond to array types in Typescript where each element has a specific type (so a tuple like (i32, i32) for example would be translated to [number, number] rather than the less specific number[] that vectors would translate into.)

We can add simple support by expanding the match &segment.arguments in our parse_type function to add this addition match arm:


// in the `parse_type` function

    ...
    // Tuple types like (i32, i32) will match here
    syn::Type::Tuple(type_tuple) => {
        output_text.push_str("[");
        for elem in type_tuple.elems.iter() {
            output_text.push_str(&parse_type(elem));
            output_text.push_str(",");
        }
        output_text.push_str("]");
    }
    ...

Standard Library Types

As a final update, let's add support for some of Rust's most common types in the standard library like Option, and HashMap.

The reason for these in particular is that they are common data types to be serialized, for example when parsed on the Javascript/Typescript side Option<T> can be thought of as T | undefined and HashMap<T, U> can be deserialized as an object in the TS form of Record<T, U>.

No doubt you can come up with more, but here are some straightforward TS mappings of some of the most common data types. We don't need to do these dynamically, we can simply prepend these pre-written types to the file output before we begin to parse:

Add the following line immediately after the variable output_text is declared in the main function:

output_text.push_str(&create_initial_types());

Then create that create_initial_types function as follows:

/// Initialize some Typescript equivalents of
/// core Rust types like Result, Option, etc
fn create_initial_types() -> String {
    let mut output_text = String::new();

    output_text
        .push_str("type HashSet<T extends number | string> = Record<T, undefined>;");
    output_text.push_str("type HashMap<T extends number | string, U> = Record<T, U>;");
    output_text.push_str("type Vec<T> = Array<T>;");
    output_text.push_str("type Option<T> = T | undefined;");
    output_text.push_str("type Result<T, U> = T | U;");

    output_text
}

The final version (of our incomplete program 😁) takes the following input:

types.rs

type NumberAlias = i32;

#[serde(tag = "t", content = "c")]
enum Colour {
    Red(i32),
    Green(i32),
    Blue(i32),
}

struct Person {
    name: String,
    age: u32,
    enjoys_coffee: bool,
}

struct ComplexType {
    colour_map: HashMap<String, Colour>,
    list_of_names: Vec<String>,
    optional_person: Option<Person>,
}

And produces the following valid Typescript output:

types.d.ts

type HashSet<T extends number | string> = Record<T, undefined>;
type HashMap<T extends number | string, U> = Record<T, U>;
type Vec<T> = Array<T>;
type Option<T> = T | undefined;
type Result<T, U> = T | U;

export type NumberAlias = number;

export type Colour =
  | { t: "Red"; c: number }
  | { t: "Green"; c: number }
  | { t: "Blue"; c: number };

export interface Person {
  name: string;
  age: number;
  enjoys_coffee: boolean;
}

export interface ComplexType {
  colour_map: HashMap<string, Colour>;
  list_of_names: Vec<string>;
  optional_person: Option<Person>;
}

So now that we have the ability to share types between Rust and Typescript!

Tests

It's always a good idea to add tests to your program/crate! Let's add some basic ones to test that we get the output for our sample types that we have been working with.

I'll start by creating a tests in src. I'm going to create three files total:

src/tests/type.rs
src/tests/enum.rs
src/tests/struct.rs

These files will just have the three basic sample types we have been working with:

src/tests/type.rs

type NumberAlias = i32;

src/tests/enum.rs

#[serde(tag = "t", content = "c")]
enum Colour {
    Red(i32),
    Green(i32),
    Blue(i32),
}

src/tests/struct.rs

struct Person {
    name: String,
    age: u32,
    enjoys_coffee: bool,
}

Next I'll write some simple test that read in these files and assert that the output matches the Typescript output that we expect, so that we can safely expand our type generating utility in the future and be able to know immediately by running our tests that we haven't broken any existing functionality:

For now I'm simply going to add this testing module down at the bottom of main.rs below the program itself and I'll use use super::* so I have access to all the same modules that the main program does:

main.rs


#[cfg(test)]
mod tests {

    use super::*;

    #[test]
    fn handles_type_alias() {
        let mut input_file = File::open("./src/tests/type.rs").unwrap();

        let mut input_file_text = String::new();

        input_file.read_to_string(&mut input_file_text).unwrap();

        let input_syntax: syn::File =
            syn::parse_file(&input_file_text).expect("Unable to parse file");

        let typescript_types = parse_syn_file(input_syntax);

        assert_eq!(r#"export type NumberAlias = number;"#, &typescript_types);
    }

    #[test]
    fn handles_struct() {
        let mut input_file = File::open("./src/tests/struct.rs").unwrap();

        let mut input_file_text = String::new();

        input_file.read_to_string(&mut input_file_text).unwrap();

        let input_syntax: syn::File =
            syn::parse_file(&input_file_text).expect("Unable to parse file");

        let typescript_types = parse_syn_file(input_syntax);

        assert_eq!(
            r#"export interface Person {name:string;age:number;enjoys_coffee:boolean;};"#,
            &typescript_types
        );
    }

    #[test]
    fn handles_enum() {
        let mut input_file = File::open("./src/tests/enum.rs").unwrap();

        let mut input_file_text = String::new();

        input_file.read_to_string(&mut input_file_text).unwrap();

        let input_syntax: syn::File =
            syn::parse_file(&input_file_text).expect("Unable to parse file");

        let typescript_types = parse_syn_file(input_syntax);

        assert_eq!(
            r#"export type Colour =  | { t: "Red" , c: number} | { t: "Green" , c: number} | { t: "Blue" , c: number};"#,
            &typescript_types
        );
    }
}

When we run the following command on the terminal:

cargo test

We would see the output:

running 3 tests
test tests::handles_type_alias ... ok
test tests::handles_enum ... ok
test tests::handles_struct ... ok

Remember to keep your tests updated as you expand your utility!

Building and Publishing

There are a few different ways we can use and share our program that we'll go over quickly.

The first is with the command we have been using to do build and run a development version:

cargo run -- --input=src/types.rs --output=types.d.ts

We can also build a release version with all the optimizations using:

cargo run --release

Then you'll find the typester binary in the target/release folder. You can now place it anywhere on your machine and run it from the command line by navigating to the directory and invoking:

./typester --input=src/types.rs --output=types.d.ts

The last method if you want to share your utility with others, or use it on other machines is to publish it to crates.io.

You'll first need to upload your project to Github and make a crates.io account if you don't have one already. This tutorial will take you through all the steps you need to publish your crate.

Now you can install your CLI locally with cargo install YOUR_CRATE_NAME.

You can see here I've published this utility to crates.io and now I can install it on any machine that has cargo installed with:

cargo install typester

Which installs by default in your $HOME/.cargo directory. Now you can use the utility on any directory on your machine!

Future Improvements

There's plenty of options for improving this crate you can pursue beyond just improving support for additional types within Rust's ecosystem.

Here's a few examples:

Target an entire project directory rather than just a single file with the walkdir crate
Capture doc comments for your types and convert them to jsdoc comments so your frontend dev team knows exactly how to use your types
Custom attributes. We've already talked about parsing serde's attributes, what what about custom ones specifically for your crate like the ability to #[ignore] types that aren't intended for frontend consumption?
Camel case. Have your CLI watch for serde's rename attribute and transform your struct fields into Typescript's standard of camelCase from Rust's snake_case
Generics! Can you include support for them. I don't even know where to start with that one, but it would probably be a great challenge.

I'm sure you can come up with other great ideas too, feel free to comment on this post with your own!

Similar Open Source Tools

If you are actually looking for a more robust version of this kind of tool and not interest in writing it yourself, there are a number of very similar and more feature rich versions of this utility out there already in the wild:

Typeshare by 1Password, the original inspiration for this blog post. It's available on crates.io.
tsync by Wulf created as part of the awesome Create Rust App utility for setting up fullstack Rust apps. It's available on crates.io

Conclusion

I hope you learned something new about how Rust itself can be parsed with crates like syn and different kinds of helpful utilities can be built out of it.

There's no reason you can't add support for other languages you use too! I simply chose Typescript as it's the one I'm most familiar with.

In a follow up blog post, we'll take a look at how to actually use this utility in a real fullstack application with an
Axum backend in Rust, and React with Vite for the frontend.

The Complete Beginner's Guide to Web Development

Alex Eagleson — Wed, 21 Sep 2022 13:30:37 +0000

Topics

Introduction
- What is a Front-End Developer?
- What is a Back-End Developer?
- What is a Full-Stack Developer?
Basic Tools and Concepts
- What Communication Tools are used?
- How do Meetings work?
- What is Project Management?
- What are Designs Systems?
- What are Designs Tools?
- How do Code Reviews work?
- What is Reading Code?
- Asking for Help
Getting Started
- Computer Hardware
- A Working Environment
- A Way to Test Your Work
- A Goal to Work Toward
- A Resource to Learn From
Web Development Basics
- How does a website get created?
- How can I build a website or web app?
- How can I build websites on my own computer?
- How do I set up a development environment on my computer?
HTML
- What is HTML?
- How do I use HTML?
- What else do I need to know about HTML?
- What are the best free resources for learning HTML?
Web Servers
- What is a web server?
- What is a web request?
- How do I run a web server?
- What else do I need to know about web servers?
- What are the best free resources for learning web servers?
CSS
- What is CSS?
- How do I use CSS?
- What else do I need to know about CSS?
- What are the best free resources for learning CSS?
Javascript
- What is Javascript?
- How do I use Javascript?
- What is the DOM?
- How do I view the DOM?
- How do I add Javascript to a website?
- What are the best free resources for learning Javascript?
- What is Node.js
- What are the best free resources for learning Node.js?
Command Line Terminal
- How do I use the command line terminal?
- What else do I need to know about the command line terminal?
- What are the best free resources for learning the command line terminal?
Git
- What is Git?
- What is a Version Control System (VCS)?
- How do I use Git?
- What is the difference between Git and Github?
- What else do I need to know about Git?
- What are the best free resources for learning Git?
APIs
- What is an API?
- How to I create an API?
- What are the best free resources for learning APIs?
Libraries and Package Management
- What is a library?
- What is a package?
- What is a package manager?
- What are NPM and Yarn?
Databases
- What is a database?
- How do I use a database?
- What are the best free resources for learning databases?
Front End Frameworks
- What is a front end framework?
- What are React, Angular, Vue and jQuery?
- What is a CSS framework?
- What are Bootstrap and TailwindCSS?
- How do I use a front end framework?
- How do I use React?
- What else do I need to know about React?
- What are the best free resources for learning React?
Typescript
- What is Typescript?
- What is the difference between Typescript and Javascript?
- How do I use Typescript?
- What are the best free resources for learning Typescript?
Web Hosting
- Where do I host a website?
- How do I put my website on the internet?
- How do I host a web app?
- How do I host a React app?
- How do I host a Node.js server?
- How do I host a Node.js app?
- What is a CDN?
Web Domains
- What is a URL?
- What is a domain name?
- What is DNS?
- How do I get a custom domain name?
- What is SSL?
- What is HTTPS?
- How do I get an SSL certificate?
SSH
- What is SSH?
- How do I use SSH?
- What are SSH keys?
- How do I generate SSH keys?
DevOps
- What is CI/CD?
- What is Docker?
Interviewing and Job Hunting
- How long until I will be ready to apply for jobs?
- How do I know when I am ready to apply to jobs?
- Where do I find jobs?
- What programming language should I learn?
- Why are there so many different programming languages?
- What if I fail?
- What if I get overwhelmed?
- What are the most important factors in getting hired?
- How do I get a job when all companies want existing work experience?
- What can I expect from an interview?
- Do I need to get a degree?
- Do I need to go to college?
- Which factors most influence potential income?
- What other factors should I consider when applying to a company?
- How do I know if my expectations are too high?
- What if I find a job, but it's not what I was expecting?
More Learning Resources
- What resources should I use to learn web development?
- What are the best free resources for learning computer science?
Conclusion

Welcome learners!

This post is targeted at anyone and everyone out there who has decided, for any reason at all -- that they would like to become a web developer.

No matter your age, or experience level. No matter whether you are a student, recent graduate, or someone who has had a lengthy career in a completely unrelated field. This guide is for you.

Although there is plenty of information that even experienced developers may find helpful, the primary audience I am aiming to target with this post are complete beginners.

I know it can seem extremely overwhelming at first, and it's important to know that this is totally normal. The purpose is to give readers an introduction to a massive range of topics. It is not meant to be a fully comprehensive deep dive into any one of those topics in particular.

Instead, at the end of each section where I discuss a concept, I will include a curated list of links and resources which you can visit to continue your learning on that topic.

With a couple exceptions, all of these resources are entirely free.

At any point a topic is spoken of or introduced, I will make an attempt to answer these three questions:

What is it?
Why is it important?
How do you use it?

These questions are critical to beginners in particular as I know it can often be extremely difficult to bridge the gap between discussion of a concept and seeing how it's actually used in practical application. For example: I know websites use a programming language called Javascript but where do I actually write that code and how do I make it run?

If you encounter anything in this tutorial that you feel is skipped over too quickly, please let me know in a comment and I will make a best effort to update it to be more clear.

My goal here is to provide you with not only the technical details you need to know, but also answers to a lot of more generalized common questions that come up like:

"How long do I need to practice before I can apply for jobs?"
"Do I need to go to college? (or back to college again?)"
"How do I know which things I even need to learn?"

In fact, maybe you're still so early in your journey that you don't even know what being a "web developer" means! And that's perfectly okay.

Perhaps you're still in college and worried about what it's going to be like to apply for jobs when you graduate. Or maybe you hate your current job and you heard on the internet that being a web developer is a real sweet gig.

Well, you're right it is. And it's a bit unique compared to some other professional fields like medicine, law or civil engineering where you really can learn and acquire all the skills necessary to succeed from the comfort of your own home entirely for free.

That may not necessarily be the most efficient route, or the one that is right for you, but it does afford potential avenues of opportunity to a lot of people who might otherwise not have the same options.

With that said it's important to state unequivocally that being accessible does not make it easy. Learning web development is difficult. Really difficult. It's a massive field.

That's the bad news.

The good news, is that it's a good kind of difficult. And you don't need to know everything. Not even close. I certainly don't.

If your goal is to get employed, you just need to learn enough.

Enough to get started and make small contributions to the big projects at the companies where you work. Enough to keep asking questions and learning as you go.

This guide is designed to introduce you to the basics of the many concepts you'll encounter in the web development world, but keep you focused on that goal.

(I'll add one last reminder that if at any time you feel overwhelmed or that anything in this guide is too technical for you, even as a complete beginner, to please comment and leave feedback. I will always be happy to try and improve things to help.)

Aspiring to becoming a web developer (which itself is just a specific area of focus for the more generalized role of software developer) means signing up for a lifetime of learning.

If that alone isn't enough to scare you aware right now, then that's a good sign you're on the right track.

The very first lesson, and the first thing you need to learn, is to become comfortable with not knowing how to do something. It's going to happen to you constantly. Even when you've got years of experience. There's a good chance that you'll encounter something you don't know how to do at first glance as frequently as every day.

If you see yourself as someone who can be in that situation, and see it as a challenge to overcome rather than an impasse, then you are fortunate enough to possess what is probably the most important prerequisite for becoming a developer.

The entire software industry exists because of people with that attitude, and it is entirely dependent on them to continue to function.

If you're still reading I'm going to assume you've agreed that you are that kind of person. Great! Let's take this opportunity to test that.

Is there a roadmap or summary of everything I should learn?

I'm going to hit you with a link that you might come across while you're beginning your journey. Maybe you've already come across it.

Before you click, acknowledge that part of being a developer is recognizing that it's perfectly okay not to know something. It's perfectly okay not to know lots of stuff!

Take a moment to skim through the link below. If you feel completely overwhelmed, remember that's totally normal.

The step-by-step roadmap to becoming a front-end developer

What do you think? That's a heck of a lot you're going to have to learn isn't it? Maybe this wasn't such a good idea?

The reality is you're going to come across a lot of similar links like this while you are learning. They are often going to make you feel like maybe it's too much, that the journey will not be possible.

I've been a developer for many years (including enough to be teaching and training others) and there's topics and tools on that list that not only have I never used -- but there's some I've never even heard of!

I'm here to tell you definitively that if your goal is simply to "become employed as a web developer" then you can safely devote 100% of your attention to just the first five things on that list:

Internet (the browser and simple web servers)
HTML (the structure of websites and web apps)
CSS (the style/appearance of websites and web apps)
Javascript (the interactivity and data for websites and web apps)
Version control systems (the ability to work with other team members and back up your project)

That's it. Almost everything else on that list is simply a tool someone has created that boils down to one of the above (or makes it easier/faster to work with them).

If you have a solid understanding of the fundamentals of those five things, then you will have the necessary knowledge to learn everything else.

I'll pick a few random ones off the list to demonstrate:

Electron: Electron is a tool that uses chromium (basically the Chrome browser) to let you write web apps that you actually install on your computer. It's popular because since it's Chrome based, you can write those apps and run them on almost any platform (Windows, Mac, Linux, etc). You write those apps in HTML, CSS and Javascript. So if you know those three, then you can easily learn Electron. Popular Electron apps include Slack, Discord and VS Code.
Sass: Sass is an enhanced version of CSS. It lets you write a bunch of extra syntax that you can't write in normal CSS in addition to regular CSS. Once it's written you use a little tool called a preprocessor to convert that file into a regular CSS file. If you understand the fundamentals of CSS, you can easily learn SASS.
React: React is a tool written in Javascript to make organizing and interacting with parts of the browser easier. Its syntax is basically a mix of HTML and Javascript together. Rather than having to manually write all the instructions for interacting with browser elements in Javascript, you can write re-usable React components to share behavior across different parts of your website or web app. Similar to SASS, a processor is then run over the React code to convert it into Javascript. If you understand the fundamentals of HTML and Javascript, then you can easily learn React.

Those are just three examples, but as I said the majority of topics on that roadmap could have similar descriptions. Most everything in the front-end ecosystem comes back around to HTML, CSS, and Javascript.

HTML, CSS and Javascript are the building blocks of the web, have been for decades, and despite what anyone says, there is no indication that is going to change anytime soon. Nothing is truly future proof, but those three are about as close as you get in the web development industry.

So that basic introduction out of the way, let's talk about what being a web developer really means.

What is a web developer?

A web developer is someone who contributes work to any part of a web page or application. It is a subset of the field of software development describing someone who specifically works on projects that are accessed by users over the internet.

It doesn't even necessarily mean writing code. For example you might build pages in Wix or Squarespace or Wordpress using drag-and-drop tools to place the elements where you want them.

Anyone who works on building websites is a web developer (at least as far as I'm concerned). They're just using different tools.

Nevertheless, the most common definition of a web developer is someone who uses HTML, CSS and Javascript to build websites and web apps.

The majority of jobs out there hiring for a "developer" position are looking for those skills at absolute minimum. The majority of this tutorial will be focused on those three technologies.

What is the different between a Front-End, Back-End and Full-Stack developer?

Front End Developer

A front-end developer is someone whose focus is on the part of we pages and apps that the user sees. Their core tools are HTML, CSS and some Javascript.

There is a very interesting article called The Great Divide that discusses the divide in the front-end development world.

At absolute minimum I would expect someone applying as a front-end developer to be able to work comfortably in HTML and CSS, with a basic knowledge of Javascript.

Back End Developer

A back-end developer is someone who works on the part of the sites and apps that relates to the data itself and how the user interacts with it.

A back-end dev needs to understand network configuration, hosting, HTTP, REST, user authentication, be familiar with web servers like Apache and Nginx, and know how to work with databases like MySQL or MongoDB.

Common programming languages you'll use to do back-end development are PHP, Python, Javascript, Java and C#.

Full Stack Developer

Full-stack developers are simply devs who have experience working on both front-end and back-end. Often they will not have as much experience as a dedicated dev in either domain, but they are considered extremely valuable to companies for being a jack of all trades.

Full-stack devs are more common now as many modern tools are built around the assumption of blurring lines between front and back end. It's more common than ever for companies to want to hire devs who can work with both.

My recommendation to anyone would be to have a preference for either front or back, but at least be willing to learn the basics of the other. It will greatly benefit your career opportunities if you do.

This guide will focus on tools and strategies needed to become a front-end developer, but we'll cover basic explanations of the fundamentals of back-end as well so you're at least aware of how they work if that's a path you decide you want to take.

Basic Tools and Concepts

What does a typical work day look like for a web developer?

Let's consider an example workday for a front-end developer. We'll focus on a junior developer position since that is likely what you are going to be working toward.

We'll use this opportunity to discuss topics and tools that are commonly used in real workplaces, that you should be aware of, but might not necessarily encounter when working alone including meetings, project management tools, design systems, and communication tools.

Communication Tools

Your day will likely start with opening your email, then opening up a communication tool like Slack or Microsoft Teams.

These tools are what you use to chat with your team in real time, particularly if you are working remote, but even within the office most communication is done through a tool like this.

For video discussion you'll likely be using something like Zoom, Google Meet or Skype. Most people these days are familiar with at least one of these tools, they are pretty interchangeable for the most part.

Companies typically use these for basic communication for work to be done, or asking questions and clarifying assumptions. Bear in mind that when it comes to actually documenting work that needs to be done, tools like these are usually discouraged as they are not designed to manage tasks. We'll cover that when we get to project management tools.

Once you've logged in and checked your emails, it's often time for the daily standup or similar meeting.

Meetings

You begin your day often with a meeting called a "stand up". This is where other developers on your team and your team lead discuss the projects they are working on, what they have done, what they are planning to do, and whether there is anything "blocking" their work.

An example of a "blocker" would be for example waiting for a client to send you a description of the new page they want built, which you need before you can begin working on it.

Meetings are very common in the developer's workplace (often a source of jokes and memes), but humour aside there is a reason they are popular. Often it is beneficial to a project to discuss in real-time work that needs to be done, and the different approaches about how you plan to accomplish it.

The most important thing to do to have successful meetings in the workplace is to document them and produce a list of actionable tasks. Everyone should walk away from a meeting having a better idea of the work that needs to be done.

At this point it's likely time to review the work that does need to be done that you are responsible for. This is usually done through issue ticketing systems. People within your organization will create tickets that represent issues with your project or work that needs to be done in one of the many project management tools that exist out there.

Project Management

What is Agile?

It's a good idea to familiarize yourself with at least one of these tools before you start looking for employment. One of the simplest ones to play around with is called Trello.

It has a free plan and it would be a good idea for you to use it to keep track of work that needs to be done even if you are only working on personal projects for yourself, just to get familiar with the workflow of creating issues and marking them complete.

Other more complex professional tools in that space you might use in your real job include JIRA, Github issues and Gitlab. You don't need to know how they work, just being aware they exist is fine.

In addition to the tools themselves, there are many different ways that you can choose to use them. This is where you may have heard terms like agile, waterfall, kanban,and scrum master.

These are different project management methodologies that sound fancy, but ultimately are just different approaches to tracking work and making sure expected projects are delivered on time. Feel free to Google the above terms to learn more if you desire.

As a developer you honestly really don't need to worry too much about these, you simply follow whatever methodology your company uses (oftentimes will be no specific methodology at all).

So at this point you have found a ticket assigned to you. We're ready to do actual development work.

The ticket looks something like this:

This is an extremely simple example I whipped together on Trello, but it includes most of the fundamentals. You'll have a basic title that describes the issue, and a description that explains the work that needs to be done.

Oftentimes you'll see additional things like a due date, priority list, tags for organizing and grouping related tickets etc. As a developer, particularly a junior developer, your only real concern will be the description of the ticket itself and marking it as "complete" when the work is finished.

You'll notice here the description says to use the design in Figma to complete the task. What is Figma?

Before we can answer that question, first let's talk briefly about Design Systems because the two are very closely connected.

Design Systems

As someone who does not pretend to be a designer nor a design expert, I have nevertheless come to appreciate the importance of good design and a good design system.

A design system is a set of standards to manage design at scale by reducing redundancy while creating a shared language and visual consistency across different pages and channels.

-- Nielsen Norman Group:

The further I go in my career the more I realize that proper design of a piece of software before development occurs can make or break the success and scalability of a product.

Pay particular attention to the terms "shared language" and "consistency". Imagine you are building a massive product with hundreds of other developers. Here's just a couple examples among countless that might occur without a proper design:

If you don't all agree on the term used for a concept in your app, you may end up with a dozen different names for the same thing like "customer" vs "user" vs "employee" across the app and two developers may think they're working with totally different data when they're actually talking about the same thing.
The designer may offer a specific hex code for "yellow" that is intended to use across the app, but if the relationship between that "yellow" is not defined between actual behavior, one developer may make it the default colour for "success" in one part of the app while another defines it as "warning" leading to a confusing UX experience when a user of your product can't easily tell if their action was correct or not.

Examples of famous design systems include Google's Material Design, Apple's Human Interface and IBM's Carbon Design System.

These systems are not necessarily exclusive to those companies, they are made publicly available specifically so they can be used by any designer and developer. Many open source tools like Material UI for example (a React component library) are built on top of these basic principles and available to use by anyone.

Depending on the company you work for, they might build on top of an existing design system while trying to modify it to their company's needs (for example just change the colours to match their brand colours). This has the benefit of being fast and inexpensive since you are working on top of existing verified designs already established.

Alternatively many larger companies will opt to build their own design systems from scratch. This has the benefit of being more unique, while also being more expensive both from the design side and the development side.

(And then of course you may be unfortunate enough to land at a company that does neither and expects the developer themselves to handle both design and development and just "make it look good". If you find yourself in that situation.... good luck!)

Once you have established what kind of design system and design language you will be using, the next step is to pick the right design tool to foster the best collaboration between designer and developer you can.

Design Tools

What is Figma?

Figma is a design tool, and pretty much the de-facto current standard in the web development industry. Other similar tools exist like Adobe XD and Zeplin but we're going to focus on Figma since it's most likely the one you will encounter.

Again, like many other tools it's more about the basic concepts. Which specific one you use doesn't make a big difference as long as you understand the fundamentals.

The general process at most companies (particularly large ones) when creating a new website or web app will be to built the wireframes about how they want it to work and function before development work actually begins.

Remember your job as a web developer isn't to make decisions about how they work or how they look (that's the job of the product manager and UI/UX designer respectively).

Your job is to implement those decisions and designs as code.

Here's a small example of what you might see when you follow the link in the ticket to the Figma design they want you to implement:

Each element built by the designer can be clicked on and you can see all the details about it, often even the CSS code you can use to build your actual implementation off of.

These designs will serve as the example of what the expectations are for how the site should look and function. When your code gets reviewed after it has been completed, the person doing the review will be comparing the work you did and checking to make sure it matches the intended design.

Remember that a website or app includes both design and function. As a front-end developer you are not just responsible for making sure images are laid out properly and that buttons are red, but also that the correct data is being displayed and correct logic is being used to display it. That's where the more complex programming aspects come into play.

It's not necessarily easy to tell from this image, but your task might include fetching data from an API which includes a real time list of all the plants the company has in stock, and exactly what their inventory levels are. This data will be changing from minute to minute, which is why you can't just write those descriptions in by hand.

All that and more is part of your responsibility as a front-end developer to create a working "New Arrivals" page. You will do this work almost entirely in the scope of HTML, CSS and Javascript.

We briefly mentioned reviews, and that is the last topic we will touch on as part of your daily experience in your front-end developer role.

Code Reviews

One last critical part of a developers job is also one of the most difficult to replicate and get experience with when you are learning: code reviews. Because you will be doing most of your work on your own, you won't get to benefit from having your work reviewed by others (unless you are lucky enough to find friends or peers online willing to do so, in which case I would highly recommend you take advantage of it).

Code reviews are a critical part of the development process for most (unfortunately not all) companies. The reason they exist is to try and ensure only that all code written and implemented as part of the larger codebase meets a certain standard of quality.

Code reviews are usually done through your version control system when your changes are committed. We'll speak more on these topics when we get to git, but the basic idea is that your code can be viewed live in a browser on a website like Github for other team members to see.

They have the ability to leave comments and either "approve" or "deny" and send back with a request to fix errors, or make general improvements.

It's important to look at code reviews as a positive opportunity for improvement. They are probably going to be one of the most important sources for your growth, and the best developers I know are those who seek out and ask for reviews from others so they can learn.

Those doing the reviews are encouraged to focus on finding and fixing potential errors, as well as issues that may result in code that is difficult to maintain in the long run. They should be done from a perspective of positive encouragement and keeping in mind that it's not something to take personally. Done properly code reviews are one of the best tools a company has to create long term maintainable systems.

Here's a great blog post about how to do a code review effectively. Well worth reading if you are going to be joining a team working with other developers.

What skills are critical for a successful junior developer?

That brings us to about the end of the day, you've successfully finished your first day as a junior developer!

There are two final closing thoughts I have that I feel are critical to be successful in a developer role that you may not necessarily get to practice much while learning on your own, so I think this is a good place to mention them:

Reading Code

Learn to Read code -- not just write it.

For many developers, particularly new ones, you will actually spend more time reading code than writing code in the real world.

The reason for that is that as someone with less experience, you will not be tasked with building new systems, but rather making small improvements to existing ones.

A lot of developers really underestimate how difficult it is to read code. Reading other people's code is hard. Harder than writing code, because when you write code you know what your intention is, but when you read it you have to try and figure out other people's intentions through the code they have written.

And unfortunately, the reality is that a lot of those code is going to be a mess. If you have any rose coloured glasses on regarding expectations of clean codebases and well documented requirements you're going to be in for a bit of a rough ride.

Oftentimes that is the exception rather than the rule, so you need to be ready to dive into something that isn't necessarily straightforward and figure out what's going on in order to make the changes that need to be made.

If this is something you would like to practice, once you have learned the basics of writing code, I would suggest looking into contributing to open source projects. It can be very intimidating at first, but also very rewarding.

Asking for Help

My rule of thumb for someone new is to try figuring out something on your own for about 10-15 mins, and if you still haven't made progress, then ask for help. That time frame will gradually increase as you get more experience.

Any good company that hires a new developer will understand that the goal is to get your trained and up to speed and will properly dedicate assets (other developers) to be available to mentor you and answer questions.

If you ever find yourself in a position where you are new and other developers are "too busy" or "don't have time to help" then please know that is a failure of the company, not a failure of yours.

Getting Started

How do I get started?

Now that we have established the mindset you want to be in, and what bring a "web developer" really means, let's talk about getting all the tools you need and setting you up for success.

There are five fundamental things that you should have before you get started

Computer Hardware
A Working Environment
A Way to Test Your Work
A Goal to Work Toward
A Resource to Learn From

Computer Hardware

To have the most user friendly and effective setup, you are going to want to have a standard computer or laptop with either Windows, Mac OS or Linux installed.

It is certainly possible to learn programming on a Chromebook or even a tablet / phone, however you will encounter a lot more challenges in trying to do so than if you can get your hands on a computer with a traditional operating system.

Whether you choose Windows, Mac or Linux makes absolutely no difference as a beginner. Even as a professional all three of them are perfectly good viable options. Don't ever let anyone tell you that you need a Mac to be a developer. Or that you have to use Linux.

You'll be perfectly fine to get through your learning experience on whatever you have available to you, and if your job ever requires a specific one, then the company will provide that one for you.

If budget is an issue, I would recommend looking for a used laptop. Something with an least 4GB of RAM (that's about the minimum for most modern coding environments).

A Working Environment

Although it is possible to learn web development entirely with tools in the browser, it will limit you significantly compared to setting up your own computer to do your development work on.

You can technically do your development in any text editor (even something as simple as Notepad).

The reason we use specialized code editors in the software field is because they can provide tons of added features to both speed up development and check for errors in your code automatically as you type it.

You will see the shorthand IDE which stands for integrated development environment often. The most popular IDE by far is Microsoft's Visual Studio Code referred to as VS Code for short.

It's totally free and you can use it to help you write code in almost any programming language in existence.

Later in this guide we will go into full detail on how to set up a simple web server on your computer that you can use to test and run your code.

A Way to Test Your Work

Once your code is written, how do you actually run it to see if it works?

If you are learning web development, then you need something that sends your HTML, CSS and Javascript code to your browser to run. That "something" is called a web server.

When you type an address into your browser's URL bar, that URL gets converted into an IP address which identifies a particular machine somewhere on the web. If that machine is running a web server, then your browser will request data from it (usually HTML, CSS and Javascript) which it then parses and displays it on your screen for you..

Later in this guide we will go into full detail on how to set up a simple web server on your computer that you can use to test and run your code.

A Goal to Work Toward

This one is relatively simple, but it's important to define it. Most people who begin their web development journey will have the goal of getting employed in a web developer job, which is a great goal!

However some might simply wish to learn more about how websites or made, or have a hobby project in mind that they would like to learn how to build. Those are great goals as well.

Regardless what your goals are it's critical that you establish a plan for how you will reach that goals.

Early on in the process you will not even be aware of everything you need to learn, so setting specific timelines might not necessarily be realistic at the beginning.

Instead try to focus on routine, decide how much time you can devote each day and each week to your learning and make sure you've done everything you need to do to ensure that time is available for you.

Regular practice is far more important than singular long sessions. For example you will learn more from practicing for one hour every day than you will doing 5 hours on Saturdays and Sundays.

The reason being that the long the gap between practice the less likely you are to retain things you have learned. Learning web development (and anything really) is all about repetition and consistent practice.

And don't forget the importance of proper sleep for solidifying what you have learned at the end of each day.

In terms of timelines, my experience from those I have worked with and talked to in the community, is that someone who dedicates themselves to learning web development full time (like a 9-5 job) will probably require at least 6 months to become proficient enough to begin applying for jobs.

Those who are only learning just a few hours a week in their spare time will require more, likely at least a full year or more.

These are by no means set in stone, everyone is different, everyone learns at different rates, so you should not hold yourself to any unrealistic deadlines that might put you at risk of burning out or giving up.

It's a marathon, not a sprint.

Once you have that plan in place then it's finally time to decide how you will go about learning.

A Resource to Learn From

One of the most difficult parts of learning web development people often find is deciding which course they should take, or which tutorial they should follow, or which project they should build.

In this guide I will provide a lot of recommendations, the vast majority of them free to use.

The most important thing to keep in mind is again, that learning to be a developer is a constant never-ending learning journey. Each tutorial you follow and project you build will teach you a little more.

Most people find that the most effective way to learn is through a combination of tutorials and self-directed projects. Start by following an online course that will guide you and hold your hand most of the way.

After completion, try to build your own project using some of the things you learned. Select that project based around something that interests you. That's a very important step to help stay motivated to see it through.

Keep the scope (size of the project) small. It's much better to start and finish a small project then to undertake a large project and quit when you realize you've taken on too much.

The more projects you build the better you will get at properly estimating the scope of the workload.

Your very first resource will be this guide. We will introduce you to the basics of HTML, CSS, and Javascript. We'll also touch on version control (git) and web servers.

Finally we'll talk in brief detail about a number of other concepts you'll want to be aware of, even if you don't understand how they work.

Web Development Basics

How does a website get created?

So how does a website get created? We'll begin by going and taking a look at one of the most poplar websites in the world: Amazon

Start by loading the home page.

Now try right clicking on any of the section titles. I've chosen "Shop by Category" but if yours appears different thats fine.

When you right click choose "Inspect" from the menu that comes up.

A menu will appear at the bottom of your screen.

This is your browser's "Developer Console". The code-looking text you are seeing is called HTML. It is a special kind of text designed to structure content.

It is one of the three primary web languages you will need to need to learn to become a web developer (along with CSS and Javascript).

What you are looking at is the complete HTML structure of the Amazon home page. This code is not designed to be hidden from you, in fact all code that runs in your browser on every site you visit is totally public and visible by using the "inspect" tool.

You will find that the browser's inspector is one of many important tools you will become familiar with while learning.

How can I build a website or web app?

The absolute easiest way for someone with no experience at all to start building their own website or web app is to use a tool called Codepen.

There are many tools like this (code editors in your browser), I have simply chosen Codepen because it's one of the simplest and easiest to get started with.

Create an account for yourself (using an account will allow you to save your work) and then click the Pen button on the left menu. You'll be taken to a screen that looks like this:

Remember we were looking at some HTMl code on the Amazon website, so let's copy it over and see it if works here! I've doing to copy and paste this from the developer console:

<h2 class="a-color-base headline truncate-1line">Shop by Category</h2>

Notice that a couple of seconds after your paste it into the HTML block, some text appears in the white section of the bottom of your screen. That is Codepen render the result of your HTML!

It looks just like it did on Amazon, even the font size is similar. The reason for that is that <h2> represents a "header" and all browsers have built-in ways of deciding how they should look. You can customize them of course, but the browser handles the defaults.

You now have an environment where you can learn and play with HTML. We'll go into more detail on HTML in a later section. For now I'd also like to introduce the other two important languages you will need to learn: CSS and Javascript.

We'll begin with Cascading Style Sheets. CSS is a special syntax used to change the way HTML looks. So as we mentioned, your browser has its own default options of how an <h2> tag should look, but with CSS we can override those default styles and make it look however we like.

In the second box on your Codepen marked as CSS let's add the following code:

h2 {
  font-size: 36px;
  color: green;
}

Notice how it changed the font size and colour of your text? That's what CSS does. The label outside the curly brackets says h2 which is called the selector. That specifically says which HTML elements we want to apply these styles to.

We'll get into CSS in much more detail in a coming section, but let's at least introduce the 3rd language you'll be learning: Javascript.

Javascript is by far the most complex of the three, it's a full fledged programming language, but believe it or not you can actually build fully complete websites with just HTML and CSS alone.

Javascript is only need to make your sites and apps interactive, and although that is an important part of the vast majority of the web these days, there are still many types of sites that need little to no Javascript.

For example a business page for a local restaurant which simply lists the menu, store hours and a contact phone number. Or a blog which is just written content and images and links between pages. Neither of those examples would require Javascript to work.

As soon as you start to want user interactivity, you are going to need it though. Let's show one of the simplest examples of Javascript at work: a button.

Add the following to your Codepen in the Javascript section:

function buyItem() {
  alert("You bought an item!");
}

What we've created there is a "function" which is basically something that performs a set of tasks. In our case the task is to show an alert banner on the screen that says "You bought an item!"

In order to actually trigger our function though, we will have to connect it to HTML. We'll need a button we can click that will "invoke" this function. Add the following code to your HTML block on codepen after the <h2> that is already there:

... <button onClick="buyItem()">Buy Item</button>

A button that says "Buy Item" will appear in your little app. try clicking on it and you should get a little pop up window that says "You bought an item!"

Honestly, that's web development in a nutshell right now. Obviously it just grows and gets bigger and more complex from here, but you now have experience using every core tool you need to build applications for the web.

All the fancy tools you hear about like React, Angular, etc ultimately just transform your code into HTML/CSS/JS before it gets sent to your browser, so if you can learn the fundamentals and learn them well then you will be in an extremely strong position to pick up the specialized tools which simply make developing apps easier and more efficient.

Easier said than done of course, but we're going to walk through everything you need to know step by step.

How can I build websites on my own computer?

There's absolutely nothing wrong with continuing to use Codepen or similar services like Codesandbox, Replit or Glitch while learning, but at some point you are probably going to want to know how to built websites using tools on your own computer.

There are many benefits to doing it this way, you'll be able to store and manage your own files, as well as share them with others. You can customize your own tools to behave how you like, and you don't need to worry about always having access to an internet connected browser to build.

Of course when you work for a real company the expectation is that you will be working on your own machine (or a company machine) so it will be expected that you know the basics of how to set up an editor.

Absolutely any text editor can be used to write HTML, CSS and JS. Even basic Notepad. The main reason we choose something like VS Code (which this tutorial will teach you to use) is that we get major benefits designed to help people who write code.

Some of the benefits include:

Project organization (see all the files for our project in one place)
Syntax highlighting (colouring text differently depending on what it does)
Intellisense (autocomplete and tooltips that tell you what your code actually does)
Error checking while you write

There are many more benefits of course, but these are just some examples of the most useful ones!

The next section will help you get VS Code installed on your machine and prepare you to setup your first web project.

In the event you don't have a computer you can install a text editor on, you can continue to use Codepen for the time being. Simply skip over the below section on setting up VS Code. You would also want to skip the section on configuring a web sever, but it's still worth reading as there are many concepts explained that are important to understand.

How do I set up a development environment on my computer?

Start by installing VS Code. You can download it here from the official site.

The benefits you will get from automatic syntax highlighting, formatting, intellisense and other programming-specific features will not only improve the quality of your code, but also your experience in writing it.

You will definitely want to download or bookmark the keyboard reference for your relevant OS:

The majority of the keyboard shortcuts in this tutorial work for both Windows and Linux, and Mac users can usually substitute the Ctrl key for the Option key.

Next we are going to create a folder called my-project. You can create this folder anywhere on your computer that you like. Once you have created it return to VS Code and select File -> Open Folder then navigate to the my-project folder you created.

Now open up the Explorer bar on the left side of VS Code. There are a few ways to do this, either press Ctrl + Shift + E or use the little papers icon:

You are now ready to create files and write code.

Before we continue any further, let's really start to dive deeper into what HTML is and teach you how to set up the shell for a standard website.

HTML

What is HTML?

HTML stands for Hypertext Markup Language and it is the building block of all content on the web, and even many web applications. It is a way to represent different types of content, for example headers, paragraphs, links, images, lists etc so that another tool (for example a browser) can display them in a way that is easy for a user to consume.

HTML is primarily concerned with the content its and not its visual style; however there is some overlap and browsers all have their own set of default styles they use when they render HTML content.

If we were to compare it to a house, HTML would be the foundation, the walls, the roof. Things like paint, trim, and even arrangement of the furniture would be someone else's job (for example the interior designer). Our job as HTML is to simply provide those building blocks to the designer.

HTML is a series of elements. An element is composed of three things:

The opening tag - Contains the element name. For example a paragraph tag is <p>. That opening tag can also have attributes inside of it which we will get to in a moment.
The content - Contains anything that goes inside the element. Can be just text, or another element, or multiple elements.
The closing tag - Tells whatever is rendering your markup that this element is finished. Looks similar to the opening tag with a slash character, so a paragraph closing tag looks like </p>. All elements have a closing tag (with a few exceptions called void elements).

So taking what we know now, a paragraph element would look like <p>Hello everyone!</p> and would render on your wep page as simply text. Here's how it looks by default in Firefox:

When we discussed the opening tag we mentioned the other important thing that an HTML elements can contain are attributes. An attribute provides additional information or identifying characteristics about that element.

A link for example which is defined by an anchor element uses the <a> tag. Of course a link is useless without specifying where you want it to go. The destination of a link is specified with the href attribute.

Attributes are always placed within the opening tag of an element, and follow the rule name="value" with the value in double quotes. So to set the href attribute on our anchor it would look like:

<a href="https://www.google.com/">Link to Google</a>

The content of the element will be the text of the link. When we place this below our paragraph tag, our page now looks like:

Click on that link will take our browser directly to Google.

Lastly let's take a look at the complete HTML content used to render these examples. The below code shows everything that is required to render out the most basic web page:

<!DOCTYPE html>

<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>My First Project</title>
  </head>

  <body>
    <p>Hello everyone!</p>
    <a href="https://www.google.com/">Link to Google</a>
  </body>
</html>

(Note that technically some of this could be removed and still rendered by the browser, when we say "minimum" we are referring to best practices of good HTML form, not the absolute minimum to get some content to appear)

You are already familiar with the <a> and <p> tags so let's look at the rest of the structure:

<!DOCTYPE html> - Simply informs the browser that you are writing valid HTML. This element used to provide more functionality than it does in modern times. Simply including it is sufficient.
<html> - This is the root element that wraps your entire page, simply specifying "this is where my HTML content can be found". The lang attribute specifies that the content we are writing is in English, this helps translation tools like Google translate know what language to start with when doing the translation.
<head> - This element wraps all the elements that provide information about your web page that don't actually contain any content that renders on the page itself.
- <meta> - This is a general tag that provides meta information about your HTML, for example above we are telling the browser that our HTML is written using the UTF-8 charset so you can expect any valid character in that set to appear. There are many other meta elements you can use as well.
- <title> - Provides the title of your web page, you can see this most commonly as the label of your browser tab. This is very important for SEO which we will get to later.
<body> - Contains the actual visible content of your site. Differentiate from the <html> element which contains both the visible body, but also the metadata in the <head>.

There is lots more to learn, but this covers the absolute fundamentals. Check out the where can I learn more section to dive deeper.

How do I use HTML?

Here we will look at how you can take your knowledge of basic HTML structure and use it to build something.

Presuming you followed the introduction about setting up VS Code, you are now ready to create your first HTML file.

Note that VS Code is not required for any of this, it's simply a text editor. If for any reason you prefer other code editors like Atom or Sublime they work just as well, and you can follow along exactly the same (just be aware the keyboard shortcuts and example images will not necessarily match up).

Begin by opening the sidebar (Ctrl + Shift + E) right clicking in it and selecting "New File". You can also press Ctrl + N to create a new file.

Our file will be called index.html. The html extension identifies that the file contains HTML, and the name index is also special too. index is the default filename a web server will look for when a browser makes a request. It's not required to name your file index.html, but if you don't then you would specifically need to tell your browser which file to get.

We'll cover this and more in the section on web servers ahead.

Once your file has been created you can copy & paste the code from the first section into it:

index.html

<!DOCTYPE html>

<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>My First Project</title>
  </head>

  <body>
    <p>Hello everyone!</p>
    <a href="https://www.google.com/">Link to Google</a>
  </body>
</html>

We were told this is valid HTML, but is there a way we can be sure? There is!

If you would like to validate your HTML to ensure it meets modern standards you can use W3's HTML validation tool.

Let's paste our code into it to be sure:

Fantastic, we've verified everything is good to go.

Before we look at getting our code running in the browser, let's wrap up our discussion on HTML. Below are some great free resources for taking your HTML learning further, and I've also mentioned a few key topics you will want to learn more about as you build your skills.

What else do I need to know about HTML?

Semantic HTML is all about using tags that properly describe your content. For example using an <h1> for a header rather than putting your text in a <div> tag. This allows things like web screen readers to better understand that the text is meaning to tell the user. More information below:

Accessibility and Semantic HTML

SEO stands for Search Engine Optimization and it is the field of creating your site in such a way that search engines (mostly Google) can scan them automatically and increase their ranking when users search for relevant terms. Things like semantic tags, page speed, colour contrast, text content and much more are considered when deciding your SEO score. More information below:

SEO

What are the best free resources for learning HTML?

There are tons of great resources on learning HTML out there, and they're free!

MDN for a complete resource that covers everything you need to know about HTML
The Odin Project HTML foundations portion of the full stack curriculum
Codeacademy for a more academic "course style" approach to learning HTML
HTML Validator - The HTML markup validation service
HTML Can do that? - A great blog post on dev.to showing some amazing browser native features that can be accomplished in pure HTML without CSS or Javascript

Web Servers

Now we are ready to get back to the HTML code we wrote in the previous section. We would like to view the contents of that HTML file in the browser.

But how do we do that? Where exactly does the browser get the HTML code from to display it? We need a way to test and view the HTML we have written.

This section will teach you everything you need to know about how that code gets up and running in your browser. This is not specific to getting it running on our machine, the concepts you are going to learn are exactly the same concepts real web servers use that let us access real websites and web apps from the internet every day!

What is a Web Server?

(Note that although I would consider this sub-section to be extremely important for every aspiring web developer to understand, it's not required understanding to complete this tutorial. If you would prefer to skip understanding how a web page gets to to your browser, you can jump ahead to How do I run a web server?

That said I would highly recommend you at least briefly read through it if only to become familiar with the basic terminology.)

A web server is a fancy name for software on a computer that is connected to a network and has the ability to send content over a protocol called HTTP. Most commonly this is an HTML page, but a web server can serve up any type of data you like.

It can be something running on a farm of supercomputers across the ocean, handling millions of requests per second, or it can be something running on your own machine to practice writing HTML.

Our tutorial, to no one's surprise, will focus on the latter.

Before we get into how to run a web server, let's start with why we need one. A web server exists to provide data to your browser, so that you (the user) can request different web pages and data. The HTML file we created in the previous section is not really any different from a web page you would request on the internet. So how does a web server know which file to send to the browser?

What is a web request?

Domain

When you enter a URL into your browser and press enter, it makes what's called a GET request to a server. Let's pretend that address is https://www.google.com/. Presumably we are trying to get the Google homepage.

How does it know which server, of all the servers in the world, to get the Google homepage from?

A server is identified the same as any other computer, with an IP address. Notice we don't see any IP address in http://www.google.com/.

When a domain name is used, the IP address is acquired through a system called DNS which transforms the domain name into an IP address. There are DNS servers all over the world with giant address books of all the domain names and the IP addresses they are registered to.

Port

So now that we have identified the Google server, your browser makes an HTTP GET request to that server. In addition to the IP address, a request also always includes a port number which helps to identify which process (think which piece of software) that request should go to.

Imagine you are mailing a letter. Think of the IP address as the house your letter is addressed to, and the port as the name of the person inside the house that the letter is for.

HTTP requests by default use port 80, so you don't have to specify it manually. You can still specify the port manually with a colon and the port number. So a request to http://www.google.com/ is equivalent to a request to http://www.google.com:80/. Try it yourself! Anything other than port 80 should give you a timeout error because no process on the Google server is listening on that port (or if they are, they aren't listening for your type of request).

Path

The final part of the URL that is relevant to our example is the path of the data we are requesting. A similar term you might also hear in this context is the route. That's the part that tells the server what data or page specifically we are looking for.

In our example it is the / on the end of the URL which is often referred to as the root directory. Think of it similar to filers and folders on your computer. It represents a path on a file system where you cannot go "up" any further.

In the context of web servers, it typically represents the directory the server is running in. This way it provides a layer of security. If you run a web server in c:\Users\my-server then requests made it it will only be able to access files within that my-server directory (and directories inside of it). Users would not have access to anything in the c:\users directory or above.

Let's give an example. I create a directory (folder) called my-server on my computer, and run my web server software inside that directory. Let's say the IP address of my computer is 127.0.0.1 (this is a actually a special IP that always refers to your own machine, so this is a valid way to make requests to your own machine).

If I enter http://127.0.0.1/ into my browser, the web server will go looking in the root directory, which means the directory the server software was started in. In our example that is the my-server directory. By default most web servers will look first for a file called index.html and send it back to whoever is requesting (our browser in this example). If it finds that file, the browser will render it, if not, you'll likely get an error.

What if we make a request next to http://127.0.0.1/something-else ?

Our web server will now look for a directory called something-else inside of the root directory and follow the same behavior: look for an index.html file. This is how we can create routes on our website. Let's say we put relevant info about our business into an index.html file in our root directory. Then we create another index.html file with our business hours and phone # and put it in a folder named contact-us.

We can now access the main page at http://127.0.0.1/ and our contact info at http://127.0.0.1/contact-us

How do I run a web server?

(This section will cover the basics of how to run a web server on your own computer so that you may test your website/app while working on it. If you are looking for how to actually put your website/app somewhere so it can be accessed publicly by anyone, check out How do I put my website on the internet)

(Furthermore, if you are looking for more information about how to a production web server for a real world website or application that needs to handle both large amounts of traffic and guarantees about service & uptime, then you will want to reach more about a professional web server like Nginx rather than the more hobby-level approaches described below)

We've covered the basics you need to understand about what a web server is an what it does. At this point you should be able to comfortably run one yourself and explain in simple terms what it is doing.

Let's now look at how to run a web server yourself and get the HTML you wrote in the previous section to render in your browser on your machine.

In this section we are going to learn how to add some extensions to the tools you are already using (VS Code or Chrome) to serve up your web content and view it in the browser.

If you would like to learn how to run your own web server on your own machine (which is surprisingly easy) I have written a separate tutorial for this: (Note that this tutorial presumes you have a basic familiarity with the terminal on your machine, at least enough to be able to copy and paste commands)

Understanding the Modern Web Stack: Running a Local Web Server

If that sounds a bit beyond your understanding, then no problem at all! For this tutorial we are going to focus on the absolutely simplest way to simply get your content running in your browser. We will provide three options, choose the one that fits your situation best:

Option 1: Choose this option if you have installed VS Code on your machine
Option 2: Choose this option if you have installed Node.js on your machine, you understand how to install an NPM package and you want more control over your server than the VS Code extension provides.
Option 3: Choose this option if you are using the web version of VS Code, or any editor other than VS Code, or are on an OS besides Windows/Linux/Mac (a Chromebook for example)
Option 4: Choose this option if you are on a public computer like a school or library where you do not have permission to install software

Option 1: VS Code Extension

We are going to install the VS Code Live Server extension. Start by clicking the Extensions icon or pressing Ctrl + Shift + X.

Next search for Live Server or the unique id of the extension: ritwickdey.liveserver and click it, then click install:

(This is a reminder if you are using the web version of VS Code that this extension does not work, please use Option 2)

Once installed, a Go Live icon will appear on the blue bar at the lower right corner of VS Code:

(Another alternative option is to simply right click on index.html and select "Open With Live Server" which is a new option that will appear when the extension is installed)

Regardless of which method you choose, you will be able to view your web page at whatever port the server decides, the default URL being http://localhost:5500.

If you have any difficulty you can refer to the full documentation here.

Option 2: HTTP Server

If you have Node.js installed on your machine and are familiar with the basics of the command line, you can get a simple web server up and running in a few seconds.

Navigate to the folder that has the web files you want to host (usually that means the directory with your index.html file) and run this command:

npx http-server . -p 8080

(Note there is a period character before the -p which indicates "this directory")

That will start the server running in the directory you are in and serve the files. The -p indicates the port which I've set as 8080 which is the default, but you can run your server on any port you choose.

You will be able to view your website's index.html file by visiting this URL:

http://localhost:8080

Obviously change the 8080 if you decide to use a different port. And remember that localhost is simply a special shorthand name for the IP address 127.0.0.1 also known as the home address.

When you are done with your server simply press ctrl+c to close it.

Option 3: Chrome Extension

(Reminder that you only need to choose one option, if you already have your HTML running in the browser then skip ahead)

If you are using the web version of VS Code or another editor and just want to get your code up in the browser, another easy option is the Chrome browser extension. This option requires you to use the Chrome browser.

(If you cannot use the Chrome browser, or would prefer to run your own on your own machine via the command line then jump ahead to option 3)

Start by installing the extension called Web Server for Chrome in your Chrome browser

Next navigate to chrome://apps/ in your Chrome browser. There you should see and be able to click on your new Chrome app.

In the menu that appears select "CHOOSE FOLDER" and navigate to your project, the /my-project directory that contains your index.html file inside of it that you created in the first section.

Finally go to the URL that the extension shows in the menu, by default it is . If all goes well you will see your page!

Option 4: No server at all

(Reminder that you only need to choose one option, if you already have your HTML running in the browser then skip ahead)

If you are unable for any reason to install software on your machine (or even extensions in your browser) then we still have an option for being able to write code and test it in the browser.

The title of "no server at all" is a bit of a misnomer as there is still a server; it's just being run by someone else and you are simply providing the code to be served.

This option would technically work even for a device like a phone or tablet.  Though I would not recommend trying to code on a small touch device; I will say that maybe a tablet with a big enough screen and a bluetooth keyboard you might be able to get yourself a viable environment for practicing (if you have no other alternatives).

In the first section of this tutorial we use Codepen to demonstrate how to quickly write some HTML, CSS and Javascript. Unfortunately Codepen does not provide an easy free way to mimic the file structure of a real project.

For this example we are going to use Replit. It's an absolutely fantastic tool that supports browser-based coding environments in all kinds of programming languages. It does require you to sign up however, but doing so will give you the big benefit of being able to save your projects, share them with others, and come back to them later.

Start by navigating to https://replit.com/ and creating an account. Once you are logged in you will see a Create option with a + button:

Click that button and you will have a number of templates you can choose from the dropdown. Select HTML, CSS, JS:

At this point you will have your own web editor that looks quite similar to VS Code. On the left are all your files generated and setup for you, with default templates provided. GO ahead and replace the content of index.html with our example and press the Run button. You will see the same output that we have on our local server:

You now have an environment you can continue to follow along and write code in directly in your browser.

In future tutorials (and as you learn more and your projects get more complex) you will likely begin to run into limitations of what you can do with this tool, but if your goal is to learn the fundamentals of HTML, CSS and JS then this tool can easily take you the majority of the way if you need it to.

What else do I need to know about web servers?

Some topics related to web servers that are worth learning about as you build your skills:

What are the best free resources for learning web servers?

There are tons of great resources on learning HTML out there, and they're free!

MDN again is your best friend for learning the fundamentals.
When you are ready to learn how to configure your own real web server then I would suggest a Digital Ocean Droplet. That page might look overwhelming, but it's just a fancy name for remote access to one of their computers you can put your web content on, and let other people access. They have a great tutorial on setting up one of the most popular web servers called Nginx (pronounced engine-X).

CSS

At this stage of the tutorial we are operating under the assumption that you were able to successfully get one of the three options for running a local server up and running, and you have the ability to view your HTML in the browser.

In this section of the tutorial we will begin to learn about how we can style our content with CSS.

What is CSS?

CSS stands for Cascading Style Sheets and it represents a syntax for describing for HTML content should look. Someone who is well versed in CSS can take the same HTML content and make two completely different looking pages out of it, such that you might hardly recognize it comes from the same HTML source.

A great resource to demonstrate this idea is CSS Zen Garden. On the right side of the page are links you can click on to apply different CSS stylesheets to the site. The HTML content doesn't change, but with the power of CSS each design feels like an entirely different site!

Before we discuss the concepts, you should be familiar with the most basic syntax. Let's begin with a simple example of some CSS, and explain what it is and how it works:

p {
  font-size: 18px;
  color: green;
}

Take a look at what happens when this CSS is applied to the HTML from the example in our previous section:

(Note you don't need to create any files or do any coding along with this section, we are just using this as an example to learn the syntax, you will learn how to create your own .css files and add them to our project in the next section)

It should be fairly straightforward what has happened based on the example above. Our "Hello everyone!" text has been formatted with the styles we described in the CSS.

These style descriptions like color: green are calledCSS properties.

Properties

CSS properties are combination of a name (e.g. color) and value (e.g. green) separated by a colon that describe what styles to apply to the elements you have selected.

There are more properties out there than any one person is likely going to learn in an entire career. Fortunately you do not need to learn them all. Many of them are either not supported anymore, or their use is strongly discouraged.

Focus on trying to learn the core properties well rather than trying to learn a lot of different properties. Like many things in life you'll find than a small number of them end up comprising the vast majority of usage.

Here is a hand picked list of the most common properties you will encounter, and the most important ones to learn (the rest can be picked up as you encounter them):

display
position
box-sizing
width (including min-width and max-width)
height (including min-height and max-height)
margin
padding
border
color
background-color
background-image
font-family
font-style
font-weight
z-index

MDN also has a slightly larger opinionated list that you can use as a quick reference to learn about these and more.

Selectors, Combinators and Specificity

In our CSS example above, why did the styles apply to the <p> element and not the <a> element? The reason is CSS selectors.

In our example we are using a tag selector for the p tag. You can see that before the opening of the first curly brace p {. This will apply all styles in that block (between the curly braces) to all <p> elements on the page.

Selectors are one of the most basic features of CSS, they describe which specific elements that your styles should be applied to; but what happens if multiple selectors try and apply the same CSS property to an element?

The default is that the styles that come later would overwrite. So in the example:

p {
  color: green;
}

p {
  color: red;
}

All <p> elements on the page with this CSS would render as red. This is where the concept of the cascade in Cascading Style Sheets comes from.

However this only applies if the selectors are exactly the same level of specificity. They are in this example because they are identical (each one is a tag selector for <p> tags.) There are many different kinds of selectors though, and ones that are considered more specific would take precedence even if they came earlier on the sheet.

Specificity is what determines which property gets applied to an element in a scenario where a property is set from more than one selector.

Let's take a look at each possible level of selector in order of specificity. This means that each one of the examples below is more specific than the last, so it would override those styles even if they appeared first on the style sheet:

Global Selector

* {
  color: green;
}

The asterisk is called the global selector and it targets every single element in your HTML. There are not many cases when you want to apply a style to every element in your page, so it doesn't get used too often.

It can be handy for debugging, for example to place a border around every element on your page to help quickly get a visual indication of the size of eah element.

Tag Selector

p {
  color: green;
}

div {
  width: 100px;
}

The above example shows two tag selectors. It will apply a green color to all <p> elements on the page and a height of 100px to all <div> elements.

If you used the * selector to set the color of all elements, this would overwrite that color value for p and div tags.

Class selector

p {
  color: red;
}

.example-class {
  color: green;
}

<p class="example-class">I will be green</p>

The class selector uses a period . before the term of your selector and will target all HTML elements with that term on the class attribute. In our example the term is example-class but that term can be anything you like.

The class selector is more specific than the tag selector, so green is applied.

ID Selector

.example-class {
  color: red;
}

#example-id {
  color: green;
}

<p class="example-class" id="example-id">I will be green</p>

The ID selector uses a pound symbol # before the term of your selector and will target all HTML elements with that term on the ID attribute. In our example the term is example-id but that term can be anything you like.

The ID selector is more specific than the class selector, so green is applied.

Inline styles

#example-id {
  color: red;
}

<p id="example-id" style="color: green;">I will be green</p>

This is not a selector per-se, since it gets set directly on your HTML element as an attribute rather than within your CSS.

HTML elements all have an attribute called style that allows you to set CSS properties directly. These properties are so specific they will override even ID selectors.

It can be convenient to quickly set styles directly on the element, however you should be cautious as it is often considered a bad practice. The reason being that they are difficult to override elsewhere, and that they exist outside of your stylesheets. Having styles in more than one location can make it difficult to reason about how styles are being applied.

!important

#example-id {
  color: red !important;
}

<p id="example-id" style="color: green;">I will be red</p>

Often called the nuclear option when all else fails and you can't figure out why your styles are being overwritten, CSS supports a very powerful option called !important that essentially says this style cannot be overwritten.

Notice in our example it even takes precedence over inline styles.

!important is nearly always considered very bad practice for reasons that should be obvious. The entire concept behind cascading style sheets is that styles are designed to be overwritten at different levels of specificity. !important breaks that model entirely.

To give an example: consider a scenario where your downloaded a library of "cool buttons" that looked fantastic and fit the needs exactly for your clients product. Your client says "they're perfect, we just need them to be blue instead of green". You find out to your dismay the button library author used background-color: green !important; when styling them. There are ways around this, but your job has just become significantly harder than it needed to be.

In summary, avoid !important entirely unless you are absolutely sure you know what you're doing.

Multiple Selectors

You can combine styles together and apply them to more than one selector with a comma. For example:

h1,
h2 {
  font-weight: bold;
}

Will apply the bold style to all h1 and h2 elements on your page.

Combinators

The last aspect of CSS we need to address before going into our example is combinators.

Combinators work in tandem with selectors. They allow you to chain multiple selectors together to get more specific. They are applied most often when dealing with nested elements. For example:

div {
  background-color: black;
}

p {
  color: black;
}

div p {
  color: white;
}

div > p {
  font-size: 20px;
}

<p>I am black text</p>
<div>
  <p>I am white text and 20px size</p>
  <section>
    <p>I am white text but regular size</p>
  </section>
</div>

There are two combinators in this example:

div p
div > p

The first one uses the space combinator, a space between selectors means all children of elements targeted by the first selector. So styles in div p would apply to every <p> element that is a child of a <div> element.

In our example this is necessary. Since we gave our <div> a black background, the text in the <p> elements would be invisible if we did not set it to white.

The second one uses the angle bracket >, it means direct child of this element, so any <p> element that is a direct child of a <div> element will be targeted with the styles we write here.

Notice the word direct in this example only applies to <p> elements that have a <div> as a direct parent. Our first <p> has no parent and our third <p> element has a <section> as a parent, so only the second one has the font-size: 20px style applied.

You can find a great quick reference for selectors and combinators here.

Now that we have covered all of the basics, let's take a look at a real world example that we can continue to iterate on for the remainder of this tutorial!

How do I use CSS?

There are three primary ways to apply CSS to your HTML:

Using the <style> tag
Using the style HTML attribute
Using a .css file

Option #3 is by far the most common and nearly always the best practice, however since we are here to learn we'll quickly begin by showing an example of options #1 and #2:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>Example</title>
    <style>
      p {
        color: green;
      }
    </style>
  </head>

  <body>
    <p>I am green</p>
    <p style="color: red;">I am red</p>
  </body>
</html>

The above example demonstrates the use of the <style> tag in the header applying a green colour to all <p> elements.

Within the HTML itself the second <p> element has a style attribute which is setting the color property to red. This has more specificity than the tag selector, so the second paragraph element will render as red.

To demonstrate the third and most common option (a .css file) we are going to construct a more complex example and take the time to break each selector and property down to get some experience working with something that looks more like a stylesheet you might encounter in a real world site.

Begin by creating an index.html file in the root directory of a project (you can use the one you already created it you are following along) and paste the following code into it:

index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>My Animal Blog</title>
    <link rel="stylesheet" href="style.css" />
  </head>

  <body>
    <header>
      <h1>My Favourite Animals</h1>
      <h2>Enjoy these furry friends</h2>
    </header>
    <main>
      <article class="card">
        <img
          src="https://res.cloudinary.com/dqse2txyi/image/upload/v1657421273/blogs/intro-to-web-development/cat_k4fcww.png"
          alt="Cat Playing Chess"
        />
        <div class="card__container">
          <h3>Chess Cat</h3>
          <p>He's planning his next move.</p>
        </div>
      </article>
    </main>
    <footer>
      <p>&copy; 2022</p>
    </footer>
  </body>
</html>

The key line to look at in the above example is this one:

<link rel="stylesheet" href="style.css" />

This element tells your browser that you want to link an external stylesheet, and the filename is style.css. It looks in the root directory by default. As of right now, you do not have a style.css file in the root directory of your project.

Without any CSS at all, this page looks like:

Not too bad, but we can make it look a lot better. Create a new file in the root directory next to index.html and call it style.css.

Leave it empty to start. We're going to add some CSS little by little to clean it up a bit and give you an idea how to craft an extremely simple, but still modern and responsive page design. We'll also explain each piece in detail as we go.

So feel free to add each of these pieces to your style.css file as you read them, one after the other (it's not necessary though if you would prefer to just read, we'll post the full file contents at the end).

So let's begin styling our animal blog:

body {
  color: #333333;
  font-family: Helvetica, sans-serif;
  margin: auto;
  max-width: 800px;
}

The body in HTML defines where all the content is. The body is inside the <html> element, which by default will span the entire width of the page.

Let's look at each property in the body tag selector, to see what they do and why we chose them.

max-width: 800px;

Typically in many modern web pages you'll find it common for the actual content not to span the entire width of a user's screen. People often use very wide monitors these days and it can be challenging to read content, particularly text that spans from one edge to the other.

For an easy example of this go to Google and search. You'll see that the search results only span a small portion of the screen before the text wraps (on my 1920px monitor the results are 600px wide).

We are doing something similar for our blog. Body content can only be a maximum of 800px wide, and will shrink down automatically on smaller screens.

margin-auto;

With only max-width our content will still be left justified on the page. We would like it to be centred. That's what having an auto margin does in this context.

On the top and bottom it will do nothing since the default height of <html> is only the height of our content; however the width is the width of thr full screen. The difference between that screen width and the 800px content will automatically become margin on the left and right.

This has the result of placing the content in the centre.

color: #333333;

Pure black text on a pure white background creates a fairly stark contrast. It's not terrible, but it's fairly common to use a not-quite-black color instead of black for text on a white background.

Here we have chosen a hex colour that is close to black, but not quite.

font-family: Helvetica, sans-serif;

After setting the color we are also setting the default font of our site. We've chosen Helvetica because it is one of the safe web fonts that are available on the majority of web browsers and operating systems.

The font-family property allows as many font names as you like separated by commas. It will chose the first one it finds from left to right that exists on a user's system. The final option sans-serif simply says worst case give me any sans-serif font you have.

Here's a look at what we have so far with our body styles:

main {
  margin-bottom: 40px;
}

header,
footer {
  text-align: center;
}

h1 {
  font-family: Arial, sans-serif;
}

h2 {
  font-style: italic;
  font-weight: lighter;
}

margin-bottom: 40px;

We added this to give a bit of space at the bottom of all the main content before the footer, so the copyright symbol doesn't press up directly against the text.

text-align: center;

As described, this centre aligns the text in the header and footer. Since the header includes both an <h1> and <h2> element each one will automatically inherit that property.

font-family, font-style, font-weight

Each property sets the font properties as described.

Similar to the previous, but now our fonts are updated and our texts are centred:

.card {
  width: 350px;
  margin: auto;
  box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2);
  border-radius: 10px;
  overflow: hidden;
}

width: 350px;

We are going to set our card elements to be 350px wide. This is simply a value I have chosen based on common screen sizes. Very few modern smartphones have a width smaller than 375px, so setting our cards to 350px means they will fit nicely (without having to be shrink) on nearly every device.

margin: auto;

Used again for centring, this time to centre the card inside the <body>. Since the card is 350px and the body is 800px this will set margins on either side to fill the difference and centre the element.

box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2);

Easily the most complex of all our properties, but a good example to break down because it will not be uncommon to encounter this type of property in your CSS journey.

First the property itself, box-shadow creates a shadow like effect around the edge of an element. The thickness of it depends on some numeric values. In this case it is the four numeric values you see 0 4px 8px 0

Whenever you see a CSS property with four consecutive numeric values it typically refers to the sides of a box in clockwise order from the top. This image from MDN illustrates:

So in our example, there is no shadow on the top, 4px on the right, 8px on the bottom and no shadow on the left. In CSS when using the number 0 it is standard to leave off the unit.

The final fifth value on box shadow indicates the colour. For an rgba colour 0, 0, 0 is black and the final value is the alpha; think of as the opacity. That's what makes the shadow look more natural and diffuse rather than a solid black at 0.2.

If you managed to follow this one, well done! If not, don't feel bad about skipping over it. This type of syntax will grow on you naturally as you encounter it more often.

border-radius: 10px;

A border radius will give us rounded corners. The larger the value the more rounded. It's common to use this property to create circles by giving a radius of 50%. For our case 10px give a nice rounded edge to the card.

overflow: hidden;

This is necessary to keep the image from going over our rounded corners. Since the image has rectangular images and sits above the card <article> it would hide the rounded corners of the card.

Hiding the card's overflow keeps the image inside the card and the edges visibly round.

.card > img {
  width: 100%;
  height: auto;
}

Setting the width of our image (using the direct child combinator) to 100% will stretch it to the edge of its part, the 350px card. Setting the height to auto ensures that the aspect ratio remains correct.

.card__container {
  padding: 16px;
}

For our class selector here we have used a common practice to help with CSS scoping. Imagine we had just called this class container. It's a pretty common term, what if we have a container somewhere else on the site? The styles of that container class would overlap and mix unintentionally with these ones.

I've chosen to prefix the class name with the name of the parent card with two underscores between. This is a common practice based on a methodology called BEM or block-element-modifier.

Essentially it's just a way of naming things to avoid unintentional collisions between unrelated elements. There are many different ways of doing this, you should choose whatever feels right to you as you gain more experience.

The property itself padding: 16px will create a buffer of space inside the border of the element. This keeps the text from pressing up against the edge of the box border, and makes things look a lot cleaner.

Now finally with all properties applied, or styling for this example is done. Take a look:

What else do I need to know about CSS?

The "cascade" is how CSS decides which styles actually get applied to an element when multiple different rules are applying values to the same property.

Cascade & inheritance

The box model describes the way every element in CSS is rendered on the page and all the factors (padding, margin, border) etc that go into deciding its size and how it flows with the rest of the content.

The Box Model

Flexbox is one of the most popular ways of laying out content on a web page. Modern browsers will support Grid which is even more powerful for full 2D layouts, but any modern CSS developer must understand Flexbox when contributing to web projects professionally:

Flexbox

Media queries are used to change which CSS styles are applied depending on what media the user is using to view it. Most commonly used to change styles for mobile devices.

Media Queries

There are many different units in CSS (px, em, rem, etc) and it's important to understand the difference between them.

CSS Units DEBUGGING CSS

What are the best free resources for learning CSS?

The Odin Project - The Odin Project CSS foundations curriculum
CSS Diner - A fun and effective way of learning CSS selectors
Flexbox Froggy - A fun and effective way of learning Flexbox
Flexbox Zombies - Another fun way to learn Flexbox
Knights of the Flexbox Table - Another fun way to learn Flexbox
Grid Garden - A fun way to learn CSS Grid
Complete Guide to CSS Flexbox
Complete Guide to CSS Grid
Specificity Cheatsheet
Selectors Reference
Properties Reference

Javascript

What is Javascript?

Javascript is a programming language, originally developer to run code in web browsers to make pages more interactive. Since then its potential usages have expanded significantly, however for the purposes of this tutorial, web page manipulation is the use we are going to focus on.

How do I use Javascript?

There are a lot of ways to use Javascript. If you just want to write your very first code you can do so in a matter of seconds. THe quickest and easiest place to write and run Javascript code is directly in your web browser.

Hit the F12 button to open up the browser's development console (alternatively you can look for a "Dev Tools" option in your browser's settings menu).

It will open be an attached bar, on either the bottom or the right side of the browser window (depending on the browser). You will see a number of tabs across the top, these values will vary between browsers, but all the major ones (Edge / Firefox / Chrome) will have a tab called console as highlighted in the below screenshot.

Within the browser's console you can type and run any code you like. We'll begin by running the simplest possible Javascript code, the traditional "hello world!" program. Type the following into your browser's console and hit enter:

console.log("hello world!");

Your output should match what you see in the screenshot above. Below the code the text "hello world!" is printed out. Your code asked for the browser's console to use its log method and print out the "hello world!" string (a string is a sequence of text characters).

Strings are a primitive value in Javascript. A type of data that is built-in and supported by the language. Another example is the number data type. You can read more about primitive data types here if you like.

Now as you can probably imagine, very few people write their Javascript directly in the browser console. For many reasons, but the simplest of which is that the code is gone as soon as you close your browser. We'd like to be able to write some Javascript that sticks around! To do that we need the HTML <script> tag.

Before we return to our code from the previous section, let's use the simplest example we can. Create an index.html file that looks like the following:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>My First Javascript Project</title>
    <script>
      console.log("hello world!");
    </script>
  </head>
  <body>
    <p>My Javascript Test Page</p>
  </body>
</html>

Notice that we have included some Javascript code in the <script> tag? Try opening up your page on your local web server and checking your result. You should see the "My Javascript Test Page" text from the body as expected, but if you open your browser console again you will see your message.

Just as before, using the console.log function we have instructed out browser to print the "hello world!" string in the console.

Great, but there's still one final step. Similar to CSS, although originally the standard was to write Javascript code directly in your HTML, at some point everyone realized that it made more sense to separate it into its own file. Let's look at how to load a Javascript file into our site.

Begin by creating a script.js file (again the filename doesn't matter, you can call it anything you like as long as it ends with a .js extension):

script.js

console.log("hello world!");

And then update your HTML file:

index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>My First Javascript Project</title>
    <script src="script.js"></script>
  </head>
  <body>
    <p>My Javascript Test Page</p>
  </body>
</html>

Notice how our <script> tag changed:

<script src="script.js"></script>

The src attribute is the filename and path of the file that you want to load.

Save both files and refresh your index.html page. You'll see once again that "hello world!" appears in the console, but this time the code is being loaded in from the .js file.

At this point you now have the knowledge of how to set up a working Javascript environment and run any code you like. Of course this is only the tip of the iceberg, the ocean of JS is very deep, but the wonderful thing is that you can even as a beginner you can accomplish a lot with just a little.

Teaching the Javascript language itself is beyond the scope of this tutorial (I would highly recommend MDN's tutorial as usual for that) but we will take the time to explore just a little bit deeper into some of the most common use cases.

One of the main usages of Javascript is for manipulating the DOM. The DOM is a term you will come across frequently in web development and it can be a little confusing at first, but we'll make sure to describe exactly what it is in the next section, and how you can use that knowledge to make your web pages more interactive.

The DOM

What is the DOM?

The DOM stands for Document Object Model and most simply put it just means the actual structure of the page built by your browser when following the instructions described by your HTML.

Think of your HTML file as the blueprints of a house, and the DOM as the house itself.

You can use a single set of blueprints (HTML) to build as many houses (pages in the browser) as you want. Maybe in one of those houses you might choose to change the colour of the exterior paint from white to blue. This change of paint colour would be analogous to what Javascript does.

When you change the paint colour of your house you change it on the house itself, you don't necessarily update the blueprints to say "all houses must be blue." Similarly, when you use Javascript to change the background colour of your page, you are changing it on the DOM and it's updating the background colour of the page in your browser. It's not changing the HTML file itself.

The DOM is represented in the browser as a tree structure. Don't worry if that looks confusing, you've already used the structure before when writing your HTML. Your elements have a hierarchy and elements can be nested inside of other elements. This represents the tree structure. All elements (except for the <html> element) have a parent element.

How do I view the DOM?

Browsers make it very easy to take a look at the DOM directly. Open up your web page again and right click on the "My Javascript Test Page" text. You will se an option to Inspect. This option is available on all browsers.

Inspecting the element will open up the dev tools console, but this time it will default to the DOM tab (it may be called "Elements" or "Inspector" depending on your browser.) Here you can see the DOM structure that has been built by your browser based on the instructions provided by the HTML.

With a simple page it is likely that the DOM structure will look identical to your HTML file. This isn't absolutely necessary though. We could have some Javascript that changed the DOM structure after it has built so that it no longer looks the same. Let's try that now.

Click on the Console tab of your browser's dev tools like we did before so that we can write some Javascript. This time, instead of using the console.log function we are going to call a different function.

All browsers provide Javascript with an interface for interacting with the DOM through code. You can change, add, remove, clone, and generally just do almost anything you can think of with the elements on the page.

First type the following code into the browser console and press enter:

document.querySelector("p");

When you hit enter you will see the result of your code directly below, it looks like a little <p> element.

Let's break this line down and see exactly what it's doing:

document - This is a Javascript variable provided by the browser. It represents a reference to the page you are currently looking at (for example in your current tab). You can use it to find out all kinds of information, for example the width of the user's page, the current URL, and many more. In this example we are using it so we know which specific page to search for the element we want to change.
querySelector - Is a function that exists on the document. It's an incredibly powerful function that allows you to use CSS Selectors that we have already learned about to target particular elements on our page with Javascript. It's great because it lets you leverage skills you already learned in CSS and apply them to Javascript.
"p" - in this context "p" is a Javascript string that represents our CSS selector. As you may remember, just p on its own is a tag selector. Our goal is to select the <p> element on our page. Although this is a very simple one, you can use any kind of CSS Selector you like. For example document.querySelector("#example") is a valid function that would get a reference to any element with id="example" that exists in the DOM.

So when you combine the three of these together you get a Javascript function that searches the DOM for a <p> element and returns it to you (if there is more than one <p> element on the page it will return the first one it finds, starting from the top and moving down the tree).

Once you have that reference you have the ability to modify it. As we mentioned before we just want to change the content that is inside of the tag, the text that says "My Javascript Test Page".

When you ran the querySelector function in your console you got that little <p> element returned on the next line. If you haven't done so already click on it. You might be overwhelmed at first by try not to be. There are far more options and bits of information on a DOM node than the average developer will ever need. Just let them wash over you like background noise.

Look for one called innerText that should have a string version of the text that we put inside of our <p> element. This is the one we are going to update to change our page.

(In case you are wondering why we chose innerText instead of innerHTML, which looks exactly the same at a glance, the answer is that if you are just working with text (like we are) rather than adding new HTML, then innerText is safer (though both would technically work). More information here if you are curious)

So now that we know the element that we want to change, how do we change it? When we run our querySelector we get that <p> element back, but we don't know how to change it yet.

There are a number of options available, but this would be a good opportunity to introduce the concept of variables. You can use variables to store information in your program that you will need at a later time.

Let's demonstrate how to save a reference to our <p> DOM node in a variable:

var myParagraph = document.querySelector("p");

This is the same code as before, except this time we have declared a variable that we've named myParagraph and set it equal to the paragraph node in the DOM. This gives us an easy way to reference and make changes to that node.

To bring it all together, the code to update our paragraph node on our page looks like this.

var myParagraph = document.querySelector("p");
myParagraph.innerText = "My Updated Javascript Test Page";

(If you're wondering about the name of the myParagraph variable, Javascript uses a naming convention called camelCase. This is not mandatory, but is considered a best practice.)

You can run it in the console yourself to see:

Now that we've seen how we can interact with the DOM, let's move this code over to our script.js file so that it runs every time we load the page. With this in place you will essentially never see the "My Javascript Test Page" text, since the script will run immediately on page load and replace it with the "My Updated Javascript Test Page" text.

There are two necessary steps. First, update your script.js file:

script.js

var myParagraph = document.querySelector("p");
myParagraph.innerText = "My Updated Javascript Test Page";

And a reminder that your index.html should look like:

index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>My First Javascript Project</title>
    <script src="script.js"></script>
  </head>
  <body>
    <p>My Javascript Test Page</p>
  </body>
</html>

If you load the page now, even though this is the same code we used in the console, you'll notice that it doesn't work! Why not?

The reason it doesn't work in this case is because of the oder we are running it in. In the console we were running our document.querySelector function in the console after the page had finished loading.

If you look at our index.html file above, imagine you are the browser parsing the file from the beginning and working your way down. You will get to the <script> element and run that .js file before you've even reached and created the <p> element. So the querySelector runs, finds nothing, and then its done its job. Nothing is telling it to run again after the page has loaded.

There is a very easy fix for this, all we need to do is instruct out <script> element to hold off and wait to run that code until the page has finished loading. We use the defer attribute for this. Update your <script> element to:

<script defer src="script.js"></script>

And try refreshing the page again., you should see "My Updated Javascript Page" as expected as soon as the page is loaded.

At this point you now have a very basic familiarity with Javascript, including how to run your code and some of the ways it can be used to manipulate DOM nodes on your page (remember that _DOM nodes is just the term used to describe HTML elements that have been built into a page in the browser)_

The last section in our Javascript introduction will go through an example of using Javascript to solve a real-world problem you might encounter.

We'll continue where we left off at the end of the CSS section with our "Animal Blog" and the cool little card that we created.

How do I add Javascript to a website?

Let's say we've been tasked with adding some new functionality to our animal blog. We want people to be able to "like" the images of cats that we are adding to our blog. A user need to be able to interact and make a change to our element, so we recognize immediately that it's a perfect use case for Javascript.

First we will pull up our previous example. In case you don't have it handy, we need three files:

index.html
style.css
script.js

index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>My Animal Blog</title>
    <link rel="stylesheet" href="style.css" />
    <script defer src="script.js"></script>
  </head>

  <body>
    <header>
      <h1>My Favourite Animals</h1>
      <h2>Enjoy these furry friends</h2>
    </header>
    <main>
      <article class="card">
        <img
          src="https://res.cloudinary.com/dqse2txyi/image/upload/v1657421273/blogs/intro-to-web-development/cat_k4fcww.png"
          alt="Cat Playing Chess"
        />
        <div class="card__container">
          <h3>Chess Cat</h3>
          <p>He's planning his next move.</p>
          <button onCLick="likeButton()">Like</button>
        </div>
      </article>
    </main>
    <footer>
      <p>&copy; 2022</p>
    </footer>
  </body>
</html>

style.css

body {
  color: #333333;
  font-family: Helvetica, sans-serif;
  margin: auto;
  max-width: 800px;
}

main {
  margin-bottom: 40px;
}

header,
footer {
  text-align: center;
}

h1 {
  font-family: Arial, sans-serif;
}

h2 {
  font-style: italic;
  font-weight: lighter;
}

.card {
  width: 350px;
  margin: auto;
  box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2);
  border-radius: 10px;
  overflow: hidden;
}

.card > img {
  width: 100%;
  height: auto;
}

.card__container {
  padding: 16px;
}

At the moment we don't have any content in our script.js file, leave it empty for now.

There is a single difference between this file and the final versions of the examples at the end of our CSS section, in index.html we have added a Like button inside the card:

<button>Like</button>

When you serve the page in your browser you'll see it rendered:

Of course at this point clicking it does nothing. In order to connect some Javascript to our button, first we have to write a Javascript function, give it a name, and finally attach it to our button.

A function is essentially a block of JS code that you don't want to run right away. Another common use for functions is to create blocks of code that you might want to run more than once. Both are great reasons to create a function.

There are a number of ways to create functions as the JS language has evolved over many years. Here for simplicity we will teach the traditional method that works in all browsers. Later as you get more comfortable you can learn about the various shorthand forms.

script.js

function likeButton() {
  alert("Like button has been pressed");
}

When you load your script.js file, the code inside of this function block (the code between the curly braces) is not run right away. You have to first call the function (sometimes called invoke) which basically means "run the code inside the function when i tell you to". The syntax in Javascript to call a function is the function name followed by open and closed parentheses.

So calling our likeButton function would look like:

likeButton();

The alert function is another built-in browser function like console.log, but this time instead of printing a string to the console, it pops it up in your face with one of those annoying pop up messages. Great for testing, but never ever use them for a real site.

So now that we have this function declared, we can update the <button> element in index.html to call the function whenever the button is click. We learned the syntax for calling the function, so the final thing we need to learn is how to connect them. We do that with the HTML onclick attribute.

The onclick attribute is available and most any elements in HTML, but for accessibility reasons you want to aim to use it primarily on elements that are designed to be click (like buttons or links).

We can tell our <button> in our index.html file to run the function when it is clicked by updating it like so:

<button onclick="likeButton()">Like</button>

As you can see, inside of the onclick attribute we can technically write any Javascript code that we want. Even as much as you want (separated by semicolons) although that gets very messy very fast, and is considered bad practice. If you want a button to perform multiple tasks, put them all into a function and simply call that function.

When we load our page and click the like button we are greeted with:

Excellent! Our page is interactive now. If your button does not work try some common troubleshooting tips:

Verify your script.js is being loaded in index.html like <script defer src="script.js"></script>
Verify that you've saved all your files after editing them
Check your spelling and case. Javascript is case sensitive so onclick and likeButton are very particular.
Make sure you are calling the function: it's onclick="likeButton()" and not onclick="likeButton"

At this point we've verified that we can create a function, and call it by clicking a button. That's fantastic progress. The next step is to update the button to show it has been "liked".

Here is the code for your script.js file. Remove the old function and replace it with the following.

script.js

var likes = 0;

// Adds 1 to the number of likes, and updates the text of the button
function likeButton() {
  likes = likes + 1;

  var myButton = document.querySelector("button");
  myButton.innerText = "👍 " + likes;
}

Let's break down each piece above to see what it does:

var likes = 0;

We create a new variable named likes that stores a number. This number keeps track of how many likes our animal picture has received so far. By default we will start at zero.

// Adds 1 to the number of likes, and updates the text of the button

This is something we haven't used yet, it's an extremely common part of programming called a comment. They are used to provide information to other team members (or yourself in the future) looking at your code to help explain what the code does.

You do not need to add comments for every line, the vast majority of code (if you use descriptive variable names like likes = 0 rather than x = 0) will be self-descriptive. Comments would only serve to add bloat. Reserve your comments for code that is less easy to understand fro ma quick glance, or to describe the entirely of what a function does as we have above.

Comments are supported in HTML, Javascript and CSS. Each one uses a slightly different syntax:

HTML 
CSS /* This is a comment */
Javascript // This is a one-line comment
Javascript /* This is a comment that can span multiple lines */

In the real world most developers, particularly senior level developers, will actually spend more of their time reading code than writing it, so good commenting and clean formatting are essential.

function likeButton() {

This creates a function called likeButton. The code inside the curly braces will not be run when the .js file is loaded, but we can call the likeButton() function at a later time to run the code when we choose.

likes = likes + 1;

This takes out likes variable and increments it by one. THe new value is equal to the current value plus one. The first time the button is pressed it will become one (0 + 1). The second time two (1 + 1) etc.

var myButton = document.querySelector("button");

As we learned in the introduction, document.querySelect uses CSS selector syntax to search for elements on a page. This will save a reference to the first <button> it finds, so it's operating on the very fragile assumption that we only have one.

If you were to add more buttons in the future you would need to return and refactor (update) this code. One option would be to add a class, say <button class="like-button">Like</button>. You could then target that specific button with the CSS class selector like so: document.querySelector(".like-button"). Notice the . prefix indicating it's a class selector.

For the moment however, the Like button is our only button, and this code gets the job done just fine.

myButton.innerText = "👍 " + likes;

The final piece here is what updates the text inside the button itself. We update and replace the innerText of the button with a thumbs up emoji, then we use the + operator to concatenate the value of the likes variable onto the end of the string.

This can be a bit tricky for newcomers as the + operator is being used for something other than math. It depends on the context. If you have two numbers on either side like 1 + 1 then Javascript will evaluate that as a math operation and return a value of 2.

If either one of those values is a string (or both) then Javascript will instead treat them both like strings and concatenate them together. So "hello" + 1 would evaluate to "hello1". It's one of those fun quirks of the language that becomes second nature after awhile, but it certainly does cause some confusion when first learning. The best way to become comfortable with it is simply repetition and practice.

So for our example, if the button has been clicked once, then "👍 " + likes will be the same as "👍 " + 1 which will be evaluated as "👍 1" and that's the string that will be placed as the innerText of our button.

Of course we won't know until we try it out. Load the page with the updated Javascript, and start mashin' that like button!

script.js

var likes = 0;

// Adds 1 to the number of likes, and updates the text of the button
function likeButton() {
  likes = likes + 1;

  var myButton = document.querySelector("button");
  myButton.innerText = "👍 " + likes;
}

How many likes can you get to before your hand gets tired? Congratulations, you've created the world's most uninteresting game.

At this point we will bring our discussion on Javascript to a close. We've only just scratched the surface of what's possible, but I hope you've got enough of an introduction and confidence to begin branching out and trying new things.

Remember don't be afraid to mess around. Copy and paste from the internet. Come up with crazy ideas and try to build them. Keep it small to start so you don't get overwhelmed, but don't be afraid to fail.

Javascript isn't something you just "learn" and then you're done. It's an ongoing process that will probably continue for your entire career. I've been writing Javascript for six years now, and I learned something new while writing this complete beginner's tutorial.

The best thing you can do in your development journey is keep your mind constantly open to learning new things and improving.

What are the best free resources for learning Javascript?

Kyle Simpson's digital book series You Don't Know JS is the gold standard for deep diving into learning all the ins and outs of the language. Highly recommend, and totally free to red online

You Don't Know JS

Written by Douglas Crockford, creator of the JSON data format, this book distills all the important stuff about the Javascript language you need to know. This is the first book I ever read about JS, and I give it a lot of credit for helping teach me the fundamentals (ok I guess this one isn't free, but it's good):

Javascript: The Good Parts

For a quick reference for all things JS, there is no better resource than:

https://javascript.info/

Node.js

What is Node.js

Node.js is a program that can run Javascript code outside of a browser.

It is one of the main reasons why Javascript has become one of the most popular programming languages in the world.

With Node, you are no longer restricted to requiring a web browser open to run your Javascript code. You can write Javascript for things like renaming photos on your computer, or sending automated emails, or even full blown desktop applications with tools like Electon.

To try Node yourself, simply follow the instructions on the site and download and install it. Once installed you should be able to open a command line terminal and type:

node --version

And get an output like v16.13.2 depending on what the latest version is when you installed it.

To you is you simply type node followed by a Javascript file. Here's a simple example, I'll create a file called script.js in the directory we are currently in:

script.js

console.log("Hello world!");

Now save it and run this command:

node script.js

You will see "Hello world!" appear on your terminal.

It's as simple as that. The applications just get bigger and more complex from there.

(Note that because your code isn't running in the browser, you will not have access to some of the common browser-specific APIs you might be familiar with. For example document.querySelector which returns a DOM node -- something that doesn't make much sense without the context of a browser)

One of the most common uses of Node.js is to build web servers. It's very popular because it allows you to write both your website (front-end) and web server (back-end) which provides the necessary data for the front-end both in Javascript.

Although you can write your whole web server entirely with code specific to Node itself, to simplify the process most people use a library like Express.

What are the best free resources for learning Node.js?

There are tons of great resources on learning Node.js out there, and they're free!

MDN will introduce you to Node and cover building a simple web server with a tool called Express
The Odin Project full free curriculum for teaching Node.js and Express
Node.js Crash Course great YouTube video series on Node.js from The Net Ninja

Command Line Terminal

If you are going to be working in any kind of web development job, you're going to need to become familiar with the command line terminal. Many of the programs you will use don't have a graphical interface and are designed to be run on the terminal.

Fortunately you don't need to be an expert, and this tutorial will go through the basic commands you need to know.

How do I use the command line terminal?

How you access the terminal will depend on your system. If you are on Mac or Linux it'll be called "Terminal", on Windows you might choose to use PowerShell which is very similar. PowerShell comes bundled with Windows automatically, you can find it in your start menu.

Or you can download something like Git bash which comes included when you install Git and will give you a unix-style terminal even if you are using Windows.

I'll be using bash for this tutorial. Even if you are using PowerShell on Windows most of the commands will be the same or similar.

When you open your terminal it will look something like this:

Most terminals will show the directory directory you are in on the left. Right now I am in the ~ directory which is an alias for my "home" directory. On Windows it would be something like c:\Users\YOUR_NAME on Linux it might be something like /home/YOUR_NAME.

Let's create a new folder called example-folder.

mkdir example-folder

That will create example-folder in your home directory. Now let's "change directory" into that folder with:

cd example-folder

Protip: use the tab key to help autocomplete when working on the terminal. It's one of the most powerful shorthands you'll be able to use for efficiency (and avoiding spelling errors).

Our terminal directory is now in the folder we just created. Let's create a file!

touch hello.txt

(This one will be different on PowerShell, you'll need to type New-Item hello.txt).

If you navigate to that folder in your graphical interface you'll be able to see it has been created!

Now let's go back to the terminal and take a look at the contents of this folder.

ls

That command means "list" and lists the contents of the directory. You should see your hello.txt file show up.

That's enough of that file, we can delete it with:

rm hello.txt

Run ls again and should be gone.

Let's return to the parent directory.

cd ..

The two dots ".." is a shorthand for "parent directory" and a single dot "." is a shorthand for "this directory".

Finally let's remove that folder we created.

rmdir example-folder

And that's the absolute basics of navigating around on the command line. You're now at least equipped with the ability to navigate between folders which is one of the most common operations you will use.

What else do I need to know about the command line terminal?

Make sure you understand the difference between absolute and relative paths. This is not just for the command line, you'll need to understand the different when writing your code as well. You often need to import "modules" from other files and you need to describe to the editor where to find those files.

Get familiar with one of the command line text editing tools so that you can quickly make changes to text and code files without having to boot up VS Code. I prefer Nano for quick edits but there are lots of options including emacs, vi, and more.

(If your terminal opens files in vi and you can't figure out how to get out, type escape then :q. Quitting vi is one of those rights of initiation for anyone working in a terminal).

You'll want to know how to open up the help pages for any program you are using. Say you just installed git but have no idea how to use it. Depending on your platform it will either be:

man git

git --help

You can replace git with the name of whatever program you are trying to learn about. Usually one of the other will get you what you want. The man command stands for "manual" and is intended to open up the manual.

What are the best free resources for learning the command line terminal?

The Odin Project command line basics

The Art of the Command Line beginner's tutorial

Unix/Linux Command Cheatsheet

Git

What is Git?

What is a Version Control System (VCS)?

Having a good understanding of Git is critically important for working in almost any software job. I'd go so far as to say that if you company doesn't use it (or at least an equivalent version control tool) you may want to consider another position.

It's that important to a healthy software lifecycle. So for that reason I want to make sure I explain it well, because despite its important, it can be very daunting to learn and understand for a beginner who isn't used to thinking this way.

Git is a version control system which is a fancy way of saying that it's a tool for taking snapshots of data at a certain point in time.

If you've ever worked on a file and made a copy of it called Resume.doc and Resume-backup.doc and Resume-FINAL.doc etc then you can imagine why a tool like Git might need to exist.

It works for any kind of file and is not specific to coding or software development, although that's where it's most commonly used.

Software developers and teams use it for two primary purposes:

To create a saved state at a certain point in time in case an issue comes up later and changes need to be reverted back to how they used to be, for one file or many files.
To allow multiple developers to work on files in a project at the same time, in the same directory and often even the same file. Git keeps track of who is making changes where, and helps you resolve and merge those changes together, ideally automatically without any input from the people making the changes at all.

For example, if you have a big Javascript file called my-code.js with 1000 lines, developer A could make edits to line 200, and developer B could make edits to line 300. Afterward they would perform a Git command called merge and Git would take care of combining the separate changes both devs made on their lines on their individual files into a single file with both changes.

There are a number of ways to use Git, the most common is through the command line terminal but there are many tools out there designed to make it easier to use. See this section for more info.

Let's take a quick look at how you can use Git

How do I use Git?

First you need to install it on your machine. Check out this link for instructions on how to install depending on what operating system you are on.

You will know you are successful when you are able to open a command line terminal and type:

git --version

You should see a response like git version 2.35.1 or whatever version is currently the newest when you install yours. If it says anything like command "git" not found then you need to double check your installation. Sometimes you need to close and re-open your terminal for newly installed programs to be visible.

Now I am going to create a new folder. You can create it anywhere that you like as long as you know how to navigate to it in your terminal. You can make it with the mkdir git-example command or you can just "Create New Folder" with your mouse and call it git-example.

After the folder is created open up your editor (I'll be using VS Code) and go File -> Open Folder and find that folder you just created.

You can see I have my VS Code terminal open there, it will automatically navigate you to the folder you have open. To open your terminal you can use ctrl + ~ or simply use the menu with View -> Terminal.

Just before we get started you'll want to run the following command:

git config --global init.defaultBranch main

If you are on a newer version of git the default branch will already be named main, but if you are on an older version it will be called master. This command just makes sure that your defaults match what we will be using for this tutorial.

The first command you will always need to use if you are creating a new "repository" (the name for a collection of folders and files tracked by the git program). The command is:

git init

You will receive a message that says Initialized empty Git repository in... with your folder path. That means git-example directory is now a git repository. Any files or folders created within that directory can be tracked as part of your "project".

Note that if you join an existing project/team, you will not need to initialize Git. When you clone a copy of their work it will already have been initialized for you, we only need to git init when we are creating a new project ourselves from scratch.

Now let's create a new file called sample.txt. To create a new file in VS Code simply right click on the left side under git-example and New File. Inside it I'm going to add some text, quite literally the phrase "Some example text".

Now I will introduce you to a couple of the first git commands you should learn, git add and git commit.

git add: Will tell git to add new files to "track" in your project. Presuming that you want to keep track of every file in the directory (this is very common) then you can simply use the . character which in most terminals is a shorthand for "this directory". So git add . means "add all files in this directory to my git project.
git commit: Will tell git that you want to save a snapshot of all tracked files at this current moment. Most commonly you will also include the -m flag which means "message". A commit message is a description of the changes you have made since the last time you committed.

(Worth noting that you can create a file called .gitignore in the root directory of your project and put the names of any files and directories you don't want to include in your repository, so even if you run the add . command those files will not get added. Useful if you have things like environment files that contain secrets and passwords you don't want to upload publicly with the rest of your code)

With that new knowledge in mind our commands will be as follows:

git add .
git commit -m 'initial commit'

You might be able to come up with a better message than that, though it's a common one for the first commit on a project. The common standard for commit message is to write your message as if it follows the start of the sentence "When applied, this commit will...".

So an example would be "change the primary button colour to blue". So that is the format you should use rather than something like "I changed the primary button to blue". Consistent subject and tense makes commit messages in large projects much easier to read and understand.

(If you want to get really deep down the rabbit hole, you can encourage your team to adopt something like conventional commits which applies strict rules about how commit messages are written.)

So now that you have created your first commit, let's learn how to create another one, and navigate back and forth through time and change the state of our project.

Add another line to your sample.txt file. Something like "Some more text" on the next line below the first line. Doesn't matter what the text says, we just want to update the file.

Now run these commands again:

git add .
git commit -m 'add a new line of text'

Technically the git add in this scenario does nothing since we didn't create any new files, but it's not a bad habit to get into as normally creating new files in projects is common, and you want to make sure you add them unless they are being explicitly excluded.

Now that you have two commits, and two "saved" states of your project, you can go back to one if you need to. Let's say that you just want to "see" how things were before, but not actually lose your newest changes.

Start with this command:

git log

If will show you a list of all your commits, with their messages, and a unique string character hash (purple arrow points to in image below) that you can use as an identifier. Note that yours will not be the same as mine, they are randomly generated.

To go back to that state simply type:

git checkout YOUR_COMMIT_HASH_HERE

So in my case it would be git checkout c849819c6f45ef72429abc36b1ee3891c3ede779 but that value will be different for you.

This brings us back to the first commit we did, notice that the content of your sample.txt doesn't have the second line you added. It's not gone, we are simply looking at a different state of the project in the timeline.

At any point you wish to return to the most up to date state just use:

git checkout main

In this context main is the name of the branch you are on. If you have been following this tutorial you will have set your default branch to main at the start. You can always double check what branch you are currently on by typing git status.

Branching is a critically important part of git. The idea of branching is that you create a separate place to track changes, outside of the main branch that acts as the central source of truth for your project. For example, whenever you release a new version of your website or app, that release is almost always done off main.

If you are working on a new feature on a separate branch, those changes won't be released when the main branch is released. Only when you merge the changes from your branch into main will those changes actually be included in the main product.

I'm not going to get into branching for this tutorial, but make sure that you are at least familiar with it before you apply for your first role. You can learn more about git branching here.

So at this stage, if you have learned the basics of add, commit, log and checkout (to switch between branches).

That covers most of the fundamentals you need to know to work with git locally as a solo developer.

But as you've probably heard, git is also used to track changes in a project between developers working on different machines. How does it do that? Well that's where additional tools like Github come into play.

What is the difference between Git and Github?

A common source of confusion for new developers is the difference between git and Github.

Git is a free open source program that runs on your computer and helps keep track of file changes within a project.

Github is a website owned and operated my Microsoft that hosts git repositories online so that you can have access to your files wherever you go and share them with other developers. Think of it like Dropbox with a bunch of extra features specifically designed around working with git repositories.

Github will always have a "copy" of your Git repository and whenever you like you can use git commands like push and pull to synchronize changes you've done on your computer with the ones hosted on Github.

If you are working by yourself it can simply act as a backup system, and help you track issues and to-do's within your project.

If you are working with other developers then it will act as the central location where all changes made by all team members are pushed to and synchronized.

Let's show an example of working with a repository on Github. I'm going to create one locally and show you how to move it up to Github from the command line so that other developers can download or even contribute code to it.

First I am going to make a new project from the command line:

mkdir example-git-project
cd example-git-project
git init

Then I am going to copy the files I created for the tutorials we did on HTML, CSS and Javascript.

If you did not complete those tutorials, don't worry, you don't actually have to understand HTMl, CSS or Javascript for this. We're not even going to run that code, we are simply trying to save a copy of it remotely where other devs can view it.

You can copy the three files directly from the Simple Website Template section.

(Don't worry about what they do, that's not the focus of this tutorial, we are simply trying to show how to push those files to your remote repository.)

After adding those files, run these commands to add them and make your first commit for this project:

git add .
git commit -m 'first commit'

When complete your example-git-project directory should look like this:

Now we are ready to move this project up to Github.

If you don't already have a Github account, create one now.

Once you have an account and are logged in, navigate to "Your repositories" and then look for the green "New" button to create a new repository.

Create a project following this basic template, give it a name and a description. The name does not have to match the same folder name you used when you created your local repository, but it helps if you want to avoid confusion between the two.

Make sure it is set to public if you want other people to be able to view and contribute to the code.

Once it is created you will get a screen that looks like the below. It will include all the commands that you need, the only difference will be that they will use your Github username rather than mine that you see in the screenshot:

All the commands you need are now visible on the screen for you. We would like to push the repository we created on our computer to this remote repository.

The first step is to tell git on our machine that this repository we created is the remote version of our repository. To do that we run the following command:

git remote add origin YOUR_REPOSITORY_URL

This command tells git to create a new remote with the name origin. The URL that you use will depend on how you have Github setup. You can use either an HTTPS or an SSH URL. SSH is normally preferred but you will likely want to go through the What is SSH section first to set it up if you are not familiar.

(The long and short of it is that HTTPS will require you to enter your username and password every time you interact with the Github repository. Setting up SSH means you don't need to do that because the auth key will exist on your machine and be provided automatically.)

Once you have added the remote repository you can confirm it is correct with this command:

git remote get-url origin

You can see the highlighted text in the above example that the origin is correct for the repository I just created.

With that complete you are ready to upload your code. To do that we use the git push command. Push says to "push the current state of my local repository to a remote one."

The first time you do this to a new repository you have to state what remote branch you want to push your current local branch to. In almost all cases this is simply a remote branch with the same name. The command looks like this:

git push -u origin main

Where -u stands for upstream. In this example we are on the main branch, but if you were working for another branch called cool-feature for example, the command would instead be git push -u origin cool-feature.

You only need to do this once, from now on just typing git push will default to pushing to the same upstream branch unless you change it.

Once this push command has been run you should see some upload status messages in your command line, and when you next refresh your Github repository URL it will look like this:

Seeing this indicates you have successfully made your first Github push. Congratulations!

You can now share your Github URL with other people (or even yourself on other machines) and they can instantly get a copy of this project.

Feel free to try it yourself. CLick the green "code" button and then copy the repository URL from there. Create a totally branch new directory anywhere on your machine and run this command:

git clone YOUR_GITHUB_URL

Where YOUR_GITHUB_URL is the URL of your repository. In my case it's git clone git@github.com:alexeagleson/example-git-project.git.

When the command finishes you will have an exact copy of your repo with all the files you added and committed in this new directory elsewhere on your machine. You can make changes and commit them to update the remote repo with the git add, git commit and git push commands same as always.

This applies to other developers on other machines as well!

There's certainly more to learn, but just having these basics down will position you very strongly for using git effectively with any large project at your company.

What else do I need to know about Git?

Important topics you should make sure to learn about include:

Branching

Working With Remote Repositories like Github for example

Merging

What are the best free resources for learning Git?

The Git Book

The Odin Project curriculum on Git basics

Learn Git Branching is a little interactive game that is great for teaching you more about the concept of branching

Oh my Git! an open-source game designed to help people learn git.

The Github documentation will teach you how to use git and Github at the same time.

APIs

What is an API?

API stands for application programming interface and (in context of web development) it has become basically a catch all term for any system that your code communicates with outside of the code.

The reason that we need an API is because we never want to gives users direct access to your data. Imagine you had a social network that displayed things like name, age, etc about your users.

If you let browsers request that raw data directly, they could also choose to display things like password, or credit card numbers, or any other sensitive data that your app might have stored for one reason or another.

Those are extreme examples but the fact is that even if something isn't critically sensitive, most of the time we want to protect our data and only share it with those people who should have access to it.

For that reason we create APIs. I will allow my web server direct access to all my data, and then the app that is running in the browser may "request" some of that data. My web server will decide if the app / user requesting it should have access to that data, and return it if so, or return an error if not.

A front-end developer needs to know how to read from an existing API. That might mean one developed by back-end developers on their team at their company, or even publicly available ones on the internet.

There are many public APIs out there you can start working with when building an app. Some popular ones include the open weather API, the PokeAPI, and the cocktail API.

These services allow you to query data and display it in your app. Regardless of what type you are working with you usually need to authenticate yourself to use them due to the fact that internet bandwidth is not free, so every time you query for data there is a cost associated with that being paid by whoever is hosting the API. (Most of them do offer free tiers for low traffic apps though.)

If you are a back-end developer you will be responsible for creating the APIs. This means you will write the code for them in a language like Javascript, or other common back-end languages like C#, Java, Go, Python, or Rust.

How do I create an API?

You can create an API very easily with just Javascript and NodeJS. Make sure you have read those sections and are familiar with those tools first.

Open up VS Code and start a new blank project. Create a single file called my-api.js with the following contents:

my-api.js

const http = require("http");

const host = "localhost";
const port = 8000;

const requestHandler = function (req, res) {
  res.setHeader("Content-Type", "application/json");
  res.writeHead(200);
  const weatherData = { toronto: { temp: 30, humidityPercent: 80 } };
  const weatherDataString = JSON.stringify(weatherData);
  res.end(weatherDataString);
};

const server = http.createServer(requestHandler);
server.listen(port, host, () => {
  console.log(`Listening at http://${host}:${port}`);
});

The first line loads the built-in HTTP module in Node.js.

Next we define variables to set the host and port that our server will run on.

The requestHandler is a function which is designed to take two arguments, variables that represent the request and response of an HTTP message. The values in these variables are populated by the Node HTTP module whenever requests are made to your server.

Inside the function we call built in Node HTTP methods like setHeader and writeHead to begin crafting the response and HTTP status codes.

If you do not understand these concepts please take the time to read up on web servers.

Next we save some imaginary API data into the weatherData variable. In a real API this data was likely fetched from a database which itself was populated by data from some kind of hardware device like a digital thermometer. Web servers are not an appropriate place to store data since all data is lost when they are rebooted or shut down, hence why we need databases.

For our simple example though we are just creating some dummy data as an example. This data represents an imaginary weather server response from someone who has made a request to our server for the current weather in Toronto, Ontario, Canada.

The weatherData object is then converted into a string so that it can be sent over the network. weatherDataString is converted into JSON, a textual string representation based on Javascript objects (hence the name).

Since it's just text, it does not technically require Javascript to use, any language can output data in JSON format. You can convert a Javascript object to a JSON string with the JSON.stringify() function.

Finally the end method is what we call when we are done settings the headers and ready to return the data to the device which is making the request to our API.

Let's give it a try!

Save your my-api.js file and open the command line terminal in the same directory as your .js file and run this command:

node my-api.js

You will see the output Listening at http://localhost:8000

Open up your browser and go to [http://localhost:8000] to get a response that looks like:

Depending on your browser it may look slightly different since different ones format JSON responses in different ways, but either way, there you have it!

Remember that the primary purpose of most APIs is to send raw data so websites and web apps can be built using that data. Rather than actually displaying it raw like we do here your app would realistically query that URL using Javascript with something like the fetch method and then display it on the screen wrapped in some pretty HTML and CSS.

What are the best free resources for learning APIs?

If you simply want to learn about the concept of APIs, why they are needed and how they are designed here's a fantastic Stack Overflow response and discussion on the topic that covers all the fundamentals.

For more information of building your own API (at least from a Javascript perspective) check out the resources listed in What are the best free resources for learning Node.js?

Libraries and Package Management

What is a library?

What is a package?

In the context of web development, a library (also sometimes called a package, or even a framework in different contexts) is typically used to refer to "someone else's Javascript code" that you download from the internet and add to your code.

Libraries are a necessary and fundamental part of building websites and web applications. Without them it would require us to re-write the same basic code over and over again each time we start a new project.

To give a very simple example:

<html>
  <head>
    <script src="https://cdn.jsdelivr.net/npm/lodash@4.17.20/lodash.min.js"></script>
  </head>
  <body>
    <p id="p-tag">hello world</p>
    <script>
      var pTag = document.getElementById("p-tag");
      pTag.innerText = _.startCase(pTag.innerText);
    </script>
  </body>
</html>

In the above example we are loading the popular lodash Javascript library from a CDN inside the <script> tag.

That loads all the Javascript code from that library. Lodash (hence the name) stores its functions on a variable whose name is just the underscore character.

We then use Javascript to grab the <p> element by its ID and replace its lowercase "hello world" text with the return value of Lodash's startCase function. The startCase function transforms any string so the first letter of each world is capitalized.

There is no built-in Javascript function which does this (only all lowercase or uppercase) so either we write one ourselves, or simply add this library to do it for us.

Of course there are many other considerations you need to be aware of, this whole library for example has a lot more code in it than just the startCase function so it might slow your websites average load time down.

Everything in web development is a trade-off so you and your team always need to consider the pros and cons when making a decision.

What is a package manager?

What are NPM and Yarn?

NPM is something called a "package manager" and it's a tool that is automatically installed on your system when you install Node.js.

Make sure you don't confuse NPM the tool (which you run on your command like like npm install PACKAGE_NAME) and NPM the online repository that stores those packages which you can explore at https://www.npmjs.com/. They both have the same name, but can mean one or the other depending on context you are talking in.

Yarn is an alternative to NPM you can install on your system which works very similarly. For a beginner you will not notice any difference between Yarn and NPM, so I would only recommend you try it if you are curious, or if your project/company is using it.

You can create a branch new NPM project in an folder with this command:

npm init

After the project has been initialized you will see a package.json file which describes all the libraries and packages your project has access to. You can add a new one anytime from the command line like so:

npm install PACKAGE_NAME

Where PACKAGE_NAME is the name of the package listed on https://www.npmjs.com/.

You would learn what functions are available to use from the library in the example above) typically by reading the library's documentation.

That documentation may exist right on the NPM page, or it might be on the Github source code for the project, or most commonly at the project's official website.

Here's an example of the API documentation for the jQuery library.

Let's take a look at a practical example using the lodash package.

Create a new folder in the terminal, navigate to it and run the following commands:

npm init

npm install lodash

Remember you can replace lodash with any other library you find on the NPM registry. That is the source for Javascript packages, and after running the npm install command you will be able to use them in your code as follows if you create a fill called script.js:

script.js

const _ = require("lodash");

console.log(_.startCase("hello world"));

The above will print the start-case'd "Hello World" to the console when you run node script.js.

Remember that libraries available on NPM are public and free to use, so don't hesitate to play and experiment!

Databases

What is a database?

In the context of web development, a database is simply the location where your website or web app will store its data. Basically a fancy hard drive that is accessible over the internet.

The main difference between and standard hard drive and a database is that a database assumes you will be accessing data constantly and will be optimized to provide ways to search for that data extremely quickly.

A database uses a concept called indexes which are exactly like you would expect if you're familiar with the index of a large book, a list of terms or keys that it creates to identify the exact location of information based on certain criteria (an index on text data might be alphabetical for example).

Databases used on the web are usually classified into one of two categories: relational and non-relational (also often called SQL and NoSQL).

(SQL stands for "structured query language" and is usually pronounced as "sequel", but some people prefer S-Q-L.)

Some examples of popular relational databases are MySQL, PostgreSQL, and Microsoft SQL Server. An example of a popular NoSQL database is MongoDB.

If you are just learning then deciding which one to use and learn makes little difference, either option will introduce you at least to the fundamentals of working with databases including indexes, schemas, creates, reads, updates, and deletes.

Typically the responsibility for working with databases will fall on the back-end developer. The front-end developer will have little to no direct exposure to the database.

The backend developer will be responsible for creating an API (see also What is an API?), which is basically code that reads information from the database and transforms it into a shape that the front-end client can use.

If your back-end is written in Javascript, then the developer would write Javascript code that uses an NPM package that gives them access to the database and write SQL queries that are sent to Javascript functions and then onward to the database.

Let's say that you want to build a page that shows current in-stock levels for a website that sells plants.

In your database you have a table which contains both the name of each plant your store carries and the number that are in stock.

(If you've used Microsoft Excel or Google Sheets before then you can think of a database as very similar, just significantly more powerful and able to store and process orders of magnitude more data.)

(Typically in a real relational database the items and stock levels would be in separate tables and marked with an ID number, a process called normalization, but we are keeping it simple for now)

A SQL query to get plant name and inventory information for plants that are currently in-stock might look something like:

SELECT (name, num_in_stock) FROM plants WHERE num_in_stock > 0;

Where plants is the name of the table and name and num_in_stock are the names of the columns that contain the data inventory data.

Once this data is returned the back-end developer writes more code to provide an API endpoint that the website can send requests to in order to get that plant inventory data and display it on the page.

An endpoint is simply an URL identifier designed to provide specific data, for example https://www.myplantstore.com/inventory.

It's important to understand that the website itself cannot query the database directly, because to do so would require you to include the login and password of your database on the website, which would then be accessible to everyone. It wouldn't long for someone (or some bot) to gain access and control of your database.

So one of the reasons we create and use an API is to provide a very specific interface via URL that only allows data to be requested from certain trusted sources, and is only programmed to return specific types of data.

Maybe we also store customer names and addresses in the database, but as long as we don't write code on our back-end to query that data and provide an interface for it to be accessed, then it will be safe as long as nobody (except those with permission) have direct access to the database.

Ultimately all you need to remember as a front-end developer is that a database is a place to permanently store data for your web app, and the back-end developer is responsible for providing you (the front-end dev) access to specific parts of it that you need in order to build your app.

How do I use a database?

This question can be interpreted in a few different ways. Someone working as a DBA (database administrator) would interpret this question as How do I install and setup a database? likely including follow up questions like what kind of data will I need to store?, should it be SQL or NoSQL?, how much data and traffic am I expecting? etc etc etc.

As a back-end developer you might may or may not need to include yourself in these discussions. Usually it depends on the size of the company. In small companies back-end developers often find themselves playing the role of the DBA as well as developing the API. At large companies that is less likely to be the case.

The way we are going to answer this question is specifically on the software development side since that is the focus of this tutorial. In most cases a database will already be set up for you, and your job will be to develop an API that can query it, maybe process the data a bit, and then return it to the user (usually front-end dev) in a shape that aligns with their needs to display it on a page to the user.

For our example we will be using a fantastic and user friendly SQL database which is totally free called SQLite. All the basic queries you would use for managing data in SQLite like SELECT, INSERT, CREATE TABLE etc are the same as you would use when working in all the industry standard SQL databases like MySQL, PostgreSQL etc.

SQLite has bindings for Node.js, which is a fancy way of saying your can just install it as an NPM package and get up and running in seconds.

We can create our SQLite database in one of two ways: either create it right in the code when the program loads (called doing it "in memory") or load up the database from a file on your computer.

Loading from a file mimics a real world use case a little closer, but requires some more configuration so we'll be doing it in memory in this example. If you want to learn how to load the database from a file here's a tutorial you can check out.

As we have learned already in the what is an API? section, an API typically involves a web server that acts as a middleman between a website/app and a database. So we are going to extend the example we showed in the API section to now include querying from a database.

We'll begin by navigating on our command line to the folder containing the my-api.js file from the How do I create an API? example and run the following command:

npm init

Go through the question prompts with all the defaults then run:

npm install sqlite3

(If you aren't sure what we are doing here you may need to revisit the What is a package manager? section)

Next, update the my-api.js as follows:

my-api.js

// Database setup

// 1
const sqlite3 = require("sqlite3").verbose();
const db = new sqlite3.Database(":memory:");

// 2
db.serialize(() => {
  // 3
  db.run("CREATE TABLE weather (city TEXT, temp INTEGER)");

  // 4
  const stmt = db.prepare("INSERT INTO weather VALUES (?, ?)");
  stmt.run("toronto", 30);
  stmt.run("vancouver", 50);
  stmt.finalize();
});

// Web server config

const http = require("http");

const host = "localhost";
const port = 8000;

const requestHandler = function (req, res) {
  res.setHeader("Content-Type", "application/json");
  res.writeHead(200);

  // 5
  let requestedCity = "";
  switch (req.url) {
    case "/toronto":
      requestedCity = "toronto";
      break;
    case "/vancouver":
      requestedCity = "vancouver";
      break;
  }

  // 6
  db.get(
    "SELECT * from weather WHERE city = ?",
    // 7
    [requestedCity],
    (err, row) => {
      // 8
      const weatherDataString = JSON.stringify(row);
      res.end(weatherDataString);
    }
  );
};

// Web server start

const server = http.createServer(requestHandler);
server.listen(port, host, () => {
  console.log(`Listening at http://${host}:${port}`);
});

This is getting a bit more complex, so I'll added a few numbered comments I'll refer to below explaining what is going on here. I do not include any explanations for the web server code since it was covered in the APIs section already.

Load the sqlite3 package that we installed from NPM. What this literally means "look at the package.json file in the node_moudles/sqlite3 directory and see if there is a field called "main" which tells me the location of a Javascript file to start loading and running code from. The next line creates a new SQLite database based on the classes defined in that JS file, and :memory: tells it we will be creating the DB from memory and not from a file.
serialize is an SQLite function that means "run all the commands inside this function one at a time making sure each one finishes before the next one starts".
Standard SQL syntax for creating a table.
Prepare an INSERT statement to add data into our database. We prepare the statement first since the statement itself doesn't change, only the values we are inserting (represented by the ? character). Once this is finalized and run we will have two rows in our database, one for each run function called, with temps for Toronto and Vancouver.
We jump ahead to the point after our web server has been initialized. We introduce one new concept to our server itself and that's routing. I wanted the user to be able to define which city they want the weather for, rather than getting every single city every time, which is a waste of resources. req.url will have the route portion of the request, so if the URL is http://localhost:8000/toronto then req.url will be /toronto. We then save that city data into a variable called requestedCity.
Next we invoke the get function on our SQLite database and run a basic SELECT SQL query for the data. The ? in the statement is replaced with the value in our variable which presumably will be either toronto or vancouver (or fail if the user requests anything else). It's important that we use the ? syntax to insert the variables that came from the user. Remember that all user provided data must be considered potentially malicious, so using prepared statements and the ? substitution instead of directly concatenating the value onto the strings to help project against SQL injection attacks.
This array is where the variable values are taken from to replace the ? characters in the strings. There should be exactly as many values in this array as there are ? characters in your statement.
The query result is returned inside of this variable we've called row because it represents a row in your database weather table. If you're not familiar with this syntax here (the function as an argument that takes row and err as arguments) it's called a callback function and it's one of the more challenging topics to learn as a new Javascript dev, so don't worry if you don't get it right away. This row is returned as an object so we can JSON.stringify() it and return it as the payload for our HTTP request.

Let's give it a try! Boot up your web server with:

node my-api.js

Visit http://localhost:8000/toronto in your browser to GET the weather from Toronto that is saved in your database! Try changing the route to /vancouver and see what happens as well!

This example here provides you with a really solid introductory example of what a "day in the life" is like of a backend developer. You'll likely be writing SQL statements, creating and updating tables in databases, writing code to query tables in the database and returning that data to users by creating "endpoints" using routes that follow a specific set of rules you control for how data is accessed and who is allowed to access it through authentication and authorization.

What are the best free resources for learning databases?

SQLite Tutorial for Node.js to keep going further with the Node.js based exmaple of working with SQL beyond what we've done so far.

MongoDB Univeristy for those devs going the NoSQL route, MongoDB is one of the most popular databases in that space (particularly in the Node and Javascript world) and one you are likely to encounter at some point in your career.

Khan Academy's Introduction to SQL includes videos and tutorials on everything SQL

CS 186 Berkeley an amazing free course that covers pretty much everything you need to know about databases as a back-end developer

The Red Book the classic book on database theory available to read free online. Although it's extremely comprehensive, it's also fairly academic and probably more than the majority of back-end developers will need. Recommended when you have all the basics down and are ready to dive deeper into the fundamentals.

Front End Frameworks

What is a front end framework?

New developers often hear that they "need to learn a framework" in order to get a job, and that of course begs the question: "What is a framework?".

There is no one-size-fits-all answer to that. Many people have different definitions, but the simplest one in the context of Javascript and web development is that it is a library/package that provides structure and opinions on how you should organize your code, and typically also provides tools and helper functions to build the code itself.

What are React, Angular, Vue, Svelte and jQuery?

Popular front end frameworks in the web development world include React, Angular and Vue. These tools are chosen because they provide much better ways of building and managing re-usable code templates for UI elements than the native browser does.

To give an example, you could create a React component called Card which includes <h2>, <p> and <img> tags wrapped in a <div>. It is created as a Javascript function that takes an object as an input argument with text/links to fill in those values. Once built you now have a "component" you can use like an HTML tag in the format <Card> over and over to build, for example, a page with a list of employee profiles.

The language used to build these components is called JSX and is basically a mix of HTML and Javascript. Here's an example of how the Card we just described would look in React:

(In the below example, props is a Javascript object with three properties: url, title, and text, each of which are strings)

Card.jsx

export const Card = (props) => {
  return (
    <div>
      <img src={props.url} />
      <h2>{props.title}</h2>
      <p>{props.text}</p>
    </div>
  );
};

Then using your custom component on your website/app looks like:

App.jsx

<div>
  <Card
    url="www.example.com/steven.png"
    title="Steven Lastname"
    text="Steven is a wonderful employee!"
  />
  <Card
    url="www.example.com/nancy.png"
    title="Nancy Lastname"
    text="Nancy is here to help!"
  />
</div>

Each framework has different levels of how opinionated they get. For example React is a lot more flexible as to how you set up your file structure, but it provides very strict rules about how its components are built.

Angular is also very opinionated about how components are structured, but much more than React enforces a particular directory and file structure within your project as well.

Vue and Svelte are much less opinionated in that regard.

Another one you may have heard of is jQuery. In its heyday, jQuery was te go-to JS tool for providing additional functions that made creating and updating the DOM (the elements on your website) much easier. Much of the reasons developers no longer reach for it is that many of its functions have now been implemented into the Javascript built-in standard itself (for example document.querySelector for DOM manipulation and fetch for AJAX calls), so the language would not be what it is today without jQuery having paved the way.

What is a CSS framework?

What are Bootstrap and TailwindCSS?

A CSS framework is a term typically used to refer to packaged code that focuses on provided pre-written CSS rather than Javascript (though many CSS frameworks including Bootstrap also include Javascript code as well to provide additional functionality).

Two popular CSS frameworks worth knowing about are Bootstrap and TailwindCSS.

Both libraries focus on simplifying and reducing the amount of CSS you need to write, and creating systems to make things like margins, paddings and layout more consistent across your site so you don't need to build and implement your own design system.

Bootstrap popularized the 12 column grid system you'll see in many layout tools these days, for example using a CSS class like col-3 to represent 3/12 (or 25%) of the screen.

Bootstrap is far more opinionated of the two (and therefore a larger footprint in terms of size in your application) but provides some great defaults for developers who are not design savvy and prefer to focus on the function of their application. Bootstrap, similar to jQuery, was far more of a standard in the earlier days of web development as has gradually fallen out of favour, but you'll still find it commonly used in many applications still in operation today.

TailwindCSS on the other hand has seen a massive rise in popularity in recent years mostly due to its small form factor and simple to use classes, as well as being easy to customize. Tailwind is a great choice for devs and designers who want to get a nice looking up up off the ground quickly, while still leaving room to customize it to your desired look and feel.

If you want to really quickly get a feel for what using a CSS framework is like, the simplest way is to simply include the <link> tag example from the Include via CDN section on the front page of https://getbootstrap.com/ in your index.html file and use the Bootstrap documentation to try out some of the different classes.

Here's a super minimal example of a site that loads and uses bootstrap. The class="btn btn-primary" and other classes on buttons is what's hooking those elements into the Bootstrap styles.

index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <link
      href="https://cdn.jsdelivr.net/npm/bootstrap@5.2.0/dist/css/bootstrap.min.css"
      rel="stylesheet"
      integrity="sha384-gH2yIJqKdNHPEq0n4Mqa/HGKIhSkIHeL5AyhkYV8i59U5AR6csBvApHHNl/vI1Bx"
      crossorigin="anonymous"
    />
  </head>
  <body>
    <button type="button" class="btn btn-primary">Primary</button>
    <button type="button" class="btn btn-secondary">Secondary</button>
    <button type="button" class="btn btn-success">Success</button>
  </body>
</html>

How do I use a front end framework?

How do I use React?

Once you have a strong grasp of the HTML/CSS/Javascript fundamentals, it's a good idea to get familiar with at least one framework (the most popular of which at the time of this writing is React).

Kent C. Dodds has a good introduction on the specific Javascript you'll want to know for React before diving in.

I highly recommend you get yourself to a comfortable level with Javascript and understand all of the topics in the above link before you start. Many frustrations people often experience with React when starting with it too early are actually Javascript frustrations.

The fastest and easiest way to start writing React is to use a pre-configured tool like Create React App or Vite.

Create React App (CRA) has been the de-facto standard for many years, also Vite has been picking up traction recently. From the perspective of a beginner learning the basics it won't make much of a difference. Let's start a react app with CRA using the following command on the command line:

npx create-react-app@latest my-cool-app

cd my-cool-app

npm run start

Notice the first command is npx and not npm. The third argument is just the name of the app you want to make. The second command cd is to change into that directory when the installation is complete.

Then when you run npm run start in that new directory you browser will be opened automatically and you'll be taken to http://localhost:3000 where your new app is being served on. It should look like a big animated spinning atom with some title text describing how to edit your app.

Create React App provides a "live reload" dev server which means that changes to your app will be reflected in the browser instantly, making it very easy to make changes to your app and quickly see and debug the results. Try making changes to the src/App.js file and seeing what happens!

What else do I need to know about React?

You should know the difference between functional components and class components. Class components are still perfectly valid, though are considered the "old" way to doing things and all modern React features and syntax are designed primarily to support functional components moving forward.

You should know what hooks are.

You should know what rendering a components means in the context of React.

You should understand React state and understand why React prefers to replace values rather than mutate (hint: it's to make it easier to keep track of when something changes to know when to update the DOM)

You should understand what a lifecycle effect and how you can use a hook like useEffect to watch values and trigger different actions/events when those values change.

You should know what global state management solutions are and why people use them. This includes bu not limited to ReduX, MobX, Zustand, Recoil, etc (there's tons). You should also understand the Context API and when it does and doesn't act as a global state management tool.

What are the best free resources for learning React?

My Blog Post on React shameless plug for my blog post where I go into more detail into how to configure a React project from the group up to get a better understanding of what's going on

Official Documentation and The Official Beta Docs written by the React team should be your first source of information. The beta docs are quite new, but they are much improved and updated to use modern React syntax, so you should always check them first before falling back on the original docs if you can't find what you're looking for.

Reactiflux mark Erikson's (developer of Redux) suggested path for learning React.

Codeacademy's React course.

The Road to React by Robin Wieruch. Not a free resource I admit, though I felt I should include it because I did read it while learning React and think it was extremely helpful for improving my understanding.

Typescript

What is Typescript?

What is the difference between Typescript and Javascript?

Typescript is an absolutely incredible programming language. It is largely responsible for adoption of Javascript to build large-scale business applications that might otherwise never have been possible or realistic without it.

Typescript is Javascript. That's one of those really critical things to realize.

Typescript is just Javascript code with additional syntax added on top of it to check for correctness and potential errors while you are writing it that otherwise would not be caught until you actually run the program with standard Javascript.

If you are proficient in Javascript, you still may not know how to write Typescript. However, if learn Typescript, then you also know Javascript as well. A good TS developer would be more than qualified for any JS job.

It can be difficult to explain to new developers why Typescript is so important. Often they will find it frustrating that they cannot get code to work, or it feels too complex with little payoff.

When it comes down to it, if you want to be employed as a web developer professionally, Typescript is unquestionably the way to go. You will severely limit your job prospects if you only learn JS.

This is probably a good time to show an example of the difference between JS and TS:

script.js

function addNumbers(a, b) {
  return a + b;
}

script.ts

function addNumbers(a: number, b: number): number {
  return a + b;
}

The first JS example is pretty simple. It takes two arguments, and adds them together. If you run the following:

const result = addNumbers(3, 4);
console.log(result);

The output will be 7

What if we did the following though?

const result = addNumbers("3", "4");
console.log(result);

What would the output be? The answer is the strong "34" because each argument was actually a string, so Javascript just assumed you meant "concatenate" when you used the plus sign.

Next let's take a look at the second example above that was labelled as script.ts. It's the same function but the parameters are annotated as number and number also appears after the parentheses to indicate that the return value should also be a number.

The screenshot below demonstrates what you would see when writing this using VS Code:

So although this is a simple example, you can immediately see one of the many benefits. Typescript has stopped us from making a simple syntax error and using our function incorrectly.

Remember that even if you look at this and think "I would never make such a simple mistake", in reality, when working on large projects for long hours everyone makes small stupid mistakes. All the time.

And even if you don't, your coworkers will.

Typescript is there to help you with a warning as soon as you make these errors so you can correct them and get back to working on your code.

So that's great, but how do you actually use it? .ts files?

Browsers do not understand Typescript. They only understand Javascript. So modern working setups are created to allow you to write Typescript code, but then transpile _convert) it into Javascript code when you are ready to run it.

How do I use Typescript?

The easiest way to practice and learn it is with the Typescript playground.

Once you move beyond that and want to start writing your own Typescript code on your machine, the most basic setup involves the following:

First, install Node.js on your machine. This will automatically install npm.
Next, create a folder for your project, then navigate to that folder on the command line and run npm init. Just hit enter to select all the defaults.
Run the following command:

npm install --save-dev typescript

That will install Typescript in your project as a "dev dependency". You should be able to see it listed with a version number in the package.json file that NPM created.

Run the following command:

npx tsc --init

(Note that's npx not npm. npx is a "package runner" which is different from your package manager. It can run packages that are installed in your project as code directly)

This will create a tsconfig.json file which has all the configuration for your Typescript project. You can learn more about tsconfig.json here

Create a script.ts file with any Typescript code you like in it. This file should be in the project root in the same directory as your tsconfig.json.

You can use the example above with the addNumbers function if you like, code below:

script.ts

function addNumbers(a: number, b: number): number {
  return a + b;
}

const result = addNumbers(3, 4);

console.log(result);

Now you need to convert that .ts file into Javascript. Run the following command.

npx tsc script.ts

That will run Typescript on the script.ts file and output a script.js file right beside it with all the type information removed.

That information was only there to help you with development, once you have verified there are no errors none if it is necessary anymore, which is great, because it means it does not add any size/bloat to the actual code that you will be including in your site.

Once the script.js file has been generated, run it with Node!

node script.js

If you copied the above example the output will be 7!

What are the best resources for learning Typescript?

Official documentation As far as I'm concerned there is no better source than the Typescript docs themselves. Lots of great examples starting at the beginner level and up from there.
No BS TS Some great videos on getting up and running with Typescript quickly

Web Hosting

Where do I host a website?

How do I put my website on the internet?

Web hosting is, simply put, the act of putting your website or web app files onto a machine or service somewhere that the general public can access them. There are basically a million different ways you can do this and a lot of factors that go into the decision of which one you might choose, so in this tutorial we are going to focus on the simplest ones.

Things that are important to us right now:

Simplicity & cost (ideally free)

Things that are not important to us right now:

Scalability (ability to handle large amounts of traffic, with large referring to hundreds of thousands of requests and beyond)

So with those in mind I'll start by showing you two great options for hosting a simple website built with HTML, CSS and JS. This type of setup we call a static site which basically means all the files used are "static" in that they are already written and don't change (as opposed to a dynamic app which generates HTML automatically, something like React for example).

The two options I would suggest depend on whether or not you have set up a Github account yet, and pushed your site to a repository.

If your code is already on Github, then I recommend Static Site Hosting Option 1: Github Pages if not, then I suggest Static Site Hosting Option 2: Netlify which is a great service with an excellent free tier for hosting projects.

Static Site Hosting Option 1: Github Pages

This section will follow up with the tutorial that teaches you how to push your site to a remote Github repository. We will be using a repository that is hosting the Basic Website Example but you can host your own custom site if you choose.

Navigate to your site's repository on Github and choose Settings -> Pages:

Use the dropdown to select which branch you want to publish (default is main if you followed the tutorial, but if you're on an older version of git it could be master and then click Save).

...that's it! You're done. You'll be given a custom generated URL based on the name of your Github account and repository that you can share with anyone. Here's mine:

https://alexeagleson.github.io/example-git-project/

Github pages is an absolutely fantastic way to quickly host and share static websites.

If you've reached a point where you are now wondering where to host more complex web apps (React apps for example) or Node apps/APIs then skip ahead to the How do I host a web app? section.

Static Site Hosting Option 2: Netlify

If your HTML, CSS and JS files are on your computer and you haven't learned how to use Github yet don't fret, Netlify is a great option.

Navigate to the Netlify website and create a free account.

Once you have logged in your will see a dashboard that gives you three options: to host a project from Github, or use a template, or upload files manually. We are going to upload the files ourselves:

I'm going to click "browse to upload" and select the directory that has my index.html, script.js and style.css files in it from the Basic Website Example. You can take those files for learning purposes, or upload your own site.

The process will take a few minutes, Netlify will be configuring its web server to load your files as well as creating a custom domain name (URL) for your site that you can share with other people.

Cool! My upload is done, Once finished yours should be publicly available just like mine. Here's the link I received with the files I uploaded, feel free to check it out:

https://stellular-brioche-c71cfe.netlify.app/

That's pretty much it. If you own your own domain you can direct it to the site you just uploaded with the tools they provide and add an SSL certificate as well if you like. Otherwise the randomized Netlify domain they provide you automatically is enough to be able to share your work with others.

Working with free services like this the main downsides you will get is that once your site reaches a certain level of traffic Netlify will likely shut it down until you registered for a paid tier.

This practice is pretty universal, you'll find that it's easy to find free hosting for projects as long as they don't get too much traffic.

If you're looking for a more scaleable option that you are willing to pay for with hosting, then check out services like AWS, Microsoft Azure, and Digital Ocean.

(I'll take this opportunity to warn you to avoid _Godaddy at all costs just in case that comes up as an option in your search, I won't waste time getting into the details, but suffice to say there are far better options. Here and here and here can get you started on reasons if you're curious)._

How do I host a web app?

How do I host a React app?

This section will discuss options for hosting web apps, we'll focus specifically on React apps, but the basic premise is the same for other popular frameworks (like Angular, Vue, etc) as well.

First we will create a basic React app with the popular tool Create React App. For a major production React app for your business, there are probably better options in 2022, but for quickly getting a simple React app up and running with little to no configuration it's still the best option out there.

(Note that the following command requires Node.js to be installed on your machine. If you haven't done that already, make sure you check out that tutorial first. It also assumes you have already learned how to hos your project on Github)

npx create-react-app example-react-app

Next I will go to my Github profile and create a project called example-react-app.

Back on my terminal I'll change directory into the project directory, add the remote Github repository I just created as the origin and then push my code.

cd example-react-app
git remote add origin YOUR_GITHUB_URL
git push -u origin main

Here is my personal repository where I've pushed by React app. Yours will have a different URL.

Next navigate over to Netlify, login to your account (or create one if you don't already have one), select "Start new project" and then "Import an existing project from a Git repository".

You will need to give Netlify permission to access your Github profile, and then either select the repository to give permission manually, or give permission to all repositories. Once permission is given you can select your React app repository.

For configuration select the main branch, and unless you've altered the default configuration, you can leave all the rest of the commands as-is. Netlify will go through the process of deploying your site, and provide you with an automatically generated public URL afterward.

That's it! It's that simple. If you want to make updates you can simply commit changes to your repository, and then choose to redeploy on Netlify. You can even configure it to watch for changes on your Github repository, and redeploy automatically whenever you push new changes to the main branch.

Here's the result of my upload and the public URL that is generated, yours should look very similar:

https://magenta-conkies-c988a1.netlify.app/

How do I host a Node.js server?

How do I host a Node.js app?

Hosting a Node app is a little bit trickier than a standard website, but not too much so. You'll have more difficulty finding a free tier for a Node app.

Before getting into other options, I should say I've heard very good things about fly.io. I've never used it personally, but it seems like a potentially great free option for Node.js hosting. If you're interested give this tutorial a try and let me know how it goes.

One other common suggestions is to use AWS Lambda. One benefit of lambda functions is that although you will still need to provide your credit card, they will only charge you based on actual usage. So if you and your friends are the only ones using the app, you're likely only looking at a few pennies a month.

I personally haven't used AWS Lambda, so I'll be focusing on a different option for this tutorial, A service provided by Digital Ocean called droplets.

Droplets are simply a fancy name given to access to a Linux server that is fully accessible to anyone on the internet. They're great because you can run pretty much anything you want on them. The lowest tier which is more than enough to run basic Node applications is only $5 per month, and will more than serve our purposes.

It's a pretty small price to pay for something that you can use to host almost any project you work on, including the websites and web apps we've already discussed.

Once you have signed up for a droplet, log into your account and select it.

You will want to make sure you can access your droplet from the command line on your machine. To do this you need to set up SSH. If you have not already created SSH keys on your machine then check out the What is SSH? tutorial to create them.

Once you have created your public key, from your digital ocean droplet dashboard click Settings -> Security -> Add SSH Key. After it is added, copy the IP address for your droplet and return to your terminal.

You can now access your remote Linux server (droplet) with this command:

ssh root@YOUR_DROPLET_IP_ADDRESS

When successful, your terminal will show a welcome message (by default) and you are in a different location/directory. This will be the home directory of the root use of your droplet. You can now create additional users if you choose, or whatever you like.

Once you have access to your droplet via SSH, you are in a good position to begin following the official guide for running a Node.js app on Digital Ocean:

Here's a quick overview of what it will show you how to do, and why each step is necessary:

Step 1 — Installing Node.js: Node is not installed on Linux by default so you need to install it on your droplet yourself.
Step 2 — Creating a Node.js Application: The tutorial assumes you will be writing the application on your droplet, which most often won't be the case. This step is where you may need to deviate a bit. If you have a Node app hosted on a Github repository, then you will want to install git on your droplet as well and then clone your repository onto your droplet. That way the app you created and pushed will be on your droplet and ready to run.
Step 3 — Installing PM2: PM2 is an amazing robust production quality tool for running Node.js apps. Technically you could run the normal command node my-app.js same as you do on your machine, but what happens if it crashes? PM2 will take care of running the app, as well as automatically rebooting it if it crashes along with a number of other quality of life features like monitoring.
Step 4 — Setting Up Nginx as a Reverse Proxy Server: Nginx is one of the most popular web servers in the world and a fantastic tool. The reason you need it in this case is to control access to your app from the outside world. If your app is running on port 300 for example and you want people to access it on a domain name you bought like mycoolapp.com, Nginx is the one responsible for directing requests to that domain name on your droplet to your app on the correct port.

So with that context provided, here is the tutorial to help get your Node app up and running live:

https://www.digitalocean.com/community/tutorials/how-to-set-up-a-node-js-application-for-production-on-ubuntu-20-04

What is a CDN?

CDN stands for content delivery network and it's basically a hosting service designed to maximize speed and uptime for resources on the web (files, images, full sites themselves, etc).

The main difference between standard hosting and a CDN is that content on a CDN will be hosted in multiple places, potentially all over the world.

So if one user from your site needs to load Cat-Picture.jpg and they are in USA, and another requests it in France, the CDN will serve the same cat picture from a physical server located in the USA to the user in the USA, and vice versa (or at least the nearest one).

Obviously there's much more to it than that, but this definition should be more than enough to get you to know what people are talking about when they use the acronym.

One of the biggest CDNs out there is called Cloudflare.

If you want to learn more about CDNs then Wikipedia is a great place to start.

Web Domains

What is a URL?

What is a domain name?

What is DNS?

A domain name is simply some text that gets converted into an IP address through a process called DNS.

An example of a domain name is www.google.com. The https:// is the scheme and technically not part of the domain name.

DNS (domain name system) servers exist all over the internet and act as kind of a global dictionary of names and the IP addresses they belong to. They are constantly being updated as names and IP addresses change. There is no one single DNS server that acts as the source of truth, there are countless DNS servers out there and you can even choose which one you want to use to resolve a domain, or host your own.

If you've ever seen the dreaded "This site cannot be reached, server's DNS address could not be found" then that's because your request could not find a way to translate the name into the address of an actual server that is hosting the website or app you are looking for.

How do I get a custom domain name?

In order to get your own custom domain name you must purchase it first from a domain name registrar. Typically these names are purchased as an annual plan that can be renewed. You will always have first choice of renewal, but if you ever opt not to renew there is nothing to stop someone else from buying the name and preventing you from ever getting it back (or having to pay a steep price to buy it from them).

I will not get into the nitty gritty of the business of domain names, but suffice to say it can be big business and common phrases in the .com space an easily go for tens or hundreds of thousands of dollars.

Fortunately there's a near infinite number of possible names out there and as long as you aren't too picky you can probably get something close enough to what you want for as low as a few dollars a year.

First you must choose where to purchase from. There are plenty of options. I'd recommend either Gandi or Namecheap or Cloudflare. Avoid Godaddy if at all possible (Google their reputation around customer service before making any decisions).

After you've purchased the name, most of the services that sell you the name will also be willing to sell you site/app hosting services as well. You can use them if you wish, but it's not necessary, they will have options available for you to point your domain name to your existing server, whether it be something like a Digital Ocean droplet or anything similar.

The final step you need to arrange after buying and configure a domain name is called an SSL certificate so that you site can be accessed over secure HTTPS rather than insecure HTTP.

What is SSL?

What is HTTPS?

How do I get an SSL certificate?

The full scope of exactly what HTTPS and SSL are is beyond the scope of this tutorial, but they are interesting topics if you wish to read more about them.

I will be focusing on the practical application of setting up an SSL certificate for your domain. The first thing I will stress to you is that generating an SSL certificate is a completely free process. I stress this because although it requires a bit of effort, there are no shortage of domain name sales services that will gladly charge you a $50+ fee to "secure" you domain for you.

It's a completely predatory charge that targets non-technical folks who don't know any better. Don't fall for it.

How you generate it will really depend on the hosting service you are using. For example if you are using Digital Ocean as I have discussed throughout this tutorial, then this guide will get you up and running.

If you are using another hosting service then simply Google their name along with "ssl" and you will almost certainly find a tutorial to generate one for free among the top results.

SSH

What is SSH?

SSH (secure shell) is a very important concept to be familiar with, though it is often challenging for beginners to understand. I'll try to explain it as simply as possible.

SSH provides a way for two machines to communicate securely. It is most commonly used over the command line. It provides a way to authenticate a machine without explicitly requiring a password (though a password can optionally be used).

Though SSH can be used to transmit data for a variety of applications and uses, the most common use of it you will encounter in web developer is authenticating your machine when logging into remote servers (for example ones that may be hosting your web app) and for authenticating for version control services like Github, so that when you want to push your code to your company's repository, your machine is already authenticated and don't need to provide your username and password.

How do I use SSH?

What are SSH keys?

So that explains what SSH is as a tool, now let's take a look at how to use it. SSH uses the concept of keys, specifically a public key and a private key.

(The explanation of how it actually works to provide security is far above the scope of this tutorial, but it's a fascinating topic and I'd recommend you check out this Wikipedia page if you want to learn more about it.)

We will focus on the most common developer's use case, using SSH to authenticate ourselves so we can push code to Github. If you're not familiar with Github you can check out the What is Github section of the tutorial and return here when you reach the point of setting up your SSH keys.

The idea behind the public and private keys is that when you generate them, your private key should remain on your machine and you should never ever share it with anyone. As soon as anyone gains access to your private key, they immediately gain the ability to impersonate you, and if you ever suspect that may have occurred (someone got access to your computer for example), you should delete your keys and generate new ones.

The public key on the other hand can freely be shared with anyone. That is what you will add to Github, and whenever a connection request is made from your machine, Github will use that public key to verify the request being sent from your computer with the private key matches.

The keys themselves are simply long strings of text characters, usually a few hundred depending on the algorithm used. When the keys are generated two files will be created, and you can differentiate between the public and private keys with the extension.

The private key will be generated as FILENAME (the filename again will depend on the algorithm used) and the public key will be generated in FILENAME.pub. So remember whenever anyone or any service asks you to provide your SSH key, you want to always make sure you provide the one in the file with the .pub extension.

How do I generate SSH keys?

Rather than go through the actual step-by-step instructions on how to genertae them, I will direct you to one of many great tutorials out there. Generating the keys is easy on any operating system using a free command line tool called ssh-keygen.

(Before showing how to generate keys from the command line, I would also take the opportunity to suggest the SSH key generation and SSH agent in 1Password. Full disclosure, this is the company I work for and I don't usually like to pitch products, but this is a rare case while I am genuinely impressed by how useful it is to manage SSH keys in 1Password and be able to share them between different machines I'm working from.)

That said, it's absolutely not necessary to use at all, just a nice convenience thing. The below links show you how to generate your own keys on the command line completely free:

Github itself also has a own tutorial if for any reason you prefer to use that, but both it and the Atlassian one should give you the same result.

(You have the option of adding a password to SSH keys that you need to enter each time you use them. SSH keys themselves are already very secure, I personally find adding a password on top of that not worth the inconvenience, but it's completely up to you.)

Once your keys are generated, make sure you know where the .pub file was creating. On macOS/Linus you'll find it in the ~/.ssh directory, and on Windows the %userprofile%/.ssh directory. You can view its contents by opening it in any text editor.

To add it to your Github profile, copy the contents of the .pub file and go to your Github dashboard. Click your profile icon then Settings -> SSH and GPG Keys. Now click the green New SSH Key button.

Paste the text of the key itself in the Key text box and give it a title. Typically the title refers to the machine the key is tied to, so if I generated it I might give it a title like "Alex's Personal Laptop" or "Alex's Work Laptop" etc. Probably good to provide even more details if you work with multiple machines.

Once that is saved you are ready to clone, push and pull Github projects over SSH. Make sure that from now on you are always copying the SSH version of the URL and not the HTTPS one. You can tell which is which based on their structure, examples below:

SSH URL Example: git@github.com:alexeagleson/example-git-project.git
HTTPS URL Example: https://github.com/alexeagleson/example-git-project.git

If you have already been working with a Github project over HTTPS and want to switch an existing project, you will need to change the remote URL. To do that, navigate to the project on your machine and run this command:

git remote set-url origin YOUR_REPOSITORY_SSH_URL

That will update the default origin to use the SSH URL and you will no longer need to enter your username and password when pushing to it.

DevOps

DevOps is one of those terms that can mean ten different things to ten different people. I will not make any attempt to assign any kind of formal definition to the term, my only concern is helping you understand the basic concept.

DevOps is at its simplest the automation of the deployment process. We've talked at great length about what a website and web app is, but maybe a little bit less about deployment. Deployment is simply the act of getting your site or app online and available to access for your consumer.

In the distant past there was a very clear divide between software developers and systems administrators (often called SysAdmins or just more generally IT personnel). As the industry matured it became clear that the divide between these two roles was not as large as once thought, and that a lot of benefits could be gained from having developers who were familiar with how their apps were deployed, as well as system administrators who were familiar with code, so that each could communicate their needs better to one another.

These days very few companies manage their own infrastructure (the servers their code is deployed on) and even fewer have their own physical servers. The majority of tech companies deploy their products using cloud services like Microsoft Azure or Amazon Web Services (AWS).

Using these services rather than purchasing and maintaining physical servers allows tech companies to focused entirely on their product and not maintain a large infrastructure team. As their application userbase grows and scales tools like AWS are designed to automatically handle that increase in traffic in real time in ways that would be far more difficult if not impossible to do if it were necessary to purchase and install more physical servers.

That feature of course comes at a cost, but that cost presumably will be less than trying to manage your infrastructure yourself.

Nevertheless, managing the needs of a high traffic application can be very complex. Tools like AWS are designed to be easily configured by code rather than through direct access to the servers themselves. DevOps personnel are particularly adept at building and maintaining configurations for cloud services so that developers do not need to think as much about how the code they write is actually deployed and served to the public.

What is CI/CD?

In addition to managing things like cloud configuration, a good DevOps engineer also takes care of building an application's CI/CD pipeline. CI/CD stands for continuous integration and continuous delivery. It essentially means creating the needed processes so that developers can continue to build and implement features for the application with minimal risk of introducing bugs or errors.

DevOps engineers will use features within tools like Gitlab or Github to ensure that in order for code to be allowed to merge into the primary main or master branch of the project, it must meet certain conditions.

Those conditions can range in anything from simple linter checks to full blow end-to-end tests which run real simulated user actions on your code and check to see if they match conditions you have set. If they fail, the code a developer submitted will not be allowed to merge.

Continuous integration refers to the idea of frequently merging new code into the priamry project branch, rather than keeping it in a separate branch. This has the benefit of keeping every developer up to date with the most recent state of the project, and avoids the risk of large merge conflicts if separate devs have been making changes to similar areas of the codebase over days or even weeks and months.

Continuous delivery refers to the idea of releasing these changes frequently to the customer. This provides the significant benefit of being able to quickly turn around customer feedback into real usable features or bug fixes. If your CI/CD automated pipeline is solid then you can be confident that any code changes which have passed all your tests are potentially ready to be released to your users without the need for extensive QA (though manual QA is always a good idea regardless of how good your pipeline is.

Containers

What is Docker?

Docker is a tool that allows you package the environment for running your application along with the application itself. You can accomplish this as simply as including a single file called Dockerfile with your project.

A developer might choose to use Docker because it provides guarantees that the versions of each tool installed (like Node, Python, NPM, MySQL, etc) are the same for every developer working on the project, regardless of what they have installed on their own machine.

If you've ever had trouble getting your environment setup properly to start your development work, that's exactly what a tool like Docker is designed to solve.

It uses a concept it calls containers which are lighter weight (require less resources) than full on virtual machines to create the environment for your application. These containers are designed to be extremely portable which means that you can quickly deploy them anywhere, and also scale up your app quickly by simply deploying more copies of your container.

All you need to do is define the requirements for your environment in the Dockerfile (for example Ubuntu 18, Node.js, etc) and every time your container is started on any machine, it will recreate exactly that environment. So you already know in advance that you will not have any issue with missing dependencies or incorrect versions.

If you would like to learn more about Docker from the perspective of a Javascript developer I have written a brief introductory Docker tutorial.

What are the best free resources for learning DevOps?

DevOps is more challenging to learn on your own than some other programming topics, simply because at its core its about empowering development teams at scale which is difficult to emulate when trying to learn as a solo developer.

Remember that we discussed how being a good DevOps engineer is about understanding the needs of the development team. Being a good DevOps engineer requires you to have a very solid understanding of a lot of developer and sysadmin fundamentals including but not limited to: Git, Linux, bash scripting, and different cloud services (particularly the one the company you work for uses).

Make sure you have a solid understanding of Git and Linux commands before you go any further:

What are the best free resources for learning the command line terminal?

What are the best free resources for learning Git?

After that depending on your code repository and hosting provider you'll want to familiarize yourself using their own tutorials and documentation:

Learn Github Actions

AWS training

Azure training

This really good reddit post about DevOps resources

Kubernetes the Hard Way

I'd like to end this by acknowledging that I am certainly no DevOps expert, the resources and descriptions I have included here are best effort and hopefully do not offend any serious DevOps people. There are still further many advanced topics I have not covered that will inevitably come up if you take the plunge into the DevOps world.

If you are interested would encourage your to at least familiarize yourself with additional tools like Kubernetes, Ansible, Jenkins and Terraform. Their use cases are mostly beyond my needs as someone who is focused primarily on the development side, but are undoubtedly critical for anyone looking to dive further into DevOps.

Interviewing and Job Hunting

In this section I'll be discussing general topics and frequently asked questions as they relate to the web development career space and software industry in general.

These responses will inevitably be a bit more opinionated than the other sections, but I do my best not to state any advice as the "only way". There are an infinite number of different paths you can take to success in this industry, it's all a matter of varying degrees of speed and effectiveness.

The vast majority of it is simply based on my own personal experiences and years spent in online developer communities and forums reading experiences from other developers at similar stages in their career.

How long until I will be ready to apply for jobs?

Everyone is different of course and there is not going to be a single answer that applies to everyone. Some people will have more time in a day to invest into their learning.

Some people will be learning in the evenings / weekends while supporting themselves with another job, others will have the freedom to make learning itself their full time job.

The important thing to remember is that while the process will be faster for some, everyone has the capability to make it eventually if it's something they are genuinely interested in.

My extremely ballpark'd answer for the majority of people would be about one year. For those who can dedicate to learning full time, it could potentially be more like 6 months. Others who are extremely busy and only able to learn a few hours a week might be looking at closer to 2 years.

How do I know when I am ready to apply to jobs?

This is a very common question, and the honest answer is that there is no specific threshold. Some companies are willing to hire people with very limited skills and experience who are willing to learn on the job, other companies have very high requirements for skill level before they are willing to hire someone.

Often there is really no way of knowing which company is strict about the requirements they ask for, and job descriptions unfortunately are notorious for not actually defining the real requirements for the job.

Many jobs that "require 5 years of experience" or "must know Angular" will end up hiring someone with 2 years experience who has never used Angular at all. The reality is that the market is very competitive and companies are often willing to settle for candidates that only tick some of the boxes who have a good attitude and sound like they will be willing to learn and train in the areas they might be missing.

The only way to know is to apply. I'm not saying to start applying right away before you can even put together a basic project, but if you have been learning for 6 months straight and get to a point where you are comfortable with the basics of HTML, CSS and Javascript then I would encourage you to think about applying for positions, even if you don't meet all the requirements.

The "worst case" is you get to practice interviews and get a better understanding of what companies are really looking for. The "best case" is you get hired! Honestly, both cases are a win for you.

For more information on preparing for your first job check out the What are the most important factors in getting hired? section.

Where do I find jobs?

Your best resource will probably be LinkedIn. Keeping your LinkedIn profile up to date is critical for people looking for work.

Start building your professional network however you can. Look for meetups, conferences, and talks in your area. Find ways to connect and start chatting with other developers. Many people find that some random network connection was the key they needed to get into their first job.

Aside from that you should look up companies that you are interested in and view the "careers" page on their website. Often applying to a company directly is an effective tool. Make sure to include the reason the company interests you in your initial point of contact.

If you are a student, do absolutely anything you get to get into an internship or a co-op program. The reality is that the most effective way to get a job is through existing professional experience, and internships and co-ops provide one of the easiest paths to get that experience. Do not pass up that opportunity under any circumstance.

Lastly, while you're learning, remember to showcase your skills. Two great methods to do that are by blogging or recording videos.

You can start to build a respectable network entirely online without actually meeting other developers in person this way.

Remember you don't need to be an expert to write a blog. Keep your topics small and focused on things you have learned, and let the intention be to share what you have learned with others. You might be surprised at some of the contacts and opportunities you can get if you keep it up.

What programming language should I learn?

Why are there so many different programming languages?

This is a great question, and there is no right answer for everyone. It really depends on what kind of work you want to do. You might wonder "why are there so many programming languages?" and the reason is that each different language is particularly good at something.

Python is known for being a great first language to learn, it's good a relatively simple syntax that is easier for beginners to understand, and it has a lot of different practical use cases including task automation, data science (crunching numbers and statistics) and building web servers.

If you want to get into mobile app development, then you'll want to learn Kotlin (for Android development) or Swift (for iOS development).

If your goal is simply to get employed, then Javascript is definitely your best bet. Although it is region dependent (you'll want to search in your local area for what kind of jobs are available) in most parts of the world Javascript (and web development in general) remains the most in-demand skill, and all signs point to it remaining that way for the forseeable future.

Everything these days is web driven, from apps on your phone to home appliances.

Maybe companies use web frameworks to help speed development, organize their code and build apps more efficiently. Some you may have heard of include React, Angular, Vue and Svelte. It's well worth it to learn at least one of these frameworks, but don't jump into them too quick, as each one is built on top of the fundamentals of HTML, CSS and Javascript and typically require a solid working knowledge base of those three in order to learn effectively.

Which framework you choose is up to you, though if you are targeting employment make sure you search the jobs in your area. In the vast majority of the world, React jobs outnumber all other frameworks by a large margin (though there is a sizeable market for Vue in Asia).

If web development doesn't sound like your thing, maybe systems programming is more up your alley (embedded controllers, hardware devices, internet of things (IoT) and etc).

To become a systems programmer there are many paths to take, but most start through the C programming language and build from there. It will be important to have a stronger grasp of computer science fundamentals in this field than in web development since you will be interacting with low level concepts like the operating system and memory directly.

Check out the section called What are the best free resources for learning computer science? to find some great resources on learning systems programming concepts.

What if I fail?

What if I get overwhelmed?

Learning software development is extremely challenging, for everyone. Any dev you see who makes it look easy almost certainly has years of practice and failure behind them, it's no different than any other skill that takes years to develop.

Along the way you are almost certainly going to find days where nothing goes right, or everything seems way too overwhelming and it's impossible to keep up with.

The only advice I can give here is to say that this is perfectly normal and part of the process. Failing at building a project will teach you a lot more about that project then not starting it at all. Sometimes you'll find that you've bitten off more than you can chew, and that's okay.

There's nothing wrong with taking a step back and slowing down. When you encounter a topic that seems too large to digest, see if you can break it down into small pieces. Take a day to learn more about just one small aspect of it. Then move on to the next thing. And the next thing.

And remember, there's no set amount of time when you will have "learned something." New devs often say things like "I'll give myself six months to learn Javascript". It doesn't really work like that. I've been writing Javascript for years and I'm still learning. Your goals should reflect concrete tangible targets, focus on learning specific aspects of the languages, rather than the whole language itself.

What are the most important factors in getting hired?

How do I get a job when all companies want existing work experience?

This is the real question right here. Honestly when it comes to software development, finding your first job will the most difficult stage of your career by a significant margin.

Work Experience

Those who are still students should do absolutely everything they can to take internships and co-op placements, having those will give you a massive leg-up over other devs in your peer group who don't when it comes to full-time hiring after graduation.

If you're looking at bootcamps then make sure you pick one specifically that offers networking support and helps connect you with employers after graduation.

For the rest of us getting into development later in life, the best thing you can do is allow the reality to sink in that it's going to be very challenging, accept it, and do your best not to allow it to deter you.

The vast majority of your applications will be rejected. And that's perfectly okay, it's almost certainly due to a lack of experience, not a lacking in skill or ability on your part.

It's important to train yourself not to take it personally. Rejection does not mean "I'm not good enough" it simply means that particular companies needs didn't align with your skills at that moment.

That's it and nothing more.

The fact is that most companies are risk averse. They see someone new to the field with no proven track record of working for an organization and they don't have the confidence that you have the skills and abilities required to contribute. The reality is that if you think you do, you probably do -- but unfortunately many companies just aren't willing to take that risk.

So since this seems like a catch-22 scenario where you can't get experience, how exactly do you break into the field?

The best advice for new devs is that you have to look at it like a numbers game. If there are 100 companies out there, and 1 of them is willing to take a chance on new devs, then you need to apply to 100 companies to get your job.

Apply like crazy! Come up with a solid but generic resume, and start sending it out. Target all the web dev jobs in your local region, look for companies online that are hiring remote. Make the expectation that most will reject you and be perfectly okay with that.

Every failed interview is experience at interviewing. The more you do the more comfortable you will get. Get your resume into as many hands as possible and do as many interviews as you can.

Be completely honest about your skills. When asked a question you don't know the best answer is always "I'm not familiar with that, but I would love to learn more about it."

All you need to remember is this: you only need one company to say yes, and you're in. Once you've got that, then you'll have work experience, and everything from that point on just gets easier and easier.

Now that we've talked about work experience, let's look at some of the other critical factors at play when it comes to getting hired.

Networking and Relationships

When I say networking I'm not talking about networking I'm talking about networking.

Which is unfortunate because if you're like a lot of people (myself included) learning the OSI model is a lot easier than building a professional network. Like many other things in life, the earlier you start the better off you'll be in the long run.

One of the most helpful things when I started to internalize this was the realization that your network doesn't necessarily mean "people you've met in person". You can build a network entirely online through contacts you make on social networks, reddit, Github, Stack Overflow, Discord, etc.

Particularly with the rise of remote work, you never know if the person you were talking to or helping out online two weeks ago could be the person who connects you with your next job. It happens all the time.

I should clarify that having a network of developers is absolutely not required for finding a job. The good old fashioned method of sending out 100 copies of your resume to 100 different companies still works great too. In fact if your goal is to get employed realistically you should be pursuing both these methods.

If you fancy yourself a writer or a speaker, don't sleep on blogs and Youtube tutorials. It's a common misconception that you need to be an expert. Certainly do your homework, but after spending some time getting confident enough with a small topic (even something as simple as loops or if statements) try writing a blog about it!

This is something many bootcamps heavily encourage students to do. It can be a great way to build confidence and also expand your personal network.

Lastly, don't sleep on LinkedIn. Yes, everyone is on there to self promote, and yes the news feed can be really difficult to stomach. The good news is you don't need to use it for that. Don't even look at it. All you need is your own profile up to date, and with some experience under your belt, the recruiters will be reaching out to you rather then the other way around.

Technical Knowledge and Interview Performance

What can I expect from an interview?

The standards for developer interviews are all over the map.

One company might simply want to chat and talk about projects you have worked on. Another might want to run you through technical aptitude tests. Another might give you a take-home project to work on in your spare time.

You should be prepared for any of these and more. As a junior developer your expectation should be a strong working knowledge of HTML, CSS and Javascript, Git, and the command line.

You'll want to supplement that with whatever additional technologies and tooling the position is asking for. So if they are hiring for a junior React position, obviously you would want to be familiar with the basics of React as well.

I think it's really important to taper your expectations and be ready to encounter scenarios where companies are hiring for junior positions, but after getting into the interview it becomes clear they are actually looking for a very experienced developer with the hopes of only paying them as a junior.

Unfortunately all you can really do in these scenarios is your best, ask lots of questions, be honest about your skills, and don't let it discourage you.

Companies will often be willing to relax their skill requirements a bit if they find someone with a good attitude who appears they are ready and willing to learn. Allow yourself to be that person.

Don't forget you are interviewing them as well. Ask questions to try and get an idea of their workload, company culture, what an average day looks like etc.

And don't hesitate to answer a question with "I don't know".

It's a much better answer than making something up or guessing wrong. You can follow it up with something like "I don't know, but here's something similar I've encountered before" or "I don't know, but here's how I would start looking for the answer if I encountered this".

Interview Styles

Show the interviewer that you are proactive when it comes to encountering situations where you don't know the answer, because as we've already discussed, these situations will happen almost every day in a developers career. Embrace that fact and leverage it in your interview.

Let's take a look at some of the different styles of interviews you might encounter in your job search: Here's a realistic look at three different hypothetical interview approaches at three different companies:

Company A: Tell me about some projects you've worked on. Tell me about why you chose the tools you did (discussion style).
Company B: Open up this online browser tool and use Javascript/Python/whatever to reverse a string (whiteboard style).
Company C: Here's an assignment to build a landing page for a website with an interactive menu. Take it home and work on it and then send us the results tomorrow (take home style).

Of course there are plenty more scenarios, but this captures sort of the three main "styles".

The first (discussion style) is based on a conversation of you discussion your knowledge and experience.

Personally I find these the best for junior positions. Most people starting out have a very difficult time programming while under pressure (even for simple stuff) and given an opportunity to discuss they knowledge, the interviewer can ask questions and probe to make sure they actually understand the things they're talking about.

The second (whiteboard style) may be done with actual code or may be done on a board where you simply use a marker. The interviewer usually isn't necessarily looking for a correct or complete solution, but an approach that shows an understanding of the problem, even if there are minor errors or edge cases.

Also important with this style of interview is the candidates ability to ask good questions and clarify assumptions. Sometimes there will even be missing information in the question intentionally and the candidate is supposed to get that information by asking questions. Despite being a coding challenge, this style of interview is very much focused on a discussion of the problem. Typically, depending on the complexity, you'll want to discuss it for up to half the allotted time before you even begin to write code.

When you get up in the higher levels and more competitive companies (often referred to as FAANG or MANGA companies), these styles of interviews get more common and much more difficult than reversing strings.

If you're interested in pushing for those top tier jobs, check out the DSA, Leetcode and Systems Design section for more information.

The final interview style (takehome style) typically involves giving the candidate a small challenge to do on their own time. This can have the benefit of allowing you to work at your own pace without the pressure of someone watching, but it also has the downside of being time consuming, and the majority of companies will not be paying for your time.

Early in your career I can understand agreeing to take home interviews when you are simply trying to get any job you can, but as you progress in your career you'll find yourself passing on this style of interview as you simply don't have the time to build multiple small projects for companies in your limited spare time. Fortunately this interview style is the least common of the three, and rarely used by the top tech companies.

Soft Skills

Don't sleep on soft skills when it comes to finding a job in the industry. Technical skills will certainly help get you the job, but good soft skills will often fast track you should levels and promotions faster than technical skills alone.

What do I mean when I say soft skills? I'm talking about things like positive attitude, the ability to communicate clearly (both spoken and written) and _willingness to share knowledge and help others.

Good mentors and teachers are often looked on very highly by companies, because it shows you can contribute to improving the level of knowledge and skill within the company. If this is something you have interest and experience in (even if it's just something like blogging or videos) make sure that it's something you share during the interview.

Many companies have what's called a cultural interview as one fo the steps in your interview process. While some companies might state they have metrics to quantify this value, the reality is that many simply evaluate you on their "feeling" about how well you would fit in with the company based on the attitude and disposition you display during the interview.

Education

Do I need to get a degree?

Do I need to go to college?

One of the great things about a career in software development is that it does not require you to complete a four year degree or college program to get hired. I can't speak for percentages, but many companies are more concerned with a candidates skill level and ability to produce working software than their educational background.

I don't want to discount how much of a benefit a university or college degree can bring however. If you are someone who has the time and resources, then in the long term a computer science degree will almost certainly be worth the investment you make into it, for a couple of reasons: First is that there are still many companies out there that do look for a degree before hiring a candidate, and second that the fundamental knowledge you gain in computer science will be incredibly valuable in teaching you problem solving skills that translate directly to work you do in the field.

If you do go the college route one of the biggest benefits career-wise are the co-op and internship programs, do not pass up those opportunities if they are available to you, they are one of the most effective ways of breaking through that initial barrier of getting your first job, which most new devs describe as the most difficult hurdle of their career. See How do I get a job when all companies want existing work experience?.

Still, for those that do not have the means to complete a full degree, if you are motivated and disciplined enough it is absolutely possible to teach yourself everything you need to know through a combination of free online resources and building your own practice projects with the knowledge you gain from those resources.

Another popular option these days is bootcamps. They can be a great middle ground between self teaching and a college degree. They are good for those people who feel they may not be able to motivate or direct themselves enough and benefit from having a very strict curriculum to follow.

Bootcamps have a reputation of being very challenging and being one of those things that really gives back the effort you put into it, so make sure you are ready to invest the huge amount of time (and often money) it takes to get the full benefits. Also make sure you do your research on the quality of the bootcamps, you'll find the full array of well regarded ones and poor quality bootcamps. If possible try to find one that also includes help with job placement as part of the package.

Luck

Let's be honest, when you ask many people how they got their first job they'll say "well, I was lucky..." and proceed to tell some anecdote about their good fortunate.

As they say, luck is the intersection of preparation and opportunity. Many of these people who describe themselves as lucky had done preparation in advance that allowed them to take these opportunities when they presented themselves.

Allow yourself to accept that there's always going to be some luck involved, but you can dramatically improve your chances of that luck finding you if you're well prepared.

Which factors most influence potential income?

I've grouped these two questions together as they often go hand in hand. How much you as a candidate appeal to a company has a direct impact on your potential value.

As you can imagine, this is a very popular question, and like many other complex questions, the most accurate answer is going to be "it depends"... but let's take some time to break down as best we can into exactly "what" it depends on.

Location

Not necessarily a huge factor when it comes to getting a job -- but quite possibly the most important factor when it comes to income. Without a doubt, both average and top end software development salaries the U.S. dwarf pretty much every other country.

That's not to say they aren't still well above average in pretty much every country, it's simply pointing out just how much "above" that already high baseline they are in the U.S. You can still live extremely comfortably as a software developer in pretty much any country in the world.

Here's an aggregate by country from Codesubmit showing the different average values between countries. The actual values themselves are less important than the variance between countries.

For developers in the U.S., here is the data directly from the U.S. Bureau of Labor Statistics directly. Similar metrics from the Government of Canada here.

If you're outside North America you should be able to find data from your government's statistics reporting with a quick Google search.

Of course there are tons of factors that go into total income. Cities with significantly higher salaries also have higher costs of living, and there are many other quality of life considerations to take into account as well.

Nevertheless, even when you account for regional cost of living, there's still pretty much no question that developers whose only goal is to maximize income should be aiming to relocate to the U.S.

That said, this particular advice really only applies to the small subset of folks who aren't already in the U.S. and are also young enough and motivated enough to uplift their entire lives.

For the rest of us (myself included in Canada) there are plenty of other factors listed ahead to consider to help maximize potential earnings without the need to relocate. The rise of remote work is (slowly) shifting the balance and bringing salaries up in non-U.S. countries, and there are more opportunities than ever to work remotely for a U.S. company while living in another country.

Product

What kind of product does the company you're applying to develop? Oftentimes in the tech world, depending on the company's market, they either view software developers as "revenue generating" or an "operating cost".

Not surprisingly, companies that view developers as a source of increased revenue tend to pay more than companies that look at devs as a cost.

Take tech companies like Facebook for example. The more developers hey hire, potentially the more features they can release and the more revenue they can generate. Hiring good devs gives them a direct line to more money, and as such they tend to pay more.

Alternatively consider a major restaurant chain. They need developers of course to manage their websites, maybe the software in their restaurants, and any number of internal tools they build to help manage their company, but ultimately a business like that is going to treat devs more as a "cost of doing business" rather than a source of revenue.

So when seeking higher paying jobs, make sure you are considering the company's product. Is the main product or service they offer software based? If so chances are they're going to value their developers higher than a brick-and-mortar business, and you're likely to get a better offer from them.

Size and Funding

Smaller companies for reasons that should be obviously simply cannot afford to pay the big software salaries that larger companies can.

Working for a small company can be a great experience though, you'll often be able to have a much bigger impact on the product and be involved in more major decisions, but in doing so there is often a price to pay income as compared to the larger companies out there.

That said, an exception to this rule can be small startups that are well funded. When a software focused startup company receives a large round of VC funding, often the biggest chunk of that funding is intended to go directly into hiring more staff to scale their product to a wider market -- so get in there and get your piece!

Base Salary vs Equity

This topic is way bigger than I can possibly go into (and I'm no expert either) so I'll simply link this great blog post on the topic

Suffice it to say that you short of guaranteed stock for an already publicly traded company, you are often best to consider any private equity or options as zero value when it comes to negotiating your salary.

Because in the case of startups (early stage ones in particular) most often that's exactly what it ends up being worth.

Having some ownership in the product you work on is great, but please don't let a company take advantage of you with promises of great value "someday" when they do public. Too many great devs have been burned by this. Make sure you get the base annual salary that you need first, no matter how small the company, if their product is solid they should be able to pay their talent. Full stop.

Once you've got that nailed down, then you can comfortably discuss options on top of that.

What other factors should I consider when applying to a company?

Remote Friendliness

Is the company you're applying to remote friendly? Do they allow permanent work from home? Hybrid model?

Remote work has become more and more common. Even for juniors starting out it's not unrealistic to aim for a remote job.

If it's something that's important to you, make sure you are upfront about it with the company before the interview process begins.

Company Culture

Before beginning the interview or even applying to a company, try and gather as much information you can about what it's like to work at the company.

Ask on developer communities like Reddit, Discord, etc if anyone has experience or anecdotes about what it's like to work for those companies.

Check sites like Glassdoor, Blind and Layoffs.fyi to get accounts from people who have worked at these companies.

try to filter out the most extreme positives and negatives from these sites if you can. Companies can and will find ways to get paid positive reviews. Focus on the ones that sound the most like a "slice of life" honest account if you can. Obviously the bigger the company there more you'll find the easier it will be to get an impression.

Finally make sure you ask in your interview as well. Don't just say "what is your company culture like?" because you'll just get a canned answer.

Ask specific questions like "what time do most developers start and finish work each day?" to try get an idea how much team pressure there might be to "work extra" even if it's not officially "required".

Another great question is "what is your training/onboarding process like?". A question like that can give a good idea whether they've really got their act together when it comes to training new hires, or whether it sounds like you'll just be thrown into a project with nothing more than a "good luck!".

There's tons more of course. Really the questions you want to ask are the ones that reflect things that are important to you. They may not be the same things that are important to someone else. Here's a blog post with some more examples. Consider picking 2-3 that best reflect the kind of work experience you are looking for.

What if I find a job, but it's not what I was expecting?

This section of the post is primarily aimed at those developers who have been been able to find work, but unfortunately find themselves struggling with unrealistic expectations, or pressure to work unpaid overtime, lack of support, or at worst -- straight up verbal harassment including insults, mocking or derision.

Let me tell you straight up so there can be absolutely no misinterpretation: GOOD JOBS, AND GOOD COMPANIES, DO NOT TREAT PEOPLE THIS WAY. EVER. PERIOD. FULL STOP.

There are no exceptions to that statement.

Is it possible that maybe you aren't keeping up, or you are struggling? Absolutely! Maybe it could be that you weren't quite ready for thr job you were hired for. That's an unfortunate reality, but it's not a death sentence.

There are ways of approaching this situation that good managers and good companies will be have plenty of experience with.

One thing I read recently that I think is really critical for developers to remember is this: nothing said in a performance review should ever come as a surprise.

If you wake up tomorrow and have a performance review (whether it's an official one, or simply your manager talking about your performance 1-on-1) and you're told that you've failed to meet expectations, what it really means is that your manager has failed at properly addressing it earlier on when they should have (unless they follow that statement with a plan to support you).

A good manager will approach the topic in a positive way, willing to work with you to come up with a solution. Do you need more training time? More time to shadow other developers? Are the tickets you are being assigned currently too far above your level and you need to spend more time tackling smaller ones to build experience?

These are the kind of questions good companies will encourage managers to ask in performance reviews. You are part of the team and that's how you should expect to be treated.

Following a discussion like this, after whatever the agreed upon period is (whether weeks or months) if you still are unable to work out an approach that helped you work at the level expected of you -- then maybe the tough discussions might start about your fit for the role.

But the key point being made here is that it should never, ever, come as a surprise.

If what I am describing here sounds totally unrealistic or like some kind of "dream company," and you find yourself in a position where you think that mistreatment and unrealistic expectations are the norm -- then I'm happy to tell you that you simply have Stockholm syndrome (I mean that in the nicest way possible).

There are no shortage of companies out there that don't treat people this way.

There has never been a better time to be a developer. I would encourage you to polish off the resume and start your search for that career you deserve.

There's not too many incredibly strong opinions I hold about the industry, but the belief that no matter your skill level that everybody deserves to be treated and spoken to with respect is one of the few complete dealbreakers for me. I'll put up with a lot of stuff, but that's where I draw the line. I think everybody should.

The more we put our foot down for behaviour like that, the more companies will be forced to address it.

Negotiating an Offer

A lot of people will tell you there is no harm in negotiating an offer. That's not exactly true, but it's still good advice.

There's really two most likely scenarios that will occur when you try to negotiate an offer:

The company will accept or counter (good)
The company will stay firm at current offer (fine)

Those are the most likely scenarios and since both are options in your favour, that's the reason most people will encourage you to negotiate.

There is a rare third scenario where the company rescinds the initial offer entirely which can happen -- but the most oft repeated advice to that is that "a company that does that probably isn't a company you want to work for anyway."

Whether you should negotiate or not really is a personal choice. I think there are good reasons and scenarios where you might choose not too (the offer is in the range you were expecting and the company is one you are interested to work for.) I personally would never fault someone for accepting an off as-is in that scenario.

Other factors to consider is the amount and your leverage.

A good rule of thumb is 10%. Most companies won't be too surprised if a candidate counters at 10% above the offer that was made to them. If they company is genuinely interested in you, 10% if probably a small price to pay to close the deal.

More than that and you should probably make sure you are confident in yourself and have leverage.

Leverage is basically evidence to show value. The further you are in your career the more track record of proven value you will have and the more leverage you will have. Negotiating up as a new graduate with no experience is possible, but much riskier.

Another way to get leverage is having multiple offers. If you are fortunate enough to have more than one offer from more than one company, and the one you would prefer to work for has offered less, you can use the other offer as leverage in a (very friendly and professional) communication that you have another offer, but would prefer to work for the other company if they would be able to match $X amount.

The final point to note is that when you negotiate or try and leverage multiple offers against each other you have to be willing to lose the offer. That's part of the risk. If you aren't willing to walk away, don't take the risk.

Pay Bands

Make sure you are as familiar as possible with the pay bands (salary range) of the role you are applying to at the company your are interviewing with.

Remember that it's in your best interest not to provide this range yourself as you are much more likely to undersell yourself. Any respectable company or recruiter should be able to provide you with the potential pay range for the role you are applying for, though it may take some pressure on your end to get it out of them, it's well worth it.

If they will not provide you with a range in advance it may be in your best interest to walk away unless you are desperate for the job. Interview processes can be very lengthy and time consuming and there is nothing worth than getting to the offer stage and finding out you had a completely different expectation than the company did.

Be sure to educate yourself in advance using tools like levels.fyi. Remember that software developer pay range is extremely location dependent so you need to ensure you are filtering not only by the company you are applying to, but also your location.

Although levels is the best indicator, particularly for tech companies, you should also make sure to check Glassdoor and Blind as well just to get the most well rounded number you can.

DSA, Leetcode and Systems Design

If your goal is to maximize your income, then you've arrived at your destination.

I mean, full disclosure -- I don't work for a FAANG company. I haven't done the grind myself and don't intend to. I work for a great company and could hardly ask for a better career, but I am familiar with the requirements and can certainly point folks in the right direction for all the resources they need to study to take that leap.

DSA stands for data structures and algorithms.

If you studied computer science in college/university then this is probably old news to you, but if you're someone who has been just learning web development on their own in their spare time it might seem a bit overwhelming. And that's okay! Let's break it down.

In order to understand why DSAs are important you need to kind of take a step back and look at the big picture of software engineering as a whole rather than web development in particular.

Regardless of whether you've learned Javascript, or Python, or C++, or Java, or Rust, or Ruby, or C# or whatever -- each programming language when you boil them down to their core elements is really just a different flavour of approaching the same topic: representing and manipulating data.

How that data is stored covers the data structures part and how it is manipulated falls into the algorithms.

Big tech companies like the FAANG/MANGA crew and similar monoliths are in a very unique position. The small mom and pop shops in your local area have to do a lot of work to seek out developers that can both do all the odds and ends dev work that needs to be done for average to below average pay -- but the big bois can sit back while every new grad and they dog climb over themselves to try to work for them.

This endless supply of potential dev talent means they have the luxury of being a lot pickier about who they hire than most companies.

But how do they decide how to filter out the best of the best?

Well as it happens, they've all done their research, and it seems like the most consistent way of identifying top talent in the software industry isn't necessarily based on work experience or knowledge of any one particular programming language -- it's a candidate's base understanding of fundamental computer science topics like data structures and algorithms.

The idea behind it is that if you can understand the thought processes behind the underlying concepts, the actual language you write the code in is pretty irrelevant.

It's much easier to teach a strong logical thinker how to write Python than it is to teach a Python developer how to think.

So with that background established we've paved the way for the birth of platforms like Leetcode to exist, which despite the stupid name, is unquestionably the most important resource available to you as a developer if your long term goal is to maximize your income.

All the top paying tech companies right now like Amazon, Apple, Facebook, etc draw most of their engineering interview questions directly from the pool of questions listed on this site. Whether similar and adjusted slightly, or in many occasions almost identically word for word.

There's no tricks or cheating involved in having access to them in advance. The whole idea is that they are challenging enough to require weeks or months of practice to internalize the concepts. If you are able to do so successfully then it demonstrates to these companies that you are capable of a level of logical thinking that they expect, and so having the solutions practices and even memorized is enough to identify you as a valuable candidate.

This is what's called the Leetcode grind, and as painful as it can be, it's widely considered to be the most valuable way you can spend your time if your goal is maximizing your income. Grind out a few problems per day, work your way up into the mediums and hards, and you'll be ready for your FAANG interview in no time.

For a new grad or someone with no experience, this is often enough to get your foot in the door. For those developers targeting the higher level positions at the big tech companies you'll also need to spend time studying system design.

Systems design in a nutshell is simply an outline of all the factors you need to consider when building a large scale software system. Consider a trivial example of a website. I make a website with HTML, CSS and Javascript and host it online with Github pages, or purchase some web hosting with Godaddy and upload my files. People love it! Every day I get more and more visitors.

Tomorrow I log on and my site is down. It got too popular! Turns out I only had 1GB of bandwidth with my hosting provider and I blew through it overnight when my site went viral. Damn. How big were the images I was hosting? What was my caching setup? What were my plans for scalability? I didn't have any?

Well no wonder everything fell apart.

In a systems design interview you'll get an example of some kind of business idea or product that might need to be built. You'll be expected to be able to discuss as many factors or considerations that need to be accounted for in building that system.

You'll need to discuss both functional requirements (what features does this product need to support?) as well as infrastructure (how can we handle potentially large amounts of traffic and data storage requirements?).

What solutions are available? What are the pros and cons of each one?

If you're still not sure exactly what we're talking about, here's a couple classic examples based on real interview questions you might encounter on this subject:

These videos will give you a great idea of how you will be expected to answer questions like this in a big tech interview.

Lastly, before moving on I should like you with a couple of the foremost standard resources in this space. The book Cracking the Coding Interview is considered the gold standard. It's not free unlike most other resources I've linked, but since you're presumably only here because you're looking to maximize your earning potential, $40 is probably a pretty small price to pay to get you there.

For systems design there are great books too, but honestly YouTube is pretty much your best friend for this one. It's really great to just listen to people walk through their thought process for different problems. Just watch to avoid the crypto shills like TechLead which I will not link out of principle.

A simple rule of thumb to follow is that if the person is only speaking about the topic and answering exactly the subject you searched for, you're probably in good hands. If they're starting or ending their video encouraging you to "check out" something totally unrelated you should proceed with extreme caution.

Good luck! Let me know in a comment if you manage to land a sweet gig :D

More Learning Resources

What resources should I use to learn web development?

My personal #1 recommendation to anyone trying to get into web development as a career would be to go through The Odin Project

It is an absolutely incredible resource that teaches all of the fundamentals of web development (including HTML, CSS, Javascript and Git) and more in a completely free digital environment.

They even have a Discord server for users to help each other out while going through the curriculum.

If you go through the entire Odin Project from start to finish and complete every step and every project I genuinely believe you will be in a position to start applying for jobs. It uses all the same technologies that real companies use and mirrors the kinds of tasks and challenges you would find in a real professional web development position.

In addition to the Odin Project, Free Code Camp is also very highly recommended as a free resource to learn web development.

What are the best free resources for learning HTML
What are the best free resources for learning CSS
What are the best free resources for learning Javascript
What are the best free resources for learning Git
What are the best free resources for learning Databases

What are the best free resources for learning computer science?

If you are interested in learning about the more generalized field of "software development" then there are some absolutely incredible free resources for that as well.

Anything that you learn about the development in general will almost certainly help propel your understanding of web development as well. The great thing about software is that ultimately most of the concepts are the same regardless of which specialty you choose.

Learning how to sort an array in C++ or Python will use all the same general concepts as sorting an array in Javascript. Learning how to do it in one language one will make learning it in the other significantly faster. Often you'll even learn tips and tricks that you might not have been exposed to working exclusively in one language.

The gold standard free resource for getting introduced to programming and software development is Harvard's free CS50 course. Seriously, watch the preview video to get an idea, it's absolutely fascinating subject material and extremely well presented. I would recommend this course to anyone who was thinking about getting into development.

After that we have the incredible OSSU which is an entire university level computer science available free online. Once again, this is one of the most amazing resources you will ever encounter, and entirely free.

We also have teach yourself CS which is very similar to OSSU in that it aims to be a complete computer science education.

Lastly, I'd like to also share two other great resources for people interested in the field of software development, depending on where you interest lies:

If you are interesting in using software and programming to solve practical everyday problems, check out Automate the Boring Stuff with Python. After completing this if you're working a white collar job using Excel in an office, there's a good chance you'll find you can automate away half the stuff you do on a daily basis.
The other is for people who are more interested in the "how does software work at the lowest level". It's a project called Nand to Tetris and it basically goes through the process of going from electronic hardware logic gates to a working game of Tetris. Very involved and complex, but incredibly rewarding.
If you are interested to go even lower, down to the bare metal, then you'll definitely want to check out this incredible video series from Ben Eater on building an 8-bit computer from scratch. If you complete a big project like this you'll learn more about computers than any single college course or video lecture will be able to teach you.

Simple Website Template

The following is a simple example of a complete website that uses HTML, CSS and Javascript. These files are used as the basis of a lot of examples in this tutorial such as using Github, website hosting, and learning HTML/CSS/JS.

The purpose of this section is to link to in those other parts of the tutorials to copy these files, but it may also serve as an example of a simple website setup that shows some of the fundamental concepts of HTML, CSS and Javascript.

Feel free to use it for any part of your learning or projects in any way that you like.

It consists of three files called index.html, script.js and style.css which are all intended to go into the same folder/directory. The contents of each of those three files is as follows:

index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>My Animal Blog</title>
    <link rel="stylesheet" href="style.css" />
    <script defer src="script.js"></script>
  </head>

  <body>
    <header>
      <h1>My Favourite Animals</h1>
      <h2>Enjoy these furry friends</h2>
    </header>
    <main>
      <article class="card">
        <img
          src="https://res.cloudinary.com/dqse2txyi/image/upload/v1657421273/blogs/intro-to-web-development/cat_k4fcww.png"
          alt="Cat Playing Chess"
        />
        <div class="card__container">
          <h3>Chess Cat</h3>
          <p>He's planning his next move.</p>
          <button onclick="likeButton()">Like</button>
        </div>
      </article>
    </main>
    <footer>
      <p>&copy; 2022</p>
    </footer>
  </body>
</html>

style.css

body {
  color: #333333;
  font-family: Helvetica, sans-serif;
  margin: auto;
  max-width: 800px;
}

main {
  margin-bottom: 40px;
}

header,
footer {
  text-align: center;
}

h1 {
  font-family: Arial, sans-serif;
}

h2 {
  font-style: italic;
  font-weight: lighter;
}

.card {
  width: 350px;
  margin: auto;
  box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2);
  border-radius: 10px;
  overflow: hidden;
}

.card > img {
  width: 100%;
  height: auto;
}

.card__container {
  padding: 16px;
}

script.js

var likes = 0;

// Adds 1 to the number of likes, and updates the text of the button
function likeButton() {
  likes = likes + 1;

  var myButton = document.querySelector("button");
  myButton.innerText = "👍 " + likes;
}

When placed together and hosted with a web server, this will be the output when you visit the URL:

Conclusion

I hope you found this guide useful and informative. It's been extremely challenging to try and decide the right format to present it so that it can be helpful to as many people as possible.

If you have any suggestions or feedback please leave a comment behind. Despite the length and scope of this guide I still consider it to be a work in progress and am very open to any ideas for improvement.

Lastly, to everyone, I wish you good luck on your learning journey. Learning a new skill, especially one as complex as software development, takes a lot of time. Go easy on yourself, and remember that the goal is not overnight success, but rather a little bit of progress each day.

All the best,

~ Alex

How to Build a Fullstack Next.js App (with Storybook & TailwindCSS)

Alex Eagleson — Thu, 21 Apr 2022 23:22:33 +0000

All code from this tutorial as a complete package is available in this repository.

If you find this tutorial helpful, please share it with your friends and colleagues! For more like it you can subscribe on Youtube or follow me on Twitter.

This tutorial is available as a video lesson if you prefer that format:

Prerequisites
Introduction
Adding Tailwind
Storybook Support for Tailwind
Scoping and Requirements
Front End Planning
Front End: Search Component
Front End: Header and Footer
Front End: Layout
Front End: Results
Back End Planning
Back End: Search Data
Back End: API Routes
Static and Dynamic Pages in Next.js
Front End Finishing Touches
Themes and Design Systems
Next Steps
Wrapping Up

Prerequisites

IMPORTANT: This tutorial is a continuation of a previous tutorial.

If you wish to align the repository with the start of this tutorial, clone the repository and git checkout 6630ca95c25e66d7b6c7b1aad92151b481c1b9c5.

After you check out that commit, create a new branch to follow along with this tutorial. An example would be something like git branch fullstack-tutorial and then git checkout fullstack-tutorial.

It should be possible to follow this tutorial with a new blank project if you choose without all the configuration from the previous setup, but I would recommend you at least read through the article to understand the project architecture before we get started.

If you wish to try starting from a fresh Next.js project, run the following commands to set up the core project:



npx create-next-app --ts

Then you will also want to install Storybook. Follow these instructions in a new project to get aligned with the beginning of this tutorial.

We also create all of our components off a base template that includes styles, stories and mock data. You can get that template from here.

Good luck, and I hope you enjoy this tutorial.

Introduction

This tutorial is the second in a series about building scaleable Next.js architecture.

In the first installment, we focused entirely on the base project setup, we didn't actually begin building an application, just a simple component template to show the process.

In this next stage we will be looking at actually building out an application. We'll be looking at how Next.js handles some fundamental things like routing, image optimization, static vs dynamic pages, building an API, and of course: styling solutions.

We'll be using the current "hot commodity" Tailwind CSS as the tool we use to organize our design system, and get styles implemented quickly while maintaining a consistent look and feel to the product.

Finally and maybe most importantly, this tutorial is also focused on trying to replicate the real software development process. So we won't just be jumping into building, we'll be looking at what the requirements are based on our goals, what the scope of the project should be, and planning out in advance how we are going to build both the front end and back end.

By the end of the tutorial our goal will be to have a functional full-stack Next.js app that we can push to a production site and continue to iterate on in the future with a team of developers following a consistent system.

If that all sounds good to you, let's jump right in!

Adding Tailwind

Tailwind CSS describes itself as:

A utility-first CSS framework packed with classes like flex, pt-4, text-center and rotate-90 that can be composed to build any design, directly in your markup.

So basically it's a way to enforce a bit of consistency and convenience, while also placing most of your styles closer to the components you're developing.

Tailwind's compiler will analyze all your code and only bundle raw CSS based on the classes you actually use, so it requires some dependencies to get up and running.

Before we get started I would very highly recommend the Tailwind CSS IntelliSense extension for VS Code. It gives you autocomplete for Tailwind styles, shows you the actual CSS values being applied, integrates with your custom theme, and generally and makes working with Tailwind so much smoother.

Now, let's begin by running the following commands in the root directory of our project:



yarn add -D tailwindcss postcss autoprefixer

Tailwind will compile into regular CSS for your final build so there is no need for it to exist as a runtime dependency in your project.

postcss and autoprefixer are tools for transforming CSS that Tailwind uses to do its job.

After Tailwind has been installed, we need to initialize it.



npx tailwindcss init -p

This will automatically create a postcss.config.js file for you. In addition to that you also need to create a tailwind.config.jsfile in the root of the project. One might get created by default as well. Its contents should include:

tailwind.config.js



module.exports = {
  content: [
    './pages/**/*.{js,ts,jsx,tsx}',
    './components/**/*.{js,ts,jsx,tsx}',
  ],
  // Ensure these match with .storybook/preview.js
  theme: {
    screens: {
      xs: '375px',
      sm: '600px',
      md: '900px',
      lg: '1200px',
      xl: '1536px',
    },
  },
  plugins: [],
};

Notice the pattern I used above is aligned with our /components and /pages directories. These are the only places I am planning to place React components (and therefore Tailwind styles since they are written on the components).

If you plan on adding more top level component directories in the future make sure you update this config.

We're almost ready to test it. We just need to add a few default baseline values to our global.css file. At this point I'm going to move it to the /pages directory because we will be building this app entirely with Tailwind and will not have any need for a global styles directory. (Note you may also need to update the import in .storybook/main.js if you do this).

If you choose not to use Tailwind, you can either keep the styles directory, or even still choose to remove it and keep your .modules.css (or SCSS, or styled-components) next to the components themselves.

Take special note of the @tailwind values at the top.

pages/global.css



@tailwind base;
@tailwind components;
@tailwind utilities;

You can remove any other browser normalizing CSS you had in global, Tailwind will take care of that for you.

I have also updated our index.tsx to get rid of Home.module.css and deleted that file:

pages/index.tsx



import CatCard from '../components/cards/cat/CatCard';
import { mockCatCardProps } from '../components/cards/cat/CatCard.mocks';
import PrimaryLayout from '../components/layouts/primary/PrimaryLayout';
import SidebarLayout from '../components/layouts/sidebar/SidebarLayout';
import { NextPageWithLayout } from './page';

const Home: NextPageWithLayout = () => {
  return (
    <section className="bg-gradient-to-r from-cyan-500 to-blue-500">
      <h1>
        Welcome to <a href="https://nextjs.org">Next.js!</a>
      </h1>
      <CatCard {...mockCatCardProps.base} />
    </section>
  );
};

export default Home;

Home.getLayout = (page) => {
  return (
    <PrimaryLayout>
      <SidebarLayout />
      {page}
    </PrimaryLayout>
  );
};

Now let's test to make sure Tailwind is installed and configured properly.

Notice that className on the section component in the above home page? That's tailwind right there, essentially just quick shorthands for the CSS properties you're already familiar with.

Without Tailwind installed and configured they won't do anything, but with Tailwind we should see a blue/cyan linear gradient background.

The nice thing is that Next.js will handle all the build process for you, you don't even have to think about it. Just start up your dev server (you may need to reboot to pick it up if it was already running):



yarn dev

And go to http://localhost:3000.

Looks like everything is setup. We only have one problem, if you try and run Storybook you're not going to see your styles. Your Next.js is setup to process your Tailwind classes, but by default Storybook is not.

Storybook Support for Tailwind

If you don't have Storybook installed and configured already, remember to read the prerequisites section of this guide.

Start by adding the PostCSS addon for Storybook:



yarn add -D @storybook/addon-postcss

OPTIONAL: If you want to keep using CSS modules as well:



yarn add -D storybook-css-modules-preset

Then update your .storybook/main.js file to:

.storybook/main.js



module.exports = {
  stories: ['../**/*.stories.mdx', '../**/*.stories.@(js|jsx|ts|tsx)'],
  /** Expose public folder to storybook as static */
  staticDirs: ['../public'],
  addons: [
    '@storybook/addon-links',
    '@storybook/addon-essentials',
    '@storybook/addon-interactions',
    'storybook-css-modules-preset',
    {
      /**
       * Fix Storybook issue with PostCSS@8
       * @see https://github.com/storybookjs/storybook/issues/12668#issuecomment-773958085
       */
      name: '@storybook/addon-postcss',
      options: {
        postcssLoaderOptions: {
          implementation: require('postcss'),
        },
      },
    },
  ],
  framework: '@storybook/react',
  core: {
    builder: '@storybook/builder-webpack5',
  },
};

I've just added our blue/cyan gradient to the BaseTemplate.tsx component to test in Storybook to ensure it's properly compiling Tailwind styles (I removed the class again immediately after the test).

Time to commit our progress with git commit -m 'feat: implement tailwind css'

If you want to align with this step of the tutorial, clone the repository and use git checkout 6630ca95c25e66d7b6c7b1aad92151b481c1b9c5.

Scoping and Requirements

One thing I would like to do with this tutorial is cover, at least at a very high level, the general software development lifecycle. Obviously this topic can span full posts and entire books, but I think it's important to touch on the concepts especially for those devs following along with the tutorial who may not have the existing experience working with real projects in the industry. That is one of the goals of this tutorial series.

So with that in mind I am going to treat it like a real project.

First I need to ask the client (in this case the client is myself): What are your goals? What are you trying to achieve?" It's possible (though very unlikely) that once discussed in detail, this challenge can actually addressed without building new software at all. Perhaps there is an existing tool out there already built that fits their needs that they are not aware of?

In our scenario my goal is to "teach people about building applications in Next.js". Alright. I think it's fair to presume I'm going to need to build a Next.js application to deliver on that goal.

It turns out that I (the client) has a list of particular topics that I would like to teach readers about as part of this tutorial. They are concepts that nearly everyone building a professional Next.js app will encounter in the process of development

Must haves:

Styling
Routing
API routes
Static and dynamic pages
Image optimization

Nice-to-haves:

Sharing state between routes
Authentication
Internationalization
Unit and end-to-end testing
Data persistence (database)

Notes: The two separate footers are not required. Just one (showing location) is enough.

Great. That really helps me decide how I am going to scope the project.

Immediately out of the gate, since I am writing multiple blog posts I am going to assign all the "nice-to-have's" into the Phase 2 of the project (in our case, future blog posts). The scope of Phase 1 will include all the "must have's".

But what kind of project will I build to meet those requirements? I'm looking for the minimum viable example I can choose which will allow me to demonstrate each of those and meet the client needs without going over time and budget.

After spending some time reviewing popular sites out there to get ideas, I have decided that for this tutorial we are going to make a very simple Google clone.

Why? Well let's review the requirements:

Styling (Google has a simple design, we'll use Tailwind CSS to recreate it)
Routing (we'll demonstrate two routes, the main "home" page and a "results" page)
API routes (we'll use the fetch API to query for some mock search data with an API route)
Static and dynamic pages (main page can be static, search page dynamic based on search query)
Image optimization (the Google logo)

Excellent! We have our requirements and scope, and now we are ready to work.

Front End Planning

Before we dive in and start making components, let's take a bit of time to look at the whole project holistically and get an idea what components we are going to need. Normally this is likely where you would involve your designer in your process and use an industry grade tool like Figma to plan out and design the components you are going to need before you even begin thinking about code.

Lucky for us we already have the best design we could possibly ask for: a fully interactive one accessible at https://www.google.com.

So we'll give the designer a break on this project and tackle it ourselves! I still want to get an idea what components I'm going to need so let's take a look at the two main pages we want to create and get an idea what the components are, and build a mental model of which pieces of it are re-used in multiple places.

(Note when I say "components" here in this context, I'm talking about the general concept of components, like the individual parts that something is composed of. I haven't gotten to the React-specific code "components" yet)

So you can see in the above I've isolated at least a few components at minimum:

Layout (likely need home and results variants)
Search (the functional part including the input, will be a form)
Nav (both header and footer variants, only difference being the background colour and top/bottom position. The elements can be child components)
Search Result (the structure of and typography of everything that does into rendering one result of a search, including title, text, url, etc)

The above is just one possible approach of a near infinite number even for something as simple as this. This is the project design stage and there honestly is no one right answer on exactly how to do it. Most people find in there career after a few years of getting the coding down, this is the stuff that ends up being the real challenge.

A good app will have the team spending much more time getting the design and plan in place, so that the absolutely minimum amount of coding needs to be done to achieve that goal. The coding and development stage is usually not only the most expensive, but it's also the most costly and complex to "undo" if requirements were not correct the first time.

I'll stop short of getting into the bureaucracy of it, because of course the reality is never this cut and dry, but hopefully you can see what I'm getting at. If at all possible, do it once, do it right, and be consistent. Other developers (and your future self) will thank you.

With that out of the way, I think we're finally ready to begin development on the front end components!

Front End: Search Component

We will be doing ALL of our component designing and testing in Storybook.

You'll find that will be a recurring theme in our development process. It's a great way to make sure that the components we build look correct in isolation, so we can validate that without interference from other parts of the app, and then place them into our app afterward once they have been verified.

For this reason I actually have the flexibility to start working on whichever component I like. I'm going to begin with the Search component first.

Create a new directory called /utility inside /components. As before, we'll start by copying our templates/base into the components/utility directory to start our component.

If you are unsure what I am describing, you can refer back to the original tutorial where we created the BaseTemplate component, or simply take it from the project repo.

Run a find and replace for each instance of BaseTemplate and replace with Search in the copied folder, including both the content of the files and the filenames themselves. Lastly change the title in Search.stories.tsx to utility/Search. When you are done it should look like this:

And in Storybook with:



yarn storybook

(You may still have some lingering Tailwind test styles on the template which can be removed. Note also that I am leaving the .module.css template on here for those who choose not to use Tailwind, but we will not be using it in this tutorial)

Alright time to begin building the component! This is the one I've outlined in green in the original planning design above and titled as Search.

Search Step 01: HTML Structure

I'm going to begin with just the HTML structure, no styles or function logic. The "Search" button and input implies I'm going to want a form.

components/utility/base/Search.tsx



export interface ISearch {}

const Search: React.FC<ISearch> = () => {
  return (
    <form>
      <input type="text" />
      <button type="submit">Google Search</button>
      <button type="submit">I&apos;m Feeling Lucky</button>
    </form>
  );
};

export default Search;

Look at that Search component, pretty incredible eh? Hit the submit button in storybook and get an error since you don't have a backend to handle it. I'd say it's basically done... well maybe not.

I'm happy with the structure though, function-wise it's got everything we need. Let's do the styling next to get the look and feel up to speed.

Search Step 02: CSS Structure

If you're not familiar with Tailwind CSS I recommend you take a read through their documentation first to get a good feel for the syntax. If you're experienced with CSS you should find it very easy, for the most part it's just convenient shorthands. Just use the search bar of ctrl + F to quickly find the Tailwind version of what you need.

Full disclosure: I've been using Tailwind now for a grand total of about... 48 hours. It's brand new to me too! But I'm acknowledging that, not as a negative, but as a positive to show how straightforward it is to learn when you already have the fundamentals down.

I chose Tailwind for two reasons: ease of development (get styles in place quickly) and consistency (the base theme and pre-set values help ensure that the different parts in our app will look and feel the same).

Now with all that said, let's start adding those classes! Here's the same component as above, just with some Tailwind styles added (and a wrapper element for the buttons).

components/utility/base/Search.tsx



export interface ISearch {}

const Search: React.FC<ISearch> = () => {
  return (
    <form className="flex flex-col items-center gap-y-5">
      <input
        type="text"
        className="rounded-full border-2 w-5/6 sm:w-96 h-12 px-3"
      />
      <div className="space-x-3">
        <button
          type="submit"
          className="border-0 p-2 px-6 bg-slate-100 rounded-md"
        >
          Google Search
        </button>
        <button
          type="submit"
          className="border-0 p-2 px-6 bg-slate-100 rounded-md"
        >
          I&apos;m Feeling Lucky
        </button>
      </div>
    </form>
  );
};

export default Search;

We can abstract those repeat classes on buttons out to a separate @apply directive to avoid repeating yourself.

Note: please read through Tailwind's extremely good documentation on this concept because it discusses how in a lot of cases the @apply solution can actually reduce future maintainability, so you just want to make sure it's the right decision first.

I'm using it here because I just want you to be aware of it and how to do it, and secondly they use an example of a global button style as one of the times that it should be used, so I feel confident using it in this example.

We just need to remove those repeat button styles and put them into pages/global.css and replace with an actual class name like so:

pages/global.css



@tailwind base;
@tailwind components;
@tailwind utilities;

@layer components {
  .btn-primary {
    @apply border-0 p-2 px-6 bg-slate-100 rounded-md;
  }
}

components/utility/base/Search.tsx



export interface ISearch {}

const Search: React.FC<ISearch> = () => {
  return (
    <form className="flex flex-col items-center gap-y-5">
      <input
        type="text"
        className="rounded-full border-2 w-5/6 sm:w-96 h-12 px-3"
      />
      <div className="space-x-3">
        <button type="submit" className="btn-primary">
          Google Search
        </button>
        <button type="submit" className="btn-primary">
          I&apos;m Feeling Lucky
        </button>
      </div>
    </form>
  );
};

export default Search;

Excellent. Our Search component is finally ready visually (I've opted not to use the magnifying icon as it is embedded within the input element which makes the CSS a bit more complex than the intended scope of this tutorial.)

Try using the screen size button within Storybook (you can see it set to sm in the screenshot) to test at different mobile breakpoints. Notice we used the default 5/6 width on the input but set to sm:w-96 once the screen begins to stretch to keep it from getting too large.

Simplifying responsive design is one of the things Tailwind really excels at.

Search Step 03: Logic and State

The last piece is to implement the management of the search state (basically keeping track of what the user has written so far).

The easiest way to do that is with the useState hook.

(Reminder once again that this is not a React tutorial, if you are not familiar with useState then you have potentially jumped the gun into Next.js a little too quickly. Not to worry! Shouldn't take you long to pick up, the new React documentation focused on hooks is probably the best way to learn straight from the source)

components/utility/base/Search.tsx



import { useState } from 'react';

export interface ISearch {}

const Search: React.FC<ISearch> = () => {
  const [searchTerm, setSearchTerm] = useState<string>();

  return (
    <form
      className="flex flex-col items-center gap-y-5"
      onSubmit={(e) => {
        e.preventDefault();
        alert(`Action requested. Search for term: ${searchTerm}`);
      }}
    >
      <input
        type="text"
        className="rounded-full border-2 w-5/6 sm:w-96 h-12 px-3"
        value={searchTerm}
        onChange={(e) => setSearchTerm(e.target.value)}
      />
      <div className="space-x-3">
        <button type="submit" className="btn-primary">
          Google Search
        </button>
        <button type="submit" className="btn-primary">
          I&apos;m Feeling Lucky
        </button>
      </div>
    </form>
  );
};

export default Search;

The above will allow you to track and react to changes in the search form on the searchTerm variable. I've also added a Javascript-based form handler (as opposed to the default HTML behavior) so we can use it later if we need it. The preventDefault steps the normal form submission behavior of making a POST to the server from occurring.

At this point we are not sure if the search term might need to be managed elsewhere in the app (other components might need to be able to read it) or how we are going to submit the form. Normally that would be part of the planning process and I would know before writing code, but I am including this default behavior here to show as an example how we will refactor later if needed.

This completes our Search component for now until we know more about what we want to do with it. Aside form the alert() it appears to do everything we need it to do, and renders without visual issues on all breakpoints, so we can consider that done for now (normally you'd update your ticket and submit to QA for approval that the execution matches the design).

Time to commit our progress with git commit -m 'feat: create Search component'

If you want to align with this step of the tutorial, clone the repository and use git checkout 676a71b50755d859f46a12e54f8ea3484bf1f208.

Front End: Header and Footer

We're gonna kick up the speed a bit here to get the basic remaining components in place.

I've decided to build the Header and Footer as separate components for the time being. There is definitely behavior that is shared between them that could be abstracted into is own component (links/buttons in a row separated on each side of the screen horizontally with flex's space-between).

However there is still a lot that's unique, the content for sure, the position and the background colour. Enough that I have decided to separate them for the sake of simplicity in this demo.

Let's get to building.

Remember in each case we are using the BaseTemplate. For Header the Story title is navigation/Header.

components/navigation/header/Header.tsx



import Link from 'next/link';

export interface IHeader extends React.ComponentPropsWithoutRef<'header'> {}

const Header: React.FC<IHeader> = ({ className, ...headerProps }) => {
  return (
    <header
      {...headerProps}
      className={`w-full flex flex-row justify-between ${className}`}
    >
      <div className="space-x-5 m-5">
        <Link href="/">
          <a className="hover:underline">About</a>
        </Link>
        <Link href="/">
          <a className="hover:underline">Store</a>
        </Link>
      </div>
      <div className="space-x-5 m-5">
        <Link href="/">
          <a className="hover:underline hidden sm:inline">Gmail</a>
        </Link>
        <Link href="/">
          <a className="hover:underline hidden sm:inline">Images</a>
        </Link>
        <button className="border-1 p-2 px-4 sm:px-6 bg-blue-500 rounded text-white">
          Sign In
        </button>
      </div>
    </header>
  );
};

export default Header;

A cool feature of the above is that the Gmail & Images links disappear on the smallest screen size. In the real app we would have a menu that includes those items so they are not inaccessible on mobile, but on larger screens we get handy shortcuts to them.

Another thing you'll notice is the special <Link /> component provided by Next.js as an alternative to the <a> anchor tag. These links are required in order to maintain stage between routes in Next which we'll get to in a little while. Learn more about it here.

Now we move onto the Footer.

components/navigation/header/Footer.tsx



export interface IFooter extends React.ComponentPropsWithoutRef<'footer'> {}

const Footer: React.FC<IFooter> = ({ className, ...footerProps }) => {
  return (
    <footer
      {...footerProps}
      className={`w-full p-5 bg-slate-100 text-slate-500 ${className}`}
    >
      <p>Canada</p>
    </footer>
  );
};

export default Footer;

We were told in our requirements that only one footer is required. Right now we have the value hard coded as Canada, but we can return to that later. Just focusing on style for now.

Front End: Layout

Presuming you've been following up with the previous blog / tutorial you will already have a layout component in place in components/layouts/primary/PrimaryLayout.tsx. This is important because we already set that layout up to persist between page routing so it doesn't reload the same layout and nav bar when you transition from one page to another.

One that note you can delete components/layouts/sidebar entirely, our new Header and Footer will replace that. Remember to delete it elsewhere in the code where SidebarLayout is imported. You can also delete pages/about.tsx for the same reason. It was just an example to show routing and no longer required in our app.

As for PrimaryLayout.tsx we will update it as follows (first remove or just blank out PrimaryLayout.module.css) then:

components/layouts/primary/PrimaryLayout.tsx



import Head from 'next/head';
import Footer from '../../navigation/footer/Footer';
import Header from '../../navigation/header/Header';

export interface IPrimaryLayout {}

const PrimaryLayout: React.FC<IPrimaryLayout> = ({ children }) => {
  return (
    <>
      <Head>
        <title>NextJs Fullstack App Template</title>
      </Head>
      <div className="min-h-screen flex flex-col items-center">
        <Header />
        <main>{children}</main>
        <div className="m-auto" />
        <Footer />
      </div>
    </>
  );
};

export default PrimaryLayout;

With our layout in place, we are ready to build the actual home page.

The way that Next.js handles routing is super simple and straightforward out of the box. Similar to a traditional webserver all you need to do is create directories.

The directory structure you create will match the path structure of your site and the page it loads is simply the index.tsx inside that directory, same as a webserver would look for an index.html by default.

For our home page accessible at the base / route of our site, we simply use pages.index.tsx. We already have the Header, Footer, Search components, and layout created, so all the home page needs to do is put those together and add the logo & language toggle link.

pages/index.tsx



import Image from 'next/image';
import Link from 'next/link';
import { useRouter } from 'next/router';
import PrimaryLayout from '../components/layouts/primary/PrimaryLayout';
import Search from '../components/utility/search/Search';
import { NextPageWithLayout } from './page';

const Home: NextPageWithLayout = () => {
  const { locale } = useRouter();

  return (
    <section className="flex flex-col items-center gap-y-5 mt-12 sm:mt-36">
      <Image
        src="/Google.png"
        alt="Google Logo"
        width={272}
        height={92}
        priority
      />
      <Search />
      <p>
        Google offered in:{' '}
        <Link href="/" locale={locale === 'en' ? 'fr' : 'en'}>
          <a className="underline text-blue-600"> Français</a>
        </Link>
      </p>
    </section>
  );
};

export default Home;

Home.getLayout = (page) => {
  return <PrimaryLayout>{page}</PrimaryLayout>;
};

(Note that I have downloaded this version of the Google logo from its Wikipedia page, named it Google.png and place it in the root public directory of the project)

There's two new Next.js specific components showcased here that I'd like to cover:

Link - Next provides a special kind of link that is used as a superpowered version of the <a> anchor tag. You still use the anchor tag, but by wrapping it in a <Link> with the href, Next will handle a click to that link in a special way that preserves state in your application without a full page load and refresh (among other benefits described in the docs)

We have also taken advantage of the locale value in the useRouter hook to handling efficiently toggling back and forth between locales. Try it yourself (you'll need to run the yarn dev server to test it since you won't have access to routing in Storybook), but it works great for toggling back and forth between languages.

Remember that our app's available locales can be customized in next.config.js on the i18n field. Right now we don't have any translation in place, so only the URL will switch (updating the text copy for i18n support will be a topic of a future tutorial.)

Image - Image handling in web development is surprisingly complicated, and as such, Next has created a special <Image> tag to replace the standard <img> which helps optimize your images on the server at build time and decide exactly the right one to serve to your users. The biggest immediate benefits here are load times (quality optimizations, PNG -> WEBP conversions as example) and also addressing Cumulative Layout Shift issues. I highly recommend you click the link to the docs to read more about it. In this example we are only using a small subset of the features available.

In addition to the Image component API docs, Next also includes a special section talking about how they manage image optimization which is well worth a read.

Thanks to a few handy Tailwind classes, with the above version of pages/index.tsx we now have a fully desktop and mobile friendly (simplified) clone of Google's homepage you can view on your dev server.

(Optional) Storybook for Pages

One could make the argument that Storybook isn't quite the right place to test full pages. It's more focused on the individual components than the complete integration of all of that.

That said however, Storybook does have full support for pages and recommendations for how to handle it, so with that in mind if you'd like to test your pages in Storybook then I'll show you the tools you'll need (at this stage) to get it working.

The main challenge is always mocking functional dependencies. So for example Next's router does not exist in Storybook. Other future challenges will be authentication and internationalization.

Each of these can be individually managed though with mock functions that provide sensible defaults, and most of the popular ones (including Next router) have addons to handle most of the config for you.

Here's how to support Next Router in Storybook. Start by installing the addon and reading its documentation.



yarn add -D storybook-addon-next-router

Then update your config files:

.storybook/main.js



module.exports = {
  ...
  addons: [
    ...
    'storybook-addon-next-router',
  ],
};

.storybook/preview.js



import { RouterContext } from 'next/dist/shared/lib/router-context';

...

export const parameters = {
  ..
  nextRouter: {
    Provider: RouterContext.Provider,
  },
};

Then create a story for your page. Since you don't want to interfere with NExt's router by placing stories in your pages directory and potentially causing errors, I have created the __stories__ directory specifically for holding any page stories.

__stories__/pages/index.stories.tsx



import { ComponentMeta, ComponentStory } from '@storybook/react';
import Home from '../../pages';

export default {
  title: 'pages/Home',
  component: Home,
  argTypes: {},
} as ComponentMeta<typeof Home>;

const Template: ComponentStory<typeof Home> = (args) => <Home {...args} />;

export const Base = Template.bind({});

And there it is. Remember that the layout (Header and Footer) are applied by Next via a separate function call, so we only have the actual page content here for testing. If you want to test the layout use the layouts/PrimaryLayout story.

Things are in a good state so time to commit our progress with git commit -m 'feat: build home page'

If you want to align with this step of the tutorial, clone the repository and use git checkout 9ff325aceb0e2096fa618d78489beec2c00dea12.

Front End: Results

We still have the "Results" page to do, but the nice thing is there's a LOT of overlap so we really only have one more custom component to build (Search Result) as well as setting a variant of the layout (home is centered on the page while the results are left-aligned).

Start by copying the BaseTemplate, rename base to search-result and replace each instance of BaseTemplate with SearchResult.

components/utility/search-result/SearchResult



import Link from 'next/link';

export interface ISearchResult extends React.ComponentPropsWithoutRef<'div'> {
  url: string;
  title: string;
  text: string;
}

const SearchResult: React.FC<ISearchResult> = ({
  url,
  title,
  text,
  className,
  ...divProps
}) => {
  return (
    <div
      {...divProps}
      className={`flex flex-col w-5/6 max-w-screen-md space-y-1 ${className} `}
    >
      <Link href={url}>
        <a
          className="cursor:pointer hover:underline"
          target="_blank"
          rel="noopener noreferrer"
        >
          <p>{url}</p>
          <p className="text-blue-600 text-xl ">{title}</p>
        </a>
      </Link>
      <p>{text}</p>
    </div>
  );
};

export default SearchResult;

Then the mock data:

components/utility/search-result/SearchResult.mocks.ts



import { ISearchResult } from './SearchResult';

const base: ISearchResult = {
  url: 'https://www.google.com',
  title: 'This is a link to a search result about product or service',
  text: 'The topic of this link is product or service.  Description of the search result. The description might be a bit long and it will tell you everything you need to know about the search result.',
};

export const mockSearchResultProps = {
  base,
};

Finally rename the story to utility/SearchResult and load Storybook, our component will look like a real Google search result (or close enough for our purposes):

With our result in place, we are ready to create the results page. Create a /results directory in the /pages directory and that's all you need to do, Next will handle the routing for you.

pages/results/index.tsx



import PrimaryLayout from '../../components/layouts/primary/PrimaryLayout';
import SearchResult from '../../components/utility/search-result/SearchResult';
import { mockSearchResultProps } from '../../components/utility/search-result/SearchResult.mocks';
import { NextPageWithLayout } from '../page';

const Results: NextPageWithLayout = () => {
  return (
    <section className="flex flex-col items-center gap-y-5">
      <div className={`flex flex-col space-y-8`}>
        {[...new Array(6)].map((_, idx) => {
          return <SearchResult key={idx} {...mockSearchResultProps.base} />;
        })}
      </div>
    </section>
  );
};

export default Results;

Results.getLayout = (page) => {
  return <PrimaryLayout justify="items-start">{page}</PrimaryLayout>;
};

Since the /results page has its layout left aligned, we need to update out PrimaryLayout.tsx to support a conditional prop. I've created the optional justify prop below and used Typescript to allow the user two options: items-center (default) and items-start.

components/layouts/primary/PrimaryLayout.tsx



import Head from 'next/head';
import Footer from '../../navigation/footer/Footer';
import Header from '../../navigation/header/Header';

export interface IPrimaryLayout extends React.ComponentPropsWithoutRef<'div'> {
  justify?: 'items-center' | 'items-start';
}

const PrimaryLayout: React.FC<IPrimaryLayout> = ({
  children,
  justify = 'items-center',
  ...divProps
}) => {
  return (
    <>
      <Head>
        <title>NextJs Fullstack App Template</title>
      </Head>
      <div {...divProps} className={`min-h-screen flex flex-col ${justify}`}>
        <Header />
        <main className="px-5">{children}</main>
        <div className="m-auto" />
        <Footer />
      </div>
    </>
  );
};

export default PrimaryLayout;

Now start your dev server with yarn dev and go to http://localhost:3000/results

This is a good time to commit our progress with git commit -m 'feat: create results page and SearchResult component'

There's a couple of things I'm going to be excluding from the clone for simplicity. Technically Google's results page still includes the search bar and even places it in the header on scroll.

You could easily create a modified version of that component and place it as a child element into this page and the header, but in terms of this tutorial we wouldn't really touch on any new Next.js-specific topics by doing that (and that's what this tutorial is focused on), so to keep things moving forward I'll leave that as an optional challenge for you if you choose.

If you want to align with this step of the tutorial, clone the repository and use git checkout 3c4cf387cfd9112fe26c5dd268c293d7c1c00f5f.

Back End Planning

Now that we have the visual aspect of the application essentially feature complete (that we know of at this stage) it's time to move onto the back end.

The great thing about Next.js is that it really is a complete full stack solution. Because pages are rendered on the server, obviously that means you have access to a server environment, and that means you can securely do things like access your database directly without needing to expose credentials to the client browser.

The primary vehicles that Next.js uses to do this depends on whether your back end functions are designed to provide data directly to pages being rendered, or if they are standard APIs that simply return data to any source in any shape (usually JSON, but not necessarily).

For the former, pages, we would use getServerSideProps and for the latter we use API routes.

In order to teach how they work, we'll be using both in this example.

Let's begin by thinking about how our app would work if we were actually querying real data. A really simplistic ELI5 version of what Google does is that it crawls through all public data on the web and index it so that it's organized in a way that is fast to search through (a simple example would be in alphabetical order).

That index would be stored by Google in some kind of database. Let's ignore the obvious differences between our little imaginary database and the worldwide distributed datacentres they use, and just simplify it to "searching through existing text in some database."

Adding a real database is beyond the scope of this tutorial (though it will be covered in a future one soon, likely using Prisma and PostgreSQL) so we're just going to create our own little pretend one that is close enough so that we can at least teach the fundamentals.

Like many other aspects of web development, once you have the fundamentals, you can very quickly learn to work with any specific tool or implementation of those ideas.

There are many ways to plan your back end, but personally I believe the most important first step is to begin with your data model. From there you build out any relationships between those data models, and modify as needed based on requirements.

If you are fortunate enough to have a rock solid data model to begin with that all parties are aligned with, and a schema to enforce correctness, you will be in an extremely good position to build your application.

In our case we have control of the data (since we are creating it) and as such I will simply design it to align with the information provided with the standard Google search results:

We already began this work when we built the SearchResult component so I am going to stick with those values for simplicity. You could definitely make the argument that description is a more apt term than text. Once again feel free to design your schema however you like, you don't need to follow what I've used to the letter.

Now that we have decided how the model for our search data will be shaped, we only need to decide how the app will get that data to our results page.

My plan for the journey is as follows:

Search value term is entered by user on home page input form
Form submission redirects to results page with user's search value as a query parameter in the URL
When rendering on server side, the results page will query an API route (we will call it /api/search) inside a getServerSideProps function, which extracts the search value from the URL query param and passes it to the API route.
The API route will query our mock database with the search value and provide the results filtered by the search value back to the getServerSideProps function on the results page.
The getServerSideProps function on theresults page will receive its search results then pass those results as props to the results page component to render the data for the user.

I'll note that in this flow technically the results page could just query the database directly in getServerSideProps. There are two main reasons I've chosen not to do that however:

In a real app, other pages or even external services might have reason to query search results with a search value, so I don't want to tie that search logic specifically to the results page
More personally, I want to demonstrate how to use both API routes and getServerSideProps in this tutorial.

Now with all that planning in place, I think we are ready to build.

Back End: Search Data

We'll begin with the mock database. When working with Node/Javascript/Typescript etc most real database that you query will be done using Node drivers for those DBs which will returns the results in JSON format. JSON is one of (if not THE) most popular formats for transmitting data on the web, so if your app can handle JSON payloads you'll be in very good shape to handle data from many different sources.

That's the reason I've chosen to put our mock data inside a JSON file.

We're going to begin using the /lib directory in the root. If you recall from the initial tutorial that is the folder I created which will store all the domain & business logic and data that our application deals with.

If "components" and "pages" are the front end directories, then "lib" is our back end directory (though we will leverage it from both sides to get all the benefits that come with that, hence the full-stack app we are building).

Create a /search directory inside of /lib. This is where we will be placing all the logic related to the concept of search data and results. Within that we'll create a file called database.json and populate it with the dummy data below:

lib/search/database.json



[
  {
    "url": "https://en.wikipedia.org/wiki/Cat",
    "title": "This is a link to a search result about cats",
    "text": "Did you know their whiskers can sense vibrations in the air?  Description of the search result. The description might be a bit long and it will tell you everything you need to know about the search result."
  },
  {
    "url": "https://en.wikipedia.org/wiki/Dog",
    "title": "This is a link to a search result about dogs",
    "text": "They sure do love to bark.  Description of the search result. The description might be a bit long and it will tell you everything you need to know about the search result."
  },
  {
    "url": "https://en.wikipedia.org/wiki/Cats_%26_Dogs",
    "title": "This is a link to a search result about both cats and dogs",
    "text": "Both of them have tails.  Description of the search result. The description might be a bit long and it will tell you everything you need to know about the search result."
  },
  {
    "url": "https://en.wikipedia.org/wiki/Broccoli",
    "title": "This is a link to a search result about broccoli",
    "text": "Broccoli was invented by crossing cauliflower with pea seeds.  Description of the search result. The description might be a bit long and it will tell you everything you need to know about the search result."
  },
  {
    "url": "https://en.wikipedia.org/wiki/Cauliflower",
    "title": "This is a link to a search result about cauliflower",
    "text": "Who invented cauliflower?  Description of the search result. The description might be a bit long and it will tell you everything you need to know about the search result."
  }
]

I've slightly modified the titles and text values so that we'll be able to perform real searches on the data and see the filtered results.

I'm also going to create a Typescript interface that aligns with this data model. We'll be using that everywhere in our app to minimize errors when working with this data.

lib/search/types.ts



export interface ISearchData {
  url: string;
  title: string;
  text: string;
}

This interface is now the source of truth for everything related to search data in the app. If we every change or add new fields, we add them here and then I want to see every API and every component in the app that uses that data to immediately break and throw a warning that I have to update those as well to handle the schema change.

For that reason there is one place I need to update already. Our SearchResult.tsx component has its own explicit type for url / title / text. Instead of that I'm going to refactor it to extend this type so they always remain aligned:

components/utility/search-result/SearchResult.tsx



import Link from 'next/link';
import { ISearchData } from '../../../lib/search/types';

export type ISearchResult = ISearchData & React.ComponentPropsWithoutRef<'div'>;

...

Everything else below the ellipsis for the component is the same, only the type and imports have been updated.

Back End: API Routes

I'm going to begin with the data and work my way out. I've already created the data in the mock database. The next connection point to that data is our API route that will be loading it and returning a filtered version of it to whoever is querying.

All API routes in Next by default begin with the /api prefix to differentiate them from routes that you would expect to visit and receive an HTML page. Our search query API will be /api/search, so create that structure now along with an index.ts file. Since this is an API dealing with data and not a React component, we can just use the .ts extension:

/pages/api/search/index.ts



// Next.js API route support: https://nextjs.org/docs/api-routes/introduction
import type { NextApiRequest, NextApiResponse } from 'next';
import database from '../../../lib/search/database.json';
import { ISearchData } from '../../../lib/search/types';

interface IApiSearchRequest extends NextApiRequest {
  body: { searchTerm?: string };
}

export type IApiSearchResponseData = ISearchData[];

export default function handler(
  req: IApiSearchRequest,
  res: NextApiResponse<IApiSearchResponseData>
) {
  const {
    body: { searchTerm },
  } = req;

  if (req.method === 'POST' && searchTerm && searchTerm.length > 0) {
    // Creates a regex search pattern for a case insensitive match from the user's search term
    const searchPattern = new RegExp(searchTerm, 'i');

    const filteredResults = database.filter((result) => {
      return (
        // Check the user's search term again either the title or the text of the database entry
        searchPattern.test(result.title) || searchPattern.test(result.text)
      );
    });
    res.status(200).json(filteredResults);
  } else {
    res.status(400).json([]);
  }
}

Let's unpack the above.

We'll start with the database. We're very spoiled to have such amazing tooling in this day and age. By default Typescript will be able to handle the import of raw JSON files and even provide types for us based on the schema that it detects on the fields in the file. We don't even need to explicitly cast it.

This behavior is enabled with the esModuleInterop and resolveJsonModule values in your tsconfig.json file in the root of your project, both of which are enabled by default in the Next.js Typescript template we are using.

The second is that we have decided that we will be expecting the user's searchTerm on the body of a POST request to get search results. If it's not a POST request or the searchTerm is missing or empty, we are going to return a 400 Bad Request along with an empty JSON array to indicate there are no results due to a poorly formatted or invalid request. The benefit of this is that regardless of term we will be able to handle an expectation of an array in the response, either empty or not.

The last key part here is the logic of the actual search. We convert the user's search term into a Javascript regular expression (aka regex) object with the "i" flag which means case insensitive.

If you are unfamiliar or just not comfortable with regexes, an alternative option that accomplishes the same result would be to check if:



result.title.toLowerCase().includes(searchTerm.toLowerCase());

The result of the string compare is used to filter out the complete list of all search results. Obviously if we were using real web indexes there's no possible way we would load ALL possible search results before processing, but this is an example and we know exactly the current size of our data, so our implementation is safe in that scope.

Now let's test our endpoint before we go any further. If you're not familiar with API testing I would suggest you look into some of the great tools out there. Postman used to be the best, but they've started really locking things down behind sign-up walls. It does still have a workable free version though. Insomnia is a great alternative.

If you're comfortable with the command line and you're on a Linux or Mac machine (or Windows with a command line version) the fastest way is to just use cURL.

Below is the command that will make a search request to your API for the term dog.

I've added a couple of echo; in the screenshot just to add newlines to make it more readable -- there are tools to display formatted JSON on the command line too if you want to look them up and get really fancy, but all we care about now is if the payload is returned and is correct.



curl -X POST -H "Content-type: application/json" -H "Accept: application/json" -d '{"searchTerm":"dog"}' "http://localhost:3000/api/search"

There's our result! If you look closely it's returned 2/5 entires from our mock database, the one about "dogs" and the one about "cats & dogs".

Since our search term was dog I'd say that's a good sign things are working well.

Let's switch gears and set up your results page to use this endpoint and get the search results to display.

Static and Dynamic Pages in Next.js

Now we are ready to introduce our first getServerSideProps function. We'll be adding it to our results page so that we can take the search term from the URL of the initial request and use it to fetch search data that we render the page with.

As soon as you introduce this function the page is no longer a candidate for static generation, which is the default behavior for pages in Next. If possible pages will always be generated when you build your app presuming they always look the same for every user. Our home page is an example of that.

Our results page however is going to look different all the time depending on the search term, so consequently Next will have to render that page dynamically every time a user requests it. The benefit obviously being the dynamic data and the downside being an increase in page load time.

We'll begin by doing a simple test of the getServerSideProps function by setting it up with a simple dummy prop.

components/utility/search-result/SearchResult.tsx



import { GetServerSideProps } from 'next';
import PrimaryLayout from '../../components/layouts/primary/PrimaryLayout';
import SearchResult from '../../components/utility/search-result/SearchResult';
import { ISearchData } from '../../lib/search/types';
import { IApiSearchResponseData } from '../api/search';
import { NextPageWithLayout } from '../page';

export interface IResults {
  searchResults: ISearchData[];
}

export const getServerSideProps: GetServerSideProps<IResults> = async ({
  query,
}) => {
  let searchResults: IApiSearchResponseData = [];
  // 1
  const searchTerm = query.search;

  if (searchTerm && searchTerm.length > 0) {
    // 2
    const response = await fetch(`http://localhost:3000/api/search`, {
      body: JSON.stringify({ searchTerm }),
      headers: {
        'Content-Type': 'application/json',
      },
      method: 'POST',
    });

    searchResults = await response.json();
  }

  return {
    // 3
    props: {
      // Will be passed to the page component as props
      searchResults,
    },
  };
};

const Results: NextPageWithLayout<IResults> = ({ searchResults }) => {
  const hasResults = searchResults.length > 0;

  return (
    <>
      <section className="flex flex-col items-center gap-y-5">
        {hasResults ? (
          <div className={`flex flex-col space-y-8`}>
            {searchResults.map((result, idx) => {
              // 4
              return <SearchResult key={idx} {...result} />;
            })}
          </div>
        ) : (
          <p>No results found.</p>
        )}
      </section>
    </>
  );
};

export default Results;

Results.getLayout = (page) => {
  return <PrimaryLayout justify="items-start">{page}</PrimaryLayout>;
};

Hopefully you are able to get an idea how data is being passed in the example above. I would encourage you to read the documentation if you haven't already.

There's a few critical things to understand and unpack here before we talk about what the actual page is doing.

First of all, it's important to be aware that getServerSideProps is a special function that must be named exactly that which is run automatically by Next as part of the page build process. Because of this you should not expect to be able to create a Story for this page in Storybook.

Think of that as a good thing, we are talking about data fetching from our API, at this point we have moved away from the real purpose of Storybook. Ideally it should not be making API calls for data. Of course we could create a mock version of our getServerSideProps function and configure Storybook to use it, but that's beyond the scope of this tutorial.

For now, while we are working on the back end we will be doing all our testing on the development build by running yarn dev.

Just before you run the dev server, let's talk about what's happening. There's a lot going on here so I've add four numbered 1-2-3-4 comments in the code above to talk about.

The query field on the context object that getServerSideProps receives will have the query parameter from the URL. So this page is expecting to receive a URL like /results?search=something and that "something" will be available as available on query.search that we extract into the searchTerm variable.
Here we are querying our own APi we created! Same values and headers we did with cURL test. The search term will be what we extract from the URL, and we'll save the result in searchResults which defaults to an empty array.
We must return an object with values on the props field, that is what our page component will receive. All this is typesafe along the way including the return value, pay close attention to the three places the IResults interface is used along the way.
We take whatever search data is returned and map it to our SearchResult component. We already know the return data matches the expected props, so we can use the spread operator to very easily pass each prop at once.

Now we are ready to run



yarn dev

And open the URL to http://localhost:3000/results?search=dog

Notice the query param I added to the URL? It works! Try changing it yourself to other terms and see if you get different results. Some examples from the mock database would be broccoli and bark.

Time to commit our progress with git commit -m 'feat: implement search API and results page query'

If you want to align with this step of the tutorial, clone the repository and use git checkout f7321a266c51528d2369bf5d5862bc4ace4fdfcb.

Front End Finishing Touches

I'm going to have to backtrack a little bit, turns out there was one more front end task that I forgot before moving to the back end.

We need to configure our Search component to redirect to the results page and put the search term into the URL when it does so that our search bar actually works.

This is quite easy to do, the necessary update to the Search.tsx component looks like this:

components/utility/search/Search.tsx



import { useRouter } from 'next/router';
import { useState } from 'react';

export interface ISearch {}

const Search: React.FC<ISearch> = () => {
  // 1
  const router = useRouter();
  const [searchTerm, setSearchTerm] = useState<string>('');

  return (
    <form
      className="flex flex-col items-center gap-y-5"
      onSubmit={(e) => {
        e.preventDefault();
        // 2
        router.push(`/results?search=${searchTerm}`);
      }}
    >
      <input
        type="text"
        className="rounded-full border-2 w-5/6 sm:w-96 h-12 px-3"
        value={searchTerm}
        onChange={(e) => setSearchTerm(e.target.value)}
      />
      <div className="space-x-3">
        <button type="submit" className="btn-primary">
          Google Search
        </button>
        <button
          onClick={() => alert('FEATURE COMING SOON!')}
          className="btn-primary"
        >
          I&apos;m Feeling Lucky
        </button>
      </div>
    </form>
  );
};

export default Search;

I've added some numbered comments on the code for reference.

We import Next's router which allows us to navigate to different pages while preserving all state.
In the onSubmit function we use the router's push function to navigate to the results page and set the search query param to the current value of the searchTerm which is set by the input field.

I've also added a silly FEATURE COMING SOON! alert to the I'm Feeling Lucky button, but don't hold your breath on that one.

I think we're finally ready to take the entire app for a test drive. Start the dev server with yarn dev and visit http://localhost:3000

How cool is that? We just built our own functioning search engine. Ready to work at Google or NASA now right?

Couple small features to keep in mind, you can return to home and search again by clicking the "Home" link. You can also search by typing your value and just pressing "enter" since it's a <form> element and the browser handles that behavior automatically by triggering onSubmit.

Time to commit our progress with git commit -m 'feat: connect search input to results page'

If you want to align with this step of the tutorial, clone the repository and use git checkout.

Themes and Design Systems

Although the app is "feature complete" as per the scope of this article, there is one final related topic that I want to touch on that I think is absolutely critical: theming.

The link I made above is not specific to Tailwind or any one particular implementation of a theme, because I wanted to first talk about the importance of theming as a concept before we apply it to our app.

As you get more experienced and build more apps you'll realize your CSS naturally starts to look something like:



.card {
  background-color: red;
  padding: 12px;
}

.nav-bar {
  background-color: red;
}

.content-section {
  padding: 12px 24px;
}

.title {
  font-size: 24px;
}

This is a really contrived example, but you can probably see where I'm going. As your app grows and your CSS grows you end up using the same values over and over.

Of course with modern CSS you can do something like --primary-color: red; and then background-color: var(--primary-color), which in itself is already a great improvement, but often what you're looking for is to create a consistent design system that automatically gets used as a default by the pieces of your app without even having to explicitly say it.

Every core component that needs a color should just have --primary-color on it by default rather than you having to be explicit about it. You should only need to do so if overriding it. Similarly with spacing, your app will feel a lot more consistent if all spacing between elements is a multiple of some value like 4px or 8px.

That's what creating a design system (like Material Design for example) aims to do. Build a consistent look for your digital product and place a meaningful framework around it. A good design system will lead to a more consistent and predictable user experience, and also provide the path of least resistance for developers implementing it.

This is just a very basic introduction, I am absolutely not a designer myself but I love working with good ones, because they make my job easier and our product better.

The final part of this tutorial is going to look at Tailwind CSS's specific implementation of a design system and how you can use it to make your app better.

Design System with Tailwind

Like everything, before we begin I always recommend you first read the documentation. Tailwind's docs are fantastic and will help you get up and running quickly.

We actually already created a basic theme in the Tailwind installation section where we established the value of the different xs sm md etc screen breakpoints for our app. The theme lives in tailwind.config.js and we are going to expand on it.

I revisited Google again to see if there's any little changes we can make to closer align the styles, a couple easy ones are: Google uses the Arial font, and the search bar is a bit wider than the max Tailwind static with we have available by default (w-96)

So rather than explicitly override our components, let's update our theme so that the rest of the app can benefit from those conventions!

tailwind.config.js



module.exports = {
  content: [
    './pages/**/*.{js,ts,jsx,tsx}',
    './components/**/*.{js,ts,jsx,tsx}',
  ],
  theme: {
    // Ensure these match with .storybook/preview.js
    screens: {
      xs: '375px',
      sm: '600px',
      md: '900px',
      lg: '1200px',
      xl: '1536px',
    },
    fontFamily: {
      sans: ['Arial', 'sans-serif'],
      serif: ['Garamond', 'serif'],
    },
    extend: {
      colors: {
        blue: {
          500: '#1a73e8',
        },
      },
      spacing: {
        128: '32rem',
      },
    },
  },
  plugins: [],
};

I've updated the fontFamily globally by setting the value there on the theme object. Within that theme object I also have a nested object called extends.

Any values I place on the theme will completely replace Tailwind's defaults, but setting values on the same values inside extends will add those values in addition to the existing ones.

I've overridden the blue-500 colour with the actual colour Google uses on their button based on using the handy eyedropper in Firefox (More Tools -> Eyedropper).

That's what I've done with the new width 128 which will translate into a w-128 Tailwind class. Let's swap out the w-96 value for w-128 on our Search component:

components/utility/search/Search.tsx



...
<input
  type="text"
  className="rounded-full border-2 w-5/6 sm:w-128 h-12 px-3"
  value={searchTerm}
  onChange={(e) => setSearchTerm(e.target.value)}
/>
...

That's it!

There's more cool stuff you can do with the theme we didn't mention here. The colour specific documentation is worth a look, as is the concept of using a self-referencing function to get access to the theme value.

For example if you wanted to set a blue colour and then later reference that exact colour on a background while still on the theme itself with theme('color.blue').

Sharing State Between Pages

One topic that is critically important for large Next.js apps that we haven't yet addressed is the ability to share state between pages.

In traditional single page React apps it's quite simple to pass your props or wrap the app in context, but how is that handled in Next when transitioning to a completely separate page?

The answer is that we leverage the top level _app.tsx component to manage our state. As long as we are using Next's built in router or the special Next <Link> component, Next will be able to handle the persistance of state in our app between pages.

The same general rules for React state still apply, if the user refreshes the page or manually enters a URL it will be lost. In those cases if you want persistance you would want to look at localStorage or a state management packaged solution that includes support for local storage like Recoil

Just for a quick demonstration of how to use it, we will be implementing a mock "auth" state that is controlled with our "Sign In" button. Our goal will be that your authenticated state will still persist even when hitting the search button and navigation to the /results page.

We will be using React context for this. Down the road when you implement a real auth service, you could potentially even connect it to this component we are going to create and replace the mock data with real data, while still using our context solution to control the UI state.

First things first I think it's time to create an additional root directory. We need a place to store React specific logic (like context and custom hooks) that is not the same as pure UI (components) or domain logic and services (lib).

Proper project structure is critically important and there are some great resources about it out there. I want to find the right balance between too compact (too much unrelated in one directory) and too abstract (directories for every different concept no matter how small).

For our use case I am going to create a root directory called /state which will be intended to hold both custom hooks and React context. The two are usually tightly related so I am comfortable keeping them together for the time being.

Within /state I will create a directory called /auth which will manage everything related to the state of authentication in our app.

state/auth/AuthContext.tsx



import { createContext, useState } from 'react';

interface IAuthContext {
  authenticated: boolean;
  login: () => void;
  logOut: () => void;
}

const defaultValue: IAuthContext = {
  authenticated: false,
  login: () => undefined,
  logOut: () => undefined,
};

const AuthContext = createContext<IAuthContext>(defaultValue);

export const AuthProvider: React.FC = ({ children }) => {
  const [authenticated, setAuthenticated] = useState(
    defaultValue.authenticated
  );
  const login = () => setAuthenticated(true);
  const logOut = () => setAuthenticated(false);

  return (
    <AuthContext.Provider value={{ authenticated, login, logOut }}>
      {children}
    </AuthContext.Provider>
  );
};

export default AuthContext;

The above component will provide context to our entire application that any component can use to check if the user is authenticated to see certain content. When that authentication state changes (using one of the two handy login/logOut functions we have provided) then all children of the context provider will re-render and update their state.

(Note when I say all children I mean ALL children, even ones that don't use the authenticated context value. This is an important concept to understand, I would recommend you read more about it if you aren't familiar with that concept. This is a place to start. It's one of the reasons why global state management libraries like Redux and Recoil are so widely used is that they have ways of working around this behavior if you need to)

We will create a new button component called AuthButton. This component is going to be dependent on the context provided by AuthContext, so we need to remember that when we use this button somewhere up the component tree we will need an AuthContext.Provider component for it to work -- the trick is to remember that's not just for our app, that applies to Storybook as well! For now though, let's just build the component.

Copy our BaseComponent over again into the /components/button directory and rename it to auth. We're going to replace all instances of BaseComponent with AuthButton including the filename. Make sure you also change the story title to buttons/AuthButton and remove any most data from the template.

The structure of the AuthButton already exists, we are going to extract it out of our Header component into its own component like so:

components/buttons/auth/AuthButton.tsx



import { useContext } from 'react';
import AuthContext from '../../../state/auth/AuthContext';
import styles from './AuthButton.module.css';

export interface IAuthButton extends React.ComponentPropsWithoutRef<'button'> {}

const AuthButton: React.FC<IAuthButton> = ({ className, ...buttonProps }) => {
  const { authenticated, login, logOut } = useContext(AuthContext);

  return (
    <button
      onClick={authenticated ? logOut : login}
      className={`${styles.container} ${className} border-1 p-2 px-4 sm:px-6 bg-blue-500 rounded text-white w-28`}
      {...buttonProps}
    >
      {authenticated ? 'Sign Out' : 'Sign In'}
    </button>
  );
};

export default AuthButton;

Pay attention to the useContext invocation, that is how twe consume the <AuthProvider> context that will be wrapping our entire application. We'll get to that part last. The next step is to take this new auth button use it in our Header:



import Link from 'next/link';
import AuthButton from '../../buttons/auth/AuthButton';

export interface IHeader extends React.ComponentPropsWithoutRef<'header'> {}

const Header: React.FC<IHeader> = ({ className, ...headerProps }) => {
  return (
    <header
      {...headerProps}
      className={`w-full flex flex-row justify-between ${className}`}
    >
      <div className="space-x-5 m-5">
        <Link href="/">
          <a className="hover:underline">Home</a>
        </Link>
        <Link href="/">
          <a className="hover:underline">Store</a>
        </Link>
      </div>
      <div className="space-x-5 m-5">
        <Link href="/">
          <a className="hover:underline hidden sm:inline">Gmail</a>
        </Link>
        <Link href="/">
          <a className="hover:underline hidden sm:inline">Images</a>
        </Link>
        <AuthButton />
      </div>
    </header>
  );
};

export default Header;

Finally we need to update _app.tsx which is the component that wraps our whole app. We want every piece of our app to have access to the Auth context, so right now that serves as the best place for it.

Technically every time the auth updates the app will re-render, but that is okay since presumably a real user would only be signing in once per session.

pages/_app.tsx



import type { AppProps } from 'next/app';
import { AuthProvider } from '../state/auth/AuthContext';
import './globals.css';
import { NextPageWithLayout } from './page';

interface AppPropsWithLayout extends AppProps {
  Component: NextPageWithLayout;
}

function MyApp({ Component, pageProps }: AppPropsWithLayout) {
  // Use the layout defined at the page level, if available
  const getLayout = Component.getLayout || ((page) => page);

  return <AuthProvider>{getLayout(<Component {...pageProps} />)}</AuthProvider>;
}

export default MyApp;

And finally, if we want to be able to access these context values for the components when we run them in Storybook, we need to create a default story template that includes that context.

For that we use Storybook decorators. Just export a const called decorators which React component(s) you want as a wrapper around all your stories.



import { AuthProvider } from '../state/auth/AuthContext';

...

export const decorators = [
  (Story) => (
    <AuthProvider>
      <Story />
    </AuthProvider>
  ),
];

Thats it! Now run yarn dev and load http://localhost:3000

When you click on the "Sign In" button if all has gone correct it will toggle to a "Sign Out" which mimics the function of having logged into the site. Doing this is basic React behavior to toggle a button state.

What is special about what we have done is when you enter a term into your search bar and hit search. It will navigate to a completely different page, the results page, but because of the React auth context wrapper your button should still show "Sign Out" if you had signed in on the home page.

And that is persistent state between routes in Next.js

Wrapping Up

Remember that all code from this tutorial as a complete package is available in this repository.

Please check some of my other learning tutorials. Feel free to leave a comment or question and share with others if you find any of them helpful:

How to Build Scalable Architecture for your Next.js Project

Alex Eagleson — Wed, 06 Apr 2022 23:45:38 +0000

All code from this tutorial as a complete package is available in this repository.

If you find this tutorial helpful, please share it with your friends and colleagues! For more like it you can subscribe on Youtube or follow me on Twitter.

This tutorial is available as a video lesson if you prefer that format:

What is Next.js?
Introduction
Project Setup
Engine Locking
Git Setup
Code Formatting and Quality Tools
Git Hooks
VS Code Configuration
Debugging
Directory Structure
Adding Storybook
Creating a Component Template
Using the Component Template
Adding a Custom Document
Adding Layouts
Deployment
Next Steps
Wrapping Up

What is Next.js?

"Next.js gives you the best developer experience with all the features you need for production: hybrid static & server rendering, TypeScript support, smart bundling, route pre-fetching, and more. No config needed."

As described in their words above, Next.js is very much an all-in-one fullstack modern application building solution. It includes first class support for Typescript and React, while offering easy solutions for some of the most common requirements in a modern application like routing, API, postCSS tools, and code-splitting.

It also supports both static site generation (for lightning fast static HTML pages that can be hosted anywhere) or with managed hosting services like Vercel/AWS/etc that run a Node server and support full on-demand data loading and server-side rendered pages.

Next.js has quickly become one of the most in demand skills in the web development space. This tutorial aims to act as kind of a "practical" extension to the documentation and help you set up a project using a lot of best practices that will improve your chances of keeping everything management as you scale.

Introduction

This tutorial is not meant to replace the official documentation, which is absolutely fantastic. I highly recommend you take a read through at least the basic features section before you begin this tutorial, so you'll be familiar with the terminology and tools and some of the components they provide that are similar, but usually "more powerful" versions of their vanilla HTML counterparts.

Please review the table of contents to get an idea of each of the topics we will be touching in this extensive tutorial. I will freely acknowledge many of them are strict and opinionated configurations, if any of tem don't appeal to you then in most cases you can simply skip over those sections and should still be able to complete the tutorial without too much trouble.

Now, with all that said, if you are ready, let's dive right in!

Project Setup

We'll begin by creating a default Next.js application with a Typescript template.



npx create-next-app --ts nextjs-fullstack-app-template

cd nextjs-fullstack-app-template

First we will test to make sure the app is working. We're going to be using yarn for this example, but you could just as easily use NPM if you choose.



yarn install

yarn dev

You should see the demo app available on http://localhost:3000

Also recommended to run



yarn build

To ensure you can successfully do a production build of the project. It's recommended (but not required) to close your dev server when running a Next.js build. Most of the time there is no issue but occasionally the build can put your dev server in a weird state that requires a restart.

You should get a nice little report on the command line of all the pages built with green coloured text implying they are small and efficient. We'll try to keep them that way as we develop the project.

Engine Locking

We would like for all developers working on this project to use the same Node engine and package manager we are using. To do that we create two new files:

.nvmrc - Will tell other uses of the project which version of Node is used
.npmrc - Will tell other users of the project which package manager is used

We are using Node v14 Fermium and yarn for this project so we set those values like so:

.nvmrc



lts/fermium

.npmrc



engine-strict=true

The reason we are using v14 instead of v16 for Node is that later in the tutorial we will be deploying on Vercel which unfortunately still does not support Node 16. Perhaps by the time you read this tutorial it might. You can follow the progress here.

You can check your version of Node with node --version and make sure you are setting the correct one. A list of Node version codenames can be found here

Note that the use of engine-strict didn't specifically say anything about yarn, we do that in package.json:

package.json



  "name": "nextjs-fullstack-app-template",
  "author": "YOUR_NAME",
  "description": "A tutorial and template for creating a production-ready fullstack Next.js application",
  "version": "0.1.0",
  "private": true,
  "license" : "MIT"
  "homepage": "YOUR_GIT_REPO_URL"
  "engines": {
    "node": ">=14.0.0",
    "yarn": ">=1.22.0",
    "npm": "please-use-yarn"
  },
  ...

The engines field is where you specify the specific versions of the tools you are using. You can also fill in your personal details if you choose.

Git Setup

This would be a good time to make our first commit to our remote repo, to make sure our changes are backed up, and to follow best practices for keeping related changes grouped within a single commit before moving to something new.

By default your Next.js project will already have a repo initialized. You can check what branch you are on with git status. It should say something like:



On branch main
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   README.md

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        .npmrc
        .nvmrc

This tells us we are on the main branch and we have not staged or made any commits yet.

Let's commit our changes so far.



git add .

git commit -m 'project initialization'

The first command will add and stage all files in your project directory that aren't ignored in .gitignore. The second will make a commit of the state of your current project with the message we wrote after the -m flag.

Hop over to your preferred git hosting provider (Github for example) and create a new repository to host this project. Make sure the default branch is se tto the same name as the branch on your local machine to avoid any confusion.

On Github you can change your global default branch name to whatever you like by going to:



Settings -> Repositories -> Repository default branch

Now you are ready to add the remote origin of your repository and push. Github will give you the exact instructions when you create it. Your syntax may be a little different than mine depending on if you are using HTTPS rather than SSH.



git remote add origin git@github.com:{YOUR_GITHUB_USERNAME}/{YOUR_REPOSITORY_NAME}.git

git push -u origin {YOUR_BRANCH_NAME}

Note that from this point on we will be using the Conventional Commits standard and specifically the Angular convention described here

The reason being like many other features in this project to simply set a consistent standard for all developers to use to minimize train-up time when contributing to the project. I personally have very little concern as to what standard is chosen, as long as everyone agrees to follow it that is the most important thing.

Consistency is everything!

Code Formatting and Quality Tools

In order to set a standard that will be used by all contributors to the project to keep the code style consistent and basic best practices followed we will be implementing two tools:

eslint - For best practices on coding standards
prettier - For automatic formatting of code files

ESLint

We'll begin with ESLint, which is easy because it automatically comes installed and pre-configured with Next.js projects.

We are just going to add a little bit of extra configuration and make it a bit stricter than it is by default. If you disagree with any of the rules it sets, no need to worry, it's very easy to disable any of them manually. We configure everything in .eslintrc.json which should already exist in your root directory:

.eslintrc.json



{
  "extends": ["next", "next/core-web-vitals", "eslint:recommended"],
  "globals": {
    "React": "readonly"
  },
  "rules": {
    "no-unused-vars": [1, { "args": "after-used", "argsIgnorePattern": "^_" }]
  }
}

In the above small code example we have added a few additional defaults, we have said that React will always be defined even if we don't specifically import it, and I have added a personal custom rule that I like which allows you to prefix variables with an underscore _ if you have declared them but not used them in the code.

I find that scenario comes up often when you are working on a feature and want to prepare variables for use later, but have not yet reached the point of implementing them.

You can test out your config by running:



yarn lint

You should get a message like:



✔ No ESLint warnings or errors
Done in 1.47s.

If you get any errors then ESLint is quite good at explaining clearly what they are. If you encounter a rule you don't like you can disable it in "rules" by simply setting it to 1 (warning) or 0 (ignore) like so:



  "rules": {
    "no-unused-vars": 0, // As example: Will never bug you about unused variables again
  }

Let's make a commit at this point with the message build: configure eslint

Prettier

Prettier will take care of automatically formatting our files for us. Let's add it to the project now.

It's only needed during development, so I'll add it as a devDependency with -D



yarn add -D prettier

I also recommend you get the Prettier VS Code extension so that VS Code can handle the formatting of the files for you and you don't need to rely on the command line tool. Having it installed and configured in your project means that VSCode will use your project's settings, so it's still necessary to add it here.

We'll create two files in the root:

.prettierrc



{
  "trailingComma": "es5",
  "tabWidth": 2,
  "semi": true,
  "singleQuote": true
}

Those values are entirely at your discretion as to what is best for your team and project.

.prettierignore



.yarn
.next
dist
node_modules

In that file I've placed a list of directories that I don't want Prettier to waste any resources working on. You can also use patterns like *.html to ignore groups of types of files if you choose.

Now we add a new script to package.json so we can run Prettier:

package.json



  ...
  "scripts: {
    ...
    "prettier": "prettier --write ."
  }

You can now run



yarn prettier

to automatically format, fix and save all files in your project you haven't ignored. By default my formatter updated about 5 files. You can see them in your list of changed files in the source control tab on the left of VS Code.

Let's make another commit with build: implement prettier.

Git Hooks

One more section on configuration before we start getting into component development. Remember you're going to want this project to be as rock solid as possible if you're going to be building on it in the long term, particularly with a team of other developers. It's worth the time to get it right at the start.

We are going to implement a tool called Husky

Husky is a tool for running scripts at different stages of the git process, for example add, commit, push, etc. We would like to be able to set certain conditions, and only allow things like commit and push to succeed if our code meets those conditions, presuming that it indicates our project is of acceptable quality.

To install Husky run



yarn add -D husky

npx husky install

The second command will create a .husky directory in your project. This is where your hooks will live. Make sure this directory is included in your code repo as it's intended for other developers as well, not just yourself.

Add the following script to your package.json file:

package.json



  ...
  "scripts: {
    ...
    "prepare": "husky install"
  }

This will ensure Husky gets installed automatically when other developers run the project.

To create a hook run



npx husky add .husky/pre-commit "yarn lint"

The above says that in order for our commit to succeed, the yarn lint script must first run and succeed. "Succeed" in this context means no errors. It will allow you to have warnings (remember in the ESLint config a setting of 1 is a warning and 2 is an error in case you want to adjust settings).

Let's create a new commit with the message ci: implement husky. If all has been setup properly your lint script should run before the commit is allowed to occur.

We're going to add another one:



npx husky add .husky/pre-push "yarn build"

The above ensures that we are not allowed to push to the remote repository unless our code can successfully build. That seems like a pretty reasonable condition doesn't it? Feel free to test it by committing this change and trying to push.

Lastly we are going to add one more tool. We have been following a standard convention for all our commit messages so far, let's ensure that everyone on the team is following them as well (including ourselves!). We can add a linter for our commit messages:



yarn add -D @commitlint/config-conventional @commitlint/cli

To configure it we will be using a set of standard defaults, but I like to include that list explicitly in a commitlint.config.js file since I sometimes forget what prefixes are available:

commitlint.config.js



// build: Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)
// ci: Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs)
// docs: Documentation only changes
// feat: A new feature
// fix: A bug fix
// perf: A code change that improves performance
// refactor: A code change that neither fixes a bug nor adds a feature
// style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
// test: Adding missing tests or correcting existing tests

module.exports = {
  extends: ['@commitlint/config-conventional'],
  rules: {
    'body-leading-blank': [1, 'always'],
    'body-max-line-length': [2, 'always', 100],
    'footer-leading-blank': [1, 'always'],
    'footer-max-line-length': [2, 'always', 100],
    'header-max-length': [2, 'always', 100],
    'scope-case': [2, 'always', 'lower-case'],
    'subject-case': [
      2,
      'never',
      ['sentence-case', 'start-case', 'pascal-case', 'upper-case'],
    ],
    'subject-empty': [2, 'never'],
    'subject-full-stop': [2, 'never', '.'],
    'type-case': [2, 'always', 'lower-case'],
    'type-empty': [2, 'never'],
    'type-enum': [
      2,
      'always',
      [
        'build',
        'chore',
        'ci',
        'docs',
        'feat',
        'fix',
        'perf',
        'refactor',
        'revert',
        'style',
        'test',
        'translation',
        'security',
        'changeset',
      ],
    ],
  },
};

Then enable commitlint with Husky by using:



npx husky add .husky/commit-msg 'npx --no -- commitlint --edit "$1"'
# Sometimes above command doesn't work in some command interpreters
# You can try other commands below to write npx --no -- commitlint --edit $1
# in the commit-msg file.
npx husky add .husky/commit-msg \"npx --no -- commitlint --edit '$1'\"
# or
npx husky add .husky/commit-msg "npx --no -- commitlint --edit $1"

Feel free to try some commits that don't follow the rules and see how they are not accepted, and you receive feedback that is designed to help you correct them.

I'm going to create a new commit now with the message ci: implement commitlint.

You can see the result of the complete culmination of this setup in the screenshot below, hopefully yours looks similar:

VS Code Configuration

Now that we have implemented ESLint and Prettier we can take advantage of some convenient VS Code functionality to have them be run automatically.

Create a directory in the root of your project called .vscode and inside a file called settings.json. This will be a list of values that override the default settings of your installed VS Code.

The reason we want to place them in a folder for the project is that we can set specific settings that only apply to this project, and we can share them with the rest of our team by including them in the code repository.

Within settings.json we will add the following values:

.vscode/settings.json



{
  "editor.defaultFormatter": "esbenp.prettier-vscode",
  "editor.formatOnSave": true,
  "editor.codeActionsOnSave": {
    "source.fixAll": true,
    "source.organizeImports": true
  }
}

The above will tell VS Code to use your Prettier extension as the default formatter (you can override manually if you wish with another one) and to automatically format your files and organize your import statements every time you save.

Very handy stuff and just another thing you no longer need to think about so you can focus on the important things like solving business problems.

I'll now make a commit with message build: implement vscode project settings.

Debugging

Let's set up a convenient environment for debugging our application in case we run into any issues during development.

Inside of your .vscode directory create a launch.json file:

launch.json



{
  "version": "0.1.0",
  "configurations": [
    {
      "name": "Next.js: debug server-side",
      "type": "node-terminal",
      "request": "launch",
      "command": "npm run dev"
    },
    {
      "name": "Next.js: debug client-side",
      "type": "pwa-chrome",
      "request": "launch",
      "url": "http://localhost:3000"
    },
    {
      "name": "Next.js: debug full stack",
      "type": "node-terminal",
      "request": "launch",
      "command": "npm run dev",
      "console": "integratedTerminal",
      "serverReadyAction": {
        "pattern": "started server on .+, url: (https?://.+)",
        "uriFormat": "%s",
        "action": "debugWithChrome"
      }
    }
  ]
}

With that script in place you have three choices for debugging. CLick the little "bug & play icon" on the left of VS Code or press Ctrl + Shift + D to access the debugging menu. You can select which script you want to run and start/stop it with the start/stop buttons.

In addition to this, or if you are not using VS Code, we can also set up some helpful debugging scripts in your project.

First we will install the cross-env which will; be necessary to set environment variables if you have teammates working on different environments (Windows, Linux, Mac, etc).



yarn add -D cross-env

With that package installed we can update our package.json dev script to look like the following:

package.json



{
  ...
  "scripts": {
    ...
    "dev": "cross-env NODE_OPTIONS='--inspect' next dev",
  },
}

This will allow you to log server data in the browser while working in dev mode, making it easier to debug issues.

At this stage I'll be making a new commit with message build: add debugging configuration

Directory Structure

This section is now going to cover setting up the folder structure in our project. This is one of those topics that many people will have extremely strong opinions about, and for good reason! Directory structure can really make or break a project in the long term when it gets out of control, especially when fellow team members have to spend unnecessary time trying to guess where to put things (or find things).

I personally like to take a fairly simplistic approach, keep things separated basically in a class model/view style. We will be using three primary folders:



/components
/lib
/pages

component - The individual UI components that make up the app will live in here
lib - Business/app/domain logic will live in here.
pages - Will be the actual routes/pages as per the required Next.js structure.

We will have other folders in addition to this to support the project, but the core of almost everything that makes up the unique app that we are building will be housed in these three directories.

Within components we will have subdirectories that kind of group similar types of components together. You can use any method you prefer to do this. I have used the MUI library quite a bit in my time, so I tend to follow the same organization they use for components in their documentation

For example inputs, surfaces, navigation, utils, layout etc.

You don't need to create these directories in advance and leave them empty. I would just create them as you go while building your components.

This section is simply designed to explain how I will be setting up this project, there are many other ways you might choose to organize yours and I would encourage you to choose whatever works best for you and your team.

At this point I will be making a commit with message rfc: create directory structure

Adding Storybook

One of the great modern tools available to us if you aren't already familiar with it is called Storybook.

Storybook gives us an environment to show off and test the React components we are building outside of the application we are using them in. It's great tool to connect developers with designers and be able to verify components we have developed look and function as per design requirements in an isolated environment without the overhead of the rest of the app.

Note that Storybook is meant as a visual testing tool, we will be implementing other tools later for functional unit testing and end-to-end testing.

The best way to learn how to use Storybook is installing it and trying it out!



npx sb init --builder webpack5

We'll be using the webpack5 version to stay up to date with the latest version of webpack (I'm unsure why it is still not yet the default. Maybe it will be by the time you are using this tutorial).

When Storybook installs it automatically detects a lot of things about your project, like how it is a React app, and other tools you are using. It should take care fo all that configuration itself.

If you get a prompt about the eslintPlugin, you can say "yes". We are going to configure it manually though, so no worries if you get a message saying it didn't auto-configure.

Open up .eslintrc.json and update it to the following:

.eslintrc.json



{
  "extends": [
    "plugin:storybook/recommended", // New
    "next",
    "next/core-web-vitals",
    "eslint:recommended"
  ],
  "globals": {
    "React": "readonly"
  },
  // New
  "overrides": [
    {
      "files": ["*.stories.@(ts|tsx|js|jsx|mjs|cjs)"],
      "rules": {
        // example of overriding a rule
        "storybook/hierarchy-separator": "error"
      }
    }
  ],
  "rules": {
    "no-unused-vars": [1, { "args": "after-used", "argsIgnorePattern": "^_" }]
  }
}

I have added // New to mark the two new sections and lines that are Storybook specific.

You'll notice that Storybook has also added as /stories directory to the root of your project with a number of examples in. If you are new to Storybook I highly recommend you look through them and leave them there until you are comfortable creating your own without the templates.

Before we run it we need to make sure we are using webpack5. Add the following to your package.json file:

package.json



{
  ...
  "resolutions": {
    "webpack": "^5"
  }
}

Then run



yarn install

To ensure webpack5 is installed.

Next we have to update the .storybook/main.js file:

storybook/main.js



module.exports = {
  stories: ['../**/*.stories.mdx', '../**/*.stories.@(js|jsx|ts|tsx)'],
  /** Expose public folder to storybook as static */
  staticDirs: ['../public'],
  addons: [
    '@storybook/addon-links',
    '@storybook/addon-essentials',
    '@storybook/addon-interactions',
  ],
  framework: '@storybook/react',
  core: {
    builder: '@storybook/builder-webpack5',
  },
};

Here we have changed the pattern for stories files so that it will pick up any .stories files inside our components (or other) directories.

We have also exposed Next.js's "public" folder as a static directory so we can test things like images, media, etc in Storybook.

Lastly, before we run Storybook itself, let's add some helpful values in storybook/preview.js. This is the file where we can control the defaults for how our stories render.

storybook/preview.js



import '../styles/globals.css';
import * as NextImage from 'next/image';

const BREAKPOINTS_INT = {
  xs: 375,
  sm: 600,
  md: 900,
  lg: 1200,
  xl: 1536,
};

const customViewports = Object.fromEntries(
  Object.entries(BREAKPOINTS_INT).map(([key, val], idx) => {
    console.log(val);
    return [
      key,
      {
        name: key,
        styles: {
          width: `${val}px`,
          height: `${(idx + 5) * 10}vh`,
        },
      },
    ];
  })
);

// Allow Storybook to handle Next's <Image> component
const OriginalNextImage = NextImage.default;

Object.defineProperty(NextImage, 'default', {
  configurable: true,
  value: (props) => <OriginalNextImage {...props} unoptimized />,
});

export const parameters = {
  actions: { argTypesRegex: '^on[A-Z].*' },
  controls: {
    matchers: {
      color: /(background|color)$/i,
      date: /Date$/,
    },
  },
  viewport: { viewports: customViewports },
};

There are a few personal preferences in the above, but you can configure it how you want. Be sure to set the default breakpoints to match whatever is important to you in your app. We are also adding a handler so that Storybook can handle Next's <Image> component without crashing.

Now we are ready to test it. Run:



yarn storybook

If all goes well you'll see a message in your console that looks like:

And you'll be able to access it on http://localhost:6006

I would encourage you to play around and get familiar with the examples if you've never used it before.

At this stage I'll be making a commit with message build: implement storybook.

Creating a Component Template

It's time to bring together all the configuration we have done and look at how we might create and implement our first component using the standards we have set for ourselves.

We'll just create a simple card. Create the following directory structure:

/components/templates/base

And inside that directory we'll create BaseTemplate.tsx. This will follow a standard pattern of filename matching the directories leading up to it. This allows us for example to have other types of cards in the cards directory like PhotoCard or TextCard etc.

BaseTemplate.tsx



export interface IBaseTemplate {}

const BaseTemplate: React.FC<IBaseTemplate> = () => {
  return <div>Hello world!</div>;
};

export default BaseTemplate;

Every single one of our components is going to follow this exact structure. Even if it does not use props it will still export an empty props interface for the component. The reason for this is it will allow us to replicate this exact structure across many components and files, and interchange components/imports using the same expected pattern and just find/replace the names of the components.

When you begin working with the stories and mock props etc it will become quickly apparent how convenient and powerful it is to maintain a consistent naming scheme and interface for all your component files.

This goes back to the consistency is everything point we made earlier.

Next I am going to make a style module file that lives next to the component. By default Next.js gives you a /styles directory which I personally do not use, but if you prefer to keep all your styles in the same place that's a fine choice. I just prefer to keep them with the components.

BaseTemplate.module.css



.component {
}

As a standard empty template for where your top level styles will go on your component. You can update your BaseTemplate as follows:

BaseTemplate.tsx



import styles from './BaseTemplate.module.css';

export interface IBaseTemplate {}

const BaseTemplate: React.FC<IBaseTemplate> = () => {
  return <div className={styles.container}>Hello world!</div>;
};

export default BaseTemplate;

Now we have a clean template for our styling.

Let's add an example prop to our template so we can handle the standard we'll be using for components props:

BaseTemplate.tsx



import styles from './BaseTemplate.module.css';

export interface IBaseTemplate {
  sampleTextProp: string;
}

const BaseTemplate: React.FC<IBaseTemplate> = ({ sampleTextProp }) => {
  return <div className={styles.container}>{sampleTextProp}</div>;
};

export default BaseTemplate;

With each component we create we're going to want a very quick and easy way to test it in different environments (Storybook for example, but also the app, and maybe our unit tests). It will be handy to have quick access to data to render the component.

Let's create a file to store some mock data for this component to use for testing:

BaseTemplate.mocks.ts



import { IBaseTemplate } from './BaseTemplate';

const base: IBaseTemplate = {
  sampleTextProp: 'Hello world!',
};

export const mockBaseTemplateProps = {
  base,
};

This structure may seem a bit convoluted, but we'll see the benefits soon. I am using very intentional consistent naming patterns so this template is very easy to copy and paste to each new component you create.

Now let's create a create a story for this component:

BaseTemplate.stories.tsx



import { ComponentStory, ComponentMeta } from '@storybook/react';
import BaseTemplate, { IBaseTemplate } from './BaseTemplate';
import { mockBaseTemplateProps } from './BaseTemplate.mocks';

export default {
  title: 'templates/BaseTemplate',
  component: BaseTemplate,
  // More on argTypes: https://storybook.js.org/docs/react/api/argtypes
  argTypes: {},
} as ComponentMeta<typeof BaseTemplate>;

// More on component templates: https://storybook.js.org/docs/react/writing-stories/introduction#using-args
const Template: ComponentStory<typeof BaseTemplate> = (args) => (
  <BaseTemplate {...args} />
);

export const Base = Template.bind({});
// More on args: https://storybook.js.org/docs/react/writing-stories/args

Base.args = {
  ...mockBaseTemplateProps.base,
} as IBaseTemplate;

I'm not going to get into all the details of what each different part of a stories file entails, for that your best resource is the official Storybook documentation.

The goal here is to create a consistent easily copy/paste-able pattern of component building and testing.

Let's try this one out. Run:



yarn storybook

If all goes well you will be greeted by your fine looking base component (if not I encourage you to revisit the previous section and check if you missed any of the configurations).

Now that we're starting to create more files it's good to get into the habit of running yarn lint before doing your commits to make sure everything is clean and ready to go. I'm going to make a commit with message build: create BaseTemplate component.

Using the Component Template

Since we have our template, let's go through the process of using it to create a real component.

Create components/cards directory. Then copy the entirety of the base directory from templates into cards and rename it cat. We are going to make a CatCard. Rename each of the files to match. When done it should look like this:

Now you can press ctrl + shift + F (or mac equivalent) in VS Code to do a full project search and replace. Include only components/cards/cat and do a replace for CatCard to replace BaseTemplate. It should look like the following:

Now you're ready to work, you've got a clean pre-generated template to work from that includes a story and mock data for your card. Pretty handy! Let's make it look like a real card:

(For the record I did not create this beautiful card, it's based on an example created here by the talented Lyon Etyo)

CatCard.tsx



import styles from './CatCard.module.css';
import Image from 'next/image';

export interface ICatCard {
  tag: string;
  title: string;
  body: string;
  author: string;
  time: string;
}

const CatCard: React.FC<ICatCard> = ({ tag, title, body, author, time }) => {
  return (
    <div className={styles.container}>
      <div className={styles.card}>
        <div className={styles.card__header}>
          <Image
            src="/time-cat.jpg"
            alt="card__image"
            className={styles.card__image}
            width="600"
            height="400"
          />
        </div>
        <div className={styles.card__body}>
          <span className={`${styles.tag} ${styles['tag-blue']}`}>{tag}</span>
          <h4>{title}</h4>
          <p>{body}</p>
        </div>
        <div className={styles.card__footer}>
          <div className={styles.user}>
            <Image
              src="https://i.pravatar.cc/40?img=3"
              alt="user__image"
              className={styles.user__image}
              width="40"
              height="40"
            />
            <div className={styles.user__info}>
              <h5>{author}</h5>
              <small>{time}</small>
            </div>
          </div>
        </div>
      </div>
    </div>
  );
};

export default CatCard;

Set the styles:

CatCard.module.css



@import url('https://fonts.googleapis.com/css2?family=Quicksand:wght@300..700&display=swap');

.container {
  margin: 1rem;
}

.container * {
  box-sizing: border-box;
  padding: 0;
  margin: 0;
}

.card__image {
  max-width: 100%;
  display: block;
  object-fit: cover;
}

.card {
  font-family: 'Quicksand', sans-serif;
  display: flex;
  flex-direction: column;
  width: clamp(20rem, calc(20rem + 2vw), 22rem);
  overflow: hidden;
  box-shadow: 0 0.1rem 1rem rgba(0, 0, 0, 0.1);
  border-radius: 1em;
  background: #ece9e6;
  background: linear-gradient(to right, #ffffff, #ece9e6);
}

.card__body {
  padding: 1rem;
  display: flex;
  flex-direction: column;
  gap: 0.5rem;
}

.tag {
  align-self: flex-start;
  padding: 0.25em 0.75em;
  border-radius: 1em;
  font-size: 0.75rem;
}

.tag-blue {
  background: #56ccf2;
  background: linear-gradient(to bottom, #2f80ed, #56ccf2);
  color: #fafafa;
}

.card__body h4 {
  font-size: 1.5rem;
  text-transform: capitalize;
}

.card__footer {
  display: flex;
  padding: 1rem;
  margin-top: auto;
}

.user {
  display: flex;
  gap: 0.5rem;
}

.user__image {
  border-radius: 50%;
}

.user__info > small {
  color: #666;
}

And set the mock data:

CatCard.mocks.ts



import { ICatCard } from './CatCard';

const base: ICatCard = {
  tag: 'Felines',
  title: `What's new in Cats`,
  body: 'Lorem ipsum dolor sit amet consectetur adipisicing elit. Sequi perferendis molestiae non nemo doloribus. Doloremque, nihil! At ea atque quidem!',
  author: 'Alex',
  time: '2h ago',
};

export const mockCatCardProps = {
  base,
};

Note that this uses an image of a cat (/time-cat.jpg) from the project's public directory. You can find it in the project repository.

The only thing we need to update in CatCard.stories is to change the story title from templates/CatCard to cards/CatCard

We do need to update our next.config.js because we are using a domain we haven't explicitly stated as permitted (for the avatar). Simply update your config file to look like this:

next.config.js



/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
  images: {
    domains: ['i.pravatar.cc'],
  },
};

module.exports = nextConfig;

Alternatively you could just place the avatar image in your own public directory, but for the sake of learning the process of using an external domain, we'll leave this setup in place.

Now nun Storybook, and if you're lucky, you'll be greeted with:

This component can then easily be dropped anywhere in your actual application. Use the mock props in the short term while testing and replace with real props when you're ready!

pages/index.tsx



import type { NextPage } from 'next';
import Head from 'next/head';
import Image from 'next/image';
import CatCard from '../components/cards/cat/CatCard';
import { mockCatCardProps } from '../components/cards/cat/CatCard.mocks';
import styles from '../styles/Home.module.css';

const Home: NextPage = () => {
  return (
    <div className={styles.container}>
      <Head>
        <title>Create Next App</title>
        <meta name="description" content="Generated by create next app" />
        <link rel="icon" href="/favicon.ico" />
      </Head>

      <main className={styles.main}>
        <h1 className={styles.title}>
          Welcome to <a href="https://nextjs.org">Next.js!</a>
        </h1>

        <div className={styles.grid}>
          <CatCard {...mockCatCardProps.base} />
          <CatCard {...mockCatCardProps.base} />
          <CatCard {...mockCatCardProps.base} />
          <CatCard {...mockCatCardProps.base} />
        </div>
      </main>

      <footer className={styles.footer}>
        <a
          href="https://vercel.com?utm_source=create-next-app&utm_medium=default-template&utm_campaign=create-next-app"
          target="_blank"
          rel="noopener noreferrer"
        >
          Powered by{' '}
          <span className={styles.logo}>
            <Image src="/vercel.svg" alt="Vercel Logo" width={72} height={16} />
          </span>
        </a>
      </footer>
    </div>
  );
};

export default Home;

Let's take a look at the final masterpiece with:



yarn dev

Adding a Custom Document

Though it is not necessary at this stage you will likely want to have more fine grained control over what is in the <head> of your app. Creating a custom _document.tsx in your pages directory allows you to do that. Create that file now.

pages/_document.tsx



import Document, { Head, Html, Main, NextScript } from 'next/document';

class MyDocument extends Document {
  render() {
    return (
      <Html>
        <Head>
          <link rel="preconnect" href="https://fonts.googleapis.com" />
          <link rel="preconnect" href="https://fonts.gstatic.com" />
          <link
            href="https://fonts.googleapis.com/css2?family=Quicksand:wght@300..700&display=swap"
            rel="stylesheet"
          />
        </Head>
        <body>
          <Main />
          <NextScript />
        </body>
      </Html>
    );
  }
}

export default MyDocument;

Note that I have removed the @import URL font from components/cards/cat/CatCard.module.css and placed that Google font here in the head to preload.

Anything else you need to do or customize in your <head> element can now be done in this file.

Be aware that this <Head> is not the same as the one imported from next/head. They will both work together and this one is used solely for data you would like loaded on every single page.

For more info on how to use a custom _document see the documentation.

Adding Layouts

Layouts are an important concept in Next.js. They help you manage state between pages. For this section we will be using the same basic template as provided in the official example and simply customizing it to fit our project.

Create a new directory called layouts in components. We will be copying our templates/case directory again two times. One to callprimary and one called sidebar. It should look as follows when complete:

Do a case-sensitive find/replace for BaseTemplate inside each of the files as well to replace with PrimaryLayout and SidebarLayout respectively.

If you have any difficulty with this step, feel free to just take the structure from the repo.

All credit to _leerob and JJ Kasper from Vercel for the structure of these layout templates_

Update the content of PrimaryLayout.tsx and PrimaryLayout.module.css to be:

components/layouts/primary/PrimaryLayout.tsx



import Head from 'next/head';
import styles from './PrimaryLayout.module.css';

export interface IPrimaryLayout {}

const PrimaryLayout: React.FC<IPrimaryLayout> = ({ children }) => {
  return (
    <>
      <Head>
        <title>Primary Layout Example</title>
      </Head>
      <main className={styles.main}>{children}</main>
    </>
  );
};

export default PrimaryLayout;

components/layouts/primary/PrimaryLayout.module.css



.main {
  display: flex;
  height: calc(100vh - 64px);
  background-color: white;
}

.main > section {
  padding: 32px;
}

Then for sidebar:

components/layouts/sidebar/SidebarLayout.tsx



import Link from 'next/link';
import styles from './SidebarLayout.module.css';

export interface ISidebarLayout {}

const SidebarLayout: React.FC<ISidebarLayout> = () => {
  return (
    <nav className={styles.nav}>
      <input className={styles.input} placeholder="Search..." />
      <Link href="/">
        <a>Home</a>
      </Link>
      <Link href="/about">
        <a>About</a>
      </Link>
      <Link href="/contact">
        <a>Contact</a>
      </Link>
    </nav>
  );
};

export default SidebarLayout;

components/layouts/sidebar/SidebarLayout.module.css



.nav {
  height: 100%;
  display: flex;
  flex-direction: column;
  width: 250px;
  background-color: #fafafa;
  padding: 32px;
  border-right: 1px solid #eaeaea;
}

.nav > a {
  margin: 8px 0;
  text-decoration: none;
  background: white;
  border-radius: 4px;
  font-size: 14px;
  padding: 12px 16px;
  text-transform: uppercase;
  font-weight: 600;
  letter-spacing: 0.025em;
  color: #333;
  border: 1px solid #eaeaea;
  transition: all 0.125s ease;
}

.nav > a:hover {
  background-color: #eaeaea;
}

.input {
  margin: 32px 0;
  text-decoration: none;
  background: white;
  border-radius: 4px;
  border: 1px solid #eaeaea;
  font-size: 14px;
  padding: 8px 16px;
  height: 28px;
}

Now that these templates have been created, we need to use them. We are going to update our Home page and create another page called about.tsx to show how to use shared layouts and persist component state between pages.

First we need to add a type which extends the default NextPage interface as for some reason it does not include the getLayout function out of the box. Create a custom type file that will handle that for us inspired by this solution

pages/page.d.ts



import { NextPage } from 'next';
import { ComponentType, ReactElement, ReactNode } from 'react';

export type NextPageWithLayout<P = {}> = NextPage<P> & {
  getLayout?: (_page: ReactElement) => ReactNode;
  layout?: ComponentType;
};

You can now use that NextPageWithLayout interface in place of NextPage when you need to create pages with custom layouts.

Now let's update our home page:

pages/index.tsx



import CatCard from '../components/cards/cat/CatCard';
import { mockCatCardProps } from '../components/cards/cat/CatCard.mocks';
import PrimaryLayout from '../components/layouts/primary/PrimaryLayout';
import SidebarLayout from '../components/layouts/sidebar/SidebarLayout';
import styles from '../styles/Home.module.css';
import { NextPageWithLayout } from './page';

const Home: NextPageWithLayout = () => {
  return (
    <section className={styles.main}>
      <h1 className={styles.title}>
        Welcome to <a href="https://nextjs.org">Next.js!</a>
      </h1>
      <CatCard {...mockCatCardProps.base} />
    </section>
  );
};

export default Home;

Home.getLayout = (page) => {
  return (
    <PrimaryLayout>
      <SidebarLayout />
      {page}
    </PrimaryLayout>
  );
};

and also create a new about page in the pages directory:

pages/about.tsx



import PrimaryLayout from '../components/layouts/primary/PrimaryLayout';
import SidebarLayout from '../components/layouts/sidebar/SidebarLayout';
import { NextPageWithLayout } from './page';

const About: NextPageWithLayout = () => {
  return (
    <section>
      <h2>Layout Example (About)</h2>
      <p>
        This example adds a property <code>getLayout</code> to your page,
        allowing you to return a React component for the layout. This allows you
        to define the layout on a per-page basis. Since we&apos;re returning a
        function, we can have complex nested layouts if desired.
      </p>
      <p>
        When navigating between pages, we want to persist page state (input
        values, scroll position, etc.) for a Single-Page Application (SPA)
        experience.
      </p>
      <p>
        This layout pattern will allow for state persistence because the React
        component tree is persisted between page transitions. To preserve state,
        we need to prevent the React component tree from being discarded between
        page transitions.
      </p>
      <h3>Try It Out</h3>
      <p>
        To visualize this, try tying in the search input in the{' '}
        <code>Sidebar</code> and then changing routes. You&apos;ll notice the
        input state is persisted.
      </p>
    </section>
  );
};

export default About;

About.getLayout = (page) => {
  return (
    <PrimaryLayout>
      <SidebarLayout />
      {page}
    </PrimaryLayout>
  );
};

Then update _app.tsx as follows:

pages/_app.tsx



import type { AppProps } from 'next/app';
import './globals.css';
import { NextPageWithLayout } from './page';

interface AppPropsWithLayout extends AppProps {
  Component: NextPageWithLayout;
}

function MyApp({ Component, pageProps }: AppPropsWithLayout) {
  // Use the layout defined at the page level, if available
  const getLayout = Component.getLayout || ((page) => page);

  return getLayout(<Component {...pageProps} />);
}

export default MyApp;

Finally, in the mocks files I have updated PrimaryLayout.mocks.ts to use children: '{{component}}' as a placeholder value to show in Storybook where a component would go, and I have removed the mock props in SidebarLayout.mocks.ts (though I do not remove the file, so I have the interface ready to go in case I ever need to add props).

I have also changed the story titles from templates/... to layouts/....

Finally we can test it out. Save and run



yarn dev

Click between the two routes on the sidebar (Home and About) to toggle between pages. Note that the layouts used will persist without needing to reload (as was our intention) and you are left with a super fast and snappy experience.

On the Storybook side, we can even view and test out layout components independent of the application. The PrimaryLayout isn't too useful without content inside of it, but the sidebar is quite nice to have.



yarn storybook

Deployment

Our final step will be to show the process of deployment of a Next.js app.

We will be using Vercel as it is the simplest and most straightforward deployment solution for a Next.js app (primarily due to the fact that Vercel owns Next and so one can always assume they will offer best in class support).

Be aware Vercel is absolutely not the only option, other major services like AWS, Netlify, etc all work as well if you choose to go that route.

Ultimately you just need a service where you can run the next start command presuming you're not using an entirely statically generated site (in which case any static hosting tool would work and no custom Next server would be required).

Deploying on Vercel as a hobby user is completely free. To begin we will create an account on Vercel.

Once logged in, click + New Project and give Vercel access to your Github repositories. You can give global access, or you can select only the repository you want to deploy. I am going to select this repository called nextjs-fullstack-app-template.

Once you have selected it you'll need to configure it. In the Build and Output Settings section make sure you replace the default NPM commands with your yarn commands (unless you are using NPM).

We have not used any environment variables yet so no need to add any.

Once that's it just click Deploy and you're done! It's that easy.

(The screenshot above is a bit out of date, I originally wrote the deployment section before the layout section, but you get the idea)

Not only is your site deployed now, it will continue to get re-deployed automatically every time you commit to your primary branch. If you don't want this behavior, then it's easy to configure in the Vercel dashboard.

The nice thing is that you already configured your yarn build command to ensure a working production build before you can push you code, so you can push with confidence in presuming your deployment will be successful.

The only thing you need to keep in mind are the differences between your two environments. It's still possible for your build to succeed locally but fail on Vercel if for example your scripts are different (using NPM instead of yarn or vice versa) or more commonly if you are missing environment variables.

We will be adding env values in future tutorials so you will need to make sure those values are configured in both your local and production environment, since they are secrets and should never be committed to a public (or even private if can be avoided) repository.

Next Steps

I hope you found this tutorial and learned something about setting up a solid and scaleable Next.js project for you and your team.

This is the first part of what is intended to be a multi-part series on creating a production quality Next.js app.

Some of my ideas for future installments are below, I'd encourage you to leave some feedback about which ones you'd find most useful (or other ones if you don't see them below).

How to build a fullstack Next.js app using API routes and Tailwind CSS
How to add a global state manager to your Next.js app with Recoil
How to implement unit and end-to-end testing in a Next.s app with jest and playwright
How to create a CI/CD pipeline with Github actions and Vercel
How to implement SSO authentication and internationalization in a Next.js app using NextAuth and i18next
How to connect a database to your Next.js app with Prisma and Supabase
How to manage multiple applications in a monorepo with Next.js and Nx

Stay tuned and please don't hesitate to ask any questions, I'm happy to answer if I can!

Wrapping Up

Remember that all code from this tutorial as a complete package is available in this repository.

Please check some of my other learning tutorials. Feel free to leave a comment or question and share with others if you find any of them helpful:

How to Connect a React App to a Notion Database

Alex Eagleson — Sun, 09 Jan 2022 02:23:44 +0000

All code from this tutorial as a complete package is available in this repository.

If you find this tutorial helpful, please share it with your friends and colleagues! For more like it you can subscribe on Youtube or follow me on Twitter.

This tutorial is available as a video lesson if you prefer that format:

What is Notion?
Introduction
Project Setup
Creating a Notion Database
Creating the Server
Querying the Server
Creating a Notion Integration
Querying the Database
Connecting the App
Wrapping Up

What is Notion?

Before we jump in I want to explain quickly a little bit what Notion is.

It's basically an organizational tool that runs in the cloud and supports multiple user collaboration at the same time.

It can be used for anything from organizing daily tasks, keeping track of school schedules, to managing the documentation of large enterprise projects.

Basically if you want to "organize" any kind of information, Notion is a great tool for that.

Similar products you might be familiar with would be something like Confluence, Evernote or OneNote.

Introduction

Recently I discovered that Notion provides an API to read and modify data on your Notion workspace.

They also have fantastic support for simple databases (even relational ones) so I thought it would be fun to try a little app that could use Notion as a quick and dirty CMS, and I had a lot of fun doing it, so I thought I would write up a little tutorial to share the process with others.

I want to be clear that I am absolutely not advocating for the use of Notion as a real database for a production application.

I do not know anything about the actual speed and performance of querying it at any scale, and I also wouldn't trust any critical data on a service that isn't specifically designed to offer a reliability guarantee.

However, for fun little projects I think it's a great option, especially for front end developers who don't have a lot of existing knowledge about databases and just want to get their feet wet.

It can also be a great way to collaborate with less technical folks and allow them the flexibility that Notion offers for creating content, and giving developers the ability to directly reference that content in code.

So without further delay, let's play around with it.

Project Setup

The structure of our project will be:

React App -> Node server -> Notion database

The reason we need the Node server is because if we were to query directly from our React app, we would have to expose our Notion account credentials and secret/database ID. Anything on the client side is always visible to the user.

By querying on the server we can keep the credentials there, out of reach of the front end, and only provide the database table data itself to the front end.

We'll begin by creating the project directory and React app. We're using Create React App here as it's still the simplest way to get an instant React project up and running with minimal complexity:



mkdir react-node-notion
cd react-node-notion
npx create-react-app@latest sample-app --template typescript
cd sample-app
npm run start

Make sure you are able to see the example React app on http://localhost:3000/ before you continue.

Creating a Notion Database

Next we are going to create our Notion workspace and database.

Navigate to:

https://www.notion.so/

You can create an account or login with an existing Google or Apple account. Notion is free to use for an individual.

Next we'll create a page where our database will live. My database is going to help me keep track of stuff I want to learn in 2022.

Click anywhere on the "My Cool Project" page and type /page. You'll have te option of creating a new page. Create one and give it an icon.

Open your new page. You can give it a cover image at the top. CLick anywhere on the blank page and type /database. You're going to select "Table Database - Inline"

The first column should be a unique value (our primary key). I'm simply going to name that column key. The second column I will name label and the third column I will name url. The key column will be of type title by default, but you will need to set the label column to text and the url column to url:

I've made the column headers lowercase on purpose since we will be referring to them with Javascript properties when we query (which are traditionally lowercase).

I will be using this database to keep track of the things I want to learn, and a URL link to the resource to learn them. This is super simple but you can come up with anything as complex as you want, we're mostly just here to give an example of how to query this data and display it in an app (or anywhere you like really).

Populate the DB with whatever data suits you best. Here's mine:

Creating the Server

We're next going to spin up a super simple Node server to serve the data. All we need is the http module and the Notion client library from NPM.

Let's begin with just the server and confirm we can query the data before we add the Notion integration:

Go back to the root directory react-node-notion before running these commands:



mkdir server
cd server
npm init -y
npm install -D typescript @types/node
npx tsc --init
mkdir src
touch src/server.ts

In case you aren't creating your files from the command line, the above instructions will install the necessary packages and create a server directory and an src directory inside with a server.ts file. Your full directory structure for the entire project should look like:



.
├── sample-app
│   └── (React app files)
└── server
    ├── src
    │   └── server.ts
    ├── tsconfig.json
    ├── package-lock.json
    └── package.json

Your server.ts file will look like:

server/src/server.ts



import http from "http";

const host = "localhost";
const port = 8000;

const server = http.createServer((req, res) => {
  // Avoid CORS errors
  res.setHeader("Access-Control-Allow-Origin", "*");
  res.setHeader("Content-Type", "application/json");
  switch (req.url) {
    // Will respond to queries to the domain root (like http://localhost/)
    case "/":
      res.writeHead(200);
      res.end(JSON.stringify({ data: "success" }));
      break;

    // Only supports the / route
    default:
      res.writeHead(404);
      res.end(JSON.stringify({ error: "Resource not found" }));
  }
});

server.listen(port, host, () => {
  console.log(`Server is running on http://${host}:${port}`);
});

Your npx tsc --init command will have created a tsconfig.json file. All the defaults are fine, you just need to add one value:

tsconfig.json



{
  ...
  "outDir": "./dist"
}

That will output the result of the tsc command into a dist folder with a JS file that you can run.

Give it a try by running:



npx tsc && node dist/server.js`
```

That says "run typescript and then use Node to run the resulting Javascript file it creates in the output folder".  

## Querying the Server

Navigate back to the `sample-app` directory and open the `src` directory.  We can delete `App.css` and the `logo.svg` file.  

We'll update the `index.css` with some super simple CSS based off [this minimalist style](https://www.swyx.io/css-100-bytes/).

`sample-app/src/index.css`
```css
html {
  max-width: 70ch;
  padding: 3em 1em;
  margin: auto;
  line-height: 1.75;
  font-size: 1.25em;
}

h1,h2,h3,h4,h5,h6 {
  margin: 3em 0 1em;
}

p,ul,ol {
  margin-bottom: 2em;
  color: #1d1d1d;
  font-family: sans-serif;
}
```

Now we update the contents of `App.tsx`.  Remove all the default content inside the file (including the imports) and replace it with the following:

`sample-app/src/App.tsx`
```tsx
function App() {

  return (
    <div>
      <h1>Things to Learn</h1>

      <button
        type="button"
        onClick={() => {
          fetch("http://localhost:8000/")
            .then((response) => response.json())
            .then((payload) => {
              console.log(payload)
            });
        }}
      >
        Fetch List
      </button>
    </div>
  );
}

export default App;
```

We use the [Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) to query the simple server we just wrote that we made listen on port 8000 and respond on the root domain route `/`.

So that means to reach that endpoint we need to query `http://localhost:8000/`.  Save and run your app, then press the "Fetch List" button.  Open the dev console with F12 and you will see:


![Fetch Success](https://res.cloudinary.com/dqse2txyi/image/upload/r_10,bo_3px_solid_darkgrey/v1641689103/blogs/notion-cms/fetch-success_sujvb8.png)

Notice the `{ data: "success" }` response there in the console.  Great!  

Our React app is connected to our server and we can query basic data.  Let's get Notion hooked up.

## Creating a Notion Integration

Before you can query data from your Notion account you need to create an _integration_ that has the necessary permissions.  You can configure integrations to have different permissions like read/write/insert depending on who you are sharing the integration secret with.

Go to the following URL:

[https://www.notion.so/my-integrations]()

And click the big **[+ New Integration]** button on the left.  

You can configure and name your integration how you like.  For mine I only want to be able to read content from my database, so I am only giving it read permissions and no access to user data:

![Integration Setup](https://res.cloudinary.com/dqse2txyi/image/upload/r_10,bo_3px_solid_darkgrey/v1641689746/blogs/notion-cms/integration-setup_piu1pt.png)

After you have created the integration you will be provided with a "secret" that gives access to your integration.  Keep this handy as we will need it soon:

![Notion Secret](https://res.cloudinary.com/dqse2txyi/image/upload/r_10,bo_3px_solid_darkgrey/v1641689915/blogs/notion-cms/notion-secret_r0nytu.png)

In addition to the secret, we also need to configure the database itself to be allowed to be read.  Go back to your "Things to Learn" database (or whatever you wrote).

At the upper right corner of your database page is a "Share" button.  Click it and then click the "Invite" button.  You will have the ability to invite your new integration that you created to have access to this database.  It will still be private and hidden from the general public.

![Share Database](https://res.cloudinary.com/dqse2txyi/image/upload/r_10,bo_3px_solid_darkgrey/v1641690016/blogs/notion-cms/share-database_d9smky.png)

The two values you need to query this database from your Node app are the Notion secret (which you already have) and the database ID.  The database ID you can get from the URL when you are looking at your database.  The URL will look something like this:

```
https://www.notion.so/aaaaaaaaaaaaaaaaaaaaaaaaaa?v=bbbbbbbbbbbbbbbbbbbbbbbbbbb
```

In the above example your `database id` is the `aaaaaaaaaaaaaaaaaaaaaa` part before the question mark.

You now have everything you need to query the data.  Back to the Node server.

## Querying the Database

We are going to need a secure place to store our Notion secret and database ID.  If we put them in our code they will become visible to anyone who checks the source when we push to a remote repository.  To get around this we will store our credentials in a `.env.` file.

Inside your `server` directory create two new files (note that both of them are hidden files that are prefix with a `.` before the filename):

`server/.env`
```
NOTION_SECRET="secret_xxxxxxxxxxxxxxxxxxxxxx"
NOTION_DATABASE_ID="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
```

Where the dummy values above are replaced by the values you took from Notion.  Remember your secret key does need the `secret_` prefix in front of it.

We also want to create a `.gitignore` file:

`server/.gitignore`
```
.env
dist
node_modules
```

This will tell `git` not to push your secret `.env` file our your automatically generated `dist` folder when you commit.  

While we're at it let's add a start script for our server that does the `tsc` build and runs the JS output:

`server/package.json`
```json
{
  ...
  "scripts": {
    "start": "tsc && node dist/server.js"
  },
}
```

Alright!  Now that that is taken care of we just need two packages.  One is the official Notion client for Node, and the other is `dotenv` library that will made it super simple to read your secret and database id values from that `.env` file:

```
npm install @notionhq/client@0.4.9 dotenv
```

Note that I have locked Notion client to `0.4.9` as the API may change since this is a relatively new product and I would like this tutorial to work for anyone who uses it in the future.  You can try running the latest build however you may need to refer to their documentation and make corrections if anything has changed.

Now we're going to update our `server.ts` file.  We'll do it all at once but I'll add lots of comments to explain everything:

`server/src/server.ts`
```ts
require("dotenv").config();
import http from "http";
import { Client } from "@notionhq/client";

// This is Typescript  interface for the shape of the object we will
// create based on our database to send to the React app
// When the data is queried it will come back in a much more complicated shape, so our goal is to
// simplify it to make it easy to work with on the front end
interface ThingToLearn {
  label: string;
  url: string;
}

// The dotenv library will read from your .env file into these values on `process.env`
const notionDatabaseId = process.env.NOTION_DATABASE_ID;
const notionSecret = process.env.NOTION_SECRET;

// Will provide an error to users who forget to create the .env file
// with their Notion data in it
if (!notionDatabaseId || !notionSecret) {
  throw Error("Must define NOTION_SECRET and NOTION_DATABASE_ID in env");
}

// Initializing the Notion client with your secret
const notion = new Client({
  auth: notionSecret,
});

const host = "localhost";
const port = 8000;

// Require an async function here to support await with the DB query
const server = http.createServer(async (req, res) => {
  res.setHeader("Access-Control-Allow-Origin", "*");

  switch (req.url) {
    case "/":
      // Query the database and wait for the result
      const query = await notion.databases.query({
        database_id: notionDatabaseId,
      });

      // We map over the complex shape of the results and return a nice clean array of
      // objects in the shape of our `ThingToLearn` interface
      const list: ThingToLearn[] = query.results.map((row) => {
        // row represents a row in our database and the name of the column is the
        // way to reference the data in that column
        const labelCell = row.properties.label;
        const urlCell = row.properties.url;

        // Depending on the column "type" we selected in Notion there will be different
        // data available to us (URL vs Date vs text for example) so in order for Typescript
        // to safely infer we have to check the `type` value.  We had one text and one url column.
        const isLabel = labelCell.type === "rich_text";
        const isUrl = urlCell.type === "url";

        // Verify the types are correct
        if (isLabel && isUrl) {
          // Pull the string values of the cells off the column data
          const label = labelCell.rich_text?.[0].plain_text;
          const url = urlCell.url ?? "";

          // Return it in our `ThingToLearn` shape
          return { label, url };
        }

        // If a row is found that does not match the rules we checked it will still return in the
        // the expected shape but with a NOT_FOUND label
        return { label: "NOT_FOUND", url: "" };
      });

      res.setHeader("Content-Type", "application/json");
      res.writeHead(200);
      res.end(JSON.stringify(list));
      break;

    default:
      res.setHeader("Content-Type", "application/json");
      res.writeHead(404);
      res.end(JSON.stringify({ error: "Resource not found" }));
  }
});

server.listen(port, host, () => {
  console.log(`Server is running on http://${host}:${port}`);
});
```

Should be good!  We'll start the server with the new script we made in `package.json`:

```
npm run start
```

## Connecting the App

A quick jump back into the React app and hit that "Fetch Data" button again.  If everything went well you will be greeted with the content of your database in your browser console:

![React Notion Query](https://res.cloudinary.com/dqse2txyi/image/upload/r_10,bo_3px_solid_darkgrey/v1641691474/blogs/notion-cms/notion-react-query_kdwswh.png)

You've now got the data in your React app, you can do whatever you want with it!  We could probably wrap up the tutorial here, but let's make one final step of turning the data into an actual list of links:

`sample-app/src/App.tsx`
```tsx
import { useState } from "react";

// Copy the payload shape interface from our server
// We want to copy (rather than import) since we we won't necessarily deploy our
// front end and back end to the same place
interface ThingToLearn {
  label: string;
  url: string;
}

function App() {
  // A state value will store the current state of the array of data which can be updated
  // by editing your database in Notion and then pressing the fetch button again
  const [thingsToLearn, setThingsToLearn] = useState<ThingToLearn[]>([]);

  return (
    <div>
      <h1>Things to Learn</h1>
      <button
        type="button"
        onClick={() => {
          fetch("http://localhost:8000/")
            .then((response) => response.json())
            .then((payload) => {
              // Set the React state with the array response
              setThingsToLearn(payload);
            });
        }}
      >
        Fetch List
      </button>

      {/* Map the resulting object array into an ordered HTML list with anchor links */}
      {/* Using index as key is harmless since we will only ever be replacing the full list */}
      <ol>
        {thingsToLearn.map((thing, idx) => {
          return (
            <li key={idx}>
              <a href={thing.url} target="_blank" rel="noopener noreferrer">
                {thing.label}
              </a>
            </li>
          );
        })}
      </ol>
    </div>
  );
}

export default App;
```

And with that, a click of the fetch button and we get a nice list of things to do which reflects the state of our Notion database and creates links to the relevant pages.  

![Final Result](https://res.cloudinary.com/dqse2txyi/image/upload/r_10,bo_3px_solid_darkgrey/v1641691929/blogs/notion-cms/final-result_irlemi.png)

Go ahead, try changing some text in your DB and hitting the button again.

## Wrapping Up

Well that's pretty neat!  Now that you know how to do this, what cool projects can you think to build?

Remember that all code from this tutorial as a complete package is available in [this repository](https://github.com/alexeagleson/react-node-notion).  

Please check some of my other learning tutorials.  Feel free to leave a comment or question and share with others if you find any of them helpful:

- [How to use Node.js to backup your personal files](https://dev.to/alexeagleson/how-to-use-nodejs-to-backup-your-personal-files-and-learn-some-webdev-skills-along-the-way-541a)

- [Introduction to Docker for Javascript Developers](https://dev.to/alexeagleson/docker-for-javascript-developers-41me)

- [Learnings from React Conf 2021](https://dev.to/alexeagleson/learnings-from-react-conf-2021-17lg)

- [How to Create a Dark Mode Component in React](https://dev.to/alexeagleson/how-to-create-a-dark-mode-component-in-react-3ibg)

- [How to Analyze and Improve your 'Create React App' Production Build ](https://dev.to/alexeagleson/how-to-analyze-and-improve-your-create-react-app-production-build-4f34)

- [How to Create and Publish a React Component Library](https://dev.to/alexeagleson/how-to-create-and-publish-a-react-component-library-2oe)

- [How to use IndexedDB to Store Local Data for your Web App ](https://dev.to/alexeagleson/how-to-use-indexeddb-to-store-data-for-your-web-application-in-the-browser-1o90)

- [Running a Local Web Server](https://dev.to/alexeagleson/understanding-the-modern-web-stack-running-a-local-web-server-4d8g)

- [ESLint](https://dev.to/alexeagleson/understanding-the-modern-web-stack-linters-eslint-59pm)

- [Prettier](https://dev.to/alexeagleson/understanding-the-modern-web-stack-prettier-214j)

- [Babel](https://dev.to/alexeagleson/building-a-modern-web-stack-babel-3hfp)

- [React & JSX](https://dev.to/alexeagleson/understanding-the-modern-web-stack-react-with-and-without-jsx-31c7)

- [Webpack: The Basics](https://dev.to/alexeagleson/understanding-the-modern-web-stack-webpack-part-1-2mn1)

- [Webpack: Loaders, Optimizations & Bundle Analysis](https://dev.to/alexeagleson/understanding-the-modern-web-stack-webpack-part-2-49bj)

---

If you find this tutorial helpful, please share it with your friends and colleagues!  For more like it you can subscribe on [Youtube](https://www.youtube.com/channel/UCV5YqK3AaInd3lYFQqlp7Lw) or follow me on [Twitter](https://twitter.com/eagleson_alex).

How to use Node.js to backup your personal files (and learn some webdev skills along the way)

Alex Eagleson — Wed, 29 Dec 2021 04:44:16 +0000

All code from this tutorial as a complete package is available in this repository. If you find this tutorial helpful, please share it with your friends and colleagues!

For more tutorials like this, follow me @eagleson_alex on Twitter

This tutorial is also available as a video:

Introduction

Like many other people out there, I have quite a few digital documents and pictures that are important to me.

Although I know I have at least one copy of these locally, and I do use Dropbox as cloud storage; I'll freely admit I don't meet the coveted 3-2-1 backup strategy standards 😳.

Over the holidays I received a new 4TB hard drive and with it, a renewed interest in backing up my data (at least the important stuff like pictures of my kids, and financial records. I guess in the worst case scenario, I could probably replace my Star Trek TNG blu-ray rips; so I'll keep those separate for now).

With that in mind, I decided to combine it with an exercise that dives a bit further than I usually go into the Node.js ecosystem.

This tutorial is the result of that exploration, and the result is a little tool for synchronizing backup copies of any directory on your machine. As a bonus we're going to configure it to support Linux, Mac and Windows.

This is one of those posts where the biggest beneficiary in many ways is myself. I wanted to try something new and simply documented my experience with learning. It follows @swyx's philosophy of learn in public and share what you've learned with the wider community so hopefully everyone can benefit.

That said, I should be very clear that this is primarily meant as a learning experience and is absolutely not meant to be the best backup solution.

If you're serious about remote storage then something like Google Drive will get the job done for you. For local backups setting up a RAID drive will cover your ass better than this little backup app will.

That said, those options are a lot less fun so if you're willing to use this as a learning opportunity to practice your Node.js skills, and get some bonus scheduled backups out of it, I think you'll find this to be a really interesting tutorial.

What You Will Learn
Try it Out (Optional)
Configuring Your Machine (Optional)
Creating the Project
Cross Platform Support
Running as a Background Process
Adding a Discord Webhook (Bonus)
Wrapping Up

What You Will Learn

Common Linux tools like rsync (copying files locally and over SSH), cron (scheduling tasks on a specific interval) and nohup (run something as a background process that doesn't stop when the terminal session is ended)
Running Node (Javascript) apps as background processes, including automatic reboot on crash using pm2, a production viable tool for Node.js servers.
More about the different values available on the process object in Node.js including title and platform
Create an app that behaves differently depending on which OS it's running on and works on both Bash (Mac/Linux) and Microsoft PowerShell (Windows)
Send messages to a webhook with HTTP POST requests, in our example that webhook will be a Discord bot

Try it Out (Optional)

If you want to try it out first, then follow the instructions below. If you want to jump into creating your own version, then skip this section.

This app will work on both Mac/Linux (Bash) and Windows (PowerShell).

All you need installed are git and nodejs.

Clone the project from this repository
Run npm install from the project directory
Create a .env file in the root with the following structure:



SOURCE_DIR="example-source/"
DESTINATION_DIR="example-destination/"
CRON_STRING="* * * * * *"
WEBHOOK_ID="DISCORD_WEBHOOK_ID"

Update SOURCE_DIR to be the directory you want to copy, and DESTINATION_DIR where you want to synchronize.

Edit CRON_STRING to determine the schedule the copying occurs on. If you are not familiar with cron strings use this tool to help you create one.

The DISCORD_WEBHOOK_ID is optional. If you don't use it it will not impact the app. If you do use it you remove the https://discord.com/api/webhooks/ portion of the webhook URLm the remainder is the DISCORD_WEBHOOK_ID.

Now you can run the app with:



node backup.js

If you plan to run it as a long-term background process you can use PM2 which is described in the Running as a Background Process section.

Configuring Your Machine (Optional)

(Note: if you already have a machine and folders that you want to use for your backups setup, then you can skip this step entirely. All we are going to cover in this section is setting up Ubuntu Linux on an old laptop, configuring it to run 24/7, and using SSH so that we can access and manage it remotely from other machines on our home network)

I'll be using this little blue Dell Inspiron with a whopping 2GB of RAM sitting around collecting dust.

Honestly it's a great machine with an SSD inside of it, unfortunately it's just too bottlenecked by its 2GB of RAM, and there is no way to upgrade it (I've tried).

So for that reason I haven't been able to get much use out of it, but that changes now.

I began by following this tutorial for installing Ubuntu with a USB stick and got the little laptop up and running with the latest Ubuntu.

Next I needed to make sure I could access the laptop from the terminal on my main machine. That will be done over SSH so I followed this tutorial on enabling SSH on Ubuntu. Afterwards I confirmed I could successfully SSH into the laptop from my machine.

Next I need to make sure that both git and node are installed on the machine, so that I can clone my project and run it. Fortunately git comes by default with Ubuntu and I can installed node with:



sudo apt update
sudo apt install nodejs

If this gives you any trouble just follow this tutorial.

Next I plug in my external hard drives that I am going to be using as backups. When I create my app I'll be pointing from one directory on one hard drive to another directory on the other hard drive to sync each week.

Lastly I need to do a bit of housekeeping for the laptop to make sure it stays running and knows what to do when the lid is closed.

To keep it from sleeping it's as simple as Settings -> Power and turn off Automatic Suspend. Basically disable anything that reads as if it will put your machine to sleep.

Next I need to deal with what happens when I close the lid. Unfortunately I don't see anything about that in Power settings so I need to edit the config file directly:



sudo gedit /etc/systemd/logind.conf

And change the following value from the default to ignore:



HandleLidSwitch=ignore

If it is commented out (prefixed with #) then remove the # and save.

That's it! I can now take the machine, close the lid and set it up in a nice cool area with the external drives attached ready to run all the time. Ideally you should plug it into your router directly with ethernet to minimize wifi issues, however the laptop I have unfortunately has no ethernet, so I'll need to settle for wifi.

Creating the Project

Let's create a directory and initialize our Javascript project inside of it:



npm init -y

Next we install the three dependency libraries for the project:



npm install cron rsync dotenv

Here's what each one is used for:

cron: Will allow us to schedule the backup at specific intervals. This package uses a Javascript implementation of the cron syntax and not the actual cron daemon meaning we don't need to worry about OS compatibility issues with this package.
rsync: This will handle the copying and syncing of files for us. This package does use the actual rsync program installed on the user's machine so we will have to manage compatibility within our Node.js app for this one.
dotenv: Allows us to read .env files from our project directory. This will let us include our personal directory paths and also our private Discord webhook without sharing that data in the git repo. Users who clone the project can provide their own values.

We'll create a Javascript file called backup.js and get the absolute basics working:

backup.js



const CronJob = require("cron").CronJob;
const Rsync = require("rsync");

// Equivalent to writing `rsync -a example-source/ example-destination/` on terminal
rsync = new Rsync()
  // The -a flag means "archive" to say we are copying the full directory not just a file
  .flags("a")
  .source("example-source/")
  .destination("example-destination/");

const job = new CronJob(
  // Run this function once every minute
  // To learn more about this cron string visit the below link
  // https://crontab.guru/#*_*_*_*_*
  "* * * * *",
  () => {
    rsync.execute((error, code, cmd) => {
      // List of rsync status codes
      // https://stackoverflow.com/a/20738063
      console.log("backup completed with status code: " + code);
    });
  },
  null,
  true,
  // Replace with your time zone
  // https://gist.github.com/diogocapela/12c6617fc87607d11fd62d2a4f42b02a
  "America/Toronto"
);

// Begin the cronjob
job.start();

Also create directories called example-source and example-destination. Inside example-source create a TXT file inside called sample-file.txt with any content inside that file you like. Doesn't matter what it is, it's just used to confirm our backup is working.

Here's what things look like before the script has been run (note the empty example-destination directory):

After the script has run once:

Things are looking good, we have a once-every-minute interval backup of our example-source directory to our example-destination directory.

At this point you could replace those directory strings with whatever folders you like and create a regular directory backup system on your machine.

Let's make it easier to customize for those who are downloading and using our tool by adding dotenv package to read .env files as part of the configuration.

If you've already followed the tutorial you have the dotenv package installed with NPM so it's just a matter of importing it. Before we do let's create the .env file. Make sure to note that is .env with a starting . (to indicate hidden files):

.env



SOURCE_DIR="example-source/"
DESTINATION_DIR="example-destination/"
CRON_STRING="* * * * *"

Now we can update our code to read from that file. We just have to require the dotenv package at the top of our code:

backup.js



require("dotenv").config();
const CronJob = require("cron").CronJob;
const Rsync = require("rsync");

// Equivalent to writing `rsync -a example-source/ example-destination/` on terminal
rsync = new Rsync()
  // The -a flag means "archive" to say we are copying the full directory not just a file
  .flags("a")
  // Reads from the `.env` file in the project directory
  .source(process.env.SOURCE_DIR)
  .destination(process.env.DESTINATION_DIR);

const job = new CronJob(
  // Run this function once every minute
  // To learn more about this cron string visit the below link
  // https://crontab.guru/#*_*_*_*_*
  process.env.CRON_STRING,
  () => {
    rsync.execute((error, code, cmd) => {
      // List of rsync status codes
      // https://stackoverflow.com/a/20738063
      console.log("backup completed with status code: " + code);
    });
  },
  null,
  true,
  // Replace with your time zone
  // https://gist.github.com/diogocapela/12c6617fc87607d11fd62d2a4f42b02a
  "America/Toronto"
);

// Begin the cronjob
job.start();

When we run node backup.js again we get the same result, but this time our source and destination directories are being read from the .env file. This will make it easier for users to add their own source/destination directories and cron string when they download the tool.

It also improves privacy as we will be adding .env to our .gitignore file so the directories that I choose to copy on my machine will not be included in the git repository for this project.

In fact, let's do that now. If you're creating this project for yourself you'll want to be able to commit it to your remote git host, so run:



git init

Then create a .gitignore file in the root directory with:

.gitignore



node_modules
.env
nohup.out

We exclude .env for reasons mentioned above, and node_modules since it will be re-created by running npm install for anyone who uses our project. The last one nohup.out will contain some logs later in the tutorial that we don't need to share with others, so we're just adding it now in advance.

Excellent! You now have a great little tool that works on Linux and Mac... but what about Windows?

The truth is, basically all the development work I do is in a Linux environment.

Even though I do all my daily development on Windows 11 with WSL2 I'm still doing everything inside a native installing of Ubuntu despite logging into Windows each day.

I honestly don't even know how to use Windows cmd or PowerShell beyond the DOS commands I still remember from when I was a kid (dir?)... but if the majority of my personal stuff (photos and documents) are stored on Windows, maybe this is a good opportunity for me to learn?

I like learning new things! I challenge myself: what do I need to do to get this working on Windows?

Turns out it was surprisingly easy.

Cross Platform Support

Most of our app will work fine on Windows as-is, the big challenge here is rsync.

As you can see from that link, rsync is a Unix copying tool that will be natively available on most Linux and mac environments; but not Windows.

The rsync package on NPM is simply a wrapper around the tool installed on your OS, so running our backup.js in PowerShell gives us an error. The error is that the rsync program does not exist.

Here's the really cool thing though: not only does Windows have a very similar tool with a similar API called robocopy, the rsync NPM package allows us to chain a method called executable() that takes a string.

That string is the name of the copying tool we want to use.

It defaults to rsync, but we can provide it with any name we want.

We can check what OS the program is running on with process.platform which will return win32 as a string when running on Windows.

Let's update backup.js:

backup.js



require("dotenv").config();
const CronJob = require("cron").CronJob;
const Rsync = require("rsync");

// The value of process.platform will be:
// Windows: win32
// Mac: darwin
// Ubuntu: linux
const syncProgram = process.platform === "win32" ? "robocopy" : "rsync";

// Equivalent to writing `rsync -a example-source/ example-destination/` on terminal
rsync = new Rsync()
  .executable(syncProgram)
  // The -a flag means "archive" to say we are copying the full directory not just a file
  .flags("a")
  // Reads from the `.env` file in the project directory
  .source(process.env.SOURCE_DIR)
  .destination(process.env.DESTINATION_DIR);

const job = new CronJob(
  // Run this function once every minute
  // To learn more about this cron string visit the below link
  // https://crontab.guru/#*_*_*_*_*
  process.env.CRON_STRING,
  () => {
    rsync.execute((error, code, cmd) => {
      let result;
      if (error) {
        // List of rsync status codes
        // https://stackoverflow.com/a/20738063
        result = `Code ${code} ${error?.message}`;
      } else {
        result = "Backup complete";
      }

      const currentDate = new Date().toISOString();
      // Write log to the console, or will be redirected to a
      // nohup.out file if using nohup
      process.stdout.write(`${currentDate}: ${result}\n`);
    });
  },
  null,
  true,
  // Replace with your time zone
  // https://gist.github.com/diogocapela/12c6617fc87607d11fd62d2a4f42b02a
  "America/Toronto"
);

// Begin the cronjob
job.start();

Notice the changes above. I did a check against process.platform and if it returns win32 we set the executable copy program to robocopy instead of rsync which will allow it to run on Windows.

Lucky for us the syntax for robocopy is exactly the same as rsync:



robocopy <source> <destination>

This means we don't have to change anything else about the way our program already works, the rsync package will call robocopy exactly the same way when we're on Windows.

We're ready to try it out. In order to get the project files on Windows, I'm going to push them to Github and then clone them on by Windows filesystem.

So here's what I do: I push this project to Github. Then I open PowerShell.

(I am a PowerShell noob, but I'm doing my best.)

Turns out I don't even have git or node installed on Windows so I'm not going to get very far here.

First I need to download git so I can clone the project, then I need to download node so I can run it.

Once downloaded and installed I can run both these commands in PowerShell and get valid output:



PS C:\Users\ME> git --version
git version 2.34.1.windows.1

PS C:\Users\ME> node --version
v16.13.1

Now that everything is set I can git clone MY_PROJECT_URL and then cd into that directory and run:



npm install

Before I can run the project though I need to create the .env file since I did not include it in the repo for privacy reasons:

.env



SOURCE_DIR="example-source/"
DESTINATION_DIR="example-destination/"
CRON_STRING="* * * * *"

Finally now in PowerShell in the project directory i run:



node backup.js

And my result:

Before the script has been run on Windows (note the empty example-destination directory)

After the script has run once on Windows:

Note the status code doesn't necessarily match up with the rsync status codes, but the result is correct: the copying process was successful.

So that's really cool. You now have a tool that will copy the contents of one directory to another on an interval of your choice. Right now we have it setup to run every minute which is pretty overkill, but thanks to tools like crontab guru it's easy to create exactly the interval you want.

For example I only need to backup my directory once a week, so I'm going to set it to run at 3am every Sunday.

We now have a backup process that runs on a schedule and works on Linux, Mac and Windows!

But... how are we going to keep this running all the time?

If we simply use node backup.js the process is going to stop as soon as we close our terminal. We need this process to be running in the background, ideally 24/7.

We are going to need a better solution. Enter pm2

Running as a Background Process

Before we get into our final solution using pm2, I'd like to quickly show Mac/Linux users how you can get this to work without installing any addition tools at all using nohup.

You can skip this section on nohup if you like and go right to the final solution using PM2 if you like, it's just here to give you a bit of a deeper insight on how to create background processes using native Unix tools.

Using nohup (Optional - Mac and Linux Only)

You can find out if you system supports nohup with:



nohup --version



$ nohup --version
nohup (GNU coreutils) 8.30

If you get a version number successfully, then this next step should work for you.



nohup node backup.js &

The leading nohup will tell your system that even when your session ends you don't want the process to stop, and the final trailing & symbol means to run it as a background daemon process.

You will likely get an output that looks like:



[1] 7604

After running the command. This is the process ID of your node program. If for any reason you lose track of it you can find it again with this command:



pstree -p

You'll get output showing all the processes running on your system with their IDs. If you notice in the backup.js code example above we used process.title and gave it a string value of node-backup-script.

This helps find and identify the process ID when we use the pstree -p command:

Notice the node-backup-sc(7604) in there showing the same PID that we were given when the script started, and also the title value we set with process.title to make it easier to find and identify.

Since we can no longer simply ctrl+C to cancel the node script execution, we have to do something different. We have to kill the process by referencing the PID directly.

To do that you can run:



kill -9 YOUR_PID

Where YOUR_PID is the ID that is given on your machine. In my example above it's 7604. The -9 tells it to override anything that might stop or intercept a system kill signal, you want to end the program no matter what.

So the nohup option will work will for Unix systems that support it as long as they are running 24/7, the process will keep running. There are a couple of issues with it though:

If your program crashes, nohup will not reboot it
This solution is Unix specific and will not work on Windows

So how do we create a solution that will run 24/7, will reboot on crash, and supports cross-platform?

That's what pm2 is for.

Using PM2 (Cross Platform)

Using PM2 we can run our backup script in the background on any system, including Windows, all we need is to have NPM installed.

We'll use NPM to install PM2 globally:



npm install -g pm2

After installing you'll be able to verify it is available with:



pm2 --version

To run your backup script:



pm2 start backup.js

So now the process is running in the background and will keep running even if you close your terminal, and will reboot on crash.

You can view the running process anytime with pm2 list and you can stop it with pm2 stop backup where "backup" is the name of the process.

You can also view the logs of your app by running pm2 logs backup. You'll get an output that looks like:

Unfortunately this will not survive a full system reboot. Configuring pm2 to start with your app automatically on reboot is beyond the scope of this tutorial, but there are very good instructions on how to handle it if you would like to do.

Here's for Unix based systems and Windows.

You can now keep this running on an old laptop, or on your own cloud server like a $5 monthly Digital Ocean Droplet for example.

Just be careful when looking at Node app hosting solutions, there are a lot of options out there, but the majority of free ones (Heroku for example) will put your app to "sleep" sometimes, which isn't a viable option as it must be awake when the scheduled backup triggers in order to work properly.

Lastly we'll add a little bonus tutorial showing how to use webhooks to send the status output of our backups to a Discord bot so that we can keep track of it easily.

Adding a Discord Webhook (Bonus)

This section will teach you how to print the resulting status code from your backup action to a bot on your Discord server in addition to the PM2 logs.

Start by creating your webhook on your server. Follow this tutorial up to the end of the section called Making a Webhook. Stop when you get to Quick Example: GitHub Webhook Integration.

All you need is to get to the point where you can click the button that says "Copy Webhook URL". It will look like:



https://discord.com/api/webhooks/YOUR_WEBHOOK_ID

Where YOUR_WEBHOOK_ID is a long string of characters which may include additional slashes. Basically everything after the https://discord.com/api/webhooks/ you want to copy and paste into the WEBHOOK_ID key in your .env file.

.env



SOURCE_DIR="example-source/"
DESTINATION_DIR="example-destination/"
CRON_STRING="* * * * * *"
WEBHOOK_ID="YOUR_WEBHOOK_ID"

Next we'll update backup.js:

backup.js



require("dotenv").config();
const CronJob = require("cron").CronJob;
const Rsync = require("rsync");
const https = require("https");

process.title = "node-backup-script";

// Will be true if there is a Discord WEBHOOK_ID set in the `.env` file
const useDiscord = !!process.env.WEBHOOK_ID;

const options = {
  hostname: "discord.com",
  path: `/api/webhooks/${process.env.WEBHOOK_ID}`,
  method: "POST",
  headers: {
    "Content-Type": "application/json",
  },
};

// process.platform will be:
// Windows: win32
// Mac: darwin
// Ubuntu: linux
const syncProgram = process.platform === "win32" ? "robocopy" : "rsync";

// Equivalent to writing `rsync -a example-source/ example-destination/` on terminal
rsync = new Rsync()
  .executable(syncProgram)
  // The -a flag means "archive" to say we are copying the full directory not just a file
  .flags("a")
  // Reads from the `.env` file in the project directory
  .source(process.env.SOURCE_DIR)
  .destination(process.env.DESTINATION_DIR);

const job = new CronJob(
  // Run this function once every minute
  // To learn more about this cron string visit the below link
  // https://crontab.guru/#*_*_*_*_*
  process.env.CRON_STRING,
  () => {
    rsync.execute((error, code, cmd) => {
      let result;
      if (error) {
        // List of rsync status codes
        // https://stackoverflow.com/a/20738063
        result = `Code ${code} ${error?.message}`;
      } else {
        result = "Backup complete";
      }

      const currentDate = new Date().toISOString();
      // Write log to the console, or will be redirected to a
      // nohup.out file if using nohup
      process.stdout.write(`${currentDate}: ${result}\n`);

      // Only sends the request if WEBHOOK_ID is defined
      if (useDiscord) {
        // Send the request to Discord with the configured options
        const req = https.request(options, (res) => {
          // do nothing with Discord response
        });

        // Discord requires a { content: string } shape for posting messages
        req.write(
          JSON.stringify({
            content: result,
          })
        );

        req.end();
      }
    });
  },
  null,
  true,
  // Replace with your time zone
  // https://gist.github.com/diogocapela/12c6617fc87607d11fd62d2a4f42b02a
  "America/Toronto"
);

// Begin the cronjob
job.start();

Presuming you set up your webhook bot correctly, you will see it post a message on your Discord channel with the status of the backup every time the cron job fires.

You can use this as a convenient way to see the log output without having to log into your server and check manually. If there is an error, the Discord bot will print the error message. For example if I change the source to a folder that does not exist:

I can lookup the status code to get more information about the problem.

So we've now handled all of the major features required for this project!

Creating a backup of one directory to another
Support for scheduled times of backups
Cross platform support
Communication of success/error (either through PM2 logs or Discord)

Wrapping Up

I hope you learned something new from this tutorial, I know it was kind of a hodge-podge of different concepts, but that was a big part of the idea. One of my favourite ways of learning new things is to connect existing skills and tools in interesting ways.

Please check some of my other learning tutorials. Feel free to leave a comment or question and share with others if you find any of them helpful:

For more tutorials like this, follow me @eagleson_alex on Twitter.

Introduction to Docker for Javascript Developers (feat Node.js and PostgreSQL)

Alex Eagleson — Wed, 22 Dec 2021 13:24:29 +0000

All code from this tutorial as a complete package is available in this repository. If you find this tutorial helpful, please share it with your friends and colleagues!

For more tutorials like this, follow me @eagleson_alex on Twitter

A video version is also available:

Introduction
What is Docker?
Prerequisites
Installing Docker
Creating a Container
Creating a Node App
Deprecating the Node App
Creating a Dockerfile
Docker Layers and Cache
Adding a Docker Volume
What is Docker-Compose?
Adding a Database
Connecting the App to the Database
Adding a Frontend
Creating a Docker Compose YML File
Adding a pgAdmin Panel (Bonus)
Useful Docker Commands
Wrapping Up

Introduction

In this tutorial you will learn what Docker is and what purpose it serves by building a fullstack Node.js app complete with frontend and PostgreSQL database.

We will use Docker Compose to connect and network each container together so that they are easy to share among project contributors, and deploy to whatever hosting service you've chosen.

What is Docker?

That said, it can be challenging to really demonstrate the need for Docker to those new to the development world who haven't yet experienced a lot of the problems that it solves.

This tutorial aims to simulate a a couple of realistic scenarios you might encounter in a work environment, and show how Docker helps to solve those issues.

The Scenario

There are two common development issues we will be replicating with this example:

Your company's project relies on an older version of a tool (in our case Node.js) than the development team has installed on their machine
We want to make it easy to test the application with a copy of the database on the developers' local machine, without requiring them to install the database software (in our case PostgreSQL)

If you follow this tutorial you will have a working application running on your machine and querying a Postgres DB without the need to have either Node.js or Postgres installed. The only tool you will need is Docker.

scalability

Prerequisites

The only prerequisite software required to have installed for this tutorial is an IDE (code editor, I use VS Code) and Docker.

How you install Docker will depend on the operating system you are running. I am running it on WSL2 on Windows 11 which is a fantastic experience. It works just as well on Mac and Linux, you simply need to follow the installation instructions for your OS.

I recommend Docker Desktop which will give you a nice GUI for working with Docker, however it is not required. For this tutorial will will be managing Docker entirely through the command line (though I may use Docker Desktop for screenshots to show what is happening).

I also suggest having Node.js installed as well. Technically you can get away without it, but in the first couple of steps we're going to run the app locally before we get Docker involved. It will also help demonstrate how Docker fixes our versioning issue.

Installing Docker

Once you have Docker installed let's make sure that it works. When you type:

docker --version

You should get a version number (as opposed to "not found"). My version right now shows 20.10.11, but any version close to that number should work fine.

Most containers are hosted on a service called Docker Hub, including the ones we will be using.

Let's begin by testing the simplest container called hello-world.

Creating a Container

Run the following command to download the hello-world image:

docker pull hello-world

That will pull the image from Docker hub. Important to get the terminology correct, we haven't created a container yet. A Docker image is a set of instructions for how to create a container. If you are familiar with web development, think of the image like HTML (blueprints) and the container like the DOM (the structure).

You can add additional instructions to the default image instructions in your Dockerfile which we will get to soon.

Presuming you got a success message like Status: Image is up to date for hello-world:latest, you are ready to create a container.

docker run hello-world

If successful, you will see this output in your terminal:

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

Congratulations! You have run your first Docker container! Although you can manage it very easily if you are using Docker Desktop, let's look at a couple of the most common commands to manage it on the command line:

docker image ls

# OR

docker container ls

Will show you a list of all the images or containers you have on your system at the moment. Because hello-world stops as soon as it's finished printing the test message, it does not keep running forever like a container running a web app would. You won't see it in your list of containers, but you will see it in your list of images.

Both the ID of the image/container and the name are important to be able to lookup because they allow you to refer to those images/containers to start/stop them.

When you stop running a container it doesn't get deleted. That is a good thing! It means it's super fast to just start it up again the next time you need it without downloading and installing it again.

While working with Docker you'll find that sometimes these images and containers begin to pile up when you change things or build new versions. To quickly remove all old/unused ones you can run:

docker image prune

# OR

docker container prune

If these don't seem too helpful now, don't worry, but keep them in mind because you will likely want to refer back to them later.

Creating a Node App

Before we get any further into Docker, let's build a small web app we can use to help demonstrate some of the more advanced features of Docker. We're going to build a simple web server in Node.js and Express:

I've created a new empty directory called docker-template and initialized an NPM repo inside of it.

mkdir docker-template
cd docker-template
npm init
npm install express

server.js

const express = require("express");
const app = express();
const port = 8080;

app.get("/", async (req, res) => {
  res.setHeader("Content-Type", "text/html");
  res.status(200);
  res.send("<h1>Hello world</h1>");
});

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`);
});

Now run your app with:

node server.js

And go to http://localhost:8080 to see:

One extra thing we would like to enable for this project is file watching and automatic reloading of the server whenever the file is changed.

The easiest way to do that is a tool called nodemon.

npm install nodemon --save-dev

Then add a start script to your package.json file:

package.json

{
  "name": "server",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "start": "nodemon server.js"
  },
  "author": "me",
  "license": "ISC",
  "dependencies": {
    "express": "^4.17.2",
  },
  "devDependencies": {
    "nodemon": "^2.0.15"
  }
}

Run your app with:

npm run start

Try editing your server.js file when your app is running (change "hello world" to "hello world!!!!" or something) and verify that your Node app reloads and you see the change in your browser when you hit the refresh button (the file watching won't trigger a browser refresh automatically).

Once that is working continue to the next step!

Deprecating the Node App

This next part is kinda fun. Let's intentionally turn this server into a legacy project.

We'll be assuming you are running a recent version of Node (15 or later). You can check by running:

node --version

My output is v16.11.1. If yours is older than 15 you can either use NVM or just read along. Installing a specific Node version of your machine is not required for this part. In fact it's exactly the problem we're going to solve with Docker in the next section.

In Node 15 there is a breaking change in the way that unhandled rejected promises are handled. Before version 15 a Javascript promise that was rejected without a catch would give a warning and keep running, but after v15 of Node an unhandled promise will crash the program.

So it's possible for use to add some code that will make our server work on versions of Node older than 15, but not work on new versions of Node.

Let's do that now:

server.js

// @ts-check

const express = require("express");
const app = express();
const port = 8080;

app.get("/", async (req, res) => {
  res.setHeader("Content-Type", "text/html");
  res.status(200);
  res.send("<h1>Hello world</h1>");
});

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`);
});

const myPromise = new Promise((resolve, reject) => {
  setTimeout(() => {
    resolve("good");
  }, 300);
  reject("bad");
});

myPromise.then(() => {
  console.log("this will never run");
});

The code above creates a new promise that always rejects. It will run (with a warning) on Node.js v14, but will crash on v15 and above. Try running it yourself on v15 and above and you get code: 'ERR_UNHANDLED_REJECTION'.

Now obviously we could just... add a catch block (or remove the code entirely), but we are trying to replicate a scenario where you are working with an older codebase and you may not necessarily have those options available to you.

Let's say for one reason or another this app must be run on Node v14 or earlier to work. Every developer on the team must be prepared to operate in that environment... but our company also has a new app that runs on Node v17! So we need that environment available too.

And while we're at it, some other tool on version X! I only have version Y on my machine! Who knows what version the other members of my team are running. Or the guy I send the app to for testing.

What do I do!?

Enter Docker.

Creating a Dockerfile

With Docker we can use code to generate the environment that our app runs in. We'll begin by searching Docker hub for a Node.js image. The official Node image is just called node.

You'll notice when you look at supported tags there are a lot of versions. Just like having a certain version on your machine, there's Docker images for pretty much every version you could want. Of course Node itself needs to be installed on some kind of operating system so that's usually the other part of the tag.

The default Node image runs on Debian, however one of the most popular versions runs on something called Alpine Linux.

The main reason Alpine is popular is because of its small size, it's a distro of Linux designed to strip out all but the most necessary parts. This means it will be faster and more cost effective to run and distribute our app on this image (assuming it meets our needs).

For our simple app, it does.

Remember we specifically want an older version of Node (older than v15 so our app runs without crashing) so I am going to choose the image tagged node:14-alpine3.12. That's Node v14 and Alpine v3.12.

We can pull the image in advance with docker pull node:14-alpine3.12 just like we did with hello-world, but it's not necessary. By adding it to our Dockerfile Docker will automatically pull it from Docker Hub if it doesn't find it on our machine.

Let's create a file called Dockerfile (no extension) in the root of our project next to server.js:

Dockerfile

# select your base image to start with
FROM node:14-alpine3.12

# Create app directory
# this is the location where you will be inside the container
WORKDIR /usr/src/app

# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm@5+)
# copying packages first helps take advantage of docker layers
COPY package*.json ./

RUN npm install
# If you are building your code for production
# RUN npm ci --only=production

# Bundle app source
COPY . .

# Make this port accessible from outside the container
# Necessary for your browser to send HTTP requests to your Node app
EXPOSE 8080

# Command to run when the container is ready
# Separate arguments as separate values in the array
CMD [ "npm", "run", "start"]

I've added a lot of comments to help explain each piece of the Dockerfile. You can learn more about Dockerfiles here, I would highly encourage you to skim through that page to get familiar with the commands that are available.

Before we continue I would like to touch briefly on Docker's layers & cache because they are very important topics!

Docker Layers and Cache

One common question for a simple Dockerfile like this is:

"Why are you using the COPY command twice? Isn't the first COPY unnecessary since the second one copies the whole directory?"

The answer is actually "no" and the reason is because of one of Docker's best features called layers.

Every time you use one of FROM, COPY, RUN, CMD it creates another image which is based off the previous layer. That image can be cached and only needs to be created again if something changes.

So by creating a specific COPY line on package-*.json we are creating a layer that is based off the content of that file before we run npm install. That means that unless we change package.json, the next time we build Docker will use the cache layer where npm install has already been run and we don't have to install all dependencies every time we run docker build. That will save us an enormous amount of time.

The next COPY looks at every file in our project directory, so that layer will be rebuilt on any file change (basically any time we update anything OTHER than package.json in our app). But that's exactly what we want.

This is just one example of efficiencies you can take advantage of when working with Docker, but I would encourage you to read the whole list of best practices for Dockerfiles.

Building the App Container

Now that your Dockerfile is created we have just one last thing we need to do before we build.

Similar to .gitignore that you're probably familiar with (used to prevent committing auto-generated files and private secrets to public repositories), Docker has a similar concept to keep you from unnecessarily copying files that your container doesn't need.

Let's create a .dockerignore file now:

.dockerignore

node_modules
npm-debug.log

Both of those will be generated inside the container, so we don't want to copy our local versions of them over.

At this point we are ready to build. Run the following command:

docker build . -t my-node-app

That will build the image describe by the Dockerfile in the current directory . and give it a name called my-node-app. When it's done you can see the image and all its details with:

docker image ls

With the image created, we are now ready to build a container off our image which will run our app:

docker run -p 3000:8080 --name my-node-app-container my-node-app

This command tells Docker to use our image to build a running container. The --name flag lets us name the container (to make it easier to identify and stop/start later, otherwise the name will be randomly generated).

I used the name my-node-app-container to differentiate it from the last argument which is the name of the image we are building from (my-node-app).

We use the -p flag to bind ports from our host machine (our computer) environment to the container environment.

If you recall we wrote EXPOSE 8080 in our Dockerfile which is the port our app runs on. The above command maps port 3000 on our machine to port 8080 in the container.

(Note you can map the same port like 8080:8080 if you like, we just mixed it up in this example to show that it's possible)

Double check that your container started successfully with:

docker container ls

My output looks like:

CONTAINER ID   IMAGE         COMMAND                  CREATED         STATUS         PORTS                    NAMES
b6523b2602e1   my-node-app   "docker-entrypoint.s…"   6 minutes ago   Up 6 minutes   0.0.0.0:3000->8080/tcp   my-node-app-container

(Sorry if the words wrap and it make it difficult to line things up)

We can see that the container is up for X minutes. That means that our app is running on port 8080, we can access that port on our machine using port 3000 so open up your browser to http://localhost:3000/ to see:

Great! You've created your first custom Docker image and container with your own app running in it!

So now that you have your environment setup, naturally one of the next things you might want to do is update your app. If you make a change to server.js and save the file, are you going to see those changes when you reload the page?

No you won't. The app is running based on a copy of server.js inside the container which has no direct relation to the one in your project directory. Is there a way that we can "connect" them somehow?

Of course there is, we need to introduce Docker volumes.

Adding a Docker Volume

Docker uses the concept of volumes to allow you to persist data between running containers.

You can imagine you might want to have your app save some data, but with the way Docker works your containers are designed to be destroyed and recreated casually.

There are two primary ways to use volumes. You can create one in advance and give it a name. This will save all the volume data by default in the /var/lib/docker/volumes directory (in a Linux environment, it would be somewhere different but equivalent on Windows).

To create a named volume (you don't need to run this command for this tutorial, it's simply an example):

docker volume create my-named-volume

Then you would map any directory in your container to that directory on your machine. You can do so by adding the --volume flag to your docker run command like so: --volume my-named-volume:/usr/src/app my-node-app.

That example would map the working directory in your container to the Docker volume on your machine. This does not help us however because we want to synchronize a specific directory (our project directory) with the one in the container so we can edit files in our project and have them update in the container.

We can do this as well.

First we need to stop the existing container (which doesn't have a volume), remove it, and then run it again with the volume:

docker container stop my-node-app-container

docker container rm my-node-app-container

docker run -p 3000:8080 --name my-node-app-container --volume  ${PWD}:/usr/src/app my-node-app

In most terminals PWD means "print working directory" so it will map the current directory to the /usr/src/app directory inside your container. This will accomplish our goal of syncing the files between our project on our computer and the one in our container.

Since we have already set up file watching and reloading with nodemon earlier in the tutorial, you should now be able to edit server.js in your project directory while the container is running (just edit the hello world text), then refresh your browser to see the changes.

And that's it! You now have a Dockerized Node app where you can make changes on your machine and see the updates happen live inside your container.

At this point we have mostly completed our introduction to Docker itself. We have completed our implementation of our first "scenario" where we use coded instructions to recreate the environment that our app requires in order to operate.

We now need to address our second common scenario: in order to function our application relies on other services, like a database for example. We could technically add the instruction to install the database in our Dockerfile, but that would not realistically mimic the environment our app would be deployed in.

It's not guaranteed that our Node app and our database would be hosted on the same server. In fact it's probably not even likely. Not only that, we don't want to have to boot up our web server to make edits to our database, and vice-versa. Is there a way that we can still use Docker, but create a separation between multiple services that rely on each other?

Yes we can.

What is Docker-Compose?

Best described in their own words:

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

The process is to define the instructions for each of your services with Dockerfiles, and then use Docker Compose to run all those containers together and facilitate network communications between them.

In this tutorial we are going to connect our Node app to a PostgreSQL database. Before we can connect them of course we need to establish the database container.

Adding a Database

Similar to Node, Docker Hub has a super simple easy to use image for PostgreSQL. Of course theres also images for MySQL, Mongo, Redis, etc, etc. There's no reason you couldn't substitute your favourite out if you want (though if you're still new to Docker I'd suggest you stick with the tutorial for now).

We search Docker Hub for the official postgres image. We don't need anything beyond the bare minimum so once again we'll choose the version running on Alpine. Image postgres:14.1-alpine.

Unlike our Node image, we don't need to copy any files or run any installation scripts, so we don't actually need a Dockerfile for our PostgreSQL installation. There are some configurations that we do need (like password and ports for example) but we can manage those with our upcoming docker-compose.yml file.

So aside from deciding which image you are going to use, there is really nothing else we need to do before we create our config file.

Connecting the App to the Database

Before we create the Docker Compose configure file to link the database container, we need to update our app to actually use it.

Our goal here is going to be to create a database with some very simple data (like a list of employees), see it with some sample data, and then query that data with our Node app.

We'll also create a simple frontend to display that data.

First we need to install the PostgreSQL NPM package:

npm install pg

Next we are going to create a .sql file that will automatically seed out database with some sample data to read from. In the root of the project create the following file:

database-seed.sql

CREATE TABLE employees
(
    id SERIAL,
    name text,
    title text,
    CONSTRAINT employees_pkey PRIMARY KEY (id)
);

INSERT INTO employees(name, title) VALUES
 ('Meadow Crystalfreak ', 'Head of Operations'),
 ('Buddy-Ray Perceptor', 'DevRel'),
 ('Prince Flitterbell', 'Marketing Guru');

(Note I got those ridiculous names from the random name generator on the "whimsical" setting)

Next we update our Node server to query these values. In addition to doing that, we are going to use express.static to serve up an entire directory rather than just sending HTML as sa string. This will allow us to serve an HTML file along with some CSS and Javascript as well, to create a full-fledged frontend.

Comments are added to explain all the new pieces:

server.js

// Import the postgres client
const { Client } = require("pg");
const express = require("express");
const app = express();
const port = 8080;

// Connect to our postgres database
// These values like `root` and `postgres` will be
// defined in our `docker-compose-yml` file
const client = new Client({
  password: "root",
  user: "root",
  host: "postgres",
});


// Serves a folder called `public` that we will create
app.use(express.static("public"));

// When a GET request is made to /employees
// Our app will return an array with a list of all
// employees including name and title
// this data is defined in our `database-seed.sql` file
app.get("/employees", async (req, res) => {
  const results = await client
    .query("SELECT * FROM employees")
    .then((payload) => {
      return payload.rows;
    })
    .catch(() => {
      throw new Error("Query failed");
    });
  res.setHeader("Content-Type", "application/json");
  res.status(200);
  res.send(JSON.stringify(results));
});

// Our app must connect to the database before it starts, so
// we wrap this in an IIFE (Google it) so that we can wait
// asynchronously for the database connection to establish before listening
(async () => {
  await client.connect();

  app.listen(port, () => {
    console.log(`Example app listening at http://localhost:${port}`);
  });
})();

const myPromise = new Promise((resolve, reject) => {
  setTimeout(() => {
    resolve("foo");
  }, 300);
  reject("oops");
});

myPromise.then(() => {
  console.log("hello");
});

In the above code update you can see that we are serving up a directory called public that we have not created yet. That directory will contain an index.html file to act as the nice looking frontend for our app.

Adding a Frontend

We'll begin by creating the public directory that is being served from our Node app:

mkdir public

Then add the following files:

public/index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>My Docker Template</title>
    <script src="script.js"></script>
    <link rel="stylesheet" href="styles.css" />
  </head>
  <body>
    <template>
      <div class="card">
        <img src="https://res.cloudinary.com/dqse2txyi/image/upload/v1639943067/blogs/docker-node/profile-picture_eav2ff.png" alt="Avatar" width="240px" />
        <div class="container">
          <h4>Placeholder</h4>
          <p>Placeholder</p>
        </div>
      </div>
    </template>
  </body>
</html>

Our index.html file takes advantage of HTML templates for the employee cards.

public/styles.css

body {
  padding: 12px;
  display: flex;
  flex-direction: row;
  column-gap: 24px;
}

.card {
  box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2);
  transition: 0.3s;
  border-radius: 5px;
  transition: 0.3s;
}

.card:hover {
  transform: scale(1.03);
}

.container {
  padding: 0 12px;
}

img {
  border-radius: 5px 5px 0 0;
}

Above in styles.css is some simple CSS to give a clean look to the employee card templates, and flex them out in a row across the page.

public/script.js

fetch("/employees")
  .then((response) => response.json())
  .then((data) => {
    data.forEach((employee) => {
      // Select the <template> we created in index.html
      const cardTemplate = document.querySelector('template');

      // Clone a copy of the template we can insert in the DOM as a real visible node
      const card = cardTemplate.content.cloneNode(true);

      // Update the content of the cloned template with the employee data we queried from the backend
      card.querySelector('h4').innerText = employee.name;
      card.querySelector('p').innerText = employee.title;

      // Append the card as a child with the employee data to the <body> element on our page
      document.body.appendChild(card);
    });
  });

When our app is loaded it will load script.js which will use the browser fetch API to query the /employees route on our Node server and get the employee information from out PostgreSQL database.

Once it is returned it will iterate through each employee and clone the HTML template that we defined in index.html to make a custom employee card with that employee's name and title.

Phew! Now that we have our app established and ready to read from the database, we are finally ready to connect our Node container and our PostgreSQL container together with Docker Compose.

Creating a Docker Compose YML File

For a brief intro to compose see here, and for more details than you can ever handle about the compose file spec see here.

We're going to be creating a simple docker-compose.yml file to link our Node app with our PostgreSQL database. Let's jump right in and create the file in our project root directory. I'll use lots of comments to explain everything:

docker-compose.yml

version: '3.8'
services:
  # These are the configurations for our Node app
  # When Docker Compose starts this container it will automatically
  # use the Dockerfile in the directory to configure it
  app:
    build: .
    depends_on:
      # Our app does not work without our database
      # so this ensures our database is loaded first
      - postgres
    ports:
      - "8080:8080"
    volumes:
      # Maps our current project directory `.` to
      # our working directory in the container
      - ./:/usr/src/app/

  # This is the configuration for our PostgreSQL database container
  # Note the `postgres` name is important, in out Node app when we refer
  # to  `host: "postgres"` that value is mapped on the network to the 
  # address of this container.
  postgres:
    image: postgres:14.1-alpine
    restart: always
    environment:
      # You can set the value of environment variables
      # in your docker-compose.yml file
      # Our Node app will use these to connect
      # to the database
      - POSTGRES_USER=root
      - POSTGRES_PASSWORD=root
      - POSTGRES_DB=root
    ports:
      # Standard port for PostgreSQL databases
      - "5432:5432"
    volumes:
      # When the PostgreSQL container is started it will run any scripts
      # provided in the `docker-entrypoint-initdb.d` directory, this connects
      # our seed file to that directory so that it gets run
      - ./database-seed.sql:/docker-entrypoint-initdb.d/database-seed.sql

So with that docker-compose.yml file in place we are finally ready to run our new and highly improved application "suite" that includes a backend, frontend and database.

From the root directory of the project, all your have to do is type:

docker-compose up --build

(Note the --build flag is used to force Docker to rebuild the images when you run docker-compose up to make sure you capture any new changes. If you simply want to restart existing containers that haven't changed you can omit it)

Once active you can finally test it out. In our docker-compose.yml config we are mapping post 8080 directly to 8080 so go to http://localhost:8080 to see:

With a cute little hover transition and everything! Congratulations!

If you are using the Docker Desktop GUI application you'll have a lot of options to stop all the containers at once, or view each one individually. If you are using the command line you can stop both containers with this simple command (run from the project root directory for context):

docker-compose down

And there you have it, a fullstack Node.js application with its own SQL database bundled along with it. You can now deploy this literally anywhere that has Docker installed and you know that it will work because you have defined all the parameters of the exact environment it needs to function.

Adding a pgAdmin Panel (Bonus)

Here's a quick little bonus for those of you who are using PostgreSQL. Adding the pgAdmin panel container to this app setup is a breeze. Simply update your docker-compose.yml config to include the following:

docker-compose.yml

version: '3.8'
services:
    app:
        build: .
        depends_on:
            # Our app does not work without our database
            # so this ensures our database is loaded first
            - postgres
        ports:
            - "8080:8080"
        volumes:
            # Maps our current project directory `.` to
            # our working directory in the container
            - ./:/usr/src/app/

    # This is the configuration for our PostgreSQL database container
    # Note the `postgres` name is important, in out Node app when we refer
    # to  `host: "postgres"` that value is mapped on the network to the 
    # address of this container.
    postgres:
        image: postgres:14.1-alpine
        restart: always
        environment:
            # You can set the value of environment variables
            # in your docker-compose.yml file
            # Our Node app will use these to connect
            # to the database
            - POSTGRES_USER=root
            - POSTGRES_PASSWORD=root
            - POSTGRES_DB=root
        ports:
            # Standard port for PostgreSQL databases
            - "5432:5432"
        volumes:
            # When the PostgresSQL container is started it will run any scripts
            # provided in the `docker-entrypoint-initdb.d` directory, this connects
            # our seed file to that directory so that it gets run
            - ./database-seed.sql:/docker-entrypoint-initdb.d/database-seed.sql

    pgadmin-compose:
        image: dpage/pgadmin4
        environment:
            PGADMIN_DEFAULT_EMAIL: "placeholder@example.com"
            PGADMIN_DEFAULT_PASSWORD: "fakepassword123!"
        ports:
            - "16543:80"
        depends_on:
            - postgres

Notice the pgAdmin panel configuration added at the bottom.

When you run docker-compose up --build now and go to:

http://localhost:16543/

You'll be greeted with the pgAdmin panel. Enter the PGADMIN_DEFAULT_EMAIL and PGADMIN_DEFAULT_PASSWORD credentials from the docker-compose.yml file to access it.

Once inside click Add New Server.

For General -> Name pick a name. Can be whatever you want.

On the Connection tab values must match the docker-compose.yml file:

Host: postgres
Username: root
Password: root

Now you can navigate from the left bar:

Servers -> whatever-you-want -> Databases -> root -> Schemas -> public -> Tables -> employees

Right click employees an Query Tool:

SELECT * FROM employees;

To see your data.

Useful Docker Commands

List all containers, images, volumes or networks, for example docker image ls.

docker {container}/{image}/{volume}/{network} ls

Remove a container, image, volume or network where ID is the id of the container/image/volume or network.

docker {container}/{image}/{volume}/{network} rm ID

Start a container in the background (as a daemon):

docker run -d IMAGE_ID

View logs of a container:

docker container logs CONTAINER_ID

View information about a container:

docker container inspect CONTAINER_ID

Open a shell inside an active container so you can run terminal commands inside of it.

docker exec -it CONTAINER_ID /bin/sh

Stop a container:

docker container stop CONTAINER_ID

Remove all dangling/unused Docker data (cached layers, volumes no longer used, etc):

docker system prune

You can also use the above command with a specific type, like docker container prune.

Wrapping Up

I hope you learned a lot about why Docker is a fantastic tool in your toolbelt and how you can use it to reduce the amount of friction related to setting up development environments. The days of fighting with WAMP and MAMP and XAMPP are thankfully long behind us (no slight against those apps, I know they're fantastic tools when configured properly).

Remember that Docker can be used both to create a baseline standard development environment on the machines of many different developers. But it's not just a development tool, Docker can be used in production as well to simplify the process fo scaling your app up with increased traffic by simply deploying more containers.

And there's plenty more to learn well beyond what's covered here, the Docker docs are the best place to start. All the best in your Docker journey.

Please check some of my other learning tutorials. Feel free to leave a comment or question and share with others if you find any of them helpful:

For more tutorials like this, follow me @eagleson_alex on Twitter

How to Create a React Typescript Monorepo with Git Submodules

Alex Eagleson — Fri, 17 Dec 2021 13:12:16 +0000

All code from this tutorial as a complete package is available in this repository and a video version of this tutorial is available below:

Do you work with large or semi-large codebases that are starting to get out of control? Do you have to deal with multiple different projects that interact with each other and have difficulty keeping versions aligned?

If you said yes to either of those things (or even if you're just anticipating encountering them in the future) then this tutorial is for you.

The purpose of this tutorial is to learn about some of the different ways that you can structure a large project which is composed primarily of smaller projects and modules.

Monorepos

One method of grouping code from multiple projects into one is called a monorepo. A monorepo is simply the practice of placing multiple different projects that are related in some way into the same repository.

The biggest benefit is that you do not need to worry about version mismatch issues between the different pieces of your project. If you update an API route in the server of your monorepo, that commit will be associated with the version of the front end that consumes it. With two different repositories you could find yourself in a situation where your v1.2 front-end is asking for data from your v1.1 backend that somebody forgot to push the latest update for.

Another big benefit is the ability to import and share code and modules between projects. Sharing types between the back-end and front-end is a common use case. Your can define the shape of the data on your server and have the front-end consume it in a typesafe way.

Git Submodules

In addition to monorepos, we also have the concept of submodules.

Let's say that we want to add a feature to our app that we have in another separate project. We don't want to move the entire project into our monorepo because it remains useful as its own independent project. Other developers will continue to work on it outside of our monorepo project.

We would like a way to include that project inside our monorepo, but not create a separate copy. Simply have the ability to pull the most recent changes from the original repository, or even make our own contributions to it from inside our monorepo. Git submodules allows you to do exactly that.

This tutorial will teach you how to create your own project that implements both of these features.

Prerequisites and Setup
Initializing the Project
Create the React App
Create the Monorepo
Create Your Repository
Sharing Code and Adding Dependencies
Create a Shared Package
Add a Git Submodule
Namespacing
Wrapping Up

Prerequisites and Setup

This tutorial assumes you have a basic familiarity with the following. Beginner level experience is fine for most as the code can be simply copy/pasted. For git you should know how to clone, pull, commit and push.

Git
React
Node.js
Typescript
NPM

This tutorial requires yarn v1 installed (we use v1.22).

Initializing the Project

To start, we need a packages directory to hold the different projects in our monorepo. Your structure should begin looking like this:

.
└── packages
    └── simple-express-app
          └── server.ts

From within the `packages/simple-express-app` directory, run:

yarn init

yarn add express

yarn add -D typescript @types/express

npx tsc --init

The final command will create a tsconfig.json file. Add the following to it:

packages/simple-express-server/tsconfig.json

{
  ...
  "outDir": "./dist",
}

Now create your server file if you haven't yet:

packages/simple-express-server/server.ts

import express from 'express';
const app = express();
const port = 3001;

app.get("/data", (req, res) => {
  res.json({ foo: "bar" });
});

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`);
});

At this point your directory structure should look like:

.
└── packages
    └── simple-express-app
          ├── server.ts
          ├── yarn.lock
          ├── package.json
          └── tsconfig.json

We'll create a simple script in package.json called start that we can run with yarn:

packages/simple-express-server/package.json

{
  "name": "simple-express-server",
  "version": "1.0.0",
  "main": "dist/server.js",
  "license": "MIT",
  "scripts": {
    "start": "tsc && node dist/server.js"
  },
  "devDependencies": {
    "@types/express": "^4.17.13",
    "typescript": "^4.5.4"
  },
  "dependencies": {
    "express": "^4.17.1"
  }
}

Open your browser to and you will see your data successfully queried:

Create the React App

Next we move onto our React app. Navigate to the packages directory and run this command:

yarn create react-app simple-react-app --template typescript

Before we do anything else we want to confirm that we can communicate with our server and get the JSON data that we are serving up.

Open up the App.tsx file in the src directory of the project generated by create-react-app. We are going to add a simple button that uses the browser fetch API to grab the data from our server and log it to the console.

packages/simple-react-app/src/App.tsx

import React from "react";
import logo from "./logo.svg";
import "./App.css";

function App() {
  return (
    <div className="App">
      <header className="App-header">
        <img src={logo} className="App-logo" alt="logo" />
        <p>
          Edit <code>src/App.tsx</code> and save to reload.
        </p>
        <a
          className="App-link"
          href="https://reactjs.org"
          target="_blank"
          rel="noopener noreferrer"
        >
          Learn React
        </a>

        { /* NEW */ }
        <button
          onClick={() => {
            fetch("http://localhost:3001/", {})
              .then((response) => response.json())
              .then((data) => console.log(data));
          }}
        >
          GET SOME DATA
        </button>

      </header>
    </div>
  );
}

export default App;

When we open the browser's development console (F12) and then click our button, we will see our server data fetched and logged in the browser:

This is great! We've accidentally created a template for a full stack React and Typescript app! But that's not the reason we're here, so let's start pushing further into scenarios we might encounter in real projects that would lead us to consider options like a monorepo and git submodules.

Before you continue take a moment to verify your project structure:

.
└── packages
    ├── simple-express-server
    │   ├── server.ts
    │   ├── yarn.lock
    │   ├── package.json
    │   └── tsconfig.json
    └── simple-react-app
        └── [default setup]

Create the Monorepo

To manage our monorepo we are going to use two tools:

Lerna: For running scripts across multiple projects and adding new dependencies. Lerna is also built to manage publishing your packages (though we will not be doing that as part of this tutorial)
Yarn workspaces: For hoisting all shared dependencies into a single node_modules folder in the root directory. Each project can still define its own dependencies, so that you don't confuse which dependencies are required for which (client vs. server) for example, but it will pool the installed packages in the root.

For yarn we are using the still most commonly used yarn v1 (current version as of this writing is v1.22).

Navigate to the root directory and run the following commands:

yarn init

yarn add -D lerna typescript

npx lerna init

Edit your Lerna configuration file:

{
  "packages": ["packages/*"],
  "version": "0.0.0",
  "npmClient": "yarn",
  "useWorkspaces": true
}

We need to specify that yarn is our NPM client and that we are using workspaces.

Next we need to define the location of those workspaces in the root package.json:

package.json

{
  "name": "monorepo-example",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "private": true,
  "workspaces": [
    "packages/*"
  ],
  "scripts": {
    "start": "lerna run --parallel start"
  },
  "devDependencies": {
    "lerna": "^4.0.0"
  }
}

We have made three changes above:

Set private to true which is necessary for workspaces to functions
Defined the location of the workspaces as packages/* which matches any directory we place in packages
Added a script that uses Lerna to run. This will allow us to use a single command to run the equivalent of yarn start in both our Express server and React app simultaneously. This way they are coupled together so that we don't accidentally forget to run one, knowing that currently they both rely on each other. The --parallel flag allows them to run at the same time.

Now we are ready to install the dependencies in root:

(Note: At this point before you run the install command, I would recommend you synchronize your Typescript version between your simple-express-server and the one that comes bundled with your simple-react-app. Make sure both versions are the same in each project's package.json and both are listed in devDependencies. Most likely the React app version will be older, so that is the one that should be changed.)

Next run the following command:

npx lerna clean -y

yarn install

The first command will clean up the old node_modules folders in each of your two packages. This is the equivalent of simply deleting them yourself.

The second command will install all dependencies for both projects in a node_modules folder in the root directory.

Go ahead and check it out! You'll see that node_modules in the root is full of packages, while the node_modules folders in simple-express-server and simple-react-app only have a couple (these are mostly symlinks to binaries that are necessary due to the way yarn/npm function).

Before we go on we should create a .gitignore file in the root to make sure we don't commit our auto-generated files:

.gitignore

node_modules/
dist/

(If you're using VS Code you'll see the folder names in the side bar go grey as soon as you sae the file, so you know it worked)

Verify your monorepo and workspaces are setup properly by running (from the root folder):

yarn start

You will see that both your Express app and React app start up at the same time! Click the button to verify that your server data is available and logs to the console.

Lastly we need to initialize Typescript in the root of the project so that our different packages can import and export between one another. Run the command:

npx tsc --init

In the root directory and it will create your .tsconfig.json. You can delete all the defaults values from this file (your individual projects will se their own configuration values.) The only field you need to include is:

tsconfig.json

{
  "compilerOptions": {
    "baseUrl": "./packages"
  }
}

Our project now looks like:

.
├── packages
|   ├── simple-express-server
|   │   ├── server.ts
|   │   ├── yarn.lock
|   │   ├── package.json
|   │   └── tsconfig.json
|   └── simple-react-app
|       └── [default setup]
├── lerna.json
├── tsconfig.json
├── package.json
└── yarn.lock

Create Your Repository

This is also a good time to commit your new project to your repository. I'll be doing that now as well, you can see the final version here.

Note that in order to learn submodules effectively, we are going to be adding a submodule from a repository that already exists, we don't want to use the one that create-react-app generated automatically.

So for that reason I am going to delete the that repository by deleting the .git directory inside packages/simple-react-app. This step is VERY IMPORTANT. Make sure there is no .git directory inside simple-react-app.

Now from the root directory you can run:

git add .
git commit -am 'first commit'
git remote add origin YOUR_GIT_REPO_ADDRESS
git push -u origin YOUR_BRANCH_NAME

Sharing Code and Adding Dependencies

So let's quickly take a look at some of the benefits we get from our monorepo.

Let's say that there's a utility library that we want to use in both our React app and on our Express server. For simplicity let's choose lodash which many people are familiar with.

Rather than adding it to each project individually, we can use lerna to install it to both. This will help us make sure that we keep the same version in sync and require us to only have one copy of it in the root directory.

From the root run the following command:

npx lerna add lodash packages/simple-*

npx lerna add @types/lodash packages/simple-* --dev

This will install lodash in any of the projects in the packages directory that match the simple-* pattern (which includes both of ours). When using this command you can install the package to dev and peer dependencies by adding --dev or --peer at the end. More info on this command here.

If you check the package.json file in both your packages you'll see that lodash has been added with the same version to both files, but the actual package itself has a single copy in the node_modules folder of your root directory.

So we'll update our server.ts file in our Express project to do a couple of new things. We'll import the shared lodash library and use one of its functions (_.snakeCase()) and we'll define a type interface that defines the shape of the data we are sending and export it so that we can also use that interface in our React app to typesafe server queries.

Update your server.ts file to look like the following:

packages/simple-express-server.ts

import express from "express";
import _ from "lodash";
const app = express();
const port = 3001;

export interface QueryPayload {
  payload: string;
}

app.use((_req, res, next) => {
  // Allow any website to connect
  res.setHeader("Access-Control-Allow-Origin", "*");

  // Continue to next middleware
  next();
});

app.get("/", (_req, res) => {
  const responseData: QueryPayload = {
    payload: _.snakeCase("Server data returned successfully"),
  };

  res.json(responseData);
});

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`);
});

(Note I have changed the key on the object from data to payload for clarity)

Next we will update our App.tsx component in simple-react-app. We'll import lodash just for no other reason to show that we can import the same package in both client and server. We'll use it to apply _.toUpper() to the "Learn React" text.

We will also import our QueryPayload interface from our simple-express-server project. This is all possible through the magic of workspaces and Typescript.

packages/simple-react-app/src/App.tsx

import React from "react";
import logo from "./logo.svg";
import "./App.css";
import _ from "lodash";
import { QueryPayload } from "simple-express-server/server";

function App() {
  return (
    <div className="App">
      <header className="App-header">
        <img src={logo} className="App-logo" alt="logo" />
        <p>
          Edit <code>src/App.tsx</code> and save to reload.
        </p>
        <a
          className="App-link"
          href="https://reactjs.org"
          target="_blank"
          rel="noopener noreferrer"
        >
          {_.toUpper("Learn React")}
        </a>
        <button
          onClick={() => {
            fetch("http://localhost:3001/", {})
              .then((response) => response.json())
              .then((data: QueryPayload) => console.log(data.payload));
          }}
        >
          GET SOME DATA
        </button>
      </header>
    </div>
  );
}

export default App;

I find this is one of the trickiest parts to get right (the importing between packages). The key to this is the installation of Typescript in the root of the project, and "baseUrl": "./packages" value in the the tsconfig.json in the root directory.

If you continue to have difficulty this is one of the best explanations I have ever come across for sharing Typescript data between projects in a monorepo.

Once everything is setup, press the button on your React application and you'll be greeted with:

Notice the snake_case response that matches the correct shape we defined. Fantastic!

Now there is one issue with our setup -- currently we are importing the QueryPayload directly from our server. That is fairly harmless, but what if we

Create a Shared Package

Using the lerna create command we can quickly and easily create new projects within our monorepo. Run the following commands from the root directory:

npx lerna create simple-shared-data

npx lerna add typescript --dev

yarn install

This will create a directory called simple-shared-data in your packages. We've already added the same version of Typescript as a dev dependency.

You can remove the lib directory that includes the default JS entrypoint as we will not be using it.

Create an index.ts file inside of packages/simple-shared-data where we will place any types or data that either our front-end, back-end or both can have access to.

packages/simple-shared-data/index.ts

export interface QueryPayload {
  payload: string;
}

And then import from this file in both our server and React app:

packages/simple-express-server/server.ts

import { QueryPayload } from 'simple-shared-data';
...

packages/simple-react-app/src/App.tsx

import { QueryPayload } from 'simple-shared-data';
...

The benefit of creating this shared project is that your front-end for example won't have a strict dependency on the existence of your server. You could deploy as:

Front-End

simple-react-ap
simple-shared-data

Back-End

simple-express-server
simple-shared-data

Now that we have all these different projects setup, lets take a look at git submodules.

Add a Git Submodule

Recently I wrote a blog post on a very simple component for a React app that adds a dark mode, a <DarkMode /> component. The component is not part of a separate library we can install with an NPM command, it exists as part of a React application that has its own repository.

Let's add it to our project, while still keeping it as its own separated repo that can be updated and managed independent of our monorepo.

From the packages/simple-react-app/src directory we'll run this command:

git submodule add git@github.com:alexeagleson/react-dark-mode.git

That will create the react-dark-mode directory (the name of the git repository, you can add another argument after the above command to name the directory yourself).

To import from the submodule it's as simple as... importing from the directory. If we're going to add the <DarkMode /> component it's as simple as adding:

packages/simple-react-app/src/App.tsx

...
import DarkMode from "./react-dark-mode/src/DarkMode";

function App() {
  return (
    <div className="App">
      ...
      <DarkMode />
    </div>
  );
}

export default App;

I've omitted some of the repetitive stuff above. Unfortunately the default background-color styles in App.css are going to override the body styles, so we need to update App.css for it to work:

packages/simple-react-app/src/App.css

...

.App-header {
  /* background-color: #282c34; */
  min-height: 100vh;
  display: flex;
  flex-direction: column;
  align-items: center;
  justify-content: center;
  font-size: calc(10px + 2vmin);
  /* color: white; */
}

.App-link {
  /* color: #61dafb; */
}

...

Comment out those color values and you're good to go!

Now you might be thinking -- couldn't I just have cloned that repo into that folder and done this? What's the difference with submodules?

Well now that we have this in place, let's look for the answer to exactly that. Run the following command:

git status

In the output you'll see new file: ../../../.gitmodules. That's something new if you've never used submodules before. It's a hidden file that has been added to the project root. Let's take a look inside it:

[submodule "packages/simple-react-app/src/react-dark-mode"]
    path = packages/simple-react-app/src/react-dark-mode
    url = git@github.com:alexeagleson/react-dark-mode.git

It stores a mapping to the directories in our project that map to other repositories.

Now if you commit your changes in the root of the monorepo and push, you'll see on Github that rather than being a regular directory inside this project -- it's actually a link to the real repository:

So you can continue to update and make changes to this monorepo without impacting that other repository. Great!

But can you update the dark mode repository from inside this one? Sure you can! (As long as you have write permission).

Let's make a trivial change to the dark mode repository from inside this one and see what happens. Navigate to:

packages/simple-react-app/src/react-dark-mode/src/DarkMode.css

...
[data-theme="dark"] {
  --font-color: #eee;
  --background-color: #333;
  --link-color: peachpuff;
}

I'm going to update the colour of the link when the app is in dark mode, from lightblue to peachpuff.

Now obviously you won't be able to update my repository, but if you're following you can continue reading to see where this is going (or you can use your own repository of course).

From this directory I make a commit and push. When I check the repository there are no new commits to the monorepo-example repository, but there IS a new commit to react-dark-mode. Even though we are still inside our monorepo project!

When working with submodules it's important to keep them up to date. Remember that other contributors could be making new commits to the submodules. The regular git pull and git fetch to your main root monorepo aren't going to automatically pull new changes to submodules. To do that you need to run:

git submodule update

To get the latest updates.

You also have new command you'll need to run when cloning a project or pulling when new submodules have been added. When you use git pull it will pull the information about relevant submodules, but it won't actually pull the code from them into your repository. You need to run:

git submodule init

To pull the code for submodules.

Lastly, in case you prefer not to run separate commands, there is a way to pull submodule updates with your regular commands you're already using like clone and pull. Simply add the --recurse-submodules flag like so:

git pull --recurse-submodules

or

git clone --recurse-submodules

Namespacing

Although I didn't use it in the tutorial, it is good practice to use namespacing for your packages. This is commonly done by prefixing with the @ character. Below I will quickly show how to update this tutorial to add a @my-namespace namespace:

Prefix the name value in each of your three package.json files with @my-namespace. For example simple-express-server/package.json will now be:

{
  "name": "@my-namespace/simple-express-server",
  ...
}

Do that for each of the three packages.

Next you need to update your imports:

packages/simple-express-server/server.ts

import { QueryPayload } from '@my-namespace/simple-shared-data';
...

packages/simple-react-app/src/App.tsx

import { QueryPayload } from '@my-namespace/simple-shared-data';
...

Finally run yarn install to update those packages inside your root node_modules directory and you're good to go!

Wrapping Up

I hope you learned something useful about monorepos and submodules. There are tons of different ways to setup a new project, and there's no one-size-fits-all answer for every team.

I'd encourage you to play around with small monorepos (even clone this example) and get get comfortable with the different commands.

Please check some of my other learning tutorials. Feel free to leave a comment or question and share with others if you find any of them helpful:

For more tutorials like this, follow me @eagleson_alex on Twitter

Learnings from React Conf 2021

Alex Eagleson — Thu, 09 Dec 2021 14:24:54 +0000

I recently had the opportunity to join the digital React Conf 2021 and listen to some talks from a really great list of speakers.

This post aims to summarize (in many ways for myself) the big take-aways that I learned from the conference. For example there were some great talks on design/UI that I found interesting, but since I am more involved with the coding side, those will be the areas that I focus on primarily.

So without further preamble, here's an unordered list of some of the new topics I found most compelling:

Conference Video Link
React 18 Release Candidate
Updating to React 18
Suspense and Server Components
React Docs
Unusual React Environments
Cross Platform with React Native
Developer Tooling
The Future of Memoization
Shopify's Hydrogen
All the Rest
Bonus Tips
Wrapping Up

Conference Video Link

The entire 5 hour conference is currently available on Youtube for anyone to watch for free.

(Note they have stated they will be releasing individual videos of each talk, so it is possible this video link will go down in the near future. I will aim to update links to the individual talks if that occurs)

React 18 Release Candidate

The official release candidate for React 18 was released to coincide with the start of the conference. This version is considered to be the "expected version for the official release in early 2022" barring any major issues found in the next couple of months.

So although you may not want to upgrade your critical production applications just yet, as of today React v18 is as close as its ever been to finalized. You're definitely encouraged to use it from the get-go with any new projects you create.

Of course the big topic of discussion in the conference were focused around the new features of React 18 itself.

Updating to React 18

Run the following command in your existing React project:

npm install react@beta react-dom@beta

Note that as of today you can also install the RC version rather than beta as 18 is now a release candidate. Check the versions on NPM to see what is available.

Then all you need to do is change the app mount point (presumably in your index file) from:

const container = document.getElementById('root');
ReactDOM.render(<App />, container);

To:

const container = document.getElementById('root');
const root = ReactDOM.createRoot(container);
root.render(<App />);

And you're ready to begin working with React 18!

Suspense and Server Components

Suspense is already available in React 17, you may have worked with code before that looks like this:

const MyComponent = React.lazy(() => import('./MyComponent'));

const App = () => {
  return (
    <Suspense fallback={<Loading />}>
      <MyComponent />
    </Suspense>
  );
}

However this use of code splitting with Suspense is happening on the client side, where development is driven now is implementing Suspense on the server side.

Currently SSR (server side rendering) is an "all or nothing approach". If we have a page with a header, nav, content and comments that we are generating with SSR -- then the time it takes to serve that page will be equal to the slowest of its individual components. If every other component takes 0.1s and comments takes 1s then you still need to wait for comments before the app can be sent to the client.

This is no longer true with React server components. It's now possible that despite being rendered on the server, you can wrap your <Comments /> in a <Suspense /> with a fallback loading spinner (or whatever you want) and the app will be sent with the fallback in place until it is ready.

Once it is ready, the bundle containing the rendered comments will be sent to the client to replace the fallback component.

As a bonus, the React is smart enough to watch for user interaction. If one suspended component has been clicked while it's still loading, React will immediately stop what it is doing elsewhere and prioritize the loading of that component!

The key takeaway for server components is that they are always rendered on the server, and never sent to be rendered on the client. There are two critical take-aways from this:

Server components will have no impact on your bundle size, since they always render on the server
Server components can have direct access to your database

If you would like to learn more about server components, here's a great talk from Dan covering all the details:

And if you would like to see Suspense in action, Dan Abramov of the React team has created a codesandbox example.

Hit the refresh button to see the effect. You'll want to open the full link in another tab to see the example of the code and how it's composed.

React Docs

The React documentation has finally been modernized with hooks-first content.

They've already been released in beta and are planning to be released officially to replace the current documentation when React 18 ships.

Discussion of the new React docs starts around 2:15

Unusual React Environments

I really enjoyed Sarah Rainsberger's talk which focused on "unusual" development environments for React.

My wife has been working with React this year for the first time and really connected with this talk, particularly the idea that someone would be able to learn React and its concept without necessarily having to go through a prerequisite stage of learning command line tools, NPM, etc.

Thanks to online coding environments like codesandbox, glitch and repl.it for example, you can begin writing real programs in React in a few seconds without installing any dependencies, and that's really powerful.

One of my favourite trends in conferences I've seen is the push to include speakers who are relatively new to the technology, talking about their experiences and aimed at new developers in similar positions. It helps a lot to break down the walls that people often hit when they are overwhelmed with the amount of new tools and concepts they need to learn when getting started.

The talk begins around 2:34

Cross Platform with React Native

The React team seems to be really pushing for parity of developer experience between all kinds of platforms.

There was a lot of talk about what React Native is being used for these days and how optimizations being applied for one platform (Android for example) end up being applied to other platforms (iOS) in ways that they were not expecting.

They discuss how React Native is being used to develop native experiences on not only mobile platforms, but OS's like Windows (parts of the Microsoft Office suite) and Xbox (the dashbaord for the new Xbox Series X) as well.

Here's an image that really demonstrates what they are trying to achieve:

If you want to watch it yourself, this talk starts around 3:34.

Developer Tooling

There was a lot of focus on new and coming updates to developer tooling for React with a talk from Brian Vaughn.

He summarized the big focuses the tooling will be seeing in the coming months:

Integrating the profiler and timeline to work together
React Native support
CPU and memory profiling

This talk starts around 1:41

The Future of Memoization

One of the most interesting talks (that at times, went completely over my head) was from Xuan Huang about the future of memoization in React.

In the context of React, memoization is used primarily to indicate to React which components may or may not need to re-render depending on some kind of state. Typically for a component, you can explicitly state that if the props do not change you do not need to re-render the component.

More info on React.memo.

With the useMemo hook we can create a memoized value that will not be recalculated if values in the dependency array do not change. Setting up this kind of scenario is notoriously cumbersome sometimes, requiring the use of a lot of extra code and potentially the useEffect hook to achieve your goal. It also has the unfortunate side effect of making the flow of your code less linear (less likely to follow logically from top to bottom).

Xuan's talk demonstrated this concept by showing an example in real time, then did a 180 and started writing a "theoretical example" of how it "could" look. To be hones the new example looked a lot more complicated than the original...

But he brought it all together by explaining that all the code he was writing -- if everything goes well (this idea is still in development) -- could be handled for you automatically. A tool that detects areas where memoization would help optimize your components and be applied automatically. Crazy stuff.

Check it out yourself: the talk begins at 1:54.

Shopify's Hydrogen

Shopify is part of the React Server Component working group and has bet heavily on the power of server components for their new React framework called Hydrogen.

Hydrogen is a brand new React-based framework designed to unify the combination of server components, smart caching, and Shopify's API.

Shiopify maintains their own documentation on Hydrogen's support for React server components which they claim to maintain their own stable abstraction layer over server components so you can use them with confidence, despite their current state.

All the Rest

There were still more talks! Some I missed simply due to time constraints, or lunch or kids or whatnot; others were outside the scope of the work I normally do (though I'm always interested in learning anyway even if it's not directly applicable.)

Either way I'd encourage you to check out some of the other talks:

Bonus Tips

Here's a couple simple and practical things I picked up that are easy to pick up and use today.

Like most of the talks, these are new features of React 18. Fortunately the upgrade path to React 18 is aimed to be very simple and completely backward compatible.

See the update section at the start of this post for a simple example of how to make the update in a couple of lines.

Batching

Previous state updates would always be batched into a single render, even if they were called multiple times in a row. There was a limitation for this however, if the function calling these state updates was asynchronous, the updates would not be batched.

This is no longer true as of React 18. The below code example in v17 would trigger two separate state updates, in v18 they will be batched together:

fetchFromApi().then(()=> {
    setLoading(false);
    setError(false);
})

Deferred State

One of my favourite new hooks I learned about that I thought was great was the useDeferredValue hook.

The first time they showed it in Shruti Kapoor's talk I thought it was fantastic. Not surprisingly it kept coming up at least another 2-3 times in other talks as it seems to be extremely useful in a lot of scenarios.

Basically what it does is allow you to specify a piece of state that you know will take longer to process than the other state of the component, and allow the component to render itself without waiting for the "big one". When the big one is ready, it will render that one in.

To give an example, large lists usually meet this criteria. If you have a filter button that changes the state of a large list of items being rendered.

The below example comes from the React documentation on the feature:

function App() {
  const [text, setText] = useState("hello");
  const deferredText = useDeferredValue(text, { timeoutMs: 2000 }); 

  return (
    <div className="App">
      {/* Keep passing the current text to the input */}
      <input value={text} onChange={handleChange} />
      ...
      {/* But the list is allowed to "lag behind" when necessary */}
      <MySlowList text={deferredText} />
    </div>
  );
 }

Wrapping Up

I hope you learned something from this post, and I hope you get the chance to check out some of these talks.

Please check some of my other learning tutorials. Feel free to leave a comment or question and share with others if you find any of them helpful:

For more tutorials like this, follow me @eagleson_alex on Twitter

How to Create a Dark Mode Component in React

Alex Eagleson — Wed, 08 Dec 2021 14:21:35 +0000

All code from this tutorial is available in this repository, and a video version of this tutorial is available below.

Live Demo
Prerequisites
Initialize the Project
Adding Styles
Adding the Toggle Button
Creating the DarkMode Component
Adding Tests (Optional)
Adding DarkMode to the App
Setting Preferred Colour Scheme
Wrapping Up

Providing users with a dark mode for your web application has become an expectation, and there are many ways to accomplish it. Typically this is most efficiently done by taking advantage of the power of CSS variables.

In this tutorial we are going to show how to bundle the entire dark mode feature into a single <DarkMode /> component that you can take with you and place inside any application.

This component will not only persist your choice of settings through a page close or refresh, it will also respect the user's prefers-color-scheme setting in their browser. Pretty cool!

So let's dive into it.

Live Demo

Before we start we'll begin by taking a look a demo of the final product, so you know what to expect from this tutorial. Once you have completed it, you will have your own <DarkMode /> component that you can drop into any application to achieve this functionality.

Prerequisites

I'll presume that you have a basic familiarity with React.

You do not need to be an expert. In fact we don't have a single stateful variable, or any hooks or lifecycle methods. The goal here (as it should always be) is to minimize complexity. We don't need them for this feature.

We will be using Create React App for this tutorial as it is an extremely easy way to quickly and easy establish a React application template that you can build on.

If you choose not to use CRA you should still be able to follow along with this tutorial. We will be writing pure CSS, but in order to copy the examples exactly you would need to have webpack setup with a CSS loader to support the import syntax for CSS files.

If you are not using webpack you can simply use a <link> element for your CSS files in your index.html rather than importing them.

We will also be using Typescript as is the default for every web project I built these days. If you are not super familiar with Typescript you should still be able to follow along, thee amount of explicit typing in these examples is minimal.

Lastly, I have included a section on adding tests for your component using React Testing Library. This section is optional.

Initialize the Project

If you are using CRA then run the following command (if you have your own existing project then disregard)



npx create-react-app dark-mode-example --template typescript

Adding Styles

When the application loads it will determine the dark/light setting in the following order of priority:

User's previous toggle setting
User's browser preferences
Light mode

We'll begin by creating the CSS that handles dark mode.

src/DarkMode.css




/* 1 */
:root {
  --font-color: #333;
  --background-color: #eee;
  --link-color: cornflowerblue;
}

/* 2 */
[data-theme="dark"] {
  --font-color: #eee;
  --background-color: #333;
  --link-color: lightblue;
}

/* 3 */
body {
  background-color: var(--background-color);
  color: var(--font-color);
}

a {
  color: var(--link-color);
}

The :root selector matches the root element representing the DOM tree. Anything you place here will be available anywhere in the application. This is where will will create the CSS variables that hold the colours for our light theme.
Here we set the colours for our dark theme. Using the attribute selector we target any element with a data-theme="dark" attribute on it. This is a custom attribute that we will be placing ourselves on the <html> element.
We set the background colour and text color of our application. This will always be the value of the --background-color and --font-color variables. The value of those variables will change depending on when the data-theme="dark" attribute is set due to the cascade. The dark values are set after the root values so if the selector applies the initial (light) value of those variables will be overwritten with the dark values.

Notice I have also added a custom link colour here which changes based on the value of the theme. You can add as many custom colours as you want here and have them all be controlled by your light/dark toggle. Try adding some more yourself!

Adding the Toggle Button

Next we will create a custom checkbox input to look like a toggle switch based on this example.

I won't comment on how this CSS works as it's not in the scope of this tutorial and not relevant to dark/light mode. The styles below are simply to override the look of the default HTML checkbox.

Add them below the above code in src/DarkMode.css

src/DarkMode.css



/* Custom Dark Mode Toggle Element */
.toggle-theme-wrapper {
  display: flex;
  flex-direction: row;
  align-items: center;
  gap: 4px;
}

.toggle-theme-wrapper span {
  font-size: 28px;
}

.toggle-theme {
  position: relative;
  display: inline-block;
  height: 34px;
  width: 60px;
}

.toggle-theme input {
  display: none;
}

.slider {
  background-color: #ccc;
  position: absolute;
  cursor: pointer;
  bottom: 0;
  left: 0;
  right: 0;
  top: 0;
  transition: 0.2s;
}

.slider:before {
  background-color: #fff;
  bottom: 4px;
  content: "";
  height: 26px;
  left: 4px;
  position: absolute;
  transition: 0.4s;
  width: 26px;
}

input:checked + .slider:before {
  transform: translateX(26px);
}

input:checked + .slider {
  background-color: cornflowerblue;
}

.slider.round {
  border-radius: 34px;
}

.slider.round:before {
  border-radius: 50%;
}

Creating the DarkMode Component

Now we will create our DarkMode component.

To start we are just going to focus on the structure of the component itself, no events or functions:

src/DarkMode.tsx



import "./DarkMode.css";

const DarkMode = () => {
  return (
    <div className="toggle-theme-wrapper">
      <span>☀️</span>
      <label className="toggle-theme" htmlFor="checkbox">
        <input
          type="checkbox"
          id="checkbox"
        />
        <div className="slider round"></div>
      </label>
      <span>🌒</span>
    </div>
  );
};

export default DarkMode;

The <input> element will be handling the state of our colour theme. When it is checked then dark mode is active, when it is not checked then light mode is active.

If you render this component you should have a nice looking custom toggle button without any functionality.

To make our toggle switch work, we have to attack some Javascript functions to the onChange event of the input that fires when the checkbox is toggled.

We also need to decide which mode we are going to show by default when the page or application is first loaded. There is a lot to unpack here; there will be explanations for what is happening with the numbered comments below the example.

src/DarkMode.tsx



import "./DarkMode.css";
import { ChangeEventHandler } from "react";

// 1
const setDark = () => {

  // 2
  localStorage.setItem("theme", "dark");

  // 3
  document.documentElement.setAttribute("data-theme", "dark");
};

const setLight = () => {
  localStorage.setItem("theme", "light");
  document.documentElement.setAttribute("data-theme", "light");
};

// 4
const storedTheme = localStorage.getItem("theme");

const prefersDark =
  window.matchMedia &&
  window.matchMedia("(prefers-color-scheme: dark)").matches;

const defaultDark =
  storedTheme === "dark" || (storedTheme === null && prefersDark);

if (defaultDark) {
  setDark();
}

// 5
const toggleTheme: ChangeEventHandler<HTMLInputElement> = (e) => {
  if (e.target.checked) {
    setDark();
  } else {
    setLight();
  }
};

const DarkMode = () => {
  return (
    <div className="toggle-theme-wrapper">
      <span>☀️</span>
      <label className="toggle-theme" htmlFor="checkbox">
        <input
          type="checkbox"
          id="checkbox"

          // 6
          onChange={toggleTheme}
          defaultChecked={defaultDark}
        />
        <div className="slider round"></div>
      </label>
      <span>🌒</span>
    </div>
  );
};

export default DarkMode;

We create functions called setDark and setLight which do exactly what the names describe. We want these to be as simple as possible. When we invoke them we expect the app to switch to either light or dark mode.
This is how we handle persistance. Using localStorage will allow us to save a value and have it persist even after the user closes the app or reloads the page. Every time light or dark mode is set, we save that value in the theme property of localStorage.
This is where we set the data-theme="dark" (or light) value on the <html> DOM element. This is what actually updates the colours in our app. When that attribute is added then the [data-theme="dark"] selector from our CSS becomes active and the dark colour variables are set (and vice versa).
The section under comment 4 is where the "initial" state is established when the page is loaded before the actual toggle switch has been used. storedTheme gets the value from localStorage if it exists. prefersDark checks a media query for the user's browser settings for prefers-color-scheme. Lastly defaultDark is meant to check both of those and decide whether to default to dark mode based on the 3 rules of priority we established at the beginning of this tutorial. If it evaluates to true, we set the app to dark mode before the component even renders. (Note the reason we can do this is we are targeting the <html> attribute which will already exist.)
This is the event handler function we have written to capture the change event that occurs when a user clicks the checkbox. If the box is checked we enable dark mode, otherwise light mode.
We place the event handler we just created onto the onChange attribute so it fires every time the checkbox changes. We also use the defaultDark boolean value we established to determine if the checkbox is enabled by default.

Adding Tests (Optional)

Before we add this component to our app we can write a few tests to ensure it works as we expect it to.

Create React App comes prepackaged with React Testing Library. It will automatically pick up any .test.tsx files you create.

src/DarkMode.test.tsx



import React from "react";
import { render, screen, fireEvent } from "@testing-library/react";
import DarkMode from "./DarkMode";

// 1
test("renders dark mode component", () => {
  render(<DarkMode />);

  // 2
  const inputElement = screen.getByRole("checkbox") as HTMLInputElement;
  expect(inputElement).toBeInTheDocument();
});

// 3
test("toggles dark mode", () => {
  render(<DarkMode />);
  const inputElement = screen.getByRole("checkbox") as HTMLInputElement;

  // 4
  expect(inputElement.checked).toEqual(false);
  fireEvent.click(inputElement);
  expect(inputElement.checked).toEqual(true);

  // 5
  expect(document.documentElement.getAttribute("data-theme")).toBe("dark");
});

A simple test to ensure the component renders.
The input has a role of checkbox so we would expect to be able to find the element by that role.
A test to ensure that the component actually activates dark mode when the checkbox is toggled
Use testing library's fireEvent function we can simulate a click on our input. We assert before clicking that it should not be checked, then after clicking it should be checked.
This component by design does have side effects and that's what this final assertion is aiming to detect. Although the component is only a small container for an input, it is designed to apply the data-theme attribute to the root <html> element. That element can be accessed directly with the Javascript variable document.documentElement. We check here that the dark value is applied to the attribute after the element is clicked.

If using the default CRA setup (or you have custom configured it to do so) we can run our tests with:



npm run test

And get our results:

Adding DarkMode to the App

Below I have simply imported and added <DarkMode /> to the default App template created when you run Create React App.

src/App.tsx



import React from "react";
import logo from "./logo.svg";
import "./App.css";
import DarkMode from "./DarkMode";

function App() {
  return (
    <div className="App">
      <header className="App-header">
        <DarkMode />
        <img src={logo} className="App-logo" alt="logo" />
        <p>
          Edit <code>src/App.tsx</code> and save to reload.
        </p>
        <a
          className="App-link"
          href="https://reactjs.org"
          target="_blank"
          rel="noopener noreferrer"
        >
          Learn React
        </a>
      </header>
    </div>
  );
}

export default App;

Lastly, we need to update the default CSS included with the CRA setup, otherwise some of the color/background-color values will overwrite our theme variables.

The below example is the default version of App.css with the color values commented out. You can delete them entirely if you like.

src/App.css



.App {
  text-align: center;
}

.App-logo {
  height: 40vmin;
  pointer-events: none;
}

@media (prefers-reduced-motion: no-preference) {
  .App-logo {
    animation: App-logo-spin infinite 20s linear;
  }
}

.App-header {
  /* background-color: #282c34; */
  min-height: 100vh;
  display: flex;
  flex-direction: column;
  align-items: center;
  justify-content: center;
  font-size: calc(10px + 2vmin);
  /* color: white; */
}

.App-link {
  /* color: #61dafb; */
}

@keyframes App-logo-spin {
  from {
    transform: rotate(0deg);
  }
  to {
    transform: rotate(360deg);
  }
}

If you followed along with everything you'll be treated to a very function app with your own custom <DarkMode /> component.

Setting Preferred Colour Scheme

We mentioned that this app supports the user's browser configuration for preferred colour scheme, but we didn't actually explain how you can set that value.

Unfortunately browsers do not make it easy, but it can be achieved with the following steps in either Chrome or Firefox:

Firefox

Type about:config into your navigation bar
If it doesn't already exist create a value called ui.systemUsesDarkTheme and set it as a Number
Se the number as 1 for dark or 0 for light

Chrome

Open developer tools (F12)
Click the ellipsis ... icon at the upper right of the tools
Click More Tools -> Rendering
Under "Emulate CSS Media" select "prefers-color-scheme: dark"

Chrome is a bit trickier so here is a screenshot showing where to find it:

Wrapping Up

I hope you enjoyed this tutorial and learned something from it! You might have picked up on the fact that although this was written from a React perspective, nearly all the code we are working with would work just as well independently of React.

Try it yourself the next time you are working with vanilla HTML/CSS, or even a different framework! You'll find that this code can be reused anywhere with very few modifications needed.

Please check some of my other learning tutorials. Feel free to leave a comment or question and share with others if you find any of them helpful:

For more tutorials like this, follow me @eagleson_alex on Twitter

How to Analyze and Improve your 'Create React App' Production Build

Alex Eagleson — Tue, 30 Nov 2021 14:04:24 +0000

If you are interested in a video version of this tutorial, check out the link below. You can follow along with the code in this blog. (The video is entirely optional, every step and instruction is covered in the blog post.)

In this tutorial we're gonna take a step-by-step look at how to analyze and optimize your Create React App project using a custom webpack configuration that is extremely easy to set up.

We will be using examples that make small adjustments and try different ways of importing modules and splitting your code to see the direct impacts it has on your bundle sizes and performance.

The goal here is to help you get a better understanding of what webpack is actually doing by seeing the exact changes that occur to a production build when you make small changes to your application.

To begin, we'll create a brand new React app called something-big-and-bloated



npx create-react-app something-big-and-bloated --template typescript

Next we'll install the dependencies we need to analyze the project.



npm install @craco/craco webpack-bundle-analyzer --save-dev

craco: A tool to use a custom webpack configuration with Create React App
webpack-bundle-analyzer: A webpack plugin for analyzing bundle size

We need to create a configuration file for craco in the root of our project to include our webpack plugin:

craco.config.js



const BundleAnalyzerPlugin =
  require("webpack-bundle-analyzer").BundleAnalyzerPlugin;

module.exports = function () {
  return {
    webpack: {
      plugins: [new BundleAnalyzerPlugin({ analyzerMode: "server" })],
    },
  };
};

If we run the usual npm run build script for a production build, it will use the standard react-scripts method.

However if we run craco build it will still run the same process, but inject in any webpack configuration you have included in your craco.config.js file. Pretty sweet.

Let's give it a try. We'll create a new entry called analyze in package.json scripts:



{
  ...
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject",
    "analyze": "craco build"
  }
}

Now run:



npm run analyze

Because we set the analyzerMode to "server" in our craco config, we will automatically get our browser open with the results served as a webpage (you can use the "json" option if you want the output without involving the browser)

You can hover over each module within the chunks or your can use the little arrow at the upper left which pops out a drawer. You'll get three different values:

stat: The size of the original source code
parsed: the size of the same code as compiled bundles
gzipped: the size of the compiled bundles after being gzipped

So for most deployments you can look at the gzipped value as how much network usage you will need, and the parsed size as how much code the browser is going to handle once unzipping is complete. Be aware that this value can be just as problematic on users with less powerful CPUs as the gzip size can be for those on slow networks. Both are important to consider.

On create-react-app v4.0.3 for this tutorial the values I get are 205kb / 135kb / 44kb for stat / parsed / gzipped respectively. You can see right out of the box we get a bit of overhead (though for most users that's a small price to pay for convenience).

Now let's try adding some libraries and look at how this value changes. We'll consider the way that we do our imports and see how we might be able to better control our bundle size by only importing what we need.

I'm gonna pick a fairly popular UI library called MUI (Material UI). It's a good example of a large package that can significantly weigh down your app if not bundled properly. It will make a good example for our tutorial.

We will need the following packages:

@mui/material - A React component library designed to give you some useful pre-styled components out of the box
@mui/icons-material: An icon package for MUI based on material icons
@emotion/react: Required for @mui/material.
@emotion/styled: Required for @mui/material.



npm install @mui/material @mui/icons-material @emotion/react @emotion/styled --save

Before we even do anything let's run our analyzer again. Remember we've added these libraries, but we aren't actually using any of them yet. Do you think our bundle size will increase? Let's find out:



npm run analyze

205kb / 135kb / 44kb again. Exact same result. That's great! That means that webpack is not including any libraries we don't actually use. It's doing its job well.

Now we'll import a component from MUI. We'll grab a relatively complex one, something more than a button. Let's use the Speed Dial! Create a new component file in src:

src/CustomSpeedDial.tsx



import React from "react";
import Box from "@mui/material/Box";
import SpeedDial from "@mui/material/SpeedDial";
import SpeedDialIcon from "@mui/material/SpeedDialIcon";
import SpeedDialAction from "@mui/material/SpeedDialAction";
import FileCopyIcon from "@mui/icons-material/FileCopyOutlined";
import SaveIcon from "@mui/icons-material/Save";
import PrintIcon from "@mui/icons-material/Print";
import ShareIcon from "@mui/icons-material/Share";

const actions = [
  { icon: <FileCopyIcon />, name: "Copy" },
  { icon: <SaveIcon />, name: "Save" },
  { icon: <PrintIcon />, name: "Print" },
  { icon: <ShareIcon />, name: "Share" },
];

export default function CustomSpeedDial() {
  return (
    <Box sx={{ height: 320, transform: "translateZ(0px)", flexGrow: 1 }}>
      <SpeedDial
        ariaLabel="SpeedDial basic example"
        sx={{ position: "absolute", bottom: 16, left: 16 }}
        icon={<SpeedDialIcon />}
      >
        {actions.map((action) => (
          <SpeedDialAction
            key={action.name}
            icon={action.icon}
            tooltipTitle={action.name}
          />
        ))}
    </Box>
  );
}

Replace the contents of your App.tsx file with the following:

src/App.tsx



import React from "react";
import CustomSpeedDial from "./CustomSpeedDial";

function App() {
  return <CustomSpeedDial />;
}

export default App;

Run the development server to check it out:



npm run start

Everything looks good. Let's see how much that impacted our build. Run our analyze command again:



npm run analyze

Our bundle size is now up to 660kb / 270kb / 88kb. That's a significant increase for one component! Of course bear in mind that it's fairly complex, and as soon as you use one you need to include all the other dependencies that make MUI function.

I'd bet if you added a second component you wouldn't get such a big jump. In fact we can try now. Add the following to your SpeedDial component:

src/CustomSpeedDial.tsx:



import React from "react";
import Box from "@mui/material/Box";
import SpeedDial from "@mui/material/SpeedDial";
import SpeedDialIcon from "@mui/material/SpeedDialIcon";
import SpeedDialAction from "@mui/material/SpeedDialAction";
import FileCopyIcon from "@mui/icons-material/FileCopyOutlined";
import SaveIcon from "@mui/icons-material/Save";
import PrintIcon from "@mui/icons-material/Print";
import ShareIcon from "@mui/icons-material/Share";

// NEW
import Button from "@mui/material/Button";

const actions = [
  { icon: <FileCopyIcon />, name: "Copy" },
  { icon: <SaveIcon />, name: "Save" },
  { icon: <PrintIcon />, name: "Print" },
  { icon: <ShareIcon />, name: "Share" },
];

export default function CustomSpeedDial() {
  return (
    <Box sx={{ height: 320, transform: "translateZ(0px)", flexGrow: 1 }}>
      {/* NEW */}
      <Button variant="contained">Hello world!</Button>
      <SpeedDial
        ariaLabel="SpeedDial basic example"
        sx={{ position: "absolute", bottom: 16, left: 16 }}
        icon={<SpeedDialIcon />}
      >
        {actions.map((action) => (
          <SpeedDialAction
            key={action.name}
            icon={action.icon}
            tooltipTitle={action.name}
          />
        ))}
      </SpeedDial>
    </Box>
  );
}

We've imported a button above and included it with our speed dial (two new lines marked with "NEW" comments.)

When we run npm run analyze again we get.... almost the same! 677kb / 278kb / 89kb. We can see that the button extremely little to the bundle size since most of the building blocks for it were already included with the Speed Dial.

But now let's compare when using the traditional commonJS import.

Add the following line to the very top of your CustomSpeedDial component (if ESLint complains about import order, place the line after all your imports statements)

src/CustomSpeedDial.tsx



const material = require("@mui/material");

Analyze again:



npm run analyze

Holy moly! 1.97*MB* / 697kb / 194kb.

What happened? Looks like we're bundling the entire MUI library. Popper? Tooltip.js? We're not using any of those, but they're taking up a ton of space in our chunk.

It turns out that when we are using ES6 modules webpack is very good at figuring out which pieces of code we are actually using based on what we import and export.

This process is called tree shaking and it requires you to use ES6 modules in order to work. You can see that doing so can have a very dramatic effect on our final bundle.

Our current program is exactly the same functionally as our previous one, but thanks to a single commonJS import it's a whopping 3x the size. Yikes!

Here's what we're going to do though. Instead of removing that require in our CustomSpeedDial we're going to leave it there, and introduce something called code spltting as another option available to you.

Code splitting is effective when you have a certain component or page or general part of your application that isn't necessary required for every visitor. It might be a datepicker that only appears when a user makes a booking, or it might be a "help" page that only a small percentage of users will need.

We can use React's lazy and suspense features to break these pieces into separate bundle chunks, and only load them when necessary.

Let's update App.tsx. There's a lot to unpack here, so we'll just show the code first and break it down:

src/App.tsx



import CircularProgress from "@mui/material/CircularProgress";
import Button from "@mui/material/Button";
import React, { Suspense, useState } from "react";

// 1
const CustomSpeedDial = React.lazy(() => import("./CustomSpeedDial"));

function App() {
  // 2
  const [showSpeedDial, setShowSpeedDial] = useState(false);

  // 4
  if (showSpeedDial) {
    return (
      // 5
      <Suspense fallback={<CircularProgress />}>
        <CustomSpeedDial />
      </Suspense>
    );
  }

  return (
    // 3
    <Button variant="contained" onClick={() => setShowSpeedDial(true)}>
      Click to load speed dial
    </Button>
  );
}

export default App;

The numbers are a little out of order, but that's intentional. You'll see it follows the actual flow of the component.

We "lazy" import the CustomSpeedDial module. Remember that's the one that uses require for the entire MUI package and comes in at 1-2MB. By using the lazy import, what happens is that the import only occurs when our primary component here (CustomSpeedDial) actually tries to render it. We'll see that it doesn't by default.
A boolean React state value to keep track of whether which component we want to render. The default false value means we will not be rendering CustomSpeedDial
Our default component is a basic Button imported directly from MUI. When this button is pressed it sets the value of showSpeedDial to true.
Once showSpeedDial is true we take this branch at the next render. The reason we get a re-render is because we updated a stateful React value (showSpeedDial).
The purpose of the Suspense component is to tells React what to render while waiting for the module to import. Depending on the size it could take a second or more. For our example we are using MUI's CircularProgress to imply a loading state while the module is loading. Once it loads it switches to render the children of the Suspense component.

Now it's time to try it out! We'll begin with an analyze:



npm run analyze

This gets really interesting. Webpack has created new separate chunks. You'll notice there are more .js chunks when you toggle the drawer on the left.

In fact this largest chunk on the left 3.5d1a4e88.chunk.js (1.52mb / 475kb / 122kb) isn't even used on the default load of our app. Based on our learnings earlier, we can see that huge chunk is definitely our CustomSpeedDial.tsx component that imports all of MUI with require commonJS import.

On the right we have the much smaller bundle 2.c5828938.chunk.js that includes things like Button and ButtonBase. This is the chunk that will load on every page load. We can look at the size (451kb / 214kb / 69kb) and validate that in a moment.

Since our ultimate goal is to make sure our production app is running as efficiently as possible, we want to run our tests on the production version of the app. Use the following command to build a production version of the app:



bpm run build

Next we'll need to serve that build directory that was created. If you have your own local serve you prefer to use, use it! If not just add the serve package:



npm install serve --save-dev

And then use it to serve the build directory:



npx serve build

You can find the page at http://localhost:3000/ (or whatever port serve specifies on the command line).

Press F12 to open your browser's Developer Tools and click the Network tab. This process should be roughly the same in both Chrome, Edge and Firefox.

I am using Firefox so the screenshots should match your experience. If you are using another browser the options will still be there, just potentially in a different location.

Click the Disable Cache checkbox so that we are loading the JS files on every refresh and not a cached version from the browser. We want to be able to see the load times and sizes.

Now hit the refresh button (F5).

As we predicted, our total transfer is 82KB with 69KB of that being the highlighted smaller c5828938 chunk we identified (remember this is a served production build, so we are working with GZIP sizes just like your real app would for real users)

There is no sign of the 122KB gzipped chunk. Let's click the "load speed dial" button on our app.

There's that 122KB chunk with our CustomSpeedDial component inside of it.

How cool is it that it only loaded the code on demand?

Wrapping Up

I hope you can start brainstorming ways to reduce the bundle size of your app and potentially introduce code splitting to improve initial load times.

Also worth noting: these tips are not exclusive to Create React App. All we did was introduce a special tool called craco to allow us to configure webpack. Any application running webpack can potentially benefit from these tips!

It would be irresponsible to me not to mention that Create React App does recommend a similar tool that doesn't require craco (although personally I feel it's not as intuitive to read the data) it still gets the job done well. Read about it here.

Continue Learning

Please check some of my other learning tutorials. Feel free to leave a comment or question and share with others if you find any of them helpful:

For more tutorials like this, follow me @eagleson_alex on Twitter

DEV Community: Alex Eagleson

How to Set Up a Fullstack Rust Project with Axum, React, Vite, and Shared Types

Table of Contents

Motivation

Project Setup

Rust Server

React Client

Sharing Types

Option 1: Run your local copy

Option 2: Run your published copy

Option 3: Run my Published Copy

Option 4: Run something more feature complete and production ready

Add Styles

Conclusion

How to Build a Rust CLI Tool to Generate Typescript Types from Rust

Table of Contents

Motivation

Introduction

Project Setup

Building the CLI

Working With Files

Basic Types

Enums

Structs

Tuples

Standard Library Types

Tests

Building and Publishing

Future Improvements

Similar Open Source Tools

Conclusion

The Complete Beginner's Guide to Web Development

Topics

Welcome learners!

Is there a roadmap or summary of everything I should learn?

What is a web developer?

What is the different between a Front-End, Back-End and Full-Stack developer?

Front End Developer

Back End Developer

Full Stack Developer

Basic Tools and Concepts

What does a typical work day look like for a web developer?

Communication Tools

Meetings

Project Management

What is Agile?

Design Systems

Design Tools

What is Figma?

Code Reviews

What skills are critical for a successful junior developer?

Reading Code

Asking for Help

Getting Started

How do I get started?

Computer Hardware

A Working Environment

A Way to Test Your Work

A Goal to Work Toward

A Resource to Learn From

Web Development Basics

How does a website get created?

How can I build a website or web app?

How can I build websites on my own computer?

How do I set up a development environment on my computer?

HTML

What is HTML?

How do I use HTML?

What else do I need to know about HTML?

What are the best free resources for learning HTML?

Web Servers

What is a Web Server?

What is a web request?

Domain

Port

Path

How do I run a web server?

Option 1: VS Code Extension

Option 2: HTTP Server

Option 3: Chrome Extension