DEV Community: Michael Hungbo

A Guide to Creating API Documentation With Redoc and GitHub Pages

Michael Hungbo — Thu, 02 Mar 2023 13:44:13 +0000

Introduction

Redoc is a free and open-source tool for creating interactive API documentation from OpenAPI definitions. It's one of the most popular API documentation generation tools, with a three-column and responsive layout. Redoc is excellent for creating documentation sites and is simple to set up locally. However, deploying your local project online is often difficult and sometimes impossible, particularly for writers who have little or no technical experience.

Recently, I’ve had two people tell me that they were able to get their Redoc-generated site to work nicely on localhost but have no idea how to deploy it online to a static site hosting service like GitHub Pages, for example. The quickstart and deployment guides in the Redoc docs only explain how to run the generated docs locally and some of the information in the guides is not straightforward and beginner friendly.

This guide is my attempt to explain how I was able to publish an example API documentation site built with Redoc on GitHub Pages.

You can take a look at the deployed documentation we’ll be creating in this guide here, and the complete code on GitHub.

Prerequisites

To get started, ensure you have a GitHub account and the following installed on your computer:

Step 1 - Create a copy of the `openapi-starter template` on GitHub

The first step in this guide will be creating a copy of the openapi-starter template on GitHub. The template is the recommended way to set up a Redoc project. It automatically creates the required folder structure and generates a basic OpenAPI definition file to try out what your final documentation site will look like.

Go to https://github.com/Redocly/openapi-starter.
Click on Use this template.
Select Create a new repository.

4. You’ll be taken to a page to create a new repository from the template.

5. Give the repository a name.

6. Choose whether you want the repository to be public or private.

7. Leave the Include all branches option unchecked.

8. Finally, select Create repository from template.

At this point, you should have a new repository created using the openapi-starter template and ready to roll!

Step 2 - Clone the repository to your computer

In this step, we’ll clone the Redoc project in the newly created repository so you can use it locally on your computer with Redoc CLI.

Click Code in the repository and copy the URL under the HTTPS tab.
Open your terminal and navigate to a directory where you’d like to clone your project.
Run the following command to clone the project:
```
git clone <your-HTTPS-URL>
```
Cd into the cloned project directory and open it in your code editor. Your project folder should look like the image below:

Step 3 - Install project dependencies

Run the command below in the root of your project to install the dependencies required for using the template:

npm install

Step 4 - Prepare API definition file

If you open the openapi folder in the project root, you’ll see an openapi.yaml file that contains API definitions that we can use to create the documentation site. However, in this guide, we’ll use Forem’s OpenAPI definition file. You can download the definition file on this page, or directly using this link. If you have an OpenAPI specification file you would like to use, you're welcome to use that instead.

Next, open the openapi.yaml file and delete its content. Copy and paste the content of the Forem OpenAPI spec file you just downloaded into the openapi.yaml file. The file should look like below:

Now, in the openapi folder in your project root, you’ll see these three folders: code-samples, components, and paths. These folders are created when you run the redocly split command to separate the bundled API spec file into different parts, which allows for a multi-file approach to API documentation.

Since we already changed the contents of the openapi.yaml file, we need to delete these folders and create new ones since their contents are not identical to what we have in the new definition file. So, go ahead and delete those folders.

Step 5 - `bundle`, `lint`, and `split`

In this step, we’ll go through the different commands you'd need to prepare your definition file to create a Redoc-generated doc site running locally. It’s important that you understand what these commands do because you’re going to be using them most of the time when you’re working with a Redoc project.

`bundle`

The bundle command is used to merge all your standalone files into a single definition file and we can also use it to create a bundled version of our API definition file. It's basically the opposite of the spilt command.

In your terminal, cd into the openapi folder and run the following command to create a bundle of the openapi.yaml file:

redocly bundle openapi.yaml --output bundled.yaml

You should get an output similar to the following, and also a bundled.yaml file in the openapi directory:

bundling openapi.yaml...
📦 Created a bundle for openapi.yaml at bundled.yaml 333ms.

At this point, your folder structure should look something like below:

`lint`

The lint command checks if an API definition file is valid and conforms to OpenAPI specifications rules.

After creating a bundled.yaml file, we can now lint the file to check if it conforms to OpenAPI specification rules and contains no errors. In the terminal, ensure you’ve cd into the openapi folder, then run the command below to lint the bundled.yaml file:

redocly lint bundled.yaml

You should get an output in the terminal that says the definition file is valid.

`split`

The split command divides the large API definition file into different parts, allowing you to follow the multi-file approach to API docs.

Splitting the definition file will make it easier to work with different parts of your API definitions if your project becomes too large or complex.

Run the command below in the openapi directory to split the bundle.yaml file into its constituent parts:

redocly split bundled.yaml --outDir .

You should get an output similar to the following and two new folders created in the openapi directory, namely components and paths:

🪓 Document:  bundled.yaml is successfully split
    and all related files are saved to the directory: .

bundled.yaml: split processed in 1068ms

These folders let you work on different parts of your API definitions independently and seamlessly to improve your writing experience and help with managing large or complete documentation.

Step 6 - Preview the definition file locally

The last step in getting to view the documentation site locally is to run the preview-docs command to create a local preview of our API reference docs.

Run the following command in the openapi directory to create a local preview of the bundled.yaml file:

redocly preview-docs bundled.yaml

You should get an output similar to this:

Using Redoc community edition.
Login with redocly login or use an enterprise license key to preview 
with the premium docs.


  🔎  Preview server running at http://127.0.0.1:8080

Bundling...


  👀  Watching bundled.yaml and all related resources for changes  

Created a bundle for bundled.yaml successfully

Navigate to http://localhost:8080 and you will see the documentation site rendered like below:

Congratulations! We just made a local preview of our OpenAPI definition file with Redoc!

NOTE: The local server also allows you to make edits such as typo fixes to bundled.yaml and the changes will be reflected immediately in the preview site.

Step 7 - Preparing to deploy

Now that we have gotten our project to work locally, it’s of no use if we don’t get it live on a website for our users to see. However, deploying a Redoc project online is where most people get stuck.

But don’t worry, in this step, we'll go through how you can prepare your project for hosting on GitHub Pages. This is also going to work if you want to use GitLab Pages or any other alternative.

In the preview site we generated in the last step, note that we used the bundled.yaml file to create the preview. However, to create a documentation site we can deploy online, we’ll need to generate a *.json file from our openapi.yaml definition file. The JSON file is what Redoc will use to create our online static docs site.

In your terminal, cd into the docs folder in the root directory and run the following command:

redocly bundle ../openapi/openapi.yaml -o dist.json

You should see the following output and a dist.json file created in the docs folder:

bundling ../openapi/openapi.yaml...
📦 Created a bundle for ../openapi/openapi.yaml at dist.json 288ms.

Next, in the docs folder, delete the contents of index.html and paste the following code into it:

<!DOCTYPE html>
<html>
  <head>
    <title>ReDoc</title>
    <!-- needed for adaptive design -->
    <meta charset="utf-8"/>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link href="https://fonts.googleapis.com/css?family=Montserrat:300,400,700|Roboto:300,400,700" rel="stylesheet">

    <!--
    ReDoc doesn't change outer page styles
    -->
    <style>
      body {
        margin: 0;
        padding: 0;
      }
    </style>
  </head>
  <body>
    <redoc spec-url='dist.json'></redoc>
    <script src="https://cdn.jsdelivr.net/npm/redoc@next/bundles/redoc.standalone.js"> </script>
  </body>
</html>

In the code above, the <script> tag fetches some JavaScript code from a CDN which will be injected into your documentation site when it’s deployed online. The code is responsible for rendering and generating the styling of your documentation site.

The <redoc> tag injects the bundled OpenAPI definition file into the site, and it’s also used by the JavaScript code to display our documentation site.

Step 8 - Deploying on GitHub Pages

In this final step, we’ll deploy the local preview of our project to a static site hosting service - GitHub Pages.

Navigate to your project root directory. Then run the following command to confirm your local project has the repository which we created in step 1 as the origin:

git remote -v

You should get an output similar to the following:

origin  https://github.com/Mich45/redoc-example.git (fetch)
origin  https://github.com/Mich45/redoc-example.git (push)

Next, run the command below to add all the changes we have made so far in the project to the staging area:

git add .

Then run the following command to commit our changes:

git commit -m ‘feature: publish docs to github pages’

Finally, run the command below to push your local changes to the template repository we created on GitHub:

git push origin

If you go back to the repository which we created in step 1, you should see that our local changes have been pushed to it. Here's what mine looks like:

So far so good.

Next, we'll generate a static docs site with GitHub Pages using this repository.

Click on the Settings tab in your repository as shown below.
In the left navigation bar, click on Pages.
In the drop-down menu under Source, select Deploy from a branch. Under Branch, select main, and under folder, select /docs.
Click Save.
Reload the page and you should see a link to your documentation site hosted on GitHub Pages as shown below:

Here’s a link to the static site generated for me on GitHub Pages: https://mich45.github.io/redoc-example/.

If you followed the steps above and everything went well, your static site should be up and running and identical to the image below:

Conclusion

Viola! That’s it. In this guide, we went through how you can setup and run a Redoc project locally, and also how to deploy a professional-looking Redoc-generated documentation to GitHub Pages.

I hope you found this guide useful for deploying your Redoc-generated sites to GitHub Pages.

Adios, until next time!

Scraping LinkedIn Data With Proxycurl Jobs API

Michael Hungbo — Tue, 09 Aug 2022 01:05:48 +0000

Cover photo by Alexander Shatov on Unsplash

Overview
Prerequisites
Getting started
Fetching listed jobs
Getting a specific job details
Limitations of the LinkedIn Job API

Overview

The LinkedIn Jobs API is one of the rich set of tools provided by the Proxycurl API for working with processed and structured LinkedIn data in your applications. The LinkedIn Jobs API, specifically can be used to access jobs listed by a company (through the Jobs Listing Endpoint) on LinkedIn, or request compact information about a particular job (through the Jobs Profile Endpoint) such as the job's title, description, employment type, etc.

Prerequisites

This tutorial is written in JavaScript (ES6) and Node.js, so I'm assuming you are comfortable writing and understanding code in both.

Create a new directory, cd into it and start a new Node.js project by running:

npm init -y

Next, we'll need to install the following packages to start our application.

express - a Node.js framework to bootstrap our server.
axios - a data fetching library to query the Jobs API endpoints.
dotenv - to load environment variables into our app.

Run the following code to install the packages:

npm install express axios dotenv

or with Yarn

yarn add express axios dotenv

Getting Started

To begin using the Jobs API, you'll need an API Key to make requests to the endpoints. Proxycurl gives free 10 credits for trial and each successful request to the API costs 1 credit. You can get additional credits by topping up your account through your dashboard. In this tutorial, we'll use the free 10 credits for starter. To get started, visit here to get your free API Key.

Next, sign-in to your dashboard and copy the API Key from the API Key and billing tab.

To protect your API Key, create a .env file in your project root directory and add the following code:

API_KEY = 'YOUR_API_KEY'

Finally, add the following code to spin up our server:

import express from 'express';
import axios from 'axios';
import dotenv from 'dotenv';

const app = express();

dotenv.config();

app.listen(8000, () => {
    console.log('App connected successfully!');
});

NOTE: Remember to add "type": "module" in package.json to let Node.js know we're writing ES6 code.

Fetching listed jobs

Let's say we need the list of jobs posted by Twitter on LinkedIn for some reason, manually scraping this type of data may be cumbersome but thanks to the Proxycurl API, we can scrape this data seamlessly.

To achieve this task, we'll use the Jobs Listing Endpoint to query jobs listed by Twitter on LinkedIn and the Company Profile Endpoint to get Twitter's search_id. The search_id is a numerical string returned in the response of the Company Profile Endpoint which we'll use as a parameter in querying the Jobs Listing Endpoint.

To get started, add the following code right before the app.listen() code block:


// {...previous code omitted for brevity}

const TWITTER_URL = 'https://www.linkedin.com/company/twitter/';  // Line 1

const COMPANY_PROFILE_ENDPOINT = 'https://nubela.co/proxycurl/api/linkedin/company';

const JOBS_LISTING_ENDPOINT = 'https://nubela.co/proxycurl/api/v2/linkedin/company/job';

const JOB_PROFILE_ENDPOINT = 'https://nubela.co/proxycurl/api/linkedin/job';

const companyProfileConfig = {  // Line 2
    url: COMPANY_PROFILE_ENDPOINT,
    method: 'get',
    headers: {'Authorization': 'Bearer ' + process.env.API_KEY},
    params: {
    url: TWITTER_URL
  }
};

const getTwitterProfile = async () => {  // Line 3
    return await axios(companyProfileConfig);
}

const profile = await getTwitterProfile();

const twitterID = profile.data.search_id;

console.log('Twitter ID:', twitterID);


const jobListingsConfig = {
    url: JOBS_LISTING_ENDPOINT,
    method: 'get',
    headers: {'Authorization': 'Bearer ' + process.env.API_KEY},
    params: {
    search_id: twitterID // Line 4
    }
}

const getTwitterListings = async () => { // Line 5
     return await axios(jobListingsConfig);
}

const jobListings = await getTwitterListings();

const jobs = jobListings.data.job;

console.log(jobs);

Let's understand what is going on in the code above.

From Line 1 above, we defined the API endpoints for the Proxycurl APIs. You can find the links from the API documentation here.
In Line 2, we defined the axios configuration for the Company profile Endpoint. The url field takes the company profile endpoint url, the headers field takes the Authorization Bearer with our API token, and the params field takes the linkedIn url of the company we'd like to query, which in our case is the TWITTER_URL variable.
In Line 3, we created a function getTwitterProfile which uses axios and returns the profile of our company using the companyProfileConfig.
In Line 4, we're simply using the search_id returned from the getTwitterProfile function as a parameter in the axios configuration for the getTwitterListings function .
Finally, in Line 5, we defined a function getTwitterListings to get the list of jobs posted by Twitter on LinkedIn and attached the result to a jobs variable and then logging the result.

At this point, if you run npm dev, the following response should be logged to the console:

Twitter ID: 96622
[
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Content Designer, Content Moderation (Canada)',
    job_url: 'https://www.linkedin.com/jobs/view/3135150334',
    list_date: null,
    location: 'Toronto, ON'
  },
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Senior Machine Learning Engineer - Ads Predictions - Revenue',
    job_url: 'https://www.linkedin.com/jobs/view/3104474438',
    list_date: null,
    location: 'Canada'
  },
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Sr. Software Engineer, Realtime Storage - Key Value Storage',
    job_url: 'https://www.linkedin.com/jobs/view/3135386201',
    list_date: null,
    location: 'Toronto, ON'
  },
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Content Designer, Content Moderation (Canada)',
    job_url: 'https://www.linkedin.com/jobs/view/3135146767',
    list_date: null,
    location: 'Canada'
  },
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Software Engineer - Content Health',
    job_url: 'https://www.linkedin.com/jobs/view/3169270490',
    list_date: null,
    location: 'Toronto, ON'
  },
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Product Design Manager, Advertiser Experience',
    job_url: 'https://www.linkedin.com/jobs/view/3020369734',
    list_date: null,
    location: 'Toronto, ON'
  },
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Engineering Manager - Content Health (Child Safety)',
    job_url: 'https://www.linkedin.com/jobs/view/3165908037',
    list_date: null,
    location: 'Toronto, ON'
  },
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Senior Software Engineer - Observability',
    job_url: 'https://www.linkedin.com/jobs/view/3158647123',
    list_date: null,
    location: 'Toronto, ON'
  },
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Outbound Sales Representative - Customer Success, Agency',
    job_url: 'https://www.linkedin.com/jobs/view/3109712849',
    list_date: null,
    location: 'Toronto, ON'
  },
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Senior Software Engineer - Data Platform, Metadata Services (Permanently Remote!)',
    job_url: 'https://www.linkedin.com/jobs/view/2939759384',
    list_date: null,
    location: 'Toronto, ON'
  },
  {
    company: 'Twitter',
    company_url: 'https://www.linkedin.com/company/twitter',
    job_title: 'Client Account Manager',
    job_url: 'https://www.linkedin.com/jobs/view/3136650462',
    list_date: null,
    location: 'Toronto, ON'
  }
]

Getting a specific job details

The Job Profile Endpoint returns processed and compact details of a specific job listed by a company on LinkedIn. To see this in action, we'll use one of the jobs listed by Twitter which we returned from the Jobs Listing Endpoint in our previous code.

Add the following code to your project:

// {...previous code omitted for brevity}

const jobProfileConfig = {
    url: JOB_PROFILE_ENDPOINT,
    method: 'get',
    headers: { 'Authorization': 'Bearer ' + process.env.API_KEY },
    params: {
        url: jobs[0].job_url   // Line 1
    }
};

const getJobDetails = async () => {  // Line 2
    return await axios(jobProfileConfig);
};

const jobDetails = await getJobDetails(); 

console.log(jobDetails.data);

Here's what we're doing above.

In Line 1, we added the url of the first job in the jobs variable as a parameter to the axios configuration in jobProfileConfig.
In Line 2, we defined a function getJobDetails to get the details of the first job in the jobs array.

Running the code logs the following response to the console:

{
    "apply_url": null,
    "company": {
        "logo": "https://media-exp1.licdn.com/dms/image/C4D0BAQHiNSL4Or29cg/company-logo_400_400/0/1519856215226?e=1661385600\u0026v=beta\u0026t=rUecQpduLPDavL3JswjLsJAUNgSu1Q2l3JS5sGp8nHk",
        "name": "Twitter",
        "url": "https://www.linkedin.com/company/twitter"
    },
    "employment_type": "Full-time",
    "industry": [
        "Internet"
    ],
    "job_description": "This role may also be remote. Note: By applying to this position you will have...",
    "job_functions": [],
    "linkedin_internal_id": "2400342303",
    "location": "Toronto, ON",
    "seniority_level": null,
    "title": "'Content Designer, Content Moderation (Canada)",
    "total_applicants": null
}

NOTE: I have truncated the job_description field content for brevity.

Limitations of the LinkedIn Job API

In the present state, the LinkedIn Job API has some limitations which limit its use. Here are some of the drawbacks a user might experience while using the API.

The API does not provide the date a job was posted.
It does not provide the qualifications or skills required for a job.

Hopefully, future updates of the API will include significant improvements and updates that will limit the above-stated drawbacks.

Building a URL Shortener with NodeJS

Michael Hungbo — Mon, 09 May 2022 15:57:59 +0000

Introduction

Hey there! In this tutorial, we'll create a URL shortener that works similarly to bit.ly or tinyurl. Our URL shortener will simply take in a URL which is arbitrarily long and shortens it to look so small so it can be shared easily.

Prerequisites

For this tutorial, you should be comfortable working with JavaScript (ES6) and Nodejs. I'm assuming you already have Nodejs installed, if you don't you can install it from here. Also, you'll need to have MongoDB installed on your computer, if you don't you can check my guide here on how to use MongoDB locally with VS Code.

How it works

Before we dive into writing the code, let's first understand how URL shortening works. The logic behind our URL shortener is as follows:

User pastes in an arbitrarily long URL to shorten
We send the long URL to the server which stores the long URL into a database along with a short unique id to identify the URL ( this id is randomly generated and is usually not more than 7-8 characters long)
The shortened URL will be our website address with the unique id that looks something like so: mysite.com/4FRwr5Y
When the user navigates to the shortened URL, we extract the unique id from the URL and find in the database which original long URL is associated with that id
Finally, we redirect the user to the original URL from the database

You can find the complete code for this project on GitHub.

Initialize the project

Now that we understand the logic behind what we'll building, let's go ahead and initialize a new app to get started.

First, we'll create a new directory (or folder, if you like) for our project on the terminal with:



mkdir url-shortener

Of course, you could name your directory anything you want but I chose url-shortener for clarity.

Next, we change directory into our new folder with:



cd url-shortener

Then, run the following command to initialize a new Nodejs project in our current directory:



npm init -y

// or if you are using Yarn

yarn init -y

At this point, we'll need to install a few packages to get started with our project. These are:

express - a Nodejs framework to bootstrap our server.
mongoose - an ODM (Object Document Modeling) to query our MongoDB database.
dotenv - allows us to load environment variables into our app effortlessly.
nodemon - to automatically restart our server when we make changes to our code.
url-exist - we'll use this package to confirm the existence of the URL submitted by the user.
nanoid - we'll use this to randomly generate unique ids for the URL.

Next, run the below command to install the packages:



npm install express dotenv mongoose url-exist nanoid

Or with Yarn:



yarn add express dotenv mongoose url-exist nanoid

I have excluded nodemon from the installation because I have it installed already. If you don't have it installed, you can install it globally with:



npm -g i nodemon



yarn -g add nodemon

And in package.json, we'll add a scripts field to include the command for starting our app like so:



"scripts": {
    "dev": "nodemon index.js"
  }

Now we can run npm dev or yarn dev to start our application.

Note: Since we'll be using import statements in our code, we'll need to add the following to the package.json file to tell Nodejs we're writing ES6 JavaScript:



"type" : "module"

In the end, your package.json should look like below:

Writing the code

Create a new file index.js (here, we'll write the bulk of our server code) in the root directory and two new directories models and public.

In index.js, add the following code:



import express from "express";
import dotenv from "dotenv";
import path from "path";
import mongoose from "mongoose";
import { nanoid } from "nanoid";
import urlExist from "url-exist";
import URL from "./models/urlModel.js";

const __dirname = path.resolve();

dotenv.config();

const app = express();

app.use(express.json());
app.use(express.URLencoded({ extended: true }));
app.use(express.static(__dirname + "/public")); // This line helps us server static files in the public folder. Here we'll write our CSS and browser javascript code

app.listen(8000, () => {
  console.log("App listening on port 8000");
});

Above, we imported the libraries we installed earlier and some core modules from Nodejs, then initialized and created a new server with Express.

You might have noticed we imported a file that doesn't exist yet from the models folder. Let's go ahead and create it.

In the models folder, create a new file named urlModel.js and add the following code:



// models/urlModel.js
import mongoose from "mongoose";

const urlSchema = new mongoose.Schema({
  url: {
    required: true,
    type: String,
    },
  id: {
    required: true,
    type: String
    }
});

const URL = mongoose.model("URL", urlSchema);

export default URL;

Here, we're defining a URL schema with mongoose, this object will let us save the URL object to the MongoDB database and perform other queries.

In modern web application development, it's common practice to not keep sensitive application data in the application code directly to prevent malicious users from exploiting our application. For this reason, we'll store our database URI in a .env file as it is a sensitive information.

In the root folder, create a .env file with the following configuration:



MONGO_DB_URI = "mongodb://localhost:27017/URL-shortener"

Info: At this point for safety purposes, we should create a .gitignore file in the root directory to prevent committing accidentally the .env file to GitHub.

Next, in the index.js file, just before where we are calling app.listen(), add the following code to connect mongoose with our MongoDB database:



mongoose.connect(process.env.MONGO_DB_URI, (err) => {
  if (err) {
    console.log(err);
  }
  console.log("Database connected successfully");
});

Note: If you followed this guide, the above code will automatically create a new database named url-shortener for us. You can confirm this by clicking on the MongoDB extension icon on the left panel in VS Code.

Writing the client-side code

In the public folder, create four new files: index.css, index.html, 404.html and index.js. These are the static files for the front-end of our app and will represent the app's UI.

In the public/index.html file, add the following code:



<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>URL Shortener</title>
    <link rel="stylesheet" href="./index.css" />
  </head>
  <body>
    <main>
      <div class="container">
        <div class="header">URL SH.RTNE.</div>
        <form class="form" id="form">
          <input
            type="text"
            name="URL"
            id="URL"
            value=""
            placeholder="Paste a link to shorten"
          />
          <div class="error"></div>
          <button type="submit" class="btn">Go!</button>
        </form>
        <div class="link-wrapper">
          <h3 class="link-text">Shortened Link</h3>
          <div class="short-link"></div>
        </div>
      </div>
    </main>
    <script src="./index.js"></script>
  </body>
</html>

And in the public/index.css file, add the following:



body {
  background-color: #0d0e12;
  color: white;
  padding: 0;
  margin: 0;
  font-family: "Roboto", sans-serif;
}

.container {
  display: flex;
  flex-direction: column;
  place-items: center;
  position: absolute;
  transform: translate(-50%, -50%);
  left: 50%;
  top: 50%;
  width: 400px;
  height: 450px;
  border-radius: 4px;
  background-color: #ef2d5e;
  padding: 10px;
}

.header {
  font-size: 36px;
  font-weight: bold;
}

.btn {
  height: 35px;
  width: 120px;
  border-radius: 4px;
  background-image: linear-gradient(to bottom, rgb(235 222 63), rgb(243 161 5));
  border: none;
  outline: none;
  color: white;
  box-shadow: 0 3px 6px #d7a827;
}

.btn:hover {
  cursor: pointer;
}

.form {
  margin-top: 30px;
  display: flex;
  flex-direction: column;
  align-items: center;
  justify-content: center;
  place-items: center;
}

input {
  height: 35px;
  width: 320px;
  border-radius: 4px;
  background-color: #fff;
  color: black;
  outline: none;
  border: none;
  margin: 10px 0;
  padding: 10px;
}

input:focus {
  border: 2px solid rgb(243 85 144);
  outline: none;
}
.error {
  color: black;
  margin: 10px 0;
  font-weight: bold;
}

.link-wrapper {
  display: none;
  flex-direction: column;
  margin: 75px 0;
  place-items: center;
  opacity: 0;
  transition: scale 1s ease-in-out;
  scale: 0;
}

.link-text {
  font-weight: bold;
  color: black;
  margin: 5px 0;
}

.short-link {
  display: flex;
  place-items: center;
  place-content: center;
  width: 300px;
  height: 50px;
  background-color: wheat;
  border-radius: 4px;
  padding: 10px;
  margin: 10px;
  color: black;
  font-weight: bold;
  box-shadow: 0 3px 6px #afada9ba;
}

.loader {
  width: 40px;
  height: 40px;
}

And in 404.html, add the following code:



<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Not Found</title>
    <style>
      @font-face {
        font-family: "Roboto";
        src: URL("/Roboto-Medium.ttf") format("truetype");
      }

      body {
        background-color: #0d0e12;
        color: white;
        padding: 0;
        margin: 0;
        font-family: "Roboto", sans-serif;
      }

      .message {
        position: absolute;
        transform: translate(-50%, -50%);
        left: 50%;
        top: 50%;
      }
    </style>
  </head>
  <body>
    <div class="message">
      <h1>Oops! Sorry, we couldn't find that URL. Please try another one.</h1>
    </div>
  </body>
</html>

We'll simply render this file when the user tries to visit a shortened link that is not valid.

Then, in public/index.js, add the following:



const form = document.getElementById("form");
const input = document.querySelector("input");
const linkWrapper = document.querySelector(".link-wrapper");
const errorDiv = document.querySelector(".error");

const shortenedLink = document.querySelector(".short-link");

const handleSubmit = async () => {
  let url = document.querySelector("#url").value;
  const response = await fetch("http://localhost:8000/link", {
    headers: {
      "Content-Type": "application/json",
    },
    method: "POST",
    body: JSON.stringify({ url }),
  }).then((response) => response.json());

  if (response.type == "failure") {
    input.style.border = "2px solid red";
    errorDiv.textContent = `${response.message}, please try another one!`;
  }
  if (response.type == "success") {
    linkWrapper.style.opacity = 1;
    linkWrapper.style.scale = 1;
    linkWrapper.style.display = "flex";
    shortenedLink.textContent = response.message;
  }
};

 // Clear input field and error message
const clearFields = () => {
  let url = document.querySelector("#url");
  url.value = '';
  url.addEventListener('focus', () => {
    errorDiv.textContent = '';
  })
}

form.addEventListener("submit", (e) => {
  e.preventDefault();
  handleSubmit();
  clearFields();
});

Above, we're making a POST request to the server using the fetch api to submit the long URL the user wants to shorten and then updating the DOM with the result from the server accordingly.

Defining the routes

Next, we'll create routes in url-shortener/index.js to serve the front-end files we just created and also handle the POST and GET requests from the user.

In url-shortener/index.js, add the following code right before where we are calling app.listen():



// {... previous code}
app.get("/", (req, res) => {
  res.sendFile(__dirname + "/public/index.html");
});

app.post("/link", validateURL, (req, res) => {
  const { URL } = req.body;

  // Generate a unique id to identify the URL
  let id = nanoid(7);

  let newURL = new URL({ URL, id });
  try {
    newURL.save();
  } catch (err) {
    res.send("An error was encountered! Please try again.");
  }
  // The shortened link: our server address with the unique id
  res.json({ message: `http://localhost:8000/${newURL.id}`, type: "success" });
});

In the first three lines in the code above, we're simply rendering the index.html file when we navigate to http://localhost:8000 in the browser, which is the homepage. This should render the following in the browser:

In the next lines, we defined a route to handle the URL we received from the user and then we generated a unique id to identify the URL and then saved it in the database.

Validating the URL

If you noticed, we added a validateURL middleware to the /link route which we haven't created yet. In this middleware, we're using url-exist to check if the URL submitted by the user is valid before saving the URL at all. If the URL submitted by the user is invalid, we'll return an "Invalid URL" message, else we'll call the next() function to proceed with saving the URL and sending the shortened link. Now, let's create the middleware. Above the previous code, add the following:



// Middleware to validate url
const validateURL = async (req, res, next) => {
  const { url } = req.body;
  const isExist = await urlExist(url);
  if (!isExist) {
    return res.json({ message: "Invalid URL", type: "failure" });
  }
  next();
};

Redirecting the user

The last part of our app is redirecting the user to the original URL when they visit the shortened link we generated. For this, we'll create a route to retrieve the unique id from the link and then find in the database the original URL associated with that id and finally, redirect the user to the original URL. Also, we're checking if the shortened link the user is querying has an original URL associated with it, if doesn't, we respond with the 404 page.



app.get("/:id", async (req, res) => {
  const id = req.params.id;

  const originalLink = await URL.findOne({ id });

  if (!originalLink) {
    return res.sendFile(__dirname + "/public/404.html");
  }
  res.redirect(originalLink.url);
});

Now if you followed this tutorial correctly and paste in any link to shorten, you should get the shortened URL of the original URL as in the following example:

Conclusion

Congratulations if you made it this far! You just built a URL shortening app! Of course, there are other features lacking in our app, but this tutorial is just to show you the basics and logic behind a URL shortening service. You can get creative and add more features if you like, e.g., a simple add-to-clipboard feature to allow our users copy the shortened link to their clipboard.

Thanks for reading. If you liked this tutorial, you may consider following me to be notified for more posts like this or say Hi on Twitter.

Client-side Rendering, Server-side Rendering, and SSG in Plain English

Michael Hungbo — Fri, 31 Dec 2021 18:46:00 +0000

Introduction

As a web developer, this may probably not be the first time you'll be hearing these terms. And if it is, I'm sure you're curious to know what the differences and similarities between these technologies are as they're almost everywhere and are essential for you to know as a web developer (You'll know just why in few minutes!).
You may have come across fantastic articles and resources that have tried to explain these terminologies, but still, you forget or maybe had no idea what some of those contents were talking about. It is my hope that this will be the last article you'll have to read to understand CSR, SSR, and SSG, using words and examples that you can easily relate with. If you believe this is possible, let's ride!

Server-side Rendering (SSR)

SSR is the traditional method of developing websites and applications before the emergence of SPAs (Single Page Applications) and web-apps. As a matter of fact, what was popular in the early days of the web was the idea of web-pages. A company develops a landing page advertising a product with very little content and a little sub-pages, if any, which are usually static texts and images without presenting any form of interaction to the user.
This architecture is the oldest in web development and is based primarily on client-server communication where the client requests for a page and the server sends the already-generated HTML file. In this type of development, the server is responsible for most of the heavy duties such as rendering and parsing the HTML into a useful file and all the browser or client does is to render the static file to the user.

But what is a web server? According to Wikipedia:

A web server is computer software and underlying hardware that accepts requests via HTTP (the network protocol created to distribute web content) or its secure variant HTTPS.

Here's further explanation of the definition if it doesn't make much sense to you:

Underlying components -> A typical web server has two components, the software and hardware components. Examples of the software components include the operating system installed on the sever, databases, file system, etc. Examples of hardware components are, the computer memory (e.g. RAM), processors, power chips etc.
1. Web content -> A server serves mainly static contents, i.e. files that do not change. Examples are, HTML files, images, and CSS.
2. HTTP (Hypertext Transfer Protocol) -> It's simply a set of standards and rules that specify how information and resources are shared between entities on the internet (e.g. a client and a server).
3. HTTPS (Hypertext Transfer Protocol Secure) -> It's a more secure variant of HTTP that adds additional layers of security to communication between entities on the internet.

Even you can create a web server. Your PC or desktop computer with an operating system (software) and uninterrupted power supply can serve a few websites depending on your system memory capacity (hardware) to anybody who can connect to it.
Examples of web servers are Apache, Nginx (pronounced 'engine x'), Google Web Server (GWS), etc. A client on the other hand is any device that requests for any content on the server. An example of a client is a user's browser.

Here is an illustration to show how server-side rendered web-pages are delivered to the end-user:

This traditional approach definitely has its benefits and also some downsides and is why there are emerging technologies such as CSR and SSG to minimize the disadvantages while preserving the advantages.

Some of the pros and cons are discussed below:

Pros

Good for Search Engine Optimization (SEO) -> Since the server sends already-generated page contents, the web-page is readily available for web crawlers to index and this helps in our website ranking across search engines.

Good for static sites -> SSR favors static sites because they require little user interactivity and this helps to reduce the load on the server and minimizes computing power and resource usage.

Cons

Not great for apps with high user-interactivity -> Applications that require high user inputs and are based on repeated client-server communication will be a bad idea to be developed with SSR. This is because a request is being made to the server on every page visit and every click of a button or any event-response based actions performed by the end user. Doing this generates heavy load on the server and may affect the server performance or in the worse case, leads to the server shutting down.

Slow subsequent page load -> The initial load times of SSR web-pages are usually fast but when subsequent page requests are made, the content is fetch all over again even if there's been no changes to the markup.

Frequent server requests -> For a web-page to be delivered in this system, the client has to make a request to the server for this each time a page is needed to be displayed. This way, too much trips are made to the server which increases the load on the server and may stress the computing power and overall performance of the server.

Browser APIs are unavailable -> Since most of the page content are processed and rendered on the server, there's really no way to interact with browser APIs that are only accessible on the client-side.

Examples of technologies we can use for server-side rendering are, NextJS, GatsbyJS, etc.

Consider SSR if:

The website you are building doesn't require end-user interaction. Examples are news websites, personal blogs, etc.
You have a little number of web-pages to deliver, consider using SSR.

Client-side Rendering (CSR)

CSR is the complete opposite of SSR. Here, the browser is solely responsible for the tasks of parsing, rendering and displaying the page contents. CSR became very popular with the emergence of Single-Page Applications (SPAs). This development architecture relies heavily on the user's browser (the client) to process, parse, and render contents before they are displayed to the user. How this works is that the server sends an almost empty HTML file to the browser and also with it the JavaScript code needed for the website functionality because the server cannot run the JS code. The browser then parses and compiles this code and then populates the web-page with the HTML, JS code, and CSS using its own resources before the page content is finally rendered to the user and before any user actions are implemented, such as clicking a button to submit a form.

There's need for CSR because of evolution of the web. The web has gone past serving end-users static contents and web-pages that are unresponsive and provide little or no interesting user experience.

It is very unlikely that you won't use any of the browser APIs when building client-side rendered applications. In fact, there is no way to build CSR apps without using any of the browser APIs. This is because they are needed to provide interactive applications and the server doesn't provide any of these. One of the browser APIs that you can't do without is the Document Object Model API (DOM API). Granted, you may not be aware of some of these APIs being used, but a third party library you are using in the project may be using it behind the scenes.

Often times, CSR applications usually require JavaScript to be enabled on the client, i.e. the browser for them to render any content. Take this React application for example:

This is a simple react application bootstrapped with yarn create react-app.

If you run this command to create a new React project, and you then start the development server, you should see the application running in your browser on localhost:3000 if everything went well:

Now, if you get the above on your browser, it means JavaScript is currently enabled in your browser. And now, we'll disable it to see if our app will work as intended or not. To disable JavaScript on your browser, if you're using Chrome open the Developer Tools with CTRL + Shift + J on Windows. Then open the Settings tab with Shift + ?, scroll down to Debugger and under it check the box that says Disable JavaScript.
Reload the page and then you'll see that your app isn't looking as before. The animated React logo is gone, everything we saw before is no more displayed. The page is empty but with just a text that says You need to enable JavaScript to run this app.

It's totally fine if you have no idea what React is or how it works. The result is pretty much the same if you're using other client-side JavaScript libraries/frameworks like Vue, Svelte, etc.
This is just a basic app in React to demonstrate the importance of JavaScript in any client-side rendered application.

Side-note: The process of rendering page contents to the user in a browser is a different topic on its own and we won't dive into it in this article. However, I'll provide links to resources that will solidify your understanding of rendering in the browser at the end of the article.

Examples of CSR Technologies

These include most JavaScript web frameworks and libraries such as ReactJS, NextJS, VueJS, AngularJS, among others.

Pros and Cons of Client-side Rendering

While a lot of web applications today are built on this architecture, it's essential we pay attention to the pros and cons of this technology. And also help us decide when each of these technologies fits our development needs best.

Pros

Great for apps with high user interactivity -> There's a minimal server-client requests being made in this type of development, therefore, websites and apps that work based on this architecture are good for applications that require a lot of user interaction.

Fast subsequent page load time -> The initial of the CSR applications downloads all the required code and assets that are needed to build the entire website. Therefore, since there are no server requests to ask for resources for page loads because most of the routes have been fetched during the initial page load, subsequent load times for pages are usually faster.

Cons

Slow initial load time -> Due to an almost empty HTML file being sent to the browser in the first page load and a heavy amount of code being received, there's a slow rendering of the page content. If the client's internet speed is slow, all they get to see on the screen is a blank page (or a fancy loading animation to keep our user's patience) which is bad for user experience.

Poor SEO -> While the browser is busy parsing the server code, web crawlers scan our site for links and text and all they see is an almost empty page or a few link and texts, they ignore our site and move to other sites with better SEO.

Consider CSR when:

Building interactive and feature-rich applications. E.g, chat apps, etc.

Static-site Generation (SSG)

Static-site generation as the name implies is the pre-rendering of contents that do not change on the server. In SSG, static page contents are rendered on build-time as opposed to SSR where page contents are rendered during run-time or request time.

What is build time? It's that process when the developer runs the code that optimizes all the static files that are required for making the site and are rendered on the server and then are delivered to a Content Delivery Network (CDN) that will eventually serve these pages to the user.
It's very much similar to SSR and have a number of advantages over SSR which we'll discuss below.

Pros

Initial and subsequent page loads are fast -> Static sites have contents that have been pre-rendered on the server and this makes the initial load time really fast. Subsequent page loads are also fast since the site's contents have been cached by a CDN that delivers these pages upon request without waiting for them to be re-rendered again.

Up-to-date content -> Static sites have up-to-date page contents since upon every change that is made the site is re-built and the user always have updated content on every page request.

Good SEO -> SSG presents good SEO as there is in SSG.

Security -> Static sites usually have better protection from security risks since they are hosted on CDNs which provide additional layer of protection against malicious attacks.

Cons

Frequent page rebuilds -> When changes are made to statically generated sites, there's need to perform rebuilds upon every change that is made in order to serve users with up-to-date content.

Build-time is proportional to site's content -> Build-times may become longer than imagined when you have a few hundreds to thousands of pages to build. This means the build-time grows as your pages and contents grow.

Examples of Static-site generators include, Next, Gatsby, Hugo, Eleventy, among others. You can check Staticgen for a comprehensive list of generators that might suit your need.

Consider SSG when:

You are building a site whose primary function is to deliver content and large sites with hundreds or thousands (Okay, I'm exaggerating. Maybe not) of pages.
You are building your next blog site.

Phew, that's a lot to take in! I wasn't expecting this article to be this long. Anyways, congratulations if you made it this far!

Conclusion

Our ability to choose the best technologies that fit our development needs will surely determine the general performance and health of the applications that we build. Now you know why it's essential that you understand the underlying principles and ideas behind these technologies.
I believe by now you should be able to make informed decisions about which option you should go with when you're building your next bad-ass web project!
And after reading this article, I hope you'll be able to answer anyone who asks you anything about CSR, SSR or SSG, confidently!

Come, let's be friends

If you found this article helpful, you may consider following me for more contents on full-stack JavaScript development and lots more!

You can also say hi on Twitter 😊

See you around!

How to use MongoDB locally with VS Code

Michael Hungbo — Wed, 22 Dec 2021 02:59:17 +0000

Recently, a friend asked me if it's possible to work with their MongoDB databases and Atlas clusters locally without the internet and my answer to that was yes. So, I decided to create a guide to show them how and any other persons who might want to test their collections and perform all kinds of database operations with their MongoDB database using VS Code as the editor.

Installing the MongoDB Community Server Edition allows us to host MongoDB databases locally unlike Atlas which is a cloud hosted database option.

Download and Install MongoDB Community Server Edition 5.0

Visit the download website to download and install MongoDB Community Edition for your OS. You can follow this guide to install MongoDB for Windows OS but the process is pretty much the same for other OS.
For Windows, you can download the .msi installation file here.

The download page should look something like below:

Run the downloaded installer
Choose Custom when prompted for the installation type
At the Service Configuration pane, select *Install MongoDB as a Service *

Install the MongoDB extension for VS Code

The MongoDB extension for VS Code allows us to create playgrounds to interact and perform query operations on our data and collections and giving us the results on the fly at the click of a button (which is supercool!).
To install, open the extension tab on VS Code and search for MongoDB for VS Code.

The extension looks something like below:

As you can see, it's already installed for me so you can just go ahead and install it.

Connect to a MongoDB deployment

In VS Code, open the MongoDB interactive panel by clicking on the leaf icon on the left sidebar menu, then click on create a connection to connect to a database instance.

Note: To connect to a deployment using a connection string, you must have a MongoDB cluster running on your machine or have one in the cloud using Atlas. We'll create one for this guide in a moment.

An example connection string for a local deployment looks like so:

localhost: specifies our host
27017: the default port used by mongodb
myDatabase: is the name of our local database

To create a sample database named myDatabase locally, we can either use MongoDB Compass, a graphical interface for working with databases or mongosh (Mongo Shell). For this guide, we'll use mongosh.

Note: You can also connect to a cloud database instance, for example, a MongoDB Atlas cluster.

Download and install Mongosh

Mongosh (MongoDB Shell) is a JavaScript REPL command line tool for interacting with MongoDB offline and online deployments. What this means is that it allows us to run valid JS code right in the command line to query your database. You can download and install Mongosh here

On successful installation, you should see this panel:

Then, open command prompt on windows and type:


 bash
  > mongosh

You should get the following:



Current Mongosh Log ID: 61bfbc4837f8722b10f9e21e
Connecting to:          mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000
Using MongoDB:          5.0.5
Using Mongosh:          1.1.7

For mongosh info see: https://docs.mongodb.com/mongodb-shell/

------
   The server generated these startup warnings when booting:
   2021-12-18T10:24:36.928-08:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
------

test >

Note:

The test shown at the bottom of the result is the default database created for us by mongosh.

To Create a new database, run the following command on the cmd:


 bash
use myDatabase

This automatically creates a new database called myDatabase for us. Which returns:


 bash
test> use myDatabase
switched to db myDatabase

myDatabase>

Great. Now, let's connect

Remember the connection string we talked about earlier? Good. We're going to use that on VS Code to connect to our newly created database.
The connection string for our database will be: mongodb://localhost:27017/myDatabase, paste this in the text bar at the top of the window, that is, step 3 in the diagram below:

Upon a successful connection, you should see the following changes:

Congratulations! You have just created a local MongoDB database and connected to it from VS Code, successfully!

Play with your database

To perform queries and other database operations on our new database, we can create a Playground in VS Code to do these.

Click on the green create playground button in VS Code to create a playground.

Your newly opened editor tab should look like below:

Delete the content in the default template and paste the following to test our myDatabase database:


 javascript
// Select the database to use
use('myDatabase');

// Insert a few documents in the continents collection
db.continents.insertMany([
    { africa: "Nigeria" },
    { asia: "India" },
    { southAmerica: "Colombia" }
    ]);

// Return all the fields in the continents collection
db.continents.find();

// Write other queries and operations ...

Click on the play button at the top-right side to run the code. A new panel should be opened with our results like so:

Congratulations, you made it!

There you have it. Now you can work with your MongoDB databases locally using VS Code, perform database operations and see the results on the fly! Isn't that awesome? Yeah, it is.

DEV Community: Michael Hungbo

A Guide to Creating API Documentation With Redoc and GitHub Pages

Introduction

Prerequisites

Step 1 - Create a copy of the openapi-starter template on GitHub

Step 2 - Clone the repository to your computer

Step 3 - Install project dependencies

Step 4 - Prepare API definition file

Step 5 - bundle, lint, and split

bundle

lint

split

Step 6 - Preview the definition file locally

Step 7 - Preparing to deploy

Step 8 - Deploying on GitHub Pages

Conclusion

Scraping LinkedIn Data With Proxycurl Jobs API

Table of Contents

Overview

Prerequisites

Getting Started

Fetching listed jobs

Getting a specific job details

Limitations of the LinkedIn Job API

Building a URL Shortener with NodeJS

Introduction

Prerequisites

How it works

Initialize the project

Writing the code

Writing the client-side code

Defining the routes

Validating the URL

Redirecting the user

Conclusion

Client-side Rendering, Server-side Rendering, and SSG in Plain English

Introduction

Server-side Rendering (SSR)

Pros

Cons

Consider SSR if:

Client-side Rendering (CSR)

Examples of CSR Technologies

Pros and Cons of Client-side Rendering

Pros

Cons

Consider CSR when:

Static-site Generation (SSG)

Pros

Cons

Consider SSG when:

Conclusion

Come, let's be friends

How to use MongoDB locally with VS Code

Download and Install MongoDB Community Server Edition 5.0

Install the MongoDB extension for VS Code

Connect to a MongoDB deployment

Download and install Mongosh

Great. Now, let's connect

Play with your database

Congratulations, you made it!

References

Follow me for more content on JavaScript and React!

You can also say hi on Twitter 🙂

Step 1 - Create a copy of the `openapi-starter template` on GitHub

Step 5 - `bundle`, `lint`, and `split`

`bundle`

`lint`

`split`