DEV Community

loading...
Cover image for URL Shortener API with express in 5 minutes

URL Shortener API with express in 5 minutes

Arsenii Gorushkin
Backend developer who wants to be Fullstack and DevOps Engineer
・9 min read

Hello Everyone!

Today, in this tutorial, we will be trying to make a URL shortener API. To achieve our goal, we will use a framework called express.js. In the future, you might be able to use this API for yourself, such as when you want to share a link with someone, but it's too long. If you ever get stuck along the tutorial, feel free to view the code in the repository, and if you have a better solution, feel free to submit a pull request.

How will our API work?

Our API is going to work in the following way:

  • On POST request, we will store the URL, and return the code of the shortened URL.
  • On GET request, we will redirect the user to the original URL.

If you want to use a different tool other then express, feel free to use it. Most of the frameworks are very similar to express (fastify for instance), but for this project I will be using express due to its simplicity.

Setup

To setup the project, you need to have node.js installed on your machine. If you don't have it, you can download it from here. After you have installed it, you need to install the dependencies.

First of all, let's make a file called package.json in the root of the project. It should contain the following: {"type": "module"}. This line will allow us to use ES6 modules in our project, instead of CommonJS.

To install the dependencies, run the following command: npm install express.

At the end, your package.json should look like this:

{
  "type": "module",
  "dependencies": {
    "express": "^4.17.1"
  }
}
Enter fullscreen mode Exit fullscreen mode

Your numbers after the express could be different, but don't worry about it.

Now let's create a file called app.js in the root of the project. This file will contain the code of our API.

After all the steps, our folder should look like this:

Alt Text

🎯 - Checklist

Step 1

Let's start off with setting up our main file, app.js.

First, let's import the express module by using the import keyword.

import express from "express";
Enter fullscreen mode Exit fullscreen mode

Now, let's declare our app and port constants.

const app = express();
const port = 8080;
Enter fullscreen mode Exit fullscreen mode

Now let's add a script that will activate our whole server.

app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});
Enter fullscreen mode Exit fullscreen mode

This turns our server on, and prints a message on the console.

Our app.js should now looks something like this:

import express from "express";

const app = express();
const port = 8080;

app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});
Enter fullscreen mode Exit fullscreen mode

This is it for our file, at least for now, so let's move to the next step.

Step 2

For our shortener to work, we would need some kind of storage, so let's create a folder called src, this is where all of our modules will be stored at. Inside this folder, we will create another folder called data which is where we will add a file called Storage.js.

Now, for the code inside Storage.js, we would first need to import a NodeJS library called fs. This library will allow us to read and write files.

import fs from "fs";
Enter fullscreen mode Exit fullscreen mode

After we've done that, let's make an object that will allow us to read and write data to a file. Before that, let's make a storage.json file in the root directory of our project. Inside the storage.json file, we will add the following:

{
  "link": []
}
Enter fullscreen mode Exit fullscreen mode

For now this array is empty, but soon we will start storing links in it.

Now, back in our Storage.js file, let's make a constant called Storage, and make it an object.

const Storage = {};
Enter fullscreen mode Exit fullscreen mode

"But this object isn't enough to write and read our data" - you might say. Yes, you will be right. But we will make a function that will allow us to write and read data to the file.

First, let's add a property called data to our object, when the script is launched, all the data from storage.json will be loaded into this property. To achieve that, we will use ternary operator to check if file exists, and if it doesn't, we will create a new file. This will result into the following code for our property

const Storage = {
  data: fs.existsSync("storage.json")
    ? JSON.parse(fs.readFileSync("storage.json", "utf8"))
    : JSON.parse(
        fs.readFileSync(
          fs.appendFileSync("storage.json", "{ links: [] }"),
          "utf-8"
        )
      ),
};
Enter fullscreen mode Exit fullscreen mode

In the code above, we are using fs.existsSync to check if the file exists with a ternary operator. If the file exists, we will use JSON.parse to parse the data from inside the file. If the file doesn't exist, we will create it, and then use JSON.parse to parse the data from inside the file.

Now we also need to think about writing data, for that purpose, we will use the fs.writeFileSync function. This function will write data to a file.

import fs from "fs";

export const Storage = {
  data: fs.existsSync("storage.json")
    ? JSON.parse(fs.readFileSync("storage.json", "utf8"))
    : JSON.parse(
        fs.readFileSync(
          "storage.json",
          fs.appendFileSync("storage.json", '{ "links": [] }')
        )
      ),

  write: () =>
    fs.existsSync("storage.json")
      ? fs.writeFileSync("storage.json", JSON.stringify(Storage.data, null, 2))
      : fs.appendFileSync(
          "storage.json",
          JSON.stringify(Storage.data, null, 2)
        ),
};
Enter fullscreen mode Exit fullscreen mode

As you can I have also added export keyword, this is made so we can later use this object in other files.

Now let's import this module into our app.js file. This will result into the following code inside app.js:

import express from "express";

import { Storage } from "./src/data/Storage.js";

const app = express();
const port = 8080;

if (Storage.data.links === undefined) {
  Storage.data.links = [];
  Storage.write();
}

app.listen(port, () => {
  console.log(`The serve is running on port ${port}`);
});
Enter fullscreen mode Exit fullscreen mode

I also added a piece of code inside app.js that will check if the links array is empty, and if it is, it will add an empty array inside it, and write the data to the file.

Step 3

Now after we finished the storage system, we need to setup a system that will allow users to add links to the storage. For that purpose, we will setup a route in our app.js file. This is how it should look like:

app.post("/cut" /*callback*/);
Enter fullscreen mode Exit fullscreen mode

Which will result into the following code inside app.js:

import express from "express";

import { Storage } from "./src/data/Storage.js";

const app = express();
const port = 8080;

if (Storage.data.links === undefined) {
  Storage.data.links = [];
  Storage.write();
}

app.post("/cut" /*callback*/);

app.listen(port, () => {
  console.log(`The serve is running on port ${port}`);
});
Enter fullscreen mode Exit fullscreen mode

Instead of the callback, we will put our method that will handle the request. We will call that method cutURL, and we'll put it inside src/cutURL.js file.

Inside our cutURL.js file, we will first import our Storage, so we can later store the URL inside the storage. To do so, we will use the this piece of code:

import { Storage } from "./data/Storage.js";
Enter fullscreen mode Exit fullscreen mode

After that, we will start writing our method, which will take the request and respose objects as parameters.

export const cutURL = (req, res) => {};
Enter fullscreen mode Exit fullscreen mode

To make sure that the URL is valid, we will use the validator regex. This library will help us to validate the URL. The regex is as shown below:

const urlRegex = /^(https?:\/\/)?[\d\w]+\.[\w]+(\/.*)*/;
Enter fullscreen mode Exit fullscreen mode

Now, to fetch the URL from the request, we will use the req.body.url property. But to do so, we will fist need to use express.json() middleware in our app.js file. While doing this, let's also import our method as well. This will result into the following code inside app.js:

import express from "express";

import { Storage } from "./src/data/Storage.js";

import { cutURL } from "./src/cutURL.js";

const app = express();
const port = 8080;

app.use(express.json());

if (Storage.data.links === undefined) {
  Storage.data.links = [];
  Storage.write();
}

app.post("/cut" cutURL);

app.listen(port, () => {
  console.log(`The serve is running on port ${port}`);
});
Enter fullscreen mode Exit fullscreen mode

Ok, now the cutURL will be activated if the requests is sent to the domain or for examples http://localhost:8080 with a rout of /cut. Now let's get back to our cutURL.js file. We will first check if the URL is valid, and if it fails the validation, we will send a response with the following code:

if (req.body.url === undefined || !urlRegex.test(req.body.url))
  return res.status(400).send("Bad request");
Enter fullscreen mode Exit fullscreen mode

If the URL isn't valid or it's undefined, it will answer with a status code of 400, and say Bad request to the user.

In case we successfully manage to validate the URL, we would need to generate a key/code for it, for this we will use replace method that will make a string of 5 hexadecimal characters.

const code = "xxxxx".replace(/x/g, () =>
  Math.floor(Math.random() * 16).toString(16)
);
Enter fullscreen mode Exit fullscreen mode

After we generatthe code, we will add it to the storage, and then we will send to the user that we have successfully added the link. Here is how we will do it:

Storage.data.links.push({
  url: req.body.url,
  code: code,
});
Storage.write();

res.status(200).send({
  code: code,
});
Enter fullscreen mode Exit fullscreen mode

First piece of code, adds a link to the array, and then we write the data to the file. After doing so, it sends a response with the status code 200, and the code that was generated, so that the user can later access it.

After doing all of these manipulations, the code should look something like this:

import { Storage } from "./data/Storage.js";

export const cutURL = (req, res) => {
  const urlRegex = /^(https?:\/\/)?[\d\w]+\.[\w]+(\/.*)*/;

  if (req.body.url === undefined || !urlRegex.test(req.body.url))
    return res.status(400).send("Bad request");

  const code = "xxxxx".replace(/x/g, () =>
    Math.floor(Math.random() * 16).toString(16)
  );

  Storage.data.links.push({
    url: req.body.url,
    code: code,
  });
  Storage.write();

  res.status(200).send({
    code: code,
  });
};
Enter fullscreen mode Exit fullscreen mode

Step 4

Now that we have done everything, it's time to make a redirect. Redirect will allow user to actually visit the shortened URL by using the key/code. To do so, we will need to add a route to our app.js file. This is how it should look like:

app.get("/[a-f0-9]{5}" /*callback*/);
Enter fullscreen mode Exit fullscreen mode

The callback will be activated every time the code is matched in the route, for instance, if the user visits http://localhost:8080/ab7da, the callback will be activated.

But, for it to work, we first need to actually write a callback. Let's do that in our src/redirect.js file.

Let's first import our Storage modules, so that we can check if the link code is valid or not. After importing, the code should look like this:

import { Storage } from "./data/Storage.js";
Enter fullscreen mode Exit fullscreen mode

After that, let's declare and export our redirect method. This will be our callback.

export const redirect = (req, res) => {};
Enter fullscreen mode Exit fullscreen mode

Inside our redirect method, we will first parse the code from the URL, and then we will check if the code is valid. If it is, we will send the user over to the URL, and if it's not, we will send a 404 status code.

First let's parse the code with the following code:

const code = req.url.replace(/\//, "");
let found = false;
Enter fullscreen mode Exit fullscreen mode

We also declare found variable and set it to false, in future if we find a match, we will set it to true, so that our Not found message will not be displayed.

Let's now write some code to run through all of our links, and check if the code matches any of them.

Storage.data.links.forEach((linkObject) => {
  if (linkObject.code === code) {
    found = true;
    res.redirect(linkObject.url);
  }
});
Enter fullscreen mode Exit fullscreen mode

This code checks for a match, and if the match is achieved it redirects the user to the URL and sets found to true.

Now let's make a fail message, if the code is not found.

if (!found) res.status(404).send("URL not found");
Enter fullscreen mode Exit fullscreen mode

This will send the 404 status code, and the message URL not found to the user. After doing all of this, it will result in the code:

import { Storage } from "./data/Storage.js";

export const redirect = (req, res) => {
  const code = req.url.replace(/\//, "");
  let found = false;

  Storage.data.links.forEach((linkObject) => {
    if (linkObject.code === code) {
      found = true;
      res.redirect(linkObject.url);
    }
  });

  if (!found) res.status(404).send("URL not found");
};
Enter fullscreen mode Exit fullscreen mode

Keep in mind, we still haven't imported our callback, so let's do that. Let's go to the app.js file and add the following code:

import { cutURL } from "./src/cutURL.js";
Enter fullscreen mode Exit fullscreen mode

and also

app.get("/[a-f0-9]{5}", cutURL);
Enter fullscreen mode Exit fullscreen mode

Here we replaced the callback with our cutURL method.

Congraulations!

You have a working URL shortener! Now you can just launch the server, and shorten a URL by sending a POST request to http:localhost:8080/cut, with {"url": "<Your URL>"} json body content. It will return a code, which you can then set as route and use it to visit the URL.

If you like this project, you can also follow me on Twitter and Github, or follow me here on Dev.to @agorushkin to receive updates. You can also do as little as leaving some feedback on how I can improve my projects in future, or just say hi.

Thanks for reading this, and cya in the future!

Discussion (11)

Collapse
michalczaplinski profile image
Michal Czaplinski • Edited

Hi Arsenii!

It's a really well written and easy to follow article so kudos! 🙂

However, I hope you won't mind if I point out some ways in which I would improve your implementation.

  1. Your Storage.data.links could be an object mapping from the shortened link to an object containing the original link instead of an array. This way you wouldn't have to call Storage.data.links.forEach((linkObject) to iterate over the whole array to find the original link - you could just do a lookup like Storage.data.links[oldLink]. This is much more efficient if you have many links stored.

  2. You could make the Storage into a class! This way you don't have to check if the storage.json file exists both in the write method and the data. You could do it just one time in the constructor

  3. Last but not least, think about what would happen if two users would try to shorten a link at exactly the same time! Or even if it was one user during two requests very very quickly. If you are using a file to store the data and writing to disk on every request to shorten the URL, there is nothing stopping you from getting your data corrupted. A slightly better design would be to use a queue data structure which periodically persists to disk. However, to make the url shortener really solid I would probably use a database like redis.io to store the links. This way you wouldn't have to worry as much about concurrency, which is a hard problem!

I hope it was useful to you and keep on blogging! 🙂

Collapse
agorushkin profile image
Arsenii Gorushkin Author

Thank you! Those were some really good pieces of advice! When making it, I didn't want to use a lot of different tools due to me either not knowing them well enough, or them not being as easy to understand. I agree, when I started writing I didn't look back for long enough to see some simple corrections that could be made, hence I didn't do 1 and 2. The argument 3 is still more than valid though, I did it the way I did, because I intended for it to be something like a small project you might want to work on when you have nothing to do, so I didn't really look deep into the problems that could be caused if you do one thing. I recently started to look into redis, but due to my knowledge of it not being big enough, I decided to settle on the solution that I have.

Thank you for your feedback!
P.S. Sorry if there are grammatical errors in my answer, English just isn't my first language, so there are still some weak spots in there that I'm filling up right now.

Collapse
michalczaplinski profile image
Michal Czaplinski

I'm a non-native speaker myself so I'm not allowed to judge people based on their english :D Besides, your english is totally fine - you definitely shouldn't need to apologize :)

I understand your reasons for not adding something like Redis to your project and it makes sense - I was only pointing out the issue with concurrency which is worth mentioning as someone might want to use similar code in production and that's something to be aware of.

Anyhow, I hope you don't take any of this as criticism because I enjoyed your article so keep on coding and writing. I've been coding for 10 years and still learn new stuff every day :)

Thread Thread
agorushkin profile image
Arsenii Gorushkin Author

:D, I have only been coding for a year or so, so I tend to listen to feedback, thanks once again!

Collapse
arunjrk profile image
ArunJRK

Was gonna write about redis 😄

Collapse
hilleer profile image
Daniel Hillmann • Edited

Thanks for your post! It is decently written and easy to follow!

One suggestion that do not seem to be mentioned already, is to use validator module's function isURL (or something similar) rather than defining your own regex for such. While it might work for some cases, a URL is probably anything but simple and it should be preferred to re-use already well-tested libraries.

Collapse
agorushkin profile image
Arsenii Gorushkin Author

Goodpoint, thanks!

Collapse
rpresb profile image
Rodrigo De Presbiteris

Would use the find() rather than for Each because it would return the required item and stop the iteration.

Collapse
agorushkin profile image
Arsenii Gorushkin Author

Yep, another comment has already mentioned it. I just didn't really think of that when I was making it :(

Collapse
stanflows profile image
Stanley Ataki

Awesome and knowledgeable. Hope to make country dial code API from this.

Collapse
agorushkin profile image
Arsenii Gorushkin Author

Glad you liked it, spent a while writing it so it will be easier to understand :D