DEV Community

Cover image for Practical Puppeteer: Create short URL with
Sony AK
Sony AK

Posted on


Practical Puppeteer: Create short URL with

Let's do fun scraping again. This time we will scrap to service to create short URL. I guess must be familiar with right? As usual we will use Puppeteer to do the scraping.

Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol. Puppeteer runs headless by default, but can be configured to run full (non-headless) Chrome or Chromium. For more information about Puppeteer please go to

We will control from Puppeteer. The step is we go to homepage, put the long URL, press the button to make it short URL and get the short URL result and display it to console in JSON format. Pretty simple.

Let's start.


Install Puppeteer

npm i puppeteer
Enter fullscreen mode Exit fullscreen mode

The code

This is the Javascript code to control Puppeteer.

File short_url_bitly.js

const puppeteer = require('puppeteer');

(async () => {
    // set some options, set headless to false so we can see the browser in action
    let launchOptions = { headless: false, args: ['--start-maximized'] };

    // launch the browser with above options
    const browser = await puppeteer.launch(launchOptions);
    const page = await browser.newPage();

    // set viewport and user agent (just in case for nice viewing)
    await page.setViewport({width: 1366, height: 768});
    await page.setUserAgent('Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36');

    // go to
    await page.goto('');

    // wait until the input selector for long URL available
    await page.waitForSelector('#shorten_url');
    await page.waitFor(2000);

    // this is sample of long URL
    // you can try with your own URL
    let urlToShorten = '';

    // type the long URL to the input selector
    await page.type('#shorten_url', urlToShorten);
    await page.waitFor(1000);

    // click the button to make it short URL
    await page.waitFor(1000);

    // evaluate the result of short URL and put on variable
    // and pass the variable urlToShorten to page.evaluate
    const urlShortener = await page.evaluate((urlToShorten) => {
        return {
                 'long_url': urlToShorten,
                 'short_url': document.getElementsByClassName('short-link')[0].textContent
    }, urlToShorten);

    // display the result (long and short url) in JSON format

    // close the browser
    await browser.close();
Enter fullscreen mode Exit fullscreen mode

Run it

node short_url_bitly.js
Enter fullscreen mode Exit fullscreen mode

If everything OK, it will display the result on your console that contains the long URL and the short URL like below.

Alt Text

Thank you and I hope you enjoy it.

The repository of this code is available at

Top comments (0)

Visualizing Promises and Async/Await 🤯

async await

☝️ Check out this all-time classic DEV post