Practical Puppeteer: Create short URL with Bit.ly

Let's do fun scraping again. This time we will scrap to Bit.ly service to create short URL. I guess must be familiar with Bit.ly right? As usual we will use Puppeteer to do the scraping.

Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol. Puppeteer runs headless by default, but can be configured to run full (non-headless) Chrome or Chromium. For more information about Puppeteer please go to https://pptr.dev.

We will control Bit.ly from Puppeteer. The step is we go to Bit.ly homepage, put the long URL, press the button to make it short URL and get the short URL result and display it to console in JSON format. Pretty simple.

Let's start.

Preparation

Install Puppeteer

npm i puppeteer

The code

This is the Javascript code to control Puppeteer.

File short_url_bitly.js

const puppeteer = require('puppeteer');

(async () => {
    // set some options, set headless to false so we can see the browser in action
    let launchOptions = { headless: false, args: ['--start-maximized'] };

    // launch the browser with above options
    const browser = await puppeteer.launch(launchOptions);
    const page = await browser.newPage();

    // set viewport and user agent (just in case for nice viewing)
    await page.setViewport({width: 1366, height: 768});
    await page.setUserAgent('Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36');

    // go to Bit.ly
    await page.goto('https://bitly.com');

    // wait until the input selector for long URL available
    await page.waitForSelector('#shorten_url');
    await page.waitFor(2000);

    // this is sample of long URL
    // you can try with your own URL
    let urlToShorten = 'https://medium.com/data-from-the-trenches/text-classification-the-first-step-toward-nlp-mastery-f5f95d525d73';

    // type the long URL to the input selector
    await page.type('#shorten_url', urlToShorten);
    await page.waitFor(1000);

    // click the button to make it short URL
    await page.click('#shorten_btn');
    await page.waitFor(1000);

    // evaluate the result of short URL and put on variable
    // and pass the variable urlToShorten to page.evaluate
    const urlShortener = await page.evaluate((urlToShorten) => {
        return {
                 'long_url': urlToShorten,
                 'short_url': document.getElementsByClassName('short-link')[0].textContent
               }
    }, urlToShorten);

    // display the result (long and short url) in JSON format
    console.log(urlShortener);

    // close the browser
    await browser.close();
})();

Run it

node short_url_bitly.js

If everything OK, it will display the result on your console that contains the long URL and the short URL like below.

Thank you and I hope you enjoy it.

The repository of this code is available at https://github.com/sonyarianto/create-short-url-with-bitly-and-puppeteer.git