DEV Community: Kim Kiragu

Extracting Text from Images using Tesseract.js

Kim Kiragu — Mon, 10 Jan 2022 09:49:31 +0000

Hello everyone and new years greetings!

We'll extract text from our scraped photos in Part 2 of our Analyzing Kenya Power Interruption Data.

Tools/Libraries used

TesseractJS - This is a pure JS port of Tesseract OCR which is a popular Optical Character Recognition engine.

Tesseract is built in C and C++, but I wanted to make the project as JS-centric as possible, so I chose the Javascript port.

Text Extraction Process

The images we grabbed from Twitter are all text-based, which is exactly what we need to map out this data. Remember that this is how our images will appear, and we want to be able to save this text as a.txt file.

This stage is straightforward and follows the steps below.

Read all files from the source folder containing our images, KenyaPower in our case.
If it does not already exist, create a folder in the text/destination folder with the same name as the source folder.
Extract text from all images in the source folder and save it to txt files in the destination folder.

The Tesseract.recognize() function accepts an image path, reads it, extracts text from it, and saves it to a txt file in this example.

await Tesseract.recognize(
   path.resolve(`${__dirname}/images/${sourceImageFolder}`, `${files[i]}`),
   "eng"
 ).then(({ data: { text } }) => {
   const textPath = path.resolve(
     `${__dirname}/text/${sourceImageFolder}`,
     `${files[i].replace(".png", "")}.txt`
   );
   const writeStream = fs.createWriteStream(textPath);
   writeStream.write(text);
});

Here's an example of how our text files look once they've been extracted. Everything stayed aligned as it was in our images, which I was extremely impressed with.

Perfomance

My initial goal was simply to be able to extract the text, and I didn't give performance a priority. However, after getting the script to run, I decided to look into ways to speed up the process, especially because the entire project flow will be automated in the end.

To calculate the time it took for my entire script to run, I used NodeJS perf hooks. It was quite useful, and I will definitely use it again if I encounter a similar situation.

Node v8.5.0 added Performance Timing API, which includes the performance#now()

A simple example of how to use perfomance.now():

const { performance } = require('perf_hooks');

const howToLive = ['lead', 'with', 'love'];

const startTime = performance.now();

for (let i = 0; i < howToLive.length; i++) {
   console.log(howToLive[i]);
}

const endTime = performance.now();
console.log(`Loop took ${endTime - startTime} milliseconds`);

I utilized 67 images to evaluate my performance, and extracting text from the images took about 803321 milliseconds, that is about 13 minutes for all and 11 seconds each image.

Conclusion

Please let me know if there are any improvements I could make to speed up this process. This process' codebase may be found here: Tesseract Text Extraction

Part 3 of our project, which I am very enthusiastic about, is data cleaning and analysis, which I will do using pandas, so don't worry, Python gurus:-)

Building Twitter Image Downloader using Puppeteer

Kim Kiragu — Tue, 09 Nov 2021 13:00:07 +0000

This is the first part of my Analyzing Kenya Power Interruption Data project. In this part we build a Twitter Image Downloader using Puppeteer.js.

Over the past 2 years Puppeteer has become my go to choice for web scraping and automation because it is JavaScript which is my main stack among other advantages in my opinion:

It is easy to configure and execute
Puppeteer is really fast, it uses headless Chrome.
It is easy to take screenshots and PDFs of pages for UI testing

Tool

Twitter Image Downloader is the tool I built to be able to scrap images from Twitter accounts, of course for educational purposes. I know several such tools exist but I decided to expand my Puppeteer and JS skills by building one myself.

The main libraries I used to build this tool are:

Puppeteer - Node.js library which provides a high-level API to control headless Chrome or Chromium or to interact with the DevTools protocol. I use it for web crawling and scarping in this project.
Request - Simplified http request client
Inquirer - An easily embeddable and beautiful command line interface for Node.js
Chalk - Chalk is a library that provides a simple and easy to use interface for applying ANSI colors and styles to your command-line output.

Puppeteer Launch

This article is not a step by step guide to building the tool rather an unofficial documentation of my thought process while building it. The instructions for running the tool can be found in the README.md here

The code below is my puppeteer config. I set headless to false in my normal developer environment so that I can be able to see what is happening especially if the scroll is effective.

const browser = await puppeteer.launch({
    headless: false,
    args: ["--disable-notifications"],
  });
  const page = await browser.newPage();
  await page.setViewport({
    width: 1366,
    height: 768,
  });

args: ["--disable-notifications"] is used to disable any notifications which may overlay and hide elements we probably want to click or get data from.

The main file is the twitter.js

The url accessed to scrape the images is found on line 67 where username is the Twitter account username entered when running the script

const pageUrl = `https://twitter.com/${username.replace("@", "")}`;

The script opens a new tab on the Chrome based browser that Puppeteer opens and gets the url of all images :

 if (response.request().resourceType() === "image") {
      /**
       * Filter to only collect tweet images and ignore profile pictures and banners.
       */
      if (url.match("(https://pbs.twimg.com/media/(.*))")) {
        /**
         * Convert twitter image urls to high quality
         */
        const urlcleaner = /(&name=([a-zA-Z0-9_]*$))\b/;
        let cleanurl = url.replace(urlcleaner, "&name=large");

        try {
          const imageDetails = cleanurl.match(
            "https://pbs.twimg.com/media/(.*)?format=(.*)&name=(.*)"
          );
          const imageName = imageDetails[1];
          const imageExtension = imageDetails[2];
          console.log(chalk.magenta("Downloading..."));
          await downloader(cleanurl, imageName, imageExtension, username);
        } catch (error) {}
      }
    }

The response.request().resourceType() === "image" part is responsible for only checking for images because that is what we are currently interested in.

Regex

We see a lot of regex matching and I am going to explain what is going on.

   url.match("(https://pbs.twimg.com/media/(.*))")

A normal Twitter user profile contains many types of images:

Their profile picture and header
Images posted/retweeted
Other retweeted users' profile pictures.

Each of these images have urls and one of my main headaches when starting out was being able to filter out only images in the 2nd category.
Luckily I found out that images posted by tweeting follow the pattern https://pbs.twimg.com/media/.. and that is what we are doing with the url.match function. We ignore all other types of images and only work with posted images.

   const urlcleaner = /(&name=([a-zA-Z0-9_]*$))\b/;
   let cleanurl = url.replace(urlcleaner, "&name=large");

Posted images all follow the same pattern except the &name= part which specifies the dimensions of the image, for example, https://pbs.twimg.com/media/FDSOZT9XMAIo6Sv?format=jpg&name=900x900 900x900 is the dimension of the image.

I needed high quality images because my use case involves extracting data from text which is why I replace the &name=... part of all image urls with &name=large to get the best quality using the urlcleaner regex to match all possibilities.

   const imageDetails = cleanurl.match(
     "https://pbs.twimg.com/media/(.*)?format=(.*)&name=(.*)"
   );
   const imageName = imageDetails[1];
   const imageExtension = imageDetails[2];

The 3rd part retrieves the results of matching the clean modified string and returns results an array where I can be able to access the image name and the extension.

   Array ["https://pbs.twimg.com/media/FDSOZT9XMAIo6Sv?format=jpg&name=large", "FDSOZT9XMAIo6Sv?", "jpg", "large"]

This is what the typical imageDetails will look like.

Autoscroll

Twitter uses infinite Scroll where tweets in current page view are loaded and to load more tweets you have to keep scrolling. This is why I needed an autoscroll function so that our browser could automatically scroll and scroll until it could not load more tweets.

async function autoScroll(page) {
  await page.evaluate(async () => {
    await new Promise((resolve, reject) => {
      let totalHeight = 0;
      const distance = 100;
      const timer = setInterval(() => {
        var scrollHeight = document.body.scrollHeight;
        window.scrollBy(0, distance);
        totalHeight += distance;
        if (totalHeight >= scrollHeight) {
          clearInterval(timer);
          resolve();
        }
      }, 300);
    });
  });
}

Download Images

The function that downloads the images can be found here downloader.js

function download(uri, name, extension, twitterUsername) {
  return new Promise((resolve, reject) => {
    request.head(uri, function (err, res, body) {
      const twitterUsernamePath = `${"./"}/images/${twitterUsername}`;
      if (!fs.existsSync(twitterUsernamePath)) {
        fs.mkdirSync(twitterUsernamePath);
      }
      const filePath = path.resolve(
        twitterUsernamePath,
        `${name}.${extension}`
      );
      request(uri).pipe(fs.createWriteStream(filePath)).on("close", resolve);
    });
  });
}

The function takes in a uri, name, extension and twitterUsername. These parameters are passed in from line 61 of twitter.js

A folder named after the Twitter Username is created here. The images are then written/downloaded to the folder one by one.

The images are named using the passed name and extension, remember the ones we were extracting using the Regex part 3.

Conclusion

There will be several images downloaded but for the purpose of the Analyzing Kenya Power Interruption project, we are interested in the images that look like this.

The code and instructions for running this tool can be found on https://github.com/Kimkykie/twitter-image-downloader

This is still a work in progress and I am open to corrections, ideas and enhancements.

The next part will be extracting text from our images and converting them into txt files. Thank you.

Analyzing Kenya Power Planned Interruption Data

Kim Kiragu — Tue, 09 Nov 2021 10:07:58 +0000

Background

Kenya Power(KPLC) is the main power distributor and retails electricity to customers throughout Kenya.
KPLC posts daily planned outages on their official care account KenyaPowerCare.

The planned interruption schedules are posted in image format and include :

The Region
The Area
Date & Time
Locations within the area

Purpose of the Project

The main inspirations for this project is:

To be able to visualize and understand which parts of the country get more power interruptions than others.
To be able to understand trends and patterns which KPLC use for their interruption schedules.
To possibly automate the process of notifying citizens (via SMS/Email) about planned outages in there areas as soon as they are posted on the KenyaPowerCare Twitter.

Project Outline

The project is going to be a series and will go from scraping the planned interruption images from Twitter to analyzing and visualizing this data.

The steps I am going to take to achieve all this include;

Scraping the images from KenyaPowerCare using this Twitter Image Downloader I built Twitter Image Downloader
Extract Text from the images and converting them to txt file using Tesseract.js
Cleaning, manipulating and analysing the extracted data using Pandas.
Visualizing the data on a map.
Automating the process.

In the next article I am going to take you through the process of building my Twitter Image Downloader.