How to visualize timeline of a Wiki article?

Sanjaya Kumar Saxena — Tue, 18 Apr 2023 16:28:24 +0000

Automatic generation of the timeline — a graphical representation of a time period, on which important events are marked — from a Wikipedia article is a fascinating idea and very useful in quickly grasping the historical perspective. This post outlines the approach to create a well formatted timeline from any Wikipedia article using WinkNLP’s API and Named Entity Recognition (NER) feature:

Fetch the article's contents and convert them into a WinkNLP document.
Iterate through detected entities and filter only DATEs.
Use shapes of dates to convert them into standard Unix time.
Using parentSentence() API, extract the sentence containing the date; also markup() the date to highlight it in the corresponding sentence.
Collect each Unix time and sentence pair in an array and sort them on Unix time.
Converts this array into a well formatted timeline using Observable capabilities along with some CSS.

The above approach is realized in about 30 lines of code:

timeLine = {
  const response = await fetch( `https://en.wikipedia.org/w/api.php?action=query&prop=extracts&titles=${WikiArticleTitle || '2022 United Nations Climate Change Conference'}&explaintext=1&formatversion=2&format=json&origin=*` );
  const body = await response.json();
  const text = body.query.pages[ 0 ].extract;

  var doc = nlp.readDoc( text || '' );
        var timeline = [];
        doc
          .entities()
          .filter( ( e ) => {
            var shapes = e.tokens().out( its.shape );
            // We only want dates that can be converted to an actual
            // time using new Date()
            return (
              e.out( its.type ) === 'DATE' &&
              (
                shapes[ 0 ] === 'dddd' ||
                ( shapes[ 0 ] === 'Xxxxx' && shapes[ 1 ] === 'dddd' ) ||
                ( shapes[ 0 ] === 'Xxxx' && shapes[ 1 ] === 'dddd' ) ||
                ( shapes[ 0 ] === 'dd' && shapes[ 1 ] === 'Xxxxx' && shapes[ 2 ] === 'dddd' ) ||
                ( shapes[ 0 ] === 'dd' && shapes[ 1 ] === 'Xxxx' &&  shapes[ 2 ] === 'dddd' ) ||
                ( shapes[ 0 ] === 'd' &&  shapes[ 1 ] === 'Xxxxx' && shapes[ 2 ] === 'dddd' ) ||
                ( shapes[ 0 ] === 'd' &&  shapes[ 1 ] === 'Xxxx' &&  shapes[ 2 ] === 'dddd' )
              )
            );
          })
          .each( ( e ) => {
            e.markup();
            let eventDate = e.out();
            if ( isNaN( eventDate[ 0 ] ) ) eventDate = '1 ' + eventDate;
            timeline.push({
              date: e.out(),
              unixTime: new Date( eventDate ).getTime() / 1000,
              sentence: e.parentSentence().out( its.markedUpText )
            })
          });

        return timeline.sort( ( a, b ) => a.unixTime - b.unixTime )
}

You can see it in action on an interactive Observable notebook — "How to visualize timeline of a Wiki article?".

About winkNLP

WinkNLP is a developer friendly JavaScript library for Natural Language Processing (NLP). It can easily process large amount of raw text at speeds over 650,000 tokens/second on a M1 Macbook Pro in both browser and Node.js environments. It even runs smoothly on a low-end smartphone's browser.

It is built ground up with a lean code base that has no external dependency. A test coverage of ~100% and compliance with the Open Source Security Foundation best practices make winkNLP the ideal tool for building production grade systems with confidence.

How to run NLP on a PDF file?

Sanjaya Kumar Saxena — Tue, 15 Dec 2020 11:26:54 +0000

On several occasions we need to extract information from PDF documents. The first step is to convert the PDF document into raw text using a PDF parser. In the following example, we use the pdf-parse NPM package to achieve this. Once we have the raw text, it can be passed to winkNLP's readDoc method to process the text. The doc object returned by this API can be used to access a variety of information such as named entities, sentences containing negation, overall or sentence wise sentiment score and many more. We have illustrated extraction of named entities here — also known as NER.

// Load wink-nlp package & helpers.
const winkNLP = require( 'wink-nlp' );
const its = require( 'wink-nlp/src/its.js' );
const model = require( 'wink-eng-lite-model' );
const nlp = winkNLP( model );

const fs = require('fs');
const pdf = require( 'pdf-parse' );

// Read PDF file.
let dataBuffer = fs.readFileSync( './sample.pdf' );

// Parse & extract entities from the dataBuffer.
pdf( dataBuffer ).then( function( data ) {
  const doc = nlp.readDoc(data.text);
  console.log( doc.entities().out( its.detail ) );
});

The above code will read the PDF file located in the current directory and print all the named entities detected along with their type i.e. DATE, TIME, MONEY, EMAIL and many more. Each entity is in the form of a Javascript object containing two properties — value and type; for example, {value: 'March 15, 1972', type: 'DATE' }.

winkNLP’s English language lite model uses a pre-trained state machine to recognize named entities.

This could be useful in extraction of meaningful information from a resume, financial document or a complete book.

Photo by Annie Spratt on Unsplash

DEV Community: Sanjaya Kumar Saxena

How to visualize timeline of a Wiki article?

About winkNLP

How to run NLP on a PDF file?