DEV Community

Cover image for NlphoseBuilder : A tool to create NLP pipelines via drag and drop
code2k13
code2k13

Posted on

NlphoseBuilder : A tool to create NLP pipelines via drag and drop

✨Checkout the live demo here

Recently I completed work on a tool called nlphoseGUIBuilder that allows creation of complex NLP pipelines visually, without writing a single line of code ! It uses Blockly to enable creation of NLP pipelines using drag and drop.

Currently following operations are supported:

  • Sentiment Analysis (AFINN)
  • NER (Spacy)
  • Language Identification (FastText)
  • Chunking (NLTK)
  • Sentiment Analysis (Transformers)
  • Question Answering (Transformers)
  • Zero shot Classification (Transformers)

The tool generates a nlphose command that can be executed in a docker container to run the pipeline. These pipelines can process streaming text like tweets or static data like files. They can be executed just like normal shell command using nlphose. Let me show you what I mean !

Below is pipeline that searches Twitter for tweets containing 'netflix' and performs named entity recognition on it.
Alt Text

It generates a nlphose command which looks like this

twint -s netflix |\ 
./twint2json.py |\ 
./entity  |\ 
./senti 
Enter fullscreen mode Exit fullscreen mode

When the above pipeline is run using nlphose, you can expect to see stream of JSON output similar to the one shown below:

....
{
  "id": "6a5fe972-e2e6-11eb-9efa-42b45ace4426",
  "text": "Wickham were returned, and to lament over his absence from the Netherfield ball. He joined them on their entering the town, and attended them to their aunt’s where his regret and vexation, and the concern of everybody, was well talked over. To Elizabeth, however, he voluntarily acknowledged that the necessity of his absence _had_ been self-imposed.",
  "afinn_score": -1.0,
  "entities": [
    {
      "label": "PERSON",
      "entity": "Wickham"
    },
    {
      "label": "ORG",
      "entity": "Netherfield"
    },
    {
      "label": "PERSON",
      "entity": "Elizabeth"
    }
  ]
}
...
Enter fullscreen mode Exit fullscreen mode

Lets try out something more, the below pipeline searches for tweets containing the word 'rainfall' and then finds the location where it rained using 'extractive question answering'. It also filters out answers with lower scores.
Alt Text

Here is the nlphose command it generates:

twint -s rainfall |\ 
./twint2json.py |\ 
./xformer.py --pipeline question-answering --param 'where did it rain' |\ 
jq 'if (.xfrmr_question_answering.score) > 0.80 then . else empty end'
Enter fullscreen mode Exit fullscreen mode

It is also possible to create a pipeline that processes multiple files from a folder :
Alt Text

The above pipeline generates this command:

./files2json.py -n 3  data/*.txt |\ 
./xformer.py --pipeline question-answering --param 'who gave the speech ?' |\ 
jq 'if (.xfrmr_question_answering.score) > 0.80 then . else empty end'
Enter fullscreen mode Exit fullscreen mode

Play with the tool here: https://ashishware.com/static/nlphose.html

Here is the link to the projects git repository: https://github.com/code2k13/nlphoseGUI

Here is a YouTube link of the tool in action:

Don't forget to checkout the repository of the companion project nlphose: https://github.com/code2k13/nlphose

Discussion (1)

Collapse
mitya8128 profile image
mitya8128

great project!

what are you going to do next? mayve you have some plans on development of this project?