text summarizer project

mitya8128 — Thu, 14 Oct 2021 20:54:26 +0000

Hey, just want to drop a link to my personal post about my new project in the field of text summarization

Hope you'll enjoy =)

graph_summarizer

mitya8128 — Thu, 14 Oct 2021 20:42:14 +0000

mitya8128 / graph_summarizer

summarize text using graphs and language vector models

Today I want to tell about my new experimental project - text summarizer based on graph algorithms and distributive language models.

Little prehistory: some time ago I worked on graph visualizations based on vector embedding of texts . This vizualisations show relational "distance" between words of sentences. So user can visualize graph of distances between word of text, also you can apply some advanced graph-theoretic approaches to further analyze input text and it's hidden features.

I want to use this approach to try on text summarization problem - in my case to extract some more informative sentences from text to compress original text.

Here's the link to the repo

You can use to play around, don't hesitate to contact me if you have any feedback!
For local use you need to download word2vec model (300d, binary format) and put it in model folder inside root folder.
Demo of using (yet only in Russian, soon I'll add another languages)

generate_summary_loop() function is an endpoint.

NB: use flag need_tag=True if you need to use pretrained model with POS-tags, else need_tag=False.

DEV Community: mitya8128

text summarizer project

graph_summarizer

mitya8128 / graph_summarizer

summarize text using graphs and language vector models