DEV Community

Cover image for Text Generation With GPT-2 in Python
James Briggs
James Briggs

Posted on

Text Generation With GPT-2 in Python

Language generation is one of those natural language tasks that can really produce an incredible feeling of awe at how far the fields of machine learning and artificial intelligence have come.

GPT-1, 2, and 3 are OpenAI’s top language models — well known for their ability to produce incredibly natural, coherent, and genuinely interesting language.

In this video, we will take a small snippet of text and learn how to feed that into a pre-trained GPT-2 model using PyTorch and Transformers to produce high-quality language generation in just eight lines of code. We cover:

PyTorch and Transformers

  • Data

Building the Model

  • Initialization

  • Tokenization

  • Generation

  • Decoding

Results

Discussion (1)

Collapse
akshatsinghania profile image
Akshat Singhania

youre so cooooooooooooooooooool