HuggingFace Transformers Bert — Unmask the Myth: Play and Deploy within 10 Lines of Codes

Bert is a fantastic model to play with. It can infer your missing word in the sentence.

In the article, we will deploy Google T5 model as a REST API service. Difficult? What about I’ll tell you: you just need to write 6 lines of codes?

---

Pinferencia makes it super easy to serve any model with just three extra lines.
HuggingFace makes it easy to use the pretrained model with just several lines.

---

Install Dependencies

HuggingFace

pip install "transformers[pytorch]"

If it doesn’t work, please visit Installation (huggingface.co) and check their official documentations.

Pinferencia

pip install "pinferencia[uvicorn]"

If it doesn’t work, please visit Install — Pinferencia (underneathall.app) and check their official documentations.

Define the Service

First let’s create the app.py to define the service:

Start the Service

    uvicorn app:service --reload

Wait for the model get downloaded. When it’s finished, you’ll see:

Call the Service

You can use curl or the interactive api page from Pinferencia.

Curl

Result:

The sentence with the highest score is: penguins cannot fly. Make sense, right?

Besides Curl, you can also use Pinferencia’s:

Interactive API Page

Mlearning.ai Submission Suggestions