Easiest Way to Deploy HuggingFace Transformers

You must know transformers. Well, I don’t mean autobots, Bumble Bee, but the famous machine learning structure.
You probably have used the huggingface transformers models. But have you ever deployed them?

With Pinferencia, just add three more lines and your model goes online!

Never heard of Pinferencia? It’s not late. Go to its GitHub to take a look. Don’t forget to give it a star if you like it.

---

HuggingFace transformer pipeline

How do you use HuggingFace transformer pipeline?

from transformers import pipeline

vision_classifier = pipeline(task="image-classification")


def predict(data):
    return vision_classifier(images=data)

And you can predict a image with its url:

predict("https://cdn.pixabay.com/photo/2018/08/12/16/59/parrot-3601194_1280.jpg")

Result:

[[{'score': 0.9489120244979858, 'label': 'macaw'},
  {'score': 0.014800671488046646, 'label': 'broom'},
  {'score': 0.009150494821369648, 'label': 'swab, swob, mop'},
  {'score': 0.0018255198374390602, 'label': "plunger, plumber's helper"},
  {'score': 0.0017631321679800749,
   'label': 'African grey, African gray, Psittacus erithacus'}]]

Deploy

Now deploy it with Pinferencia, just add three lines and save as app.py

from transformers import pipeline
from pinferencia import Server

vision_classifier = pipeline(task="image-classification")


def predict(data):
    return vision_classifier(images=data)


service = Server()
service.register(model_name="vision", model=predict)

Now go to the terminal and run

uvicorn app:service --reload

Your service is online! Go to http://127.0.0.1:8000 and check out the API.
Now you can send a request:

curl --location --request POST 'http://127.0.0.1:8000/v1/models/vision/predict' \
    --header 'Content-Type: application/json' \
    --data-raw '{
        "data": "https://cdn.pixabay.com/photo/2018/08/12/16/59/parrot-3601194_1280.jpg"
    }'

The result:

[[{'score': 0.9489120244979858, 'label': 'macaw'},
  {'score': 0.014800671488046646, 'label': 'broom'},
  {'score': 0.009150494821369648, 'label': 'swab, swob, mop'},
  {'score': 0.0018255198374390602, 'label': "plunger, plumber's helper"},
  {'score': 0.0017631321679800749,
   'label': 'African grey, African gray, Psittacus erithacus'}]]

Or just use the interactive UI Pinferencia provides:

Simple enough, huh?

---

If you like Pinferencia go https://github.com/underneathall/pinferencia/star and give it a star.