DEV Community

Julien Simon
Julien Simon

Posted on • Originally published at julsimon.Medium on

NLP models: from the Hugging Face hub to Amazon SageMaker… and back!

In this video, I start from a pre-trained model and a dataset hosted on the Hugging Face hub. Running a Jupyter notebook in SageMaker Studio, I pre-process the data, and I fine-tune a sentiment analysis model on SageMaker infrastructure. Then, I deploy the model on a SageMaker endpoint and predict with it.

Next, I retrieve the trained model in S3 and I use the Hugging Face CLI to push the model to the Hugging Face hub. From there, I use the open source Transformers library to work with the model, just like I would do with any Hugging Face model.

Finally, using the SageMaker SDK, I redeploy the model directly from the Hugging Face hub to a SageMaker endpoint.

Dataset and notebook: https://huggingface.co/juliensimon/reviews-sentiment-analysis/tree/main

New to Transformers? Check out the Hugging Face course at https://huggingface.co/course

Discussion (0)