DEV Community

Cover image for How to deploy our model on huggingface using Gradio
卜乐
卜乐

Posted on

How to deploy our model on huggingface using Gradio

  • Preparing our data

1、define a function of searching images

from duckduckgo_search import DDGS
from fastcore.all import *
import time, json
path = Path('D:\\pycodes\\transport')
transport_types = ['bike', 'car', 'bus']
def search_images(keywords, max_images=200):
    return L(DDGS().images(keywords, max_results=max_images)).itemgot('image')
Enter fullscreen mode Exit fullscreen mode

2、download images from urls

from fastdownload import download_url
from fastai.vision.all import *
# # We will put each in a separate folder
for o in transport_types:
    dest = (path/o)
    dest.mkdir(exist_ok=True, parents=True)
    results = search_images(f'{o}')
    download_images(dest, urls=results)
Enter fullscreen mode Exit fullscreen mode

3、remove the photos which might not be downloaded correctly causing our model training to fail

fns = get_image_files(path)
# check if there are some failed images
failed = verify_images(fns)
# remove all the failed images using unlink on each of them
failed.map(Path.unlink)
Enter fullscreen mode Exit fullscreen mode
from PIL import Image
import warnings

def check_images_in_folder(path):
    for fldr in os.listdir(path):
        sub_folder_path = os.path.join(path, fldr)
        for filee in os.listdir(sub_folder_path):
            file_path = os.path.join(sub_folder_path, filee)
            with warnings.catch_warnings(record=True) as w:
                # 忽略所有其他警告
                warnings.simplefilter("always")
                try:
                    # 尝试打开图片
                    im = Image.open(file_path)
                    # 转换为 RGB 格式
                    rgb_im = im.convert('RGB')
                except Exception as e:
                    # 如果发生异常,打印出图片路径和异常信息
                    print(f"Error opening image: {file_path} - {e}")
                    Path(file_path).unlink()
                # 检查是否有警告信息
                for warning in w:
                    if "Palette images with Transparency expressed in bytes should be converted to RGBA images" in str(warning.message):
                        print(f"Warning opening image: {file_path} - {warning.message}")
                        Path(file_path).unlink()
# 调用函数检查图片
check_images_in_folder(path)
Enter fullscreen mode Exit fullscreen mode
  • Training our model and using it to clean data

1、create the dataloaders using a DataBlock

transports = DataBlock(
    # ImageBlock: independent var, used to make predictions
    # CategoryBlock: target(type of bears)
    blocks=(ImageBlock, CategoryBlock),
    # how to get the list of all the items 
    # The get_image_files fun takes a path, and returns a list of all images in that path
    get_items=get_image_files,  
    splitter=RandomSplitter(valid_pct=0.2, seed=42),
    get_y=parent_label,
    item_tfms=RandomResizedCrop(224, min_scale=0.8),
    batch_tfms=aug_transforms(mult=2)
)
dls = transports.dataloaders(path, bs=32).cuda()
Enter fullscreen mode Exit fullscreen mode

2、finetune the model on our data

learn = vision_learner(dls, resnet18, metrics=error_rate)
learn.fine_tune(4)
Enter fullscreen mode Exit fullscreen mode

Image description

3、create a confusion matrix

interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix()
Enter fullscreen mode Exit fullscreen mode

Image description

4、plot the images with the top losses in our dataset

interp.plot_top_losses(6, nrows=2)
Enter fullscreen mode Exit fullscreen mode

Image description

5、using the ImageClassifierCleaner to clean our data

Image description

  • Retraining our model
transports = DataBlock(
    # ImageBlock: independent var, used to make predictions
    # CategoryBlock: target(type of bears)
    blocks=(ImageBlock, CategoryBlock),
    # how to get the list of all the items 
    # The get_image_files fun takes a path, and returns a list of all images in that path
    get_items=get_image_files,  
    splitter=RandomSplitter(valid_pct=0.2, seed=42),
    get_y=parent_label,
    item_tfms=RandomResizedCrop(224, min_scale=0.8),
    batch_tfms=aug_transforms(mult=2)
)
dls = transports.dataloaders(path, bs=32).cuda()
learn = vision_learner(dls, resnet18, metrics=error_rate)
learn.fine_tune(4)
learn.export('D:\\pycodes\\transport\\model.pkl')
Enter fullscreen mode Exit fullscreen mode

Image description

Look! The error rate is dropping!

  • Deploy our model

1、create a Gradio interface

from fastai.vision.all import *
import gradio as gr
import platform
import pathlib
plt = platform.system()
if plt != 'Windows':
  pathlib.WindowsPath = pathlib.PosixPath

learn = load_learner('model.pkl')

categories = ('bike', 'bus', 'car')

def classify_image(img):
    pred, idx, probs = learn.predict(img)
    return dict(zip(categories, map(float, probs)))

image = gr.Image(width=240, height=240)
label = gr.Label()
examples = ['bike.jpg', 'bus.jpg', 'car.jpg']

intf = gr.Interface(fn=classify_image, inputs=image, outputs=label, examples=examples)
intf.launch(inline=False)
Enter fullscreen mode Exit fullscreen mode

2、deploy our model on hugging face

from fastai.vision.all import *
import gradio as gr
import platform
import pathlib
plt = platform.system()
if plt != 'Windows':
  pathlib.WindowsPath = pathlib.PosixPath

learn = load_learner('model.pkl')

categories = ('bike', 'bus', 'car')

def classify_image(img):
    pred, idx, probs = learn.predict(img)
    return dict(zip(categories, map(float, probs)))

image = gr.Image(width=240, height=240)
label = gr.Label()
examples = ['bike.jpg', 'bus.jpg', 'car.jpg']

intf = gr.Interface(fn=classify_image, inputs=image, outputs=label, examples=examples)
intf.launch(inline=False)
Enter fullscreen mode Exit fullscreen mode

Do your career a big favor. Join DEV. (The website you're on right now)

It takes one minute, it's free, and is worth it for your career.

Get started

Community matters

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay