YOLOv9 vs. YOLOv8: Segmentation & Fine-Tuning Guide

#computerscience #vision #python #tutorial

Introduction

The YOLOv9 model for object segmentation was released recently, offering superior performance to the previous YOLOv8 model. This article will compare YOLOv8 and YOLOv9, showcase YOLOv9 segmentation, and include a guide for fine-tuning YOLOv9 on your own datasets.

Comparison and Showcase

In this section, we will compare YOLOv8 and YOLOv9 performance and quickly showcase YOLOv9 segmentation.

YOLOv8 vs. YOLOv9

The new YOLO model uses techniques such as Programmable Gradient Information (PGI) and Generalized Efficient Layer Aggregation Network (GELAN) to improve performance.

YOLOv9 demonstrates improved accuracy and efficiency for object segmentation, achieving higher precision and recall rates compared to YOLOv8【12†source】【13†source】.

Demo

YOLOv9e segmentation demo:

Training

It’s possible to train YOLOv9 on your own segmentation data. The process can be divided into three steps: (1) Installation, (2) Dataset Creation, and (3) Fine-tuning/Training.

Installation

Begin by installing the Ultralytics framework:

pip install ultralytics

Dataset

Choose or create the dataset you need. The dataset needs to be in YOLO segmentation format, meaning each image shall have a corresponding text file (.txt) with the following content:

<class-index> <x1> <y1> <x2> <y2> ... <xn> <yn>
...
<class-index> <x1> <y1> <x2> <y2> ... <xn> <yn>

After finding a dataset and completing the image annotations, organize the dataset in the following way:

path/to/dataset/
├─ train/
│  ├─ img_0000.jpg
│  ├─ img_0000.txt
│  ├─ ...
│  ├─ img_0999.jpg
│  ├─ img_0999.txt
├─ val/
│  ├─ img_1000.jpg
│  ├─ img_1000.txt
│  ├─ ...
│  ├─ img_1099.jpg
│  ├─ img_1099.txt

Fine-Tuning

This section is for you if you want to train YOLOv9 on your custom data. If you’re just looking to use the model, skip ahead to the section Inference and Segmentation.

First, begin by creating a training configuration file:

# train.yaml
path: path/to/dataset
train: train
val: val

names:
  0: person
  1: bicycle
  2: car
  # ...
  77: teddy bear
  78: hair drier
  79: toothbrush

The configuration file shall contain the paths to the training and validation sets, class names, and class mapping.

Finally, train the model using the Ultralytics framework:

from ultralytics import YOLO

model = YOLO("yolov9c-seg.yaml")
model.train(data="path/to/train.yaml", epochs=100)

Make sure to use the correct segmentation model depending on your time constraints and hardware:

yolov9c-seg.yaml
yolov9e-seg.yaml

Best Practices

If you want to optimize the training performance, read this guide:

YOLOv8: Best Practices for Training

Inference and Segmentation

Run inference:

results = model("images/sofa.jpg")

Plot segmented masks:

import numpy as np
import matplotlib.pyplot as plt
import cv2

for result in results:
    height, width = result.orig_img.shape[:2]
    background = np.ones((height, width, 3), dtype=np.uint8) * 255

    masks = result.masks.xy
    for mask in masks:
        mask = mask.astype(int)
        cv2.drawContours(background, [mask], -1, (0, 255, 0), thickness=cv2.FILLED)

    plt.imshow(background)
    plt.title('Segmented objects')
    plt.axis('off')
    plt.show()

    plt.imsave('segmented_objects.jpg', background)

Plot segmentation mask with original colors:

import cv2
import numpy as np
import matplotlib.pyplot as plt

for result in results:
    height, width = result.orig_img.shape[:2]
    background = np.ones((height, width, 3), dtype=np.uint8) * 255
    masks = result.masks.xy
    orig_img = result.orig_img

    for mask in masks:
        mask = mask.astype(int)
        mask_img = np.zeros_like(orig_img)

        cv2.fillPoly(mask_img, [mask], (255, 255, 255))
        masked_object = cv2.bitwise_and(orig_img, mask_img)
        background[mask_img == 255] = masked_object[mask_img == 255]

    background_rgb = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)

    plt.imshow(background_rgb)
    plt.title('Segmented objects')
    plt.axis('off')
    plt.show()

    cv2.imwrite('segmented_objects.jpg', background)

Bonus: Comparison with Other Technologies

Let's briefly compare YOLO with another popular computer vision technology, such as Mask R-CNN, in terms of performance and ease of use.

Performance: YOLO models are generally faster but might be less accurate for complex segmentation tasks compared to Mask R-CNN.
Ease of Use: YOLO is simpler to set up and use for real-time applications, while Mask R-CNN can be more complex but offers finer segmentation.

Word of the Day

Generalized Efficient Layer Aggregation Network (GELAN) - A technique used in YOLOv9 to enhance model performance by efficiently aggregating information across layers.

Conclusion

YOLOv9 offers significant improvements over YOLOv8, particularly in accuracy and efficiency for object segmentation tasks. Training and fine-tuning your own YOLOv9 model can be straightforward with the right dataset and tools. Don't hesitate to experiment and share your results!

Limitations:

The limitation of YOLO algorithm is that it struggles with small objects with in the image. The algorithm might not be able to detect very small object in the image, due to spatial constraints of the algorithm.