Introduction
Image segmentation, one of the most basic procedures in computer vision, allows a system to decompose and analyze various regions within an image. Whether you're dealing with object recognition, medical imaging, or autonomous driving, segmentation is what breaks images down into meaningful parts.
Although deep learning models continue to be increasingly popular in this task, traditional techniques in digital image processing still are powerful and practical. The approaches being reviewed in this post include thresholding, edge detection, region-based, and clustering by implementing a well-recognized dataset for the analysis of cell images, the MIVIA HEp-2 Image Dataset.
MIVIA HEp-2 Image Dataset
The MIVIA HEp-2 Image Dataset is a set of pictures of the cells used to analyze the pattern of antinuclear antibodies (ANA) through HEp-2 cells. It consists of 2D pictures taken through fluorescence microscopy. This makes it very suitable for segmentation tasks, most importantly those concerned with medical image analysis, where cellular region detection is most important.
Now, let's move on to the segmentation techniques used to process these images, comparing their performance based on the F1 scores.
1. Thresholding Segmentation
Thresholding is the process whereby grayscale images get converted into binary images based on pixel intensities. In the MIVIA HEp-2 dataset, this process is useful in cell extraction from the background. It is simple and effective to a relatively large level, especially with Otsu's method, as it self-computes the optimum threshold.
Otsu's Method is an automatic thresholding method, where it tries to find the best threshold value to yield the minimal intra-class variance, thereby separating the two classes: foreground (cells) and background. The method examines the image histogram and computes the perfect threshold, where the sum of the pixel intensity variances in each class is minimized.
# Thresholding Segmentation
def thresholding(img):
# Convert image to grayscale
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
# Apply Otsu's thresholding
_, thresh = cv.threshold(gray, 0, 255, cv.THRESH_BINARY + cv.THRESH_OTSU)
return thresh
2. Edge Detection Segmentation
Edge detection pertains to identifying boundaries of objects or regions, such as cell edges in the MIVIA HEp-2 dataset. Of the many available methods for detecting abrupt intensity changes, the Canny Edge Detector is the best and hence most appropriate method to be used for detecting cellular boundaries.
Canny Edge Detector is a multi-stage algorithm that can detect the edges by detecting areas of strong gradients of intensity. The process embodies smoothing with a Gaussian filter, calculation of intensity gradients, application of non-maximum suppression to eliminate spurious responses, and a final double thresholding operation for the retention of only salient edges.
# Edge Detection Segmentation
def edge_detection(img):
# Convert image to grayscale
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
# Apply Gaussian blur
gray = cv.GaussianBlur(gray, (3, 3), 0)
# Calculate lower and upper thresholds for Canny edge detection
sigma = 0.33
v = np.median(gray)
lower = int(max(0, (1.0 - sigma) * v))
upper = int(min(255, (1.0 + sigma) * v))
# Apply Canny edge detection
edges = cv.Canny(gray, lower, upper)
# Dilate the edges to fill gaps
kernel = np.ones((5, 5), np.uint8)
dilated_edges = cv.dilate(edges, kernel, iterations=2)
# Clean the edges using morphological opening
cleaned_edges = cv.morphologyEx(dilated_edges, cv.MORPH_OPEN, kernel, iterations=1)
# Find connected components and filter out small components
num_labels, labels, stats, _ = cv.connectedComponentsWithStats(
cleaned_edges, connectivity=8
)
min_size = 500
filtered_mask = np.zeros_like(cleaned_edges)
for i in range(1, num_labels):
if stats[i, cv.CC_STAT_AREA] >= min_size:
filtered_mask[labels == i] = 255
# Find contours of the filtered mask
contours, _ = cv.findContours(
filtered_mask, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE
)
# Create a filled mask using the contours
filled_mask = np.zeros_like(gray)
cv.drawContours(filled_mask, contours, -1, (255), thickness=cv.FILLED)
# Perform morphological closing to fill holes
final_filled_image = cv.morphologyEx(
filled_mask, cv.MORPH_CLOSE, kernel, iterations=2
)
# Dilate the final filled image to smooth the edges
final_filled_image = cv.dilate(final_filled_image, kernel, iterations=1)
return final_filled_image
3. Region-Based Segmentation
Region-based segmentation groups similar pixels together into regions, dependent on certain criteria such as intensity or color. The Watershed segmentation technique can be used to help in segmenting HEp-2 cell images to be able to detect those regions that represent cells; it considers pixel intensities as a topographic surface and outlines distinguishing regions.
Watershed segmentation treats the intensities of pixels as a topographical surface. The algorithm identifies "basins" in which it identifies local minima and then gradually floods these basins to enlarge distinct regions. This technique is quite useful when one wants to separate touching objects, like in the case of cells within microscopic images, but it can be sensitive to noise. The process can be guided by markers and over-segmentation can often be reduced.
# Region-Based Segmentation
def region_based(img):
# Convert image to grayscale
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
# Apply Otsu's thresholding
_, thresh = cv.threshold(gray, 0, 255, cv.THRESH_BINARY_INV + cv.THRESH_OTSU)
# Apply morphological opening to remove noise
kernel = np.ones((3, 3), np.uint8)
opening = cv.morphologyEx(thresh, cv.MORPH_OPEN, kernel, iterations=2)
# Dilate the opening to get the background
sure_bg = cv.dilate(opening, kernel, iterations=3)
# Calculate the distance transform
dist_transform = cv.distanceTransform(opening, cv.DIST_L2, 5)
# Threshold the distance transform to get the foreground
_, sure_fg = cv.threshold(dist_transform, 0.2 * dist_transform.max(), 255, 0)
sure_fg = np.uint8(sure_fg)
# Find the unknown region
unknown = cv.subtract(sure_bg, sure_fg)
# Label the markers for watershed algorithm
_, markers = cv.connectedComponents(sure_fg)
markers = markers + 1
markers[unknown == 255] = 0
# Apply watershed algorithm
markers = cv.watershed(img, markers)
# Create a mask for the segmented region
mask = np.zeros_like(gray, dtype=np.uint8)
mask[markers == 1] = 255
return mask
4. Clustering-Based Segmentation
Clustering techniques such as K-Means tend to group the pixels into similar clusters, which works fine when wanting to segment cells in multi-colored or complex environments, as seen in HEp-2 cell images. Fundamentally, this could represent different classes, such as a cellular region versus a background.
K-means is an unsupervised learning algorithm for clustering images based on the pixel similarity of color or intensity. The algorithm randomly selects K centroids, assigns each pixel to the nearest centroid, and updates the centroid iteratively until it converges. It is particularly effective in segmenting an image that has multiple regions of interest that are very different from one another.
# Clustering Segmentation
def clustering(img):
# Convert image to grayscale
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
# Reshape the image
Z = gray.reshape((-1, 3))
Z = np.float32(Z)
# Define the criteria for k-means clustering
criteria = (cv.TERM_CRITERIA_EPS + cv.TERM_CRITERIA_MAX_ITER, 10, 1.0)
# Set the number of clusters
K = 2
# Perform k-means clustering
_, label, center = cv.kmeans(Z, K, None, criteria, 10, cv.KMEANS_RANDOM_CENTERS)
# Convert the center values to uint8
center = np.uint8(center)
# Reshape the result
res = center[label.flatten()]
res = res.reshape((gray.shape))
# Apply thresholding to the result
_, res = cv.threshold(res, 0, 255, cv.THRESH_BINARY + cv.THRESH_OTSU)
return res
Evaluating the Techniques Using F1 Scores
The F1 score is a measure that combines precision and recall together to compare the predicted segmentation image with the ground truth image. It is the harmonic mean of precision and recall, which is useful in cases of high data imbalance, such as in medical imaging datasets.
We calculated the F1 score for each segmentation method by flattening both the ground truth and the segmented image and calculating the weighted F1 score.
def calculate_f1_score(ground_image, segmented_image):
ground_image = ground_image.flatten()
segmented_image = segmented_image.flatten()
return f1_score(ground_image, segmented_image, average="weighted")
We then visualized the F1 scores of different methods using a simple bar chart:
Conclusion
Although many recent approaches for image segmentation are emerging, traditional segmentation techniques such as thresholding, edge detection, region-based methods, and clustering can be very useful when applied to datasets such as the MIVIA HEp-2 image dataset.
Each method has its strength:
- Thresholding is good for simple binary segmentation.
- Edge Detection is an ideal technique for the detection of boundaries.
- Region-based segmentation is very useful in separating connected components from their neighbors.
- Clustering methods are well-suited for multi-region segmentation tasks.
By evaluating these methods using F1 scores, we understand the trade-offs each of these models has. These methods may not be as sophisticated as what is developed in the newest models of deep learning, but they are still fast, interpretable, and serviceable in a broad range of applications.
Thanks for reading! I hope this exploration of traditional image segmentation techniques inspires your next project. Feel free to share your thoughts and experiences in the comments below!
Top comments (0)