[Part 1] Your first steps in Computer Vision on AWS: ASL dataset + preprocessing

This is the first part of the series "Your first steps in Computer Vision on AWS". If you are just starting, this tutorial will guide you step by step from the initial dataset to a deployed app.

Problem

Computer vision (CV) can help us to solve many problems. Sign Language Recognition (SLR) addresses the communication barrier between Deaf and Hard of Hearing (DHH) individuals and hearing non-signers. Our system should provide an automated character transcription without requiring human interpreters.

Fact: Currently, more than 1.5 billion people (nearly 20% of the global population) live with hearing loss; 430 million of them have disabling hearing loss. [Source: World Health Organization]

The core purpose of CV in this domain is to transform high-dimensional, unstructured visual data (RGB images) into structured categories.

Photo by Assad Tanoli on Unsplash

Dataset

The ASL Alphabet dataset (Kaggle) comprises 87,000 training images and 29 test images. Images are 200x200 pixels and in 3-channel RGB format. The dataset has 29 classes: 26 English letters (A-Z) and 3 control signs (SPACE, DELETE, NOTHING). As you can see, it is perfectly balanced: 3,000 images per class in the training set.

One possible limitation is that images were captured under controlled, repeatable environments. They have static, isolated hand postures. We will see in later parts how we can address and potentially overcome such limitations.

Which computer vision task type to select?

Nice overview of CV task types is provided by IBM. Potentially, we can consider the following task types:

• Image Classification: Categorizes an entire input image into a pre-defined class.
• Object Detection: Localizes target objects with spatial bounding boxes while simultaneously predicting class labels.
• Semantic/Instance Segmentation: Assigns class labels to individual pixels, creating fine-grained boundary masks.

For our given dataset, the most sense would be to do image classification: the task requires distinguishing among 29 discrete hand shape classes from pre-cropped images.

❓ Why not object detection?
Hand regions are already centered within the frame, making bounding box detection redundant.

❓ Why not semantic/ instance segmentation?
It has high annotation cost and computational latency, the images are not annotated for this task.

Uploading to S3

After downloading the zipped archive from Kaggle, you will need to upload it to S3.

One way is to use the management console. But with such amount of files, it could take more than a day!

Much better option would be to use AWS CLI. (How to install/update AWS CLI?)

Then run the commands to configure the appropriate concurrency (considering you have the default profile):

aws configure set default.s3.max_concurrent_requests 50

(Maximum number of parallel requests allowed at once)

aws configure set default.s3.max_queue_size 10000

(Maximum number of tasks allowed in the transfer queue)

aws configure set default.s3.multipart_threshold 64MB

(File size threshold before switching to a multipart transfer)

Unzip the downloaded archive. To copy the files:

cd archive/asl_alphabet_train
aws s3 sync ./asl_alphabet_train s3://your-bucket/raw/asl_alphabet_train/

plaintext

So your data in the bucket is organized in the following way:

s3://YOUR-BUCKET-NAME/

raw/
└── asl_alphabet_train/
    ├── A/
    ├── B/
    └── ...

markdown

⚠️ Note: if any file failed to be uploaded, use this command to find which one:

aws s3 sync ./asl_alphabet_train s3://your-bucket/raw/asl_alphabet_train/ --dryrun

Because aws s3 sync compares your source and destination, the --dryrun output will print a list of only the files that have not been copied yet.

Create SageMaker notebook instance

Next, we need some compute to preprocess our data. One of the ways to do it on AWS is to create a SageMaker notebook instance.

Since we are not training any model yet (only processing 200x200 images one at a time), a bare CPU instance with additional disk space would be enough.

The ml.t3.medium general-purpose instance provides 2 vCPUs and 4 GiB of internal RAM. Because we are going to store the original dataset and its 2 preprocessed copies on the disk, we will need to add a 30-40 GB EBS volume.

I created an ml.t3.medium instance with a 50 GB EBS volume:

Pricing: t3.medium is one of the cheapest options. The exact price can vary depending on the region; see this page for on-demand pricing reference.

⚠️ Don't forget to stop and terminate your instance once the data is processed and uploaded to S3.

Let's preprocess our data

Once the notebook instance is ready, you can Open Jupyter / Open JupyterLab.

Upload the data_preprocessing.ipynb Jupyter notebook from the GitHub repo.

Run cell by cell. You might need to restart the Kernel after running pip install of libraries. (first cell)

⚠️ Don't forget to replace the placeholder and put your S3 bucket name:

BUCKET = "YOUR-BUCKET-NAME"

What it does (high-level):

1. Downloads the data from S3
2. Creates the train/validation/test split
3. Separately applies Gaussian filtering and Canny edge detection to each image
4. Uploads the preprocessed images to S3

Why do we need to preprocess data?

Gaussian filtering subtly softens the image, preserving color while smoothing high-frequency noise.
In our code, it is achieved using a simple function from the OpenCV package:

    filtered = cv2.GaussianBlur(
        image,
        GAUSSIAN_KERNEL,
        sigmaX=GAUSSIAN_SIGMA
    )

We are using (5, 5) as GAUSSIAN_KERNEL and 1.0 as GAUSSIAN_SIGMA. Adjust these values to regulate the amount of smoothing.

Learn more about Gaussian blurring.

Canny edge detection isolates the high-contrast skeletal boundaries of hand postures and finger configurations while discarding background textures and skin illumination gradients.
In our code, it is achieved in 2 steps. First, we convert the image to grayscale:

    gray = cv2.cvtColor(
        image,
        cv2.COLOR_BGR2GRAY
    )

Then, we use a simple function from the OpenCV package:

    edges = cv2.Canny(
        gray,
        threshold1=CANNY_LOW,
        threshold2=CANNY_HIGH
    )

We specified 100 and 200 as CANNY_LOW and CANNY_HIGH thresholds. These values help to decide which faint edges are real and which are just random noise, ensuring that true object boundaries stay connected without breaking up.

Learn more about Canny Edge Detection.

Here is an example of what a letter ' B ' would look like in a raw image, Gaussian-filtered image, and Canny edge image:

Preprocessed data and upload:

The split ratio is 70% for train, 15% for validation, and 15% for held-out test.

Our split sizes:

• train 60,900 images
• validation 13,050 images
• test 13,050 images

⚠️ Note: In some sources, the final held-out dataset is called 'validation'.
Here, we are using validation set for the model training/adjusting during multiple epochs.
The final held-out dataset is called 'test' (we will do the final metric evaluation against it).

Instead of physically moving raw data to separate folders during split, we use the CSV file to track which file corresponds to which split set.
This file maps every image to its three representations.

The file contains several columns: 'split', 'label', 'original_path', 'filtered_path', 'edge_path'.
Our code in the following part will read file paths from the CSV and take the corresponding file from the required split.

Here is what this CSV file looks like (truncated):

After running all the notebook cells, the data is uploaded to S3. Your bucket should have a new processed/ directory:

s3://YOUR-BUCKET-NAME/

processed/
├── filtered/
│   ├── train/
│   ├── validation/
│   └── test/
│
├── edges/
│   ├── train/
│   ├── validation/
│   └── test/
│
└── manifest.csv

Again, the full code in Jupyter notebook is available in the GitHub repo.

Conclusion

In this part, we uploaded the ASL dataset to S3 and created a SageMaker notebook instance. By running the Jupyter notebook, we applied Gaussian filtering and Canny edge detection processing to raw images. Finally, we uploaded processed images to S3.

We will keep all 3 image versions to compare which image representation would give the best model performance. In the next part, we will train a CNN from scratch. Stay tuned!