What I Learned Building a 97%-Accuracy Tumor Classifier (and Why Augmentation Mattered More Than the Model)

#computerscience #ai #deeplearning #python

I want to walk through the pipeline decisions including the ones that didn't work because that's the part that's actually useful to AI engineer.

I recently published my research in MDPI Technologies journal, a unified deep learning framework for multi-class tumor classification.

I designed and built the entire pipeline:

the data flow
the augmentation strategy
model selection, and ablation design.

We hit 97% accuracy across 10 classes on independent datasets. Here's how it was actually built, and the architectural choices that moved the needle.

The problem with a single dataset

Medical imaging whether we're talking about brain MRI or skin lesions, has a massive data problem.
Public datasets are small, heavily imbalanced across classes, and inconsistent in their conditions.
Train on one dataset, and you risk learning dataset specific pattern instead of the actual pathology.

To prove this pipeline actually understand the pattern, the study used 2 dataset design covering both skin and brain MRI data.
This was designed specifically to stress-test whether the pipeline generalized or if it was just over-fitting to one source's.

This is the first decision worth calling out: a single-dataset 97% is a much weaker claim than a dual-dataset 97%. It’s a cheap thing to add if you design your pipeline for it from day one.

Stage 1: A binary gate before the real problem

Before attempting a 10-class tumor classification, the pipeline runs a U-Net segmentation model as a binary gate: Is there a tumor region present at all, and where is it?

Why bother with this stage instead of feeding raw images straight into the classifier? Two reasons:

It forces the downstream classifier to focus on the relevant region instead of learning shortcuts from background or context pixels.
It's a natural checkpoint to catch garbage input early, before it costs you a wrong 10-way classification.

This is the exact same instinct behind a lot of production ML pipelines: don't let your hardest model see noisy input it doesn't need to handle.

Stage 2: The ablation that actually mattered (DCGAN vs. Augmentor)

This is the part I think is most useful to share, because it's where intuition and actual experimental results diverged.

The obvious move for a small, imbalanced medical imaging dataset is synthetic data generation. So, I trained a DCGAN to generate additional samples, balancing out the classes to about 7,000 images per class. But I also ran a much simpler baseline: classical augmentation via Augmentor (rotations, flips, elastic distortions, etc.) with no generative model at all.

Here is what actually happened: the Augmentor baseline did most of the heavy lifting. DCGAN helped marginally on the rarest classes, but the extra GAN training cost bought only a small edge over classical augmentation

This is a genuinely useful finding for anyone building in this space. Sometimes the "simple" baseline does more work than the sophisticated generative approach, and you only know that if you run the ablation.

Stage 3: The classifier, Fine-tuned Inception V3

The final 10-class classification stage used a fine-tuned Inception V3, which successfully reached 97% accuracy on our held-out test sets.

Rather than training a massive architecture from scratch, leveraging transfer learning with a frozen backbone and strategically unfreezing the head gave the model the exact feature extraction power it needed without immediately over-fitting to our 7k/class dataset.

What I'd do differently

it's usually the most skipped one. If I were to rebuild this pipeline from scratch, here is what I would look at closer:

The U-Net Bottleneck: Did the U-Net gate ever drop false negatives that never even got the chance to reach the classifier?
The Edge Cases: Where the dual-dataset generalization struggled the most, and which specific skin/brain classes got confused with each other.

the confusion matrix of the classifier model