Data augmentation from scratch: turning one labelled image into an endless training set

Deep nets are greedy for data, and a small dataset invites overfitting — the model memorises exact pixels instead of learning the class, driving training accuracy to 100% while test accuracy stalls. The cleanest cure is more data, but labelling is slow and expensive. Data augmentation is the cheat: manufacture new, correctly-labelled examples from the ones you already have by applying transforms that change the pixels but not the label. A cat that is flipped, rotated, cropped, dimmed, or partly hidden is still a cat. I built the whole pipeline from scratch on an HTML5 canvas to see exactly how each transform works.

Two canvases, one pristine source

Draw the labelled image once onto an offscreen source canvas. Every augmentation reads from that pristine source and paints a fresh result elsewhere — the original is never mutated, so you can resample forever.

Geometric transforms are one matrix

Flips and rotations teach spatial invariance — the object is the same however it sits in the frame. Translate the origin to the centre first, then rotate, then scale, because a negative scale factor mirrors an axis:

ctx.translate(W / 2, H / 2);           // work about the centre
ctx.rotate(p.rot * Math.PI / 180);     // degrees → radians
ctx.scale(p.hflip ? -1 : 1,            // negative x = mirror horizontally
          p.vflip ? -1 : 1);           // negative y = mirror vertically

Doing it around the centre keeps the subject in frame instead of spinning off a corner.

Random-resized-crop: the strongest single augmentation

The nine-argument drawImage takes a source rectangle and stretches it over the whole destination. Shrink that rectangle (zoom in) and slide it to a random offset, and you have a random-resized-crop — the most effective augmentation for natural images, because the model learns to recognise the class from just an ear or a paw:

const sw = src.width  / p.zoom;        // zoom>1 → smaller source window
const sh = src.height / p.zoom;
const sx = (src.width  - sw) * p.offX; // offX,offY in [0,1] → random position
const sy = (src.height - sh) * p.offY;
ctx.drawImage(src, sx, sy, sw, sh, -W/2, -H/2, W, H);

Photometric jitter and noise

Colour changes teach the model to ignore lighting. Brightness and contrast come free from ctx.filter — set it before drawing and it bakes into the pixels. For per-pixel effects like sensor noise, pull the raw bytes with getImageData, add a Gaussian sample to each channel, and write them back:

ctx.filter = `brightness(${p.brightness}) contrast(${p.contrast})`;
drawCropped(ctx, p);
ctx.filter = "none";

function addNoise(ctx, sigma) {
  const img = ctx.getImageData(0, 0, W, H), d = img.data;
  for (let i = 0; i < d.length; i += 4) {
    const n = gauss() * sigma;              // Box–Muller: uniforms → normal
    d[i] += n; d[i+1] += n; d[i+2] += n;    // alpha untouched
  }
  ctx.putImageData(img, 0, 0);
}

Cutout: hide part of the image on purpose

Cutout drops a random square to a constant fill. Why help the model by hiding the input? Because it stops the network leaning on one giveaway feature — cover the ears at random and it must also learn the whiskers, so it spreads attention over the whole object and grows robust to real occlusion. The label is untouched: a partly hidden cat is still a cat.

Augmentation is a distribution, not a fixed edit

The key idea: each time you need a training example, sample fresh random parameters and run the whole pipeline. Same source, endless outputs — the model trains on a dataset far larger than the one you collected and never memorises a single frame:

function samplePolicy() {                  // one random draw = one variant
  return {
    hflip: Math.random() < 0.5,
    vflip: false,                          // an upside-down cat is rare
    rot:   rand(-25, 25),
    zoom:  rand(1.0, 1.8), offX: Math.random(), offY: Math.random(),
    brightness: rand(0.75, 1.25), contrast: rand(0.8, 1.2),
    sigma: rand(0, 22),
    cut:   Math.random() < 0.5 ? rand(0.15, 0.35) : 0,
  };
}

Every canvas call maps one-to-one onto a line of torchvision.transforms, which composes them into the training Dataset so a fresh variant is produced on every fetch.

The one rule that matters

Label-preserving is domain-specific, and this is the whole discipline. A horizontal flip is fine for a cat but turns a 6 into something else and mirrors text into gibberish; heavy rotation is fine for aerial photos but wrong for upright digits. Apply a transform only if a domain expert would still give the result the same label — and only if the result could plausibly appear at test time. And augment the training set only: the validation and test sets must stay clean, or you measure performance on distorted images instead of real ones.

Toggle the transforms, drag the sliders to morph one image, then hit Resample to watch a single picture explode into a batch of variants:
https://dev48v.infy.uk/dl/day35-data-augmentation.html