A histogram is the crudest density estimate there is — drop a smooth bump on every point, sum them, and bandwidth is everything

A histogram is the first thing anyone reaches for to see the shape of some data, and it's also the crudest density estimate there is. You choose a bin width and an origin, drop every point into a box, and count — and the picture depends on choices that have nothing to do with the data. Shift the bins half a width and a bump can appear or vanish; the estimate is a staircase that jumps at every edge even though the truth is smooth. Kernel density estimation keeps the histogram's one good instinct — let each point vote for density nearby — and removes the arbitrariness. I built it from scratch, live, in about a dozen lines of JavaScript.

Sum a smooth bump per point

Instead of a hard box, every point contributes a smooth kernel K((x−xᵢ)/h). Add the bumps up and divide by n·h and out comes a continuous density that integrates to exactly 1 — a real probability density that no longer cares where you put bin edges. That's the whole method.

const KERNELS = {
  gaussian:     u => Math.exp(-0.5*u*u) / Math.sqrt(2*Math.PI),  // ∞ support
  epanechnikov: u => Math.abs(u) < 1 ? 0.75*(1 - u*u) : 0,       // MSE-optimal
  tophat:       u => Math.abs(u) < 1 ? 0.5 : 0                   // moving box
};
function kde(x, samples, h, K){
  let s = 0;
  for (const xi of samples) s += K((x - xi) / h);
  return s / (samples.length * h);   // f̂(x) = 1/(nh) Σ K((x−xᵢ)/h)
}

The kernel shape barely matters — Gaussian, Epanechnikov and tophat all land in nearly the same place. The 1/h keeps each bump's area at 1/n, so the total is a genuine density. To draw the curve, evaluate kde at a few hundred x-values across the range and connect the dots.

The bandwidth is the bias-variance dial

The one knob that matters is h, and it's the entire story. Make it small and each bump is a spike — the curve overfits the sample, high variance, a wiggly mess that mistakes noise for structure. Make it large and everything smears into one oversmoothed blob — high bias, real peaks washed away. Setting h is the bias-variance trade-off made literal, and there are two standard ways to pick it.

Silverman's rule — a bandwidth for free

The fast default. Assuming the data is roughly Gaussian, the bandwidth minimising mean integrated squared error is a closed form. Using the interquartile range instead of the standard deviation makes it robust to outliers and mild multimodality.

// h = 0.9 · min(σ, IQR/1.34) · n^(−1/5)
const A = iqr > 0 ? Math.min(sd, iqr/1.349) : sd;
const h = 0.9 * A * Math.pow(n, -1/5);

Leave-one-out CV — let the data pick h

When Silverman's Gaussian assumption is wrong — bimodal, skewed — cross-validate. For a candidate h, hold out each point, estimate its density from the rest, and sum the log-densities. Too small starves the held-out points; too large blurs them; the peak in between is the bandwidth the data itself prefers, with no ground truth needed. Grid-search h and take the argmax.

function loocvLogLik(x, h, K){
  let ll = 0;
  for (let i = 0; i < x.length; i++){
    let s = 0;
    for (let j = 0; j < x.length; j++) if (j !== i) s += K((x[i]-x[j])/h);
    ll += Math.log(s / ((x.length-1)*h) + 1e-12);   // density at held-out point
  }
  return ll;
}

It generalises up a dimension — then it breaks

The same sum works in 2-D with a product kernel, K((x−xᵢ)/h)·K((y−yᵢ)/h), rendering density as a heatmap. But that's also where it ends: in d dimensions you need exponentially more points to fill space, so a fixed-width kernel runs out of neighbours and KDE degrades badly past ~5–6 dims — the curse of dimensionality. It's a superb 1-D/2-D tool and a poor high-dimensional one; reach for a parametric model or dimensionality reduction there.

So the trade is assumptions vs data hunger. A single fitted Gaussian needs almost no data but is wrong the moment the truth has two peaks. A histogram assumes nothing about shape but pays with discontinuity and fragile bin edges. KDE is the smooth, bin-free middle ground, collapsing all the arbitrariness into one continuous knob. Click to scatter points, slide h to feel spiky-vs-blurry, and hunt the sweet spot with cross-validation:

https://dev48v.infy.uk/ml/day50-kernel-density-estimation.html