Forty shops. One has 860 ratings. Five have exactly one.
Shop #0 has a single rating, and that rating says 80.43. It sits at the top of the leaderboard. Its true quality is 57.61 — above average, but nowhere near the top of anything.
Partial pooling estimates it at 53.12, and across all forty shops it cuts total squared error from 1763.35 to 332.54.
👉 Live, with the shrinkage weight recomputed as you drag the sample size: https://dev48.in/ml/day83-partial-pooling.html
The data is synthetic and seeded
Every number here comes from a seeded generator: 40 groups, sizes 1, 1, 1, 1, 1, 2, … 688, 860 for 4,345 observations, true group means drawn from Normal(50, 4²) and observations from Normal(θ_g, 12²). That is the entire point of synthesising it — the true group means are known exactly, so every estimator is scored against the truth rather than against another estimate. Nothing on the page is fitted to real shop data and none of these figures are claims about real shops.
Three estimators, scored against truth
| estimator | what it does | total squared error |
|---|---|---|
| no pooling | each group's own mean | 1763.35 |
| complete pooling | one global mean for all 40 | 794.38 |
| partial pooling | shrink each toward the mean | 332.54 |
5.30x better than trusting every group, 2.39x better than trusting none of them. Nothing told it which groups were small.
The weight is a ratio of two variances, not a tuning knob
lambda_g = tau^2 / (tau^2 + sigma^2 / n_g)
estimate = lambda_g * ybar_g + (1 - lambda_g) * mu
sigma²/n is how much of a group's average is sampling noise. tau² is how much real spread there is between groups. lambda is the share of what the group said that is signal. It is bounded in [0,1] by construction, because it is a ratio whose denominator contains its numerator.
At sigma = 12:
| spread | n = 1 | n = 3 | n = 10 | n = 30 | n = 100 | n = 860 |
|---|---|---|---|---|---|---|
| τ = 1 | 0.0069 | 0.0204 | 0.0649 | 0.1724 | 0.4098 | 0.8566 |
| τ = 4 | 0.1000 | 0.2500 | 0.5263 | 0.7692 | 0.9174 | 0.9896 |
| τ = 16 | 0.6400 | 0.8421 | 0.9467 | 0.9816 | 0.9944 | 0.9993 |
Complete pooling is this method at lambda = 0. No pooling is this method at lambda = 1. They are not rivals of it, they are its two endpoints — and notice there is no threshold anywhere. No group is ever dropped for being "too small"; it is continuously discounted in proportion to how much of its number is noise.
Both variances have to be estimated, and one of them is hard
sigma² is easy: pool the within-group deviations over N − G = 4345 − 40 = 4305 degrees of freedom. The one-rating shops contribute nothing to it, which is correct — a single point has no spread.
tau² is the hard one, because you never see the true means. The naive answer, the spread of the 40 observed group averages, is 83.658 against a truth of 16 — a 5.23x overstatement, because the one-rating shops are scattered by ±12 points of pure noise. Use that and lambda goes to nearly 1 and you shrink nothing.
The moment identity fixes it. The precision-weighted scatter Q of the group averages has expectation G − 1 = 39 if the groups are truly identical, so the excess is real spread:
| quantity | value |
|---|---|
| Q observed | 421.61 |
| E[Q] if all groups identical | 39 |
| τ̂² recovered | 14.697 (true 16.000) |
| σ̂² recovered | 148.689 (true 144.000) |
Over 300 seeded worlds the estimates average 144.069 and 16.124 against truths of 144 and 16, and τ̂² was clamped to zero in 0 of the 300. The clamp is not decoration — a sample can genuinely look tighter than chance predicts, and the honest answer there is τ̂² = 0, which collapses the whole method to complete pooling.
One group, sample size varied, truth held fixed
The true score of shop #0 is 57.61 in every row — the true means come from their own seeded stream, so changing the sample size cannot disturb them. Only the evidence changes.
| ratings | λ | raw average | partial pooled | pulled by |
|---|---|---|---|---|
| 1 | 0.0900 | 80.43 | 53.12 | 27.31 |
| 4 | 0.2855 | 63.67 | 54.24 | 9.44 |
| 16 | 0.6137 | 59.23 | 55.87 | 3.36 |
| 128 | 0.9309 | 59.51 | 58.89 | 0.62 |
| 2000 | 0.9968 | 57.78 | 57.75 | 0.02 |
It is not a hack — it is the estimator with lower expected error
Over 300 independently seeded worlds:
| estimator | mean total squared error | best world | worst world |
|---|---|---|---|
| no pooling | 1421.02 | 337.75 | 2977.61 |
| complete pooling | 660.40 | 312.53 | 1280.33 |
| partial pooling | 290.45 | 96.52 | 507.22 |
Partial pooling beat both at once in 300 of 300. Its worst world (507.22) is better than no pooling's best one (337.75).
That is the James–Stein result in applied form. For a normal mean in three or more dimensions, using each group's own average — unbiased, maximum likelihood, the obvious thing — is inadmissible: a shrunken estimator has lower expected total squared error for every possible set of true means. On 30 groups of 5, the classical James–Stein estimator scores 350.76 against the MLE's 806.59, a 2.30x improvement, winning 58 of 60 seeded worlds. Not 60 of 60 — inadmissibility is a statement about the expectation, and a page claiming otherwise would be lying to you.
Error moved out of variance and into bias, deliberately
Hold the forty true means fixed and redraw only the observations, 400 times. Now "bias" means something.
| estimator, group | n | bias | bias² | variance | MSE |
|---|---|---|---|---|---|
| no pooling, shop #0 | 1 | −0.107 | 0.011 | 143.386 | 143.398 |
| partial, shop #0 | 1 | −7.029 | 49.403 | 1.071 | 50.473 |
| no pooling, shop #39 | 860 | −0.022 | 0.000 | 0.166 | 0.167 |
| partial, shop #39 | 860 | 0.021 | 0.000 | 0.164 | 0.164 |
The one-rating group's no-pooling estimate is almost perfectly unbiased and almost perfectly useless — its error is essentially all variance. Partial pooling takes on a real, systematic, never-averaging-away pull of −7.03 toward the middle, and the variance collapses by 134x. Total error falls 2.84x. The 860-rating group is left alone.
Being unbiased was never the objective. Being close was.
The leaderboard lesson, measured
Pick the single top shop, 200 seeded worlds:
| ranking | true score of the winner | names the genuinely best shop | groups with ≤3 ratings in its top 5 |
|---|---|---|---|
| by raw average | 53.76 | 32 / 200 | 2.87 of 5 |
| by partial pooling | 57.12 | 78 / 200 | 0.29 of 5 |
| best possible | 58.77 | — | — |
Twelve of the forty groups have three ratings or fewer. The raw top five holds on average 2.87 of them — not because small groups are good, but because small groups have the widest distribution of outcomes and a maximum picks from the tails.
What this costs
14 of 40 individual groups come out worse. Shop #11 has 3 ratings and a raw error of 0.157; partial pooling moves it to 4.052. The method optimises the sum, and individual groups pay for it.
So if your decision is about one named group — a safety alert, a fraud case, a single hospital's mortality rate — the shrunken number is the wrong one to argue from. And the pooling assumption has to hold: it says the groups are exchangeable draws from one population, and shrinking toward a mean that no group belongs to makes things worse, not better.
What this does not cover
One normal-normal model with a scalar outcome, no predictors, no crossed or nested levels, and no full Bayes — tau² is a point estimate here and the uncertainty in it is never propagated. Group sizes vary but the within-group variance is assumed common across groups. Outcomes are Gaussian, so nothing here is a binomial or count hierarchy.
37 in-page assertions, 56 verifier assertions, 0 failures. The verifier shares none of the page's machinery: a different generator, and the shrinkage weight recovered three other ways — trapezoid quadrature over the Bayes posterior, a golden-section minimiser on the penalised least-squares objective, and a grid search over λ against empirical mean squared error on 300,000 draws.
Top comments (0)