An autoencoder detects anomalies by being bad at them. Train it to reproduce normal windows through a bottleneck and it will reproduce normal windows well and abnormal ones poorly, so the reconstruction error is the score. The whole difficulty is turning that score into a threshold, and most published advice on the threshold is wrong by an order of magnitude.
What the model is actually trained to do
The training objective is reconstruction: encode a window of the series into a representation smaller than the window, decode it back, and minimise the squared difference. Because the representation is smaller than the input, the model cannot memorise; it must spend its capacity on whatever structure is most common in the training data. Feed it only normal data and the common structure is normality.
Three design decisions matter more than the architecture. The window length must be long enough to contain the pattern you consider normal — if the daily cycle is the structure, a window shorter than a day cannot encode it and the model will learn the local level instead. The bottleneck must be tight enough to prevent identity: an autoencoder with a latent dimension close to the input dimension learns to copy, reconstructs anomalies perfectly, and produces a flat error series that detects nothing. And the training data must actually be clean, because any anomaly present in training is a pattern the model learns to reconstruct and will not flag later.
That last requirement is the awkward one. The method is usually sold as unsupervised and it is not: it is semi-supervised, requiring a period you are willing to declare normal. Where no such period exists, contamination at a low rate is survivable — the model spends little capacity on rare patterns — but it degrades quietly rather than failing, which is worse.
From reconstruction to a per-timestamp error
Windows overlap, so a given timestamp appears in many windows and has many reconstruction errors. Collapsing them is a choice with consequences: the mean over windows is smooth and delays detection at the start of an event, the maximum is responsive and noisy. Taking the error only from the last position of each window makes it usable online, at the cost of never getting the benefit of the model seeing the timestamp in context.
For multivariate series there is a further step, and the standard treatment is Malhotra and colleagues’ 2016 encoder-decoder paper: collect the per-channel errors at each timestamp into an error vector, fit a distribution to those vectors on a normal validation set, and score each timestamp by its distance under that distribution rather than by a per-channel comparison. That matters because channels have different error scales and are correlated, and a Euclidean norm over raw errors is dominated by whichever channel happens to be noisiest.
Malhotra et al., LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection, arXiv:1607.00148
Deriving the threshold from the distribution
The threshold should come from the empirical distribution of errors on held-out normal data, and the useful way to choose it is by the alert volume it implies rather than by the percentile it names. Suppose a clean validation set of 10,000 windows produces this error distribution. These figures are an illustrative validation set, not a measurement; the arithmetic on them is the thing to keep.
assumed empirical quantiles of reconstruction error on clean data
p50 0.021 p99 0.061
p90 0.038 p99.9 0.092
p95 0.045 max 0.140
data arrives every 5 minutes -> 288 windows per series per day
you monitor 200 series
threshold at p99 : 0.01 x 288 = 2.88 alerts/series/day -> 576/day total
threshold at p99.9 : 0.001 x 288 = 0.29 alerts/series/day -> 58/day total
threshold at max : 0 -> 0/day, and
nothing detected
until it exceeds
anything ever seen
Five hundred and seventy-six alerts a day is not a monitoring system, it is a filter somebody will turn off. Fifty-eight is still more than most teams will look at. That is the real content of the threshold decision, and it is arithmetic rather than statistics: the false alarm rate is multiplied by the sampling rate and by the number of series, and both multipliers are usually large. Two consequences follow. Deduplicate consecutive alerts into events before counting, which typically cuts the volume by the average event length. And set the threshold per series rather than globally, because a single threshold across 200 series with different error scales concentrates almost all alerts on the few noisiest ones.
Why mean plus three sigma is not the 99.87th percentile
The common rule is to set the threshold at the mean error plus three standard deviations, on the understanding that this is the 99.87th percentile and so about one alert in 750. That inference requires the errors to be normally distributed. Reconstruction errors are a sum of squares, bounded below by zero and with a long right tail, so they are not; a lognormal is much closer.
Work it through on the distribution above. Fit a lognormal to the median of 0.021 and the 99th percentile of 0.061:
sigma_log = ln(0.061 / 0.021) / 2.326 = ln(2.905) / 2.326 = 1.066 / 2.326 = 0.4585
mean = 0.021 x exp(0.4585^2 / 2) = 0.021 x 1.1108 = 0.02333
sd = 0.02333 x sqrt(exp(0.4585^2) - 1) = 0.02333 x 0.4837 = 0.01129
mean + 3 sd = 0.02333 + 0.03386 = 0.05718
what percentile is 0.05718?
z = ln(0.05718 / 0.021) / 0.4585 = 1.0016 / 0.4585 = 2.184
Phi(2.184) = 0.9855 -> 1.45% of normal windows exceed it
The rule promised 0.13 per cent and delivers 1.45 per cent, roughly eleven times the alert volume expected, before multiplying by 288 windows a day and 200 series. Nothing here is exotic: it is what happens when a symmetric rule is applied to a skewed distribution, and it is why the empirical quantile is the right tool. If you want a parametric threshold, take logs first and apply the rule there, which at least matches the shape of the data.
What you still need labels for
The threshold above controls false alarms and says nothing about detection. You cannot know whether 0.061 catches the faults you care about without some labelled examples, and the honest position is that a handful of labelled incidents is worth more than any amount of threshold theory. With even a small labelled set you can choose the threshold to maximise an F-score weighted toward recall, which is the approach Malhotra and colleagues take, and you can report a detection rate rather than a percentile.
- Score by event, not by timestamp. A fault lasting two hours contributes 24 five-minute windows. Counting those as 24 detections inflates recall and counting a single missed window as a miss deflates it; the unit that matters is the event.
- Allow a detection delay. A detection is normally counted as correct if it falls anywhere within the event, or within a stated tolerance of its start. State which convention you used, because the two produce noticeably different numbers on the same detector.
- Watch for the model relearning the anomaly. If you retrain periodically on recent data, a persistent fault enters the training set and stops being anomalous. Retraining windows should exclude confirmed incident periods.
- Compare against something simple. A rolling z-score or an Isolation Forest on engineered features is a genuine competitor here and costs a fraction of the infrastructure. An autoencoder earns its place when the normal pattern is a multivariate shape rather than a range, and not automatically.
Top comments (0)