DEV Community

dustin liang
dustin liang

Posted on

scipy savgol_filter in Python: Choosing window_length and polyorder Without Flattening the Peaks

scipy.signal.savgol_filter is the smoother people reach for when a moving average flattens the very peaks they care about. It fits a low-order polynomial to each window by least squares and keeps the fitted centre value, so short bumps keep most of their height while the noise is averaged away. The catch is that the result depends heavily on two numbers, window_length and polyorder, plus a third argument, mode, that decides what happens at the edges. Pick them badly and you get either a noisy line or a moving average in disguise.

This tutorial walks through savgol_filter in Python on a noisy daily series: how it compares with a rolling mean, how to choose window_length and polyorder with a small grid search, why the edges and missing values need special care, how to get a smooth slope with deriv=1, and why the default centred filter quietly uses future data. Every number below comes from running the code.

Code runs on SciPy 1.18, NumPy 2.5 and pandas 3.1. The data is synthetic and seeded: a slow seasonal trend, one narrow event about a week wide, one broad event about a month wide, and Gaussian noise. Think of it as a daily attention or activity index; because we generated it, we know the clean signal and can measure errors exactly.

A noisy daily series with a known clean signal

import numpy as np
import pandas as pd
from scipy.signal import savgol_filter

rng = np.random.default_rng(7)
n = 180
t = np.arange(n)
dates = pd.bdate_range("2026-01-05", periods=n)
trend = 40 + 10 * np.sin(2 * np.pi * t / 120)
bump1 = 30 * np.exp(-0.5 * ((t - 60) / 4.0) ** 2)   # narrow event, ~1 week wide
bump2 = 18 * np.exp(-0.5 * ((t - 130) / 12.0) ** 2) # broad event, ~1 month wide
clean = trend + bump1 + bump2
noisy = clean + rng.normal(0, 4, n)
s = pd.Series(noisy, index=dates, name="heat")
print(s.describe().round(2))
print("true peak 1:", round(clean[60], 2), " true peak 2:", round(clean[130], 2))
Enter fullscreen mode Exit fullscreen mode
count    180.00
mean      46.09
std       10.60
min       23.59
25%       38.85
50%       46.50
75%       52.62
max       73.27
Name: heat, dtype: float64
true peak 1: 70.0  true peak 2: 63.0
Enter fullscreen mode Exit fullscreen mode

The narrow event peaks at 70 and the broad one at 63. The raw series wobbles by about 4 points per day around that, so the question is how much of each peak a smoother keeps.

savgol_filter vs rolling mean

We compare a centred rolling mean with savgol_filter at the same window length. Besides the error against the clean signal (rmse_vs_clean), we record the smoothed height of both peaks and a simple roughness measure: the standard deviation of day-to-day changes.

def compare(win, poly=None):
    rm = s.rolling(win, center=True, min_periods=1).mean().to_numpy()
    out = {"rolling_mean": rm}
    for p in ([poly] if poly is not None else [2, 3]):
        out[f"savgol_p{p}"] = savgol_filter(s.to_numpy(), win, p)
    rows = []
    for name, y in out.items():
        rows.append({
            "method": name, "window": win,
            "peak1": round(y[55:66].max(), 2),
            "peak2": round(y[120:141].max(), 2),
            "rmse_vs_clean": round(np.sqrt(np.mean((y - clean) ** 2)), 2),
            "roughness": round(np.std(np.diff(y)), 2),
        })
    return pd.DataFrame(rows)

print("raw roughness:", round(np.std(np.diff(noisy)), 2))
print(pd.concat([compare(11), compare(21)], ignore_index=True).to_string(index=False))
Enter fullscreen mode Exit fullscreen mode
raw roughness: 5.33
      method  window  peak1  peak2  rmse_vs_clean  roughness
rolling_mean      11  62.89  63.32           1.82       1.01
   savgol_p2      11  70.03  65.27           1.67       1.42
   savgol_p3      11  70.03  65.27           1.68       1.43
rolling_mean      21  56.17  61.09           3.39       0.68
   savgol_p2      21  65.02  64.39           1.60       1.06
   savgol_p3      21  65.02  64.39           1.61       1.05
Enter fullscreen mode Exit fullscreen mode

With an 11-day window the rolling mean cuts the narrow peak from 70 to about 63; savgol keeps it at 70. At 21 days the rolling mean drops it to about 56 while savgol still reaches 65. The price is roughness: savgol output is less smooth than a rolling mean of the same width (1.42 vs 1.01 at 11 days). That is the core trade-off. A moving average is the best possible noise reducer for a flat signal, and the worst choice for a signal with peaks.

Notice also that polyorder=2 and polyorder=3 give identical numbers. That is not a bug: for the centre point of a symmetric window, an even order 2k and the next odd order 2k+1 produce the same smoothing coefficients. If you are smoothing (not differentiating), only try even orders, or only odd ones; testing both wastes time.

Choosing window_length and polyorder

There is no universal rule, but with a known clean signal (or a hand-labelled sample of real data) a small grid makes the trade-off visible. The first table is the overall RMSE; the second is how far the narrow peak is under- or over-estimated.

grid = []
for win in [7, 11, 15, 21, 31, 41]:
    for poly in [1, 2, 3, 4]:
        if poly >= win:
            continue
        y = savgol_filter(noisy, win, poly)
        grid.append({"window": win, "poly": poly,
                     "rmse": np.sqrt(np.mean((y - clean) ** 2)),
                     "peak1_err": y[55:66].max() - clean[60]})
g = pd.DataFrame(grid)
print(g.pivot(index="window", columns="poly", values="rmse").round(2))
print()
print(g.pivot(index="window", columns="poly", values="peak1_err").round(2))
Enter fullscreen mode Exit fullscreen mode
poly       1     2     3     4
window                        
7       1.57  2.03  2.03  2.59
11      1.83  1.67  1.68  1.99
15      2.42  1.51  1.54  1.76
21      3.38  1.60  1.61  1.56
31      4.56  2.47  2.47  1.57
41      5.40  3.30  3.29  2.16

poly        1      2      3     4
window                           
7       -2.78   0.55   0.55  0.72
11      -7.11   0.03   0.03  0.69
15     -10.45  -2.15  -2.15  0.61
21     -13.83  -4.98  -4.98 -0.83
31     -17.80  -9.37  -9.37 -3.58
41     -20.50 -13.21 -13.21 -7.50
Enter fullscreen mode Exit fullscreen mode

Three patterns are worth remembering:

  • polyorder=1 is a moving average. A straight-line fit evaluated at the window centre is just the window mean, so its peak error grows quickly with the window.
  • Higher order lets you use a longer window. The lowest RMSE for poly 2/3 is at window 15 (1.51); for poly 4 it moves to windows 21–31 (about 1.56) with much less damage to the narrow peak (-0.83 at window 21 against -4.98 for poly 2).
  • Window length should match the width of the features you want to keep. Our narrow event is about a week wide; once the window passes roughly two to three times that width, even poly 4 starts eating it.

A practical starting point for daily data: window_length around the width of the shortest feature you care about (odd, and larger than polyorder), polyorder 2 or 3, then check peak heights on a few known events before trusting it.

What the coefficients tell you

savgol_filter is a fixed linear filter: each output is a weighted sum of the window. savgol_coeffs shows the weights. The sum of squared weights tells you what fraction of white-noise variance survives.

from scipy.signal import savgol_coeffs
print("smoothing coeffs, window=7, poly=2:")
print(savgol_coeffs(7, 2).round(4))
print("noise variance kept, by (window, poly):")
for win, poly in [(11, 2), (21, 2), (21, 4), (41, 4)]:
    c = savgol_coeffs(win, poly)
    print(f"  ({win:2d},{poly}) -> {np.sum(c**2):.3f}   moving average -> {1/win:.3f}")
Enter fullscreen mode Exit fullscreen mode
smoothing coeffs, window=7, poly=2:
[-0.0952  0.1429  0.2857  0.3333  0.2857  0.1429 -0.0952]
noise variance kept, by (window, poly):
  (11,2) -> 0.207   moving average -> 0.091
  (21,2) -> 0.108   moving average -> 0.048
  (21,4) -> 0.169   moving average -> 0.048
  (41,4) -> 0.086   moving average -> 0.024
Enter fullscreen mode Exit fullscreen mode

The negative weights at the edges of the window are what let the filter keep peaks: it subtracts some of the neighbours to undo the flattening a plain average would cause. They are also why savgol keeps more noise than a moving average of the same length (0.207 vs 0.091 of the variance at window 11). If you need both peak preservation and very low noise, increase the window and the order together rather than stacking two filters.

Edge handling: the mode argument

Near the start and end of the series there is no full window. mode decides how the missing side is filled.

for mode in ["interp", "mirror", "nearest", "constant", "wrap"]:
    y = savgol_filter(noisy, 21, 3, mode=mode)
    edge = np.r_[y[:5] - clean[:5], y[-5:] - clean[-5:]]
    print(f"{mode:9s} first={y[0]:6.2f} last={y[-1]:6.2f}  edge_rmse={np.sqrt(np.mean(edge**2)):5.2f}")
print("clean     first=", round(clean[0], 2), " last=", round(clean[-1], 2))
Enter fullscreen mode Exit fullscreen mode
interp    first= 39.79 last= 41.25  edge_rmse= 0.77
mirror    first= 39.79 last= 40.94  edge_rmse= 0.63
nearest   first= 39.90 last= 40.67  edge_rmse= 0.51
constant  first= 22.05 last= 22.64  edge_rmse=11.63
wrap      first= 40.32 last= 40.40  edge_rmse= 0.60
clean     first= 40.0  last= 40.53
Enter fullscreen mode Exit fullscreen mode

The default interp fits a polynomial to the last full window and evaluates it at the edge points, which works well here. mirror and nearest are also reasonable for a series that is roughly flat at its ends. constant pads with zeros (cval=0.0), which drags both ends down to about 22, a classic source of fake "drops" on the last day of a chart. wrap treats the series as periodic, which is only right if the end really connects to the start. For daily business data, stick to interp or nearest, and never constant unless your padding value is meaningful.

Missing values: savgol_filter spreads NaN

savgol_filter does not skip NaN. Every window that contains a missing value returns NaN.

s_gap = s.copy()
s_gap.iloc[[40, 41, 95]] = np.nan
y_nan = savgol_filter(s_gap.to_numpy(), 21, 3)
print("3 missing inputs -> NaN outputs:", int(np.isnan(y_nan).sum()))

filled = s_gap.interpolate(limit=3)
mask = s_gap.isna()
y_fill = pd.Series(savgol_filter(filled.to_numpy(), 21, 3), index=s.index)
y_fill[mask] = np.nan
print("NaN outputs after fill + re-mask:", int(y_fill.isna().sum()))
Enter fullscreen mode Exit fullscreen mode
3 missing inputs -> NaN outputs: 43
NaN outputs after fill + re-mask: 3
Enter fullscreen mode Exit fullscreen mode

Three missing days turned into 43 missing outputs, because each gap poisons a full 21-day window around it. The usual fix: interpolate short gaps with a limit, smooth, then put NaN back where the input was missing so you do not present invented values as data. For long gaps, smooth each continuous segment separately instead of interpolating across the hole.

Smoothed slope with deriv=1

Because savgol fits a polynomial, it can return the polynomial's derivative directly. This is much better than differencing a noisy series.

slope = savgol_filter(noisy, 21, 3, deriv=1)          # units: heat per day
naive = np.gradient(noisy)
true_slope = np.gradient(clean)
for name, d in [("np.gradient(raw)", naive), ("savgol deriv=1", slope)]:
    print(f"{name:17s} rmse vs true slope = {np.sqrt(np.mean((d - true_slope) ** 2)):.3f}")
print("true peak-1 day:", s.index[60].date())
rising = pd.Series(slope > 0.5, index=s.index)
runs = (rising != rising.shift()).cumsum()
spans = rising.groupby(runs).agg(["first", "size"])
print(spans[spans["first"]].assign(start=lambda x: [s.index[runs == k][0].date() for k in x.index])[["start", "size"]].to_string())
Enter fullscreen mode Exit fullscreen mode
np.gradient(raw)  rmse vs true slope = 2.483
savgol deriv=1    rmse vs true slope = 0.406
true peak-1 day: 2026-03-30
        start  size
2  2026-01-09     7
4  2026-03-13    11
6  2026-05-08     1
8  2026-05-25    33
Enter fullscreen mode Exit fullscreen mode

The savgol slope is about six times closer to the true slope than np.gradient on raw data. Thresholding it (here: rising by more than 0.5 per day) gives clean "rising" segments: the run starting 13 March covers the climb into the narrow event that peaks on 30 March, and the long run from late May is the broad event plus the seasonal upswing. The one-day run on 8 May shows that a threshold still needs a minimum duration filter in practice.

The look-ahead problem: centred vs real-time smoothing

The default savgol_filter is centred: the smoothed value for a day uses 10 days before and 10 days after it. That is fine for a chart of history and wrong for anything that must decide in real time, such as an alert or a backtest. The honest real-time version uses only the window that ends today and keeps the last fitted point.

win, poly = 21, 3
centered = savgol_filter(noisy, win, poly)

def last_point_savgol(x, win, poly):
    out = np.full(len(x), np.nan)
    for i in range(win - 1, len(x)):
        out[i] = savgol_filter(x[i - win + 1:i + 1], win, poly, mode="interp")[-1]
    return out

realtime = last_point_savgol(noisy, win, poly)
ok = ~np.isnan(realtime)
print("centered rmse vs clean :", round(np.sqrt(np.mean((centered[ok] - clean[ok]) ** 2)), 2))
print("real-time rmse vs clean:", round(np.sqrt(np.mean((realtime[ok] - clean[ok]) ** 2)), 2))
print("max revision of a day's value once future data arrives:",
      round(np.nanmax(np.abs(centered - realtime)), 2))
for name, y in [("clean", clean), ("centered", centered), ("real-time", realtime)]:
    first = 45 + int(np.argmax(y[45:80] > 60))
    print(f"{name:9s} first day above 60 near event 1: {s.index[first].date()}")
print("real-time roughness:", round(np.nanstd(np.diff(realtime)), 2),
      " centered roughness:", round(np.std(np.diff(centered)), 2))
Enter fullscreen mode Exit fullscreen mode
centered rmse vs clean : 1.61
real-time rmse vs clean: 3.0
max revision of a day's value once future data arrives: 9.61
clean     first day above 60 near event 1: 2026-03-24
centered  first day above 60 near event 1: 2026-03-24
real-time first day above 60 near event 1: 2026-03-25
real-time roughness: 2.83  centered roughness: 1.05
Enter fullscreen mode Exit fullscreen mode

Two things stand out. First, the centred value for a day can change by almost 10 points once future data arrives, so a centred smoother on a live dashboard will revise its recent history every day. Second, the real-time estimate is noisier (RMSE 3.0 vs 1.61, roughness 2.83 vs 1.05) because the end of a polynomial fit is its least stable point. It is not much slower here (it crossed 60 one day after the clean signal), but it needs a lower polyorder or a longer window if you want it smooth. In a backtest, using the centred output is a look-ahead bug even though no code explicitly "peeks".

FAQ

What does polyorder mean in savgol_filter?
It is the degree of the polynomial fitted to each window. It must be less than window_length. Order 1 equals a moving average; orders 2–4 are typical for smoothing.

Does window_length have to be odd?
Not in recent SciPy versions (an even length runs fine on SciPy 1.18), but an odd length keeps the window symmetric around the output point, which is what most people expect. Use odd lengths unless you have a reason not to.

Why are polyorder 2 and 3 giving the same result?
For the centre point of a symmetric window, orders 2k and 2k+1 share the same smoothing coefficients. They differ for derivatives and for edge points.

Is savgol_filter causal?
No. It is centred by default and uses future samples. For real-time use, fit the window that ends at the current point and keep the last value, as in the last section.

savgol_filter or rolling mean?
Rolling mean for maximum noise reduction on slow signals; savgol_filter when short peaks or turning points must keep their height and timing.

Conclusion

savgol_filter keeps peaks that a rolling mean flattens, at the cost of passing a bit more noise. Choose window_length from the width of the features you care about, use polyorder 2–4 (even and odd orders smooth identically), keep mode="interp" or "nearest" at the edges, fill short gaps before smoothing and re-mask them afterwards, and use deriv=1 for a clean slope. Above all, remember that the default filter is centred: great for describing history, a look-ahead bug for anything that has to work on today's data.

For a real example of a daily sector attention series where peak height and timing matter, Lava's AI sector heat monitor tracks news-flow heat for the AI concept board day by day.

中文简介:本文用一条带固定种子的合成日度序列(已知真实信号)讲解 scipy.signal.savgol_filter:与同窗口滚动均值对比,savgol 能保住窄峰高度但保留更多噪声;用网格搜索展示 window_length 与 polyorder 的取舍(polyorder=1 等于移动平均,偶数阶与下一个奇数阶平滑结果相同);用 savgol_coeffs 解释负权重与噪声方差;比较 mode 边界处理(constant 会把首尾拉低);说明 NaN 会扩散到整个窗口及“插值—平滑—回填缺失”的做法;用 deriv=1 求平滑斜率;最后指出默认居中滤波会用到未来数据,并给出只用当日及之前数据的实时版本及其误差代价。

Top comments (0)