The One-Line Summary:
RandomForestClassifierdefaultsmax_featuresto'sqrt'andRandomForestRegressordefaults it to1.0, and the common advice to "fix" the regressor by copying'sqrt'across is the one move the numbers actually punish — across three dataset shapes the default won on accuracy every time,'sqrt'lost by up to 21 points, and what the default really costs you is roughly 2x the fit time and the highest tree correlation of any setting.
Two Defaults, One Library
Check the constructor signatures rather than taking my word for it:
import inspect, sklearn
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
for cls in (RandomForestClassifier, RandomForestRegressor):
d = inspect.signature(cls.__init__).parameters['max_features'].default
print(f"{cls.__name__:<24} max_features={d!r}")
print("sklearn", sklearn.__version__)
RandomForestClassifier max_features='sqrt'
RandomForestRegressor max_features=1.0
sklearn 1.7.2
So the regressor considers every column at every split. Day one established why that matters: feature subsampling is the knob that decorrelates the trees, and correlation is the floor on the variance of an average. The regressor ships with that knob wide open — which is to say, as plain bagged trees.
The obvious conclusion is that you should close it. That conclusion is wrong, and it took measuring to find out.
The Measurement
Three dataset shapes, four settings, five seeds each, averaged. Correlation is the mean pairwise correlation between individual tree predictions on held-out rows.
MAX_FEATURES ON REGRESSION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
A: p=20, 6 informative, noise=12
setting R2 rho fit s beats dflt
1.0 0.8723 0.696 0.48s -
0.5 0.8708 0.627 0.28s 1/5
0.33 0.8473 0.515 0.22s 0/5
sqrt 0.8036 0.402 0.20s 0/5
B: p=60, 5 informative, noise=8 (sparse signal)
1.0 0.8940 0.745 1.17s -
0.5 0.8855 0.663 0.64s 1/5
0.33 0.8562 0.542 0.48s 0/5
sqrt 0.6831 0.248 0.27s 0/5 <- -21 pts
C: p=20, 18 informative, noise=3 (dense signal)
1.0 0.6828 0.346 0.46s -
0.5 0.6820 0.318 0.31s 3/5
0.33 0.6787 0.278 0.22s 1/5
sqrt 0.6631 0.240 0.19s 0/5
Every shape: default best on R2, slowest, most correlated.
Every shape: 'sqrt' worst on R2.
Reproduce shape A:
import numpy as np, time
from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
for s in [1.0, 0.5, 0.33, 'sqrt']:
r2, rho, secs = [], [], []
for sd in range(5):
X, y = make_regression(n_samples=2000, n_features=20, n_informative=6,
noise=12.0, random_state=sd)
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=sd)
t0 = time.perf_counter()
m = RandomForestRegressor(n_estimators=150, max_features=s,
random_state=42, n_jobs=-1).fit(Xtr, ytr)
secs.append(time.perf_counter() - t0)
r2.append(m.score(Xte, yte))
P = np.array([t.predict(Xte) for t in m.estimators_])
C = np.corrcoef(P)
rho.append(np.nanmean(C[np.triu_indices_from(C, k=1)]))
print(f"{str(s):>6} R2={np.mean(r2):.4f} rho={np.mean(rho):.3f} "
f"{np.mean(secs):.2f}s")
1.0 R2=0.8723 rho=0.696 0.48s
0.5 R2=0.8708 rho=0.627 0.28s
0.33 R2=0.8473 rho=0.515 0.22s
sqrt R2=0.8036 rho=0.402 0.20s
Why Regression Is Not Classification Here
The decorrelation story from day one still holds — lowering max_features really does cut correlation, monotonically, in every shape above. What changes is the price.
A classifier only needs the majority to be right. Individual trees can be sloppy and the vote still lands, so you can spend a lot of per-tree accuracy on diversity and come out ahead.
A regressor averages the actual numbers. Every tree's error goes into the mean, undiluted. Starve a tree of columns and it splits on whatever it was handed, and that bias is baked into the average — and per-tree bias does not average away, however many trees you add. On shape B, 'sqrt' gave each split 4 of 60 columns with only 5 carrying signal, so most trees never saw a useful feature at all. Correlation fell to 0.248, the lowest of any setting, and R2 fell off a cliff with it.
Lowest correlation, worst model. That is the whole lesson: decorrelation is a means, not a goal.
What To Actually Do
Leave the regression default alone unless fit time hurts. It cost 2.0–2.4x the fit time of 0.33 in these runs, and that is the real bill. If you are refitting in a loop or tuning against the clock, 0.5 is the setting to reach for: within 0.002–0.009 R2 of the default in all three shapes, at roughly 1.7x the speed.
Never copy 'sqrt' across from the classifier. Worst of the four on every shape tested, and catastrophic when signal is sparse in a wide frame.
Tune it, don't inherit it. The peak is interior and shape-dependent — 0.5 beat the default 3 times out of 5 on the dense-signal shape and only once on the others. Put it in your search space over [1.0, 0.7, 0.5, 0.33] and let held-out data decide.
Key Takeaways
The two defaults genuinely differ —
'sqrt'for the classifier,1.0for the regressor. That asymmetry is deliberate, not an oversight.The regression default was the most accurate setting in all three shapes tested. Its cost is ~2x fit time and the highest tree correlation, not accuracy.
'sqrt'on regression was the worst setting every time, losing 21 points of R2 when 5 of 60 features carried the signal.0.5is the pragmatic compromise — statistically tied with the default, meaningfully faster.Average over seeds before believing any of this. Single splits moved by more than the gaps between several of these settings.
The One-Sentence Summary
The advice to copy max_features='sqrt' into your regressor is the rare tuning tip that is both widely repeated and measurably backwards: the regression default was the most accurate setting on every dataset shape I tried, 'sqrt' was the least accurate on all of them, and the only thing the default actually costs you is about twice the fit time — so change it to buy speed, never to buy accuracy.
What's Next?
- The Random Forest cheat sheet — every knob from this series on one printable page.
- The series recap — the one number that explains all six posts.
- Gradient Boosting — where the sequential family gets serious.
Follow me for the next article in the Random Forests Deep Dive series!
Let's Connect!
If you have been copying 'sqrt' into regressors, this one is worth a re-run on your own data.
Questions? Ask in the comments — I read and respond to every one.
Has a "well-known" default ever cost you more than the model did? This post began as a takedown of the regression default and ended up defending it. 🌲
I wrote the first draft convinced the default was a bug. Three dataset shapes later the data said the default was fine and the popular fix was the problem. The honest version is less satisfying and considerably more useful.
Top comments (0)