The One-Line Summary:
feature_importances_measures how much each column reduced impurity on the training rows, which quietly rewards columns with many possible split points instead of columns that actually predict anything — so a column of random IDs can outrank a genuinely useful binary flag, exactly like a company promoting the man who signs everything and firing the woman who prevents every disaster.
The Parable of the Company That Counted Signatures
Last time, a village blindfolded twelve judges and got better verdicts out of worse jurors. This time, a freight company spends eleven years not noticing that its one unarguable number measures the opposite of what it thinks.
Harrowgate & Vale moves cargo. Three hundred and forty staff, four warehouses, one rule everybody knows by heart: the Ledger decides.
The Ledger
Every time a job gets done, whoever did it signs the Ledger. At year end the partners count signatures. Most signatures, biggest bonus. Fewest signatures, cleaned desk. Nobody can argue with arithmetic.
THE LEDGER, YEAR ELEVEN
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
NAME SIGNATURES WHAT THEY SIGN FOR
Devon 38,412 every parcel that arrives
Marcus 4,510 the dock log, odd weeks
Wei 4,488 the dock log, even weeks
Otis 1,204 whichever floor he's on
Priya 96 the manifest, before a truck
leaves
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Devon signs for every parcel that comes through the door. He inspects nothing, decides nothing. His signature is on everything, which is another way of saying it tells you nothing: he signed all of them, the good and the ruined alike.
Priya does one thing. Twice a week she reads a departing truck's manifest against what is on the truck and writes one word: checked. Ninety-six words a year. When Priya checks a truck, it arrives whole. The fortnight she had shingles, three trucks went out wrong and Harrowgate lost its second-largest customer.
Marcus and Wei share the dock log on alternating weeks. Between them they cover every shift, and Marcus writing dock clear means identically the same thing as Wei writing it.
Otis signs for whichever floor he is standing on. Nobody knows why.
WHAT THE LEDGER DECIDED
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Devon 38,412 signatures → DIRECTOR OF OPERATIONS
Marcus 4,510 signatures → "middling. let him go."
Wei 4,488 signatures → "middling. let him go."
Priya 96 signatures → "contributes nothing."
Promoted: the man whose signature means nothing.
Fired: the woman who prevented every disaster,
and BOTH people who ran the dock.
Three months later, four trucks left with the wrong manifests, the dock ran two days behind for a quarter, and Devon — now Director of Operations — requisitioned a larger stamp.
Shuffle Week
Ines Vaugh had been a junior partner for six weeks and had already learned not to attack the Ledger directly. So she attacked the question instead.
"Stop counting what people touch. Start measuring what breaks when what they do stops making sense."
Pick one person. For one week, secretly scramble their output: every mark they make gets randomly reassigned to a different job. Devon's signatures land on the wrong parcels. Priya's checked marks land on trucks she never looked at. Nothing else changes — same people, same routines, same number of marks. The marks just stop meaning anything.
Then count how many shipments go wrong, on next month's work — not on the eleven years of Ledger the company had already written about itself.
SHUFFLE WEEK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
whose marks were shipments
scrambled gone wrong verdict
nobody (baseline) 29 —
Devon 29 no damage at all
Otis 30 no damage at all
Marcus 30 no damage at all
Wei 30 no damage at all
Priya 71 SEVERE
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Devon's marks were noise for eleven years, and
scrambling noise produces noise. Nothing moved.
Priya's one word a week was holding up the roof.
But read the dock rows literally and you get
"neither Marcus nor Wei matters" — wrong. Nothing
broke because the office simply read the OTHER
man's entries instead.
scramble one → no damage
scramble both → the dock stops
The Ledger had ranked Devon four hundred times above Priya. Shuffle Week ranked Priya first and Devon at zero — and still fired both dock men.
Two people doing one job each look replaceable, because each one is replaceable, right up to the moment you replace both. So Ines added a second, expensive test, the empty chair: send the person home for a month and run the company without them. And when two people share a job, empty the chair, not the person.
Why the Ledger Lied
WHAT THE LEDGER WAS ACTUALLY MEASURING
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
The Ledger counted OPPORTUNITIES TO SIGN.
Devon had 38,412 chances to be involved.
Priya had 96.
Someone who touches everything will always
outscore someone who touches one thing, even when
the one thing is the only thing that matters.
Three lessons, one story:
1. Counting activity is not measuring value.
→ this is IMPURITY-BASED IMPORTANCE (MDI)
2. Break someone on purpose, then measure the
damage — on work nobody has seen yet.
→ this is PERMUTATION IMPORTANCE
3. Two people doing the same job each look
replaceable, and both get fired.
→ this is the CORRELATED FEATURE problem
Everything below is those three sentences, written in Python.
What Is Feature Importance?
A trained Random Forest hands you feature_importances_ for free: one non-negative number per column, summing to 1.0. It looks like an answer. It is a measurement, and you need to know what got measured. Three different questions hide under "how important is this feature":
| Question | Method | Measured on |
|---|---|---|
| How much did this column reduce training impurity? | MDI / Gini importance | training rows |
| How much worse is the model if this column becomes nonsense? | Permutation importance (MDA) | held-out rows |
| How much worse would a model be that never saw this column? | Drop-column importance | held-out rows, after refitting |
feature_importances_ answers the first. Almost nobody wants the first.
The Math of MDI
Gini impurity of a node is , and a split's impurity decrease is
MDI for feature sums that over every node in every tree that split on , weighted by the samples reaching each node, then averages over the trees:
sklearn then normalizes the vector to sum to 1 — which is why MDI feels like a percentage of credit, and why people quote it to stakeholders.
Notice what is not in that formula: no held-out data. Not one row. Every quantity — , , the chosen threshold — comes from the rows the tree was grown on. MDI is a training score, and we would never report training accuracy to a stakeholder.
Where the Bias Comes From
WHERE THE CARDINALITY BIAS COMES FROM
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
A feature with u distinct values offers u-1
candidate thresholds at every single node.
binary flag u = 2 → 1 threshold
floor number u = 3 → 2 thresholds
4-level code u = 4 → 3 thresholds
unique id u = 1400 → 1399 thresholds
The tree keeps the BEST of those candidates.
Best-of-1399 lucky splits beats best-of-1 honest
split on the training rows almost every time —
and the winner's luck is recorded as impurity
reduction, which is reported as IMPORTANCE.
MDI pays for OPPORTUNITY and calls it VALUE.
That is Devon's 38,412 signatures. Two biases, one root:
- Cardinality bias. High-cardinality columns get more chances to look good: IDs, timestamps, hashes, customer numbers, latitudes.
- High-variance / in-sample bias. Impurity decrease is scored on the same rows used to choose the split, so any column that helps the tree memorize gets paid. Fully-grown trees make it worse: deep nodes hold few samples, and almost anything shatters them.
Permutation Importance, Formally
Fit once. Score on held-out data for a baseline . Then, for feature , shuffle that one column and re-score:
Shuffling breaks the relationship between column and while leaving 's marginal distribution untouched. If the model leaned on , the score falls. If it didn't, nothing happens — and if was actively misleading the model, the score rises and importance goes negative. Negative importance is a feature, not a bug: the column is worse than useless.
Step by Step
- Fit the forest on the training set. Never refit — you are auditing this exact model.
- Score it on held-out rows for your baseline .
- For each feature , for each of repeats: shuffle column , keep where it is, score, record .
- Average over the repeats. Keep the standard deviation.
- Sort. Anything within a couple of standard deviations of zero is noise, and say so out loud.
Feature Importance with Scikit-Learn
Harrowgate & Vale as a dataset. Five columns, five employees:
-
priya_check— a binary flag that genuinely drives the outcome -
marcus_log— a 4-level code that genuinely drives the outcome -
wei_log— a byte-for-byte duplicate ofmarcus_log -
devon_seq— 2,000 unique integers, pure noise, no relationship to the label -
floor_no— 3 levels, also pure noise
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
FEATURES = ["priya_check", "marcus_log", "wei_log", "devon_seq", "floor_no"]
def build_review_data(n=2000, seed=42):
"""Five 'employees'. Only two of them actually do anything."""
rng = np.random.default_rng(seed)
priya_check = rng.integers(0, 2, n) # BINARY, genuinely predictive
marcus_log = rng.integers(0, 4, n) # 4 levels, genuinely predictive
wei_log = marcus_log.copy() # EXACT duplicate of Marcus
devon_seq = rng.permutation(n) # 2000 unique values, PURE NOISE
floor_no = rng.integers(1, 4, n) # 3 levels, pure noise
log_effect = np.array([-1.5, -0.45, 0.45, 1.5])
logit = 3.0 * priya_check + log_effect[marcus_log] - 1.5
y = (rng.random(n) < 1.0 / (1.0 + np.exp(-logit))).astype(int)
X = np.column_stack([priya_check, marcus_log, wei_log,
devon_seq, floor_no]).astype(float)
return X, y
X, y = build_review_data()
X_tr, X_te, y_tr, y_te = train_test_split(
X, y, test_size=0.3, random_state=42, stratify=y
)
rf = RandomForestClassifier(n_estimators=200, random_state=42,
n_jobs=-1).fit(X_tr, y_tr)
print(f"train accuracy: {rf.score(X_tr, y_tr):.4f}")
print(f"test accuracy: {rf.score(X_te, y_te):.4f}")
print()
print("THE PERFORMANCE REVIEW (feature_importances_)")
for i in np.argsort(rf.feature_importances_)[::-1]:
bar = "#" * int(round(rf.feature_importances_[i] * 50))
print(f" {FEATURES[i]:>11} {rf.feature_importances_[i]:.4f} {bar}")
train accuracy: 1.0000
test accuracy: 0.7083
THE PERFORMANCE REVIEW (feature_importances_)
devon_seq 0.5602 ############################
priya_check 0.2722 ##############
marcus_log 0.0711 ####
wei_log 0.0663 ###
floor_no 0.0303 ##
The most important feature in this model, by a factor of two, is a column of random integers. devon_seq holds 56% of the total, and it has no relationship to the label whatsoever — rng.permutation, drawn independently of y. The genuinely predictive binary flag comes second at 27%. And marcus_log and wei_log — a column that genuinely matters, present twice — sit near the bottom at 0.0711 and 0.0663. Middling. Let them go.
The model got promoted to production and fired Priya.
Note train accuracy: 1.0000 beside test accuracy: 0.7083. The forest memorized the training set perfectly, and devon_seq is how. MDI is not measuring prediction; it is measuring participation in memorization.
The Same Model, Audited Properly
Same forest, same rows. We change only the question.
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import permutation_importance
from sklearn.model_selection import train_test_split
FEATURES = ["priya_check", "marcus_log", "wei_log", "devon_seq", "floor_no"]
def build_review_data(n=2000, seed=42):
rng = np.random.default_rng(seed)
priya, marcus = rng.integers(0, 2, n), rng.integers(0, 4, n)
devon, floor = rng.permutation(n), rng.integers(1, 4, n)
logit = 3.0 * priya + np.array([-1.5, -0.45, 0.45, 1.5])[marcus] - 1.5
y = (rng.random(n) < 1.0 / (1.0 + np.exp(-logit))).astype(int)
# marcus appears twice: Marcus and Wei do the identical job
return np.column_stack([priya, marcus, marcus,
devon, floor]).astype(float), y
X, y = build_review_data()
X_tr, X_te, y_tr, y_te = train_test_split(
X, y, test_size=0.3, random_state=42, stratify=y
)
rf = RandomForestClassifier(n_estimators=200, random_state=42,
n_jobs=-1).fit(X_tr, y_tr)
mdi = rf.feature_importances_
tr = permutation_importance(rf, X_tr, y_tr, n_repeats=10,
random_state=42, n_jobs=-1)
te = permutation_importance(rf, X_te, y_te, n_repeats=10,
random_state=42, n_jobs=-1)
print(f"{'feature':>11} {'MDI':>8} {'perm TRAIN':>11} {'perm TEST':>11} {'+/-':>7}")
for i in np.argsort(te.importances_mean)[::-1]:
print(f"{FEATURES[i]:>11} {mdi[i]:>8.4f} {tr.importances_mean[i]:>+11.4f} "
f"{te.importances_mean[i]:>+11.4f} {te.importances_std[i]:>7.4f}")
feature MDI perm TRAIN perm TEST +/-
priya_check 0.2722 +0.2926 +0.1488 0.0172
floor_no 0.0303 +0.1921 -0.0053 0.0126
devon_seq 0.5602 +0.2821 -0.0133 0.0167
marcus_log 0.0711 +0.1452 -0.0258 0.0087
wei_log 0.0663 +0.1381 -0.0263 0.0096
perm TEST is the column that tells the truth. Exactly one feature is positive: priya_check, at +0.1488 with a standard deviation of 0.0172 — eight sigma from zero, unambiguous. devon_seq scores -0.0133: shuffling the random ID improves held-out accuracy, because the model was memorizing through it. The feature MDI called most important is the one the model would be better off without.
perm TRAIN is the trap inside the fix. Permutation importance inherits the sins of whatever rows you run it on. On the training set, devon_seq scores +0.2821 — neck and neck with the real signal at +0.2926 — and even floor_no, three levels of pure noise, scores +0.1921. Of course: the model memorized through those columns, so scrambling them destroys the memorization. Permutation importance on training data is a different way of computing the same lie. The held-out set is not a nice-to-have; it is the entire mechanism.
Watching Cardinality Buy Importance
Keep a single genuinely predictive binary feature, add exactly one noise feature, and vary nothing but the number of distinct values that noise may take.
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(42)
n = 2000
priya_check = rng.integers(0, 2, n) # the real signal, 2 levels
logit = 3.0 * priya_check - 1.5
y = (rng.random(n) < 1.0 / (1.0 + np.exp(-logit))).astype(int)
print(f"{'noise levels':>12} {'noise MDI':>10} {'priya MDI':>10} {'test acc':>9}")
for k in [2, 5, 20, 100, 500, 2000]:
noise = rng.integers(0, k, n) # pure noise, k distinct values
X = np.column_stack([priya_check, noise]).astype(float)
X_tr, X_te, y_tr, y_te = train_test_split(
X, y, test_size=0.3, random_state=42, stratify=y
)
rf = RandomForestClassifier(n_estimators=200, random_state=42,
n_jobs=-1).fit(X_tr, y_tr)
a, b = rf.feature_importances_
print(f"{k:>12} {b:>10.4f} {a:>10.4f} {rf.score(X_te, y_te):>9.4f}")
noise levels noise MDI priya MDI test acc
2 0.0018 0.9982 0.8033
5 0.0160 0.9840 0.8033
20 0.0772 0.9228 0.8033
100 0.2800 0.7200 0.8000
500 0.5118 0.4882 0.7367
2000 0.5717 0.4283 0.6850
Nothing about that column's relationship to the label changed across those six rows — it is uniform random garbage in all of them. Its apparent importance went from 0.18% to 57.17%, a 300-fold swing, bought entirely with distinct values.
Now the last column. Test accuracy holds flat at 0.8033 while the noise stays coarse, then collapses to 0.6850 once it is fine enough to memorize with. The feature MDI promotes hardest is the one doing the most damage — the same mechanism read from both ends.
The Correlated Pair That Fools Everything
The hard case: the one that survives switching to permutation importance, and the one I have watched cost real teams real accuracy.
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
def build_review_data(n=2000, seed=42):
rng = np.random.default_rng(seed)
priya, marcus = rng.integers(0, 2, n), rng.integers(0, 4, n)
devon, floor = rng.permutation(n), rng.integers(1, 4, n)
logit = 3.0 * priya + np.array([-1.5, -0.45, 0.45, 1.5])[marcus] - 1.5
y = (rng.random(n) < 1.0 / (1.0 + np.exp(-logit))).astype(int)
return np.column_stack([priya, marcus, marcus,
devon, floor]).astype(float), y
X, y = build_review_data()
def train_on(cols, seeds=(42, 43, 44)):
"""Refit from scratch on a column subset -> (mean acc, first model)."""
a, b, c, d = train_test_split(X[:, cols], y, test_size=0.3,
random_state=42, stratify=y)
ms = [RandomForestClassifier(n_estimators=200, random_state=s,
n_jobs=-1).fit(a, c) for s in seeds]
return float(np.mean([m.score(b, d) for m in ms])), ms[0]
# --- 1. what a duplicate does to MDI ---
_, both = train_on([0, 1, 2, 3, 4])
_, solo = train_on([0, 1, 3, 4]) # Wei deleted from the payroll
m, w = both.feature_importances_[1], both.feature_importances_[2]
print("MDI for the log-keeping job:")
print(f" two people share it: marcus={m:.4f} + wei={w:.4f} = {m + w:.4f}")
print(f" one person does it: marcus={solo.feature_importances_[1]:.4f}")
# --- 2. drop-column importance: the only honest audit ---
full, _ = train_on([0, 1, 2, 3, 4])
print(f"\nDROP-COLUMN IMPORTANCE (baseline test acc = {full:.4f})")
for label, cols in [("fire priya_check", [1, 2, 3, 4]),
("fire marcus_log ", [0, 2, 3, 4]),
("fire wei_log ", [0, 1, 3, 4]),
("fire devon_seq ", [0, 1, 2, 4]),
("fire floor_no ", [0, 1, 2, 3]),
("fire BOTH logs ", [0, 3, 4])]:
acc, _ = train_on(cols)
print(f" {label} acc={acc:.4f} importance={full - acc:+.4f}")
MDI for the log-keeping job:
two people share it: marcus=0.0711 + wei=0.0663 = 0.1374
one person does it: marcus=0.1470
DROP-COLUMN IMPORTANCE (baseline test acc = 0.7083)
fire priya_check acc=0.5628 importance=+0.1456
fire marcus_log acc=0.7083 importance=+0.0000
fire wei_log acc=0.7083 importance=+0.0000
fire devon_seq acc=0.7750 importance=-0.0667
fire floor_no acc=0.7017 importance=+0.0067
fire BOTH logs acc=0.6433 importance=+0.0650
MDI splits the credit almost exactly in half. One column doing the log job scores 0.1470. Two identical columns doing it score 0.0711 and 0.0663 — summing to 0.1374, the same total, halved. Duplicate a column ten times and you have ten features that all look like rounding error. The information did not get less useful. It got divided.
Permutation importance doesn't split the credit, it destroys it. In the previous table marcus_log scored -0.0258 and wei_log -0.0263 on held-out data. Not small — negative. Shuffling Marcus does nothing, because the forest reads Wei instead; shuffling Wei does nothing, because the forest reads Marcus. An analyst reading that table drops both, ships it, and loses accuracy.
Drop-column gets it right — but only if you drop the group. Firing marcus_log costs +0.0000. Firing wei_log costs +0.0000 (identical, because it is literally the same column). Firing both costs +0.0650, the second-biggest number in the analysis. One at a time, free. Together, expensive.
And the punchline the Ledger never saw coming: firing devon_seq scores -0.0667. Deleting the model's "most important feature" makes it 6.7 points more accurate, 0.7083 to 0.7750. Devon was not useless. Devon was the most expensive employee at Harrowgate & Vale.
THE HIERARCHY OF HONESTY
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
devon priya marcus wei
MDI (train) 0.5602 0.2722 0.0711 0.0663
↑ #1 ↑ #2 ↑ weak ↑ weak
permutation (test)-0.0133 +0.1488 -0.0258 -0.0263
↑ zero ↑ #1 ↑ negative!
drop-column -0.0667 +0.1456 +0.0000 +0.0000
drop the GROUP — — +0.0650 (!)
MDI got EVERYTHING wrong.
permutation got the noise right, the pair wrong.
drop-group got all of it right, for p+1 fits.
How to Actually Handle Correlated Features
- Compute the Spearman rank correlation matrix of your features.
- Hierarchically cluster on
1 - |rho|(scipy.cluster.hierarchy.ward, thenfcluster). - Keep one representative per cluster, or permute and drop whole clusters.
- Report importance per cluster, not per column.
If you take one habit from this article, take that one. It converts an unanswerable question ("is Marcus important?") into an answerable one ("is the dock log important?").
Permutation Importance from Scratch
Twelve lines of NumPy. No gradients, no impurity bookkeeping, no access to the model's internals — just predict, a shuffle, and subtraction. Which is why it works on anything: forests, boosters, neural nets, ONNX blobs behind an HTTP call.
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import permutation_importance
from sklearn.model_selection import train_test_split
FEATURES = ["priya_check", "marcus_log", "wei_log", "devon_seq", "floor_no"]
def my_permutation_importance(model, X, y, n_repeats=10, seed=42):
"""Permutation importance in 12 lines. This is the whole algorithm."""
rng = np.random.default_rng(seed)
baseline = float((model.predict(X) == y).mean()) # score once, honestly
scores = np.zeros((X.shape[1], n_repeats))
Xp = X.copy() # never touch caller's X
for j in range(X.shape[1]):
original = Xp[:, j].copy()
for r in range(n_repeats):
Xp[:, j] = rng.permutation(original) # break j <-> y only
shuffled = float((model.predict(Xp) == y).mean())
scores[j, r] = baseline - shuffled # how much did we lose?
Xp[:, j] = original # put the column back
return scores.mean(axis=1), scores.std(axis=1)
def build_review_data(n=2000, seed=42):
rng = np.random.default_rng(seed)
priya, marcus = rng.integers(0, 2, n), rng.integers(0, 4, n)
devon, floor = rng.permutation(n), rng.integers(1, 4, n)
logit = 3.0 * priya + np.array([-1.5, -0.45, 0.45, 1.5])[marcus] - 1.5
y = (rng.random(n) < 1.0 / (1.0 + np.exp(-logit))).astype(int)
return np.column_stack([priya, marcus, marcus,
devon, floor]).astype(float), y
X, y = build_review_data()
X_tr, X_te, y_tr, y_te = train_test_split(
X, y, test_size=0.3, random_state=42, stratify=y
)
rf = RandomForestClassifier(n_estimators=200, random_state=42,
n_jobs=-1).fit(X_tr, y_tr)
mine, mine_sd = my_permutation_importance(rf, X_te, y_te, n_repeats=10, seed=42)
ref = permutation_importance(rf, X_te, y_te, n_repeats=10,
random_state=42, n_jobs=-1)
print(f"{'feature':>11} {'mine':>9} {'sklearn':>9} {'diff':>8}")
for i in range(X.shape[1]):
print(f"{FEATURES[i]:>11} {mine[i]:>+9.4f} {ref.importances_mean[i]:>+9.4f} "
f"{abs(mine[i] - ref.importances_mean[i]):>8.4f}")
print(f"\nmax absolute disagreement: "
f"{np.max(np.abs(mine - ref.importances_mean)):.4f}")
print(f"monte-carlo noise (sklearn std): {ref.importances_std.mean():.4f}")
feature mine sklearn diff
priya_check +0.1518 +0.1488 0.0030
marcus_log -0.0272 -0.0258 0.0013
wei_log -0.0263 -0.0263 0.0000
devon_seq -0.0045 -0.0133 0.0088
floor_no -0.0077 -0.0053 0.0023
max absolute disagreement: 0.0088
monte-carlo noise (sklearn std): 0.0130
The largest disagreement between my twelve lines and sklearn's implementation is 0.0088 — smaller than sklearn's own repeat-to-repeat standard deviation of 0.0130. Same estimator, different seeds. sklearn.inspection parallelizes, supports arbitrary scoring, and handles pandas; that is the entire difference.
That 0.0130 deserves a stare. Your importances have error bars about as wide as the gap between third and fifth place, and n_repeats=5 on a small test set will reorder your leaderboard between runs. Always print the standard deviation. If you are going to delete a column because of a number, earn it with n_repeats=30.
When Does Each Method Help Most?
FEATURE IMPORTANCE: THE HONEST ASSESSMENT
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
REACH FOR MDI WHEN:
✓ You want a zero-cost first glance
✓ All features share similar cardinality
✓ You will verify before acting on it
✗ Never when a high-cardinality column exists
✗ Never in a slide shown to a stakeholder
REACH FOR PERMUTATION IMPORTANCE WHEN:
✓ You need a number you can defend
✓ You care about a specific metric (AUC, recall)
✓ Refitting the model is expensive
✓ You have a genuine held-out set
✗ Not when features are strongly correlated
REACH FOR DROP-COLUMN WHEN:
✓ You are actually deciding what to delete
✓ p+1 refits fit in your compute budget
✓ You drop correlated GROUPS, not members
NONE OF THEM TELL YOU:
✗ Causation. Not even slightly.
✗ Direction of effect (does it push up or down?)
✗ Anything about ONE specific prediction
✗ Whether a feature would help a DIFFERENT model
Two caveats. Permutation importance evaluates the model off the data manifold: shuffle height independently of weight and you invent 140 cm adults weighing 110 kg, so the model gets scored on inputs that cannot exist. That is a genuine objection and the reason conditional permutation schemes exist; for most tabular work it is second-order, and for tightly coupled physical quantities it is not.
And none of this explains a prediction. Everything here is global: this column matters to this model overall. When a declined loan applicant asks why their application failed, global importance is the wrong tool — that is what SHAP values are for, per-row additively-decomposed attributions with real axioms behind them, and the subject of its own article later in this series. Global importance ranks columns; SHAP explains rows.
The Knobs That Change the Answer
| Parameter | Where | What it does | Guidance |
|---|---|---|---|
n_repeats |
permutation_importance |
shuffles per feature | 10 to look, 30+ to delete |
scoring |
permutation_importance |
which metric gets damaged | Set it explicitly. Accuracy-based importance on an imbalanced problem is meaningless. |
max_samples |
permutation_importance |
subsample the eval rows | Lower for speed on big test sets |
random_state |
permutation_importance |
the shuffle order |
42, or your leaderboard moves |
n_jobs |
permutation_importance |
parallelism |
-1. Embarrassingly parallel. |
max_depth, min_samples_leaf
|
the forest | how deep MDI's in-sample bias reaches | Shallower trees shrink the bias, never remove it |
max_features |
the forest | which columns get a chance per split | Low values spread MDI across a correlated group |
One anti-pattern worth naming: permutation_importance(rf, X_train, y_train), because that is the variable in scope. We measured the cost — pure noise at +0.2821, a hair below the real signal. That is not a milder version of the right answer. It is the wrong answer with extra compute.
Quick Reference Card
FEATURE IMPORTANCE: CHEAT SHEET
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
THE THREE METHODS:
MDI impurity drop on TRAIN rows.
free (already computed), badly biased.
PERMUTATION score drop after shuffling column j
on HELD-OUT rows. R·p predicts, no refit.
DROP-COLUMN refit without column j. exact, p+1 fits.
MDI's TWO BIASES:
cardinality u distinct values → u-1 thresholds →
more chances to get lucky → more credit
in-sample it is a training score wearing a hat
WHAT BREAKS MDI *AND* PERMUTATION:
correlated features. MDI splits the credit,
permutation masks it (each twin covers the other).
Fix: cluster on Spearman rho, drop whole groups.
DEFAULTS THAT ARE ACTUALLY GOOD:
n_repeats = 10 to look, 30 to decide
scoring = whatever you optimise in production
evaluate = ALWAYS on held-out rows
report = mean AND standard deviation
WHAT NOT TO DO:
✗ Ship feature_importances_ to a stakeholder
✗ Run permutation importance on the training set
✗ Drop a correlated group one member at a time
✗ Read any of it as causation
SKLEARN:
from sklearn.inspection import permutation_importance
rf.feature_importances_ # MDI — handle with care
Key Takeaways
feature_importances_is a training-set statistic. It sums impurity decrease over the rows the tree was grown on and never looks at held-out data. You would not quote training accuracy; stop quoting this.MDI is biased toward high cardinality. distinct values buy thresholds per node, and the tree keeps the luckiest. Pure noise climbed from 0.18% to 57.17% importance on nothing but distinct values.
A random ID beat a genuinely predictive binary flag, 0.5602 to 0.2722. Not a contrived edge case — an unpruned forest, one unique-per-row column, the defaults everyone uses.
Permutation importance fixes it, but only on held-out rows. The same audit on training data scored the noise column +0.2821 against +0.2926 for the real signal. The held-out set is the mechanism, not a formality.
Negative importance is information.
devon_seqscored -0.0133 on permutation and -0.0667 on drop-column: actively harmful, and deleting it moved accuracy from 0.7083 to 0.7750.Correlated features break MDI by division and permutation by masking. Two identical columns split 0.1470 of MDI into 0.0711 and 0.0663, and each scored negative under permutation because the other covered for it.
Dropping one of a correlated pair costs nothing; dropping both costs everything. +0.0000 each, +0.0650 together. Drop groups, never members.
Drop-column importance is ground truth and costs refits — cheap when you are about to delete columns from production. And no global method explains a single prediction; that is SHAP's job.
The One-Sentence Summary
feature_importances_ counts how often and how usefully each column got to help memorize the training set, so it pays for opportunity rather than value and will hand its top rank to a column of random IDs while a genuinely decisive binary flag looks like rounding error — audit your model the way Ines audited Harrowgate & Vale: scramble one column at a time on rows nobody has seen, believe the damage rather than the activity, and when two columns do the same job, empty the chair instead of firing the person.
What's Next?
Now that you know your forest's self-assessment is unreliable, you're ready for:
- Out-of-Bag Error — tomorrow's deep dive. Bootstrapping leaves roughly 36.8% of rows out of every tree, so you already own a free held-out set living inside your model — and it is the cheapest legitimate place to compute the permutation importance you just learned.
- Extra Trees — a third source of randomness: stop searching for the best split and pick thresholds at random. Faster, sometimes better, and it moves the cardinality bias in an interesting direction.
- AdaBoost — the algorithm that made weak learners fashionable, where trees grow in sequence and each is paid to fix the last one's mistakes.
-
The
max_featuresdefault — Friday's short post, and the most expensive one-word default in scikit-learn regression. For classification it is'sqrt'. For regression it is not.
Follow me for the next article in the Random Forests Deep Dive series!
Let's Connect!
If Harrowgate & Vale's Ledger made feature importance click, drop a heart!
Questions? Ask in the comments — I read and respond to every one.
Have you ever shipped a feature-importance chart you'd now take back? I have. A customer-ID column once made it into a "top drivers of churn" deck of mine, and a very kind data scientist asked, in front of everyone, what it would mean to increase a customer's ID. 📋
Every organisation I have worked in has had a Ledger — a number easy to count, impossible to argue with, and quietly measuring the wrong thing. Story points. Lines of code. Tickets closed. Messages sent. The failure mode is always identical: the metric rewards whoever touches the most surface area, and the person holding the whole thing together with one decision a week gets a note about visibility. Machine learning did not invent this mistake. It just made it fast enough to watch.
Share this with whoever on your team is about to delete a feature because the bar chart was short. Ask them to empty the chair first.
Top comments (0)