Feature Engineering Trade-offs & Multi-Metric

H4dis — Tue, 04 Aug 2026 20:49:58 +0000

Adding manual signal features (Impact Severity, Energy Ratios) boosted Class 2 F1 score but caused metric drop in Class 0 & 1, decreasing total leaderboard performance.
Root Causes

Multicollinearity: Re-creating Peak-to-RMS ratio when** Crest Factor** was already present. - Dimensional Mismatch: Direct summation of unscaled velocity (mm/s) and acceleration (g). -** Outlier Explosion**: Division by small RMS values creating extreme data spikes.
Metric Trade-off: Optimizing
features for a single class without controlling multi-metric balance.
Key Learnings & Fixes
Outlier Suppression: Apply logarithmic transformation (np.log1p) on raw ratio features to normalize distributions.
Feature Stacking: Isolate noisy Class 2 features into a binary sub-model, injecting only its output probability (prob_class2) into the main classifier.
Multi-Threshold Post-Processing: Use scipy.optimize.minimize (Nelder-Mead) on prediction probabilities to optimize overall multi-metric leaderboard score instead of manual feature hacking.
Competition vs Engineering: Recognized that high leaderboard scores often rely on prior distribution alignment and post-processing hacks rather than pure signal domain analysis.

DEV Community: H4dis

Feature Engineering Trade-offs & Multi-Metric