Why Does Your Backtest Look Great But Lose Money Live?
Classic quant beginner story: find an "interesting" indicator → backtest → great results → trade live → lose money.
Why?
Because "looks effective" and "statistically significant" are two very different things. Quantitative factor research has a rigorous evaluation framework: IC, IR, and Barra risk neutralization. Skip this and your backtest is just mining noise.
Part 1: IC — The Measuring Stick for Factor Effectiveness
IC (Information Coefficient) = correlation between current-period factor exposure and next-period stock returns.
Use RankIC (Spearman) over Pearson IC in practice—it's more robust to outliers.
import scipy.stats as stats
def calc_rank_ic(factor_series, return_series):
rank_factor = factor_series.rank()
rank_return = return_series.rank()
ic, _ = stats.spearmanr(rank_factor, rank_return)
return ic
Thresholds
| Metric | Threshold | Meaning |
|---|---|---|
| Mean IC | > 0.03 (Pearson) / > 0.05 (RankIC) | Basic bar for validity |
| IC positive rate | > 55% | Directional stability |
| IC std dev | Lower is better | Consistency |
IC = 0.05 sounds tiny, but in noisy markets like China A-shares, this is genuinely meaningful.
Part 2: IR — Stability Matters More Than Average Effectiveness
IR = Mean IC / Std Dev of IC
- IR > 0.5: Minimum for real-world use
- IR > 1.0: Excellent factor
Recommended combination method: ICIR weighting
Weight each factor proportional to its IR. This rewards both effectiveness and consistency—better than equal-weight combinations.
Part 3: Barra CNE5/CNE6 — The "OS" for Multi-Factor Models
Barra models serve three functions:
- Risk attribution: Where is your return coming from?
- Factor neutralization: Strip sector/size noise from your alpha factor
- Portfolio optimization: Maximize alpha while controlling style exposure
CNE5: 10 Core Style Factors
| Factor | Meaning |
|---|---|
| BETA | Market sensitivity |
| MOMENTUM | 525-day weighted return (excl. last 21 days) |
| SIZE | ln(market cap) |
| EARNYILD | Earnings yield composite |
| RESVOL | Residual volatility, orthogonalized to BETA |
| GROWTH | Composite revenue/earnings growth |
| BTOP | Book-to-price (value) |
| LEVERAGE | Composite financial leverage |
| LIQUIDTY | Turnover rate composite |
| SIZENL | Non-linear size (cube of SIZE, orthogonalized) |
CNE6 Additions (Better for 2024+)
- Quality (ROE stability, earnings quality)
- Sentiment (analyst rating changes, fund flows)
- Dividend Yield
Part 4: The Complete Factor Research Workflow
Step 1: Factor construction (winsorize → fill missing → standardize)
Step 2: Factor neutralization (regress out industry and size)
Step 3: Single-factor testing (RankIC series, ICIR, quintile backtest)
Step 4: Multi-factor combination (ICIR-weighted)
Step 5: Portfolio construction (sector constraints, minimize tracking error)
Step 6: Evaluation (excess return, max drawdown, Sharpe, IR)
Part 5: Five Pitfalls You Must Avoid
- No industry neutralization → Overweights one sector → sector rotation causes massive drawdown
- No outlier handling → Financial metrics have extreme values; use winsorizing (3σ or MAD)
- Look-ahead bias → China financial reports have disclosure delays; always use announcement date
- Ignoring transaction costs → ~1.5% round-trip in A-shares destroys high-turnover strategies
- Data mining bias → Testing 200 factors and keeping top 20 is just noise. Always OOS validate
Part 6: Factor Effectiveness in China A-Shares (2026)
| Factor | Status | Notes |
|---|---|---|
| Small cap | ⚠️ Declining | Pressure since 2024 registration reform |
| Low volatility | ✅ Stable | Defensive returns hold in volatile markets |
| Quality (ROE stability) | ✅ Effective | CNE6 addition; institutions prefer it |
| Momentum | ⚠️ Unstable | Short-term (20–60 days) only |
| Value | ⚠️ Weak | Growth > Value environment persists |
| Sentiment | ✅ Short-term | Good for daily/weekly strategies |
2026 recommendation: Focus on Quality + Low Volatility. Most resilient in current environment.
Conclusion
Mastering IC/ICIR + Barra neutralization is like installing a filter on your strategy research pipeline. It doesn't guarantee good factors—but it effectively screens out the bad ones that merely look good.
This is basic hygiene for quantitative strategy research.
Data: Zheshang Securities Financial Engineering Report, BigQuant, Barra CNE5/CNE6 methodology docs. Factor assessments based on 2025–2026 China A-share practice.
Top comments (0)