Estimating Tail Risk With Python: VaR, Expected Shortfall, GARCH, and Extreme Value Theory

Disclosure: AI assistance was used in preparing this article. The code examples are illustrative and are not connected to a live trading system.

A trading strategy can exhibit stable daily returns while remaining exposed to substantial losses during extreme market conditions.

This is why standard volatility estimates are not sufficient for comprehensive risk analysis.

Quantitative systems need to measure both the probability of entering a loss tail and the severity of losses within that tail.

This article demonstrates a practical framework using Python, Value at Risk (VaR), Expected Shortfall (ES), and the Generalized Pareto Distribution (GPD).

1. Calculate Empirical VaR and Expected Shortfall

Let (L) represent portfolio loss, where positive values indicate losses.

VaR at confidence level (\alpha) is the corresponding loss quantile:

[
VaR_\alpha(L)=F_L^{-1}(\alpha)
]

Expected Shortfall can be expressed as:

[
ES_\alpha(L)=
\frac{1}{1-\alpha}
\int_\alpha^1 F_L^{-1}(u)\,du
]

For a continuous distribution, this is equivalent to the conditional average loss beyond the VaR threshold.

For empirical data, however, the quantile-integral definition is preferable because it handles probability mass at the threshold.

The following implementation calculates both measures from historical losses.

import numpy as np


def empirical_var_es(losses, confidence=0.99):
    losses = np.asarray(losses, dtype=float)
    losses = losses[np.isfinite(losses)]

    if len(losses) == 0:
        raise ValueError("No valid loss observations.")

    if not 0 < confidence < 1:
        raise ValueError("Confidence must be between 0 and 1.")

    ordered = np.sort(losses)
    n = len(ordered)

    # Empirical inverse-CDF quantile.
    var = np.quantile(
        ordered,
        confidence,
        method="inverted_cdf"
    )

    # Integrate the empirical quantile function
    # over the upper (1 - confidence) probability tail.
    left = np.arange(n) / n
    right = np.arange(1, n + 1) / n

    weights = np.maximum(
        0.0,
        right - np.maximum(left, confidence)
    )

    es = np.sum(ordered * weights) / (1 - confidence)

    return {
        "observations": n,
        "confidence": confidence,
        "VaR": var,
        "ES": es
    }


# Illustrative daily portfolio losses in USD.
losses = np.array([
    1200, 1500, 900, 1800, 1100,
    2100, 1400, 1600, 1300, 2500,
    1700, 1900, 1250, 1450, 2200,
    3100, 2800, 4200, 3600, 8500
])

result = empirical_var_es(losses, confidence=0.90)

print(result)

This example uses a deliberately small dataset to demonstrate the calculation.

A real 99% tail-risk estimate requires substantially more observations, and the effective amount of independent information may be smaller when losses are serially dependent.

The empirical method also cannot reliably extrapolate far beyond the largest observed loss.

That is where parametric tail modeling may become useful.

2. Fit a Generalized Pareto Distribution

The Peaks Over Threshold method models losses exceeding a selected threshold.

For threshold (u):

[
Y=L-u,\qquad L>u
]

The excess distribution can be approximated by a GPD:

[
G_{\xi,\beta}(y)=
1-\left(1+\frac{\xi y}{\beta}\right)^{-1/\xi}
]

A practical implementation using SciPy is:

import numpy as np
from scipy.stats import genpareto


def fit_gpd_tail(losses, threshold_quantile=0.95):
    losses = np.asarray(losses, dtype=float)
    losses = losses[np.isfinite(losses)]

    if len(losses) < 100:
        raise ValueError(
            "Insufficient observations for this illustrative fit."
        )

    if not 0 < threshold_quantile < 1:
        raise ValueError("Invalid threshold quantile.")

    threshold = np.quantile(
        losses,
        threshold_quantile
    )

    excesses = losses[losses > threshold] - threshold

    if len(excesses) < 20:
        raise ValueError(
            "Too few exceedances for a meaningful fit."
        )

    shape, loc, scale = genpareto.fit(
        excesses,
        floc=0
    )

    return {
        "threshold": threshold,
        "exceedances": len(excesses),
        "shape": shape,
        "scale": scale
    }


tail_model = fit_gpd_tail(
    losses,
    threshold_quantile=0.80
)

print(tail_model)

The threshold values in this example are illustrative, not recommended universal settings.

A production implementation should evaluate multiple thresholds and inspect:

• Mean excess plots.
• Shape-parameter stability.
• Goodness-of-fit diagnostics.
• Number of independent exceedance events.
• Sensitivity of estimated VaR and ES to threshold selection.

A high threshold may leave too few observations, while a low threshold can introduce substantial approximation bias.

The fitted distribution should not be used for extreme extrapolation without these checks.

3. Combine GARCH With Extreme Value Analysis

Historical volatility is not constant.

A GARCH model estimates conditional volatility using previous returns and volatility shocks.

A common specification is:

[
\sigma_t^2=
\omega+\alpha\epsilon_{t-1}^2+
\beta\sigma_{t-1}^2
]

After fitting the model, standardized residuals can be examined for tail behavior.

The general workflow is:

1. Fit a conditional volatility model.
2. Calculate standardized residuals.
3. Test the residual distribution for remaining dependence and volatility clustering.
4. Fit an appropriate tail distribution to the residuals.
5. Reconstruct conditional VaR and ES using current volatility estimates.

McNeil and Frey's research demonstrates this combination of GARCH filtering and extreme value methods for financial risk estimation.

The key benefit is separating changing volatility from the underlying tail characteristics of standardized innovations.

However, the approach still depends on model assumptions. Structural breaks and clustered extreme events can invalidate a simplistic independent-residual interpretation.

4. Validate Risk Estimates

A risk model should be tested against observations that were not used to fit it.

For VaR, record each exceedance:

[
I_t =
\mathbf{1}(L_t>VaR_{\alpha,t})
]

At a 99% confidence level, the expected exceedance frequency is approximately 1%, assuming correct calibration and a suitable evaluation design.

But exceedance frequency alone is insufficient.

A model can have the correct overall number of violations while failing to capture their clustering.

Validation should therefore examine:

• Unconditional exceedance frequency.
• Independence and clustering of violations.
• Conditional performance across market regimes.
• Joint VaR and ES forecast performance.
• Parameter and threshold sensitivity.

For ES, joint scoring and backtesting methods are especially important because ES cannot be adequately evaluated by counting VaR violations alone.

5. Integrate Risk Estimates Into Portfolio Controls

The final step is to connect statistical risk estimates with operational constraints.

For example:

• Reduce new exposure when forecast ES approaches a predefined limit.
• Tighten leverage constraints when volatility and correlation increase together.
• Reassess portfolio concentration during liquidity stress.
• Restrict new positions when execution capacity deteriorates.
• Escalate to independent risk controls when model reliability becomes uncertain.

For KAEL AI's quantitative research, tail-risk modeling is a useful bridge between predictive models and portfolio-level risk controls.

A strategy should not be evaluated only by expected returns under ordinary market conditions. It should also be examined for its behavior when losses move into the extreme regions of the distribution.

Conclusion

VaR identifies a loss quantile. Expected Shortfall measures the severity of losses across the remaining tail. Extreme Value Theory provides tools for modeling rare observations beyond a selected threshold.

Python makes these methods accessible, but implementation alone does not establish model reliability.

Threshold selection, dependence, sample size, conditional volatility, and independent backtesting remain essential.

The objective is to build a risk framework that can identify extreme-loss exposure and translate it into meaningful portfolio constraints.

References

McNeil, A. J., & Frey, R. (2000). Estimation of Tail-Related Risk Measures for Heteroscedastic Financial Time Series: An Extreme Value Approach. Journal of Empirical Finance, 7(3–4), 271–300.

McNeil, A. J., Frey, R., & Embrechts, P. (2015). Quantitative Risk Management: Concepts, Techniques and Tools. Princeton University Press.

Acerbi, C., & Tasche, D. (2002). On the Coherence of Expected Shortfall. Journal of Banking & Finance, 26(7), 1487–1503.