Most panel data tutorials stop at "run the Hausman test, and if p < 0.05 use fixed effects." That rule is fine as a starting point, but it hides a few things that matter once you look at real output. So here is a small worked example you can run yourself, with the numbers I got.
The setup
I used the wage_panel dataset that ships with linearmodels: 545 individuals observed over 8 years, with log wages as the outcome. The regressors are experience squared, marital status and union membership.
bash
pip install linearmodels
python
import numpy as np
from scipy import stats
from linearmodels.panel import PanelOLS, RandomEffects
from linearmodels.datasets import wage_panel
df = wage_panel.load().set_index(["nr", "year"])
y = df["lwage"]
X = df[["expersq", "married", "union"]]
the RE model needs an intercept, the FE (within) model does not
X_re = X.copy()
X_re.insert(0, "const", 1.0)
fe = PanelOLS(y, X, entity_effects=True).fit()
re = RandomEffects(y, X_re).fit()
The two sets of estimates
variable fixed effects random effects
expersq 0.0037 0.0032
married 0.1073 0.1316
union 0.0828 0.1035
The random effects numbers for married and union are noticeably larger. That gap is what the Hausman test formalises. If the unobserved person-level effects (ability, motivation, whatever drives wages and also correlates with marrying or joining a union) are correlated with the regressors, the RE estimator picks that correlation up as if it were an effect of the regressor, and the coefficients drift upward.
Computing the Hausman statistic by hand
linearmodels doesn't ship a one-line Hausman test, but the statistic is short:
python
cols = fe.params.index
b = fe.params - re.params[cols]
V = fe.cov - re.cov.loc[cols, cols]
stat = float(b @ np.linalg.inv(V) @ b)
p = 1 - stats.chi2.cdf(stat, len(cols))
print(f"Hausman chi2 = {stat:.3f}, df = {len(cols)}, p = {p:.4f}")
Output on this data:
Hausman chi2 = 112.118, df = 3, p = 0.0000
A clear rejection. The RE assumption (effects uncorrelated with the regressors) doesn't hold here, so the fixed effects estimates are the ones to report.
Three things the textbook rule skips
A rejection is evidence against RE, not a proof that FE is right. FE only removes confounding that is constant over the sample window. Anything that changes over time and correlates with both wages and marriage is still in your error term.
You pay for FE with the time-invariant variables. The within estimator can't estimate the effect of anything that doesn't change for a person (education at entry, say, or a fixed demographic). I left those out above for exactly that reason. If they are the point of your study, FE alone won't answer your question, and a Mundlak or correlated random effects specification is worth a look.
The classic Hausman test assumes homoskedastic errors. With clustered standard errors, which is the usual choice in applied work, the difference of covariance matrices isn't the right object anymore. A regression-based version (the Mundlak approach: add the group means of each regressor to the RE model and test them jointly) works with robust errors.
If you'd rather not write code
For a quick check, or to hand something to a colleague who doesn't use Python, there is a panel data regression online tool that fits fixed and random effects models in the browser. Full disclosure: I'm connected to that site. For anything you plan to publish, I'd still keep the script above so the analysis is reproducible.
Curious whether people go Mundlak, or just default to FE and move on.RE model and test them jointly) works with robust errors.
If you'd rather not write code
For a quick check, or to hand something to a colleague who doesn't use Python, there is a panel data regression online tool that fits fixed and random effects models in the browser. Full disclosure: I'm connected to that site. For anything you plan to publish, I'd still keep the script above so the analysis is reproducible.
What do you use when the Hausman test and clustered errors disagree? Curious whether people go Mundlak, or just default to FE and
Top comments (0)