Every February 2nd, a groundhog in Pennsylvania either sees his shadow or doesn’t, and half the US media treats it as a real forecast. I wanted to know if there’s anything to it, so I pulled the actual prediction record going back to the 1880s and checked it against real March temperatures. Short answer: no, and the “no” is more interesting than I expected.
The data
The dataset tracks Punxsutawney Phil’s yearly prediction (full shadow = “six more weeks of winter,” no shadow = “early spring”) alongside recorded February and March average temperatures for Pennsylvania, the Northeast, and the Midwest, going back to 1886.
After cleaning:
Dropped a summary row at the bottom of the file that wasn’t an actual year (labeled “1901–2000,” clearly an aggregate row someone left in).
Of 131 real yearly rows, 118 had an actual Full Shadow / No Shadow call - the rest were marked “No Record” (12 years) or, in exactly one case, “Partial Shadow.” I excluded those 13, since there’s no clean way to score a non-prediction.
115 of the remaining 118 had usable March temperature data to check the prediction against.
The class split is heavily lopsided: 100 Full Shadow predictions vs. only 15 No Shadow predictions - Phil calls for six more weeks of winter about 86% of the time, regardless of what actually happens. That imbalance turns out to matter a lot for how to read the accuracy number.
How I scored “correct”
Phil’s prediction is really a directional claim: Full Shadow implies a colder-than-normal March, No Shadow implies a warmer one. So for each year, I compared his call against whether that year’s actual Pennsylvania March average temperature came in above or below the long-run mean across the whole dataset (36.0°F).
Result
Accuracy: 52.2% (60 correct out of 115 years).
That sounds barely-better-than-a-coin-flip, and it is, but the more telling comparison isn’t 50%, it’s against a naive strategy that ignores Phil entirely and just always guesses “colder than average” every single year (which would nearly match his own bias toward predicting Full Shadow). That naive always-guess-colder strategy scores 49.6% statistically indistinguishable from Phil’s actual 52.2%.
I also ran a direct comparison of average March temperature in Full Shadow years vs. No Shadow years:
- Full Shadow years: 35.95°F average
- No Shadow years: 36.32°F average
That’s in the “correct” direction (Full Shadow years were very slightly colder), but the gap is tiny and a Welch’s t-test puts the p-value at 0.72 nowhere close to statistically significant. There’s no real signal here.
Checking it wasn’t a fluke of one region or one era
Before concluding “no skill,” I checked whether the result held up under different slices, since a null result on one narrow cut can hide something real:
- Northeast temperatures: same story, 52.2% accuracy, p=0.50.
- Midwest temperatures: 51.3% accuracy, p=0.73.
Pre-1960 vs. 1960-onward: 51.7% vs. 50.9%, no meaningful shift even across a period that includes real, measurable climate warming.
Consistent near-50% across three regions and two eras is a stronger “no effect” than a single test would have shown, and it’s the main reason I trust the null result here rather than writing it off as noise.
Why I think this happens
The lopsided class split explains most of it: because Phil predicts Full Shadow around 86% of the time regardless of the actual weather, his “track record” mostly just reflects how often a below-average March happens by chance which, across 115 years, is close to half the time anyway. There’s no evidence the tradition is tracking anything about the actual weather; it’s closer to a fixed, mostly-one-sided coin that happens to land near a 50–50 real-world split.
Where this could be wrong
Small sample for the “No Shadow” class (only 15 cases) means that side of the comparison has wide uncertainty a handful of different outcomes in those 15 years could shift the picture more than the same shift would in the Full Shadow group.
I used the Pennsylvania column as primary since that’s where Phil actually is, but “correct” is inherently a simplification real early-spring vs. late-winter isn’t fully captured by one month’s average temperature relative to a century-long mean.
I excluded the “No Record” and “Partial Shadow” years rather than trying to impute a guess for them, which is the more honest choice but does shrink the sample further.
Code and data
Full cleaning and analysis: github.com/DostonUr/GrounHog-prediction-Project-DS
If you’ve seen a different cut of this data or think the scoring method should be set up differently, I’d like to hear it reply here or find me on LinkedIn.
Top comments (0)