Most statistical mistakes in published work are not math errors. The arithmetic is done by software and the software gets it right. The mistake happens one step earlier, when someone picks a test that does not fit the data, the software runs it anyway without complaint, and prints a p-value that looks exactly as authoritative as a correct one.
That gap is what statsage was built for. It is a free, MIT licensed Python package that behaves like a careful statistician. Give it a table and the columns to compare, and it checks the assumptions, chooses the test those assumptions allow, reports the effect size and confidence interval, draws the matching figure, and writes the methods and results text you can paste into a manuscript. It runs offline, with no account and no API key.
How the Wrong Test Still Returns a P-Value
A t-test on badly skewed data returns a number. An ANOVA with wildly unequal variances returns a number. A chi-square with expected cell counts of two returns a number. None of those numbers mean what a reader will assume they mean, and nothing in the output says so.
The knowledge to avoid this sits in every statistics textbook, so what is missing is not knowledge, it is a step in the workflow. Under deadline, most people reach for the test they already know how to run and interpret, and the tool cooperates.
What Assumption Checking Actually Looks Like
Before choosing anything, statsage runs the checks the choice depends on: Shapiro-Wilk or D'Agostino for normality, Levene's test for equal variances, sample size rules, IQR outlier flags, and expected cell counts for categorical data.
Those checks then drive the decision. Two groups default to Welch's t-test, which does not assume equal variances, with Student's t-test and Mann-Whitney U available when they fit. Paired designs get the paired t-test or Wilcoxon signed-rank. Three or more groups get one-way ANOVA with Tukey HSD, Welch's ANOVA with Games-Howell, or Kruskal-Wallis with Dunn's test and Holm adjustment. Categorical data gets chi-square or Fisher's exact. Correlation gets Pearson or Spearman.
Effect sizes arrive with the test rather than as an afterthought: Hedges' g, Cohen's dz, rank-biserial r, eta squared, epsilon squared and Cramer's V, each with a plain language interpretation.
Reading the Decision Path
The part that matters most is not the chosen test, it is the sentence explaining the choice. Every report includes its decision path, in the form of "normality failed in group B and n is under 30, so Mann-Whitney U was chosen".
That one line does two things. It teaches the statistics while you get the answer, and it gives you a defensible reply when a reviewer asks why you used that test. You are not saying the software picked it. You are pointing at the check that failed and the rule that followed from it.
It also makes the tool arguable, which is the property you want. Someone who disagrees can go after the specific step instead of the whole output.
Getting there needs clean input, which is its own chore, so the same collection includes labparse for turning raw instrument exports into the tidy tables this expects.
When the Model Is Allowed to Write
statsage is fully functional with no AI at all. If a model is available, through the Claude CLI, the Codex CLI, an Anthropic key, or an OpenAI compatible endpoint, it is used for exactly one job: polishing the written narrative.
The model is never allowed to change a number. If its rewrite drops or alters any value, the rewrite is discarded and the template text stands. The statistics come from SciPy backed computation, not from a language model's memory of what a p-value looks like, and --llm off removes it from the loop entirely.
That boundary is worth copying in any tool that mixes computation with generated text. Let the model handle prose, never the arithmetic, and check its work before accepting it.
What This Looks Like in Practice
Install is pip install statsage, and the command line form is one line:
statsage growth.csv --outcome od600 --group strain
The report prints to the terminal as markdown and saves as a self-contained HTML file with the figure embedded, drawn in the colorblind safe Okabe-Ito palette at 300 dpi, ready for a manuscript or a poster.
The source and issue tracker are on GitHub, and it sits alongside the rest of the free science tools we maintain. If your design is not covered yet, that is worth an issue rather than a workaround.
Top comments (0)