Choosing the wrong statistical test is one of the most common — and most avoidable — mistakes in applied data analysis. This is a practical decision map: answer a few questions about your data and you land on the right test, plus how to report it in APA style.
Start with your question, not your data
Before touching any calculator, name what you are actually asking:
- Comparing groups? Does group A differ from group B (or from several groups)?
- Measuring a relationship? Does one variable move with another?
- Checking categories or proportions? Are two categorical variables independent?
The answer picks the family of tests. Everything after that is a refinement based on how your data are shaped.
Comparing two groups
The default is the independent-samples t-test, which compares the means of two groups. It assumes the outcome is roughly normally distributed within each group.
When that assumption breaks — the data are skewed, ordinal (like Likert ratings), or the sample is small — switch to the Mann-Whitney U test. Instead of comparing means, it compares rank distributions, so it makes no assumption about the shape of the data.
If the two measurements are paired (before/after on the same subjects), use a paired t-test, or its non-parametric cousin, the Wilcoxon signed-rank test.
Comparing three or more groups
For three or more groups, reach for one-way ANOVA when the normality and equal-variance assumptions hold. When they do not, the Kruskal-Wallis test is the rank-based alternative. A significant omnibus result only tells you some group differs — follow up with a post-hoc test to see which pairs drive it.
Measuring a relationship
To quantify how two continuous variables move together, use Pearson correlation for linear, normally distributed data, and Spearman correlation when the relationship is monotonic but not linear, or the data are ranked. If you want to predict one variable from another, that is regression.
Categories and proportions
To test whether two categorical variables are independent, use the chi-square test of independence. When your table has small expected counts, chi-square becomes unreliable — use Fisher's exact test instead.
Frequently asked questions
Which test compares two independent groups?
If the outcome is roughly normal, use an independent-samples t-test. If it is skewed, ordinal, or the sample is small, use the Mann-Whitney U test.
What is the non-parametric alternative to ANOVA?
The Kruskal-Wallis test. It compares three or more groups using ranks instead of means, so it does not require normally distributed data.
When should I use Fisher's exact test instead of chi-square?
When your contingency table has small expected cell counts. Fisher's exact test computes an exact p-value and stays valid where chi-square's approximation does not.
How do I report a result in APA style?
Report the test statistic, its degrees of freedom, the p-value, and an effect size (such as Cohen's d for a t-test or eta-squared for ANOVA). State the direction of the effect in plain language, not just the numbers.
A quick reference
- Two groups, normal → independent-samples t-test
- Two groups, skewed/ordinal/small → Mann-Whitney U
- Paired measurements → paired t-test / Wilcoxon signed-rank
- Three or more groups, normal → one-way ANOVA
- Three or more groups, non-normal → Kruskal-Wallis
- Relationship, linear → Pearson correlation
- Relationship, monotonic/ranked → Spearman correlation
- Two categorical variables → chi-square (Fisher's exact for small counts)
Running these without installing software
If you want to run any of these tests in the browser, StatMate is a free calculator suite that returns APA-formatted output and cross-checks its results against R. The calculators are available in English, Korean, and Japanese, and need no signup. It also includes a guided "which test should I use?" wizard that walks through the same decision map above.
Whatever tool you use, the habit that saves you is the same: decide the test from your question and your data's shape before you run anything.
Top comments (0)