Hypothesis test chooser
Which test to use for which question, with nonparametric alternatives.
The p-value is the chance of seeing a result at least this extreme if nothing were really different. If p is below α (usually 0.05), the difference is statistically significant. Always check whether the difference is large enough to matter.
| Question | Data | Test | Nonparametric option |
|---|---|---|---|
| Is the mean equal to a target? | 1 group, measurements | 1-sample t | 1-sample Wilcoxon |
| Are two means different? | 2 independent groups | 2-sample t (Welch) | Mann-Whitney |
| Did the same parts change? | Before and after pairs | Paired t | Wilcoxon signed rank |
| Are three or more means different? | 3+ groups | One-way ANOVA, then Tukey | Kruskal-Wallis |
| Are the variances different? | 2+ groups | F test, Levene, Bartlett | Levene |
| Is a proportion equal to a target? | Pass/fail counts | 1-proportion test | Exact binomial |
| Are two proportions different? | Pass/fail, 2 groups | 2-proportion test | Fisher's exact |
| Are two categories related? | Counts in a table | Chi-square | Fisher's exact |
| Are two measurements related? | Paired measurements | Pearson correlation, regression | Spearman |
| Is the data normal? | 1 group | Anderson-Darling | - |
| Term | Meaning |
|---|---|
| α (Type I error) | Calling a difference real when it is not. Usually 0.05. |
| β (Type II error) | Missing a real difference. |
| Power | : the chance of detecting a real difference. Aim for 0.80 or more. |
| Null hypothesis | The starting assumption that there is no difference or effect. |
