# Statistics helper > Compare two groups the careful way: see every data point, get the difference with its confidence interval, the usual tests and an effect size, explained in… Source: https://khushisingh.in/tools/statistics/ Author: Khushi Singh (khushisingh.in). Licence: CC BY 4.0 for text and original illustrations, unless a credit says otherwise. Evidence labels used on this site: established, strong evidence, emerging, hypothesis, debated. Where a page has Simple, Curious and Expert versions, all three follow. --- Tools Compare two groups the careful way: see every data point, get the difference with its confidence interval, the usual tests and an effect size, explained in plain words. Plan how many replicates you need. Your numbers stay here Everything is calculated by your browser; nothing is uploaded or stored. The results match R’s `t.test` and `wilcox.test` on their standard examples. *[Here the web page has an interactive activity; open the source URL to use it.]* ## Choosing and reading the results - **Paired or not?** Paired means each value in one group belongs with one value in the other: the same mouse before and after, or treated and control wells from the same experiment day. Pairing removes the day-to-day variation and usually gives a much sharper answer. - **Welch’s t-test is the default** for two independent groups, because it does not assume the groups have equal spreads, and it loses almost nothing when they do. (evidence: strong evidence) - **The rank tests** (Mann–Whitney, Wilcoxon signed-rank) do not assume normal distributions. They ask a slightly different question: whether values in one group tend to be larger. Use them for skewed data or scores; with very small samples they cannot reach small p-values at all. - **Read the interval, not just p.** The 95% confidence interval shows the range of differences that are compatible with your data. A p-value is not the probability that the result is due to chance, and p above 0.05 does not show that there is no effect. (evidence: established) - **Effect sizes** (Cohen’s d, Hedges’ g) put the difference on a common scale. Cohen’s words “small, medium, large” were meant only as a last resort; what matters is the size of the effect in your biology. ## What counts as n The n in a test must be the number of **independent** replicates: separate animals, separate cultures or experiments on different days. Hundreds of cells from one dish are one replicate, not hundreds; treating them as independent makes p-values misleadingly small. Average the cells within each experiment, compare the experiment means, and show both (a “SuperPlot”). (evidence: established) ## What it does not do Two groups only. For three or more groups, repeated measures, or several factors (for example genotype × treatment), use ANOVA or a mixed model in R, Python or a statistics package, ideally with advice from a statistician before the experiment. Testing many pairs one by one with this page inflates false positives. ## References 1. Student. 1908. The probable error of a mean. Biometrika 6:1–25. [doi:10.2307/2331554](https://doi.org/10.2307/2331554) 2. Cushny AR, Peebles AR. 1905. The action of optical isomers. II. Hyoscines. Journal of Physiology 32:501–510. (The example data.) [doi:10.1113/jphysiol.1905.sp001097](https://doi.org/10.1113/jphysiol.1905.sp001097) 3. Welch BL. 1947. The generalization of “Student’s” problem when several different population variances are involved. Biometrika 34:28–35. [doi:10.1093/biomet/34.1-2.28](https://doi.org/10.1093/biomet/34.1-2.28) 4. Delacre M, Lakens D, Leys C. 2017. Why psychologists should by default use Welch’s t-test instead of Student’s t-test. International Review of Social Psychology 30:92–101. [doi:10.5334/irsp.82](https://doi.org/10.5334/irsp.82) 5. Mann HB, Whitney DR. 1947. On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics 18:50–60. [doi:10.1214/aoms/1177730491](https://doi.org/10.1214/aoms/1177730491) 6. Wilcoxon F. 1945. Individual comparisons by ranking methods. Biometrics Bulletin 1:80–83. [doi:10.2307/3001968](https://doi.org/10.2307/3001968) 7. Hedges LV. 1981. Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics 6:107–128. [doi:10.3102/10769986006002107](https://doi.org/10.3102/10769986006002107) 8. Lakens D. 2013. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology 4:863. [doi:10.3389/fpsyg.2013.00863](https://doi.org/10.3389/fpsyg.2013.00863) 9. Cohen J. 1992. A power primer. Psychological Bulletin 112:155–159. [doi:10.1037/0033-2909.112.1.155](https://doi.org/10.1037/0033-2909.112.1.155) 10. Cumming G. 2014. The new statistics: why and how. Psychological Science 25:7–29. [doi:10.1177/0956797613504966](https://doi.org/10.1177/0956797613504966) 11. Wasserstein RL, Lazar NA. 2016. The ASA statement on p-values: context, process, and purpose. The American Statistician 70:129–133. [doi:10.1080/00031305.2016.1154108](https://doi.org/10.1080/00031305.2016.1154108) 12. Weissgerber TL, et al. 2015. Beyond bar and line graphs: time for a new data presentation paradigm. PLOS Biology 13:e1002128. [doi:10.1371/journal.pbio.1002128](https://doi.org/10.1371/journal.pbio.1002128) 13. Vaux DL, Fidler F, Cumming G. 2012. Replicates and repeats—what is the difference and is it significant? EMBO Reports 13:291–296. [doi:10.1038/embor.2012.36](https://doi.org/10.1038/embor.2012.36) 14. Lord SJ, et al. 2020. SuperPlots: communicating reproducibility and variability in cell biology. Journal of Cell Biology 219:e202001064. [doi:10.1083/jcb.202001064](https://doi.org/10.1083/jcb.202001064)