khushi singh.in

Tools

Statistics helper

Compare two groups the careful way: see every data point, get the difference with its confidence interval, the usual tests and an effect size, explained in plain words. Plan how many replicates you need.

0 values

0 values

Planning: how many replicates?

For a two-sided t-test. The effect size d is the difference you want to detect divided by the standard deviation; take the SD from a pilot or published data, not from wishful thinking. Effects seen in small pilots tend to be overestimates, so plan for a smaller one. Power calculated afterwards from the observed effect says nothing new about a finished experiment.

→

64 independent replicates per group give 80% power to detect d = 0.5 at α = 0.05.

0%25%50%75%100%n = 64replicates per group (up to 128)

n means independent biological replicates (separate animals, cultures or days), not technical repeats or cells from the same dish; counting cells as replicates makes p-values far too small.

Choosing and reading the results

  • Paired or not? Paired means each value in one group belongs with one value in the other: the same mouse before and after, or treated and control wells from the same experiment day. Pairing removes the day-to-day variation and usually gives a much sharper answer.
  • Welch’s t-test is the default for two independent groups, because it does not assume the groups have equal spreads, and it loses almost nothing when they do. Strong evidence
  • The rank tests (Mann–Whitney, Wilcoxon signed-rank) do not assume normal distributions. They ask a slightly different question: whether values in one group tend to be larger. Use them for skewed data or scores; with very small samples they cannot reach small p-values at all.
  • Read the interval, not just p. The 95% confidence interval shows the range of differences that are compatible with your data. A p-value is not the probability that the result is due to chance, and p above 0.05 does not show that there is no effect. Established
  • Effect sizes (Cohen’s d, Hedges’ g) put the difference on a common scale. Cohen’s words “small, medium, large” were meant only as a last resort; what matters is the size of the effect in your biology.

What counts as n

The n in a test must be the number of independent replicates: separate animals, separate cultures or experiments on different days. Hundreds of cells from one dish are one replicate, not hundreds; treating them as independent makes p-values misleadingly small. Average the cells within each experiment, compare the experiment means, and show both (a “SuperPlot”).Established

What it does not do

Two groups only. For three or more groups, repeated measures, or several factors (for example genotype × treatment), use ANOVA or a mixed model in R, Python or a statistics package, ideally with advice from a statistician before the experiment. Testing many pairs one by one with this page inflates false positives.

References

  1. Student. 1908. The probable error of a mean. Biometrika 6:1–25. doi:10.2307/2331554
  2. Cushny AR, Peebles AR. 1905. The action of optical isomers. II. Hyoscines. Journal of Physiology 32:501–510. (The example data.) doi:10.1113/jphysiol.1905.sp001097
  3. Welch BL. 1947. The generalization of “Student’s” problem when several different population variances are involved. Biometrika 34:28–35. doi:10.1093/biomet/34.1-2.28
  4. Delacre M, Lakens D, Leys C. 2017. Why psychologists should by default use Welch’s t-test instead of Student’s t-test. International Review of Social Psychology 30:92–101. doi:10.5334/irsp.82
  5. Mann HB, Whitney DR. 1947. On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics 18:50–60. doi:10.1214/aoms/1177730491
  6. Wilcoxon F. 1945. Individual comparisons by ranking methods. Biometrics Bulletin 1:80–83. doi:10.2307/3001968
  7. Hedges LV. 1981. Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics 6:107–128. doi:10.3102/10769986006002107
  8. Lakens D. 2013. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology 4:863. doi:10.3389/fpsyg.2013.00863
  9. Cohen J. 1992. A power primer. Psychological Bulletin 112:155–159. doi:10.1037/0033-2909.112.1.155
  10. Cumming G. 2014. The new statistics: why and how. Psychological Science 25:7–29. doi:10.1177/0956797613504966
  11. Wasserstein RL, Lazar NA. 2016. The ASA statement on p-values: context, process, and purpose. The American Statistician 70:129–133. doi:10.1080/00031305.2016.1154108
  12. Weissgerber TL, et al. 2015. Beyond bar and line graphs: time for a new data presentation paradigm. PLOS Biology 13:e1002128. doi:10.1371/journal.pbio.1002128
  13. Vaux DL, Fidler F, Cumming G. 2012. Replicates and repeats—what is the difference and is it significant? EMBO Reports 13:291–296. doi:10.1038/embor.2012.36
  14. Lord SJ, et al. 2020. SuperPlots: communicating reproducibility and variability in cell biology. Journal of Cell Biology 219:e202001064. doi:10.1083/jcb.202001064