Trust

How the calculator is tested

How the numbers are computed, which published results they are checked against, and how closely they agree.

Sources checked on November 21, 2026. Results computed by the site's engine and tested against R and SciPy.

Numerical methods

The t distribution's tail areas come from the regularized incomplete beta function, evaluated with a continued fraction (modified Lentz method) in log space, following NIST's Digital Library of Mathematical Functions. The two-sided p uses x = df/(df + t²), so small tails come straight from the fraction without subtracting nearly equal numbers. Very large df (above ten million) use a second-order expansion about the normal distribution. Percentiles are found by Newton's method with a bracketing safeguard on the exact tail area, solved on the tail itself so tiny alphas keep their accuracy. If a method ever fails to converge, the calculator says it could not compute rather than show a wrong number.

Sums use compensated addition and variances use the two-pass method (mean first, then squared deviations), so data with large offsets keep their precision. Exact rank-test p-values count every arrangement, as R does.

Accuracy targets

Against SciPy 1.18.1 (which uses the Boost library), p-values must agree to a relative error of 1e-10 (1e-8 below 1e-100), and percentiles to 1e-10. The build stops if any check fails. In the current build the largest relative error across 8,100 stored SciPy values was about 2e-12.

Every reference case

Each row is a published or constructed case. Labels say where it comes from: an official worked example or table, published software output, a textbook, a published example entered another way (for example as summary statistics), or our own constructed case.

CaseKindChecksResult
Fourteen reference values of the incomplete gamma and beta functions and the t distributionReference values from SciPy14Pass
APA sentence for a paired t-testOur constructed case1Pass
APA sentence for a Welch test with p below .001Our constructed case1Pass
APA sentence for a Welch test with two-decimal dfOur constructed case1Pass
NIST car mileage (AUTO83B), Student and Welch from summary statisticsOfficial worked example9Pass
NIST ZARR13 one-sample interval and critical valuesOfficial worked example8Pass
R: Welch test of 1 to 10 against 7 to 20Published software output5Pass
R: the same with an extreme value of 200 addedPublished software output5Pass
R: mtcars fuel economy by gearbox (Welch)Published software output5Pass
R: one-sample test on the sleep dataPublished software output6Pass
R: paired test on Student's sleep data, with Cohen's d_zPublished software output7Pass
Sleep data as two independent groups: Welch, Student, d_s and gOur case on published data, partly checked against published output12Pass
OpenStax Example 10.1, Welch from summary statisticsTextbook example5Pass
NIST critical values of t at df 1, 2, 5, 10, 30, 100 and infinityOfficial printed table54Pass
Paired sleep test entered as a summary of differencesPublished example entered another way4Pass
Paired sleep test from two means, two SDs and rPublished example entered another way4Pass
Paired sleep test, one-sided in the predicted directionPublished example entered another way2Pass
Paired sleep test, one-sided in the wrong directionPublished example entered another way2Pass
R's 1 to 10 against 7 to 20, entered as summariesPublished example entered another way2Pass
Student's test, pooled SD, d_s, g and Glass's deltaOur constructed case11Pass
mtcars Welch interval at 99% confidencePublished example entered another way2Pass
p from t: a NIST table value, t = 2.228 with 10 dfOfficial printed table3Pass
p from t: a NIST table value, t = 12.706 with 1 dfOfficial printed table3Pass
p from t: a NIST table value, t = 3.385 with 30 dfOfficial printed table3Pass
p from t: a NIST table value, t = 1.96 with one billion dfOfficial printed table3Pass
R: Wilcoxon signed rank, depression scores (V = 40)Published software output4Pass
R: Mann-Whitney, permeability constants (W = 35)Published software output3Pass
R: exact binomial test (used inside the rank test checks)Published software output1Pass

Where a source prints rounded values, we check that our value rounds to the printed digits; otherwise we compare full precision with SciPy. NIST's one-sample example prints t from unrounded data, so from its printed summaries our t agrees to five significant figures, and the OpenStax textbook's p of about 0.0054 is the two-sided value.

Sources

SourceRecheck groupChecked
R source code, stats package, t.test.Rreference softwareNovember 21, 2026
R reference example output, stats-Ex.Rout.savereference softwareNovember 21, 2026
R documentation: The Student t Distribution (TDist)reference softwareNovember 21, 2026
SciPy documentation: scipy.stats.ttest_indreference softwareNovember 21, 2026
SciPy documentation: scipy.stats.ttest_relreference softwareNovember 21, 2026
SciPy documentation: scipy.stats.mannwhitneyureference softwareNovember 21, 2026
SciPy documentation: scipy.stats.wilcoxonreference softwareNovember 21, 2026
SciPy documentation: scipy.special.betaincreference softwareNovember 21, 2026
NIST/SEMATECH e-Handbook of Statistical Methods, 1.3.5.3 Two-Sample t-Test for Equal MeanssourcesNovember 21, 2026
NIST/SEMATECH e-Handbook of Statistical Methods, 1.3.5.2 Confidence Limits for the MeansourcesNovember 21, 2026
NIST/SEMATECH e-Handbook of Statistical Methods, 1.3.6.7.2 Critical Values of the Student's t DistributionsourcesNovember 21, 2026
OpenStax, Introductory Statistics 2e, section 10.1: Two Population Means with Unknown Standard DeviationssourcesNovember 21, 2026
Lakens D. (2013) Calculating and reporting effect sizes to facilitate cumulative science. Frontiers in Psychology 4:863sourcesNovember 21, 2026
Delacre M., Lakens D., Leys C. (2017) Why psychologists should by default use Welch's t-test instead of Student's t-test. International Review of Social Psychology 30(1):92-101sourcesNovember 21, 2026
APA Style, Numbers and Statistics Guide (7th edition handout)styleNovember 21, 2026
Microsoft Support: T.TEST functionreference softwareNovember 21, 2026
IBM SPSS Statistics 29 documentation: Independent-Samples T Testreference softwareNovember 21, 2026
NIST Digital Library of Mathematical Functions, chapter 8 (incomplete gamma and beta functions)sourcesNovember 21, 2026

What we have not confirmed

These are not stated as fact anywhere on the site:

  • Whether the Excel Analysis ToolPak rounds Welch's df before computing p: not described on the site.
  • SPSS output row labels and menu names in current versions: not checked in SPSS; only IBM's documented description is used.
  • TI-84 2-SampTTest Pooled default: not checked on a calculator; the site does not say which setting is the default.
  • A-level and IB biology t-test formulas: specifications not checked; no curriculum option built.
  • Lakens d_rm equation and confidence intervals for d (noncentral t): d_rm not checked against the printed paper; CI for d is planned for a later phase.
  • Cohen 1988 benchmarks from the book itself: quoted via Lakens 2013 only, and always called rough conventions.

Corrections

If a result looks wrong, write to [email protected] with the inputs and the output you expected. We rerun the case, and a confirmed fix becomes a new test case.

Sources