How the calculator is tested
How the numbers are computed, which published results they are checked against, and how closely they agree.
Sources checked on November 21, 2026. Results computed by the site's engine and tested against R and SciPy.
Numerical methods
The t distribution's tail areas come from the regularized incomplete beta function, evaluated with a continued fraction (modified Lentz method) in log space, following NIST's Digital Library of Mathematical Functions. The two-sided p uses x = df/(df + t²), so small tails come straight from the fraction without subtracting nearly equal numbers. Very large df (above ten million) use a second-order expansion about the normal distribution. Percentiles are found by Newton's method with a bracketing safeguard on the exact tail area, solved on the tail itself so tiny alphas keep their accuracy. If a method ever fails to converge, the calculator says it could not compute rather than show a wrong number.
Sums use compensated addition and variances use the two-pass method (mean first, then squared deviations), so data with large offsets keep their precision. Exact rank-test p-values count every arrangement, as R does.
Accuracy targets
Against SciPy 1.18.1 (which uses the Boost library), p-values must agree to a relative error of 1e-10 (1e-8 below 1e-100), and percentiles to 1e-10. The build stops if any check fails. In the current build the largest relative error across 8,100 stored SciPy values was about 2e-12.
Every reference case
Each row is a published or constructed case. Labels say where it comes from: an official worked example or table, published software output, a textbook, a published example entered another way (for example as summary statistics), or our own constructed case.
| Case | Kind | Checks | Result |
|---|---|---|---|
| Fourteen reference values of the incomplete gamma and beta functions and the t distribution | Reference values from SciPy | 14 | Pass |
| APA sentence for a paired t-test | Our constructed case | 1 | Pass |
| APA sentence for a Welch test with p below .001 | Our constructed case | 1 | Pass |
| APA sentence for a Welch test with two-decimal df | Our constructed case | 1 | Pass |
| NIST car mileage (AUTO83B), Student and Welch from summary statistics | Official worked example | 9 | Pass |
| NIST ZARR13 one-sample interval and critical values | Official worked example | 8 | Pass |
| R: Welch test of 1 to 10 against 7 to 20 | Published software output | 5 | Pass |
| R: the same with an extreme value of 200 added | Published software output | 5 | Pass |
| R: mtcars fuel economy by gearbox (Welch) | Published software output | 5 | Pass |
| R: one-sample test on the sleep data | Published software output | 6 | Pass |
| R: paired test on Student's sleep data, with Cohen's d_z | Published software output | 7 | Pass |
| Sleep data as two independent groups: Welch, Student, d_s and g | Our case on published data, partly checked against published output | 12 | Pass |
| OpenStax Example 10.1, Welch from summary statistics | Textbook example | 5 | Pass |
| NIST critical values of t at df 1, 2, 5, 10, 30, 100 and infinity | Official printed table | 54 | Pass |
| Paired sleep test entered as a summary of differences | Published example entered another way | 4 | Pass |
| Paired sleep test from two means, two SDs and r | Published example entered another way | 4 | Pass |
| Paired sleep test, one-sided in the predicted direction | Published example entered another way | 2 | Pass |
| Paired sleep test, one-sided in the wrong direction | Published example entered another way | 2 | Pass |
| R's 1 to 10 against 7 to 20, entered as summaries | Published example entered another way | 2 | Pass |
| Student's test, pooled SD, d_s, g and Glass's delta | Our constructed case | 11 | Pass |
| mtcars Welch interval at 99% confidence | Published example entered another way | 2 | Pass |
| p from t: a NIST table value, t = 2.228 with 10 df | Official printed table | 3 | Pass |
| p from t: a NIST table value, t = 12.706 with 1 df | Official printed table | 3 | Pass |
| p from t: a NIST table value, t = 3.385 with 30 df | Official printed table | 3 | Pass |
| p from t: a NIST table value, t = 1.96 with one billion df | Official printed table | 3 | Pass |
| R: Wilcoxon signed rank, depression scores (V = 40) | Published software output | 4 | Pass |
| R: Mann-Whitney, permeability constants (W = 35) | Published software output | 3 | Pass |
| R: exact binomial test (used inside the rank test checks) | Published software output | 1 | Pass |
Where a source prints rounded values, we check that our value rounds to the printed digits; otherwise we compare full precision with SciPy. NIST's one-sample example prints t from unrounded data, so from its printed summaries our t agrees to five significant figures, and the OpenStax textbook's p of about 0.0054 is the two-sided value.
Sources
What we have not confirmed
These are not stated as fact anywhere on the site:
- Whether the Excel Analysis ToolPak rounds Welch's df before computing p: not described on the site.
- SPSS output row labels and menu names in current versions: not checked in SPSS; only IBM's documented description is used.
- TI-84 2-SampTTest Pooled default: not checked on a calculator; the site does not say which setting is the default.
- A-level and IB biology t-test formulas: specifications not checked; no curriculum option built.
- Lakens d_rm equation and confidence intervals for d (noncentral t): d_rm not checked against the printed paper; CI for d is planned for a later phase.
- Cohen 1988 benchmarks from the book itself: quoted via Lakens 2013 only, and always called rough conventions.
Corrections
If a result looks wrong, write to [email protected] with the inputs and the output you expected. We rerun the case, and a confirmed fix becomes a new test case.
Sources
- NIST Digital Library of Mathematical Functions, chapter 8 (incomplete gamma and beta functions) checked November 21, 2026