permutation test concept
| Definition | A test that computes the p-value by reassigning the labels under the null, every way or a few thousand random ways, and counting the fraction of statistics at least as extreme as the observed. It needs no assumption about the shape of the data, and the sample size fixes the smallest p-value it can give. Primer S, equation S.15. Also permut. |
|---|---|
| Example | Groups (8, 9) and (4, 5) have six reassignments, two with a difference of at least 4 in size, so the two-sided p-value is one third. |
| Book | Data Mining as Observation, draft 0.2, commit f3914f0; entry id permutation-test, kind concept. |
| Status | no ledger row names this entry. Corrections: none recorded. |
| Defining equation | Book equation S.15. |
| Assumptions and scope | none |
| Prior art | none recorded |
| Evidence | none |
| Reviewed | semantic review 2026-09-09; generated 2026-09-10 from records at the commits on the provenance page. |
Equation
Book equation S.15.
\[p=\Pr\big[\,|T|\ge|t_{\mathrm{obs}}|\ \big|\ \text{null}\,\big]=\frac{\#\{\text{reassignments with } |T|\ge|t_{\mathrm{obs}}|\}}{\#\{\text{reassignments}\}}.\]
Conditions
none
Ledger
none
First stated
Primer S section S.7 of Data Mining as Observation, added in draft 0.3 (2026-09-09) for the ECE 514 readers whose first courses are far behind. The idea is standard and TSK Appendix C covers it at length.
Measurements
none
Failures and corrections
none
Invariance envelope
none declared
Machine checked
none
Used in
Data Mining as Observation primer S, chapters 0, 7, 8, 9, 14.
Related
null hypothesis; p-value; null model; seed; type I and type II error, power.
See also
none
Status
Generated 2026-09-10 by encyclopedia/generate.py; book
at observation-data-mining f3914f0; the commit of every record is listed
in the encyclopedia’s provenance.