standard error concept

Definition	The spread of an estimate across repeated samples. Chapter 0 section 0.9.
Example	A spread of 0.4 over 16 seeds gives standard error 0.1, and over 64 seeds 0.05.
Book	Data Mining as Observation, draft 0.2, commit `f3914f0`; entry id `standard-error`, kind concept.
Status	no ledger row names this entry. Corrections: none recorded.
Defining equation	Book equation 8.2.
Assumptions and scope	The spread of an estimate across repeated samples, the spread of one draw over the square root of the number of independent draws. It is positive, falls as the draws grow, halves only when the draws quadruple, and tends to zero. Dependent draws do not shrink it this way. A thousand repeated folds shrink the naive standard error by a factor near 31.6 and the corrected one by 15.84 less, which is the Nadeau and Bengio correction.
Prior art	none recorded
Evidence	`lean/DataMiningAsObservation/StandardError.lean`
Reviewed	not yet reviewed; generated 2026-09-10 from records at the commits on the provenance page.

The spread of an estimate over repetitions, falling as one over root n.

Equation

Book equation 8.2.

\[\begin{gathered} \widehat{\operatorname{Var}}_{\mathrm{NB}}=\Big(\frac1J+\frac{n_{\mathrm{test}}}{n_{\mathrm{train}}}\Big)\hat\sigma^{2}, \\ J=1000,\ \frac{n_{\mathrm{test}}}{n_{\mathrm{train}}}=\frac14\ \Rightarrow\ 1+250=251,\ \ \sqrt{251}=15.84. \end{gathered}\]

Conditions

The spread of an estimate across repeated samples, the spread of one draw over the square root of the number of independent draws. It is positive, falls as the draws grow, halves only when the draws quadruple, and tends to zero.
Dependent draws do not shrink it this way. A thousand repeated folds shrink the naive standard error by a factor near 31.6 and the corrected one by 15.84 less, which is the Nadeau and Bengio correction.

Conditions are curated in entries.toml rather than read from a record.

Ledger

none

First stated

Chapter 0 section 0.9 of Data Mining as Observation, with the program’s own inflation in constraint-gap/review/FINDINGS.md:1-35.

Measurements

Where the book states it	Numbers, as the book’s sources table records them	Source
chapter 8 section 8.5	variance inflation 251, SE inflation 15.84, J_eff 3.98, about 62 needed, 3 of 9 at full, 7 at half, 8 at a third	`constraint-gap/review/FINDINGS.md:1-35`

Failures and corrections

none

Invariance envelope

none declared

Machine checked

lean/DataMiningAsObservation/StandardError.lean, theorems se_pos, se_quarter, se_antitone, se_tendsto_zero, thousand_folds, at observation-data-mining f3914f0; what the check covers is stated in the book’s appendix C.

Used in

Data Mining as Observation primer S, chapters 0, 6, 7, 8, 13, 14.

Nadeau and Bengio correction; harness; multiple comparisons; preregistration.

Status

Generated 2026-09-10 by encyclopedia/generate.py; book at observation-data-mining f3914f0; the commit of every record is listed in the encyclopedia’s provenance.

← standard deviationstandardization →