cross-corpus gate instrument

Definition	The validation rule that an encoder votes only if its held-out AUROC on a corpus it was not trained on clears a preregistered margin over a bag-of-words null. Chapters 12 and 14. Also cross-corpus.
Example	The rights encoder needed a margin of 0.10 over a bag-of-words null on a corpus it never saw; it scored 0.509 against the null’s 0.512 and failed.
Book	Data Mining as Observation, draft 0.2, commit `f3914f0`; entry id `cross-corpus-gate`, kind instrument.
Status	no ledger row names this entry. Corrections: none recorded.
Defining equation	Book equation 12.3.
Assumptions and scope	An encoder votes only if its held-out AUROC on a corpus it was not trained on clears both nulls, the untrained encoder and the bag-of-words baseline, by the preregistered margin of one tenth. A validated encoder therefore beats each null by the margin. Since AUROC is at most one, a null above nine tenths cannot be cleared by any encoder, and the gate’s verdict is unchanged by any strictly monotone recalibration of the score. The rights encoder fell below the untrained baseline and was retired rather than tuned, and the within-corpus scores that had collapsed on a second corpus were the reason the gate is cross-corpus. The gate compares the AUROC as the encoder orients its score, so a reversed encoder fails rather than passing at 1 minus A, and the bag-of-words null is scored the same way.
Prior art	none recorded
Evidence	`lean/DataMiningAsObservation/CrossCorpusGate.lean`
Reviewed	semantic review 2026-09-06; generated 2026-09-10 from records at the commits on the provenance page.

An encoder votes only if its AUROC on a corpus it never saw clears the margin over a bag-of-words null.

Equation

Book equation 12.3.

\[\text{validated}\iff \mathrm{AUROC}_{\text{cross}}-\max\big(\mathrm{AUROC}_{\text{untrained}},\ \mathrm{AUROC}_{\text{BoW}}\big)\ \ge\ 0.10.\]

Conditions

An encoder votes only if its held-out AUROC on a corpus it was not trained on clears both nulls, the untrained encoder and the bag-of-words baseline, by the preregistered margin of one tenth. A validated encoder therefore beats each null by the margin.
Since AUROC is at most one, a null above nine tenths cannot be cleared by any encoder, and the gate’s verdict is unchanged by any strictly monotone recalibration of the score.
The rights encoder fell below the untrained baseline and was retired rather than tuned, and the within-corpus scores that had collapsed on a second corpus were the reason the gate is cross-corpus.
The gate compares the AUROC as the encoder orients its score, so a reversed encoder fails rather than passing at 1 minus A, and the bag-of-words null is scored the same way.

Conditions are curated in entries.toml rather than read from a record.

Ledger

none

First stated

The moral-embedding program, xbse/README.md:104-130,160-172, and chapter 12 section 12.5 and chapter 14 section 14.2 of Data Mining as Observation.

Measurements

Where the book states it	Numbers, as the book’s sources table records them	Source
chapter 8 section 8.6	rights encoder 0.467 below untrained baseline, the AUROC lesson, cross-corpus same-sign fix	`xbse/README.md:160-172`; `xbse/experiments/rights_r6_summary.json`
chapter 12 section 12.5	within 0.75 to 0.955 collapsing to 0.47 to 0.55, cross-corpus positives as the fix, adversary not load-bearing	`xbse\README.md:104-130,160-172`

Failures and corrections

none

Invariance envelope

none declared

Machine checked

lean/DataMiningAsObservation/CrossCorpusGate.lean, theorems clears_bow, clears_untrained, not_validated_of_saturated, margin_example, validated_comp, at observation-data-mining f3914f0; what the check covers is stated in the book’s appendix C.

Used in

Data Mining as Observation chapters 8, 12, 14.

reliability weight; harness; leakage; preregistration.

Status

Generated 2026-09-10 by encyclopedia/generate.py; book at observation-data-mining f3914f0; the commit of every record is listed in the encyclopedia’s provenance.

← coveragecross-entropy →