A companion to the variance theory, this work develops a way to estimate uncertainty by rerunning the learning and evaluation procedure on outcomes generated from a fitted conditional model. It distinguishes uncertainty conditional on the observed covariates from uncertainty that also includes sampling those covariates, and examines the assumptions needed for useful performance intervals and model comparisons.
This manuscript is in preparation. A public manuscript link will be added when available.