Each observation influences a cross-validation estimate through both validation and model training. This work uses those two roles to study how the learning procedure, performance metric, and resampling design contribute to variance. The framework compares balanced and unbalanced designs and clarifies what changing the number of folds or repetitions can change about uncertainty.
This manuscript is in preparation. A public manuscript link will be added when available.