Proposition: Finite-sample coverage for the declared score

Suppose each actual output law is within total variation $e$ of the specified uniform ideal law, and each recorded score approximates $W_f$ with deterministic error at most $\kappa$. For independent repetitions with the same actual law, put

Then, with probability at least $1-\alpha$,

An additional calibration-failure probability $\beta$ enlarges the failure probability to at most $\alpha +\beta$.

Proof. A function in $[0,1]$ changes in expectation by at most the total variation distance. Score rounding changes the sample mean by at most $\kappa$.

For completeness, for any $W\in [0,1]$, the log moment generating function of $W-\mathbb{E}W$ has value and derivative zero at zero. Its second derivative is a variance under an exponentially tilted law, at most $1/4$. Hence

Independence, exponential Markov inequality, and optimization over $t$ give

Taking $r=\sqrt{\ln (2/\alpha )/(2N)}$, then adding the model and rounding allowances, proves (eq:coverage). The calibration assertion is a union bound. This is Hoeffding's bounded-variable argument Hoeffding.