Manuscript Psy01-A@1 · Psy01-A.tex · theorem · 94477fa3

Theorem: Accuracy and surrogate stability

Stability, hard-mask discontinuity, and schedule dependence › Accuracy needs a margin assumption · lines 1219–1289 · proof included or linked

Psy01-A@1#thm:accuracystability, Psy01-A@1#eq:accuracyexpectation, Psy01-A@1#eq:accuracyuniform, Psy01-A@1#eq:surrogatestability

Back to manuscript

Let

D = \norm z_{θ} - z_{θ^{'}}_{\infty}

under the common input/noise coupling, and let $Q_{θ}$ denote the corresponding accuracy. For every $t > 0$ ,

\abs Q_{θ} - Q_{θ^{'}} \leq P {\abs m (z_{θ}, Y) \leq t} + \frac{2 E D}{t} .

If $D \leq η$ almost surely, then

\abs Q_{θ} - Q_{θ^{'}} \leq P {\abs m (z_{θ}, Y) \leq 2 η} .

P {\abs m (z_{θ}, Y) \leq t} \leq C t^{β}, β > 0,

then (eq:accuracyexpectation) yields a $O ((E D)^{β / (β + 1)})$ modulus after optimizing $t$ . Under the uniform bound and $β = 1$ , accuracy is Lipschitz in the certified logit perturbation.

For the bounded ramp

ρ_{γ} (m) = ⎩ ⎨ ⎧ 0, m / γ, 1, m \leq 0, 0 < m < γ, m \geq γ,

one has the unconditional estimate

\abs E ρ_{γ} (m_{θ}) - E ρ_{γ} (m_{θ^{'}}) \leq \frac{2}{γ} E D .

Proof. Correctness can change only when $\abs m (z_{θ}, Y) \leq 2 D$ . This event is contained in

{\abs m (z_{θ}, Y) \leq t} \cup {D > t /2} .

Markov's inequality gives (eq:accuracyexpectation). The uniform version is immediate. Substitution of $C t^{β}$ and minimization over $t$ gives the stated modulus.

The ramp has Lipschitz constant $1/ γ$ . Equation (eq:marginlip) gives

\abs ρ_{γ} (m_{θ}) - ρ_{γ} (m_{θ^{'}}) \leq 2 D / γ .

Taking expectations proves (eq:surrogatestability).

Displayed formulas in this passage

Psy01-A@1#eq:accuracyexpectation · line 1227
$\abs Q_{θ} - Q_{θ^{'}} \leq P {\abs m (z_{θ}, Y) \leq t} + \frac{2 E D}{t} .$
unlabeled · line 1222
$D = \norm z_{θ} - z_{θ^{'}}_{\infty}$
Psy01-A@1#eq:accuracyuniform · line 1235
$\abs Q_{θ} - Q_{θ^{'}} \leq P {\abs m (z_{θ}, Y) \leq 2 η} .$
unlabeled · line 1242
$P {\abs m (z_{θ}, Y) \leq t} \leq C t^{β}, β > 0,$
unlabeled · line 1252
$ρ_{γ} (m) = ⎩ ⎨ ⎧ 0, m / γ, 1, m \leq 0, 0 < m < γ, m \geq γ,$
Psy01-A@1#eq:surrogatestability · line 1261
$\abs E ρ_{γ} (m_{θ}) - E ρ_{γ} (m_{θ^{'}}) \leq \frac{2}{γ} E D .$
unlabeled · line 1273
${\abs m (z_{θ}, Y) \leq t} \cup {D > t /2} .$
unlabeled · line 1284
$\abs ρ_{γ} (m_{θ}) - ρ_{γ} (m_{θ^{'}}) \leq 2 D / γ .$