Theorem: Cue-sensitivity certificate

For the unstressed network,

Let $q(c)=\mathbb{P}\{g_{\theta }(X,c;0)=Y\}$. Then

In particular, this bounds both $\text{abs}\Delta$ and $\text{abs}{\Delta }_0$ by using $c^{\prime }=c_{\mathrm{a}}$ and $c^{\prime }=0$, respectively.

Proof. ReLU is $1$-Lipschitz in Euclidean norm. The first-layer difference is at most $\text{norm}{B_{1,c}(c-c^{\prime })}_2$, and propagation through the remaining layers gives the logit bound.

If the absolute signed margin exceeds $2\eta (c,c^{\prime })$, (eq:marginlip) prevents a change in the sign of the margin. Correctness therefore cannot change. Ties are included in the exceptional event. Taking expectations proves (eq:cuecertificate).