Theorem: Frozen-gate response with an explicit exit error

For $\sigma >0$, the noisy ReLU network and (eq:frozengaussian) agree with probability at least $1-{\epsilon }_{\mathrm{gate}}(u,\sigma )$.

For true label $y$, let

and put

For zero variance use $p_j=1\{m_j>0\}$. Assume no zero-variance pair has $m_j=0$; otherwise its deterministic tie is handled directly. Conditional stressed accuracy $s_{\theta }(u,y;\sigma )$ satisfies

Proof. The first hidden gate is unaffected by hidden noise because its input is deterministic. At a later layer, the frozen preactivation at coordinate $i$ has mean ${a_{\ell ,i}}^0$ and variance ${\sigma }^2{v_{\ell ,i}}^2$. Its sign differs from the clean sign with the probability appearing in (eq:gateerror).

If none of the frozen preactivation signs differs, induction over the layers shows that the actual and frozen networks coincide. A union bound proves the claimed agreement probability.

Each frozen pairwise margin is Gaussian:

Its probability of being positive is $p_j$. Correct classification requires all three pairwise margins to be positive. A union bound gives the lower bound $1-{\sum }_{j\ne y}(1-p_j)$, and containment in each pairwise event gives the upper bound ${\min }_{j\ne y}{p}_j$. Changing from the frozen to the actual network costs at most ${\epsilon }_{\mathrm{gate}}$. Truncation gives the displayed bounds.