Proposition: Positive rescaling and stress calibration

Let $D_1,D_2,D_3$ be positive diagonal matrices, with $D_0=I$. Transform a clean ReLU network by

The clean logits are unchanged.

Under the same isotropic noise intensity in the transformed coordinates, its stressed logits equal those of the original network with hidden noises

Thus preserving the clean function does not preserve a fixed-coordinate isotropic stress experiment.

Proof. For positive diagonal $D$, $\mathrm{ReLU}(Da)=D\mathrm{ReLU}(a)$. Induction gives $h_{\ell }^{\prime }=D_{\ell }{h}_{\ell }$ in the clean network and identical head logits. With noise, write $G_{\ell }=D_{\ell }({D_{\ell }}^{-1}{G}_{\ell })$ and apply the same induction.