Manuscript Psy01-A@1 · Psy01-A.tex · remark · 94477fa3

Remark: A spectral norm need not decrease after masking

Stability, hard-mask discontinuity, and schedule dependence › Logit perturbation · lines 1190–1204

For example,

W = (11 1 - 1), W \circ M = (10 1 - 1)

have operator norms $2$ and $(1 + 5) /2$ , respectively. Thus a universal bound using only the unpruned operator norm is not justified by masking. The always valid bound $\norm W \circ M_{op} \leq \norm W_{F}$ is weaker.

Displayed formulas in this passage

unlabeled · line 1192
$W = (11 1 - 1), W \circ M = (10 1 - 1)$

Nearby in this section

section: Logit perturbation
theorem: Theorem: Parameter-to-logit stability
section: Logit perturbation