Manuscript Psy03-D@1 · Psy03-D-v1.tex · proposition · 3e18951d

Proposition: Numerical permanence does not prohibit future regrowth

Masked gradients and reference-preserving repair › Zeroing is not permanent eligibility exclusion · lines 1431–1447 · proof included or linked

Psy03-D@1#prop:remove

Back to manuscript

Removing a masking reparameterization while retaining its current zeroed tensor does not by itself impose a permanent zero constraint under later optimization. There are losses for which a zeroed coordinate immediately becomes nonzero under a subsequent gradient step.

Proof. After mask removal, take a scalar zero parameter and loss

A gradient step of size gives .

Displayed formulas in this passage

  • unlabeled · line 1443

Nearby in this section