Proposition: Ambient effective-weight scores remain available

Let $\tilde{L}(B)$ be the same computational loss viewed as a function of the effective tensor. Its derivative at $B=W\circ M$ can be nonzero at an inactive coordinate, even though the derivative with respect to $W$ is zero there.

Recording the effective tensor's autograd derivative computes this ambient derivative. It does not apply a signed update or guarantee a loss decrease.

Proof. For

the effective derivative is

It can be nonzero at any coordinate with nonzero $x_i$. Equation (eq:maskedgradient) then zeros the corresponding stored derivative when $M_i=0$.