Manuscript Psy01-A@1 · Psy01-A.tex · definition · 94477fa3

Definition: Structural and numerical density

Definitions and exact pruning operators › Architecture and stochastic forward map · lines 239–248

The structural density is

d_{mask} = \frac{\sum _{ℓ = 1} ^{4} \norm M _{ℓ} _{0}}{99840} .

Numerical density counts nonzero stored parameters, possibly including biases. It is a different quantity. An active mask position can contain a numerically zero weight.

Displayed formulas in this passage

unlabeled · line 241
$d_{mask} = \frac{\sum _{ℓ = 1} ^{4} \norm M _{ℓ} _{0}}{99840} .$

Nearby in this section

section: Architecture and stochastic forward map