Manuscript Psy03-A@1 · Psy03-A.tex · lemma · b30eb6e9

Lemma: Layerwise propagation

Stress, margins, and complete perturbation budgets › A general path lemma and the damped certificate · lines 575–597 · proof included or linked

Psy03-A@1#lem:path

Suppose two computations with the same matrices and activations have hidden differences satisfying

d_{ℓ} \leq a \norm B_{ℓ}_{op} d_{ℓ - 1} + e_{ℓ}, ℓ = 1, 2, 3.

Then

\norm z - z^{'}_{2} \leq a^{3} (ℓ = 1 \prod^{4} \norm B_{ℓ}_{op}) d_{0} + r = 1 \sum^{3} a^{3 - r} (ℓ = r + 1 \prod^{4} \norm B_{ℓ}_{op}) e_{r} .

Proof. Iterate the three scalar recurrences and multiply the final hidden difference by $\norm B_{4}_{op}$ .

Displayed formulas in this passage

unlabeled · line 579
$d_{ℓ} \leq a \norm B_{ℓ}_{op} d_{ℓ - 1} + e_{ℓ}, ℓ = 1, 2, 3.$
unlabeled · line 584
$\norm z - z^{'}_{2} \leq a^{3} (ℓ = 1 \prod^{4} \norm B_{ℓ}_{op}) d_{0} + r = 1 \sum^{3} a^{3 - r} (ℓ = r + 1 \prod^{4} \norm B_{ℓ}_{op}) e_{r} .$