Manuscript Psy03-C@1 · Psy03-C-v1.tex · proposition · d3380b90

Proposition: Tube-averaged Jacobian products

Gate-aware recurrent stability › Finite-horizon propagation · lines 678–709 · proof included or linked

Psy03-C@1#prop:jacproduct

Suppose the unprimed update maps are continuously differentiable on the state comparison segments. This holds for external tanh and for a verified nonzero ReLU gate tube. Set

A_{t} = \int_{0}^{1} D Ψ_{t} (h_{t - 1}^{'} + v (h_{t - 1} - h_{t - 1}^{'})) d v,

and

e_{t} = Ψ_{t} (h_{t - 1}^{'}) - Ψ_{t}^{'} (h_{t - 1}^{'}) .

Then exactly

h_{t} - h_{t}^{'} = A_{t} (h_{t - 1} - h_{t - 1}^{'}) + e_{t} .

Consequently,

d_{T} \leq \norm A_{T} \dots A_{1} d_{0} + t = 1 \sum^{T} \norm A_{T} \dots A_{t + 1} \norm e_{t}_{2} .

Proof. Apply the fundamental theorem of calculus on each state segment and iterate the resulting vector identity.

Displayed formulas in this passage

unlabeled · line 685
$A_{t} = \int_{0}^{1} D Ψ_{t} (h_{t - 1}^{'} + v (h_{t - 1} - h_{t - 1}^{'})) d v,$
unlabeled · line 690
$e_{t} = Ψ_{t} (h_{t - 1}^{'}) - Ψ_{t}^{'} (h_{t - 1}^{'}) .$
unlabeled · line 694
$h_{t} - h_{t}^{'} = A_{t} (h_{t - 1} - h_{t - 1}^{'}) + e_{t} .$
unlabeled · line 698
$d_{T} \leq \norm A_{T} \dots A_{1} d_{0} + t = 1 \sum^{T} \norm A_{T} \dots A_{t + 1} \norm e_{t}_{2} .$

Proposition: Tube-averaged Jacobian products

Displayed formulas in this passage

Nearby in this section