Manuscript Psy03-C@1 · Psy03-C-v1.tex · theorem · d3380b90

Theorem: Gate-aware cell Lipschitz constants

Gate-aware recurrent stability › Explicit constants · lines 533–582 · proof included or linked

Psy03-C@1#thm:celllip, Psy03-C@1#eq:celllip, Psy03-C@1#eq:Lconstants

On the declared region,

\norm F_{θ} (u, h) - F_{θ} (u^{'}, h^{'})_{2} \leq L_{x} \norm u - u^{'}_{2} + L_{h} \norm h - h^{'}_{2},

where

L_{x} = L_{h} = (1 - ζ) (\norm A_{n} + \frac{C _{R}}{4} \norm A_{r}) + \frac{R + 1}{4} \norm A_{ζ}, ζ + (1 - ζ) (\norm B_{n} + \frac{C _{R}}{4} \norm B_{r}) + \frac{R + 1}{4} \norm B_{ζ} .

With coarse gate bounds, these constants are valid for all inputs $u, u^{'}$ , even if their domain is unbounded.

Proof. Sigmoid is $1/4$ -Lipschitz and tanh is $1$ -Lipschitz. Writing $w = B_{n} h + b_{hn}$ , the candidate arguments differ by

A_{n} (u - u^{'}) + r \circ B_{n} (h - h^{'}) + (r - r^{'}) \circ w^{'} .

The last term is bounded by $C_{R} \norm r - r^{'}_{2}$ . This gives the candidate constants

N_{x} = \norm A_{n} + C_{R} \norm A_{r} /4, N_{h} = \norm B_{n} + C_{R} \norm B_{r} /4.

The exact output decomposition is

F - F^{'} = ζ \circ (h - h^{'}) + (1 - ζ) \circ (n - n^{'}) + (ζ - ζ^{'}) \circ (h^{'} - n^{'}) .

Since $\norm h^{'} - n^{'}_{\infty} \leq R + 1$ , substitution gives (eq:Lconstants). The coarse bounds do not restrict input magnitude.

Displayed formulas in this passage

Psy03-C@1#eq:Lconstants · line 542
$L_{x} = L_{h} = (1 - ζ) (\norm A_{n} + \frac{C _{R}}{4} \norm A_{r}) + \frac{R + 1}{4} \norm A_{ζ}, ζ + (1 - ζ) (\norm B_{n} + \frac{C _{R}}{4} \norm B_{r}) + \frac{R + 1}{4} \norm B_{ζ} .$
Psy03-C@1#eq:celllip · line 536
$\norm F_{θ} (u, h) - F_{θ} (u^{'}, h^{'})_{2} \leq L_{x} \norm u - u^{'}_{2} + L_{h} \norm h - h^{'}_{2},$
unlabeled · line 561
$A_{n} (u - u^{'}) + r \circ B_{n} (h - h^{'}) + (r - r^{'}) \circ w^{'} .$
unlabeled · line 567
$N_{x} = \norm A_{n} + C_{R} \norm A_{r} /4, N_{h} = \norm B_{n} + C_{R} \norm B_{r} /4.$
unlabeled · line 573
$F - F^{'} = ζ \circ (h - h^{'}) + (1 - ζ) \circ (n - n^{'}) + (ζ - ζ^{'}) \circ (h^{'} - n^{'}) .$