Architecture and stochastic forward map

Let

The input is $u=(x,c)$, with $x\in {\mathbb{R}}^2$ and $c\in {\mathbb{R}}^2$. For layer $\ell$, let

and let $b_{\ell }\in {\mathbb{R}}^{n_{\ell }}$ be an unpruned bias.

For independent standard Gaussian vectors $G_{\ell }\in {\mathbb{R}}^{n_{\ell }}$, define

Here $\theta =(B_{\ell },b_{\ell }{)}_{\ell =1}^4$. A deterministic tie convention, such as choosing the smallest maximizing class index, is fixed once and for all:

Equation (eq:forwardhidden) matches the supplied code: noise is added after each hidden ReLU, including during evaluation. The perturbed hidden state can therefore have negative coordinates.

The weight matrices contain

entries. There are $644$ bias parameters, so the total parameter count is $100484$.