Dimensions, masks, and the PyTorch cell

The source uses input dimension $d=2$, hidden dimension $H=384$, four output classes, and repeated sequences of length $T=20$.

With biases enabled, its parameter counts are

Thus

The configured feed-forward hidden_dims do not determine this GRU architecture.

For each weight tensor, let $M$ be its active mask and $S$ its permanent-scar mask. The admissibility condition is

Effective weights are stored with deleted coordinates zero. Biases are not pruned.

Split the effective input and recurrent weights into reset, update, and candidate blocks:

The packed tensors are ordered $r,\zeta ,n$. For $u\in {\mathbb{R}}^d$ and $h\in {\mathbb{R}}^H$, the actual cell is

These are the PyTorch GRUCell equations PyTorchGRU. In particular, the reset gate multiplies the hidden affine term including $b_{hn}$. The alternative expression $B_n(r\circ h)+b_{hn}$ is not substituted.

The candidate uses tanh and the gates use sigmoid even when the source switches its external activation to ReLU.