Training as part of the state

Write $\vartheta$ for an augmented training state containing weights, masks, optimizer variables, and any required random-generator states. A specified training procedure is a map

where $U$ records the minibatch order and other exogenous choices.

The supplied training function creates a fresh Adam optimizer at each invocation. Hence its fine-tuning map includes an optimizer reset. Results for this map must not silently be transferred to a procedure that retains Adam moments.

A phased experiment has the form

The notation $A(d_e,f)$ or $S(d_e,f,\sigma )$ suppresses the baseline, training procedure, and random inputs. These hidden arguments matter.