The augmented intervention state

The source creates a fresh Adam optimizer in each training function. An exact description therefore includes these resets. A phase-boundary checkpoint can omit discarded optimizer moments; a checkpoint intended to resume a live optimizer cannot.

Ordinary Python modulation attributes and masks held by a separate manager are not automatically included in a module's state_dict. Parameters and persistent buffers are included, and the returned dictionary must be cloned when an immutable snapshot is intended PyTorchModule.