Psy01-A Phased Magnitude Pruning in Task-Gated ReLU Networks: Cue Certificates, Stability, and Conditional Selectivity–Fragility Trade-offs

Phased pruning simulations suggest that late connection removal can alter resistance to cue interference and vulnerability to internal noise. Turning these observations into mathematics requires distinguishing statements about a particular trained network from universal statements about an architecture class. We provide a self-contained analytic framework for the task-gated ReLU architecture used in the 067A model. We prove cue-utilization bounds, an indistinguishability obstruction for conflicting tasks, parameter perturbation estimates, Gaussian stress certificates, exact identities for nested magnitude pruning, and conditional concentration bounds. An explicit sparse construction disproves a universal cue-loss threshold near twenty percent density. Fixed-weight nested pruning is shown to agree with one-shot pruning at the same integer retention counts; schedule dependence instead requires learning, re-ranking, or a change of allocation rule. Hard finite-width pruning does not generally define a Lipschitz density surface. A differentiable relaxation and an exactly solved affine-Gaussian comparison model provide valid replacements, including a conditional, non-monotonic selectivity–fragility trade-off and an increasing maximizing curve. We also establish that identical masks and identical clean outputs need not determine identical stress tolerance. Cheung's landmark manuscripts inform the separation of construction, fluctuation, observation, and causal claims, but their quantum results are not identified with neural-network theorems. A source-code audit supplies exact-count pruning and random-stream repairs. No empirical density threshold, biological pruning mechanism, or treatment recommendation is asserted as a mathematical consequence.

Keywords: magnitude pruning; task cues; ReLU networks; Gaussian perturbations; margin certificates; developmental models; concentration; causal interpretation.

Cheung's phased-pruning study introduces a task-gated multilayer perceptron with early density manipulation, subsequent magnitude pruning, cue diagnostics, and hidden-activation noise CheungPruning. The supplied implementation 067a_code_sub_v2.py and accompanying manuscript Combined 067A Num v3 (1).docx are the specific objects audited here CheungSupplied067A.

The mathematical question is not whether one can attach theorem labels to the reported numerical tables. It is which conclusions follow from explicitly stated assumptions, which require a checkpoint-specific certificate, and which are false for the architecture class.

Probabilistic pruning theory already provides approximation results under specified weight laws QianKlabjan. Recent one-shot magnitude-pruning analysis likewise uses explicitly restricted models Koyuncu. These works do not establish a universal task-cue threshold or a developmental trade-off for the trained 067A network. The lottery-ticket literature also distinguishes pruning, initialization, and subsequent optimization FrankleCarbin.

principal corrections. The following distinctions organize this paper.

1. Density is a connection-count control, not a sufficient statistic for cue use or noise sensitivity.

2. A magnitude mask is an orthogonal coordinate projection in parameter space. It is not generally an orthogonal projection of hidden representations.

3. Fixed masks commute. Fixed-weight nested magnitude pruning also has an exact nesting identity.

4. Learning between pruning operations can change rankings and therefore invalidate that identity.

5. Accuracy is a discontinuous decision functional. Lipschitz control of logits does not, without a margin assumption, give Lipschitz control of accuracy.

6. A smooth surrogate pruning surface is a changed mathematical object, not the finite hard-pruning surface.

7. An ambiguous cue has no intrinsically preferred task. Accuracy against visual labels measures visual-rule retention, not task-neutral ambiguity resolution.

of the results. All propositions and theorems below are proved either directly or by an explicitly identified standard concentration inequality. The affine-Gaussian phase diagram is an exactly solved comparison model. Its applicability to a saved ReLU network is conditional on a stated approximation certificate. The supplied numerical values are treated as reported observations, not independently reproduced results.

Let

The input is $u=(x,c)$, with $x\in {\mathbb{R}}^2$ and $c\in {\mathbb{R}}^2$. For layer $\ell$, let

and let $b_{\ell }\in {\mathbb{R}}^{n_{\ell }}$ be an unpruned bias.

For independent standard Gaussian vectors $G_{\ell }\in {\mathbb{R}}^{n_{\ell }}$, define

Here $\theta =(B_{\ell },b_{\ell }{)}_{\ell =1}^4$. A deterministic tie convention, such as choosing the smallest maximizing class index, is fixed once and for all:

Equation (eq:forwardhidden) matches the supplied code: noise is added after each hidden ReLU, including during evaluation. The perturbed hidden state can therefore have negative coordinates.

The weight matrices contain

entries. There are $644$ bias parameters, so the total parameter count is $100484$.

The structural density is

Numerical density counts nonzero stored parameters, possibly including biases. It is a different quantity. An active mask position can contain a numerically zero weight.

Let $Y$ be uniform on $\{0,1,2,3\}$, and let

where

Define the fixed-point-free permutation

Thus ${\pi }^2$ is the identity and $\pi (y)\ne y$.

The visual task has label $Y$, whereas the voice task has label $\pi (Y)$. Their input distributions are identical. For theoretical comparisons it is useful to evaluate both rules on the same latent sample $(X,Y)$, even though the supplied implementation generates separate visual and voice test datasets.

The cues are

All probabilities below are fractions in $[0,1]$, not percentages. Unless stated otherwise, cue diagnostics use $\sigma =0$. Define

We abbreviate $A_{\theta }=A_{\theta }(1/2)$.

The name “ambiguity tolerance” is retained for correspondence with the simulation. Mathematically $A_{\theta }$ is visual-reference accuracy under a midpoint cue. Interpreting it as selectivity requires successful endpoint performance and an explicit task interpretation. It is not, by itself, a measure of cognitive flexibility or clinical resistance to distraction.

For each matrix, order its eligible positions by descending absolute weight, breaking ties by increasing flattened index. Let $H_K$ retain exactly the first $K$ eligible positions. Eligibility is restricted to the current mask, so deleted positions cannot be reactivated.

For an early target $d_e$, use

Removing a fraction $f$ of the surviving positions means

Consequently,

with equality zero possible at integer-aligned targets.

Thus $d_f=d_e(1-f)$ is the continuous target law. The exact finite-network density is determined by the integer counts.

Write $\vartheta$ for an augmented training state containing weights, masks, optimizer variables, and any required random-generator states. A specified training procedure is a map

where $U$ records the minibatch order and other exogenous choices.

The supplied training function creates a fresh Adam optimizer at each invocation. Hence its fine-tuning map includes an optimizer reset. Results for this map must not silently be transferred to a procedure that retains Adam moments.

A phased experiment has the form

The notation $A(d_e,f)$ or $S(d_e,f,\sigma )$ suppresses the baseline, training procedure, and random inputs. These hidden arguments matter.

For the task distribution above:

1. $$q_{\mathrm{a}}+q_{\mathrm{w}}\le 1,\Delta \ge q_{\mathrm{v}}+q_{\mathrm{a}}-1.$$

2. If the classifier is cue invariant on the test distribution, then

   $$q_{\mathrm{v}}+q_{\mathrm{a}}\le 1,\Delta =0.$$

   If it is also invariant under cue removal, then ${\Delta }_0=0$.

3. At the midpoint cue, the task-balanced accuracy satisfies

   $$\frac{1}{2}\mathbb{P}\{g_{\theta }(X,c_{1/2};0)=Y\}+\frac{1}{2}\mathbb{P}\{g_{\theta }(X,c_{1/2};0)=\pi (Y)\}\le \frac{1}{2}.$$

The same bounds hold for a randomized classifier if its randomization does not reveal the intended task.

Proof. For every realization of $(X,Y)$, the two events

are disjoint because $\pi (Y)\ne Y$. Their indicator sum is at most one. Taking expectations proves $q_{\mathrm{a}}+q_{\mathrm{w}}\le 1$, and rearrangement gives

If the predictions under $c_{\mathrm{v}}$ and $c_{\mathrm{a}}$ agree almost surely, then $q_{\mathrm{w}}=q_{\mathrm{v}}$, so $\Delta =0$, and the same disjointness argument gives $q_{\mathrm{v}}+q_{\mathrm{a}}\le 1$. Agreement with the zero-cue prediction gives ${\Delta }_0=0$.

At the midpoint cue a single prediction can equal at most one of $Y$ and $\pi (Y)$. Multiplying the corresponding indicator sum by $1/2$ proves (eq:balancedambiguous). Conditioning on the classifier's randomization proves the last assertion.

If

then

This is a certificate of cue-dependent rule use, not a theorem that a particular density must produce it.

For logits $z$ and true label $y$, define the signed margin

The margin is $2$-Lipschitz in the logit sup norm:

Split the first-layer matrix as

with $B_{1,c}\in {\mathbb{R}}^{256\times 2}$. Put

For the unstressed network,

Let $q(c)=\mathbb{P}\{g_{\theta }(X,c;0)=Y\}$. Then

In particular, this bounds both $\text{abs}\Delta$ and $\text{abs}{\Delta }_0$ by using $c^{\prime }=c_{\mathrm{a}}$ and $c^{\prime }=0$, respectively.

Proof. ReLU is $1$-Lipschitz in Euclidean norm. The first-layer difference is at most $\text{norm}{B_{1,c}(c-c^{\prime })}_2$, and propagation through the remaining layers gives the logit bound.

If the absolute signed margin exceeds $2\eta (c,c^{\prime })$, (eq:marginlip) prevents a change in the sign of the margin. Correctness therefore cannot change. Ties are included in the exceptional event. Taking expectations proves (eq:cuecertificate).

The condition $B_{1,c}(c_{\mathrm{v}}-c_{\mathrm{a}})=0$ makes the two endpoint representations equal, but need not make them equal to the zero-cue representation. The stronger condition $B_{1,c}=0$ eliminates all first-layer cue dependence. Downstream cancellation can also eliminate functional cue influence without eliminating cue weights.

There is a member of the stated architecture class, obtainable by layer-wise magnitude pruning to ten percent target density, with perfect endpoint switching on the noiseless cluster centers. For Gaussian cluster noise $\nu >0$, the same construction satisfies

In particular, cue use does not follow from density alone, and a universal bound $\Delta \le 1/2+O(\nu )$ at all densities below a threshold near $0.2$ is false as $\nu \to 0$.

Proof. Write ${\mu }_j=3(s_{j,1},s_{j,2})$ with $s_{j,i}\in \{-1,1\}$, and put $t=c_1-c_2$. Use eight first-layer units indexed by $j\in \{0,1,2,3\}$ and $s\in \{-1,1\}$:

At $x={\mu }_j$ and $t=s$, exactly the matching unit has value $3$; every other unit has value zero.

Relay these eight nonnegative coordinates through eight identity connections in each of the next two hidden layers. The head sends $(j,+1)$ to visual label $j$ and $(j,-1)$ to voice label $\pi (j)$. There are $32+8+8+8=56$ essential nonzero weights.

Now let $x={\mu }_j+\epsilon$. On

the matching unit remains positive and every other unit remains zero. Thus both tasks are correctly classified on $E$.

For independent ${\epsilon }_i\sim N(0,{\nu }^2)$, the variables ${\epsilon }_1+{\epsilon }_2$ and ${\epsilon }_1-{\epsilon }_2$ are independent $N(0,2{\nu }^2)$ variables, and

Therefore $\mathbb{P}(E)=p_{\nu }$. We have $q_{\mathrm{v}},q_{\mathrm{a}}\ge p_{\nu }$, and Theorem thm:cueobstruction gives $\Delta \ge 2p_{\nu }-1$.

It remains to realize the prescribed layer-wise counts. At ten percent density the counts are

Pad the essential connections with weights of magnitude $1/2$ in dormant units. First-layer dormant units have sufficiently negative biases to vanish on the compact set $E$ around all four centers. In the next two layers, place padding connections inside dormant-to-dormant blocks and keep their biases negative. In the head, padding uses dormant columns. The available block sizes exceed all required padding counts.

Give every position intended for deletion a magnitude strictly below $1/2$. Essential magnitudes are at least one. Layer-wise magnitude pruning therefore retains all essential connections and exactly the required padding. Padding contributes nothing on $E$. On noiseless centers, $p_0=1$ and $\Delta =1$. As $\nu \to 0$, $p_{\nu }\to 1$, disproving the proposed asymptotic upper bound.

The supplied table reports impaired voice performance at ten percent and successful endpoint switching at twenty percent. This identifies a transition between those tested settings for that run. It does not establish a universal threshold, and the proposed inequality for all $d_e\le 0.2$ is already inconsistent with the reported successful twenty-percent network.

For an independent Bernoulli retention mask with probability $p$, and a first-layer cue block containing $512$ nonzero entries, the probability that the entire block is deleted is

This calculation does not apply to trained magnitude masks, but it demonstrates why independent-mask concentration is not a proof that ten-percent density removes all cue channels.

Margin-based robustness arguments are standard Tsuzuku. Gaussian randomized smoothing also has a rigorous literature Cohen. The observable here is different: $S_{\theta }(\sigma )$ is ordinary accuracy averaged over hidden perturbations, not the certified adversarial accuracy of a smoothed classifier.

Put

with the usual empty-product convention.

For any input and Gaussian realization,

Let $H=n_1+n_2+n_3=640$, let $0<\delta <1$, and define

Then

The right side may be truncated below at zero.

Proof. Apply the $1$-Lipschitz property of ReLU recursively. The noise introduced at hidden layer $r$ is propagated through the downstream operator norms, giving (eq:stresspathbound).

For a standard scalar Gaussian $G$, $\mathbb{P}(\text{abs}G>r)\le 2e^{-r^2/2}$. A union bound over the $H$ hidden-noise coordinates shows that all coordinates have magnitude at most $R_{\delta }$ with probability at least $1-\delta$. On that event,

If the clean true-label margin exceeds $2\sigma R_{\delta }{C}_{\theta }$, equation (eq:marginlip) preserves correct classification. Subtracting the probability of the exceptional noise event proves (eq:globalstress).

This bound is conservative but applies to the full nonlinear network without assuming fixed activation gates.

For a fixed input $u$, let

where $h^0$ is the clean forward pass. Freeze these gates and define

Let

Then

where $G\in {\mathbb{R}}^{640}$ is standard Gaussian.

For $\ell =2,3$, define

Assume the deterministic preactivations at these gates are nonzero. Define

where a summand is zero if $v_{\ell ,i}=0$.

For $\sigma >0$, the noisy ReLU network and (eq:frozengaussian) agree with probability at least $1-{\epsilon }_{\mathrm{gate}}(u,\sigma )$.

For true label $y$, let

and put

For zero variance use $p_j=1\{m_j>0\}$. Assume no zero-variance pair has $m_j=0$; otherwise its deterministic tie is handled directly. Conditional stressed accuracy $s_{\theta }(u,y;\sigma )$ satisfies

Proof. The first hidden gate is unaffected by hidden noise because its input is deterministic. At a later layer, the frozen preactivation at coordinate $i$ has mean ${a_{\ell ,i}}^0$ and variance ${\sigma }^2{v_{\ell ,i}}^2$. Its sign differs from the clean sign with the probability appearing in (eq:gateerror).

If none of the frozen preactivation signs differs, induction over the layers shows that the actual and frozen networks coincide. A union bound proves the claimed agreement probability.

Each frozen pairwise margin is Gaussian:

Its probability of being positive is $p_j$. Correct classification requires all three pairwise margins to be positive. A union bound gives the lower bound $1-{\sum }_{j\ne y}(1-p_j)$, and containment in each pairwise event gives the upper bound ${\min }_{j\ne y}{p}_j$. Changing from the frozen to the actual network costs at most ${\epsilon }_{\mathrm{gate}}$. Truncation gives the displayed bounds.

Averaging these bounds over $(X,Y)$ yields population stress certificates. Crucially, a smaller lower bound is not proof of a smaller actual accuracy. A strict comparison needs an upper bound for one network and a lower bound for the other.

Let $L_0$ be an averaged lower bound for $S_{{\theta }_0}(\sigma )$, and let $U_1$ be an averaged upper bound for $S_{{\theta }_1}(\sigma )$, obtained from Theorem thm:frozengates or another valid certificate. If $U_1<L_0$, then

There are networks with identical masks and identical clean logits for every input, but different stressed accuracies at every fixed $\sigma >0$.

Proof. Disconnect the first two hidden layers from a selected third-layer unit by setting its incoming weights to zero. Give that unit bias $t>0$, so its clean activation is $t$ and its stressed activation is $t+\sigma G$. Set its weight to output class $0$ equal to $1/t$. Let the other class logits be $0,-1,-1$, using output biases.

For every input, the clean logits are

For a test distribution with label $0$, stressed correctness is equivalent to

and hence

All masks and all nonzero-position counts are unchanged as $t$ varies. Upstream noise cannot influence this output. The clean function is unchanged, but the stressed accuracy is strictly increasing in $t$.

The counterexample uses a simple test law to disprove a universal density-only assertion. It is not presented as a fit to the four-cluster experiment.

Let $D_1,D_2,D_3$ be positive diagonal matrices, with $D_0=I$. Transform a clean ReLU network by

The clean logits are unchanged.

Under the same isotropic noise intensity in the transformed coordinates, its stressed logits equal those of the original network with hidden noises

Thus preserving the clean function does not preserve a fixed-coordinate isotropic stress experiment.

Proof. For positive diagonal $D$, $\mathrm{ReLU}(Da)=D\mathrm{ReLU}(a)$. Induction gives $h_{\ell }^{\prime }=D_{\ell }{h}_{\ell }$ in the clean network and identical head logits. With noise, write $G_{\ell }=D_{\ell }({D_{\ell }}^{-1}{G}_{\ell })$ and apply the same induction.

Positive rescaling also changes weight magnitudes and can change the selected pruning mask while preserving the original clean function. A magnitude-pruning theorem therefore needs a specified parameterization, not just a specified input–output function.

The following model is a deliberately declared affine-Gaussian comparison. It is not a differentiable rewriting of the actual finite magnitude mask.

Let $e\in [0,1]$, $f\in [0,F]$, $0<F<1$, and set

where $0<e_0<1$ and $k,\lambda ,r,h_0,v_0>0$. For two conflicting rules, take the visual-versus-voice logit difference to be

where $Z,G$ are independent standard Gaussians and $\tau >0$. The visual label is selected when $L>0$.

At zero stress,

For the correct visual cue,

Assume

and

Under Assumption ass:comparison:

1. For $e\le e_0$, $A_{\mathrm{lin}}(e,f)$ is maximized at $f=0$.

2. For $e>e_0$, its unique maximizer is

   $$f^{\ast }(e)=\frac{k(e-e_0)}{2\lambda }\in (0,F).$$

   This curve is strictly increasing.

3. For $e>e_0$,

   $$A_{\mathrm{lin}}(e,f)>A_{\mathrm{lin}}(e,0)$$

   whenever

   $$0<f<\min \left\{F,\frac{k(e-e_0)}{\lambda }\right\}.$$

   The ambiguity surface is non-monotonic in $f$, with its peak at (eq:explicitpeak).

4. For every fixed $\sigma >0$, $S_{\mathrm{lin}}(e,f,\sigma )$ is strictly decreasing in $f$. It is also strictly decreasing as $e$ increases along the maximizing curve $f=f^{\ast }(e)$, $e>e_0$.

5. Endpoint cue use remains nonzero. More precisely, if

   $$\gamma =\underset{e,f}{\min }\{h(f)-\text{abs}b(e,f)\}>0,$$

   then the zero-stress cue drop satisfies

   $${\Delta }_{\mathrm{lin}}(e,f)\ge 2\Phi (\gamma /\tau )-1>0.$$

   The bound can be made arbitrarily close to one by increasing the separation $h_0/\tau$.

Proof. Because $\Phi$ is strictly increasing, maximizing $A_{\mathrm{lin}}$ is equivalent to maximizing

Its derivative is $k(e-e_0)-2\lambda f$, and its second derivative is $-2\lambda <0$. For $e\le e_0$, the maximum is at zero. For $e>e_0$, the stationary point is (eq:explicitpeak), and the first parameter condition puts it inside $(0,F)$. Also,

which proves the strict-gain interval.

Put

Condition (eq:cuepreservation) gives $\mu >0$. Furthermore,

Therefore ${\partial }_f(\mu /D)<0$, proving the fixed-$e$ stress ordering.

Along the maximizing curve,

The denominator $D(f^{\ast }(e))$ is strictly increasing. The ratio, and consequently $S_{\mathrm{lin}}$, is strictly decreasing along that curve.

Finally, the two endpoint probabilities against the visual reference are

The first is at least $\Phi (\gamma /\tau )$, and the second is at most $\Phi (-\gamma /\tau )$. Their difference proves (eq:comparisoncue).

Take

Then

so the assumptions hold. Increasing $h_0$ strengthens endpoint cue use without changing the maximizing curve.

The number $e_0$ is a parameter of the comparison model. Theorem thm:comparisontradeoff does not estimate a threshold for the trained 067A architecture. The term $kf(e-e_0)-\lambda f^2$ explicitly encodes an initial interference reduction followed by excessive-loss damage. The theorem proves the consequences of that mechanism rather than deriving it from magnitude ranking.

For two specified ReLU checkpoints ${\theta }_0,{\theta }_1$, suppose a declared comparison model gives observables ${A_i}^{\mathrm{cmp}},{S_i}^{\mathrm{cmp}}$, and valid error certificates give

If

and

then

Proof. Subtract the two accuracy inequalities:

Apply the same argument to $S_{{\theta }_0}-S_{{\theta }_1}$.

The errors must include activation-gate exits, approximation of the input distribution, and any reduction of a four-class decision to a single competing margin. A binary Gaussian formula alone is not a certificate for a four-class nonlinear network.

Consider two networks $\theta ,{\theta }^{\prime }$, evaluated on the same input and the same hidden Gaussian vectors. Define

For common stress intensity $\sigma$,

The expectation of the right side is finite for the stated Gaussian task distribution and fixed finite parameters.

Proof. For hidden layers, common additive noise cancels when the two states are subtracted. Hence

The same inequality, without ReLU, holds at the head. Starting from equal inputs and iterating proves (eq:parameterstability).

The Gaussian input has finite first moment. Recursively,

so all hidden first moments are finite.

For example,

have operator norms $\sqrt{2}$ and $(1+\sqrt{5})/2$, respectively. Thus a universal bound using only the unpruned operator norm is not justified by masking. The always valid bound $\text{norm}{W\circ M}_{\mathrm{op}}\le \text{norm}{W}_{\mathrm{F}}$ is weaker.

If $r_{\ell }$ entries of magnitude at most $q_{\ell }$ are removed from a frozen matrix, then

This provides a deterministic pruning-error input for Theorem thm:parameterstability, in the spirit of network approximation bounds QianKlabjan.

Let

under the common input/noise coupling, and let $Q_{\theta }$ denote the corresponding accuracy. For every $t>0$,

If $D\le \eta$ almost surely, then

If

then (eq:accuracyexpectation) yields a $O((\mathbb{E}D{)}^{\beta /(\beta +1)})$ modulus after optimizing $t$. Under the uniform bound and $\beta =1$, accuracy is Lipschitz in the certified logit perturbation.

For the bounded ramp

one has the unconditional estimate

Proof. Correctness can change only when $\text{abs}m(z_{\theta },Y)\le 2D$. This event is contained in

Markov's inequality gives (eq:accuracyexpectation). The uniform version is immediate. Substitution of $C{t}^{\beta }$ and minimization over $t$ gives the stated modulus.

The ramp has Lipschitz constant $1/\gamma$. Equation (eq:marginlip) gives

Taking expectations proves (eq:surrogatestability).

With frozen weights and exact retention counts, the hard-pruning map is a step function of the requested density. It can have nonzero jumps in logits and accuracy. Consequently a general Lipschitz theorem for $(d_e,f)\mapsto (A,S)$ is false without further assumptions.

Proof. The count $\lceil dN\rceil$ is locally constant between its integer thresholds and changes by one at a threshold. Retaining or deleting a nonzero weight therefore gives a parameter jump.

For an accuracy jump, consider a constant ReLU feature equal to one and two output logits

Retaining the positive output weight gives class $0$; deleting it gives class $1$. Extra inactive units and output classes embed the example in the stated architecture. For a class-$0$ test law, accuracy jumps from one to zero. Thus continuity, and hence a global Lipschitz bound, fails.

On a region containing no count transition, frozen hard pruning is constant, so a local Lipschitz statement there is trivial. Fine-tuning does not restore a general continuity theorem: its dependence on masks, rankings, and training inputs must also be controlled.

For fixed weights and temperature $T>0$, define

Choose $q_{\ell }(p)$ by

The left side is continuous, strictly decreasing from one to zero, so $q_{\ell }(p)$ exists uniquely. Define $B_{\ell ,i}(p)=W_{\ell ,i}{s}_{\ell ,i}(q_{\ell }(p))$.

On a compact parameter set with $p$ bounded away from zero and one, define

Then $r_{\ell }$ has a positive minimum, and

Moreover,

The ramp observables in Theorem thm:accuracystability are Lipschitz in $(d_e,f)$ on such compact sets, with constants obtained from (eq:parameterstability), (eq:surrogatestability), and (eq:softderivative).

Proof. Implicit differentiation of (eq:softcount) gives the formula for $q_{\ell }^{\prime }$. Each summand $s(1-s)$ is positive, and continuity on a compact interval gives a positive minimum for $r_{\ell }$. Differentiating the sigmoid and using $s(1-s)\le 1/4$ proves (eq:softderivative). The stability estimates then give the stated Lipschitz bound.