Theorem: No universal twenty-percent cue-loss threshold

There is a member of the stated architecture class, obtainable by layer-wise magnitude pruning to ten percent target density, with perfect endpoint switching on the noiseless cluster centers. For Gaussian cluster noise $\nu >0$, the same construction satisfies

In particular, cue use does not follow from density alone, and a universal bound $\Delta \le 1/2+O(\nu )$ at all densities below a threshold near $0.2$ is false as $\nu \to 0$.

Proof. Write ${\mu }_j=3(s_{j,1},s_{j,2})$ with $s_{j,i}\in \{-1,1\}$, and put $t=c_1-c_2$. Use eight first-layer units indexed by $j\in \{0,1,2,3\}$ and $s\in \{-1,1\}$:

At $x={\mu }_j$ and $t=s$, exactly the matching unit has value $3$; every other unit has value zero.

Relay these eight nonnegative coordinates through eight identity connections in each of the next two hidden layers. The head sends $(j,+1)$ to visual label $j$ and $(j,-1)$ to voice label $\pi (j)$. There are $32+8+8+8=56$ essential nonzero weights.

Now let $x={\mu }_j+\epsilon$. On

the matching unit remains positive and every other unit remains zero. Thus both tasks are correctly classified on $E$.

For independent ${\epsilon }_i\sim N(0,{\nu }^2)$, the variables ${\epsilon }_1+{\epsilon }_2$ and ${\epsilon }_1-{\epsilon }_2$ are independent $N(0,2{\nu }^2)$ variables, and

Therefore $\mathbb{P}(E)=p_{\nu }$. We have $q_{\mathrm{v}},q_{\mathrm{a}}\ge p_{\nu }$, and Theorem thm:cueobstruction gives $\Delta \ge 2p_{\nu }-1$.

It remains to realize the prescribed layer-wise counts. At ten percent density the counts are

Pad the essential connections with weights of magnitude $1/2$ in dormant units. First-layer dormant units have sufficiently negative biases to vanish on the compact set $E$ around all four centers. In the next two layers, place padding connections inside dormant-to-dormant blocks and keep their biases negative. In the head, padding uses dormant columns. The available block sizes exceed all required padding counts.

Give every position intended for deletion a magnitude strictly below $1/2$. Essential magnitudes are at least one. Layer-wise magnitude pruning therefore retains all essential connections and exactly the required padding. Padding contributes nothing on $E$. On noiseless centers, $p_0=1$ and $\Delta =1$. As $\nu \to 0$, $p_{\nu }\to 1$, disproving the proposed asymptotic upper bound.