Manuscript Oai119-A@1 · Oai119-A.tex · lemma · fdff7292

Lemma: The classical information identity

Stage 0: objects, normalizations, and claim status › Three different scalar functions · lines 208–249 · proof included or linked

Oai119-A@1#lem:infoidentity, Oai119-A@1#eq:MI

Back to manuscript

Given $u : Ω_{n} \to [- 1, 1]$ , construct a sign $U$ with

P (U = b ∣ X = x) = \frac{1 + b u ( x )}{2}, b \in {- 1, 1},

using auxiliary randomness independent of the noise conditional on $X$ . Then

I (U; X) = I (u), I (U; Y) = I (T_{ρ} u) .

For Boolean $f$ , one may take $U = f (X)$ , and

I_{2} (f (X); Y) = h_{2} (\frac{1 + m}{2}) - E h_{2} (\frac{1 + T _{ρ} f}{2}), m = E f .

Proof. The conditional mean of $U$ given $X$ is $u (X)$ , so its conditional entropy is $E h (u)$ , while its mean is $E u$ . This proves the first identity. Conditional expectation and the stated conditional independence give

E [U ∣ Y] = E [u (X) ∣ Y] = T_{ρ} u (Y) .

The same entropy calculation proves the second identity. For Boolean $f$ , $U = f (X)$ has zero conditional entropy given $X$ , and division by $ℓ$ gives (eq:MI).

Displayed formulas in this passage

Oai119-A@1#eq:MI · line 223
$I_{2} (f (X); Y) = h_{2} (\frac{1 + m}{2}) - E h_{2} (\frac{1 + T _{ρ} f}{2}), m = E f .$
unlabeled · line 211
$P (U = b ∣ X = x) = \frac{1 + b u ( x )}{2}, b \in {- 1, 1},$
unlabeled · line 217
$I (U; X) = I (u), I (U; Y) = I (T_{ρ} u) .$
unlabeled · line 239
$E [U ∣ Y] = E [u (X) ∣ Y] = T_{ρ} u (Y) .$

Lemma: The classical information identity

Displayed formulas in this passage

Nearby in this section