# Welcome to Randal Douc's wiki

A collaborative site on maths but not only!

• Theatre
• Research
• Teaching

world:pinsker

# The Pinsker Inequality.


2017/10/07 23:39 ·

## What?

The inequality is the following:

The Pinsker inequality. For all probability measures $\sigma$, $\pi$, $$|\int f (\rmd \sigma-\rmd \pi)|^2\leq 2 \Var_{\mu}(f) KL(\sigma|| \pi)$$ where $\mu$ is the probability measure $\mu=2\pi/3+\sigma/3$.

If $0\leq f\leq 1$, then $\Var_{\mu}(f)=\PE[f^2]-\PE^2[f]\leq \PE[f]-\PE^2[f]=\PE[f](1-\PE[f]) \leq 1/4$ and we get $$\rmd_{tv}(\sigma,\pi) \leq KL(\sigma|| \pi)/2$$

## The proof

$\bproof$ Here are the main ideas. For simplicity, we assume $\sigma(x)>0$ for all $x$. First note that for all $y> 0$,

$$(y-1)^2 \leq \frac{2}{3}(2y+1) (-\log y +y-1) \quad (\star)$$

Indeed, set $h(y)=\frac{2}{3}(2y+1)(-\log y+y-1)-(y-1)^2$ and check that $h(1)=h'(1)=0$ and $h''(y)=2(x-1)^2/(3y^2)>0$ for all $y \geq 0$. Therefore, $h$ is a convex function on $\rset^+$ and attains its minimum at $y=1$. Therefore, $h(y) \geq 0$ for all $y\geq 0$, which proves $(\star)$. Then, using $(\star)$ with $y=\frac{\pi(x)}{\sigma(x)}$, and taking $\bar f=f-\mu(f)$ where $\mu=2\pi/3 +\sigma/3$,

\begin{align*} |\int f (\rmd \sigma-\rmd \pi)|^2=|\int \bar f (\rmd \sigma-\rmd \pi)|^2 &\leq \left(\int |\bar f(x)|\ |\sigma(x)-\pi(x)|\rmd x\right)^2 \\ &=\left(\int |\bar f(x)|\ \sigma(x)\ |\frac{\pi(x)}{\sigma(x)}-1|\rmd x\right)^2 \\ &=\left(\int \sigma(\rmd x) \sqrt{\frac{2\bar f^2(x)}{3}\left(2\frac{\pi(x)}{\sigma(x)}+1\right)} \sqrt{\left( -\log \frac{\pi(x)}{\sigma(x)} +\frac{\pi(x)}{\sigma(x)}-1\right)}\right)^2 \end{align*} And we conclude by applying Holder's inequality: $\left(\int \sigma(\rmd x) \sqrt{a(x)}\sqrt{b(x)} \right)^2\leq \int \sigma(\rmd x) a(x) \times \int \sigma(\rmd x) b(x)$ $\eproof$