Wiki
Wiki
Courses and public working groups
Courses and public working groups
Private Working Groups
Private Working Groups
- New!!! Reading Group
- Theatre
- Admin
- Research
- Teaching
User Tools
Sidebar
world:de-finetti
This is an old revision of the document!
Table of Contents
on
$$ \newcommand{\arginf}{\mathrm{arginf}} \newcommand{\argmin}{\mathrm{argmin}} \newcommand{\argmax}{\mathrm{argmax}} \newcommand{\asconv}[1]{\stackrel{#1-a.s.}{\rightarrow}} \newcommand{\Aset}{\mathsf{A}} \newcommand{\b}[1]{{\mathbf{#1}}} \newcommand{\ball}[1]{\mathsf{B}(#1)} \newcommand{\bbQ}{{\mathbb Q}} \newcommand{\bproof}{\textbf{Proof :}\quad} \newcommand{\bmuf}[2]{b_{#1,#2}} \newcommand{\card}{\mathrm{card}} \newcommand{\chunk}[3]{{#1}_{#2:#3}} \newcommand{\condtrans}[3]{p_{#1}(#2|#3)} \newcommand{\convprob}[1]{\stackrel{#1-\text{prob}}{\rightarrow}} \newcommand{\Cov}{\mathbb{C}\mathrm{ov}} \newcommand{\cro}[1]{\langle #1 \rangle} \newcommand{\CPE}[2]{\PE\lr{#1| #2}} \renewcommand{\det}{\mathrm{det}} \newcommand{\dimlabel}{\mathsf{m}} \newcommand{\dimU}{\mathsf{q}} \newcommand{\dimX}{\mathsf{d}} \newcommand{\dimY}{\mathsf{p}} \newcommand{\dlim}{\Rightarrow} \newcommand{\e}[1]{{\left\lfloor #1 \right\rfloor}} \newcommand{\eproof}{\quad \Box} \newcommand{\eremark}{</WRAP>} \newcommand{\eqdef}{:=} \newcommand{\eqlaw}{\stackrel{\mathcal{L}}{=}} \newcommand{\eqsp}{\;} \newcommand{\Eset}{ {\mathsf E}} \newcommand{\esssup}{\mathrm{essup}} \newcommand{\fr}[1]{{\left\langle #1 \right\rangle}} \newcommand{\falph}{f} \renewcommand{\geq}{\geqslant} \newcommand{\hchi}{\hat \chi} \newcommand{\Hset}{\mathsf{H}} \newcommand{\Id}{\mathrm{Id}} \newcommand{\img}{\text{Im}} \newcommand{\indi}[1]{\mathbf{1}_{#1}} \newcommand{\indiacc}[1]{\mathbf{1}_{\{#1\}}} \newcommand{\indin}[1]{\mathbf{1}\{#1\}} \newcommand{\itemm}{\quad \quad \blacktriangleright \;} \newcommand{\jointtrans}[3]{p_{#1}(#2,#3)} \newcommand{\ker}{\text{Ker}} \newcommand{\klbck}[2]{\mathrm{K}\lr{#1||#2}} \newcommand{\law}{\mathcal{L}} \newcommand{\labelinit}{\pi} \newcommand{\labelkernel}{Q} \renewcommand{\leq}{\leqslant} \newcommand{\lone}{\mathsf{L}_1} \newcommand{\lp}[1]{\mathsf{L}_{{#1}}} \newcommand{\lrav}[1]{\left|#1 \right|} \newcommand{\lr}[1]{\left(#1 \right)} \newcommand{\lrb}[1]{\left[#1 \right]} \newcommand{\lrc}[1]{\left\{#1 \right\}} \newcommand{\lrcb}[1]{\left\{#1 \right\}} \newcommand{\ltwo}[1]{\PE^{1/2}\lrb{\lrcb{#1}^2}} \newcommand{\Ltwo}{\mathrm{L}^2} \newcommand{\mc}[1]{\mathcal{#1}} \newcommand{\mcbb}{\mathcal B} \newcommand{\mcf}{\mathcal{F}} \newcommand{\meas}[1]{\mathrm{M}_{#1}} \newcommand{\norm}[1]{\left\|#1\right\|} \newcommand{\normmat}[1]{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert #1 \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} \newcommand{\nset}{\mathbb N} \newcommand{\N}{\mathcal{N}} \newcommand{\one}{\mathsf{1}} \newcommand{\PE}{\mathbb E} \newcommand{\pminfty}{_{-\infty}^\infty} \newcommand{\PP}{\mathbb P} \newcommand{\projorth}[1]{\mathsf{P}^\perp_{#1}} \newcommand{\Psif}{\Psi_f} \newcommand{\pscal}[2]{\langle #1,#2\rangle} \newcommand{\pscal}[2]{\langle #1,#2\rangle} \newcommand{\psconv}{\stackrel{\PP-a.s.}{\rightarrow}} \newcommand{\qset}{\mathbb Q} \newcommand{\revcondtrans}[3]{q_{#1}(#2|#3)} \newcommand{\rmd}{\mathrm d} \newcommand{\rme}{\mathrm e} \newcommand{\rmi}{\mathrm i} \newcommand{\Rset}{\mathbb{R}} \newcommand{\rset}{\mathbb{R}} \newcommand{\rti}{\sigma} \newcommand{\section}[1]{==== #1 ====} \newcommand{\seq}[2]{\lrc{#1\eqsp: \eqsp #2}} \newcommand{\set}[2]{\lrc{#1\eqsp: \eqsp #2}} \newcommand{\sg}{\mathrm{sgn}} \newcommand{\supnorm}[1]{\left\|#1\right\|_{\infty}} \newcommand{\thv}{{\theta_\star}} \newcommand{\tmu}{ {\tilde{\mu}}} \newcommand{\Tset}{ {\mathsf{T}}} \newcommand{\Tsigma}{ {\mathcal{T}}} \newcommand{\ttheta}{{\tilde \theta}} \newcommand{\tv}[1]{\left\|#1\right\|_{\mathrm{TV}}} \newcommand{\unif}{\mathrm{Unif}} \newcommand{\weaklim}[1]{\stackrel{\mathcal{L}_{#1}}{\rightsquigarrow}} \newcommand{\Xset}{{\mathsf X}} \newcommand{\Xsigma}{\mathcal X} \newcommand{\Yset}{{\mathsf Y}} \newcommand{\Ysigma}{\mathcal Y} \newcommand{\Var}{\mathbb{V}\mathrm{ar}} \newcommand{\zset}{\mathbb{Z}} \newcommand{\Zset}{\mathsf{Z}} $$
De Finetti's Representation Theorem for Exchangeable Random Elements
Theorem
De Finetti's Theorem: Let $(X_i)_{i\in\mathbb{N}}$ be a family of exchangeable random elements taking balues on a measurable space $(\mathsf{X},\mathcal{X})$. Then, there exists a $\sigma$-field $\mathcal{G}_\infty$ such that, conditionally on $\mathcal{G}_\infty$, the random variables $(X_i)_{i\in\mathbb{N}}$ are independent and identically distributed (i.i.d.).
The proof is based on the paper “Uses of exchangeability” by J. F. Kingman Click here to see the paper.
Proof
Without loss of generality, we model $(X_i)_{i\in\mathbb{N}}$ as the coordinate projections on the canonical probability space $(\mathsf{X}^{\mathbb{N}},\mathcal{X}^{\otimes\mathbb{N}},\mathbb{P})$. We proceed as follows:
1. Reverse filtration construction:
- For each $n\in\mathbb{N}$, let $\mathcal{G}_n$ be the $\sigma$-field generated by all measurable functions $f:\mathsf{X}^{\mathbb{N}}\to\mathbb{R}$ invariant under any permutation of the first $n$ coordinates, i.e. for any permutation $\pi$ of $\{1,\ldots,n\}$ and any $x\in\mathsf{X}^{\mathbb{N}}$,
$$ f(x_1,\ldots,x_n,x_{n+1},\ldots)=f(x_{\pi(1)},\ldots,x_{\pi(n)},x_{n+1},\ldots). $$
- $(\mathcal{G}_n)_{n\in\mathbb{N}}$ is a reverse filtration with $\mathcal{G}_n\supseteq\mathcal{G}_{n+1}$ and
$$ \mathcal{G}_\infty=\bigcap_{n\in\mathbb{N}}\mathcal{G}_n. $$
- $\mathcal{G}_\infty$ is generated by all functions invariant under any finite permutation of coordinates.
2. Conditional expectation and empirical averages:
- For any bounded measurable $h:\mathsf{X}\to\mathbb{R}$, the empirical average $\frac1n\sum_{i=1}^n h(X_i)$ is $\mathcal{G}_n$-measurable.
- For any $A\in\mathcal{G}_n$, exchangeability implies
$$ \mathbb{E}\!\left[\left(\frac1n\sum_{i=1}^n h(X_i)\right)\mathbf1_A\right] =\frac1n\sum_{i=1}^n \mathbb{E}\!\left[\left( h(X_i)\right)\mathbf1_A\right] =\mathbb{E}[h(X_1)\mathbf1_A]. $$
- The two previous item show the amazing formula:
$$ \frac1n\sum_{i=1}^n h(X_i)=\mathbb{E}[h(X_1)\mid\mathcal{G}_n], \quad a.s. $$
- By the reverse martingale convergence theorem (see for example Upcrossing Inequality and Martingale Convergence),
$$ \frac1n\sum_{i=1}^n h(X_i)\xrightarrow{\mathrm{a.s.}}\mathbb{E}[h(X_1)\mid\mathcal{G}_\infty]. $$
3. Multivariate functions:
- For bounded measurable $f:\mathsf{X}^k\to\mathbb{R}$,
$$ \mathbb{E}[f(X_1,\ldots,X_k)\mid\mathcal{G}_\infty] =\lim_{n\to\infty}\frac1{n(n-1)\cdots(n-k+1)} \sum_{\substack{i_{1:k} \in [1:n]^k\\ i_j\neq i_\ell}} f(X_{i_1},\ldots,X_{i_k}). $$
- But we have
$$ \frac1{n(n-1)\cdots(n-k+1)} \sum_{\substack{i_{1:k} \in [1:n]^k\\ i_j\neq i_\ell}} f(X_{i_1},\ldots,X_{i_k}) + O\lr{\frac1n} = \frac1{n^k}\sum_{i_1=1}^n\cdots\sum_{i_k=1}^n f(X_{i_1},\ldots,X_{i_k}). $$
- Hence, for product functions $f(x_1,\ldots,x_k)=f_1(x_1)\cdots f_k(x_k)$,
$$ \mathbb{E}[f_1(X_1)\cdots f_k(X_k)\mid\mathcal{G}_\infty] =\prod_{\ell=1}^k\mathbb{E}[f_\ell(X_1)\mid\mathcal{G}_\infty]. $$
- Thus, conditionally on $\mathcal{G}_\infty$, $(X_i)$ are independent.
If $(X_i)$ are real-valued random variables, then, since the distribution of any random variable $X$ is completely determined by the values $\mathbb{P}(X \le x)$ for $x \in \mathbb{Q}$, we can replace $\mathcal{G}_\infty$ with the $\sigma$-field generated by the countable family of random variables \[ \left\{ \mathbb{E}\big[\mathbf{1}_{\{X_1 \le x\}} \mid \mathcal{G}_\infty \big] : x \in \mathbb{Q} \right\}. \] It follows that there exists a random variable $S$ such that, conditional on $S$, the sequence $(X_i)$ is independent and identically distributed. $\square$
Comments: convergence of Empirical Averages for i.i.d. Random Variables
The previous approach allows to prove the strong law of large numbers (for a proof of the LLN using only the dominated convergence theorem, click here)
Let \((X_i)\) be a sequence of independent and identically distributed (i.i.d.) random variables such that \(\mathbb{E}[|X_1|] < \infty\). Then, \[ \frac{1}{n} \sum_{i=1}^n X_i \xrightarrow{a.s.} \mathbb{E}[X_1]. \]
Proof: By the reverse martingale convergence theorem (see Upcrossing Inequality and Martingale Convergence), the empirical average converges almost surely to the conditional expectation with respect to the tail σ-field \(\mathcal{G}_\infty\): \[ \frac{1}{n} \sum_{i=1}^n X_i = \mathbb{E}[X_1|\mathcal{G}_n] \xrightarrow{a.s.} \mathbb{E}[X_1|\mathcal{G}_\infty]. \]
The Hewitt-Savage 0-1 Law states that any \(\mathcal{G}_\infty\)-measurable random variable is almost surely constant. Consequently, the conditional expectation \(\mathbb{E}[X_1|\mathcal{G}_\infty]\) is almost surely equal to its unconditional expectation: \[ \mathbb{E}[X_1|\mathcal{G}_\infty] = \mathbb{E}[X_1] \quad \text{a.s.} \]
Thus, the empirical average converges almost surely to the theoretical expectation: \[ \frac{1}{n} \sum_{i=1}^n X_i \xrightarrow{a.s.} \mathbb{E}[X_1]. \]
world/de-finetti.1770456761.txt.gz · Last modified: 2026/02/07 10:32 by rdouc
