Welcome to Randal Douc's wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
{{page>:defs}}


====== De Finetti's Representation Theorem for Exchangeable Random Elements ======

===== Theorem =====

<WRAP center round tip 80%>
**De Finetti's Theorem:**  
Let $(X_i)_{i\in\mathbb{N}}$ be a family of **exchangeable random elements** taking values on a measurable space $(\mathsf{X},\mathcal{X})$. Then, there exists a $\sigma$-field $\mathcal{G}_\infty$ such that, **conditionally on $\mathcal{G}_\infty$**, the random variables $(X_i)_{i\in\mathbb{N}}$ are **independent and identically distributed (i.i.d.)**.

</WRAP>

The proof is based on the paper "Uses of exchangeability" by J. F. Kingman [[https://projecteuclid.org/journalArticle/Download?urlId=10.1214%2Faop%2F1176995566|Click here to see the paper]]. 
===== Proof =====
Without loss of generality, we model $(X_i)_{i\in\mathbb{N}}$ as the coordinate projections on the canonical probability space $(\mathsf{X}^{\mathbb{N}},\mathcal{X}^{\otimes\mathbb{N}},\mathbb{P})$. We proceed as follows:

1. **Reverse filtration construction:**  
    * For each $n\in\mathbb{N}$, let $\mathcal{G}_n$ be the $\sigma$-field generated by all measurable functions $f:\mathsf{X}^{\mathbb{N}}\to\mathbb{R}$ invariant under any permutation of the first $n$ coordinates, i.e. for any permutation $\pi$ of $\{1,\ldots,n\}$ and any $x\in\mathsf{X}^{\mathbb{N}}$,
$$
f(x_1,\ldots,x_n,x_{n+1},\ldots)=f(x_{\pi(1)},\ldots,x_{\pi(n)},x_{n+1},\ldots).
$$
    * $(\mathcal{G}_n)_{n\in\mathbb{N}}$ is a reverse filtration with $\mathcal{G}_n\supseteq\mathcal{G}_{n+1}$ and
$$
\mathcal{G}_\infty=\bigcap_{n\in\mathbb{N}}\mathcal{G}_n.
$$
    * $\mathcal{G}_\infty$ is generated by all functions invariant under any finite permutation of coordinates.

2. **Conditional expectation and empirical averages:**  
    * For any bounded measurable $h:\mathsf{X}\to\mathbb{R}$, the empirical average $\frac1n\sum_{i=1}^n h(X_i)$ is $\mathcal{G}_n$-measurable.
    * For any $A\in\mathcal{G}_n$, exchangeability implies
$$
\mathbb{E}\!\left[\left(\frac1n\sum_{i=1}^n h(X_i)\right)\mathbf1_A\right]
=\frac1n\sum_{i=1}^n \mathbb{E}\!\left[\left( h(X_i)\right)\mathbf1_A\right]
=\mathbb{E}[h(X_1)\mathbf1_A].
$$
    * The two previous item show the amazing formula: 
$$
\frac1n\sum_{i=1}^n h(X_i)=\mathbb{E}[h(X_1)\mid\mathcal{G}_n], \quad a.s. 
$$
    * By the **reverse martingale convergence** theorem (see for example [[world:forward-downward-martingale|Upcrossing Inequality and Martingale Convergence]]),
$$
\frac1n\sum_{i=1}^n h(X_i)\xrightarrow{\mathrm{a.s.}}\mathbb{E}[h(X_1)\mid\mathcal{G}_\infty].
$$

3. **Multivariate functions:**  
    * For bounded measurable $f:\mathsf{X}^k\to\mathbb{R}$,
$$
\mathbb{E}[f(X_1,\ldots,X_k)\mid\mathcal{G}_\infty]
=\lim_{n\to\infty}\frac1{n(n-1)\cdots(n-k+1)}
\sum_{\substack{i_{1:k} \in [1:n]^k\\ i_j\neq i_\ell}}
f(X_{i_1},\ldots,X_{i_k}).
$$
    * But we have
$$
\frac1{n(n-1)\cdots(n-k+1)}
\sum_{\substack{i_{1:k} \in [1:n]^k\\ i_j\neq i_\ell}}
f(X_{i_1},\ldots,X_{i_k}) + O\lr{\frac1n} = \frac1{n^k}\sum_{i_1=1}^n\cdots\sum_{i_k=1}^n f(X_{i_1},\ldots,X_{i_k}).
$$
    * Hence, for product functions $f(x_1,\ldots,x_k)=f_1(x_1)\cdots f_k(x_k)$,
$$
\mathbb{E}[f_1(X_1)\cdots f_k(X_k)\mid\mathcal{G}_\infty]
=\prod_{\ell=1}^k\mathbb{E}[f_\ell(X_1)\mid\mathcal{G}_\infty].
$$
    * Thus, conditionally on $\mathcal{G}_\infty$, $(X_i)$ are independent. $\blacksquare$

<WRAP center round todo 80%>
If $(X_i)$ are real-valued random variables, then, since the distribution of any random variable $X$ is completely determined by the values $\mathbb{P}(X \le x)$ for $x \in \mathbb{Q}$, we can replace $\mathcal{G}_\infty$ with the $\sigma$-field generated by the countable family of random variables
\[
\left\{ \mathbb{E}\big[\mathbf{1}_{\{X_1 \le x\}} \mid \mathcal{G}_\infty \big] : x \in \mathbb{Q} \right\}.
\]
It follows that there exists a random variable $S$ such that, conditional on $S$, the sequence $(X_i)$ is independent and identically distributed. 

</WRAP>


==== Comments: Another proof of the Law of Large Numbers for i.i.d. Random Variables ====

The previous approach allows to prove the strong law of large numbers (for a proof of the LLN using only the dominated convergence theorem,  [[world:lln| click here]])

<WRAP center round tip 80%>
Let \((X_i)\) be a sequence of **independent and identically distributed (i.i.d.)** random variables such that \(\mathbb{E}[|X_1|] < \infty\). Then, 
\[
\frac{1}{n} \sum_{i=1}^n X_i \xrightarrow{a.s.} \mathbb{E}[X_1].
\]
</WRAP>


**Proof:**
By the **reverse martingale convergence theorem** (see [[world:forward-downward-martingale|Upcrossing Inequality and Martingale Convergence]]), the empirical average converges almost surely to the conditional expectation with respect to the tail σ-field \(\mathcal{G}_\infty\):
\[
\frac{1}{n} \sum_{i=1}^n X_i = \mathbb{E}[X_1|\mathcal{G}_n] \xrightarrow{a.s.} \mathbb{E}[X_1|\mathcal{G}_\infty].
\]

The [[world:hewitt-savage|Hewitt-Savage 0-1 Law]] states that any \(\mathcal{G}_\infty\)-measurable random variable is almost surely constant. Consequently, the conditional expectation \(\mathbb{E}[X_1|\mathcal{G}_\infty]\) is almost surely equal to its unconditional expectation:
\[
\mathbb{E}[X_1|\mathcal{G}_\infty] = \mathbb{E}[X_1] \quad \text{a.s.}
\]

Thus, the empirical average converges almost surely to the theoretical expectation:
\[
\frac{1}{n} \sum_{i=1}^n X_i \xrightarrow{a.s.} \mathbb{E}[X_1].
\]
Welcome to Randal Douc's wiki

User Tools

Site Tools

Page Tools
