world:optimal-classifier
Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
|
world:optimal-classifier [2023/11/04 12:18] rdouc created |
world:optimal-classifier [2023/11/04 14:03] (current) rdouc [Bayes Optimal Classifier] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| {{page>:defs}} | {{page>:defs}} | ||
| - | === Bayes Optimal Classifier === | + | ====== Bayes Optimal Classifier ====== |
| Let $(X,Y)$ be random vector on $ (\Omega,\mcf,\PP) $, taking values on $\rset^k \times [1:p]$. We are interested in solving the optimization problem | Let $(X,Y)$ be random vector on $ (\Omega,\mcf,\PP) $, taking values on $\rset^k \times [1:p]$. We are interested in solving the optimization problem | ||
| Line 9: | Line 9: | ||
| where $\sf{F}$ is the set of measurable functions from $\rset^k$ to $[1:p]$ where we equip $\rset^k$ with the $\sigma$-field $\mcbb(\rset^k)$ and $[1:p]$ with the $\sigma$-field $\mc{P}([1:p])$. | where $\sf{F}$ is the set of measurable functions from $\rset^k$ to $[1:p]$ where we equip $\rset^k$ with the $\sigma$-field $\mcbb(\rset^k)$ and $[1:p]$ with the $\sigma$-field $\mc{P}([1:p])$. | ||
| - | Proposition | + | <WRAP center round tip 90%> |
| + | **__Proposition__** | ||
| $$ | $$ | ||
| \inf_{\phi \in \sf{F}} \PP(Y\neq \phi(X))= \PE\lrb{\min_{i \in [1:p]} \PP(Y \neq i|X)}=\PP(Y \neq \phi^\star(X)) | \inf_{\phi \in \sf{F}} \PP(Y\neq \phi(X))= \PE\lrb{\min_{i \in [1:p]} \PP(Y \neq i|X)}=\PP(Y \neq \phi^\star(X)) | ||
| $$ | $$ | ||
| where $\phi^\star(X)=\argmax_{i \in [1:p]} \PP(Y=i|X)$. | where $\phi^\star(X)=\argmax_{i \in [1:p]} \PP(Y=i|X)$. | ||
| + | </WRAP> | ||
| - | Proof | + | ===== Proof ===== |
| For any classifier $\phi$, decomposing into the disjoint events $\{\phi(X)=i\}$ and using the tower property, | For any classifier $\phi$, decomposing into the disjoint events $\{\phi(X)=i\}$ and using the tower property, | ||
| Line 27: | Line 28: | ||
| \phi^\star(X)= \argmin_{i \in [1:p]} \PP(Y \neq i|X)=\argmax_{i \in [1:p]} \PP(Y=i|X) | \phi^\star(X)= \argmin_{i \in [1:p]} \PP(Y \neq i|X)=\argmax_{i \in [1:p]} \PP(Y=i|X) | ||
| $$ | $$ | ||
| - | which concludes the proof. | + | where the last equality follows from the identity $\PP(Y \neq i|X)=1-\PP(Y=i|X)$. This concludes the proof. |
world/optimal-classifier.1699096730.txt.gz · Last modified: 2023/11/04 12:18 by rdouc
Page Tools
