Let be random vector on , taking values on . We are interested in solving the optimization problem where is the set of measurable functions from to where we equip with the -field and with the -field .

For any classifier , decomposing into the disjoint events and using the tower property, Moreover the inequality above becomes an equality for which satisfies the property: where the last equality follows from the identity . This concludes the proof.