We set , and we let and such that

We assume there exist a Markov kernel on and a sequence of real numbers such that

Define iteratively the sequence $X_{n+1}=X_{n}-\gamma_{n+1} U_{n+1}$ where and . The aim of this short note is to prove that . We follow a version of the proof proposed by François Roueff.

Write and set

Considering \eqref{eq:def:Sn}, to obtain the convergence of , we only need to prove that and converge -a.s. as goes to infinity.

1. The convergence of follows from
   • is a submartingale since
   • .
2. The convergence of follows from .

Finally, \eqref{eq:def:Sn} implies that for some -a.s. finite random variable . To complete the proof, it remains to show that is almost surely null.

First, write the Doob decomposition for the submartingale that is: where Note that is a non-decreasing previsible non-negative process and that is a martingale.

To conclude, it is sufficient to prove that for any , . To this aim, we will show that

2. .

To get the second property (2), note that so that . Finally, is a martingale, with a positive part which is uniformly bounded in . Therefore, converges -a.s. And since , it also implies that converges a.s. so that .

We now turn to the first property (1). Let . Then, there exists such that for all , where . Moreover, by continuity of , and this implies that for all , This shows that and the proof is completed.