#LyX 1.6.1 created this file. For more info see http://www.lyx.org/ \lyxformat 345 \begin_document \begin_header \textclass article \begin_preamble \usepackage[bookmarksnumbered,colorlinks,linkcolor=black,urlcolor=blue]{hyperref} \usepackage{enumerate} \usepackage{amsthm} \newtheorem{theorem}{Theorem} \newtheorem{lemma}{Lemma} \newtheorem*{corollary}{Corollary} \newtheorem*{claim}{Claim} \newcommand{\defterm}[1]{{\it #1}} \newcommand{\cD}{{\mathcal D}} \newcommand{\cF}{{\mathcal F}} \newcommand{\cG}{{\mathcal G}} \newcommand{\cH}{{\mathcal H}} \newcommand{\cI}{{\mathcal I}} \newcommand{\cJ}{{\mathcal J}} \newcommand{\cL}{{\mathcal L}} \newcommand{\cN}{{\mathcal N}} \newcommand{\cS}{{\mathcal S}} \newcommand{\cT}{{\mathcal T}} \newcommand{\C}{{\mathbb C}} \newcommand{\E}{{\mathbb E}} \newcommand{\N}{{\mathbb N}} \renewcommand{\P}{{\mathbb P}} \newcommand{\Q}{{\mathbb Q}} \newcommand{\R}{{\mathbb R}} \newcommand{\Z}{{\mathbb Z}} \newcommand{\mS}{{\mathrm{m}\cS}} \newcommand{\mSp}{{\mS^+}} \newcommand{\var}{\mathop{\mathrm{Var}}} \newcommand{\cov}{\mathop{\mathrm{Cov}}} \newcommand{\titlesigma}{{\texorpdfstring{$\sigma$}{sigma}}} \newcommand{\titleSigma}{{\texorpdfstring{$\sigma$}{Sigma}}} % The only way I can find to get LyX to give me a math environment % inside an equation tag \newcommand{\mtag}[1]{\tag{$#1$}} \end_preamble \use_default_options false \language british \inputencoding auto \font_roman lmodern \font_sans default \font_typewriter default \font_default_family default \font_sc false \font_osf false \font_sf_scale 100 \font_tt_scale 100 \graphics default \paperfontsize default \spacing single \use_hyperref false \papersize a4paper \use_geometry true \use_amsmath 1 \use_esint 1 \cite_engine basic \use_bibtopic false \paperorientation portrait \leftmargin 3cm \topmargin 2cm \rightmargin 3.5cm \bottommargin 2cm \secnumdepth 3 \tocdepth 3 \paragraph_separation indent \defskip medskip \quotes_language english \papercolumns 1 \papersides 1 \paperpagestyle default \tracking_changes false \output_changes false \author "" \author "" \end_header \begin_body \begin_layout Title Probability and Measure \end_layout \begin_layout Author Martin Orr \end_layout \begin_layout Subsection Introduction \end_layout \begin_layout Standard This is a set of notes for the Probability and Measure course in Part II of the Cambridge Mathematics Tripos. They were written as my own attempt to understand the course. \end_layout \begin_layout Standard These notes are currently only a draft, so many things could be improved. Please send me your comments or corrections. If you can't find something in which is claimed to be in the Appendix, that is because the Appendix has only partially been written yet. \begin_inset VSpace defskip \end_inset \end_layout \begin_layout Standard These notes are intended to be used in combination with Prof Rogers' examples sheets (at \begin_inset Flex URL status open \begin_layout Plain Layout http://www.statslab.cam.ac.uk/~chris/ \end_layout \end_inset ). However, I have sometimes included results which are on the example sheets and I have made no attempt to ensure that material is presented here in a sensible order for doing the example sheets. \end_layout \begin_layout Standard The main sources for these notes were Williams' \shape italic Probability with Martingales \shape default , Billingsley's \shape italic Probability and Measure \shape default and my lecture notes from Prof Rogers' lectures in Michaelmas 2007. Thanks are also due to my supervisor Dr Tehranchi for helping me to understand much of the material, and to Jacob Davis for comments and corrections. \end_layout \begin_layout Standard Responsibility for the material in here, including errors, is entirely my own. Some of my own biases will have shown through. In particular, these are unashamedly the notes of a pure mathematician and I tend to emphasise analysis rather than probability. \end_layout \begin_layout Section Sets and measures \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status collapsed \begin_layout Plain Layout [4] Measure spaces, \begin_inset Formula $\sigma$ \end_inset -algebras, \begin_inset Formula $\pi$ \end_inset -systems and uniqueness of extension, statement *and proof* of Carathéodory's extension theorem. Construction of Lebesgue measure on \begin_inset Formula $\mathbb{R}$ \end_inset . The Borel \begin_inset Formula $\sigma$ \end_inset -algebra of \begin_inset Formula $\mathbb{R}$ \end_inset . Existence of non-measurable subsets of \begin_inset Formula $\mathbb{R}$ \end_inset . \end_layout \end_inset \end_layout \begin_layout Subsection Probability: A reminder \end_layout \begin_layout Standard In probability theory, we work with a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{sample space} \end_layout \end_inset \begin_inset Formula $\Omega$ \end_inset of possible \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{outcomes} \end_layout \end_inset . An \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{event} \end_layout \end_inset is a subset \begin_inset Formula $A$ \end_inset of \begin_inset Formula $\Omega$ \end_inset to which we assign a probability \begin_inset Formula $\P(A)$ \end_inset . This should satisfy the following axioms: \end_layout \begin_layout Enumerate \begin_inset Formula $0\leq\P(A)\leq1$ \end_inset for all events \begin_inset Formula $A$ \end_inset \end_layout \begin_layout Enumerate \begin_inset Formula $\P(\Omega)=1$ \end_inset \end_layout \begin_layout Enumerate If \begin_inset Formula $\{A_{n}\}$ \end_inset is a finite or countable collection of pairwise disjoint events, \begin_inset Newline newline \end_inset then \begin_inset Formula $\P(\bigcup_{n}A_{n})=\sum_{n}\P(A_{n})$ \end_inset \end_layout \begin_layout Standard If \begin_inset Formula $\Omega$ \end_inset is finite or countable, then we can quite happily let all subsets of \begin_inset Formula $\Omega$ \end_inset be events, and indeed specifying \begin_inset Formula $\P(\{\omega\})$ \end_inset for each singleton \begin_inset Formula $\{\omega\}$ \end_inset is sufficient to determine the probability of all subsets by the above axioms. \end_layout \begin_layout Subsection Non-measurable sets \end_layout \begin_layout Standard When we turn to uncountable \begin_inset Formula $\Omega$ \end_inset things are not so simple. Suppose we try to model the uniform distribution on \begin_inset Formula $\Omega=(0,1]$ \end_inset . By \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{uniform distribution} \end_layout \end_inset I mean that probabilities are unchanged by translations: for any fixed \begin_inset Formula $t$ \end_inset , \begin_inset Formula $\P(\{\omega\in\Omega|\omega+t\bmod1\in A\})=\P(A)$ \end_inset . It turns out that (as long as we assume the Axiom of Choice) it is not possible to define a probability for all subsets of \begin_inset Formula $(0,1]$ \end_inset which satisfies this condition as well as the above axioms. \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Vitali's Theorem] \end_layout \end_inset There is no translation-invariant probability defined on all subsets of \begin_inset Formula $(0,1]$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Suppose we have a translation-invariant probability \begin_inset Formula $\P$ \end_inset defined on all subsets of \begin_inset Formula $(0,1]$ \end_inset . \end_layout \begin_layout Standard Define an equivalence relation \begin_inset Formula $\sim$ \end_inset on \begin_inset Formula $(0,1]$ \end_inset by \begin_inset Formula $x\sim y$ \end_inset if \begin_inset Formula $x-y\in\Q$ \end_inset . By the Axiom of Choice, we can form a set \begin_inset Formula $A$ \end_inset which contains one element of each equivalence class. \end_layout \begin_layout Standard For each \begin_inset Formula $q\in\Q$ \end_inset , let \begin_inset Formula $A_{q}=\{a\in(0,1]|a+q\bmod1\in A\}$ \end_inset . Then the \begin_inset Formula $A_{q}$ \end_inset are disjoint, have union \begin_inset Formula $(0,1]$ \end_inset and there are countably many. So \begin_inset Formula $\sum_{q}\P(A_{q})=\P((0,1])=1$ \end_inset . \end_layout \begin_layout Standard However, \begin_inset Formula $\P(A_{q})=\P(A)$ \end_inset for all \begin_inset Formula $q\in\Q$ \end_inset . So if \begin_inset Formula $\P(A)=0$ \end_inset then \begin_inset Formula $\sum_{q}\P(A_{q})=0$ \end_inset while if \begin_inset Formula $\P(A)=\epsilon>0$ \end_inset then \begin_inset Formula $\sum_{q}\P(A_{q})=\infty$ \end_inset , giving a contradiction. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard Another property we might expect the uniform distribution to have is that \begin_inset Formula $\P((a,b])=b-a$ \end_inset for all \begin_inset Formula $a,b$ \end_inset . It is not too hard to show that this is implied by translation invariance, but does the reverse implication hold? Does there exist a probability on \begin_inset Formula $(0,1]$ \end_inset which takes the right value on all intervals? In fact this turns out to lead deep into set theory and be undecidable with the usual axioms. For those who care about such things, the existence of such a probability is equiconsistent with the existence of a measurable cardinal. \end_layout \begin_layout Subsection \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash titleSigma \end_layout \end_inset -fields \end_layout \begin_layout Standard Since we can't satisfactorily assign a probability to all subsets of \begin_inset Formula $(0,1]$ \end_inset , we restrict our notion of what an event is. Let \begin_inset Formula $\cF\subset\wp(\Omega)$ \end_inset be the set of events. To properly define the \begin_inset Formula $\cF$ \end_inset we will work on, we need to take a much more abstract approach, and write down the properties we would like \begin_inset Formula $\cF$ \end_inset to have, then define \begin_inset Formula $\cF$ \end_inset to be the smallest collection of sets with these properties. This has the benefit that it allows us to develop a more general theory that can be applied to any collection of sets with the required properties, not just subsets of \begin_inset Formula $(0,1]$ \end_inset . \end_layout \begin_layout Standard The main properties we require of \begin_inset Formula $\cF$ \end_inset are that the sets mentioned in the axioms for probability should exist. It also seems natural that if \begin_inset Formula $A$ \end_inset is an event, \begin_inset Formula $\text{not-}A$ \end_inset should be as well. So the properties are: \end_layout \begin_layout Enumerate \begin_inset Formula $\Omega\in\cF$ \end_inset . \end_layout \begin_layout Enumerate If \begin_inset Formula $A\in\cF$ \end_inset then \begin_inset Formula $A^{c}\in\cF$ \end_inset (complement with respect to \begin_inset Formula $\Omega$ \end_inset ). \end_layout \begin_layout Enumerate If \begin_inset Formula $\{A_{n}\}$ \end_inset is a finite or countable subcollection of \begin_inset Formula $\cF$ \end_inset , then \begin_inset Formula $\bigcup_{n}A_{n}\in\cF$ \end_inset . \end_layout \begin_layout Standard Any collection \begin_inset Formula $\cF\subset\wp(\Omega)$ \end_inset with these properties is called a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$ \backslash sigma$-field} \end_layout \end_inset . Note that, by 2 and 3, \begin_inset Formula $\cF$ \end_inset is closed under countable intersections as well. \end_layout \begin_layout Standard A \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{measurable space} \end_layout \end_inset is a pair \begin_inset Formula $(\Omega,\cF)$ \end_inset where \begin_inset Formula $\Omega$ \end_inset is some set and \begin_inset Formula $\cF$ \end_inset is a \begin_inset Formula $\sigma$ \end_inset -field on \begin_inset Formula $\Omega$ \end_inset . \end_layout \begin_layout Standard A subset \begin_inset Formula $A$ \end_inset of \begin_inset Formula $\Omega$ \end_inset is a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{measurable set} \end_layout \end_inset with respect to a \begin_inset Formula $\sigma$ \end_inset -field \begin_inset Formula $\cF$ \end_inset if \begin_inset Formula $A\in\cF$ \end_inset . (This is the same as the probabilistic term \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{event} \end_layout \end_inset .) \end_layout \begin_layout Standard A \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{measure} \end_layout \end_inset is a function \begin_inset Formula $\mu:\cF\rightarrow[0,\infty]$ \end_inset on a \begin_inset Formula $\sigma$ \end_inset -field s.t. if \begin_inset Formula $\{A_{n}\}\subset\cF$ \end_inset is finite or countable and the \begin_inset Formula $A_{n}$ \end_inset are pairwise disjoint, then \begin_inset Formula $\mu(\bigcup_{n}A_{n})=\sum_{n}\mu(A_{n})$ \end_inset . \end_layout \begin_layout Standard \begin_inset Formula $\mu$ \end_inset is a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{probability (measure)} \end_layout \end_inset if \begin_inset Formula $\mu(\Omega)=1$ \end_inset (note that this agrees with the earlier definition of probability). \end_layout \begin_layout Standard \begin_inset Formula $\mu$ \end_inset is a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{finite measure} \end_layout \end_inset if \begin_inset Formula $\mu(\Omega)<\infty$ \end_inset . \end_layout \begin_layout Standard \begin_inset Formula $\mu$ \end_inset is a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$ \backslash sigma$-finite measure} \end_layout \end_inset if there are countably many sets \begin_inset Formula $A_{n}$ \end_inset , each with \begin_inset Formula $\mu(A_{n})<\infty$ \end_inset , s.t. \begin_inset Formula $\bigcup_{n}A_{n}=\Omega$ \end_inset . \end_layout \begin_layout Standard A \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{measure space} \end_layout \end_inset is a triple \begin_inset Formula $(\Omega,\cF,\mu)$ \end_inset where \begin_inset Formula $\Omega$ \end_inset is a set, \begin_inset Formula $\cF$ \end_inset is a \begin_inset Formula $\sigma$ \end_inset -field on \begin_inset Formula $\Omega$ \end_inset and \begin_inset Formula $\mu$ \end_inset is a measure on \begin_inset Formula $\cF$ \end_inset . \end_layout \begin_layout Subsection The Borel \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash titlesigma \end_layout \end_inset -field \end_layout \begin_layout Standard To continue with the scheme outlined in the previous section, we want to define our uniform probability on the smallest \begin_inset Formula $\sigma$ \end_inset -field on \begin_inset Formula $(0,1]$ \end_inset containing the half-open intervals. What does this mean? \end_layout \begin_layout Standard Observe that for any collection of \begin_inset Formula $\sigma$ \end_inset -fields on some base set \begin_inset Formula $\Omega$ \end_inset , their intersection is itself a \begin_inset Formula $\sigma$ \end_inset -field (it is straightforward to check the definition). So, given any collection \begin_inset Formula $C$ \end_inset of subsets of \begin_inset space ~ \end_inset \begin_inset Formula $\Omega$ \end_inset , \begin_inset Formula $\bigcap\{\sigma\textrm{-fields containing }C\}$ \end_inset is itself a \begin_inset Formula $\sigma$ \end_inset -field. It is contained in every \begin_inset Formula $\sigma$ \end_inset -field containing \begin_inset Formula $C$ \end_inset , so it makes sense to call this the smallest \begin_inset Formula $\sigma$ \end_inset -field containing \begin_inset Formula $C$ \end_inset . Note that this really exists because \begin_inset Formula $\wp(\Omega)$ \end_inset is itself a \begin_inset Formula $\sigma$ \end_inset -field, so there is at least one \begin_inset Formula $\sigma$ \end_inset -field containing \begin_inset Formula $C$ \end_inset . (Compare this with defining the closure of a subset \begin_inset Formula $S$ \end_inset of a topological space as the smallest closed set containing \begin_inset Formula $S$ \end_inset .) \end_layout \begin_layout Standard The smallest \begin_inset Formula $\sigma$ \end_inset -field containing \begin_inset Formula $C$ \end_inset is called the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$ \backslash sigma$-field generated by $C$} \end_layout \end_inset , written \begin_inset Formula $\sigma(C)$ \end_inset . \end_layout \begin_layout Standard For a topological space \begin_inset Formula $(X,T)$ \end_inset , the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{Borel $ \backslash sigma$-field} \end_layout \end_inset is \begin_inset Formula $\sigma(T)$ \end_inset , the \begin_inset Formula $\sigma$ \end_inset -field generated by the open sets. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset The \begin_inset Formula $\sigma$ \end_inset -field on \begin_inset Formula $(0,1]$ \end_inset generated by half-open intervals \begin_inset Formula $(a,b]$ \end_inset is the same as \begin_inset Formula $B((0,1])$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Let \begin_inset Formula $S$ \end_inset be the set of half-open intervals \begin_inset Formula $(a,b]$ \end_inset and \begin_inset Formula $T$ \end_inset the open sets of \begin_inset Formula $(0,1]$ \end_inset . \end_layout \begin_layout Standard If \begin_inset Formula $b<1$ \end_inset , then \begin_inset Formula $(a,b]=\bigcap_{n}(a,b+1/n)\in B((0,1])$ \end_inset and if \begin_inset Formula $b=1$ \end_inset , certainly, \begin_inset Formula $(a,b]\in B((0,1])$ \end_inset . So \begin_inset Formula $S\subset B((0,1])$ \end_inset , so as \begin_inset Formula $B((0,1])$ \end_inset is a \begin_inset Formula $\sigma$ \end_inset -field, \begin_inset Formula $\sigma(S)\subset B((0,1])$ \end_inset . \end_layout \begin_layout Standard The open interval \begin_inset Formula $(a,b)=\bigcup_{n}(a,b-1/n]\in\sigma(S)$ \end_inset , and any interval \begin_inset Formula $(a,1]\in\sigma(S)$ \end_inset . Every open subset of \begin_inset Formula $(0,1]$ \end_inset can be written as a countable union of open intervals and sets of the form \begin_inset Formula $(a,1]$ \end_inset (since every connected component of our open set contains a rational). So \begin_inset Formula $T\subset\sigma(S)$ \end_inset , and \begin_inset Formula $B((0,1])=\sigma(T)\subset\sigma(S)$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection Carathéodory Extension Theorem \end_layout \begin_layout Standard Now we have our \begin_inset Formula $\sigma$ \end_inset -field \begin_inset Formula $B((0,1])$ \end_inset of events, we have to ensure that there is in fact a suitable measure defined on them. We do this using the Carathéodory Extension Theorem. \end_layout \begin_layout Standard In order to state this theorem, we need weaker versions of some of the definitio ns concerning measures. There are several definitions of families of sets obeying certain closure properties, most of which are unimportant and I have relegated to a table at the end of this section. What we need now is a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{field} \end_layout \end_inset , with the properties: \end_layout \begin_layout Enumerate If \begin_inset Formula $A\in\cF$ \end_inset then \begin_inset Formula $A^{c}\in\cF_{0}$ \end_inset (complement with respect to \begin_inset Formula $\Omega$ \end_inset ). \end_layout \begin_layout Enumerate \begin_inset Formula $\Omega\in\cF_{0}$ \end_inset . \end_layout \begin_layout Enumerate If \begin_inset Formula $\{A_{n}\}$ \end_inset is a finite subcollection of \begin_inset Formula $\cF_{0}$ \end_inset , then \begin_inset Formula $\bigcup_{n}A_{n}\in\cF_{0}$ \end_inset . \end_layout \begin_layout Standard A function \begin_inset Formula $\mu_{0}:\cF_{0}\rightarrow[0,\infty]$ \end_inset on a field \begin_inset Formula $\cF_{0}$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{countably additive} \end_layout \end_inset if whenever \begin_inset Formula $\{A_{n}\}\subset\cF_{0}$ \end_inset is finite or countable, the \begin_inset Formula $A_{n}$ \end_inset are pairwise disjoint and \begin_inset Formula $\bigcup_{n}A_{n}\in\cF_{0}$ \end_inset , then \begin_inset Formula $\mu_{0}(\bigcup_{n}A_{n})=\sum_{n}\mu_{0}(A_{n})$ \end_inset . (So a measure is a countably additive function on a \begin_inset Formula $\sigma$ \end_inset -field.) \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{Finitely additive} \end_layout \end_inset is defined similarly, with the condition only applied to finite subfamilies \begin_inset Formula $\{A_{n}\}$ \end_inset . \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Carathéodory Extension Theorem] \end_layout \end_inset If \begin_inset Formula $\cF_{0}$ \end_inset is a field and \begin_inset Formula $\mu_{0}:\cF_{0}\rightarrow[0,\infty]$ \end_inset is countably additive, then there is a measure \begin_inset Formula $\mu$ \end_inset on \begin_inset Formula $\sigma(\cF_{0})$ \end_inset s.t. \begin_inset Formula $\mu(A)=\mu_{0}(A)$ \end_inset for all \begin_inset Formula $A\in\cF_{0}$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard The proof of this theorem is starred in the schedules, and is boring and mostly technical, so it is given in the Appendix. (Note that the unstarred sentence in the schedules \begin_inset Quotes eld \end_inset Construction of Lebesgue measure on \begin_inset Formula $\R$ \end_inset \begin_inset Quotes erd \end_inset might require the definition of outer measure.) \end_layout \begin_layout Standard \end_layout \begin_layout Subsection Uniqueness of extension \end_layout \begin_layout Standard That's all very well; this tells us that, given a countably additive function on a field, there is some measure on the generated \begin_inset Formula $\sigma$ \end_inset -field which extends it. But there might be more than one such measure. We want to prove that there is only one. To do this requires defining a couple more types of sets, and some fairly abstract manipulation of them. I find it impossible to visualise the sets worked with here, which are larger than \begin_inset Formula $\R$ \end_inset , but this doesn't really matter: you can just follow the properties defining them and only care about what they actually contain at the beginning and end. \end_layout \begin_layout Standard A \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$ \backslash pi$-system} \end_layout \end_inset is a family \begin_inset Formula $\cG$ \end_inset of subsets of \begin_inset Formula $\Omega$ \end_inset s.t. if \begin_inset Formula $A,B\in\cG$ \end_inset then \begin_inset Formula $A\cap B\in\cG$ \end_inset . Note that any field is a \begin_inset Formula $\pi$ \end_inset -system. \end_layout \begin_layout Standard A \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$d$-system} \end_layout \end_inset is a family \begin_inset Formula $\cD$ \end_inset of subsets of \begin_inset Formula $\Omega$ \end_inset s.t. \end_layout \begin_layout Enumerate \begin_inset Formula $\Omega\in\cD$ \end_inset . \end_layout \begin_layout Enumerate If \begin_inset Formula $A,B\in\cD$ \end_inset and \begin_inset Formula $A\supset B$ \end_inset then \begin_inset Formula $A\setminus B\in\cD$ \end_inset . \end_layout \begin_layout Enumerate If \begin_inset Formula $\{A_{n}\}\subset\cD$ \end_inset is countable and \begin_inset Formula $A_{n}\uparrow A$ \end_inset (i.e. \begin_inset Formula $A_{n}\subset A_{n+1}\forall n$ \end_inset and \begin_inset Formula $A=\bigcup_{n}A_{n}$ \end_inset ) then \begin_inset Formula $A\in\cD$ \end_inset . \end_layout \begin_layout Standard It is easy to check that the intersection of a family of \begin_inset Formula $d$ \end_inset -systems is a \begin_inset Formula $d$ \end_inset -system, so for any collection \begin_inset Formula $C$ \end_inset of subsets of \begin_inset Formula $\Omega$ \end_inset we can define \begin_inset Formula $d(C)$ \end_inset , the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$d$-system generated by $C$} \end_layout \end_inset , as the smallest \begin_inset Formula $d$ \end_inset -system containing \begin_inset Formula $C$ \end_inset , just as with \begin_inset Formula $\sigma$ \end_inset -fields. \end_layout \begin_layout Standard These are connected to \begin_inset Formula $\sigma$ \end_inset -fields by the following two lemmas. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset A collection \begin_inset Formula $\cG$ \end_inset of subsets of \begin_inset Formula $\Omega$ \end_inset is a \begin_inset Formula $\sigma$ \end_inset -field iff it is both a \begin_inset Formula $\pi$ \end_inset -system and a \begin_inset Formula $d$ \end_inset -system. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset The forwards direction is easy. \end_layout \begin_layout Standard Now suppose \begin_inset Formula $\cG$ \end_inset is both a \begin_inset Formula $\pi$ \end_inset -system and a \begin_inset Formula $d$ \end_inset -system. \end_layout \begin_layout Standard From the definition of a \begin_inset Formula $d$ \end_inset -system, \begin_inset Formula $\Omega\in\cG$ \end_inset and for \begin_inset Formula $A\in\cG$ \end_inset , \begin_inset Formula $A^{c}=\Omega\setminus A\in\cG$ \end_inset . \end_layout \begin_layout Standard Suppose \begin_inset Formula $\{A_{n}\}\subset\cG$ \end_inset is countable. Then \begin_inset Formula $A_{n}^{c}\in\cG$ \end_inset for each \begin_inset Formula $n$ \end_inset , so as \begin_inset Formula $\cG$ \end_inset is a \begin_inset Formula $\pi$ \end_inset -system, we have \begin_inset Formula \[ \left(\bigcup_{n=1}^{N}A_{n}\right)^{c}=\bigcap_{n=1}^{N}A_{n}^{c}\in\cG\] \end_inset \end_layout \begin_layout Standard So \begin_inset Formula $\bigcup_{n=1}^{N}A_{n}\in\cG$ \end_inset for each \begin_inset Formula $N$ \end_inset . Hence as \begin_inset Formula $\bigcup_{n=1}^{N}A_{n}\uparrow\bigcup_{n}A_{n}$ \end_inset , we get \begin_inset Formula $\bigcup_{n}A_{n}\in\cG$ \end_inset . \end_layout \begin_layout Standard So \begin_inset Formula $\cG$ \end_inset is a \begin_inset Formula $\sigma$ \end_inset -field. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{lemma}[Dynkin's Lemma] \end_layout \end_inset If \begin_inset Formula $\cI$ \end_inset is a \begin_inset Formula $\pi$ \end_inset -system, then \begin_inset Formula $d(\cI)=\sigma(\cI)$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Certainly, \begin_inset Formula $\sigma(\cI)$ \end_inset is a \begin_inset Formula $\sigma$ \end_inset -field, so a \begin_inset Formula $d$ \end_inset -system, containing \begin_inset Formula $\cI$ \end_inset , so \begin_inset Formula $\sigma(\cI)\supset d(\cI)$ \end_inset . \end_layout \begin_layout Standard We need to show that \begin_inset Formula $d(\cI)$ \end_inset is a \begin_inset Formula $\pi$ \end_inset -system. Then by the previous lemma, it is a \begin_inset Formula $\sigma$ \end_inset -field, so \begin_inset Formula $d(\cI)\supset\sigma(\cI)$ \end_inset . \end_layout \begin_layout Standard To show this, we will construct a subset of \begin_inset Formula $d(\cI)$ \end_inset and show that the subset is itself a \begin_inset Formula $d$ \end_inset -system containing \begin_inset Formula $\cI$ \end_inset , so containing, and therefore equal to, \begin_inset Formula $d(\cI)$ \end_inset . We will need to do this twice. \end_layout \begin_layout Standard Let \begin_inset Formula \[ \cD_{1}=\{A\in d(\cI)\colon A\cap X\in d(\cI)\forall X\in\cI\}\] \end_inset \end_layout \begin_layout Standard Certainly \begin_inset Formula $\Omega\in\cD_{1}$ \end_inset . \end_layout \begin_layout Standard If \begin_inset Formula $A,B\in\cD_{1}$ \end_inset and \begin_inset Formula $A\supset B$ \end_inset , then \begin_inset Formula $(A\setminus B)\cap X=(A\cap X)\setminus(B\cap X)\in d(\cI)$ \end_inset for all \begin_inset Formula $X\in\cI$ \end_inset , so \begin_inset Formula $A\setminus B\in\cD_{1}$ \end_inset . \end_layout \begin_layout Standard If \begin_inset Formula $\{A_{n}\}\subset\cD_{1}$ \end_inset is countable and \begin_inset Formula $A_{n}\uparrow A$ \end_inset , then \begin_inset Formula $(A_{n}\cap X)\uparrow(A\cap X)$ \end_inset so \begin_inset Formula $A\cap X\in d(\cI)$ \end_inset for all \begin_inset Formula $X\in\cI$ \end_inset , so \begin_inset Formula $A\in\cD_{1}$ \end_inset . \end_layout \begin_layout Standard So \begin_inset Formula $\cD_{1}$ \end_inset is a \begin_inset Formula $d$ \end_inset -system. Also, since \begin_inset Formula $\cI$ \end_inset is a \begin_inset Formula $\pi$ \end_inset -system, \begin_inset Formula $\cI\subset\cD_{1}$ \end_inset . \end_layout \begin_layout Standard So \begin_inset Formula $d(\cI)\subset\cD_{1}$ \end_inset ; and by definition, \begin_inset Formula $\cD_{1}\subset d(\cI)$ \end_inset . So \begin_inset Formula $d(\cI)=\cD_{1}$ \end_inset . \end_layout \begin_layout Standard Now let \begin_inset Formula \[ \cD_{2}=\{B\in d(\cI)\colon B\cap Y\in d(\cI)\forall Y\in d(\cI)\}\] \end_inset \end_layout \begin_layout Standard By the same argument as before, \begin_inset Formula $\cD_{2}$ \end_inset is a \begin_inset Formula $d$ \end_inset -system. \end_layout \begin_layout Standard If \begin_inset Formula $X\in\cI$ \end_inset , then \begin_inset Formula $A\cap X\in d(\cI)$ \end_inset for all \begin_inset Formula $A\in\cD_{1}=d(\cI)$ \end_inset , so \begin_inset Formula $X\in\cD_{2}$ \end_inset . \end_layout \begin_layout Standard Hence \begin_inset Formula $\cI\subset\cD_{2}$ \end_inset so \begin_inset Formula $d(\cI)\subset\cD_{2}$ \end_inset so \begin_inset Formula $d(\cI)=\cD_{2}$ \end_inset . \end_layout \begin_layout Standard Hence \begin_inset Formula $d(\cI)$ \end_inset is closed under finite intersections, so is a \begin_inset Formula $\pi$ \end_inset -system. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard With those out of the way, the following theorem tells us that if two measures agree on a \begin_inset Formula $\pi$ \end_inset -system, then they agree on the \begin_inset Formula $\sigma$ \end_inset -field generated by the \begin_inset Formula $\pi$ \end_inset -system. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset \begin_inset CommandInset label LatexCommand label name "lemma:pi-unique-measure" \end_inset If \begin_inset Formula $\cI$ \end_inset is a \begin_inset Formula $\pi$ \end_inset -system and \begin_inset Formula $\mu_{1},\mu_{2}$ \end_inset are measures on \begin_inset Formula $\sigma(\cI)$ \end_inset with \begin_inset Formula $\mu_{1}(\Omega)=\mu_{2}(\Omega)<\infty$ \end_inset and \begin_inset Formula $\mu_{1}(A)=\mu_{2}(A)$ \end_inset for all \begin_inset Formula $A\in\cI$ \end_inset , then \begin_inset Formula $\mu_{1}=\mu_{2}$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Let \begin_inset Formula \[ \cD=\{A\in\cI\colon\mu_{1}(A)=\mu_{2}(A)\}\] \end_inset \end_layout \begin_layout Standard We show that \begin_inset Formula $\cD$ \end_inset is a \begin_inset Formula $d$ \end_inset -system. We are given that \begin_inset Formula $\cI\subset\cD$ \end_inset , so then \begin_inset Formula $\cD\supset d(\cI)=\sigma(\cI)$ \end_inset . \end_layout \begin_layout Standard We are given that \begin_inset Formula $\Omega\in\cD$ \end_inset . \end_layout \begin_layout Standard If \begin_inset Formula $A,B\in\cD$ \end_inset and \begin_inset Formula $A\supset B$ \end_inset then \begin_inset Formula \[ \mu_{1}(A\setminus B)=\mu_{1}(A)-\mu_{1}(B)=\mu_{2}(A)-\mu_{2}(B)=\mu_{2}(A\setminus B)\] \end_inset so \begin_inset Formula $A\setminus B\in\cD$ \end_inset . \end_layout \begin_layout Standard If \begin_inset Formula $\{A_{n}\}\subset\cD$ \end_inset is countable and \begin_inset Formula $A_{n}\uparrow A$ \end_inset , then let \begin_inset Formula $B_{n}=A_{n}\setminus\bigcup_{m=1}^{n-1}A_{m}$ \end_inset . \end_layout \begin_layout Standard Then the \begin_inset Formula $B_{n}$ \end_inset are pairwise disjoint, \begin_inset Formula $A_{N}=\bigcup_{n=1}^{N}B_{n}$ \end_inset and \begin_inset Formula $A=\bigcup_{n=1}^{\infty}B_{n}$ \end_inset so \begin_inset Formula \[ \mu_{j}(A)=\sum_{n=1}^{\infty}\mu_{j}(B_{n})=\lim_{N\rightarrow\infty}\sum_{n=1}^{N}\mu_{j}(B_{n})=\lim_{N\rightarrow\infty}\mu_{j}(A_{N})\] \end_inset for \begin_inset Formula $j=1,2$ \end_inset , so as \begin_inset Formula $\mu_{1}(A_{N})=\mu_{2}(A_{N})$ \end_inset for each \begin_inset Formula $N$ \end_inset , we get \begin_inset Formula $\mu_{1}(A)=\mu_{2}(A)$ \end_inset so \begin_inset Formula $A\in\cD$ \end_inset . \end_layout \begin_layout Standard So \begin_inset Formula $\cD$ \end_inset is a \begin_inset Formula $d$ \end_inset -system. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection Borel measure on \begin_inset Formula $\R$ \end_inset \end_layout \begin_layout Standard To apply this theory to \begin_inset Formula $B((0,1])$ \end_inset , let \begin_inset Formula $\cF_{0}$ \end_inset be the family of finite unions of half-open intervals i.e. sets of the form \begin_inset Formula $\bigcup_{i=1}^{n}(a_{i},b_{i}]$ \end_inset . This is clearly a field, and \begin_inset Formula $\sigma(\cF_{0})=B((0,1])$ \end_inset . Every element of \begin_inset Formula $\cF_{0}$ \end_inset can be written as a finite union of disjoint intervals, and we can define \begin_inset Formula $\mu_{0}(\bigcup_{i=1}^{n}(a_{i},b_{i}])=\sum_{i=1}^{n}(b_{i}-a_{i})$ \end_inset if the intervals \begin_inset Formula $(a_{i},b_{i}]$ \end_inset are disjoint. It is straightforward to check that this is well-defined and finitely additive, but we need to check that it is countably additive. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset \begin_inset Formula $\mu_{0}$ \end_inset is countably additive. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset We begin by noting an alternative definition of countably additive: a function on a field is countably additive iff it is finitely additive and for any countable sequence \begin_inset Formula $\{F_{n}\}\subset\cF_{0}$ \end_inset s.t. \begin_inset Formula $F_{n}\downarrow\emptyset$ \end_inset (i.e. \begin_inset Formula $F_{n}\supset F_{n+1}\forall n$ \end_inset and \begin_inset Formula $\bigcap_{n}F_{n}=\emptyset$ \end_inset ), we have \begin_inset Formula $\mu_{0}(F_{n})\downarrow0$ \end_inset . This is because for any countable collection \begin_inset Formula $\{A_{n}\}\subset\cF_{0}$ \end_inset of disjoint sets s.t. \begin_inset Formula $A=\bigcup_{n}A_{n}\in\cF_{0}$ \end_inset , we have \begin_inset Formula $F_{n}=A\setminus\bigcup_{i=1}^{n}A_{i}\in\cF_{0}$ \end_inset with \begin_inset Formula $F_{n}\downarrow0$ \end_inset and \begin_inset Formula $\mu_{0}(F_{n})=\mu_{0}(A)-\sum_{i=1}^{n}\mu_{0}(A_{n})$ \end_inset . \end_layout \begin_layout Standard So given a sequence \begin_inset Formula $\{F_{n}\}\subset\cF_{0}$ \end_inset s.t. \begin_inset Formula $F_{n}\downarrow0$ \end_inset and \begin_inset Formula $\epsilon>0$ \end_inset , we need to show that \begin_inset Formula $\mu_{0}(F_{n})<\epsilon$ \end_inset for large enough \begin_inset Formula $n$ \end_inset . \end_layout \begin_layout Standard Each \begin_inset Formula $F_{n}$ \end_inset is a finite union of disjoint half-open intervals; we can make these intervals slightly smaller to get \begin_inset Formula $G_{n}\in\cF_{0}$ \end_inset such that \begin_inset Formula $\mu_{0}(F_{n}\setminus G_{n})<\epsilon2^{-n}$ \end_inset and \begin_inset Formula $\overline{G}_{n}\subset F_{n}$ \end_inset (the closure of \begin_inset Formula $G_{n}$ \end_inset in the topological sense). \end_layout \begin_layout Standard \begin_inset Formula $\overline{G}_{n}$ \end_inset are closed subsets of the compact space \begin_inset Formula $[0,1]$ \end_inset , and \begin_inset Formula $\overline{G}_{n}\downarrow\emptyset$ \end_inset so there is some \begin_inset Formula $N$ \end_inset for which \family roman \series medium \shape up \size normal \emph off \bar no \noun off \color none \begin_inset Formula \[ \bigcap_{n=1}^{N}\overline{G}_{n}=\emptyset\] \end_inset \end_layout \begin_layout Standard Now \begin_inset Formula \[ F_{N}=F_{N}\setminus\bigcap_{n=1}^{N}G_{n}=\bigcup_{n=1}^{N}(F_{N}\setminus G_{n})\subset\bigcup_{n=1}^{N}(F_{n}\setminus G_{n})\] \end_inset \end_layout \begin_layout Standard so \begin_inset Formula \[ \mu_{0}(F_{N})\leq\sum_{n=1}^{N}\mu_{0}(F_{n}\setminus G_{n})<\sum_{n=1}^{N}\epsilon2^{-n}<\epsilon\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard The Carathéodory Extension Theorem then tells us that we can extend \begin_inset Formula $\mu_{0}$ \end_inset to \begin_inset Formula $\sigma(\cF_{0})=B((0,1])$ \end_inset . Because \begin_inset Formula $\cF_{0}$ \end_inset is a \begin_inset Formula $\pi$ \end_inset -system, the extension is unique. This gives us the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{Borel measure} \end_layout \end_inset on \begin_inset Formula $B((0,1])$ \end_inset . \end_layout \begin_layout Subsection Lebesgue measure on \begin_inset Formula $\R$ \end_inset : Final details \end_layout \begin_layout Standard To get Lebesgue measure, there is one more step (which is rarely important). Let \begin_inset Formula $(\Omega,\cF,\mu)$ \end_inset be a measure space. If \begin_inset Formula $N\subset M\subset\Omega$ \end_inset and \begin_inset Formula $\mu(M)=0$ \end_inset , then we would expect that \begin_inset Formula $\mu(N)=0$ \end_inset . However, \begin_inset Formula $N$ \end_inset need not be in \begin_inset Formula $\cF$ \end_inset . So let the set of \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{null sets} \end_layout \end_inset be \begin_inset Formula \[ \cN=\{N\subset\Omega\colon N\subset M\text{ for some }M\text{ with }\mu(M)=0\}\] \end_inset \end_layout \begin_layout Standard Then define \begin_inset Formula $\cF^{*}=\{A\cup N\colon A\in\cF,N\in\cN\}$ \end_inset , the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{completion of $ \backslash cF$ with respect to $ \backslash mu$} \end_layout \end_inset . You can check that \begin_inset Formula $\cF^{*}$ \end_inset is a \begin_inset Formula $\sigma$ \end_inset -field, and there is a unique extension of \begin_inset Formula $\mu$ \end_inset to \begin_inset Formula $\cF^{*}$ \end_inset (example sheet 1, question 9). \end_layout \begin_layout Standard The \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{Lebesgue $ \backslash sigma$-field} \end_layout \end_inset of \begin_inset Formula $\R$ \end_inset is defined to be the completion of the Borel \begin_inset Formula $\sigma$ \end_inset -field with respect to Borel measure, and \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{Lebesgue measure} \end_layout \end_inset the corresponding measure. \end_layout \begin_layout Standard We know that Lebesgue measure satisfies \begin_inset Formula $\mu((a,b])=b-a$ \end_inset . But we started off looking for a measure satisfying the stronger condition of translation-invariance. You can check that in fact Lebesgue measure is translation-invariant, using example sheet 1, question 10. \end_layout \begin_layout Standard Note also that we can define Lebesgue measure on all of \begin_inset Formula $\R$ \end_inset , not just \begin_inset Formula $(0,1]$ \end_inset , by taking a copy on \begin_inset Formula $(n,n+1]$ \end_inset for each \begin_inset Formula $n\in\Z$ \end_inset , and letting a general set's measure be given by summing the measures of its intersections with each interval \begin_inset Formula $(n,n+1]$ \end_inset . \end_layout \begin_layout Subsection \begin_inset Formula $\liminf$ \end_inset and \begin_inset Formula $\limsup$ \end_inset \end_layout \begin_layout Standard \begin_inset Formula $\liminf$ \end_inset and \begin_inset Formula $\limsup$ \end_inset are operations on sequences from elementary analysis. They are not part of the schedules for any IA or IB course, so I will define them now. \end_layout \begin_layout Standard \begin_inset Formula \[ \limsup_{n\rightarrow\infty}x_{n}=\lim_{n\rightarrow\infty}\sup_{m\geq n}x_{m}\] \end_inset \begin_inset Formula \[ \liminf_{n\rightarrow\infty}x_{n}=\lim_{n\rightarrow\infty}\inf_{m\geq n}x_{m}\] \end_inset \end_layout \begin_layout Standard The values \begin_inset Formula $\sup_{m\geq n}x_{m}$ \end_inset are decreasing as \begin_inset Formula $n$ \end_inset increases, so the \begin_inset Formula $\limsup$ \end_inset always exists, providing we allow it to take the values \begin_inset Formula $\pm\infty$ \end_inset . These have the properties: \begin_inset Formula \[ \liminf x_{n}\leq\limsup x_{n}\] \end_inset \begin_inset Formula \[ \liminf x_{n}=\limsup x_{n}\mbox{ iff }x_{n}\mbox{ converges, in which case they are equal to }\lim x_{n}\] \end_inset \end_layout \begin_layout Standard Also useful are similar definitions for sequences of sets: \begin_inset Formula \[ \limsup_{n\rightarrow\infty}A_{n}=\bigcap_{n}\bigcup_{m\geq n}A_{n}\] \end_inset \begin_inset Formula \[ \liminf_{n\rightarrow\infty}A_{n}=\bigcup_{n}\bigcap_{m\geq n}A_{n}\] \end_inset \end_layout \begin_layout Standard When the \begin_inset Formula $A_{n}$ \end_inset are events, \begin_inset Formula $\limsup A_{n}$ \end_inset is the event \begin_inset Quotes eld \end_inset \begin_inset Formula $A_{n}$ \end_inset happens infinitely often \begin_inset Quotes erd \end_inset (i.e. for infinitely many \begin_inset Formula $n$ \end_inset ) and \begin_inset Formula $\liminf A_{n}$ \end_inset is the event \begin_inset Quotes eld \end_inset eventually \begin_inset Formula $A_{n}$ \end_inset always happens \begin_inset Quotes erd \end_inset (i.e. for all \begin_inset Formula $n$ \end_inset greater than some \begin_inset Formula $n_{0}$ \end_inset ). \end_layout \begin_layout Subsection Set table \end_layout \begin_layout Standard The different types of families of subsets of \begin_inset Formula $\Omega$ \end_inset are shown in the table. Note that a ring of sets is a ring in the algebraic sense, with symmetric difference as the addition operation and intersection as the multiplication. A field of sets is not an algebraic field. \end_layout \begin_layout Standard \align center \begin_inset Tabular \begin_inset Text \begin_layout Plain Layout Type of set \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout Closed under (definition) \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout Implies closed under \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout ring \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout symmetric difference \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout subtraction \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout finite intersection \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout finite union \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \begin_inset Formula $\sigma$ \end_inset -ring \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout symmetric difference \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout subtraction \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout countable intersection \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout countable union \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout field \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout contains \begin_inset Formula $\Omega$ \end_inset \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout finite intersection \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout (also called \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout complement \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout symmetric difference \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout algebra) \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout finite union \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout subtraction \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \begin_inset Formula $\sigma$ \end_inset -field \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout contains \begin_inset Formula $\Omega$ \end_inset \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout countable intersection \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout (or \begin_inset Formula $\sigma$ \end_inset -algebra) \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout complement \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout symmetric difference \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout countable union \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout subtraction \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \begin_inset Formula $\pi$ \end_inset -system \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout finite intersection \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \begin_inset Formula $d$ \end_inset -system \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout contains \begin_inset Formula $\Omega$ \end_inset \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout complement \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout subtraction (where \begin_inset Formula $A\supset B)$ \end_inset \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout increasing countable union \end_layout \end_inset \begin_inset Text \begin_layout Plain Layout \end_layout \end_inset \end_inset \end_layout \begin_layout Section Random variables \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status collapsed \begin_layout Plain Layout [2] Lebesgue-Stieltjes measure and probability distribution functions. \end_layout \begin_layout Plain Layout Measurable functions, random variables. \end_layout \end_inset \end_layout \begin_layout Subsection Measurable functions \end_layout \begin_layout Standard Having defined measure spaces, we now define their structure-preserving maps. This definition is essentially the same as the definition of continuous maps from one topological space to another: \end_layout \begin_layout Standard For measurable spaces \begin_inset Formula $(S_{1},\cS_{1}),(S_{2},\cS_{2})$ \end_inset , a function \begin_inset Formula $f:S_{1}\rightarrow S_{2}$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{measurable} \end_layout \end_inset (or properly \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$ \backslash cS_1$/$ \backslash cS_2$-measurable} \end_layout \end_inset ) if \begin_inset Formula $f^{-1}(A)\in\cS_{1}$ \end_inset for all \begin_inset Formula $A\in\cS_{2}$ \end_inset . \end_layout \begin_layout Standard The equivalent probabilistic concept is a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{random variable} \end_layout \end_inset , defined as a measurable function from a probability space to \begin_inset Formula $(\R,B(\R))$ \end_inset . \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset If \begin_inset Formula $\cS_{2}=\sigma(\cT)$ \end_inset and \begin_inset Formula $f:S_{1}\rightarrow S_{2}$ \end_inset satisfies \begin_inset Formula $f^{-1}(A)\in\cS_{1}$ \end_inset for all \begin_inset Formula $A\in\cT$ \end_inset , then \begin_inset Formula $f$ \end_inset is measurable. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Note that \begin_inset Formula $f^{-1}$ \end_inset preserves set operations: \begin_inset Formula $f^{-1}(\bigcup_{i}A_{i})=\bigcup_{i}f^{-1}(A_{i})$ \end_inset etc., so \begin_inset Formula $\cG=\{A\subset S_{2}\colon f^{-1}(A)\in\cS_{1}\}$ \end_inset has the same closure properties as \begin_inset Formula $\cS_{1}$ \end_inset , so is a \begin_inset Formula $\sigma$ \end_inset -field. \end_layout \begin_layout Standard We are given that \begin_inset Formula $\cG\supset\cT$ \end_inset , so \begin_inset Formula $\cG\supset\sigma(\cT)=\cS_{2}$ \end_inset , so \begin_inset Formula $f$ \end_inset is \begin_inset Formula $\cS_{1}$ \end_inset / \begin_inset Formula $\cS_{2}$ \end_inset -measurable. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard This lemma makes it easier to prove that functions are measurable by telling us two things: \end_layout \begin_layout Enumerate If the two spaces are the Borel \begin_inset Formula $\sigma$ \end_inset -fields of topological spaces, and \begin_inset Formula $f$ \end_inset is continuous, then \begin_inset Formula $f$ \end_inset is measurable. (Apply the lemma with \begin_inset Formula $\cT=\{\text{open sets of }S_{2}\}$ \end_inset .) \end_layout \begin_layout Enumerate If \begin_inset Formula $(S_{2},\cS_{2})=(\R,B(\R))$ \end_inset , then we only need to check that \begin_inset Formula $f^{-1}((-\infty,c))$ \end_inset is measurable for all \begin_inset Formula $c\in\R$ \end_inset , since \begin_inset Formula $B(\R)=\sigma(\{(-\infty,c)\})$ \end_inset . \end_layout \begin_layout Standard Measurable functions with range \begin_inset Formula $(\R,B(\R))$ \end_inset are the most common case, and we should check that these are closed under arithmetic operations. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset If \begin_inset Formula $f_{1},f_{2}:(S_{1},\cS_{1})\rightarrow(\R,B(\R))$ \end_inset are measurable, then \begin_inset Formula $f_{1}+f_{2}$ \end_inset , \begin_inset Formula $f_{1}f_{2}$ \end_inset (pointwise multiplication) and \begin_inset Formula $\lambda f_{1}(\lambda\in\R)$ \end_inset are measurable. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset We will only do \begin_inset Formula $f_{1}+f_{2}$ \end_inset here. The others are similar. \end_layout \begin_layout Standard \begin_inset Formula \[ f_{1}(\omega)+f_{2}(\omega)X(\omega)$ \end_inset and arbitrarily close to \begin_inset Formula $X(\omega)$ \end_inset s.t. \begin_inset Formula $F(t)\geq\omega$ \end_inset . So by the right-continuity of \begin_inset Formula $F$ \end_inset , \begin_inset Formula $F(X(\omega))\geq\omega$ \end_inset . \end_layout \begin_layout Standard So if \begin_inset Formula $X(\omega)\leq c$ \end_inset , then as \begin_inset Formula $F$ \end_inset is increasing, \begin_inset Formula $\omega\leq F(X(\omega))\leq F(c)$ \end_inset . \end_layout \begin_layout Standard Conversely, if \begin_inset Formula $\omega\leq F(c)$ \end_inset , then it is immediate from the definition of \begin_inset Formula $X$ \end_inset that \begin_inset Formula $X(\omega)\leq c$ \end_inset . \end_layout \begin_layout Standard Hence \begin_inset Formula $\{\omega:X\leq c\}=\{\omega:\omega\leq F(c)\}$ \end_inset , which is measurable, so \begin_inset Formula $X$ \end_inset is measurable and \begin_inset Formula \[ F_{X}(c)=\P(X\leq c)=\P(0<\omega\leq F(c))=F(c)\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard The distribution of \begin_inset Formula $X$ \end_inset gives us a measure \begin_inset Formula $\mu_{F}$ \end_inset for which the random variable \begin_inset Formula $I(\omega)=\omega$ \end_inset has distribution function \begin_inset Formula $F$ \end_inset . \end_layout \begin_layout Standard This is called the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{Lebesgue-Stieltjes measure} \end_layout \end_inset corresponding to \begin_inset Formula $F$ \end_inset . \end_layout \begin_layout Standard You can also construct the Lebesgue-Stieltjes measure by starting from \begin_inset Formula $\mu_{F}((a,b])=F(b)-F(a)$ \end_inset and following the same procedure as for the construction of Lebesgue measure, using the Carathéodory Extension Theorem. The only bit which requires a bit of extra work is checking that given a finite union of half-open intervals you can fit a compact set inside it while reducing its measure by less than \begin_inset Formula $\epsilon2^{n}$ \end_inset , which uses the right-continuity of \begin_inset Formula $F$ \end_inset . \end_layout \begin_layout Subsection Generated \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash titlesigma \end_layout \end_inset -fields \end_layout \begin_layout Standard For a function \begin_inset Formula $f:\Omega\rightarrow(S_{2},\cS_{2})$ \end_inset , the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$ \backslash sigma$-field generated by $f$} \end_layout \end_inset , \begin_inset Formula $\sigma(f)$ \end_inset , is defined to be the smallest \begin_inset Formula $\sigma$ \end_inset -field on \begin_inset Formula $\Omega$ \end_inset with respect to which \begin_inset Formula $f$ \end_inset is measurable. This can be defined, as with the \begin_inset Formula $\sigma$ \end_inset -field generated by a set, as the intersection of all \begin_inset Formula $\sigma$ \end_inset -fields for which \begin_inset Formula $f$ \end_inset is measurable, but more directly \begin_inset Formula \[ \sigma(f)=\{f^{-1}(A)\colon A\in\cS_{2}\}\] \end_inset This is because \begin_inset Formula $\{f^{-1}(A)\colon A\in\cS_{2}\}$ \end_inset certainly must be contained in a \begin_inset Formula $\sigma$ \end_inset -field for \begin_inset Formula $f$ \end_inset to be measurable, but you can check that it is itself a \begin_inset Formula $\sigma$ \end_inset -field. \end_layout \begin_layout Standard If \begin_inset Formula $X$ \end_inset is a random variable on \begin_inset Formula $(\Omega,\cF)$ \end_inset , then of course \begin_inset Formula $\sigma(X)\subset\cF$ \end_inset , but it can be much smaller. For example, if \begin_inset Formula $(\Omega,\cF)=((0,1],B((0,1]))$ \end_inset and \begin_inset Formula \[ X(\omega)=\begin{cases} 0 & \text{if }0<\omega\leq\frac{1}{2}\\ 1 & \text{if }\frac{1}{2}<\omega\leq1\end{cases}\] \end_inset \end_layout \begin_layout Standard then \begin_inset Formula $\sigma(X)$ \end_inset is just \begin_inset Formula $\{\emptyset,(0,\frac{1}{2}],(\frac{1}{2},1],(0,1]\}$ \end_inset . \end_layout \begin_layout Standard More generally, given a collection \begin_inset Formula $\{f_{\lambda}\colon\lambda\in\Lambda\}$ \end_inset of functions \begin_inset Formula $\Omega\rightarrow(S_{2},\cS_{2})$ \end_inset , the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$ \backslash sigma$-field generated by $ \backslash {f_ \backslash lambda \backslash }$} \end_layout \end_inset , \begin_inset Formula $\sigma(\{f_{\lambda}\})$ \end_inset , is the smallest \begin_inset Formula $\sigma$ \end_inset -field on \begin_inset Formula $\Omega$ \end_inset with respect to which all the \begin_inset Formula $f_{\lambda}$ \end_inset are measurable. \end_layout \begin_layout Standard Intuitively, if \begin_inset Formula $X_{\lambda}$ \end_inset are random variables, then \begin_inset Formula $\sigma(X_{\lambda})$ \end_inset is the set of events \begin_inset Formula $A$ \end_inset for which, if we know the values of all the variables \begin_inset Formula $X_{\lambda}$ \end_inset , then we know whether \begin_inset Formula $A$ \end_inset occurred or not. \end_layout \begin_layout Section Independence \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status collapsed \begin_layout Plain Layout [2] Independence of events, independence of \begin_inset Formula $\sigma$ \end_inset -algebras. The Borel-Cantelli lemmas. Kolmogorov's 0-1 law. \end_layout \begin_layout Plain Layout Independence of random variables. \end_layout \end_inset \end_layout \begin_layout Subsection Definitions \end_layout \begin_layout Standard So far, the ideas we have covered are interesting both from a pure mathematical and from a probabilistic perspective, although probability has often provided the easiest motivations and examples. Independence however is a fundamentally probabilistic concept. We work on a probability space \begin_inset Formula $(\Omega,\cF,\P)$ \end_inset . \end_layout \begin_layout Standard A sequence \begin_inset Formula $(A_{n})$ \end_inset of events is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{independent} \end_layout \end_inset if for any distinct \begin_inset Formula $n_{1},\ldots,n_{k}$ \end_inset , we have \begin_inset Formula \[ \P(A_{n_{1}}\cap\ldots\cap A_{n_{k}})=\prod_{j=1}^{k}\P(A_{n_{j}})\] \end_inset \end_layout \begin_layout Standard A sequence \begin_inset Formula $(\cG_{n})$ \end_inset of sub- \begin_inset Formula $\sigma$ \end_inset -fields of \begin_inset Formula $\cF$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{independent} \end_layout \end_inset if for any distinct \begin_inset Formula $n_{1},\ldots,n_{k}$ \end_inset and events \begin_inset Formula $A_{j}\in\cG_{n_{j}},$ \end_inset we have \begin_inset Formula \[ \P(A_{1}\cap\ldots\cap A_{k})=\prod_{j=1}^{k}\P(A_{j})\] \end_inset \end_layout \begin_layout Standard The same definition will apply for \begin_inset Formula $\pi$ \end_inset -systems (or indeed any families of events). We only really care about \begin_inset Formula $\sigma$ \end_inset -fields, but having the definition for \begin_inset Formula $\pi$ \end_inset -systems makes it easier to show that \begin_inset Formula $\sigma$ \end_inset -fields are independent, using the following lemma. It is proved here only for two \begin_inset Formula $\pi$ \end_inset -systems, but can be extended to all sequences (to get it for infinite sequences , use the fact that a sequence of \begin_inset Formula $\sigma$ \end_inset -fields/ \begin_inset Formula $\pi$ \end_inset -systems is independent iff all its finite subsequences are). \end_layout \begin_layout Standard A sequence \begin_inset Formula $(X_{n})$ \end_inset of random variables is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{independent} \end_layout \end_inset if the \begin_inset Formula $\sigma$ \end_inset -fields \begin_inset Formula $\sigma(X_{n})$ \end_inset are independent. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset \begin_inset CommandInset label LatexCommand label name "lemma:pi-independent" \end_inset If \begin_inset Formula $\cI,\cJ$ \end_inset are independent \begin_inset Formula $\pi$ \end_inset -systems with \begin_inset Formula $\sigma(\cI)=\cG,\sigma(\cJ)=\cH$ \end_inset , then \begin_inset Formula $\cG,\cH$ \end_inset are independent. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Fix \begin_inset Formula $A\in\cI$ \end_inset . \end_layout \begin_layout Standard Define measures \begin_inset Formula $\mu_{1},\mu_{2}$ \end_inset on \begin_inset Formula $\cH$ \end_inset by \begin_inset Formula \[ \mu_{1}(B)=\P(A\cap B),\:\mu_{2}(B)=\P(A)\P(B)\] \end_inset \end_layout \begin_layout Standard It is easy to check that these are measures. We are given that \begin_inset Formula $\mu_{1}(B)=\mu_{2}(B)$ \end_inset for all \begin_inset Formula $B\in\cJ$ \end_inset , and of course \begin_inset Formula $\mu_{1}(\Omega)=\mu_{2}(\Omega)$ \end_inset , so by Lemma \begin_inset CommandInset ref LatexCommand ref reference "lemma:pi-unique-measure" \end_inset , \begin_inset Formula $\mu_{1}=\mu_{2}$ \end_inset . \end_layout \begin_layout Standard Applying this for all \begin_inset Formula $A\in\cI$ \end_inset , we get \begin_inset Formula $\P(A\cap B)=\P(A)\P(B)$ \end_inset for all \begin_inset Formula $A\in\cI,B\in\cH$ \end_inset . \end_layout \begin_layout Standard Now we repeat the other way round: fix \begin_inset Formula $B\in\cH$ \end_inset and define measures \begin_inset Formula $\mu_{3},\mu_{4}$ \end_inset on \begin_inset Formula $\cG$ \end_inset by \begin_inset Formula \[ \mu_{3}(A)=\P(A\cap B),\:\mu_{4}(A)=\P(A)\P(B)\] \end_inset \end_layout \begin_layout Standard We have just shown that \begin_inset Formula $\mu_{3}(A)=\mu_{4}(A)$ \end_inset for all \begin_inset Formula $A\in\cI$ \end_inset , so using Lemma \begin_inset CommandInset ref LatexCommand ref reference "lemma:pi-unique-measure" \end_inset again, we get \begin_inset Formula $\mu_{3}=\mu_{4}$ \end_inset . \end_layout \begin_layout Standard Again, this holds for arbitrary \begin_inset Formula $B$ \end_inset , so \begin_inset Formula $\P(A\cap B)=\P(A)\P(B)$ \end_inset for all \begin_inset Formula $A\in\cG,B\in\cH$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection The Borel-Cantelli lemmas \end_layout \begin_layout Standard The two Borel-Cantelli lemmas are useful tools to prove results about sequences of events. The proof of the first lemma appears very quick and simple, but it is probably the more important of the two. There are many problems where, if you don't quote this lemma, you will re-write its proof from scratch -- and it's harder to come up with in the context of a particular problem than as a general statement. Note that the first lemma does not require independence. Indeed it holds not only in probability spaces, but in general measure spaces. \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{lemma}[First Borel-Cantelli Lemma] \end_layout \end_inset If \begin_inset Formula $(A_{n})$ \end_inset is a sequence of events with \begin_inset Formula $\sum\P(A_{n})<\infty$ \end_inset , then \begin_inset Formula $\P(\limsup A_{n})=0$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset \begin_inset Formula \[ \P(\limsup A_{n})=\P(\bigcap_{k}\bigcup_{n\geq k}A_{n})\leq\P(\bigcup_{n\geq k}A_{n})\leq\sum_{n\geq k}\P(A_{n})\rightarrow0\text{ as }k\rightarrow\infty\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard The second lemma does require independence. This is easy to see because for example if \begin_inset Formula $A_{n}$ \end_inset is the same event for all \begin_inset Formula $n$ \end_inset , then \begin_inset Formula $\sum\P(A_{n})=\infty$ \end_inset but \begin_inset Formula $\P(\limsup A_{n})\neq1$ \end_inset . The proof is more involved and will use the following lemma. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset If \begin_inset Formula $(p_{n})$ \end_inset is a real sequence with \begin_inset Formula $0\leq p_{n}\leq1$ \end_inset and \begin_inset Formula $\sum p_{n}=\infty$ \end_inset , then \begin_inset Formula $\prod(1-p_{n})=0$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset \begin_inset Formula \[ 1-x\leq exp(-x)\] \end_inset so \begin_inset Formula \[ \prod_{n=1}^{N}(1-p_{n})\leq\prod_{n=1}^{N}exp(-p_{n})=exp(-\sum_{n=1}^{N}p_{n})\rightarrow0\mbox{ since }\sum p_{n}\rightarrow\infty\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{lemma}[Second Borel-Cantelli Lemma] \end_layout \end_inset If \begin_inset Formula $(A_{n})$ \end_inset is a sequence of independent events with \begin_inset Formula $\sum\P(A_{n})=\infty$ \end_inset , then \begin_inset Formula $\P(\limsup A_{n})=1$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset It is usually easier to prove that a probability is zero rather than one, so we consider \begin_inset Formula $(\limsup A_{n})^{c}=\liminf A_{n}^{c}$ \end_inset instead. Note that if \begin_inset Formula $(A_{n})$ \end_inset is independent, then so is \begin_inset Formula $(A_{n}^{c})$ \end_inset . \end_layout \begin_layout Standard Let \begin_inset Formula $p_{n}=\P(A_{n})$ \end_inset . For any \begin_inset Formula $k$ \end_inset , \begin_inset Formula \[ \P(\bigcap_{n\geq k}A_{n}^{c})\leq\P(\bigcap_{n=k}^{m}A_{n}^{c})=\prod_{n=k}^{m}\P(A_{n}^{c})=\prod_{n=k}^{m}(1-p_{n})\rightarrow0\mbox{ as }m\rightarrow\infty\] \end_inset \end_layout \begin_layout Standard by the previous lemma. So \begin_inset Formula \[ \P(\bigcap_{n\geq k}A_{n}^{c})=0\] \end_inset \end_layout \begin_layout Standard for all \begin_inset Formula $k$ \end_inset , and taking a countable union, \begin_inset Formula \[ \P(\liminf A_{n}^{c})=\P(\bigcup_{k}\bigcap_{n\geq k}A_{n}^{c})=0\qedhere\] \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection Tail \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash titlesigma \end_layout \end_inset -fields \end_layout \begin_layout Standard Given a sequence \begin_inset Formula $(X_{n})$ \end_inset of random variables, let \begin_inset Formula $\cT_{n}=\sigma(X_{n},X_{n+1},\ldots)$ \end_inset . \end_layout \begin_layout Standard Then the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{tail $ \backslash sigma$-field} \end_layout \end_inset of the sequence is \begin_inset Formula $\cT=\bigcap_{n}\cT_{n}$ \end_inset . \end_layout \begin_layout Standard So \begin_inset Formula $\cT$ \end_inset is the \begin_inset Formula $\sigma$ \end_inset -field of events which are determined by the values of \begin_inset Formula $X_{n}$ \end_inset , but are not affected by any finite number of these values. For example, the event \begin_inset Quotes eld \end_inset \begin_inset Formula $\lim X_{n}$ \end_inset exists \begin_inset Quotes erd \end_inset is in \begin_inset Formula $\cT$ \end_inset , and the random variables \begin_inset Formula $\limsup X_{n}$ \end_inset , \begin_inset Formula $\liminf X_{n}$ \end_inset are \begin_inset Formula $\cT$ \end_inset -measurable. \end_layout \begin_layout Standard Kolmogorov's 0-1 Law tells us that if the \begin_inset Formula $X_{n}$ \end_inset are independent, then a tail event has probability either 0 or 1. However, it may not be easy to tell which. (Sometimes, but not always, the Borel-Cantelli lemmas can help.) \end_layout \begin_layout Standard A \begin_inset Formula $\sigma$ \end_inset -field \begin_inset Formula $\cF$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{trivial} \end_layout \end_inset (with respect to a probability measure \begin_inset Formula $\P$ \end_inset ) if \begin_inset Formula $\P(A)=0\mbox{ or }1$ \end_inset for all \begin_inset Formula $A\in\cF$ \end_inset . The usual way to prove that a \begin_inset Formula $\sigma$ \end_inset -field is trivial is to show that it is independent of itself, as then for any \begin_inset Formula $A\in\cF$ \end_inset , \begin_inset Formula \[ \P(A)=\P(A\cap A)=\P(A)\P(A)\] \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Kolmogorov's 0-1 Law] \end_layout \end_inset The tail \begin_inset Formula $\sigma$ \end_inset -field of a sequence of independent random variables is trivial. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Let \begin_inset Formula $\cF_{n}=\sigma(X_{1},\ldots,X_{n})$ \end_inset . \end_layout \begin_layout Standard First we show that \begin_inset Formula $\cF_{n}$ \end_inset and \begin_inset Formula $\cT_{n+1}$ \end_inset are independent (i.e. the first \begin_inset Formula $n$ \end_inset variables are independent of all the rest). \end_layout \begin_layout Standard Let \begin_inset Formula \[ \cI_{n}=\{\{\omega\colon X_{1}\leq c_{1},\ldots,X_{n}\leq c_{n}\}\colon c_{1},\ldots,c_{n}\in\R\},\] \end_inset \begin_inset Formula \[ \cJ_{n}=\{\{\omega\colon X_{n}\leq c_{n},\ldots,X_{n+r}\leq c_{n+r}\}\colon c_{n},\ldots,c_{n+r}\in\R,r\in\N\}\] \end_inset \end_layout \begin_layout Standard Then \begin_inset Formula $\cI_{n},\cJ_{n+1}$ \end_inset are \begin_inset Formula $\pi$ \end_inset -systems, generate \begin_inset Formula $\cF_{n},\cT_{n+1}$ \end_inset respectively, and are certainly independent. So by lemma \begin_inset CommandInset ref LatexCommand ref reference "lemma:pi-independent" \end_inset , \begin_inset Formula $\cF_{n}$ \end_inset and \begin_inset Formula $\cT_{n+1}$ \end_inset are independent. \end_layout \begin_layout Standard Since \begin_inset Formula $\cT\subset\cT_{n+1}$ \end_inset , we get that \begin_inset Formula $\cT$ \end_inset is independent of \begin_inset Formula $\cF_{n}$ \end_inset for all \begin_inset Formula $n$ \end_inset . This is certainly what we would expect intuitively: tail events are independent of finite subsequences of \begin_inset Formula $(X_{n})$ \end_inset . \begin_inset VSpace defskip \end_inset \end_layout \begin_layout Standard Now we show that \begin_inset Formula $\cT$ \end_inset is independent of \begin_inset Formula $\cT_{1}=\sigma(X_{1},X_{2},\ldots)$ \end_inset . \end_layout \begin_layout Standard Because \begin_inset Formula $\cF_{n}\subset\cF_{n+1}$ \end_inset and they are closed under intersection, the union \begin_inset Formula $\cI_{\infty}=\bigcup_{n}\cF_{n}$ \end_inset is a \begin_inset Formula $\pi$ \end_inset -system. It must be contained in any \begin_inset Formula $\sigma$ \end_inset -field on which all \begin_inset Formula $X_{n}$ \end_inset are measurable, so \begin_inset Formula $\cI_{\infty}\subset\cT_{1}$ \end_inset ; and all \begin_inset Formula $X_{n}$ \end_inset are measurable on any \begin_inset Formula $\sigma$ \end_inset -field containing \begin_inset Formula $\cI_{\infty}$ \end_inset , so \begin_inset Formula $\sigma(\cI_{\infty})\supset\cT_{1}$ \end_inset . So \begin_inset Formula $\sigma(\cI_{\infty})=\cT_{1}$ \end_inset . \end_layout \begin_layout Standard If \begin_inset Formula $A\in\cI_{\infty}$ \end_inset and \begin_inset Formula $B\in\cT$ \end_inset , then \begin_inset Formula $A\in\cF_{n}$ \end_inset for some \begin_inset Formula $n$ \end_inset and \begin_inset Formula $\cF_{n}$ \end_inset is independent of \begin_inset Formula $\cT$ \end_inset , so \begin_inset Formula $\P(A\cap B)=\P(A)\P(B)$ \end_inset . So \begin_inset Formula $\cI_{\infty}$ \end_inset and \begin_inset Formula $\cT$ \end_inset are independent, so \begin_inset Formula $\cT_{1}$ \end_inset and \begin_inset Formula $\cT$ \end_inset are independent. \end_layout \begin_layout Standard Finally \begin_inset Formula $\cT\subset\cT_{1}$ \end_inset , so \begin_inset Formula $\cT$ \end_inset is independent of itself. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard It follows that if \begin_inset Formula $Y$ \end_inset is a tail-measurable random variable, then \begin_inset Formula $\P(Y\leq c)=0\mbox{ or }1$ \end_inset for all \begin_inset Formula $c\in\R$ \end_inset , so \begin_inset Formula $Y$ \end_inset is almost surely constant. \end_layout \begin_layout Section Integration \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status collapsed \begin_layout Plain Layout [2] Construction of the integral, expectation. \end_layout \begin_layout Plain Layout Fatou's lemma, monotone and dominated convergence, differentiation under the integral sign. \end_layout \end_inset \end_layout \begin_layout Subsection Definition of integration \end_layout \begin_layout Standard Let \begin_inset Formula $(S,\cS,\mu)$ \end_inset be a \begin_inset Formula $\sigma$ \end_inset -finite measure space and write \begin_inset Formula $\mS$ \end_inset for the set of measurable functions \begin_inset Formula $(S,\cS)\rightarrow(\R,B(\R))$ \end_inset and \begin_inset Formula $\mSp$ \end_inset for the set of non-negative measurable functions. For \begin_inset Formula $f\in\mSp$ \end_inset , we aim to define the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{integral} \end_layout \end_inset of \begin_inset Formula $f$ \end_inset with respect to \begin_inset Formula $\mu$ \end_inset . (Allowing \begin_inset Formula $f$ \end_inset to take negative values introduces complications we will consider later.) There are a confusingly large number of notations for this: \begin_inset Formula \[ \mu(f)=\int_{S}fd\mu=\int_{S}f(x)\mu(dx)\] \end_inset \end_layout \begin_layout Standard In probability, the equivalent concept is expectation. For a random variable \begin_inset Formula $X:(\Omega,\cF,\P)\rightarrow\R$ \end_inset , the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{expectation} \end_layout \end_inset of \begin_inset Formula $X$ \end_inset is \begin_inset Formula \[ \E(X)=\int_{\Omega}Xd\P\] \end_inset \end_layout \begin_layout Standard We begin with the indicator functions of measurable sets. For these, if \begin_inset Formula $A$ \end_inset is measurable, then \begin_inset Formula \[ \int I_{A}d\mu=\mu(A)\] \end_inset (In probabilistic terms, this is the statement \begin_inset Formula $\E(I_{A})=\P(A)$ \end_inset .) \end_layout \begin_layout Standard The integration operator should be linear, so we can extend this to finite combinations of indicators. A function \begin_inset Formula $f:S\rightarrow\R$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{simple} \end_layout \end_inset if \begin_inset Formula \[ f=\sum_{k=1}^{n}\alpha_{k}I_{A_{k}}\mbox{ for some measurable sets }A_{k}\mbox{ and constants }\alpha_{k}\in\R\] \end_inset The set of simple functions is denoted \begin_inset Formula $SF$ \end_inset , and of non-negative simple functions \begin_inset Formula $SF^{+}$ \end_inset . For \begin_inset Formula $f\in SF^{+}$ \end_inset , we define \begin_inset Formula \[ \int\left(\sum_{k=1}^{n}\alpha_{k}I_{A_{k}}\right)d\mu=\sum_{k=1}^{n}\alpha_{k}\mu(A_{k})\] \end_inset We need to check that this is well-defined, because a single simple function will have many different representations of the form \begin_inset Formula $f=\sum_{k=1}^{n}\alpha_{k}I_{A_{k}}$ \end_inset . This is fairly boring and routine. Note that we can always find such a representation with the sets \begin_inset Formula $A_{k}$ \end_inset disjoint, as a simple function can take only finitely many values \begin_inset Formula $c_{1},\ldots,c_{k}$ \end_inset ; then let \begin_inset Formula $A_{k}=f^{-1}(\{c_{k}\})$ \end_inset . Note also that (by writing the functions on each side as suitable sums of indicators) if \begin_inset Formula $f_{j}$ \end_inset are simple functions and \begin_inset Formula $\beta_{j}\geq0$ \end_inset then \begin_inset Formula \[ \int\left(\sum_{j=1}^{n}\beta_{j}f_{j}\right)d\mu=\sum_{j=1}^{n}\beta_{j}\int f_{j}d\mu\] \end_inset \end_layout \begin_layout Standard The simple functions are good enough to approximate all measurable functions from below: given \begin_inset Formula $f\in\mSp$ \end_inset , let \begin_inset Formula \[ f_{n}(x)=2^{-n}\left\lfloor 2^{n}\min\{f(x),n\}\right\rfloor =2^{-n}\sum_{j=1}^{n2^{n}}I_{\{x:f(x)\geq j2^{-n}\}}\] \end_inset Then \begin_inset Formula $f_{n}$ \end_inset is an increasing sequence which converges to \begin_inset Formula $f$ \end_inset pointwise, and indeed \begin_inset Formula $f_{n}\rightarrow f$ \end_inset uniformly on any set of the form \begin_inset Formula $\{x\colon f(x)\leq k\}$ \end_inset . \end_layout \begin_layout Standard So for general \begin_inset Formula $f\in\mSp$ \end_inset , we define \begin_inset Formula \[ \int fd\mu=\sup\{{\textstyle \int}gd\mu\colon g\leq f,g\in SF^{+}\}\] \end_inset This may be \begin_inset Formula $+\infty$ \end_inset . \begin_inset Formula $f\in\mSp$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{integrable} \end_layout \end_inset if \begin_inset Formula $\int fd\mu<\infty$ \end_inset . \end_layout \begin_layout Subsection Monotone Convergence Theorem \end_layout \begin_layout Standard The key theorem about integrals is the Monotone Convergence Theorem: \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Monotone Convergence Theorem] \end_layout \end_inset If \begin_inset Formula $f_{n}\in\mSp$ \end_inset and \begin_inset Formula $f_{n}\uparrow f$ \end_inset (pointwise) then \begin_inset Formula \[ \int f_{n}d\mu\:\big\uparrow\int fd\mu\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard To prove this theorem, we go through a couple of restricted versions. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset If \begin_inset Formula $f_{n}\in SF^{+},A\in\cS$ \end_inset and \begin_inset Formula $f_{n}\uparrow I_{A}$ \end_inset , then \begin_inset Formula $\int f_{n}d\mu\uparrow\mu(A)$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Certainly \begin_inset Formula $\int f_{n}d\mu\leq\mu(A)$ \end_inset for each \begin_inset Formula $n$ \end_inset and \begin_inset Formula $\int f_{n}d\mu$ \end_inset is increasing. We just need to show that \begin_inset Formula $\int f_{n}d\mu$ \end_inset becomes arbitrarily close to \begin_inset Formula $\mu(A)$ \end_inset . \end_layout \begin_layout Standard Given \begin_inset Formula $\epsilon>0$ \end_inset , let \begin_inset Formula $A_{n}=\{\omega\colon f_{n}(\omega)\geq1-\epsilon\}$ \end_inset . \end_layout \begin_layout Standard Since \begin_inset Formula $f_{n}\uparrow I_{A}$ \end_inset , \begin_inset Formula $A_{n}\uparrow A$ \end_inset and so \begin_inset Formula $\mu(A_{n})\uparrow\mu(A)$ \end_inset . \end_layout \begin_layout Standard Hence we can choose \begin_inset Formula $n$ \end_inset s.t. \begin_inset Formula $\mu(A_{n})\geq(1-\epsilon)\mu(A)$ \end_inset . \end_layout \begin_layout Standard Note that, by the definition of \begin_inset Formula $A_{n}$ \end_inset , \begin_inset Formula $f_{n}\geq(1-\epsilon)I_{A_{n}}$ \end_inset , so \begin_inset Formula \[ \int f_{n}d\mu\geq(1-\epsilon)\mu(A_{n})\geq(1-\epsilon)^{2}\mu(A)\] \end_inset \end_layout \begin_layout Standard Since \begin_inset Formula $\epsilon$ \end_inset is arbitrary, we are done. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{corollary} \end_layout \end_inset If \begin_inset Formula $f_{n},f\in SF^{+}$ \end_inset and \begin_inset Formula $f_{n}\uparrow f$ \end_inset , then \begin_inset Formula $\int f_{n}d\mu\uparrow\int fd\mu$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{corollary} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Write \begin_inset Formula $f=\sum_{k}\alpha_{k}I_{A_{k}}$ \end_inset where \begin_inset Formula $A_{k}$ \end_inset are disjoint measurable sets. Then \begin_inset Formula $f_{n}I_{A_{k}}\uparrow\alpha_{k}I_{A_{k}}$ \end_inset for each \begin_inset Formula $k$ \end_inset , so we can apply the lemma for each \begin_inset Formula $k$ \end_inset and add up. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset If \begin_inset Formula $f_{n}\in SF^{+},f\in\mSp$ \end_inset and \begin_inset Formula $f_{n}\uparrow f$ \end_inset then \begin_inset Formula $\int f_{n}d\mu\uparrow\int fd\mu$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Again \begin_inset Formula $\int f_{n}d\mu\leq\int fd\mu$ \end_inset for all \begin_inset Formula $n$ \end_inset and \begin_inset Formula $\int f_{n}d\mu$ \end_inset is increasing, so we just need to check that \begin_inset Formula $\int f_{n}d\mu$ \end_inset becomes arbitrarily close to \begin_inset Formula $\int fd\mu$ \end_inset . \end_layout \begin_layout Standard Given \begin_inset Formula $\epsilon>0$ \end_inset , there exists some \begin_inset Formula $g\in SF^{+}$ \end_inset s.t. \begin_inset Formula $g\leq f$ \end_inset and \begin_inset Formula $\int gd\mu>\int fd\mu-\epsilon$ \end_inset (by the definition of \begin_inset Formula $\int fd\mu$ \end_inset ). \end_layout \begin_layout Standard Let \begin_inset Formula $g_{n}=\min\{g,f_{n}\}$ \end_inset . Now \begin_inset Formula $g_{n}\uparrow g$ \end_inset and \begin_inset Formula $g_{n},g$ \end_inset are simple, so by the above corollary we have \begin_inset Formula $\int g_{n}d\mu\uparrow\int gd\mu$ \end_inset . \end_layout \begin_layout Standard Then for large enough \begin_inset Formula $n$ \end_inset , \end_layout \begin_layout Standard \begin_inset Formula \[ {\textstyle \int}f_{n}d\mu\geq{\textstyle \int}g_{n}d\mu>{\textstyle \int}gd\mu-\epsilon>{\textstyle \int}fd\mu-2\epsilon\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard Now we are ready for the main theorem. The proof uses the following lemma from elementary analysis, left as an exercise: \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset If \begin_inset Formula $(a_{nk})_{n,k\in\N}$ \end_inset are real numbers increasing with both \begin_inset Formula $n$ \end_inset and \begin_inset Formula $k$ \end_inset , then \begin_inset Formula \[ \lim_{n\rightarrow\infty}\lim_{k\rightarrow\infty}a_{nk}=\lim_{n\rightarrow\infty}a_{nn}\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Monotone Convergence Theorem] \end_layout \end_inset If \begin_inset Formula $f_{n}\in\mSp$ \end_inset and \begin_inset Formula $f_{n}\uparrow f$ \end_inset then \begin_inset Formula $\int f_{n}d\mu\uparrow\int fd\mu$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset First note that \begin_inset Formula $f^{-1}((-\infty,c])=\bigcap_{n}f_{n}^{-1}((-\infty,c])$ \end_inset so \begin_inset Formula $f$ \end_inset is measurable and \begin_inset Formula $\int fd\mu$ \end_inset is defined. \end_layout \begin_layout Standard For each \begin_inset Formula $n$ \end_inset , let \begin_inset Formula $(g_{nk})_{k\in\N}$ \end_inset be a sequence of simple functions s.t. \begin_inset Formula $g_{nk}\uparrow f_{n}$ \end_inset as \begin_inset Formula $k\rightarrow\infty$ \end_inset . \end_layout \begin_layout Standard For each \begin_inset Formula $n,k$ \end_inset , let \begin_inset Formula $h_{nk}=\max\{g_{mk}:m\leq n\}$ \end_inset . The \begin_inset Formula $h_{nk}$ \end_inset are certainly increasing in both \begin_inset Formula $n$ \end_inset and \begin_inset Formula $k$ \end_inset , and since \begin_inset Formula $g_{nk}\leq h_{nk}\leq f_{n}$ \end_inset , \begin_inset Formula $h_{nk}\uparrow f_{n}$ \end_inset . \end_layout \begin_layout Standard So for each \begin_inset Formula $\omega$ \end_inset we get \begin_inset Formula \[ \lim_{n\rightarrow\infty}h_{nn}(\omega)=\lim_{n\rightarrow\infty}\lim_{k\rightarrow\infty}h_{nk}(\omega)=\lim_{n\rightarrow\infty}f_{n}(\omega)=f(\omega)\] \end_inset \end_layout \begin_layout Standard So \begin_inset Formula $h_{nn}\uparrow f$ \end_inset . Since \begin_inset Formula $h_{nk}$ \end_inset is the max of a finite collection of simple functions, \begin_inset Formula $h_{nk}$ \end_inset is itself simple. So by the earlier lemma, we get \begin_inset Formula \[ \int fd\mu=\lim_{n\rightarrow\infty}\int h_{nn}d\mu=\lim_{n\rightarrow\infty}\lim_{k\rightarrow\infty}\int h_{nk}d\mu=\lim_{n\rightarrow\infty}\int f_{n}d\mu\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection Integration of functions with negative values, and \begin_inset Formula $\cL^{1}$ \end_inset \end_layout \begin_layout Standard Functions which can take negative values require a little more care. \end_layout \begin_layout Standard Write the positive and negative parts of \begin_inset Formula $f$ \end_inset as \begin_inset Formula $f^{+}=\max\{f,0\},f^{-}=\max\{-f,0\}$ \end_inset , so that \begin_inset Formula $f=f^{+}-f^{-}$ \end_inset and \begin_inset Formula $|f|=f^{+}+f^{-}$ \end_inset . \end_layout \begin_layout Standard Then \begin_inset Formula $f\in\mS$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{integrable} \end_layout \end_inset if \begin_inset Formula $\int|f|d\mu<\infty$ \end_inset , and \begin_inset Formula \[ \int fd\mu=\int f^{+}d\mu-\int f^{-}d\mu\] \end_inset \end_layout \begin_layout Standard It is necessary to rule out \begin_inset Formula $\int f^{+}d\mu=\int f^{-}d\mu=\infty$ \end_inset to avoid an undefined \begin_inset Formula $\infty-\infty$ \end_inset , and to rule out cases in which just one of \begin_inset Formula $\int f^{+}d\mu,\int f^{-}d\mu=\infty$ \end_inset to ensure that the set of integrable functions is closed under addition and subtraction. \end_layout \begin_layout Standard Write \begin_inset Formula $\cL^{1}(S,\cS,\mu)$ \end_inset for the set of integrable functions on a \begin_inset Formula $\sigma$ \end_inset -finite measure space \begin_inset Formula $(S,\cS,\mu)$ \end_inset . \end_layout \begin_layout Standard The monotone convergence theorem extends to functions in \begin_inset Formula $\cL^{1}$ \end_inset , as long as \begin_inset Formula $\lim\int f_{n}d\mu$ \end_inset is finite. \end_layout \begin_layout Subsection Other convergence theorems \end_layout \begin_layout Standard There are a number of important consequences of the monotone convergence theorem. Note that the first applies only to nonnegative functions. \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{lemma}[Fatou's Lemma] \end_layout \end_inset If \begin_inset Formula $f_{n}\in\mSp$ \end_inset then \begin_inset Formula \[ \int\liminf f_{n}d\mu\leq\liminf\int f_{n}d\mu\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Let \begin_inset Formula $g_{n}={\displaystyle \inf_{m\geq n}}f_{m}$ \end_inset . (This is why we need that \begin_inset Formula $f_{n}\geq0$ \end_inset -- it follows that \begin_inset Formula $g_{n}\geq0$ \end_inset too, so \begin_inset Formula $\int g_{n}$ \end_inset is defined. If we allowed functions with negative values then \begin_inset Formula $g_{n}$ \end_inset need not be integrable.) \end_layout \begin_layout Standard Then \begin_inset Formula $g_{n}\uparrow\liminf f_{n}$ \end_inset so by MCT, \begin_inset Formula $\int g_{n}d\mu\uparrow\int\liminf f_{n}d\mu$ \end_inset . \end_layout \begin_layout Standard Also \begin_inset Formula $\int g_{n}d\mu\leq\int f_{m}d\mu$ \end_inset for all \begin_inset Formula $m\geq n$ \end_inset so \begin_inset Formula \[ {\textstyle \int}g_{n}d\mu\leq\inf_{m\geq n}{\textstyle \int}f_{m}d\mu\] \end_inset \end_layout \begin_layout Standard And so \begin_inset Formula \[ {\textstyle \int}\liminf f_{n}d\mu=\lim{\textstyle \int}g_{n}d\mu\leq\liminf{\textstyle \int}f_{m}d\mu\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{lemma}[Reverse Fatou's Lemma] \end_layout \end_inset If \begin_inset Formula $f_{n}\in\cL^{1}$ \end_inset and there exists an integrable function \begin_inset Formula $g$ \end_inset s.t. \begin_inset Formula $f_{n}\leq g$ \end_inset for all \begin_inset Formula $n$ \end_inset , then \begin_inset Formula \[ \int\limsup f_{n}d\mu\geq\limsup\int f_{n}d\mu\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Apply Fatou's lemma to the non-nonnegative functions \begin_inset Formula $g-f_{n}$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Dominated Convergence Theorem] \end_layout \end_inset If \begin_inset Formula $f_{n}\in\cL^{1}$ \end_inset , there exists an integrable \begin_inset Formula $g$ \end_inset s.t. \begin_inset Formula $\vert f_{n}\vert\leq g$ \end_inset for all \begin_inset Formula $n$ \end_inset , and \begin_inset Formula $f_{n}\rightarrow f$ \end_inset (pointwise) then \begin_inset Formula \[ \int|f_{n}-f|d\mu\rightarrow0\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Since \begin_inset Formula $f_{n}\rightarrow f$ \end_inset we have \begin_inset Formula $\limsup|f_{n}-f|=0$ \end_inset . \end_layout \begin_layout Standard Also, by the triangle inequality, \begin_inset Formula \[ |f_{n}-f|\leq2g\] \end_inset \end_layout \begin_layout Standard so we can apply reverse Fatou to get \family roman \series medium \shape up \size normal \emph off \bar no \noun off \color none \begin_inset Formula \[ 0={\textstyle \int}\limsup|f_{n}-f|d\mu\geq\limsup{\textstyle \int}|f_{n}-f|d\mu\geq0\] \end_inset \end_layout \begin_layout Standard and so \begin_inset Formula \[ {\textstyle \int}|f_{n}-f|d\mu\rightarrow0\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{corollary}[Bounded Convergence Theorem] \end_layout \end_inset If \begin_inset Formula $X_{n}$ \end_inset are random variables on a probability space \begin_inset Formula $(\Omega,\cF,\P)$ \end_inset , there is a constant \begin_inset Formula $K$ \end_inset s.t. \begin_inset Formula $\vert X_{n}\vert\leq K$ \end_inset for all \begin_inset Formula $n$ \end_inset , and \begin_inset Formula $X_{n}\rightarrow X$ \end_inset (pointwise) then \begin_inset Formula $\E(|X_{n}-X|)\rightarrow0$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{corollary} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Because \begin_inset Formula $\P(\Omega)$ \end_inset is finite, the constant random variable \begin_inset Formula $K$ \end_inset is integrable. So we can simply apply the dominated convergence theorem. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection The standard machine \end_layout \begin_layout Standard The standard machine refers to a technique for proving general facts about integrals, by working through the stages of the definition: indicator functions (perhaps starting with indicators of special sets e.g. intervals), simple functions, non-negative functions (using monotone convergenc e), and finally general integrable functions. As an illustration of the technique, we shall prove that integration is linear and that if \begin_inset Formula $X,Y$ \end_inset are independent random variables, then \begin_inset Formula $\E(XY)=\E(X)\E(Y)$ \end_inset . \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset If \begin_inset Formula $f,g\in\cL^{1}(S,\cS,\mu)$ \end_inset then \begin_inset Formula $\int(\alpha f+\beta g)d\mu=\alpha\int fd\mu+\beta\int gd\mu$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset We have observed that this fact holds whenever \begin_inset Formula $f,g$ \end_inset are simple, directly from the definition of integration. \end_layout \begin_layout Standard If \begin_inset Formula $f,g\in\mSp$ \end_inset and \begin_inset Formula $\alpha,\beta\geq0$ \end_inset , then let \begin_inset Formula $f_{n}\uparrow f,g_{n}\uparrow f$ \end_inset be sequences of simple functions. Then \begin_inset Formula $\alpha f_{n}+\beta g_{n}\uparrow\alpha f+\beta g$ \end_inset so by the Monotone Convergence Theorem, \begin_inset Formula \[ \int\alpha f+\beta gd\mu=\lim\int\alpha f_{n}+\beta g_{n}d\mu=\lim\left(\alpha\int f_{n}d\mu+\beta\int g_{n}d\mu\right)=\alpha\int fd\mu+\beta\int gd\mu\] \end_inset \end_layout \begin_layout Standard Finally, for \begin_inset Formula $f,g\in\cL^{1}$ \end_inset : assume that \begin_inset Formula $\alpha,\beta\geq0$ \end_inset , by switching the sign of \begin_inset Formula $\alpha$ \end_inset and \begin_inset Formula $f$ \end_inset or \begin_inset Formula $\beta$ \end_inset and \begin_inset Formula $g$ \end_inset if necessary. \end_layout \begin_layout Standard Let \begin_inset Formula $A=\{x:\alpha f(x)+\beta g(x)\geq0\},B=\{x:\alpha f(x)+\beta g(x)\leq0\}$ \end_inset . \end_layout \begin_layout Standard Now \begin_inset Formula $(\alpha f+\beta g)^{+}=(\alpha f^{+}I_{A}+\beta g^{+}I_{A})-(\alpha f^{-}I_{A}+\beta g^{-}I_{A})$ \end_inset and \begin_inset Formula $(\alpha f+\beta g)^{+},(\alpha f^{-}I_{A}+\beta g^{-}I_{A})\geq0$ \end_inset so using what we have just proved, \begin_inset Formula \[ \int(\alpha f+\beta g)^{+}d\mu+\int(\alpha f^{-}I_{A}+\beta g^{-}I_{A})d\mu=\int(\alpha f^{+}I_{A}+\beta g^{+}I_{A})d\mu\] \end_inset \begin_inset Formula \[ \therefore\int(\alpha f+\beta g)^{+}d\mu=\alpha\int f^{+}I_{A}d\mu+\beta\int g^{+}I_{A}d\mu-\alpha\int f^{-}I_{A}d\mu-\beta\int g^{-}I_{A}d\mu\] \end_inset \end_layout \begin_layout Standard Similarly \begin_inset Formula $-(\alpha f+\beta g)^{-}=(\alpha f^{+}I_{B}+\beta g^{+}I_{B})-(\alpha f^{-}I_{B}+\beta g^{-}I_{B})$ \end_inset , so \begin_inset Formula \[ -\int(\alpha f+\beta g)^{-}d\mu=\alpha\int f^{+}I_{B}d\mu+\beta\int g^{+}I_{B}d\mu-\alpha\int f^{-}I_{B}d\mu-\beta\int g^{-}I_{B}d\mu\] \end_inset \end_layout \begin_layout Standard We also have \begin_inset Formula $f^{+}=f^{+}I_{A}+f^{+}I_{B}$ \end_inset and similarly for \begin_inset Formula $f^{-},g^{+},g^{-}$ \end_inset , so putting these together (and using linearity for nonnegative functions again) gives \begin_inset Formula \[ \int(\alpha f+\beta g)^{+}d\mu-\int(\alpha f+\beta g)^{-}d\mu=\alpha\int f^{+}d\mu+\beta\int g^{+}d\mu-\alpha\int f^{-}d\mu-\beta\int g^{-}d\mu\] \end_inset which is what we want. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset If \begin_inset Formula $X,Y$ \end_inset are independent r.v.s and \begin_inset Formula $g,h:\R\rightarrow\R$ \end_inset measurable functions s.t. \begin_inset Formula $g(X),h(Y)$ \end_inset are integrable, then \begin_inset Formula $\E(g(X)h(Y))=\E(g(X))\E(h(Y))$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Stating the theorem in this form, rather than just \begin_inset Formula $\E(XY)=\E(X)\E(Y)$ \end_inset allows us to apply the standard machine to \begin_inset Formula $g,h$ \end_inset . \end_layout \begin_layout Standard If \begin_inset Formula $g,h$ \end_inset are both indicators of measurable sets, say \begin_inset Formula $g=I_{A},h=I_{B}$ \end_inset , then \begin_inset Formula \[ \begin{aligned}\E(g(X)h(Y)) & =\P(X\in A\mbox{ and }Y\in B)\\ & =\P(X\in A)\P(Y\in B)\mbox{ by definition of independence for }X,Y\\ & =\E(g(X))\E(h(Y))\end{aligned} \] \end_inset \end_layout \begin_layout Standard By linearity, this extends to \begin_inset Formula $g,h$ \end_inset simple functions. \end_layout \begin_layout Standard If \begin_inset Formula $g,h\geq0$ \end_inset , then take sequences \begin_inset Formula $g_{n},h_{n}$ \end_inset of simple functions s.t. \begin_inset Formula $g_{n}\uparrow g,h_{n}\uparrow h$ \end_inset . Then \begin_inset Formula $g_{n}(X)h_{n}(Y)\uparrow g(X)h(Y)$ \end_inset so by the Monotone Convergence Theorem, \begin_inset Formula \[ \E(g(X)h(Y))=\lim_{n\rightarrow\infty}\E(g_{n}(X)h_{n}(Y))=\lim_{n\rightarrow\infty}\E(g_{n}(X))\E(h_{n}(Y))=\E(g(X))\E(h(Y))\] \end_inset \end_layout \begin_layout Standard Finally for general \begin_inset Formula $g,h$ \end_inset , let \begin_inset Formula $f(\omega)=g(X)h(Y)$ \end_inset . \end_layout \begin_layout Standard Then \begin_inset Formula $f^{+}=g^{+}(X)h^{+}(Y)+g^{-}(X)h^{-}(Y)$ \end_inset and \begin_inset Formula $f^{-}=g^{+}(X)h^{-}(Y)+g^{-}(X)h^{+}(Y)$ \end_inset so \begin_inset Formula \[ \begin{aligned}\E(g(X)h(Y)) & = & & \E(f^{+})-\E(f^{-})\\ & = & & \E(g^{+}(X)h^{+}(Y))+\E(g^{-}(X)h^{-}(Y))\\ & = & & \E(g^{+}(X))\E(h^{+}(Y))+\E(g^{-}(X))\E(h^{-}(Y))\\ & & & -\E(g^{+}(X))\E(h^{-}(Y))-\E(g^{-}(X))\E(h^{+}(Y))\\ & = & & (\E(g^{+}(X))-\E(g^{-}(X)))(\E(h^{+}(X))-\E(h^{-}(X)))\\ & = & & \E(g(X))\E(h(Y))\end{aligned} \] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection Differentiation under the integral sign \end_layout \begin_layout Standard This is fairly unimportant but is explicitly required by the schedules. \end_layout \begin_layout Standard Suppose we have a function \begin_inset Formula $f(s,x)$ \end_inset of two variables, and let \begin_inset Formula $F(x)=\int f(s,x)\mu(ds)$ \end_inset . It would be nice if we could obtain \begin_inset Formula $F'(x)$ \end_inset by interchanging the order of differentiation and integration, and getting \begin_inset Formula $F'(x)=\int\frac{\partial}{\partial x}f(s,x)\mu(ds)$ \end_inset . \end_layout \begin_layout Standard Here we give conditions for this to be valid: \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset Let \begin_inset Formula $f:(S,\cS,\mu)\times\R\rightarrow\R$ \end_inset be a function s.t. \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{enumerate}[(i)] \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash item \end_layout \end_inset For each \begin_inset Formula $x\in\R$ \end_inset , \begin_inset Formula $s\mapsto f(s,x)$ \end_inset is a \begin_inset Formula $\cS$ \end_inset -measurable function (so \begin_inset Formula $F(x)$ \end_inset has a chance of existing). \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash item \end_layout \end_inset For each \begin_inset Formula $s\in S$ \end_inset , \begin_inset Formula $x\mapsto f(s,x)$ \end_inset is differentiable. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash item \end_layout \end_inset \begin_inset Formula $s\mapsto f(s,x)\in L^{1}(S,\cS,\mu)$ \end_inset (so \begin_inset Formula $F(0)$ \end_inset exists and is finite). \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash item \end_layout \end_inset There is a function \begin_inset Formula $g\in L^{1}(S,\cS,\mu)$ \end_inset s.t. \begin_inset Formula $\left|\dfrac{f(s,x)-f(s,y)}{x-y}\right|\leq g(s)\forall x\neq y$ \end_inset (so approximations to \begin_inset Formula $\frac{\partial}{\partial x}f(s,x)$ \end_inset are dominated by \begin_inset Formula $g$ \end_inset ). \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash end{enumerate} \end_layout \end_inset \end_layout \begin_layout Standard Then \begin_inset Formula $F(x)=\int_{S}f(s,x)\mu(ds)$ \end_inset is differentiable, and \begin_inset Formula $F'(x)=\int\frac{\partial}{\partial x}f(s,x)\mu(ds)$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Condition (iii) together with (iv) with \begin_inset Formula $y=0$ \end_inset ensures that \begin_inset Formula $F(x)$ \end_inset exists and is finite for all \begin_inset Formula $x\in\R$ \end_inset . \end_layout \begin_layout Standard Now for fixed \begin_inset Formula $x$ \end_inset , \begin_inset Formula \begin{align*} \frac{F(x+h)-F(x)}{h} & =\int_{S}\frac{f(s,x+h)-f(s)}{h}\mu(ds)\\ & \rightarrow\int_{S}\frac{\partial}{\partial x}f(s,x)\mu(ds)\mbox{ as }h\rightarrow0\end{align*} \end_inset by dominated convergence. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Section Modes of convergence \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status collapsed \begin_layout Plain Layout [1] Convergence in measure and convergence almost everywhere. \end_layout \end_inset \end_layout \begin_layout Subsection Convergence almost everywhere \end_layout \begin_layout Standard A sequence of functions \begin_inset Formula $f_{n}:S\rightarrow\R$ \end_inset converges \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$ \backslash mu$-almost everywhere} \end_layout \end_inset to \begin_inset Formula $f$ \end_inset if \begin_inset Formula \[ \mu(\{\omega\colon f_{n}(\omega)\not\rightarrow f(\omega)\})=0\] \end_inset In other words, \begin_inset Formula $f_{n}\rightarrow f$ \end_inset pointwise except on a set of measure zero. In probability, this is called \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{almost sure convergence} \end_layout \end_inset . So far as we are concerned in measure theory, this is just as good as pointwise convergence; in particular the Monotone and Dominated Convergence Theorems continue to hold if we replace pointwise convergence with almost everywhere convergence (just consider \begin_inset Formula $f_{n}I_{A}\rightarrow fI_{A}$ \end_inset where \begin_inset Formula $A=\{\omega\colon f_{n}(\omega)\rightarrow f(\omega)\}$ \end_inset ). \end_layout \begin_layout Subsection \begin_inset CommandInset label LatexCommand label name "subsec:convergence-in-measure" \end_inset Convergence in measure \end_layout \begin_layout Standard A sequence of functions \begin_inset Formula $f_{n}:S\rightarrow\R$ \end_inset converges \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{in $ \backslash mu$-measure} \end_layout \end_inset to \begin_inset Formula $f$ \end_inset if, for any \begin_inset Formula $\epsilon,\delta>0$ \end_inset , there exists \begin_inset Formula $N$ \end_inset s.t. for all \begin_inset Formula $n\geq N$ \end_inset , \begin_inset Formula \[ \mu(\{\omega\colon|f_{n}(\omega)-f(\omega)|>\epsilon\})<\delta\] \end_inset In probability, this is called \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{convergence in probability} \end_layout \end_inset . This is a weak definition of convergence and does not imply that \begin_inset Formula $f_{n}(\omega)\rightarrow f(\omega)$ \end_inset at any point, as the sets \begin_inset Formula $\{\omega:|f_{n}(\omega)-f(\omega)|>\epsilon\}$ \end_inset must become small in measure but they can jump back and forward across the entire space. \end_layout \begin_layout Standard For example, let \begin_inset Formula \[ \begin{array}{cccc} f_{1}=I_{(0,1]},\\ f_{2}=2I_{(0,\frac{1}{2}]}, & & f_{3}=2I_{(\frac{1}{2},1]}\\ f_{4}=4I_{(0,\frac{1}{4}]}, & f_{5}=4I_{(\frac{1}{4},\frac{1}{2}]}, & f_{6}=4I_{(\frac{1}{2},\frac{3}{4}]}, & f_{7}=4I_{(\frac{3}{4},1]}\end{array}\] \end_inset and so on. \end_layout \begin_layout Standard For small \begin_inset Formula $\epsilon$ \end_inset we have \family roman \series medium \shape up \size normal \emph off \bar no \noun off \color none \begin_inset Formula $\mu(\{\omega:|f_{n}(\omega)|>\epsilon\})=2^{-k}$ \end_inset when \begin_inset Formula $2^{k}\leq n<2^{k+1}$ \end_inset , so \begin_inset Formula $f_{n}\rightarrow0$ \end_inset in measure, but \begin_inset Formula $f_{n}(\omega)\not\rightarrow0$ \end_inset for any \begin_inset Formula $\omega\in(0,1]$ \end_inset . Also \begin_inset Formula $\int f_{n}d\mu=1$ \end_inset for all \begin_inset Formula $n$ \end_inset , so \begin_inset Formula $\int f_{n}d\mu\not\rightarrow0$ \end_inset . \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset If \begin_inset Formula $(S,\cS,\mu)$ \end_inset is a finite measure space and \begin_inset Formula $f_{n}\rightarrow f$ \end_inset a.e., then \begin_inset Formula $f_{n}\rightarrow f$ \end_inset in measure. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Fix \begin_inset Formula $\epsilon>0$ \end_inset . Let \begin_inset Formula $A_{n}=\{\omega\colon|f_{m}(\omega)-f(\omega)|\leq\epsilon\mbox{ for all }m\geq n\}$ \end_inset . \end_layout \begin_layout Standard Then \begin_inset Formula $A_{n}\uparrow A$ \end_inset where \begin_inset Formula $A=\{\omega\colon f_{n}(\omega)\rightarrow f(\omega)\}$ \end_inset , and as \begin_inset Formula $f_{n}\rightarrow f$ \end_inset a.e., \begin_inset Formula $\mu(A)=\mu(A_{n})$ \end_inset . \end_layout \begin_layout Standard So \begin_inset Formula $\mu(A_{n})\uparrow\mu(S)$ \end_inset . \end_layout \begin_layout Standard Since \begin_inset Formula $\mu(S)$ \end_inset is finite, \begin_inset Formula $\mu(A_{n}^{c})=\mu(S)-\mu(A_{n})\downarrow0$ \end_inset . \end_layout \begin_layout Standard Now \begin_inset Formula $\{\omega\colon|f_{n}(\omega)-f(\omega)|>\epsilon\}\subset A_{n}^{c}$ \end_inset , so \begin_inset Formula $f_{n}\rightarrow f$ \end_inset in measure. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset If \begin_inset Formula $f_{n}\rightarrow f$ \end_inset in measure, then there is a subsequence \begin_inset Formula $(f_{n_{k}})$ \end_inset s.t. \begin_inset Formula $f_{n_{k}}\rightarrow f$ \end_inset almost everywhere. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Say that \begin_inset Formula $(f_{n})$ \end_inset \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{converges fast in measure} \end_layout \end_inset to \begin_inset Formula $f$ \end_inset if for all \begin_inset Formula $\epsilon>0$ \end_inset , \begin_inset Formula \[ \sum_{n}\mu(\{\omega\colon|f_{n}(\omega)-f(\omega)|>\epsilon\})<\infty\] \end_inset This is clearly a strengthening of the definition of convergence in measure, and it rules out things like the above counter-example because the sets on which it is far from zero get small too slowly. In fact it is a sufficient strengthening that fast convergence in measure implies almost-everywhere convergence (proved below). \end_layout \begin_layout Standard It addition it is not too hard to show that if a sequence converges in measure, then it has a subsequence which converges fast in measure: just choose \begin_inset Formula $n_{k}$ \end_inset such that, for all \begin_inset Formula $n\geq n_{k}$ \end_inset ,. \begin_inset Formula \[ \mu(\{\omega\colon|f_{n}(\omega)-f(\omega)|>2^{-k}\})<2^{-k}\] \end_inset \end_layout \begin_layout Standard For any \begin_inset Formula $\epsilon>0$ \end_inset , choose \begin_inset Formula $K$ \end_inset s.t. \begin_inset Formula $2^{-K}<\epsilon$ \end_inset , and then \begin_inset Formula \[ \sum_{k=K}^{\infty}\mu(\{\omega\colon|f_{n_{k}}(\omega)-f(\omega)|>\epsilon\})<\sum_{k=K}^{\infty}2^{-k}<\infty\] \end_inset \end_layout \begin_layout Standard To complete the proof, suppose that \begin_inset Formula $f_{n}\rightarrow f$ \end_inset fast in measure. We show that \begin_inset Formula $f_{n}\rightarrow f$ \end_inset a.e. \end_layout \begin_layout Standard For any \begin_inset Formula $\epsilon>0$ \end_inset , Borel-Cantelli I gives us \begin_inset Formula \[ \mu(\limsup_{n\rightarrow\infty}\{\omega\colon|f_{n}(\omega)-f(\omega)|>\epsilon\})=0\] \end_inset \end_layout \begin_layout Standard Now \begin_inset Formula \[ f_{n}(\omega)\not\rightarrow f(\omega)\Leftrightarrow\exists\epsilon>0\mbox{ s.t. for infinitely many }k,|f_{n}(\omega)-f(\omega)|>\epsilon\] \end_inset \begin_inset Formula \[ \therefore\{\omega\colon f_{n}(\omega)\not\rightarrow f(\omega)\}=\bigcup_{m}\limsup_{n\rightarrow\infty}\{\omega\colon|f_{n}(\omega)-f(\omega)|>1/m\}\] \end_inset \begin_inset Formula \[ \therefore\mu(\{\omega\colon f_{n}(\omega)\not\rightarrow f(\omega)\})\leq\sum_{m}\mu(\limsup_{n\rightarrow\infty}\{\omega\colon|f_{n}(\omega)-f(\omega)|>1/m\})=0\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection The space \begin_inset Formula $L^{1}$ \end_inset \end_layout \begin_layout Standard Recall that \begin_inset Formula $\cL^{1}(S,\cS,\mu)=\{f\in\mS\colon\int\vert f\vert d\mu<\infty\}$ \end_inset . This is a vector space and \begin_inset Formula $f\mapsto\int fd\mu$ \end_inset is a linear functional on \begin_inset Formula $\cL^{1}$ \end_inset . \end_layout \begin_layout Standard Defining \begin_inset Formula $\Vert f\Vert=\int\vert f\vert d\mu$ \end_inset gives something which almost satisfies the definition of a normed space, except that \begin_inset Formula $\Vert f\Vert=0\not\Rightarrow f=0$ \end_inset , only that \begin_inset Formula $f=0$ \end_inset almost everywhere. Such a space is called a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{seminormed space} \end_layout \end_inset and we can get a normed space by taking the subspace \begin_inset Formula $W=\{f\in\cL^{1}\colon f=0\mbox{ a.e.}\}$ \end_inset and letting \begin_inset Formula $L^{1}(S,\cS,\mu)$ \end_inset be the quotient space \begin_inset Formula $\cL^{1}/W$ \end_inset . You should check that this does give a well-defined normed space. \end_layout \begin_layout Standard Now \begin_inset Formula $\Vert f\Vert=\int\vert f\vert d\mu$ \end_inset is a norm on \begin_inset Formula $L^{1}$ \end_inset , and \begin_inset Formula $f\rightarrow\int fd\mu$ \end_inset is still a well-defined linear functional on \begin_inset Formula $L^{1}$ \end_inset . \end_layout \begin_layout Standard I don't promise to always be careful about distinguishing between \begin_inset Formula $\cL^{1}$ \end_inset and \begin_inset Formula $L^{1}$ \end_inset . \end_layout \begin_layout Subsection Convergence in \begin_inset Formula $L^{1}$ \end_inset \end_layout \begin_layout Standard We can define convergence in \begin_inset Formula $L^{1}$ \end_inset in the usual way for normed spaces: \begin_inset Formula $f_{n}$ \end_inset \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{converges in $L^1$} \end_layout \end_inset to \begin_inset Formula $f$ \end_inset if \begin_inset Formula $\Vert f_{n}-f\Vert\rightarrow0$ \end_inset , or in other words \begin_inset Formula $\int\vert f_{n}-f\vert d\mu\rightarrow0$ \end_inset . In probability, this is called \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{convergence in mean} \end_layout \end_inset . \end_layout \begin_layout Standard Note that this is not the same thing as simple convergence of the means or integrals: \begin_inset Formula $\int f_{n}d\mu\rightarrow\int fd\mu$ \end_inset tells us very little, and the \begin_inset Formula $f_{n}$ \end_inset can vary wildly so long as positive and negative parts of \begin_inset Formula $f_{n}-f$ \end_inset cancel out. Considering \begin_inset Formula $\vert f_{n}-f\vert$ \end_inset prevents this. However, \begin_inset Formula $f_{n}\rightarrow f$ \end_inset in \begin_inset Formula $L^{1}$ \end_inset does imply that \begin_inset Formula $\int f_{n}d\mu\rightarrow\int fd\mu$ \end_inset (an analyst would describe this by saying that \begin_inset Formula $\int\cdot\, d\mu$ \end_inset is a continuous function \begin_inset Formula $L^{1}\rightarrow\R$ \end_inset ). \end_layout \begin_layout Standard How does this relate to our earlier types of convergence? Convergence almost everywhere does not imply \begin_inset Formula $L^{1}$ \end_inset convergence -- this is just the fact from Analysis II that pointwise convergenc e does not imply convergence of integrals. The example in Section \begin_inset CommandInset ref LatexCommand ref reference "subsec:convergence-in-measure" \end_inset shows that convergence in measure does not imply \begin_inset Formula $L^{1}$ \end_inset convergence, and a slight modification of this example (taking each \begin_inset Formula $f_{n}$ \end_inset to be the indicator of an interval, instead of \begin_inset Formula $2^{k}$ \end_inset times an indicator) shows that \begin_inset Formula $L^{1}$ \end_inset convergence does not imply almost everywhere convergence. However we do have: \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset If \begin_inset Formula $f_{n}\rightarrow f$ \end_inset in \begin_inset Formula $L^{1}$ \end_inset , then \begin_inset Formula $f_{n}\rightarrow f$ \end_inset in measure. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Given \begin_inset Formula $\epsilon,\delta>0$ \end_inset , we can use \begin_inset Formula $L^{1}$ \end_inset convergence to find \begin_inset Formula $N$ \end_inset s.t. for all \begin_inset Formula $n>N$ \end_inset , \end_layout \begin_layout Standard \begin_inset Formula \[ \epsilon\delta>\int\vert f_{n}-f\vert d\mu\geq\epsilon\mu(\{\omega\colon\vert f_{n}(\omega)-f(\omega)\vert>\epsilon\})\] \end_inset \end_layout \begin_layout Standard Then \begin_inset Formula \[ n>N\Rightarrow\mu(\{\omega\colon\vert f_{n}(\omega)-f(\omega)\vert>\epsilon\})<\delta\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{lemma}[Scheffé's Lemma] \end_layout \end_inset If \begin_inset Formula $f_{n},f\in L^{1}$ \end_inset , \begin_inset Formula $f_{n}\geq0$ \end_inset , \begin_inset Formula $f_{n}\rightarrow f$ \end_inset a.e., and \begin_inset Formula $\int f_{n}d\mu\rightarrow\int fd\mu$ \end_inset , then \begin_inset Formula $f_{n}\rightarrow f$ \end_inset in \begin_inset Formula $L^{1}$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Consider the positive and negative parts of \begin_inset Formula $f_{n}-f$ \end_inset separately. \end_layout \begin_layout Standard We have \begin_inset Formula $0\leq(f_{n}-f)^{-}=(f-f_{n})^{+}\leq f$ \end_inset a.e. (using \begin_inset Formula $f_{n}\geq0$ \end_inset ) and \begin_inset Formula $(f_{n}-f)^{-}\rightarrow0$ \end_inset a.e. so by DCT, \begin_inset Formula $\int(f_{n}-f)^{-}d\mu\rightarrow0$ \end_inset . \end_layout \begin_layout Standard We are given that \begin_inset Formula $\int(f_{n}-f)^{+}d\mu-\int(f_{n}-f)^{-}d\mu=\int(f-f_{n})d\mu\rightarrow0$ \end_inset , so \begin_inset Formula $\int(f_{n}-f)^{+}d\mu\rightarrow0$ \end_inset . \end_layout \begin_layout Standard Now \begin_inset Formula $\int\vert f-f_{n}\vert d\mu=\int(f_{n}-f)^{+}d\mu+\int(f_{n}-f)^{-}d\mu\rightarrow0$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Section Fubini's theorem \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status collapsed \begin_layout Plain Layout [1] Discussion of product measure and statement of Fubini's theorem. \end_layout \end_inset \end_layout \begin_layout Subsection Product \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash titlesigma \end_layout \end_inset -fields \end_layout \begin_layout Standard Let \begin_inset Formula $(S_{1},\cS_{1},\mu_{1}),(S_{2},\cS_{2},\mu_{2})$ \end_inset be two finite measure spaces. \end_layout \begin_layout Standard We define a product \begin_inset Formula $\sigma$ \end_inset -field \begin_inset Formula $\cS$ \end_inset on \begin_inset Formula $S=S_{1}\times S_{2}$ \end_inset in essentially the same way as we would define the product of topological spaces, with continuous functions replaced by measurable: it is the smallest \begin_inset Formula $\sigma$ \end_inset -field on which the projections \begin_inset Formula $\pi_{1},\pi_{2}$ \end_inset are measurable; in other words it is \begin_inset Formula $\sigma(A\times S_{2}\colon A\in\cS_{1},S_{1}\times B\colon B\in\cS_{2})$ \end_inset . \end_layout \begin_layout Standard This will certainly contain \begin_inset Formula $\cI=\{B_{1}\times B_{2}\colon B_{i}\in\cS_{i}\}$ \end_inset , so \begin_inset Formula $\cS=\sigma(\cI)$ \end_inset (which again is like the definition of product topology as the topology generated by products of open sets). \end_layout \begin_layout Subsection Product measures \end_layout \begin_layout Standard This is the hard bit in this section. \end_layout \begin_layout Standard Let \begin_inset Formula $\mu_{1},\mu_{2}$ \end_inset be finite measures on \begin_inset Formula $(S_{1},\cS_{1}),(S_{2},\cS_{2})$ \end_inset . \end_layout \begin_layout Standard To define the product measure \begin_inset Formula $\mu$ \end_inset on \begin_inset Formula $(S,\cS)$ \end_inset , we start with \begin_inset Formula $\mu(B_{1}\times B_{2})=\mu_{1}(B_{1})\mu_{2}(B_{2})$ \end_inset on \begin_inset Formula $\cI$ \end_inset (think of our two starting spaces as \begin_inset Formula $[0,1]$ \end_inset so \begin_inset Formula $S$ \end_inset is the unit square, and \begin_inset Formula $\mu$ \end_inset is area). Since \begin_inset Formula $\cI$ \end_inset is a \begin_inset Formula $\pi$ \end_inset -system generating \begin_inset Formula $\cS$ \end_inset , there is at most one finite measure extending this. \end_layout \begin_layout Standard If you were an applied mathematician, you might try to write the product measure of a general set by integrating the measures of \begin_inset Quotes eld \end_inset slices \begin_inset Quotes erd \end_inset through it: \begin_inset Formula \[ \mu(A)=\int_{S_{1}}\mu_{2}(\{s_{2}\in S_{2}\colon(s_{1},s_{2})\in A\})\mu_{1}(ds_{1})\] \end_inset or the same thing with 1 and 2 swapped. \end_layout \begin_layout Standard We shall do the same, but we have to do some work with \begin_inset Formula $d$ \end_inset -systems and \begin_inset Formula $\sigma$ \end_inset -fields to show that it is valid. \end_layout \begin_layout Standard Let \begin_inset Formula $J_{1A}(s_{1})=\mu_{2}(\{s_{2}\in S_{2}\colon(s_{1},s_{2})\in A\})$ \end_inset . We need to check that the slices involved are measurable. This is true because all \begin_inset Formula $s_{1}$ \end_inset -slices through sets in \begin_inset Formula $\cI$ \end_inset are certainly \begin_inset Formula $\cS_{2}$ \end_inset -measurable, and the family of subsets of \begin_inset Formula $S$ \end_inset whose \begin_inset Formula $s_{1}$ \end_inset -slices are measurable is closed under complements and countable unions, so is a \begin_inset Formula $\sigma$ \end_inset -field, so contains \begin_inset Formula $\sigma(\cI)$ \end_inset . \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset \begin_inset Formula $J_{1A}$ \end_inset is \begin_inset Formula $\cS_{1}$ \end_inset -measurable for all \begin_inset Formula $A\in\cS$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Let \begin_inset Formula $\cD=\{A\in\cS\colon J_{1A}\mbox{ is }\cS_{1}\mbox{-measurable}\}$ \end_inset . We shall show that \begin_inset Formula $\cD$ \end_inset is a \begin_inset Formula $d$ \end_inset -system. \end_layout \begin_layout Standard Certainly \begin_inset Formula $S\in\cD$ \end_inset . \end_layout \begin_layout Standard If \begin_inset Formula $A,B\in\cD$ \end_inset with \begin_inset Formula $A\supset B$ \end_inset then \begin_inset Formula $J_{1(A\setminus B)}=J_{1A}-J_{1B}$ \end_inset is \begin_inset Formula $\cS_{1}$ \end_inset -measurable, so \begin_inset Formula $A\setminus B\in\cD$ \end_inset . \end_layout \begin_layout Standard If \begin_inset Formula $A_{n}\uparrow A$ \end_inset and \begin_inset Formula $A_{n}\in\cD$ \end_inset then \begin_inset Formula $J_{1A_{n}}\uparrow J_{1A}$ \end_inset so \begin_inset Formula $J_{1A}$ \end_inset is \begin_inset Formula $\cS_{1}$ \end_inset -measurable and \begin_inset Formula $A\in\cD$ \end_inset . \end_layout \begin_layout Standard So \begin_inset Formula $\cD$ \end_inset is a \begin_inset Formula $d$ \end_inset -system; and it certainly contains the \begin_inset Formula $\pi$ \end_inset -system \begin_inset Formula $\cI$ \end_inset , so \begin_inset Formula $\cD\supset d(\cI)=\sigma(\cI)=\cS$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard Hence \begin_inset Formula $\mu(A)=\int_{S_{1}}J_{1A}d\mu_{1}$ \end_inset is well-defined. \end_layout \begin_layout Standard It is finitely additive because \begin_inset Formula $\mu_{2}$ \end_inset is and because integration is linear, and it is countably additive because if \begin_inset Formula $A=\bigcup_{n}A_{n}$ \end_inset ( \begin_inset Formula $A_{n}$ \end_inset pairwise disjoint) then \begin_inset Formula \[ \sum_{j=1}^{n}\mu(A_{j})=\int_{S_{1}}\sum_{j=1}^{n}J_{1A_{j}}d\mu_{1}\uparrow\int_{S_{1}}\sum_{j=1}^{\infty}J_{1A_{j}}d\mu_{1}=\mu(A)\] \end_inset by the Monotone Convergence Theorem. \end_layout \begin_layout Standard So \begin_inset Formula $\mu$ \end_inset is a measure on \begin_inset Formula $(S,\cS)$ \end_inset , it takes the right values on \begin_inset Formula $\cI$ \end_inset , and it is finite because \begin_inset Formula $\mu(S)=\mu_{1}(S_{1})\mu_{2}(S_{2})<\infty$ \end_inset . \end_layout \begin_layout Standard We could alternatively have defined \begin_inset Formula $J_{2A}(s_{2})=\mu_{1}(\{s_{1}\in S_{1}\colon(s_{1},s_{2})\in A\}),\mu(A)=\int_{S_{2}}J_{2A}d\mu_{2}$ \end_inset , and because there is a unique measure taking the right values on \begin_inset Formula $\cI$ \end_inset , this gives the same thing. \end_layout \begin_layout Standard We have set this all up for finite measures, which is necessary in order to use the uniqueness of extension lemma, but it can now be extended to \begin_inset Formula $\sigma$ \end_inset -finite measures \begin_inset Formula $\mu_{1},\mu_{2}$ \end_inset just by working on an increasing sequence of finite subspaces whose union is all of \begin_inset Formula $S_{1}$ \end_inset or \begin_inset Formula $S_{2}$ \end_inset . \end_layout \begin_layout Subsection Integration on product spaces \end_layout \begin_layout Standard The results of Fubini's and Tonelli's theorems tell us that a function \begin_inset Formula $f$ \end_inset on the product space can be integrated by integrating it in one variable first, then the other, provided that \begin_inset Formula $f\geq0$ \end_inset or \begin_inset Formula $\vert f\vert$ \end_inset is \begin_inset Formula $\mu$ \end_inset -integrable. An important consequence is that it doesn't matter which order the integrations are performed in. \end_layout \begin_layout Standard The proofs of these are rather easy, now that we have shown how to define the product measure by one-dimensional integrals. \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Tonelli's Theorem] \end_layout \end_inset If \begin_inset Formula $f\colon S\rightarrow\R$ \end_inset is \begin_inset Formula $\cS$ \end_inset -measurable and \begin_inset Formula $f\geq0$ \end_inset then \begin_inset Formula \[ \int_{S}fd\mu=\int_{S_{1}}\left(\int_{S_{2}}f(s_{1},s_{2})\mu_{2}(ds_{2})\right)\mu_{1}(ds_{1})=\int_{S_{2}}\left(\int_{S_{1}}f(s_{1},s_{2})\mu_{1}(ds_{1})\right)\mu_{2}(ds_{2})\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset We use the standard machine. \end_layout \begin_layout Standard First suppose that \begin_inset Formula $f$ \end_inset is the indicator function of a \begin_inset Formula $\cS$ \end_inset -measurable set \begin_inset Formula $A$ \end_inset . Then this is just the statement \begin_inset Formula $\mu(A)=\int_{S_{1}}J_{1A}(s_{1})\mu_{1}(ds_{1})=\int_{S_{2}}J_{2A}(s_{2})\mu_{2}(ds_{2})$ \end_inset . \end_layout \begin_layout Standard By linearity, the theorem holds for non-negative simple functions on \begin_inset Formula $(S,\cS)$ \end_inset . \end_layout \begin_layout Standard For general \begin_inset Formula $f\in\mSp$ \end_inset , take a sequence \begin_inset Formula $f_{n}$ \end_inset of nonnegative simple functions s.t. \begin_inset Formula $f_{n}\uparrow f$ \end_inset . \end_layout \begin_layout Standard Then by monotone convergence, \begin_inset Formula $\int_{S_{2}}f_{n}(s_{1},s_{2})\mu_{2}(ds_{2})\uparrow\int_{S_{2}}f(s_{1},s_{2})\mu_{2}(ds_{2})$ \end_inset and so \begin_inset Formula \[ \int_{S_{1}}\left(\int_{S_{2}}f_{n}(s_{1},s_{2})\mu_{2}(ds_{2})\right)\mu_{1}(ds_{1})\,\big\uparrow\int_{S_{1}}\left(\int_{S_{2}}f(s_{1},s_{2})\mu_{2}(ds_{2})\right)\mu_{1}(ds_{1})\] \end_inset \end_layout \begin_layout Standard (And similarly with 1 and 2 swapped.) \end_layout \begin_layout Standard Also by monotone convergence, \begin_inset Formula $\int_{S}f_{n}d\mu\uparrow\int_{S}fd\mu$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Fubini's Theorem] \end_layout \end_inset If \begin_inset Formula $f\in\cL^{1}(S,\cS,\mu)$ \end_inset (i.e. \begin_inset Formula $\int_{S}\vert f\vert d\mu<\infty$ \end_inset ) then \begin_inset Formula \[ \int_{S}fd\mu=\int_{S_{1}}\left(\int_{S_{2}}f(s_{1},s_{2})\mu_{2}(ds_{2})\right)\mu_{1}(ds_{1})=\int_{S_{2}}\left(\int_{S_{1}}f(s_{1},s_{2})\mu_{1}(ds_{1})\right)\mu_{2}(ds_{2})\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Just apply Tonelli's theorem to \begin_inset Formula $f^{+}$ \end_inset and \begin_inset Formula $f^{-}$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard Note that the two theorems can be used together: to show that a general \begin_inset Formula $f$ \end_inset satisfies the condition \begin_inset Formula $\int_{S}\vert f\vert d\mu<\infty$ \end_inset in Fubini's theorem, check that \begin_inset Formula $\int_{S_{1}}\left(\int_{S_{2}}\vert f\vert\mu_{2}(ds_{2})\right)\mu_{1}(ds_{1})<\infty$ \end_inset and use Tonelli on \begin_inset Formula $\vert f\vert$ \end_inset . In Cambridge, it is common to combine the two theorems together and call them both Fubini's theorem. \end_layout \begin_layout Subsection Borel-Cantelli from Tonelli \end_layout \begin_layout Standard Fubini's and Tonelli's theorems are more powerful than they may at first appear (as indeed is true of a lot of measure theory), because you can choose to apply it to a wide range of measure spaces. As an illustration, we will derive the first Borel-Cantelli lemma from Tonelli. \end_layout \begin_layout Standard Let \begin_inset Formula $(\Omega,\cF,\P)$ \end_inset be a probability space and \begin_inset Formula $(\N,\wp(\N),\#)$ \end_inset be the natural numbers with counting measure (i.e. \begin_inset Formula $\mu(A)$ \end_inset is simply the number of elements of \begin_inset Formula $A$ \end_inset ; then \begin_inset Formula $\int_{\N}fd\#=\sum_{n=1}^{\infty}f(n)$ \end_inset ). \end_layout \begin_layout Standard Given a sequence \begin_inset Formula $(A_{n})$ \end_inset of events in \begin_inset Formula $\cF$ \end_inset , we define a function \begin_inset Formula $f:\Omega\times\N\rightarrow\R$ \end_inset by \begin_inset Formula $f(\omega,n)=I_{A_{n}}(\omega)$ \end_inset . \end_layout \begin_layout Standard This is nonnegative, so Tonelli's theorem gives us \begin_inset Formula \[ \begin{aligned}\int_{\Omega\times\N}fd(\P,\mu) & =\int_{\N}\int_{\Omega}fd\P d\#=\int_{\N}\P(A_{n})\#(dn)=\sum_{n=1}^{\infty}\P(A_{n})\\ & =\int_{\Omega}\int_{\N}fd\#d\P=\int_{\Omega}\sum_{n=1}^{\infty}I_{A_{n}}d\P=\E(N)\end{aligned} \] \end_inset where \begin_inset Formula $N(\omega)$ \end_inset is the number of values \begin_inset Formula $n\in\N$ \end_inset s.t. \begin_inset Formula $\omega\in A_{n}$ \end_inset . \end_layout \begin_layout Standard The condition of BCI is that \begin_inset Formula $\sum_{n=1}^{\infty}\P(A_{n})<\infty$ \end_inset . Then we get that \begin_inset Formula $\E(N)<\infty$ \end_inset , so \begin_inset Formula $\P(N=\infty)=0$ \end_inset . But the event \begin_inset Formula $\{N=\infty\}$ \end_inset is precisely \begin_inset Formula $\limsup A_{n}$ \end_inset . \end_layout \begin_layout Section Uniform integrability \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status collapsed \begin_layout Plain Layout [1] Uniform integrability. \end_layout \end_inset \end_layout \begin_layout Subsection Definition of uniform integrability \end_layout \begin_layout Standard I shall only deal with probability spaces here because that is the important case; at least some of this extends to \begin_inset Formula $\sigma$ \end_inset -finite measure spaces, but it becomes a bit more complicated. \end_layout \begin_layout Standard A family \begin_inset Formula $\{X_{\lambda}\}$ \end_inset of random variables is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{uniformly integrable} \end_layout \end_inset if, for all \begin_inset Formula $\epsilon>0$ \end_inset , there is some \begin_inset Formula $K$ \end_inset s.t. for all \begin_inset Formula $\lambda$ \end_inset , \begin_inset Formula \[ \int_{\vert X_{\lambda}\vert>K}\vert X_{\lambda}\vert d\P<\epsilon\] \end_inset Intuitively this says that the tails of the distributions of all the random variables become small together. (Uniform integrability is related to compactness in some way which I haven't figured out yet, but note that the next theorem looks a bit like the Ascoli-Arz elà theorem). Uniform integrability is defined for any set of random variables, but we usually apply it to sequences. \end_layout \begin_layout Standard The following theorem gives an equivalent characterisation which is often more useful. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset A family \begin_inset Formula $\{X_{\lambda}\}$ \end_inset of random variables is uniformly integrable iff both the following conditions \end_layout \begin_layout Standard hold: \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{enumerate}[(i)] \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash item \end_layout \end_inset The set \begin_inset Formula $\{X_{\lambda}\}$ \end_inset is bounded in \begin_inset Formula $L^{1}$ \end_inset (i.e. \begin_inset Formula $\exists M$ \end_inset s.t. \begin_inset Formula $\forall\lambda\;\E(\vert X_{\lambda}\vert)0$ \end_inset , there is some \begin_inset Formula $\delta>0$ \end_inset s.t. for all \begin_inset Formula $\lambda$ \end_inset and all events \begin_inset Formula $A$ \end_inset with \begin_inset Formula $\P(A)<\delta$ \end_inset , we have \begin_inset Formula \[ \int_{A}\vert X_{\lambda}\vert d\P<\epsilon\] \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash end{enumerate} \end_layout \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset First suppose that \begin_inset Formula $\{X_{\lambda}\}$ \end_inset is UI. \end_layout \begin_layout Standard For (i), choose \begin_inset Formula $K$ \end_inset s.t. for all \begin_inset Formula $\lambda$ \end_inset , \begin_inset Formula \[ \int_{\vert X_{\lambda}\vert>K}\vert X_{\lambda}\vert d\P<1\] \end_inset Then \begin_inset Formula $\E(\vert X_{\lambda}\vert)=\int_{\vert X_{\lambda}\vert\leq K}\vert X_{\lambda}\vert d\P+\int_{\vert X_{\lambda}\vert>K}\vert X_{\lambda}\vert d\P0$ \end_inset , choose \begin_inset Formula $K$ \end_inset as in the definition of uniform integrability, and \begin_inset Formula $\delta$ \end_inset s.t. \begin_inset Formula $K\delta<\epsilon$ \end_inset . Then for \begin_inset Formula $\P(A)<\delta$ \end_inset , \begin_inset Formula \[ \int_{A}\vert X_{\lambda}\vert d\P=\int_{A\cap\{\vert X_{\lambda}\vert\leq K\}}\vert X_{\lambda}\vert d\P+\int_{A\cap\{\vert X_{\lambda}\vert>K\}}\vert X_{\lambda}\vert d\P0$ \end_inset , choose suitable \begin_inset Formula $\delta$ \end_inset as in (ii), and \begin_inset Formula $M$ \end_inset as in (i). Set \begin_inset Formula $K=M/\delta$ \end_inset . \end_layout \begin_layout Standard Now \begin_inset Formula $M>\E(\vert X_{\lambda}\vert)\geq K\P(\vert X_{\lambda}\vert>K)$ \end_inset so \begin_inset Formula $\P(\vert X_{\lambda}\vert>K)<\delta$ \end_inset for all \begin_inset Formula $\lambda$ \end_inset . So putting \begin_inset Formula $A=\{\vert X_{\lambda}\vert>K\}$ \end_inset in (ii) gives exactly what is required. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection Sufficient conditions for uniform integrability \end_layout \begin_layout Standard A single integrable random variable \begin_inset Formula $X$ \end_inset is UI since the random variables \begin_inset Formula $Z_{n}=\vert X\vert I_{\{\vert X|>n\}}$ \end_inset are dominated by \begin_inset Formula $X$ \end_inset and converge a.s. to \begin_inset Formula $0$ \end_inset , so by DCT, \begin_inset Formula \[ \int_{\vert X\vert>n}\vert X\vert d\P=\E(Z_{n})\rightarrow0\] \end_inset \end_layout \begin_layout Standard Any finite set \begin_inset Formula $\{X_{1},\ldots,X_{n}\}$ \end_inset of integrable r.v.s is UI since for each \begin_inset Formula $\epsilon>0$ \end_inset , we can find \begin_inset Formula $K_{1},\ldots,K_{n}$ \end_inset for which \begin_inset Formula $\int_{\vert X_{n}\vert>K_{n}}\vert X_{n}\vert d\P<\epsilon$ \end_inset , then take \begin_inset Formula $K=\max\{K_{1},\ldots,K_{n}\}$ \end_inset . \end_layout \begin_layout Standard If there is some \begin_inset Formula $X\in L^{1}$ \end_inset s.t. \begin_inset Formula $\vert X_{\lambda}\vert\leq\vert X\vert$ \end_inset for all \begin_inset Formula $\lambda$ \end_inset , then \begin_inset Formula $\{X_{\lambda}\}$ \end_inset is UI, since given \begin_inset Formula $\epsilon>0$ \end_inset , we may choose \begin_inset Formula $K$ \end_inset s.t. \begin_inset Formula $\int_{\vert X\vert>K}\vert X\vert d\P<\epsilon$ \end_inset . Then for each \begin_inset Formula $\lambda$ \end_inset , \begin_inset Formula $\{\omega\colon\vert X_{\lambda}\vert>K\}\subset\{\omega\colon\vert X\vert>K\}$ \end_inset , so \begin_inset Formula \[ \int_{\vert X_{\lambda}|>K}\vert X_{\lambda}\vert d\P\leq\int_{\vert X|>K}\vert X_{\lambda}\vert d\P\leq\int_{\vert X|>K}\vert X\vert d\P<\epsilon\] \end_inset \end_layout \begin_layout Standard If \begin_inset Formula $\{X_{\lambda}\}$ \end_inset and \begin_inset Formula $\{Y_{\lambda'}\}$ \end_inset are UI, then \begin_inset Formula $\{X_{\lambda}+Y_{\lambda'}\}$ \end_inset is UI, by the alternative characterisation. \end_layout \begin_layout Subsection \begin_inset Formula $L^{1}$ \end_inset convergence and uniform integrability \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset A sequence \begin_inset Formula $(X_{n})$ \end_inset of integrable random variables converges to \begin_inset Formula $X$ \end_inset in \begin_inset Formula $L^{1}$ \end_inset iff \begin_inset Formula $X_{n}\rightarrow X$ \end_inset in probability and \begin_inset Formula $\{X_{n}\}$ \end_inset is UI. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset First suppose that \begin_inset Formula $X_{n}\rightarrow X$ \end_inset in \begin_inset Formula $L^{1}$ \end_inset . \end_layout \begin_layout Standard Then \begin_inset Formula $X\in L^{1}$ \end_inset so \begin_inset Formula $\{X_{n}-X\}$ \end_inset UI implies that \begin_inset Formula $\{X_{n}\}$ \end_inset is UI, so we may assume wlog that \begin_inset Formula $X=0$ \end_inset . \end_layout \begin_layout Standard We already know that \begin_inset Formula $X_{n}\rightarrow0$ \end_inset in probability. \end_layout \begin_layout Standard Given \begin_inset Formula $\epsilon>0$ \end_inset , we can choose \begin_inset Formula $N$ \end_inset s.t. \begin_inset Formula $n>N\Rightarrow\E(\vert X_{n}\vert)<\epsilon$ \end_inset , so certainly \begin_inset Formula $\int_{\vert X_{n}\vert>K}\vert X_{n}\vert d\P<\epsilon$ \end_inset for any \begin_inset Formula $K$ \end_inset . \end_layout \begin_layout Standard So we just have to choose \begin_inset Formula $K$ \end_inset s.t. \begin_inset Formula $\int_{\vert X_{n}\vert>K}\vert X_{n}\vert d\P<\epsilon$ \end_inset for \begin_inset Formula $1\leq n\leq N$ \end_inset , which we can do since the finite set \begin_inset Formula $\{X_{1},\ldots,X_{N}\}$ \end_inset is UI. \begin_inset VSpace defskip \end_inset \end_layout \begin_layout Standard Now suppose that \begin_inset Formula $X_{n}\rightarrow X$ \end_inset in probability and \begin_inset Formula $\{X_{n}\}$ \end_inset is UI. \end_layout \begin_layout Standard First we show that \begin_inset Formula $X\in L^{1}$ \end_inset (which seems to take several lines, but they are quite easy). \end_layout \begin_layout Standard Take a subsequence \begin_inset Formula $(X_{n_{k}})$ \end_inset s.t. \begin_inset Formula $X_{n_{k}}\rightarrow X$ \end_inset almost surely. \end_layout \begin_layout Standard Let \begin_inset Formula $Y_{n_{k}}=\min\{X_{n_{k}}^{+},X^{+}\}$ \end_inset so \begin_inset Formula $Y_{n_{k}}\uparrow X^{+}$ \end_inset a.s. and by MCT \begin_inset Formula $\E(Y_{n_{k}})\uparrow\E(X^{+})$ \end_inset . \end_layout \begin_layout Standard \begin_inset Formula $\E(Y_{n_{k}})\leq\E(\vert X_{n_{k}}\vert)\leq M$ \end_inset for some \begin_inset Formula $M$ \end_inset since \begin_inset Formula $\{X_{n_{k}}\}$ \end_inset is UI, so \begin_inset Formula $\E(X^{+})\leq M$ \end_inset . \end_layout \begin_layout Standard In particular, \begin_inset Formula $\E(X^{+})<\infty$ \end_inset and likewise \begin_inset Formula $\E(X^{-})<\infty$ \end_inset so \begin_inset Formula $X\in L^{1}$ \end_inset . \begin_inset VSpace defskip \end_inset \end_layout \begin_layout Standard Hence \begin_inset Formula $\{X_{n}\}$ \end_inset UI implies that \begin_inset Formula $\{X_{n}-X\}$ \end_inset is UI, so we may assume wlog that \begin_inset Formula $X=0$ \end_inset . \end_layout \begin_layout Standard Given \begin_inset Formula $\epsilon>0$ \end_inset , choose \begin_inset Formula $\delta$ \end_inset s.t. \begin_inset Formula $\P(A)<\delta\Rightarrow\int_{A}\vert X_{n}\vert d\P<\epsilon$ \end_inset . \end_layout \begin_layout Standard Then, by convergence in probability, choose \begin_inset Formula $N$ \end_inset s.t. \begin_inset Formula $n>N\Rightarrow\P(\vert X_{n}\vert>\epsilon)<\delta$ \end_inset . \end_layout \begin_layout Standard Now for \begin_inset Formula $n>N$ \end_inset , we have \begin_inset Formula \[ \E(\vert X_{n}\vert)=\int_{\vert X_{n}\vert\leq\epsilon}\vert X_{n}\vert d\P+\int_{\vert X_{n}\vert>\epsilon}\vert X_{n}\vert d\P<\epsilon\P(\vert X_{n}\vert\leq\epsilon)+\epsilon<2\epsilon\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard Because a sequence dominated by an integrable r.v. is UI, this theorem strengthens the Dominated Convergence Theorem to require only convergence in probability rather than almost sure convergence. \end_layout \begin_layout Section Inequalities and Banach spaces \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status collapsed \begin_layout Plain Layout [3] Chebyshev's inequality, tail estimates. Jensen's inequality. Completeness of \begin_inset Formula $L^{p}$ \end_inset for \begin_inset Formula $1\leq p\leq\infty$ \end_inset . The Hölder and Minkowski inequalities. \end_layout \end_inset \end_layout \begin_layout Standard \end_layout \begin_layout Subsection Chebyshev's inequality \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Chebyshev's Inequality] \end_layout \end_inset If \begin_inset Formula $f$ \end_inset is a measurable function \begin_inset Formula $S\rightarrow\R$ \end_inset then \begin_inset Formula \[ \mu(\{\omega\colon\vert f(\omega)\vert\geq t\})\leq\frac{1}{t^{2}}\int_{S}f^{2}d\mu\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset \begin_inset Formula \[ \int_{S}f^{2}d\mu\geq\int_{\vert f\vert\geq t}f^{2}d\mu\geq t^{2}\mu(\{\omega\colon\vert f(\omega)\vert\geq t\})\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard In the probability case where \begin_inset Formula $X$ \end_inset is a random variable, this becomes \begin_inset Formula $\P(|X-\E X|\geq t)\leq\frac{1}{t^{2}}\var(X)$ \end_inset , where \begin_inset Formula $\var(X)$ \end_inset has the usual definition of \begin_inset Formula $\E((X-\E X)^{2})$ \end_inset . \end_layout \begin_layout Subsection Jensen's inequality \end_layout \begin_layout Standard Jensen's inequality is an important result concerning the averaging of convex functions. A \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{convex function} \end_layout \end_inset is a function \begin_inset Formula $\varphi:\R\rightarrow\R$ \end_inset s.t. whenever \begin_inset Formula $0\leq a,b\leq1$ \end_inset and \begin_inset Formula $a+b=1$ \end_inset , \begin_inset Formula $\varphi(ax+by)\leq a\varphi(x)+b\varphi(y)$ \end_inset . It is often easiest to test this using the fact that twice-differentiable function is convex iff \begin_inset Formula $\varphi''(x)\geq0$ \end_inset for all \begin_inset Formula $x$ \end_inset , or just by drawing a graph. \end_layout \begin_layout Standard Note that this result holds only in probability spaces; it cannot even be generalised (like most of the results I have given for probability spaces) to finite measure spaces. \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Jensen's Inequality] \end_layout \end_inset If \begin_inset Formula $X$ \end_inset is an integrable random variable and \begin_inset Formula $\varphi$ \end_inset a measurable convex function \begin_inset Formula $\R\rightarrow\R$ \end_inset , then \begin_inset Formula \[ \varphi(\E(X))\leq\E(\varphi(X))\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Properly we need a condition that \begin_inset Formula $\E(\varphi(X))$ \end_inset exists. \end_layout \begin_layout Standard The key step in the proof is that through every point \begin_inset Formula $(x,\varphi(x))$ \end_inset in the graph of \begin_inset Formula $\varphi$ \end_inset , there is a line which is always below or on the graph. If \begin_inset Formula $\varphi$ \end_inset is differentiable at \begin_inset Formula $x$ \end_inset , this is just the tangent. If not, then we have to work a bit harder. \end_layout \begin_layout Standard Given \begin_inset Formula $z\in\R$ \end_inset , define \begin_inset Formula $\psi_{z}(x)={\displaystyle \frac{\varphi(x)-\varphi(z)}{x-z}}$ \end_inset for \begin_inset Formula $x\neq z$ \end_inset . \end_layout \begin_layout Standard From the definition of convexity, \begin_inset Formula $\psi_{z}$ \end_inset is increasing, and in particular, if \begin_inset Formula $x_{1}z$ \end_inset , so \begin_inset Formula $\alpha_{z}x+\beta_{z}\leq\varphi(x)$ \end_inset for all \begin_inset Formula $x$ \end_inset . \end_layout \begin_layout Standard Now apply this result with \begin_inset Formula $z=\E(X)$ \end_inset . \end_layout \begin_layout Standard By the choice of \begin_inset Formula $\beta_{z}$ \end_inset , we have \begin_inset Formula $\varphi(\E(X))=\varphi(z)=\alpha_{z}z+\beta_{z}=\alpha_{z}\E(X)+\beta_{z}$ \end_inset . \end_layout \begin_layout Standard But also \begin_inset Formula $\alpha_{z}x+\beta_{z}\leq\varphi(x)$ \end_inset for all x, so taking expectations, \begin_inset Formula \[ \varphi(\E(X))=\alpha_{z}\E(X)+\beta_{z}\leq\E(\varphi(X))\qedhere\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection Hölder's inequality \end_layout \begin_layout Standard There are many proofs of this important inequality. Analysts like to prove it again and again, once for finite sequences, once for infinite sequences ( \begin_inset Formula $l^{p}$ \end_inset spaces), once for integrable functions ( \begin_inset Formula $L^{p}$ \end_inset spaces). The measure theoretic version gives us all of these at once, by taking a suitable underlying measure space (a finite set with counting measure, \begin_inset Formula $\N$ \end_inset with counting measure, Lebesgue measure). This proof uses Jensen's inequality, but on a specially defined probability measure, so the theorem still holds for general \begin_inset Formula $\sigma$ \end_inset -finite measure spaces. \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Hölder's Inequality] \end_layout \end_inset If \begin_inset Formula $f,g\in\mS$ \end_inset , \begin_inset Formula $10$ \end_inset , choose \begin_inset Formula $N$ \end_inset s.t. \begin_inset Formula $m,n>N\Rightarrow\Vert f_{m}-f_{n}\Vert_{p}<\delta^{1/p}\epsilon$ \end_inset . \end_layout \begin_layout Standard Now \begin_inset Formula $\Vert f_{m}-f_{n}\Vert_{p}^{p}\geq\epsilon^{p}\mu(\{\omega\colon\vert f_{m}(\omega)-f_{n}(\omega)\vert>\epsilon\})$ \end_inset so \begin_inset Formula $m,n>N\Rightarrow\mu(\{\omega\colon\vert f_{m}(\omega)-f_{n}(\omega)\vert>\epsilon\})<\delta$ \end_inset . \end_layout \begin_layout Standard \begin_inset VSpace defskip \end_inset \end_layout \begin_layout Standard We earlier had a theorem that if a sequence converges in measure, then a subsequence converges almost everywhere (compare the next part of this proof to the proof of that theorem). In fact, it is sufficient for the sequence to be Cauchy in measure. \end_layout \begin_layout Standard Choose a subsequence \begin_inset Formula $(f_{n_{k}})$ \end_inset s.t. \begin_inset Formula $m,n\geq n_{k}\Rightarrow\mu(\{\omega\colon\vert f_{m}(\omega)-f_{n}(\omega)\vert>2^{-k}\})<2^{-k}$ \end_inset , so that \begin_inset Formula \[ \sum_{k}\mu(\{\omega\colon\vert f_{n_{k}}(\omega)-f_{n_{k+1}}(\omega)\vert>2^{-k}\})<\infty\] \end_inset and so by Borel-Cantelli I, \begin_inset Formula $\mu(A)=0$ \end_inset where \begin_inset Formula \[ A=\limsup_{k\rightarrow\infty}\{\omega\colon\vert f_{n_{k}}(\omega)-f_{n_{k+1}}(\omega)\vert>2^{-k}\}\] \end_inset \end_layout \begin_layout Standard Unpacking the definition of \begin_inset Formula $A$ \end_inset says that if \begin_inset Formula $\omega\in A^{c}$ \end_inset , then there is some \begin_inset Formula $k$ \end_inset (depending on \begin_inset Formula $\omega$ \end_inset ) s.t. \begin_inset Formula $j\geq k\Rightarrow\vert f_{n_{j}}(\omega)-f_{n_{j+1}}(\omega)\vert\leq2^{-j}$ \end_inset , and so \begin_inset Formula $i,j\geq k\Rightarrow\vert f_{n_{i}}(\omega)-f_{n_{j}}(\omega)\vert\leq\sum_{j\geq k}2^{-j}=2^{-k+1}$ \end_inset . \end_layout \begin_layout Standard Hence for \begin_inset Formula $\omega\in A^{c}$ \end_inset , the sequence \begin_inset Formula $(f_{n_{k}}(\omega))$ \end_inset is Cauchy (in \begin_inset Formula $\R$ \end_inset ), so has a limit. \end_layout \begin_layout Standard \begin_inset VSpace defskip \end_inset \end_layout \begin_layout Standard Define \begin_inset Formula $f(\omega)=\limsup f_{n_{k}}(\omega)$ \end_inset . This is measurable, and if \begin_inset Formula $\omega\in A^{c}$ \end_inset then \begin_inset Formula $f(\omega)=\lim f_{n_{k}}(\omega)$ \end_inset so \begin_inset Formula $f_{n_{k}}\rightarrow f$ \end_inset almost everywhere. \end_layout \begin_layout Standard We need to check that \begin_inset Formula $f\in L^{p}$ \end_inset . This follows from Fatou since \begin_inset Formula $\vert f\vert^{p}=\liminf_{k}\vert f_{n_{k}}\vert^{p}$ \end_inset a.e., so \begin_inset Formula $\int\vert f\vert^{p}d\mu\leq\liminf_{k}\int\vert f_{n_{k}}\vert^{p}d\mu$ \end_inset and \begin_inset Formula $\int\vert f_{n_{k}}\vert^{p}d\mu\leq(\Vert f_{n_{k}}-f_{n_{1}}\Vert_{p}+\Vert f_{n_{1}}\Vert_{p})^{p}\leq(1+\Vert f_{n_{1}}\Vert_{p})^{p}$ \end_inset for all \begin_inset Formula $k$ \end_inset . \end_layout \begin_layout Standard \begin_inset VSpace defskip \end_inset \end_layout \begin_layout Standard Finally we show that \begin_inset Formula $\Vert f_{n}-f\Vert_{p}\rightarrow0$ \end_inset . \end_layout \begin_layout Standard Given \begin_inset Formula $\epsilon>0$ \end_inset , choose \begin_inset Formula $k$ \end_inset s.t. \begin_inset Formula $m,n\geq n_{k}\Rightarrow\Vert f_{n}-f_{m}\Vert_{p}<\epsilon$ \end_inset . \end_layout \begin_layout Standard For \begin_inset Formula $m\geq n_{k}$ \end_inset , \begin_inset Formula $\vert f_{m}-f\vert^{p}=\lim_{k}\vert f_{m}-f_{n_{k}}\vert^{p}=\liminf_{k}\vert f_{m}-f_{n_{k}}\vert^{p}$ \end_inset a.e. so by Fatou, \begin_inset Formula \[ \int\vert f_{m}-f\vert^{p}d\mu\leq\liminf_{k\rightarrow\infty}\int\vert f_{m}-f_{n_{k}}\vert^{p}d\mu\leq\epsilon^{p}\] \end_inset \end_layout \begin_layout Standard So \begin_inset Formula $m\geq n_{k}\Rightarrow\Vert f_{m}-f\Vert_{p}\leq\epsilon$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection The space \begin_inset ERT status open \begin_layout Plain Layout \backslash texorpdfstring{$L^ \backslash infty$}{L-infinity} \end_layout \end_inset \end_layout \begin_layout Standard There is one more space to consider: the space \begin_inset Formula $L^{\infty}$ \end_inset of essentially bounded functions. \end_layout \begin_layout Standard A function \begin_inset Formula $f\in\mS$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{essentially bounded} \end_layout \end_inset if there is some \begin_inset Formula $K$ \end_inset s.t. \begin_inset Formula $\mu(\{\omega\colon\vert f\vert>K\})=0$ \end_inset . \end_layout \begin_layout Standard We define a norm (the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{essential supremum} \end_layout \end_inset ) by \begin_inset Formula $\Vert f\Vert_{\infty}={\displaystyle \inf_{N\in\cS,\mu(N)=0}\sup_{\omega\not\in N}\vert f(\omega)\vert}$ \end_inset . \end_layout \begin_layout Standard Let \begin_inset Formula $\cL^{\infty}$ \end_inset be the set of essentially bounded functions \begin_inset Formula $(S,\cS)\rightarrow\R$ \end_inset , and identify a.e.-equal functions to form the quotient \begin_inset Formula $L^{\infty}$ \end_inset as before. \begin_inset Formula $\Vert\cdot\Vert_{\infty}$ \end_inset is a norm on \begin_inset Formula $L^{\infty}$ \end_inset ; the triangle inequality is a piece of straightforward analysis. \end_layout \begin_layout Standard A sequence \begin_inset Formula $f_{n}\rightarrow f$ \end_inset in \begin_inset Formula $L^{\infty}$ \end_inset iff there is a null set \begin_inset Formula $N$ \end_inset s.t. \begin_inset Formula $f_{n}\rightarrow f$ \end_inset uniformly on \begin_inset Formula $S\backslash N$ \end_inset (this uses the fact that a countable union of null sets is null). The relationship between \begin_inset Formula $L^{\infty}$ \end_inset convergence and uniform convergence is the same as the relationship between almost everywhere convergence and pointwise convergence. \end_layout \begin_layout Standard We can use this to show that \begin_inset Formula $L^{\infty}$ \end_inset is complete, by showing that a Cauchy sequence in \begin_inset Formula $L^{\infty}$ \end_inset is uniformly Cauchy except on a null set, then using the completeness of the uniform norm. \end_layout \begin_layout Section \begin_inset Formula $L^{2}$ \end_inset and conditional expectation \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status collapsed \begin_layout Plain Layout [1] \begin_inset Formula $L^{2}$ \end_inset as a Hilbert space. Orthogonal projection, relation with conditional probability. Variance and covariance. \end_layout \end_inset \end_layout \begin_layout Subsection Conditional probability \end_layout \begin_layout Standard In elementary probability, for an event \begin_inset Formula $B$ \end_inset with \begin_inset Formula $\P(B)>0$ \end_inset , and for any event \begin_inset Formula $A$ \end_inset , we define the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{conditional probability of $A$ given $B$} \end_layout \end_inset as \begin_inset Formula \[ \P(A|B)=\frac{\P(A\cap B)}{\P(B)}\] \end_inset \end_layout \begin_layout Standard Suppose we perform an experiment with a finite set of possible outcomes, with events \begin_inset Formula $B_{1},\ldots,B_{n}$ \end_inset corresponding to each outcome (these events are disjoint and cover \begin_inset Formula $\Omega$ \end_inset , and must all have non-zero probability). After gaining this information, we want to know the probability of some other event \begin_inset Formula $A$ \end_inset ; this will be \begin_inset Formula $\P(A|B_{j})$ \end_inset for the appropriate \begin_inset Formula $j$ \end_inset . Hence our guess as to the probability of \begin_inset Formula $A$ \end_inset after the experiment is itself a random variable: \begin_inset Formula \[ g(\omega)=\P(A\cap B_{j})/\P(B_{j})\mbox{ when }\omega\in B_{j}\] \end_inset \end_layout \begin_layout Standard Now consider a more sophisticated experiment, where the set of possible outcomes may be infinite and may contain zero-probability events. Let \begin_inset Formula $\cG$ \end_inset be the set of all events for which the experiment tells us whether they occur or not; these form a sub- \begin_inset Formula $\sigma$ \end_inset -field of \begin_inset Formula $\cF.$ \end_inset (In the above finite case, \begin_inset Formula $\cG=\sigma(B_{1},\ldots,B_{n})$ \end_inset is the set of all possible unions of the \begin_inset Formula $B_{j}$ \end_inset . If the experiment measures some random variable \begin_inset Formula $X$ \end_inset , then \begin_inset Formula $\cG=\sigma(X)$ \end_inset .) \end_layout \begin_layout Standard Again, we want a random variable \begin_inset Formula $g$ \end_inset which gives the probability of \begin_inset Formula $A$ \end_inset after knowing the results of this experiment. \end_layout \begin_layout Standard The value of \begin_inset Formula $g$ \end_inset must of course depend only on events in \begin_inset Formula $\cG$ \end_inset ; for example, in the case \begin_inset Formula $\cG=\sigma(B_{1},\ldots,B_{n})$ \end_inset , \begin_inset Formula $g$ \end_inset is constant on each \begin_inset Formula $B_{j}$ \end_inset . This condition is made precise by saying that \begin_inset Formula $g$ \end_inset must be \begin_inset Formula $\cG$ \end_inset -measurable (note that since \begin_inset Formula $\cG\subset\cF$ \end_inset , this is a stronger condition than being \begin_inset Formula $\cF$ \end_inset -measurable). \end_layout \begin_layout Standard Secondly, consider \begin_inset Formula $\P(A\cap B)$ \end_inset for any \begin_inset Formula $B\in\cG$ \end_inset . Once we have performed the experiment, we know either that \begin_inset Formula $B$ \end_inset did not occur, so \begin_inset Formula $A\cap B$ \end_inset certainly does not occur, or \begin_inset Formula $B$ \end_inset did occur, in which case \begin_inset Formula $A\cap B$ \end_inset occurs with conditional probability \begin_inset Formula $g(\omega)$ \end_inset . If we add (integrate) this for all \begin_inset Formula $\omega$ \end_inset , then we should get \begin_inset Formula $\P(A\cap B)$ \end_inset : \begin_inset Formula $\int_{B}gd\P=\P(A\cap B)$ \end_inset . \end_layout \begin_layout Standard \begin_inset VSpace defskip \end_inset \end_layout \begin_layout Standard So we define a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{(version of the) conditional probability of $A$ given $ \backslash cG$} \end_layout \end_inset to be a random variable \begin_inset Formula $\P(A|\cG)$ \end_inset s.t. \end_layout \begin_layout Enumerate \begin_inset Formula $\P(A|\cG)$ \end_inset is \begin_inset Formula $\cG$ \end_inset -measurable \end_layout \begin_layout Enumerate For any \begin_inset Formula $B\in\cG$ \end_inset , \begin_inset Formula $\int_{B}\P(A|\cG)d\P=\P(A\cap B)$ \end_inset \end_layout \begin_layout Standard It is not at all obvious that such a random variable exists in general, or that it is unique (hence the need to talk about versions); we shall worry about this later. \end_layout \begin_layout Subsection Conditional expectation \end_layout \begin_layout Standard If we fix \begin_inset Formula $\cG$ \end_inset and \begin_inset Formula $\omega$ \end_inset , and allow \begin_inset Formula $A$ \end_inset to vary, then \begin_inset Formula $\P(A|\cG)(\omega)$ \end_inset gives a probability measure on \begin_inset Formula $(\Omega,\cF)$ \end_inset . We can integrate with respect to this measure to get the conditional expectatio n of a random variable \begin_inset Formula $X$ \end_inset : \begin_inset Formula \[ \E(X|\cG)(\omega)=\int_{\Omega}Xd(\P(-|\cG)(\omega))\] \end_inset \end_layout \begin_layout Standard Now fixing \begin_inset Formula $X$ \end_inset and considering this as a function of \begin_inset Formula $\omega$ \end_inset , we get a new random variable \begin_inset Formula $\E(X|\cG)$ \end_inset . This will have the properties (these follow from the corresponding properties for conditional probability, by the standard machine): \end_layout \begin_layout Enumerate \begin_inset Formula $\E(X|\cG)$ \end_inset is \begin_inset Formula $\cG$ \end_inset -measurable \end_layout \begin_layout Enumerate For any \begin_inset Formula $B\in\cG$ \end_inset , \begin_inset Formula $\int_{B}\E(X|\cG)d\P=\int_{B}Xd\P$ \end_inset \end_layout \begin_layout Standard We define a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{(version of the) conditional expectation of $X$ given $ \backslash cG$} \end_layout \end_inset to be a random variable with these two properties. \end_layout \begin_layout Standard Again it is not clear that such a random variable exists. And it is only unique up to almost sure equality: if \begin_inset Formula $g$ \end_inset is a version of \begin_inset Formula $\E(X|\cG)$ \end_inset and \begin_inset Formula $f$ \end_inset a \begin_inset Formula $\cG$ \end_inset -measurable r\SpecialChar \@. v\SpecialChar \@. which is zero almost surely, then \begin_inset Formula $g+f$ \end_inset is still a version of \begin_inset Formula $\E(X|\cG)$ \end_inset . \end_layout \begin_layout Standard However, if \begin_inset Formula $g_{1},g_{2}$ \end_inset are two versions of \begin_inset Formula $\E(X|\cG)$ \end_inset , then let \begin_inset Formula $B=\{\omega:g_{1}(\omega)\geq g_{2}(\omega)+\epsilon\}$ \end_inset . \begin_inset Formula $B\in\cG$ \end_inset since \begin_inset Formula $g_{1},g_{2}$ \end_inset are \begin_inset Formula $\cG$ \end_inset -measurable so \begin_inset Formula $\int_{B}g_{1}d\P=\int_{B}Xd\P=\int_{B}g_{2}d\P$ \end_inset but \begin_inset Formula $\int_{B}(g_{1}-g_{2})d\P\geq\epsilon\P(B)$ \end_inset so \begin_inset Formula $\P(B)=0$ \end_inset for any \begin_inset Formula $\epsilon$ \end_inset . Hence \begin_inset Formula $g_{1}=g_{2}$ \end_inset almost surely. \end_layout \begin_layout Subsection \begin_inset Formula $L^{2}$ \end_inset as a Hilbert space \end_layout \begin_layout Standard We shall prove the existence of conditional expectation for square-integrable random variables, by exploiting the fact that \begin_inset Formula $L^{2}(\Omega,\cF,\P)$ \end_inset is a Hilbert space i.e. its norm is given by an inner product. (Linear Analysis yields more light on why this is the case, specifically that \begin_inset Formula $L^{2}$ \end_inset is self-dual). The inner product is \begin_inset Formula \[ \left\langle f,g\right\rangle =\int fgd\mu\] \end_inset which is finite for \begin_inset Formula $f,g\in L^{2}$ \end_inset by Hölder (the case \begin_inset Formula $p=q=2$ \end_inset , often called the Schwarz or Cauchy-Schwarz inequality). \end_layout \begin_layout Standard In the case of probability, the inner product has an interpretation in terms of covariance: the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{covariance} \end_layout \end_inset of \begin_inset Formula $X,Y\in L^{2}$ \end_inset is \begin_inset Formula $\cov(X,Y)=\left\langle X-\E X,Y-\E Y\right\rangle =\left\langle X,Y\right\rangle -\E(X)\E(Y)$ \end_inset . \end_layout \begin_layout Standard The \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{variance} \end_layout \end_inset of a random variable is \begin_inset Formula $\var(X)=\cov(X,X)=\Vert X-\E X\Vert_{2}^{2}$ \end_inset , which is finite iff \begin_inset Formula $X\in L^{2}$ \end_inset . \end_layout \begin_layout Standard The benefit of Hilbert space is that we can apply geometrical ideas. In particular there is an idea of orthogonality: \begin_inset Formula $f,g\in L^{2}$ \end_inset are \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{orthogonal} \end_layout \end_inset if \begin_inset Formula $\left\langle f,g\right\rangle =0$ \end_inset . We also have the parallelogram law: \begin_inset Formula \[ \Vert f+g\Vert^{2}+\Vert f-g\Vert^{2}=2(\Vert f\Vert^{2}+\Vert g\Vert^{2})\] \end_inset \end_layout \begin_layout Subsection Orthogonal projection \end_layout \begin_layout Standard An important geometrical property of Hilbert spaces is the following, proved in Linear Analysis: \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset If \begin_inset Formula $G$ \end_inset is a closed subspace of a Hilbert space \begin_inset Formula $H$ \end_inset , then there is a unique continuous linear map \begin_inset Formula $\pi:H\rightarrow G$ \end_inset s.t. \begin_inset Formula $\left\langle f-\pi f,g\right\rangle =0$ \end_inset for all \begin_inset Formula $f\in H,g\in G$ \end_inset . This map has the properties: \end_layout \begin_layout Enumerate \begin_inset Formula $\Vert f-\pi f\Vert\leq\Vert f-g\Vert$ \end_inset for all \begin_inset Formula $g\in G$ \end_inset \end_layout \begin_layout Enumerate \begin_inset Formula $\pi g=g$ \end_inset for all \begin_inset Formula $g\in G$ \end_inset \end_layout \begin_layout Enumerate \family roman \series medium \shape up \size normal \emph off \bar no \noun off \color none \begin_inset Formula $\left\langle \pi f,g\right\rangle =\left\langle f,\pi g\right\rangle $ \end_inset for all \begin_inset Formula $f,g\in H$ \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard This map \begin_inset Formula $\pi$ \end_inset is called the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{orthogonal projection} \end_layout \end_inset from \begin_inset Formula $H$ \end_inset to \begin_inset Formula $G$ \end_inset . Note that \begin_inset Quotes eld \end_inset \begin_inset Formula $G$ \end_inset a closed subspace \begin_inset Quotes erd \end_inset means that \begin_inset Formula $G$ \end_inset must be closed in the topology on \begin_inset Formula $H$ \end_inset and must be a subspace of \begin_inset Formula $H$ \end_inset as a vector space. \end_layout \begin_layout Standard Work with \begin_inset Formula $H=L^{2}(\Omega,\cF,\P)$ \end_inset and let \begin_inset Formula $\cG$ \end_inset be a sub- \begin_inset Formula $\sigma$ \end_inset -field of \begin_inset Formula $\cF$ \end_inset . The space \begin_inset Formula $G=L^{2}(\Omega,\cG,\P)$ \end_inset is a closed subspace of \begin_inset Formula $H$ \end_inset (closed because it is complete), so we may let \begin_inset Formula $\pi$ \end_inset be the orthogonal projection \begin_inset Formula $H\rightarrow G$ \end_inset . \end_layout \begin_layout Standard For any \begin_inset Formula $X\in H$ \end_inset , \begin_inset Formula $\pi(X)$ \end_inset is \begin_inset Formula $\cG$ \end_inset -measurable since \begin_inset Formula $\pi(X)\in G$ \end_inset . And for any \begin_inset Formula $B\in\cG$ \end_inset , \begin_inset Formula $I_{B}\in G$ \end_inset so \begin_inset Formula \[ \int_{B}\pi(X)d\P=\left\langle I_{B},\pi(X)\right\rangle =\left\langle \pi(I_{B}),X\right\rangle =\left\langle I_{B},X\right\rangle =\int_{B}Xd\P\] \end_inset \end_layout \begin_layout Standard Hence \begin_inset Formula $\pi(X)$ \end_inset is a conditional expectation of \begin_inset Formula $X$ \end_inset given \begin_inset Formula $\cG$ \end_inset . Since we are working with \begin_inset Formula $L^{2}$ \end_inset rather than \begin_inset Formula $\cL^{2}$ \end_inset , this is only defined up to almost sure equality, corresponding to the uniqueness property of conditional expectation. \end_layout \begin_layout Standard Intuitively, a projection is an operation into a subspace which discards information lying outside that subspace: for example, consider \begin_inset Formula $\R^{n}$ \end_inset with basis \begin_inset Formula $\{e_{1},\ldots,e_{n}\}$ \end_inset . The orthogonal projection onto the subspace generated by \begin_inset Formula $\{e_{1},\ldots,e_{m}\}$ \end_inset corresponds to throwing away coordinates after the \begin_inset Formula $m$ \end_inset -th. In the conditional expectation case, \begin_inset Formula $\pi$ \end_inset is throwing away information not contained in the \begin_inset Formula $\sigma$ \end_inset -field \begin_inset Formula $\cG$ \end_inset . \end_layout \begin_layout Standard The property \begin_inset Formula $\Vert X-\E(X|\cG)\Vert_{2}\leq\Vert X-Y\Vert_{2}$ \end_inset for all \begin_inset Formula $Y\in G$ \end_inset shows that for \begin_inset Formula $X\in L^{2}$ \end_inset , \begin_inset Formula $\E(X|\cG)$ \end_inset is the best estimate we can give for \begin_inset Formula $X$ \end_inset after knowing the information represented by \begin_inset Formula $\cG$ \end_inset , if we measure \begin_inset Quotes eld \end_inset best \begin_inset Quotes erd \end_inset by minimising \begin_inset Formula $\E(\vert X-Y\vert^{2})$ \end_inset . \end_layout \begin_layout Section Ergodicity \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status open \begin_layout Plain Layout [4] The strong law of large numbers, proof for independent random variables with bounded fourth moments. Measure preserving transformations, Bernoulli shifts. Statements *and proofs* of maximal ergodic theorem and Birkhoff's almost everywhere ergodic theorem, proof of the strong law. \end_layout \end_inset \end_layout \begin_layout Subsection Strong law of large numbers \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Strong law of large numbers] \end_layout \end_inset If \begin_inset Formula $X_{n}$ \end_inset are i.i.d. r.v.s with \begin_inset Formula $\E\vert X_{1}\vert<\infty$ \end_inset then \begin_inset Formula \[ \overline{X}_{n}=\frac{X_{1}+\ldots+X_{n}}{n}\rightarrow\E X_{1}\mbox{ almost surely.}\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard The strong law of large numbers tells us that if we take finite samples from some probability distribution, as we increase the sample size, the sample mean converges almost surely to the expectation of the distribution. This is important because it justifies the intuitive understanding of expectati on and many applications in statistics. \end_layout \begin_layout Standard The weak law is a strictly weaker result than the strong law: it says that, with the same hypothesis, the sample mean converges in probability to the expectation. It may still be useful because it has a simpler proof than the strong law. Indeed it is non-trivial that \family roman \series medium \shape up \size normal \emph off \bar no \noun off \color none \begin_inset Formula $\{\omega\colon\overline{X}_{n}\rightarrow\E X_{1}\}$ \end_inset is measurable; this holds because \begin_inset Formula $\overline{X}_{n}$ \end_inset is certainly measurable, so \begin_inset Formula $\limsup_{n}\overline{X}_{n},\liminf_{n}\overline{X}_{n}$ \end_inset are measurable, and so \begin_inset Formula $\{\limsup\overline{X}_{n}=\liminf\overline{X}_{n}=\E X_{1}\}$ \end_inset is measurable. This is not a problem for the weak law, which only talks about events \begin_inset Formula $\{\vert\overline{X}_{n}-\E X_{1}\vert>\epsilon\}$ \end_inset . \end_layout \begin_layout Standard There are many proofs of the strong law, which vary in the conditions they require and techniques they use. We begin with an easy proof, subject to the assumption that the random variables have bounded fourth moments. Note that this proof does not require the variables to be identically distribut ed; the condition that \begin_inset Formula $\E X_{n}=0$ \end_inset is just for simplicity and can be obtained by translating. \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset If \begin_inset Formula $X_{n}$ \end_inset are independent r.v.s with \begin_inset Formula $\E X_{n}=0$ \end_inset and \begin_inset Formula $\E(X_{n}^{4})0$ \end_inset , we show that \begin_inset Formula $\P(\vert\overline{X}_{n}\vert>\epsilon\mbox{ infinitely often})=0$ \end_inset . Then we are done because \begin_inset Formula $\overline{X}_{n}\not\rightarrow0\Leftrightarrow((\overline{X}_{n}>\vert j^{-1}\vert\mbox{ infinitely often})\mbox{ for some }j\in\N)$ \end_inset , expressing \begin_inset Formula $\overline{X}_{n}\not\rightarrow0$ \end_inset as a countable union of probability \begin_inset Formula $0$ \end_inset events. \end_layout \begin_layout Standard Let \begin_inset Formula $S_{n}=X_{1}+\ldots+X_{n}$ \end_inset and consider \begin_inset Formula $S_{n}^{4}=\sum_{i=1}^{n}\sum_{j=1}^{n}\sum_{k=1}^{n}\sum_{l=1}^{n}X_{i}X_{j}X_{k}X_{l}$ \end_inset . \end_layout \begin_layout Standard If there is one of \begin_inset Formula $i,j,k,l$ \end_inset not equal to any of the rest (say \begin_inset Formula $i$ \end_inset ) then by independence, \begin_inset Formula $\E(X_{i}X_{j}X_{k}X_{l})=\E(X_{i})\E(X_{j}X_{k}X_{l})=0$ \end_inset . \end_layout \begin_layout Standard So the only non-zero terms in the expansion of \begin_inset Formula $\E(S_{n}^{4})$ \end_inset have form \begin_inset Formula $\E(X_{i}^{2}X_{j}^{2})$ \end_inset or \begin_inset Formula $\E(X_{i}^{4})$ \end_inset . \end_layout \begin_layout Standard By Hölder, \begin_inset Formula $\E(X_{i}^{2}X_{j}^{2})\leq\left(\E(X_{i}^{4})\right)^{1/2}\left(\E(X_{j}^{4})\right)^{1/2}\epsilon^{4}n^{4})\leq\E(S_{n}^{4})<3n^{2}c$ \end_inset . \end_layout \begin_layout Standard So \begin_inset Formula $\P(\vert S_{n}\vert/n>\epsilon)<3c\epsilon^{-4}n^{-2}$ \end_inset and \begin_inset Formula $\sum_{n}\P(\vert S_{n}\vert/n>\epsilon)<3c\epsilon^{-4}\sum n^{-2}<\infty$ \end_inset . \end_layout \begin_layout Standard Borel-Cantelli I gives us \begin_inset Formula $\P(\limsup\{\vert\overline{X}_{n}\vert>\epsilon\})=0$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection Measure-preserving transformations \end_layout \begin_layout Standard A \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{measure-preserving transformation} \end_layout \end_inset is a measurable function \begin_inset Formula $T:(S,\cS,\mu)\rightarrow(S,\cS)$ \end_inset s.t. \begin_inset Formula $\mu(T^{-1}A)=\mu(A)$ \end_inset for all \begin_inset Formula $A\in\cS$ \end_inset . (Measurability of \begin_inset Formula $T$ \end_inset ensures that \begin_inset Formula $T^{-1}A$ \end_inset is always measurable. Note that the definition contains \begin_inset Formula $T^{-1}$ \end_inset rather than \begin_inset Formula $T$ \end_inset .) \end_layout \begin_layout Standard We can restate this definition in terms of measurable functions on \begin_inset Formula $(S,\cS)$ \end_inset using the standard machine: for any \begin_inset Formula $A\in\cS$ \end_inset , \begin_inset Formula $\omega\in T^{-1}A\Leftrightarrow T\omega\in A$ \end_inset so \begin_inset Formula \[ \int I_{A}d\mu=\int I_{T^{-1}A}d\mu=\int(I_{A}\circ T)d\mu\] \end_inset Linearity and monotone convergence give us \begin_inset Formula $\int fd\mu=\int(f\circ T)d\mu$ \end_inset for all measurable \begin_inset Formula $f\colon S\rightarrow\R$ \end_inset . The converse is also true: if \begin_inset Formula $\int fd\mu=\int(f\circ T)d\mu$ \end_inset for all measurable \begin_inset Formula $f\colon S\rightarrow\R$ \end_inset then \begin_inset Formula $f$ \end_inset is measure-preserving. \end_layout \begin_layout Standard A measurable set \begin_inset Formula $A$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{$T$-invariant} \end_layout \end_inset if \begin_inset Formula $\mu(A\triangle T^{-1}A)=0$ \end_inset ( \begin_inset Formula $\triangle$ \end_inset is symmetric difference). In other words, \begin_inset Formula $A$ \end_inset is almost the same as \begin_inset Formula $T^{-1}A$ \end_inset , except for a null set. Note that if \begin_inset Formula $\mu$ \end_inset is finite then the \begin_inset Formula $T$ \end_inset -invariant sets form a \begin_inset Formula $\sigma$ \end_inset -field. \end_layout \begin_layout Subsection Stationary sequences \end_layout \begin_layout Standard An important example of measure-preserving transformations comes from stationary sequences. \end_layout \begin_layout Standard A sequence of random variables \begin_inset Formula $(X_{n})$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{stationary} \end_layout \end_inset if \begin_inset Formula $(X_{n})_{n\geq1}$ \end_inset has the same distribution as \begin_inset Formula $(X_{n+1})_{n\geq1}$ \end_inset . \end_layout \begin_layout Standard Examples of stationary sequences are any i.i.d. sequence and the values of an aperiodic positive-recurrent Markov chain with an invariant distribution as the initial distribution. \end_layout \begin_layout Standard Instead of considering \begin_inset Formula $(X_{n})$ \end_inset as a sequence of functions \begin_inset Formula $\Omega\rightarrow\R$ \end_inset , it is convenient to consider it as a single function taking values in \begin_inset Formula $\R^{\N}$ \end_inset , the space of real-valued sequences. We use the product \begin_inset Formula $\sigma$ \end_inset -field \begin_inset Formula $B(\R^{\N})$ \end_inset on \begin_inset Formula $\R^{\N}$ \end_inset ; this is defined as the smallest \begin_inset Formula $\sigma$ \end_inset -field on which all the projection maps (picking out one element of the sequence) are measurable; since we are taking a product of only countably many copies of \begin_inset Formula $\R$ \end_inset , this is the same as the \begin_inset Formula $\sigma$ \end_inset -field generated by sets of the form \begin_inset Formula $\prod_{n=1}^{\infty}A_{n}$ \end_inset where \begin_inset Formula $A_{n}\in B(\R)$ \end_inset . Hence we can define a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{random sequence} \end_layout \end_inset to be a function \begin_inset Formula $X\colon(\Omega,\cF,\P)\rightarrow\R^{\N}$ \end_inset which is measurable w.r.t. \begin_inset Formula $B(\R^{\N})$ \end_inset . Then \begin_inset Formula $X=(X_{1},X_{2},\ldots)$ \end_inset is a random sequence iff \begin_inset Formula $X_{1},X_{2},\ldots$ \end_inset are random variables. \end_layout \begin_layout Standard To obtain a measure-preserving transformation from a stationary sequence, consider the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{Bernoulli shift} \end_layout \end_inset : the map \begin_inset Formula $S:\R^{\N}\rightarrow\R^{\N}$ \end_inset which throws away the first element of the sequence, i.e. \begin_inset Formula $S(x_{1},x_{2},\ldots)=(x_{2},x_{3},\dots)$ \end_inset . Note that this need not correspond to any transformation on the original probability space \begin_inset Formula $\Omega$ \end_inset (i.e. there need not be any transformation \begin_inset Formula $T:\Omega\rightarrow\Omega$ \end_inset such that \begin_inset Formula $X\circ T=S\circ X$ \end_inset ). \end_layout \begin_layout Standard Nevertheless, the sequence \begin_inset Formula $X$ \end_inset induces a measure \begin_inset Formula $\P\circ X^{-1}$ \end_inset on \begin_inset Formula $(\R^{\N},B(\R^{\N}))$ \end_inset . We can then consider the Bernoulli shift as a transformation \begin_inset Formula $(\R^{\N},B(\R^{\N}),\P\circ X^{-1})\rightarrow(\R^{\N},B(\R^{\N}))$ \end_inset . \begin_inset Formula $S$ \end_inset is always measurable w.r.t. \begin_inset Formula $B(\R^{\N})$ \end_inset , and it is easy to see that \begin_inset Formula $S$ \end_inset preserves the measure \begin_inset Formula $\P\circ X^{-1}$ \end_inset iff the sequence is stationary (simply restating definitions). \end_layout \begin_layout Standard Conversely, any measure-preserving transformation \begin_inset Formula $T$ \end_inset on \begin_inset Formula $(\Omega,\cF,\P)$ \end_inset together with a random variable \begin_inset Formula $f:(\Omega,\cF)\rightarrow(\R,B(\R))$ \end_inset gives rise to a stationary sequence defined by \begin_inset Formula $X_{n}(\omega)=f(T^{n}(\omega))$ \end_inset . \end_layout \begin_layout Subsection Ergodic theorems \end_layout \begin_layout Standard Birkhoff's (pointwise) ergodic theorem is a generalisation of the strong law of large numbers which applies to general stationary sequences. The maximal ergodic theorem is used as a lemma in the proof, and I believe it has other applications in analysis but I don't know about them. The proofs are in the Appendix. \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Maximal Ergodic Theorem] \end_layout \end_inset If \begin_inset Formula $T:(S,\cS,\mu)\rightarrow(S,\cS,\mu)$ \end_inset is measure-preserving, \begin_inset Formula $f\in L^{1}(S,\cS,\mu)$ \end_inset , \begin_inset Formula $S_{n}(\omega)=\sum_{k=1}^{n}f(T^{k-1}\omega)$ \end_inset and \begin_inset Formula $A=\{\omega\colon\sup S_{n}>0\}$ \end_inset , then \begin_inset Formula $\int_{A}fd\mu\geq0$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Birkhoff's Ergodic Theorem] \end_layout \end_inset If \begin_inset Formula $T:(\Omega,\cF,\P)\rightarrow(\Omega,\cF,\P)$ \end_inset is measure-preserving, \begin_inset Formula $\cI$ \end_inset is the invariant \begin_inset Formula $\sigma$ \end_inset -field of \begin_inset Formula $T$ \end_inset , and \begin_inset Formula $f\in L^{1}$ \end_inset then \begin_inset Formula \[ \frac{1}{n}\sum_{k=1}^{n}f(T^{k-1}\omega)\rightarrow\E(f|\cI)\mbox{ almost surely.}\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard This can also be viewed from the point of view of dynamics: we take our probability space as the space of states for some dynamical system and let \begin_inset Formula $T$ \end_inset be the transformation corresponding to advancing by a single unit of time (Liouville's theorem in dynamics tells us that this transformation is measure-p reserving). The invariant \begin_inset Formula $\sigma$ \end_inset -field \begin_inset Formula $\cI$ \end_inset consists of the components of the state space which a particle (almost surely) does not enter or leave. \end_layout \begin_layout Standard Then \begin_inset Formula $\lim_{n\rightarrow\infty}\frac{1}{n}\sum_{k=1}^{n}f(T^{k-1}\omega)$ \end_inset is the average over time of the value of \begin_inset Formula $f$ \end_inset for a single particle, and \begin_inset Formula $\E(f|\cI)$ \end_inset is the average value of \begin_inset Formula $f$ \end_inset over all points in a component of the state space, at an instant in time. Birkhoff's ergodic theorem tells us that the time average and space average are the same. \end_layout \begin_layout Subsection Ergodic transformations \end_layout \begin_layout Standard The \begin_inset Formula $\E(f|\cI)$ \end_inset on the right hand side of Birkhoff's ergodic theorem suggests that we should be particularly interested in transformations for which \begin_inset Formula $\cI$ \end_inset is trivial (i.e. \begin_inset Formula $\P(A)=0\mbox{ or }1\;\forall A\in\cI$ \end_inset ). Since \begin_inset Formula $\E(f|\cI)$ \end_inset is a \begin_inset Formula $\cI$ \end_inset -measurable r.v., for such a \begin_inset Formula $\sigma$ \end_inset -field it will be almost surely constant. So we define: \end_layout \begin_layout Standard A measure-preserving transformation \begin_inset Formula $T$ \end_inset is \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{ergodic} \end_layout \end_inset if its invariant \begin_inset Formula $\sigma$ \end_inset -field is trivial. \end_layout \begin_layout Standard Note that the value which an a.s. constant r.v. takes with probability 1 must be equal to its expectation. So \begin_inset Formula $\E(f|\cI)=\E(\E(f|\cI))=\E(f)$ \end_inset almost surely. \end_layout \begin_layout Standard So to prove the strong law of large numbers, all we need to prove is: \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset If \begin_inset Formula $(X_{1},X_{2},\ldots)$ \end_inset is a sequence of i.i.d. r.v.s, then the shift operator \begin_inset Formula $S$ \end_inset is ergodic. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset As observed above, we are treating \begin_inset Formula $S$ \end_inset as a measure-preserving transformation on the probability space \begin_inset Formula $(\R^{\N},B(\R^{\N}),\P\circ X^{-1})$ \end_inset . \end_layout \begin_layout Standard Call a measurable set \begin_inset Formula $A$ \end_inset \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{strictly invariant} \end_layout \end_inset (non-standard terminology) if \begin_inset Formula $S^{-1}A=A$ \end_inset . The strictly invariant sets form a \begin_inset Formula $\sigma$ \end_inset -field \begin_inset Formula $\cJ$ \end_inset . \end_layout \begin_layout Standard We shall show that \begin_inset Formula $\cJ$ \end_inset is contained in the tail \begin_inset Formula $\sigma$ \end_inset -field \begin_inset Formula $\cT$ \end_inset so by Kolmogorov's 0-1 law \begin_inset Formula $\cJ$ \end_inset is trivial. We shall also show that an invariant set differs from a strictly invariant set by a null set, so \begin_inset Formula $\cI$ \end_inset is trivial and \begin_inset Formula $S$ \end_inset is ergodic. \end_layout \begin_layout Standard Informally, what we do is observe that \begin_inset Formula $\omega\in A\Leftrightarrow S\omega\in A$ \end_inset and that whether \begin_inset Formula $S\omega$ \end_inset is in \begin_inset Formula $A$ \end_inset can be determined by looking at just \begin_inset Formula $X_{2},X_{3},\ldots$ \end_inset , so \begin_inset Formula $A\in\sigma(X_{2},X_{3},\ldots)$ \end_inset . \end_layout \begin_layout Standard Doing this formally requires quite a bit of manipulation of \begin_inset Formula $\sigma$ \end_inset -fields: \end_layout \begin_layout Standard Note that \begin_inset Formula $X_{n}(S\omega)=X_{n+1}(\omega)$ \end_inset so \begin_inset Formula $X_{n}\circ S=X_{n+1}$ \end_inset . So the functions \begin_inset Formula $X_{n}\circ S$ \end_inset are all measurable on \begin_inset Formula $\sigma(X_{2},X_{3},\ldots)$ \end_inset , and \begin_inset Formula $\sigma(X_{1}\circ S,X_{2}\circ S,\ldots)\subset\sigma(X_{2},X_{3},\ldots)$ \end_inset . \end_layout \begin_layout Standard Let \begin_inset Formula $\cF_{0}=\{A\subset\Omega\colon S^{-1}A\in\sigma(X_{1}\circ S,X_{2}\circ S,\ldots)\}$ \end_inset . It is straightforward to check that this is a \begin_inset Formula $\sigma$ \end_inset -field. If \begin_inset Formula $B$ \end_inset is a measurable subset of \begin_inset Formula $\R$ \end_inset , then \begin_inset Formula $S^{-1}(X_{n}^{-1}(B))=(X_{n}\circ S)^{-1}(B)\in\sigma(X_{1}\circ S,X_{2}\circ S,\ldots)$ \end_inset so \begin_inset Formula $X_{n}^{-1}(B)\in\cF_{0}$ \end_inset for all \begin_inset Formula $n$ \end_inset . So \begin_inset Formula $X_{n}$ \end_inset is \begin_inset Formula $\cF_{0}$ \end_inset -measurable for all \begin_inset Formula $n$ \end_inset , so \begin_inset Formula $\cF_{0}\supset\sigma(X_{1},X_{2},\ldots)$ \end_inset . \end_layout \begin_layout Standard Now given \begin_inset Formula $A\in\cJ$ \end_inset , we have \begin_inset Formula $A\in\sigma(X_{1},X_{2},\ldots)\subset\cF_{0}$ \end_inset so \begin_inset Formula $A=S^{-1}A\in\sigma(X_{2},X_{3},\ldots)$ \end_inset . \end_layout \begin_layout Standard Similarly, \begin_inset Formula $A\in\sigma(X_{n},X_{n+1},\ldots)$ \end_inset for all \begin_inset Formula $n$ \end_inset , so \begin_inset Formula $A\in\cT$ \end_inset . \end_layout \begin_layout Standard \begin_inset VSpace defskip \end_inset \end_layout \begin_layout Standard Given \begin_inset Formula $A\in\cI$ \end_inset , we need to check that there is \begin_inset Formula $B\in\cJ$ \end_inset s.t. \begin_inset Formula $A\triangle B$ \end_inset is a null set. There ought to be an easier way than the following, but this is the best I have come up with. \end_layout \begin_layout Standard Let \begin_inset Formula $B=\limsup_{n\geq0}S^{-n}A$ \end_inset . Then \begin_inset Formula $S^{-1}B=\limsup_{n\geq1}S^{-n}A=B$ \end_inset so \begin_inset Formula $B\in\cJ$ \end_inset . \end_layout \begin_layout Standard By lots of set chasing, \begin_inset Formula \begin{align*} A\triangle B=(A\setminus B)\cup(B\setminus A) & =\liminf(A\setminus S^{-n}A)\cup\limsup(S^{-n}A\setminus A)\\ & \subset\liminf(A\triangle S^{-n}A)\cup\limsup(A\triangle S^{-n}A)\\ & =\limsup(A\triangle S^{-n}A)\end{align*} \end_inset \end_layout \begin_layout Standard Now \begin_inset Formula $A\triangle S^{-n}A\subset\bigcup_{k=1}^{n}(S^{-k+1}A\triangle S^{-k}A)$ \end_inset and as \begin_inset Formula $S$ \end_inset is measure-preserving, \begin_inset Formula \[ \mu(S^{-k+1}A\triangle S^{-k}A)=\mu(S^{-1}A\triangle A)=0\] \end_inset so \begin_inset Formula $\mu(A\triangle S^{-n}A)=0$ \end_inset for all \begin_inset Formula $n$ \end_inset . \end_layout \begin_layout Standard Hence \begin_inset Formula $\mu(A\triangle B)=0$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Section Characteristic functions \end_layout \begin_layout Standard \begin_inset Box Framed position "t" hor_pos "c" has_inner_box 0 inner_pos "t" use_parbox 0 width "100col%" special "none" height "1in" height_special "totalheight" status collapsed \begin_layout Plain Layout [2] The Fourier transform of a finite measure, characteristic functions, uniqueness and inversion. Weak convergence, statement of Lévy's convergence theorem for characteristic functions. \end_layout \end_inset This discussion of Fourier transforms is of course more rigorous than the treatment in IB Methods, but it is still not entirely satisfactory. For example, there is annoying asymmetry between the domains on which the Fourier transform and the inverse Fourier transform are defined. \end_layout \begin_layout Subsection Fourier transforms \end_layout \begin_layout Standard The \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{Fourier transform} \end_layout \end_inset of a finite measure \begin_inset Formula $\mu$ \end_inset on \begin_inset Formula $(\R,B(\R))$ \end_inset is the function \begin_inset Formula $\hat{\mu}:\R\rightarrow\C$ \end_inset \begin_inset Formula \[ \hat{\mu}(t)=\int e^{itx}\mu(dx)\] \end_inset (Note of course the usual variety of conventions in defining Fourier transforms : it may be defined as the conjugate of this, or with a constant multiplier. Note also that we extend integration from real-valued to complex-valued functions just by adding real and imaginary parts, so long as \begin_inset Formula $\int\vert f\vert d\mu<\infty$ \end_inset .) \end_layout \begin_layout Standard The Fourier transform defined in applied maths as \begin_inset Formula $\hat{f}(t)=\int_{-\infty}^{\infty}e^{itx}f(x)dx$ \end_inset is the Fourier transform in this sense of the measure with density function \begin_inset Formula $f$ \end_inset . (Not all measures have density functions; I will add a definition and discussio n of them somewhere eventually.) \end_layout \begin_layout Standard In the case of a random variable \begin_inset Formula $X$ \end_inset , the Fourier transform with respect to its Lebesgue-Stieltjes measure is the \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{characteristic function} \end_layout \end_inset \begin_inset Formula $\varphi_{X}(t)=\E(e^{itX})$ \end_inset . \end_layout \begin_layout Standard Note that \begin_inset Formula $\hat{\mu}(0)=\mu(\R)$ \end_inset and \begin_inset Formula $\vert\hat{\mu}(t)\vert\leq\mu(\R)$ \end_inset for all \begin_inset Formula $t$ \end_inset . \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{lemma} \end_layout \end_inset The Fourier transform \begin_inset Formula $\varphi$ \end_inset of a finite measure \begin_inset Formula $\mu$ \end_inset is uniformly continuous. \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{lemma} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset We have \begin_inset Formula $\vert e^{itx}-1\vert\rightarrow0$ \end_inset as \begin_inset Formula $t\rightarrow0$ \end_inset , \begin_inset Formula $\vert e^{itx}-1\vert\leq2$ \end_inset for all \begin_inset Formula $t$ \end_inset , and the constant \begin_inset Formula $2$ \end_inset is integrable since \begin_inset Formula $\mu$ \end_inset is finite, so by the Dominated Convergence Theorem, \begin_inset Formula \[ \int\vert e^{itx}-1\vert\mu(dx)\rightarrow0\mbox{ as }t\rightarrow0\] \end_inset \end_layout \begin_layout Standard Now given \begin_inset Formula $\epsilon>0$ \end_inset , we can find \begin_inset Formula $\delta>0$ \end_inset s.t. \begin_inset Formula $\vert h\vert<\delta\Rightarrow\int\vert e^{ihx}-1\vert\mu(dx)<\epsilon$ \end_inset . For any \begin_inset Formula $t,h\in\R$ \end_inset with \begin_inset Formula $\vert h\vert<\delta$ \end_inset , we have \begin_inset Formula \[ \vert\hat{\mu}(t+h)-\hat{\mu}(t)\vert=\left|\int e^{itx}(e^{ihx}-1)\mu(dx)\right|\leq\int\vert e^{itx}\vert\vert e^{ihx}-1\vert\mu(dx)=\int\vert e^{ihx}-1\vert\mu(dx)<\epsilon\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Subsection Inverse of Fourier transform \end_layout \begin_layout Standard Given a function \begin_inset Formula $\hat{\mu}:\R\rightarrow\C$ \end_inset which is the Fourier transform of some finite measure \begin_inset Formula $\mu$ \end_inset , we would like to recover \begin_inset Formula $\mu$ \end_inset ; in particular it will be important to know that there is only one measure with given Fourier transform. (We do not consider here conditions for such a measure to exist; we simply assume that the \begin_inset Formula $\hat{\mu}$ \end_inset we are given is the transform of some measure.) \end_layout \begin_layout Standard Recall the inversion formula from Methods: \begin_inset Formula $f(x)=\frac{1}{2\pi}\int_{-\infty}^{\infty}e^{-itx}\hat{f}(t)dt$ \end_inset . But we do not know that the RHS is integrable; or from another point of view, this formula constructs a density \begin_inset Formula $f$ \end_inset for \begin_inset Formula $\mu$ \end_inset , so can only work if \begin_inset Formula $\mu$ \end_inset has a density function. \end_layout \begin_layout Standard To allow us to use this formula, we smooth out \begin_inset Formula $\mu$ \end_inset so that it does have a density. This can be done by convolving with a \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{mollifier} \end_layout \end_inset , a smooth function with properties that allow us to reconstruct the original measure by taking a limit. This convolution of course corresponds to multiplication of Fourier transforms. \end_layout \begin_layout Standard A suitable function to use (both this function and its Fourier transform are smooth) is \begin_inset Formula $\exp(-\epsilon t^{2}/2)$ \end_inset . As \begin_inset Formula $\epsilon\rightarrow0$ \end_inset , this tends to 1 and so \begin_inset Formula $\exp(-\epsilon t^{2}/2)\hat{\mu}(t)\rightarrow\hat{\mu}(t)$ \end_inset . (Another way of looking at this is that we are mixing a small amount of Gaussian noise into the distribution.) \end_layout \begin_layout Standard Now we can define \begin_inset Formula \[ f_{\epsilon}(x)=\frac{1}{2\pi}\int e^{-itx}\exp(-\epsilon t^{2}/2)\hat{\mu}(t)dt\] \end_inset This integral is defined and finite because \begin_inset Formula $e^{-itx}\exp(-\epsilon t^{2}/2)\hat{\mu(t)}$ \end_inset is continuous, so measurable, and it is dominated by \begin_inset Formula $\exp(-\epsilon t^{2}/2)\mu(\R)$ \end_inset which is integrable. \end_layout \begin_layout Standard Now \begin_inset Formula $f_{\epsilon}(x)$ \end_inset should be an approximate density for \begin_inset Formula $\mu$ \end_inset . In fact we have \begin_inset Formula \begin{align*} f_{\epsilon}(x) & =\frac{1}{2\pi}\int e^{-itx}\exp(-\epsilon t^{2}/2)\left(\int e^{itx'}\mu(dx')\right)dt\\ & =\frac{1}{2\pi}\int\int\exp(it(x'-x)-\epsilon t^{2}/2)\mu(dx')dt\\ & =\frac{1}{2\pi}\int\int e^{it(x'-x)}\exp(-\epsilon t^{2}/2)dt\,\mu(dx')\mbox{ by Fubini}\end{align*} \end_inset (The application of Fubini is justified because the integrand is dominated by \begin_inset Formula $\exp(-\epsilon t^{2}/2)$ \end_inset and \begin_inset Formula $\mu$ \end_inset is finite, so \begin_inset Formula $\int\int\exp(-\epsilon t^{2}/2)\mu(dx')dt<\infty$ \end_inset .) \end_layout \begin_layout Standard This contains the Fourier transform of a Gaussian (just with different labels), which we know how to compute, so we get \begin_inset Formula \[ f_{\epsilon}(x)=\frac{1}{\sqrt{2\pi\epsilon}}\int\exp(-(x'-x)^{2}/2\epsilon)\mu(dx')\] \end_inset \end_layout \begin_layout Standard This is the density of an approximation to \begin_inset Formula $\mu$ \end_inset , so we get the measure by integrating. Note that \begin_inset Formula $g_{\epsilon}(x,x')=\exp(-(x'-x)^{2}/2\epsilon)$ \end_inset is non-negative, so we can apply Tonelli's (a.k.a. Fubini's) theorem without having to check that the integrals involved are finite. Define \begin_inset Formula \begin{align*} F_{\epsilon}(a) & =\int_{-\infty}^{a}f_{\epsilon}(x)dx\\ & =\frac{1}{\sqrt{2\pi\epsilon}}\int_{-\infty}^{a}\int\exp(-(x'-x)^{2}/2\epsilon)\mu(dx')dx\\ & =\frac{1}{\sqrt{2\pi\epsilon}}\int\int_{-\infty}^{a}\exp(-(x-x')^{2}/2\epsilon)dx\,\mu(dx')\mbox{ (Tonelli)}\\ & =\int\Phi\left(\frac{a-x'}{\sqrt{\epsilon}}\right)\mu(dx')\end{align*} \end_inset where \begin_inset Formula $\Phi(x)=\int_{-\infty}^{x}\exp(-t^{2}/2)dt$ \end_inset is the standard Gaussian distribution function. \end_layout \begin_layout Standard Now as \begin_inset Formula $\epsilon\rightarrow0$ \end_inset , \begin_inset Formula \[ \Phi\left(\frac{a-x'}{\sqrt{\epsilon}}\right)\rightarrow\begin{cases} 0 & \mbox{if }ax'\end{cases}\] \end_inset and it is dominated by 1, so by DCT we get \begin_inset Formula \[ F_{\epsilon}(a)\rightarrow\int{\textstyle \frac{1}{2}}I_{\{a=x'\}}+I_{\{a>x'\}}\mu(dx')=\mu((-\infty,a))+{\textstyle \frac{1}{2}}\mu(\{a\})\] \end_inset \end_layout \begin_layout Standard This is sufficient to determine \begin_inset Formula $\mu$ \end_inset on the \begin_inset Formula $\pi$ \end_inset -system \begin_inset Formula $\{(-\infty,a):a\in\R\}$ \end_inset , using the fact that \begin_inset Formula $\mu((-\infty,a))$ \end_inset is increasing and right-continuous, and so to determine \begin_inset Formula $\mu$ \end_inset . \end_layout \begin_layout Subsection Weak convergence \end_layout \begin_layout Standard The modes of convergence we considered earlier were convergence of functions. Now we consider a form of convergence of measures. \end_layout \begin_layout Standard Let \begin_inset Formula $(S,\cS)$ \end_inset be a metric space with its Borel \begin_inset Formula $\sigma$ \end_inset -field. \end_layout \begin_layout Standard A sequence of measures \begin_inset Formula $\mu_{n}$ \end_inset on \begin_inset Formula $(S,\cS)$ \end_inset \begin_inset ERT status open \begin_layout Plain Layout \backslash defterm{converges weakly} \end_layout \end_inset to \begin_inset Formula $\mu$ \end_inset if \begin_inset Formula $\int fd\mu_{n}\rightarrow\int fd\mu$ \end_inset for all bounded, continuous functions \begin_inset Formula $f:S\rightarrow\R$ \end_inset . We write this \begin_inset Formula $\mu_{n}\Rightarrow\mu$ \end_inset . \end_layout \begin_layout Standard Note that weak convergence can be considered as a mode of convergence of random variables, by treating it as weak convergence of the corresponding Lebesgue-Stieltjes measure. Regarded in this way, weak convergence of r.v.s is implied by convergence in probability, and so by any of the modes of convergence considered earlier. \end_layout \begin_layout Standard There is an alternative characterisation of weak convergence which is often easier to prove: \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{theorem} \end_layout \end_inset \begin_inset Formula $\mu_{n}\Rightarrow\mu$ \end_inset iff \begin_inset Formula $\mu(A)\leq\liminf\mu_{n}(A)$ \end_inset for all open sets \begin_inset Formula $A$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Suppose that \begin_inset Formula $\mu_{n}\Rightarrow\mu$ \end_inset and let \begin_inset Formula $A$ \end_inset be an open set. \end_layout \begin_layout Standard Consider the function \begin_inset Formula $h_{n}(x)=\inf\{1,n\vert x-y\vert\colon y\in A^{c}\}$ \end_inset , which is a bounded and continuous approximation to \begin_inset Formula $I_{A}$ \end_inset . \begin_inset Formula $(A^{c}$ \end_inset closed ensures that \begin_inset Formula $\inf\{\vert x-y\vert:y\in A^{c}\}>0$ \end_inset for \begin_inset Formula $x\in A$ \end_inset .) \end_layout \begin_layout Standard Specifically, \begin_inset Formula $0\leq h_{n}\uparrow I_{A}$ \end_inset . \end_layout \begin_layout Standard So for any \begin_inset Formula $k$ \end_inset we have \begin_inset Formula \begin{align*} \liminf\mu_{n}(A)\geq\liminf{\textstyle \int}h_{k}d\mu_{n} & ={\textstyle \int}h_{k}d\mu & & \mbox{(weak convergence)}\\ & \uparrow{\textstyle \int}I_{A}d\mu=\mu(A) & & \mbox{(monotone convergence)}\end{align*} \end_inset \end_layout \begin_layout Standard Conversely, suppose that \begin_inset Formula $\mu(A)\leq\liminf\mu_{n}(A)$ \end_inset for all open sets \begin_inset Formula $A$ \end_inset and let \begin_inset Formula $f$ \end_inset be a bounded, continuous function. Since \begin_inset Formula $f$ \end_inset is bounded, we can add a constant to it to get \begin_inset Formula $f\geq0$ \end_inset . \end_layout \begin_layout Standard The integral of a measurable function is defined by looking at sums of indicator s of measurable sets. Here we only know things about the measure of open sets, but on the other hand \begin_inset Formula $f$ \end_inset is continuous so can be approximated by sums of indicators of open sets. (Essentially we are computing the Riemann instead of the Lebesgue integral of \begin_inset Formula $f$ \end_inset .) \end_layout \begin_layout Standard Fix \begin_inset Formula $m\in\N$ \end_inset (higher \begin_inset Formula $m$ \end_inset means better approximation of \begin_inset Formula $f$ \end_inset ). \end_layout \begin_layout Standard For \begin_inset Formula $j\in\N$ \end_inset , let \begin_inset Formula $A_{j}=\{\omega\colon f(\omega)>j/m\}$ \end_inset , which is open. \end_layout \begin_layout Standard Now \begin_inset Formula $I_{A_{j}}(\omega)=1$ \end_inset iff \begin_inset Formula $1\leq j0$ \end_inset , we have \begin_inset Formula $\mu((-\infty,c-\delta))\leq\liminf\mu_{n}((-\infty,c-\delta))\leq\liminf\mu_{n}((-\infty,c])$ \end_inset . \end_layout \begin_layout Standard Since \begin_inset Formula $F$ \end_inset is continuous at \begin_inset Formula $c$ \end_inset , \begin_inset Formula $F(c-2\delta)\uparrow F(c)$ \end_inset as \begin_inset Formula $\delta\rightarrow0$ \end_inset and \begin_inset Formula $F(c-2\delta)\leq\mu((-\infty,c-\delta))\leq F(c)$ \end_inset , so \begin_inset Formula $\mu((-\infty,c-\delta))\uparrow\mu((-\infty,c])$ \end_inset . \end_layout \begin_layout Standard Hence \begin_inset Formula $\mu((-\infty,c])\leq\liminf\mu_{n}((-\infty,c])$ \end_inset . \end_layout \begin_layout Standard Also, \begin_inset Formula $\mu((c,\infty))\leq\liminf\mu_{n}((c,\infty))$ \end_inset so \begin_inset Formula $\mu((-\infty,c])\geq\limsup\mu_{n}((-\infty,c])$ \end_inset . \end_layout \begin_layout Standard Hence \begin_inset Formula $\lim\mu((-\infty,c])=\mu((-\infty,c])$ \end_inset . \begin_inset VSpace defskip \end_inset \end_layout \begin_layout Standard Now the other way. Suppose that \begin_inset Formula $F_{n}\rightarrow F$ \end_inset , and let \begin_inset Formula $A$ \end_inset be an open subset of \begin_inset Formula $\R$ \end_inset . We want to show that \begin_inset Formula $\mu(A)\geq\liminf\mu_{n}(A)$ \end_inset . \end_layout \begin_layout Standard All open subsets of \begin_inset Formula $\R$ \end_inset are a union of countably many disjoint open intervals, so \begin_inset Formula $A=\bigcup_{n}(a_{n},b_{n})$ \end_inset . \end_layout \begin_layout Standard It is enough to show that \begin_inset Formula $\mu((a,b))\leq\liminf\mu_{n}((a,b))$ \end_inset for a single open interval \begin_inset Formula $(a,b)$ \end_inset , as we can then add these up. \end_layout \begin_layout Standard \begin_inset Formula $F$ \end_inset has only countably many points of discontinuity, so there is a sequence \begin_inset Formula $a_{m}\downarrow a$ \end_inset s.t. \begin_inset Formula $F$ \end_inset is continuous at \begin_inset Formula $a_{m}$ \end_inset for all \begin_inset Formula $m$ \end_inset . Then \begin_inset Formula $\limsup F_{n}(a)\leq\limsup F_{n}(a_{m})=F(a_{m})$ \end_inset for all \begin_inset Formula $m$ \end_inset , so by right continuity of \begin_inset Formula $F$ \end_inset , \begin_inset Formula $\limsup F_{n}(a)\leq F(a)$ \end_inset . \end_layout \begin_layout Standard Similarly, there is a sequence \begin_inset Formula $b_{m}\uparrow b$ \end_inset s.t. \begin_inset Formula $b_{m}0\}$ \end_inset , then \begin_inset Formula $\int_{A}fd\mu\geq0$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Let \begin_inset Formula $M_{n}=\max\{0,S_{1},\ldots,S_{n}\}$ \end_inset , and \begin_inset Formula $A_{n}=\{\omega\colon M_{n}>0\}$ \end_inset . Then \begin_inset Formula $M_{n}$ \end_inset is increasing, and \begin_inset Formula $A_{n}\uparrow A$ \end_inset so \begin_inset Formula $fI_{A_{n}}\rightarrow fI_{A}$ \end_inset . And \begin_inset Formula $\vert fI_{A_{n}}\vert\leq\vert f\vert\in L^{1}$ \end_inset so by the Dominated Convergence Theorem, \begin_inset Formula \[ \int_{A_{n}}fd\mu\rightarrow\int_{A}fd\mu\] \end_inset \end_layout \begin_layout Standard Hence we just need to show \begin_inset Formula $\int_{A_{n}}fd\mu\geq0$ \end_inset for each \begin_inset Formula $n$ \end_inset . \end_layout \begin_layout Standard Consider the shifted sums \begin_inset Formula $S_{n}'=S_{n}\circ T=\sum_{k=1}^{n}f(T^{k}\omega)$ \end_inset . \end_layout \begin_layout Standard Let \begin_inset Formula $M_{n}'=M_{n}\circ T=\max\{0,S_{1}',\ldots,S_{n}'\}$ \end_inset and \begin_inset Formula $A_{n}'=T^{-1}(A_{n})=\{\omega\colon M_{n}'>0\}$ \end_inset . \end_layout \begin_layout Standard Now \begin_inset Formula $S_{n}=f+S_{n-1}'$ \end_inset so if \begin_inset Formula $M_{n}>0$ \end_inset then \begin_inset Formula $M_{n}=f+\max\{S_{1}',\ldots,S_{n-1}'\}\leq f+M_{n-1}'$ \end_inset . \end_layout \begin_layout Standard Hence \begin_inset Formula $M_{n}I_{A_{n}}-M_{n-1}'I_{A_{n}}\leq fI_{A_{n}}$ \end_inset . \end_layout \begin_layout Standard But we can control \begin_inset Formula $M_{n-1}'I_{A_{n}}$ \end_inset using \begin_inset Formula $M_{n-1}'I_{A_{n}}\leq M_{n}'I_{A_{n}}\leq M_{n}'I_{A_{n}'}=(M_{n}I_{A_{n}})\circ T$ \end_inset , so \begin_inset Formula \[ \int_{A_{n}}f\geq\int(M_{n}-M_{n-1}')I_{A_{n}}\geq\int M_{n}I_{A_{n}}-\int(M_{n}I_{A_{n}})\circ T=0\] \end_inset (the last equality since \begin_inset Formula $T$ \end_inset is measure-preserving). \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{theorem}[Birkhoff's Ergodic Theorem] \end_layout \end_inset If \begin_inset Formula $T:(\Omega,\cF,\P)\rightarrow(\Omega,\cF,\P)$ \end_inset is measure-preserving, \begin_inset Formula $\cI$ \end_inset is the invariant \begin_inset Formula $\sigma$ \end_inset -field of \begin_inset Formula $T$ \end_inset , and \begin_inset Formula $f\in L^{1}$ \end_inset then \begin_inset Formula \[ \frac{1}{n}\sum_{k=1}^{n}f(T^{k-1}\omega)\rightarrow\E(f|\cI)\mbox{ almost surely.}\] \end_inset \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{theorem} \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash begin{proof} \end_layout \end_inset Assume wlog that \begin_inset Formula $\E(f|\cI)=0$ \end_inset . \end_layout \begin_layout Standard Let \begin_inset Formula $X_{n}=f(T^{n-1}\omega),S_{n}=\sum_{k=1}^{n}X_{k}$ \end_inset . \end_layout \begin_layout Standard Given \begin_inset Formula $\epsilon>0$ \end_inset , let \begin_inset Formula $A=\{\omega\colon\limsup_{n}S_{n}/n>\epsilon\}$ \end_inset . We need to show that \begin_inset Formula $\P(A)=0$ \end_inset . \end_layout \begin_layout Standard Let \begin_inset Formula $f'=(f-\epsilon)I_{A}$ \end_inset , \begin_inset Formula $X_{n}'=(X_{n}-\epsilon)I_{A}$ \end_inset and \begin_inset Formula $S_{n}'=\sum_{k=1}^{n}X_{n}'=(S_{n}-n\epsilon)I_{A}$ \end_inset . \end_layout \begin_layout Standard Now if \begin_inset Formula $A$ \end_inset occurs then \begin_inset Formula $S_{n}>n\epsilon$ \end_inset for some (in fact infinitely many) \begin_inset Formula $n$ \end_inset , so \begin_inset Formula $\sup S_{n}'>0$ \end_inset . \end_layout \begin_layout Standard And if \begin_inset Formula $\sup S_{n}'>0$ \end_inset , then \begin_inset Formula $I_{A}=1$ \end_inset so \begin_inset Formula $A$ \end_inset occurs. \end_layout \begin_layout Standard So \begin_inset Formula $A=\{\omega\colon\sup S_{n}'>0\}$ \end_inset . \end_layout \begin_layout Standard Now for any fixed \begin_inset Formula $\omega$ \end_inset , \begin_inset Formula $X_{1}/n\rightarrow0$ \end_inset as \begin_inset Formula $n\rightarrow\infty$ \end_inset so \begin_inset Formula $\limsup_{n}S_{n}/n=\limsup_{n}S_{n-1}'/n=\limsup_{n}S_{n}'/n$ \end_inset . Hence \begin_inset Formula $A=T^{-1}(A)$ \end_inset and \begin_inset Formula $X_{n}'$ \end_inset is stationary. \end_layout \begin_layout Standard Hence by the maximal ergodic theorem, \begin_inset Formula \[ 0\leq\int_{A}X_{1}'d\P=\int_{A}X_{1}d\P-\epsilon\P(A)\] \end_inset \end_layout \begin_layout Standard Now \begin_inset Formula $A\in\cI$ \end_inset so the definition of conditional probability gives \begin_inset Formula \[ \int_{A}X_{1}d\P=\int_{A}\E(X_{1}|\cI)d\P=0\] \end_inset so we get \begin_inset Formula $\epsilon\P(A)\leq0$ \end_inset giving \begin_inset Formula $\P(A)=0$ \end_inset . \begin_inset ERT status collapsed \begin_layout Plain Layout \backslash end{proof} \end_layout \end_inset \end_layout \end_body \end_document