Probability Space and Probability Axioms

Random Experiments and Probability Spaces

To describe a random experiment in mathematical terms we need what is called a probability space. Such a space consists of three things:

  1. The set of all possible outcomes of the random experiment. We will denote this set as \(U\) for universe.
  2. A collection of subsets of \(U\). A subspace \(A\subset U\) corresponds with an event. For example when throwing a dice a possible event is to throw a 6. Another possibiliy is the event corresponding with throwing an odd number of points.
  3. A probability measure \(\P\) that maps each subset \(A\subset U\) (event) onto a positive scalar \(\P(A)\)

Probability Axioms

Let \(A\) and \(B\) both be subsets of the probability space (the universum) \(U\), i.e. \(A\) and \(B\) are events. Surprisingly maybe, the whole theory of probability is based upon just three axioms:

\[\begin{split}(A_1)&\quad \P(A)\geq 0\\ (A_2)&\quad \P(U)=1\\ (A_3)&\quad A\cap B=\emptyset \Rightarrow \P(A\cup B) = \P(A) + \P(B)\end{split}\]

The first axiom simply states that probabilities are non negative (real) numbers. The second axioma states that the probability for the universe \(U\) equals 1. The third axiom is the only axiom that relates probabilities of different events: if event \(A\) and event \(B\) are mutually exclusive (i.e. you can’t have \(A\) and \(B\) being the simulatanuous outcome of one random experiment) then the sum of either \(A\) or event \(B\) occuring equals the sum of \(\P(A)\) and \(\P(B)\).

Venn Diagrams

To gain insight and some intuition of probability space it is useful to depict that space as a two dimensional subset of the plane (the Venn diagrams of sets).

Consider the Venn diagram corresponding with the third axiom. We have the universe \(U\) and the two events \(A\) and \(B\), both subset of \(U\). The assumption that \(A\cap B =\emptyset\) follows is visualised as the empty intersection of the two sets.

You may take the visualization somewhat further and assume the relative area of set \(A\) with respect to the area of set \(U\) corresponds with the probability \(\P(A)\):

\[\P(A) = \frac{\text{area}(A)}{\text{area}(U)}\]

Please note that this is NOT the definition of \(\P(A)\) ! It is stating the rule to interpret the Venn diagrams.

Simple Probability Theorems

The definitions and axioms in a mathematical theory are the starting point, but the interesting parts are the theorems, the things that can be proven to be true given the definitions and the axioms. Probability theory is no exception.

Let us start with a simple definition: the complement. Let \(A\) be an event, i.e. subset of \(U\). Then we define the complement \(\neg A\) as:

\[\neg A = U \setminus A\]

where \(\setminus\) denotes the ‘set minus’, i.e. everything in the universe that is not in \(A\). This is just a definition. A theorem however is the statement:

\[\P(\neg A) = 1 - \P(A)\]

The proof obviously needs the third axiom as we have to relate the probabilities of two events: \(A\) and \(\neg A\). Observe that:

\[U = A \cup \neg A\]

By definition we have that \(A\) and \(\neg A\) are disjunct, i.e. \(A\cap \neg A=\emptyset\), and thus we case utilize the third axiom:

\[\P(U) = \P(A) + \P(\neg A)\]

Using the second axiom and rewriting the equation we get:

\[\P(\neg A) = 1 - \P(A)\]

and that completes the proof.

Now consider the situation where events \(A\) and \(B\) are not disjunct. Then we cannot apply the third axiom directly. Look at a Venn diagram of two subsets \(A\) and \(B\) or \(U\) with non empty intersection. From such a figure it is intuively clear that

\[\P(A\cup B) = \P(A) + \P(B) - \P(A\cap B).\]

The proof is simple as well. We leave that to an exercise.