=====================
 Joint Distributions
=====================




Two Discrete Distributions
==========================

Consider a random experiment where we observe two random variables $X$
and $Y$. Assume both RV's are discrete. The probability for outcomes
$X=x$ and $Y=y$ is given by the **joint probability mass function
$p_{XY}$

.. math::
   p_{XY}(x,y) = \P(X=x,Y=y)

Here $X=x, Y=y$ denotes $X=x \cap Y=y$.
Again the sum of all possible outcomes of the experiment should be 1:

.. math::
   \sum_{x=-\infty}^{\infty} \sum_{y=-\infty}^{\infty} p_{XY}(x,y) = 1

Note that we don't have to run the summation over all of $\setZ$ in
case we know that $p_{XY}(x,y)=0$ outside a given interval for $x$ and
$y$.

Let's consider an example where $X$ can have the values 1,2 and 3, and
$Y$ can take on the values 1 and 2. Assume we know all probabilities
we can set up the **joint distribution table**:

.. list-table::
   :widths: 1, 1, 2
   :header-rows: 1

   * - $x$
     - $y$
     - $p_{XY}(x,y)$

   * - 1
     - 1
     - 0.10


   * - 1
     - 2
     - 0.20

   * - 2
     - 1
     - 0.20

   * - 2
     - 2
     - 0.25

   * - 3
     - 1
     - 0.15

   * - 3
     - 2
     - 0.10

The joint distribution probability functions (and hence the joint
distribution table) is all there is to know about the random
experiment. So we may also calculate $P(X=x)$ from it:

.. math::
   \P(X=x) = p_X(x) = \sum_{y} p_{XY}(x,y)

We can also calculate:

.. math::
   \P(X=1\given Y=1) &= \frac{\P(X=1,Y=1)}{\P(Y=1)}\\
                     &= \frac{p_{XY}(1,1)}{\sum_x p_{XY}(x,1)}\\
                     &= \frac{0.10}{0.10+0.20+0.15}\\
                     &= \frac{0.10}{0.45}


The indepence of two discrete random variables $X$ and $Y$ is defined as:

.. math::
   \forall x, \forall y : p_{XY}(x,y) = p_X(x)\, p_Y(y)







Two Continuous Distributions
============================

In case $X$ and $Y$ are continuous random variables we have a joint
probability density function $f_{XY}(x,y)$ let $A\subset\setR^2$ then

.. math::
   \P((X,Y)\in A) = \iint_A f_{XY}(x,y)\,dxdy

Note that $f_{XY}$ is a density function in two variables. Therefore
$f_{XY}(x,y)dxdy$ is a probability.

The integral of the pdf over $\setR^2$ should equal 1 (because the
universe in this case is $\setR^2$:

.. math::
   \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f_{XY}(x,y)dx dy = 1


Like for the discrete counterpart we can calculate $f_X$ from $f_{XY}$:

.. math::
   f_X(x) = \int_{-\infty}^{\infty} f_{XY}(x,y)dy


Independent continuous random variables $X$ and $Y$ are defined with:

.. math::
   \forall x, \forall y : f_{XY}(x,y) = f_X(x)\,f_Y(y)



Multiple Random Variables
=========================

Now we consider the case that we are observing $n$ random variables
from one random experiment. In case that $X_1,X_2,\ldots,X_n$ are all
discrete RV's we have:

.. math::
   p_{X_1 X_2\cdots X_n}(x_1,\ldots,x_n) = \P(X_1=x_1,\ldots,X_n=x_n)


For continuous RV's we have:

.. math::
   \P((X_1,\ldots,X_n)\in A) = \int\cdots\int_A f_{X_1 X_2\cdots X_n}(x_1,\ldots,x_n) dx_1 dx_2\cdots dx_n