Joint Distributions

Two Discrete Distributions

Consider a random experiment where we observe two random variables \(X\) and \(Y\). Assume both RV’s are discrete. The probability for outcomes \(X=x\) and \(Y=y\) is given by the **joint probability mass function \(p_{XY}\)

\[p_{XY}(x,y) = \P(X=x,Y=y)\]

Again the sum of all possible outcomes of the experiment should be 1:

\[\sum_{x=-\infty}^{\infty} \sum_{y=-\infty}^{\infty} p_{XY}(x,y) = 1\]

Note that we don’t have to run the summation over all of \(\setZ\) in case we know that \(p_{XY}(x,y)=0\) outside a given interval for \(x\) and \(y\).

Let’s consider an example where \(X\) can have the values 1,2 and 3, and \(Y\) can take on the values 1 and 2. Assume we know all probabilities we can set up the joint distribution table:

\(x\)	\(y\)	\(p_{XY}(x,y)\)
1	1	0.10
1	2	0.20
2	1	0.20
2	2	0.25
3	1	0.15
3	2	0.10

The joint distribution probability functions (and hence the joint distribution table) is all there is to know about the random experiment. So we may also calculate \(P(X=x)\) from it:

\[\P(X=x) = p_X(x) = \sum_{y} p_{XY}(x,y)\]

We can also calculate:

\[\begin{split}\P(X=1\given Y=1) &= \frac{\P(X=1,Y=1)}{\P(Y=1)}\\ &= \frac{p_{XY}(1,1)}{\sum_x p_{XY}(x,1)}\\ &= \frac{0.10}{0.10+0.20+0.15}\\ &= \frac{0.10}{0.45}\end{split}\]

In case \(X\) and \(Y\) are independent we have:

\[\forall x, \forall y : p_{XY}(x,y) = p_X(x)\, p_Y(y)\]

Two Continuous Distributions

In case \(X\) and \(Y\) are continuous random variables we have a joint probability density function \(f_{XY}(x,y)\) let \(A\subset\setR^2\) then

\[\P((X,Y)\in A) = \iint_A f_{XY}(x,y)\,dxdy\]

Note that \(f_{XY}\) is a density function in two variables. Therefore \(f_{XY}(x,y)dxdy\) is a probability.

The integral of the pdf over \(\setR^2\) should equal 1 (because the universe in this case is \(\setR^2\):

\[\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f_{XY}(x,y)dx dy = 1\]

Like for the discrete counterpart we can calculate \(f_X\) from \(f_{XY}\):

\[f_X(x) = \int_{-\infty}^{\infty} f_{XY}(x,y)dy\]

In case \(X\) and \(Y\) are independent we have:

\[\forall x, \forall y : f_{XY}(x,y) = f_X(x)\,f_Y(y)\]

Multiple Random Variables

Now we consider the case that we are observing \(n\) random variables from one random experiment. In case that \(X_1,X_2,\ldots,X_n\) are all discrete RV’s we have:

\[p_{X_1 X_2\cdots X_n}(x_1,\ldots,x_n) = \P(X_1=x_1,\ldots,X_n=x_n)\]

For continuous RV’s we have:

\[\P((X_1,\ldots,X_n)\in A) = \int\cdots\int_A f_{X_1 X_2\cdots X_n}(x_1,\ldots,x_n) dx_1 dx_2\cdots dx_n\]