2.8. Calculations with Random Variables

2.8.1. Scaled and Shifted Distributions

We have seen simple calculations with continuous random variables before where we defined \(Y=aX+b\) where \(X\) is a random variable and \(a\) and \(b\) are constant values. When discussing the expectation and variance of random variables we have shown that

\[\begin{split}&\E(aX+b) = a\E(X)+b\\ &\Var(aX+b) = a^2\Var(X)\end{split}\]

Evidently \(Y\) is a random variable itself. In general when you make calculations involving random variables the result will be a random variable.

But what about the distribution of \(Y\). For this operation on \(X\) the distribution of \(Y\) can be expressed as a scaled and shifted version of the distribution of \(X\).

We start with \(Y=aX+b\) where \(a>0\). The cumulative distribution function \(F_Y\) then is:

\[\begin{split}\begin{align} F_Y(y) &= P(Y\leq y) \quad && \text{by definition}\\ &= P(aX+b\leq y) && \text{substituting } Y=aX+b\\ &= P(aX \leq y - b )&& \text{rearranging terms}\\ &= P(X \leq \frac{y-b}{a} ) && \text{because }a>0\\ &= F_X( \frac{y-b}{a} ) && \text{by definition} \end{align}\end{split}\]

so \(F_Y\) is a translated and horizontally scaled version of the distribution \(F_X\). Using the fact that \(f_X\) is the derivative of \(F_X\) we have:

\[\begin{split}f_Y(y) &= \frac{d}{dy}F_y(y)\\ &= \frac{d}{dy} F_X(\frac{y-b}{a})\\ &= F^\prime_X(\frac{y-b}{a}) \frac{1}{a}\\ &= \frac{1}{a}f_X(\frac{y-b}{a})\end{split}\]

Again we have the horizontal shift and horizontal scaling but now also a vertical scaling (that will keep the area under the curve equal to 1).

Such scaled and shifted versions of a pdf are very common. For instance in case you need a uniformly distributed number in the range from \(-\pi/6\) to \(\pi/6\) but you only have a random number generator \(X\sim\Uniform(0,1)\) every programmer intuively will write code defining the random variable \(Y\):

\[Y = \frac{\pi}{6}(2 X - 1)\]

Scaling and shifting of the normal distribution is of great practical importance as well. Given a random variable \(X\sim\Normal(\mu,\sigma^2)\) we have:

\[Z = \frac{X-\mu}{\sigma} \sim \Normal(0,1)\]

The random variable \(Z\) is often called the Z-score. It transforms any normally distributed value into a standard normal distributed value.

In the old pre computer days only the standard normal distribution was available in tables and scaling and shifting was needed to obtain the values for any normal distribution.

Whereas the Z-score takes a \(\Normal(\mu, \sigma^2)\) distribution and transforms it into a standard normal distribution \(\Normal(0,1)\) we can go the opposite way as well. Let \(X\sim\Normal(0,1)\) then

\[Y = \sigma X + \mu \sim \Normal(\mu,\sigma^2)\]

2.8.2. The Sum of Random Variables

Consider the random variables \(X\) and \(Y\) and their sum \(Z=X+Y\). First consider the expectation:

\[\E(Z) = \E(X+Y) = \E(X) + \E(Y)\]

To prove this we have to start with the joint probability mass or density function. Here we assume both \(X\) and \(Y\) are discrete RV’s. Then:

\[\begin{split}\E(Z) = \E(X+Y) &= \sum_x \sum_y (x+y)\, p_{XY}(x,y)\\ &= \sum_x \sum_y x\, p_{XY}(x,y) + \sum_x \sum_y y\, p_{XY}(x,y)\\ &= \sum_x x \sum_y p_{XY}(x,y) + \sum_y y \sum_x p_{XY}(x,y)\\ &= \sum_x x\, p_X(x) + \sum_y y\, p_Y(y)\\ &= \E(X) + \E(Y)\end{split}\]

For the variance of \(Z\) things are a bit more complicated:

\[\Var(X+Y) = \Var(X) + \Var(Y) + 2\Cov(X,Y)\]

where

\[\Cov(X,Y) = \E\left( (X-\E(X)) (Y-\E(Y) \right)\]

is called the covariance of \(X\) and \(Y\). Covariance and independence are related but certainly not equal. We have that if \(X\) and \(Y\) are independent then \(\Cov(X,Y)=0\). Please note that a zero covariance (we also say then that \(X\) and \(Y\) are uncorrelated) does not imply that \(X\) and \(Y\) are independent.