Glossary

Select one of the keywords on the left…

ProbabilityConditional Expectation

Reading time: ~10 min

The conditional expectation of Y given \{X=x\} is defined to be the expectation of Y calculated with respect to its conditional distribution given \{X=x\}. For example, if X and Y are continuous random variables, then

\begin{align*}E[Y | X = x] = \int_{-\infty}^{\infty}y f_{Y | \{X = x\}} (y) \, \mathrm{d} y.\end{align*}

Example
Suppose that f is the function which returns 2 for any point in the triangle with vertices (0,0), (1,0), and (0,1) and otherwise returns 0. If (X,Y) has joint pdf f, then the conditional density of Y given \{X = x\} is the mean of the uniform distribution on the segment [x,1], which is \frac{1+x}{2}.

The conditional variance of Y given \{X = x\} is defined to be the variance of Y with respect to its conditional distribution of Y given \{X=x\}.

Example
Continuing with the example above, the conditional density of Y given \{X = x\} is the variance of the uniform distribution on the segment [x,1], which is \frac{(1-x)^2}{12}.

We can regard the conditional expectation of Y given X as a random variable, denoted \mathbb{E}[Y | X] by coming up with a formula for \mathbb{E}[Y | \{X = x\}] for each x \in \mathbb{R}, and then substituting X for x. And likewise for conditional variance.

Example
With Y and X as defined above, we have \mathbb{E}[Y | X] = \frac{1+X}{2} and \operatorname{Var}[Y | X] = \frac{(1-X)^2}{12}.

Exercise
Find the conditional expectation of Y given X where the pair (X,Y) has density x + y on [0,1]^2.

Solution. We calculate the conditional density as

\begin{align*}\frac{f_{X,Y}(x,y)}{f_X(x)} = \frac{x + y}{x + \frac{1}{2}},\end{align*}

which means that

\begin{align*}\mathbb{E}[Y | X = x] = \int_0^1 \frac{y(x+y)}{x + \frac{1}{2}} \, \mathrm{d} x = \frac{3x+2}{6(x+\frac{1}{2})}.\end{align*}

So \mathbb{E}[Y | X] = \frac{3X+2}{6(X+\frac{1}{2})}

The tower law

Conditional expectation can be helpful for calculating expectations, because of the tower law.

Theorem (Tower law of conditional expectation)
If X and Y are random variables defined on a probability space, then

\begin{align*}\mathbb{E}[\mathbb{E}[Y | X]] = \mathbb{E}[Y].\end{align*}

Exercise
Consider a particle which splits into two particles with probability p \in (0,1) at time t=1. At time t = 2, each extant particle splits into two particles independently with probability p.

Find the expected number of particles extant just after time t = 2. Hint: define X to be 1 or 0 depending on whether the particle splits at time t = 1, and use the tower law with X.

Solution. If Y is the number of particles and X is the indicator of the event that the particle split at time 1, then

\begin{align*}\mathbb{E}[Y | \{X = 0\}] = 2(p) + 1(1-p) = 1+p\end{align*}

while

\begin{align*}\mathbb{E}[Y | \{X = 1\}] = 2(1+p) = 2 + 2p.\end{align*}

Therefore, \mathbb{E}[Y | \{X = x\}] = (1+p)(1+X). By the tower law, we have

\begin{align*}\mathbb{E}[Y] = \mathbb{E}[\mathbb{E}[Y | X]] = (1+p)(1+\mathbb{E}[X]) = (1+p)^2.\end{align*}

Bruno
Bruno Bruno