Convergence of random variables

In probability theory, several different notions of convergence of random variables are investigated. These will be presented here. Throughout, we assume that (X_n) is a sequence of random variables, and X is a random variable, and all of them are defined on the same probability space (Ω, Pr).

We say that the sequence X_n converges towards X in distribution, if

lim_n→∞ Pr(X_n ≤ a) = Pr(X ≤ a)

for every real number a at which the cumulative distribution function of the limiting random variable X is continuous.

Essentially, this means that the likelyhood that the value of X is in a given range is very similar to the likelyhood that the value of X_n is in that range, if only n is large enough. This notion of convergence is used in the central limit theorems.

We say that the sequence X_n converges towards X in probability or weakly if

lim_n→∞Pr(|X_n - X| > ε) = 0

for every ε > 0.

This means that if you pick a tolerance ε and choose n large enough, then the value of X_n will be almost guaranteed to be within that tolerance of the value of X. This notion of convergence is used in the weak law of large numbers.

Convergence in probability implies convergence in distribution.

We say that the sequence X_n converges almost surely or almost everywhere or with probability 1 or strongly towards X if

Pr ( {ω in Ω : lim_n→∞ X_n(ω) = X(ω)} ) = 1

This means that you are virtually guaranteed that the values of X_n approach the value of X. This notion of convergence is used in the strong law of large numbers.

Almost sure convergence implies convergence in probability.

We say that the sequence X_n converges towards X in mean or in the L¹ norm if

lim_n→∞ E(|X_n - X|) = 0

where E denotes the expected value.

This means that the expected difference between X_n and X gets as small as desired if n is chosen big enough. This convergence is considered in L^p spaces (where p = 1).

Convergence in the mean implies convergence in probability. There is no general relation between convergence in mean and almost sure convergence however.

We say that the sequence X_n converges towards X in mean square or in the L² norm if

lim_n→∞ E(|X_n - X|²) = 0.

This means that the expected squared difference between X_n and X gets as small as desired if n is chosen big enough. This convergence is considered in L^p spaces (where p = 2).

Convergence in mean square implies convergence in mean.