Central Limit Theorem

EC484_Econometrics_Analysis

Like Law of Large Numbers, it is also a tool for asymptotic analysis.

The core idea of this theorem is that When you add together a large number of random variables, regardless of their individual distributions, their sum (or mean) will tend towards a normal distribution.

Formal Statement

Suppose we have an i.i.d random variables: $X_1,X_2,\dots ,X_n$

The sample mean is defined as:

$${\bar{X}}_n=\frac{1}{n}\sum _{i=1}^n{X}_i$$

The expected value of each $X_i$ is $\mathbb{E}[X_i]=\mu$, with expected Variance $\text{Var}(X_i)={\sigma }^2<\infty$.

From Law of Large Numbers, we know that as $n\to \infty$, ${\bar{X}}_n{\to }^p\mu$.

We the normalize it:

$$Z_n=\frac{\sqrt{n}({\bar{X}}_n-\mu )}{\sigma }$$

And we would know that:

$$Z_n{\to }^{d}N(0,1)$$

Why $\sqrt{n}$?

We already know that since

$${\bar{z}}_n=\frac{1}{n}\sum _{i=1}^n{z}_i$$

The Variance of ${\bar{z}}_n$ is:

$$\begin{array}{rl}\text{Var}({\bar{z}}_n)& =\text{Var}\left(\frac{1}{n}\sum _{i=1}^n{z}_i\right)\\ & =\frac{1}{n^2}\text{Var}\left(\sum _{i=1}^n{z}_i\right)\\ & =\frac{1}{n^2}\sum _{i=1}^n(\text{Var}(z_i))(\text{becuase it is i.i.d.})\\ & =\frac{1}{n^2}n{\sigma }^2\\ & =\frac{{\sigma }^2}{n}\end{array}$$

By Weak Law of Large Numbers, we know that ${\bar{z}}_n{\to }^p\mu$, the distribution would be more and more concentrated around $\mu$ as $n$ increases, eventually a degenerate distribution at $\mu$.

To avoid this, we multiply by $\sqrt{n}$, thus the variance would be:

$$\text{Var}(\sqrt{n}{\bar{z}}_n)=n\cdot \frac{{\sigma }^2}{n}={\sigma }^2$$