The Central Limit Theorem Mathematical Proof

The central limit theorem says that the sample means of a any probability distribution with the sample size large enough, the sample means follow a normal distribution. This is useful in a way that, in real life, we don't certainly know what distribution things follow, and we may use the central limit theorem to model real life random events as long as we take big enough samples.

Moment Generating Function(MGF)

Moment generating function is a way to describe a probability distribution just like a probability density function(PDF) or a cumulative distribution function(CDF) does.

It is intuitive that two probability distributions are the same if their PDFs are the same. 

What if their MGFs are the same? 

Definition

An MGF is defined as Mx(t) = E[etx], and if we write out its Taylor series expansion, we get 
Therefore,
If we plug 0 into t, we get Mx(0) = E[e0] = 1.
If we take the first derivative of Mx(t) with respect to t, we get
Then if we plug 0 into t, we get E[x].

So, we can get the nth moment of a probability distribution by taking its nth derivative with respect to t and plug 0 into t.

Uniqueness

If two probability distributions have the same MGFs, they have same moments of all order. 

For example. two exact probability distributions have the same mean and the same variance, mean being E[x] and the variance being E[x2] - E[x]2, and their first and second moments must be the same.

On the other hand, if two probability distributions are not the same, there must exist a moment of order n being different.

Central limit theorem proof

To proof the central limit theorem, we need one property of MGF.
If x and y are independent:
Mx+y(t) = E[et(x+y)] = E[etxety] = E[etx] E[ety] = Mx(t) My(t) 

Proof

Suppose we take samples of size n form a probability distribution and  treat the sample means as random variable, we can prove that the sample means have a MGF of a normal distribution.

Let a random sample be x1 + x2 + x3 +...+ xn and its mean be S
         S = (x1 + x2 + x3 +...+ xn) / n

The mathematical derivation is as the following:
Suggested reading: Application of the Central Limit Theorem: Modeling Normal Distribution Using javascript

Comments

Post a Comment

Popular posts from this blog

How to Make A Reusable Image Slideshow HTML Component With Vanilla JavaScript

Image
The image slide show I am making has no animation. The mechanism for displaying different slide is by using the display CSS property. The image with display none will not show, while the one with display block will. I will start build one that relys on HTML code then I will build one that relys on JavaScipt that renders the slide show onto the webpage. HTML Slideshow Setup A HTML image slide show can follow the following logic: An image is shown and the others are hidden, by defining the CSS display property to "block" to show and "none" to hide. An index is used to control the CSS display property of each slide with the initial index of 1. On initiation, all image tags for the slide show will be given the CSS property correspoding to whether it's shown or not. A left arrow and a right arrow are used to control the index with onclick event handlers. In order to position the left and right control...
Read more

HTML Tags and Inline CSS that Work In Plotly.js Title

When we create a plot object with Plotly.js, we throw in 3 basic parameters in the Plotly object which are the plot container id, data and layout, like this one: Plotly.newPlot('myDiv', data, layout); And in the layont object, there is a property, called title. The title property can be defined with an string, which will simply be just the title text that is going to display on the top of the plot.  I want to style my chart title when I use Plotly.js.; var layout = { title:`Plot Title` } Or in the text object within the title property. var layout = { title:{text:`Plot Title`} } Plot;y'js has some built-in title properties which we can use to style the title, sush as font color, family, size, padding, and position with x or y. var layout = { title: { text:'Plot Title', font: { family: 'Courier New, monospace', size: 24 }, xref: 'paper', x: 0.05, } }; Problems But I want to add a HTML tag...
Read more

How to Type Spaces In HTML Input And Display In the Browser

Image
When we type a space in an input and use it in JavaScript. The space will be recorded as text just like regular texts. Type In Multiple Spaces When we type some text with many spaces and displays the text in the browser, muiltple spaces will be parsed into one space, if they are connected. Type some text with spaces here: <input oninput="show_1.innerHTML = this.value" /><br /> <div id="show_1"></div> What does JavaScript Do to the Spaces? JavaScript does nothing to the spaces. It preservs all the spaces we type. The following code converts spaces into the letter "a". Space is represnted by \x20 in JavaScript. Type some text with spaces here: <input oninput="show_2.innerHTML = this.value.replace(/\x20/g,'a');" /><br /> <div id="show_2"></div> Why doesn't the Spaces Show in the Browser? Because the connected spaces are parsed into one space in HTML. ...
Read more