In my classes, I use one "simple" situation that might help you wonder and perhaps develop a gut feeling for what a degree of freedom may mean.

It is kind of a "Forrest Gump" approach to the subject, but it is worth the try.

Consider you have 10 independent observations $X_1, X_2, \ldots, X_{10}\sim N(\mu,\sigma^2)$ that came right from a normal population whose mean $\mu$ and variance $\sigma^2$ are unknown.

Your observations bring to you collectively information both about $\mu$ and $\sigma^2$. After all, your observations tend to be spread around one central value, which ought to be close to the actual and unknown value of $\mu$ and, likewise, if $\mu$ is very high or very low, then you can expect to see your observations gather around a very high or very low value respectively. One good "substitute" for $\mu$ (in the absence of knowledge of its actual value) is $\bar X$, the average of your observation.

Also, if your observations are very close to one another, that is an indication that you can expect that $\sigma^2$ must be small and, likewise, if $\sigma^2$ is very large, then you can expect to see wildly different values for $X_1$ to $X_{10}$.

If you were to bet your week's wage on which should be the actual values of $\mu$ and $\sigma^2$, you would need to *choose* a pair of values in which you would bet your money. Let's not think of anything as dramatic as losing your paycheck unless you guess $\mu$ correctly until its 200th decimal position. Nope. Let's think of some sort of prizing system that the closer you guess $\mu$ and $\sigma^2$ the more you get rewarded.

In some sense, your better, more informed, and more polite guess for $\mu$'s value could be $\bar X$. In that sense, you *estimate* that $\mu$ must be some value around $\bar X$. Similarly, one good "substitute" for $\sigma^2$ (not required for now) is $S^2$, your sample variance, which makes a good estimate for $\sigma$.

If your were to believe that those substitutes are the actual values of $\mu$ and $\sigma 2$, you would probably be wrong, because very slim are the chances that you were so lucky that your observations coordinated themselves to get you the gift of $\bar X$ being equal to $\mu$ and $S^2$ equal to $\sigma^2$. Nah, probably it didn't happen.

But you could be at different levels of wrong, varying from a bit wrong to really, really, **really** miserably wrong (a.k.a., "Bye-bye, paycheck; see you next week!").

Ok, let's say that you took $\bar X$ as your guess for $\mu$. Consider just two scenarios: $S^2=2$ and $S^2=20,000,000$. In the first, your observations sit pretty and close to one another. In the latter, your observations vary wildly. In which scenario you should be more concerned with your potential losses? If you thought of the second one, you're right. Having a estimate about $\sigma^2$ changes your confidence on your bet very reasonably, for the larger $\sigma^2$ is, the wider you can expect $\bar X$ to variate.

But, beyond information about $\mu$ and $\sigma^2$, your observations also carry some amount of just pure random fluctuation that is not informative neither about $\mu$ nor about $\sigma^2$.

How can you notice it?

Well, let's assume, for sake of argument, that there is a God and that He has spare time enough to give Himself the frivolity of telling you specifically the real (and so far unknown) values of both $\mu$ and $\sigma$.

And here is the annoying plot twist of this lysergic tale: He tells it to you **after** you placed your bet. Perhaps to enlighten you, perhaps to prepare you, perhaps to mock you. How could you know?

Well, that makes the information about $\mu$ and $\sigma^2$ contained in your observations quite useless now. Your observations' central position $\bar X$ and variance $S^2$ are no longer of any help to get closer to the actual values of $\mu$ and $\sigma^2$, for you already know them.

One of the benefits of your good acquaintance with God is that you actually know by how much you failed to guess correctly $\mu$ by using $\bar X$, that is, $(\bar X - \mu)$ your estimation error.

Well, since $X_i\sim N(\mu,\sigma^2)$, then $\bar X\sim N(\mu,\sigma^2/10)$ (trust me in that if you will), also $(\bar X - \mu)\sim N(0,\sigma^2/10)$ (ok, trust me in that on too) and, finally,
$$
\frac{\bar X - \mu}{\sigma/\sqrt{10}} \sim N(0,1)
$$
(guess what? trust me in that one as well), which carries absolutely no information about $\mu$ or $\sigma^2$.

You know what? If you took any of your individual observations as a guess for $\mu$, your estimation error $(X_i-\mu)$ would be distributed as $N(0,\sigma^2)$. Well, between estimating $\mu$ with $\bar X$ and any $X_i$, choosing $\bar X$ would be better business, because $Var(\bar X) = \sigma^2/10 < \sigma^2 = Var(X_i)$, so $\bar X$ was less prone to be astray from $\mu$ than an individual $X_i$.

Anyway, $(X_i-\mu)/\sigma\sim N(0,1)$ is also absolutely non informative about neither $\mu$ nor $\sigma^2$.

"Will this tale ever end?" you may be thinking. You also may be thinking "Is there any more random fluctuation that is non informative about $\mu$ and $\sigma^2$?".

[I prefer to think that you are thinking of the latter.]

Yes, there is!

The square of your estimation error for $\mu$ with $X_i$ divided by $\sigma$,
$$
\frac{(X_i-\mu)^2}{\sigma^2}
= \left(\frac{X_i-\mu}{\sigma}\right)^2
\sim \chi^2
$$
has a Chi-squared distribution, which is the distribution of the square $Z^2$ of a standard Normal $Z\sim N(0,1)$, which I am sure you noticed has absolutely no information about either $\mu$ nor $\sigma^2$, but conveys information about the variability you should expect to face.

That is a very well known distribution that arises naturally from the very scenario of you gambling problem for every single one of your ten observations and also from your mean:
$$
\frac{(\bar X-\mu)^2}{\sigma^2/10}
= \left(\frac{\bar X-\mu}{\sigma/\sqrt{10}}\right)^2
= \left(N(0,1)\right)^2
\sim\chi^2
$$
and also from the gathering of your ten observations' variation:
$$
\sum_{i=1}^{10} \frac{(X_i-\mu)^2}{\sigma^2/10}
=\sum_{i=1}^{10} \left(\frac{X_i-\mu}{\sigma/\sqrt{10}}\right)^2
=\sum_{i=1}^{10} \left(N(0,1)\right)^2
=\sum_{i=1}^{10} \chi^2.
$$
Now that last guy doesn't have a Chi-squared distribution, because he is the sum of ten of those Chi-squared distributions, all of them independent from one another (because so are $X_1, \ldots, X_{10}$). Each one of those single Chi-squared distribution is one contribution to the amount of random variability you should expect to face, with roughly the same amount of contribution to the sum.

The value of each contribution is not mathematically equal to the other nine, but all of them have the same expected behavior in distribution. In that sense, they are somehow symmetric.

Each one of those Chi-square is one contribution to the amount of pure, random variability you should expect in that sum.

If you had 100 observations, the sum above would be expected to be bigger **just because it have more sources of contibutions**.

Each of those "sources of contributions" with the same behavior can be called **degree of freedom**.

Now take one or two steps back, re-read the previous paragraphs if needed to accommodate the sudden arrival of your quested-for *degree of freedom*.

Yep, each degree of freedom can be thought of as one unit of variability that is obligatorily expected to occur and that brings nothing to the improvement of guessing of $\mu$ or $\sigma^2$.

The thing is, you start to count on the behavior of those 10 equivalent sources of variability. If you had 100 observations, you would have 100 independent equally-behaved sources of strictly random fluctuation to that sum.

That sum of 10 Chi-squares gets called a **Chi-squared distributions with ***10* degrees of freedom from now on, and written $\chi^2_{10}$. We can describe what to expect from it starting from its probability density function, that can be mathematically derived from the density from that single Chi-squared distribution (from now on called **Chi-squared distribution with ***one* degree of freedom and written $\chi^2_1$), that can be mathematically derived from the density of the normal distribution.

"So what?" --- you might be thinking --- "That is of any good only if God took the time to tell me the values of $\mu$ and $\sigma^2$, of all the things He could tell me!"

Indeed, if God Almighty were too busy to tell you the values of $\mu$ and $\sigma^2$, you would still have that 10 sources, that 10 degrees of freedom.

Things start to get weird (Hahahaha; only now!) when you rebel against God and try and get along all by yourself, without expecting Him to patronize you.

You have $\bar X$ and $S^2$, estimators for $\mu$ and $\sigma^2$. You can find your way to a safer bet.

You could consider calculating the sum above with $\bar X$ and $S^2$ in the places of $\mu$ and $\sigma^2$:
$$
\sum_{i=1}^{10} \frac{(X_i-\bar X)^2}{S^2/10}
=\sum_{i=1}^{10} \left(\frac{X_i-\bar X}{S/\sqrt{10}}\right)^2,
$$
but that is not the same as the original sum.

"Why not?" The term inside the square of both sums are very different. For instance, it is unlikely but possible that all your observations end up being larger than $\mu$, in which case $(X_i-\mu) > 0$, which implies $\sum_{i=1}^{10}(X_i-\mu) > 0$, but, by its turn, $\sum_{i=1}^{10}(X_i-\bar X) = 0$, because $\sum_{i=1}^{10}X_i-10 \bar X =10 \bar X - 10 \bar X = 0$.

Worse, you can prove easily (Hahahaha; right!) that $\sum_{i=1}^{10}(X_i-\bar X)^2 \le \sum_{i=1}^{10}(X_i-\mu)^2$ with strict inequality when at least two observations are different (which is not unusual).

"But wait! There's more!"
$$
\frac{X_i-\bar X}{S/\sqrt{10}}
$$
doesn't have standard normal distribution,
$$
\frac{(X_i-\bar X)^2}{S^2/10}
$$
doesn't have Chi-squared distribution with one degree of freedom,
$$
\sum_{i=1}^{10} \frac{(X_i-\bar X)^2}{S^2/10}
$$
doesn't have Chi-squared distribution with 10 degrees of freedom
$$
\frac{\bar X-\mu}{S/\sqrt{10}}
$$
doesn't have standard normal distribution.

"Was it all for nothing?"

No way. Now comes the magic! Note that
$$
\sum_{i=1}^{10} \frac{(X_i-\bar X)^2}{\sigma^2}
=\sum_{i=1}^{10} \frac{[X_i-\mu+\mu-\bar X]^2}{\sigma^2}
=\sum_{i=1}^{10} \frac{[(X_i-\mu)-(\bar X-\mu)]^2}{\sigma^2}
=\sum_{i=1}^{10} \frac{(X_i-\mu)^2-2(X_i-\mu)(\bar X-\mu)+(\bar X-\mu)^2}{\sigma^2}
=\sum_{i=1}^{10} \frac{(X_i-\mu)^2-(\bar X-\mu)^2}{\sigma^2}
=\sum_{i=1}^{10} \frac{(X_i-\mu)^2}{\sigma^2}-\sum_{i=1}^{10} \frac{(\bar X-\mu)^2}{\sigma^2}
=\sum_{i=1}^{10} \frac{(X_i-\mu)^2}{\sigma^2}-10\frac{(\bar X-\mu)^2}{\sigma^2}
=\sum_{i=1}^{10} \frac{(X_i-\mu)^2}{\sigma^2}-\frac{(\bar X-\mu)^2}{\sigma^2/10}
$$
or, equivalently,
$$
\sum_{i=1}^{10} \frac{(X_i-\mu)^2}{\sigma^2}
=\sum_{i=1}^{10} \frac{(X_i-\bar X)^2}{\sigma^2}
+\frac{(\bar X-\mu)^2}{\sigma^2/10}.
$$
Now we get back to those known faces.

The first term has Chi-squared distribution with 10 degrees of freedom and the last term has Chi-squared distribution with one degree of freedom(!).

We simply split a Chi-square with 10 independent equally-behaved sources of variability in two parts, both positive: one part is a Chi-square with one source of variability and the other we can prove (leap of faith? win by W.O.?) to be also a Chi-square with 9 (= 10-1) independent equally-behaved sources of variability, with both parts independent from one another.

This is already a good news, since now we have its distribution.

Alas, it uses $\sigma^2$, to which we have no access (recall that God is amusing Himself on watching our struggle).

Well,
$$
S^2=\frac{1}{10-1}\sum_{i=1}^{10} (X_i-\bar X)^2,
$$
so
$$
\sum_{i=1}^{10} \frac{(X_i-\bar X)^2}{\sigma^2}
=\frac{\sum_{i=1}^{10} (X_i-\bar X)^2}{\sigma^2}
=\frac{(10-1)S^2}{\sigma^2}
\sim\chi^2_{(10-1)}
$$
therefore
$$
\frac{\bar X-\mu}{S/\sqrt{10}}
=\frac{\frac{\bar X-\mu}{\sigma/\sqrt{10}}}{\frac{S}{\sigma}}
=\frac{\frac{\bar X-\mu}{\sigma/\sqrt{10}}}{\sqrt{\frac{S^2}{\sigma^2}}}
=\frac{\frac{\bar X-\mu}{\sigma/\sqrt{10}}}{\sqrt{\frac{\frac{(10-1)S^2}{\sigma^2}}{(10-1)}}}
=\frac{N(0,1)}{\sqrt{\frac{\chi^2_{(10-1)}}{(10-1)}}},
$$
which is a distribution that is not the standard normal, but whose density can be derived from the densities of the standard normal and the Chi-squared with $(10-1)$ degrees of freedom.

One very, very smart guy did that math[^1] in the beginning of 20th century and, as an unintended consequence, he made his boss the absolute world leader in the industry of Stout beer. I am talking about William Sealy Gosset (a.k.a. Student; yes, *that* Student, from the $t$ distribution) and Saint James's Gate Brewery (a.k.a. Guinness Brewery), of which I am a devout.

[^1]: @whuber told in the comments below that Gosset did not do the math, but *guessed* instead! I really don't know which feat is more surprising for that time.

That, my dear friend, is the origin of the $t$ distribution with $(10-1)$ degrees of freedom. The ratio of a standard normal and the squared root of an independent Chi-square divided by its degrees of freedom, which, in an unpredictable turn of tides, wind up describing the expected behavior of the estimation error you undergo when using the sample average $\bar X$ to estimate $\mu$ and using $S^2$ to estimate the variability of $\bar X$.

There you go. With an awful lot of technical details grossly swept behind the rug, but not depending solely on God's intervention to dangerously bet your whole paycheck.

3

Also see this question "What are degrees of freedom?"

– Jeromy Anglim – 2011-10-12T22:12:59.437Check out this explanation

– George Dontas – 2010-07-28T10:26:32.273