I must confess I'm not an educator, but I like this question and at the very least I can answer with the intuitive picture I use in my own head.
The $n$-th central moment $\mu_n$ of a random variable $X$ is defined by:
$$\mu_n \equiv E[(X-E[X])^n]$$
For what follows we will take the measure space of $X$ to be the real numbers, since that's the easiest to visualize and the most common thing to deal with in statistics. Now, with this knowledge we can rewrite the moment as
$$\mu_n = \int_{-\infty}^{\infty} (x-E[X])^nf_X(x)dx$$
where $f_X$ is the pdf of $X$. To help complete the mental image, it is important to keep in mind that $E[X]$ is a constant and $f_X$ is a normalized function that is non-negative. So, our integrand looks like some single term polynomial centered at the mean of $X$, multiplying the non-negative pdf of $X$. This is a lot clearer once you see what I'm talking about, so I will show some photos and explain.
Second Moment (and even moments in general):
Here is a picture illustrating the integral I was talking about for the second moment using some pdf I made up. The dotted line is the pdf, the dashed line is the polynomial term, and the solid line is the total integrand:
Keep in mind that the moment is defined as the integral of the solid curve. There's a couple of things worth noting here:
- The moment weights parts of the pdf that are further from the mean heavier. For instance, the right bump in the integrand has more area underneath it than the left one, even though the left bump of the pdf is more likely.
- It's not pictured, but hopefully it's clear that higher moments will weight things further from the mean even more heavily.
- For even moments, the polynomial term is non-negative, so the integrand cannot be negative anywhere. This means that there is no "cancellation", as we'll see there is for odd moments.
Summarizing these points, we see that even moments give us a measure of how far from the mean we can expect our random variable to fall. As we go to higher and higher moments, these will give more and more weight to parts of our distribution that lie farther from the mean. So, if for whatever reason there's a bump in our pdf super far from the mean, or a tail that doesn't decay as quickly, it will lead to large higher order moments.
Third Moment (and odd moments in general):
Here is an analogous picture for the third moment of the same pdf:
We can see that odd moments work just like even moments for bullets one and two in the previous section. However, now there's an added complication-- our polynomial is negative for some values, so the integral from the left side of the mean will cancel the integral from the right side to some extent. Thus, odd moments no longer just measure how far things are distributed from the mean. They also provide a measure of the asymmetry of the pdf-- a pdf perfectly symmetric about the mean will have null odd moments. This gives them more information than even moments in some sense, but the drawback is that they're harder to interpret taken alone-- a third moment close to zero could mean that you have a perfectly symmetric, widely distributed pdf, or a narrowly distributed, highly asymmetric pdf. As before, higher order moments will accentuate features that lie far from the mean, so a tail that decays faster on one side than on the other will make a large impact in higher order even moments.
Why standardized moments?
Most of the previous discussion took place using raw moments, but a lot of times the standardized moment is of more interest, ie
$$\tilde{\mu_n} = \mu_n/\sigma^n$$
where
$$\sigma = \sqrt{E[(X-E[X])^2]}$$
is the standard deviation.
There are two main reasons for this:
- It combines the knowledge of two different moments into one. This is especially useful for odd moments, as it can help distinguish between the two degenerate examples given in the previous section: the asymmetric, narrow distribution would have a high skewness while the nearly symmetric, wide distribution would have a small one, even though they both have a small third moment.
- It is scale invariant. It is straightforward to show that $\sigma_{\lambda X} = \lambda \sigma_X$ and $\mu_n(\lambda X)=\lambda^n \mu_n(X)$ If $\lambda$ is a real number scaling $X$. Together, this means $\tilde{\mu}_n(\lambda X)=\tilde{\mu}_n(X)$. This is an important property because in real life, we often are taking statistics of objects with units, and transformations between units are described by scaling them. You can describe the heights of people in a sample using feet or centimeters, but either way they're describing the same thing and so we would like our statistical measures to not change.
TL;DR:
Ultimately, it's impossible to give a completely general yet useful interpretation for what any one moment means. There will always be vastly different distributions that have the same moment, because fundamentally you're trying to describe something with an infinite number of degrees of freedom (the pdf) with a single number. However, there are three rough guidelines that help give a bit of intuition:
- Even moments tell you something about how far things are distributed from the mean.
- Odd moments tell you something about how asymmetric the distribution is about the mean (skewed left = negative, right = positive, and zero = symmetric).
- Higher order moments put more of an emphasis on how the distribution behaves further from the mean, so tails and anomalous pdf bumps far away matter more than the bulk of the distribution that sits close to the mean.
