5
$\begingroup$

In prob/stats, students mostly get the idea of mean/expected value, and variance/s.d. slowly sinks in. Even the third and fourth moments (about the mean) have "standard" interpretations - skewness and kurtosis. The latter is something one can at least think of geometrically, e.g in terms of the tails.

Does anyone have ideas for a good, student-centered geometric interpretation (or other intuitive one) for higher moments? I realize that there may not be any great ones - see for instance this answer - but even a nice graphic describing 3rd versus 5th moment, or how the moments change as they go higher (even versus odd) would be useful.


Yes, I did get this question. To be fair, it's a math major course, not intro stats for social sciences or something.

$\endgroup$
  • 4
    $\begingroup$ The answer to 'how do I picture 15 dimensions?' is 'you don't; picture 3'. Maybe this is the same - you just think of it as 'the higher version of this thing I do understand'? $\endgroup$ – Jessica B Oct 2 '19 at 11:30
  • $\begingroup$ That does make some sense, but I have a better sense of how to bridge that gap when talking of higher-dimensional spaces. E.g. we have projections and slices, visualizations, or at least can talk about the vector of variables involved in predicting the weather. I have no intuition at all here for how to make it plausible that picturing 3 is a realistic way to understand 15, if that makes sense. Partly because I have very little background in this material, I admit! $\endgroup$ – kcrisman Oct 2 '19 at 12:35
  • $\begingroup$ I also have very little intuition in this case. But my point is more that there's a point where you stop relying on intuition, and rely on formalism instead, having used the cases you can picture to give you enough understanding of what the formalism achieves. $\endgroup$ – Jessica B Oct 2 '19 at 20:09
7
$\begingroup$

I must confess I'm not an educator, but I like this question and at the very least I can answer with the intuitive picture I use in my own head.

The $n$-th central moment $\mu_n$ of a random variable $X$ is defined by:

$$\mu_n \equiv E[(X-E[X])^n]$$

For what follows we will take the measure space of $X$ to be the real numbers, since that's the easiest to visualize and the most common thing to deal with in statistics. Now, with this knowledge we can rewrite the moment as

$$\mu_n = \int_{-\infty}^{\infty} (x-E[X])^nf_X(x)dx$$

where $f_X$ is the pdf of $X$. To help complete the mental image, it is important to keep in mind that $E[X]$ is a constant and $f_X$ is a normalized function that is non-negative. So, our integrand looks like some single term polynomial centered at the mean of $X$, multiplying the non-negative pdf of $X$. This is a lot clearer once you see what I'm talking about, so I will show some photos and explain.

Second Moment (and even moments in general):

Here is a picture illustrating the integral I was talking about for the second moment using some pdf I made up. The dotted line is the pdf, the dashed line is the polynomial term, and the solid line is the total integrand:

second moment

Keep in mind that the moment is defined as the integral of the solid curve. There's a couple of things worth noting here:

  • The moment weights parts of the pdf that are further from the mean heavier. For instance, the right bump in the integrand has more area underneath it than the left one, even though the left bump of the pdf is more likely.
  • It's not pictured, but hopefully it's clear that higher moments will weight things further from the mean even more heavily.
  • For even moments, the polynomial term is non-negative, so the integrand cannot be negative anywhere. This means that there is no "cancellation", as we'll see there is for odd moments.

Summarizing these points, we see that even moments give us a measure of how far from the mean we can expect our random variable to fall. As we go to higher and higher moments, these will give more and more weight to parts of our distribution that lie farther from the mean. So, if for whatever reason there's a bump in our pdf super far from the mean, or a tail that doesn't decay as quickly, it will lead to large higher order moments.

Third Moment (and odd moments in general):

Here is an analogous picture for the third moment of the same pdf:

third moment

We can see that odd moments work just like even moments for bullets one and two in the previous section. However, now there's an added complication-- our polynomial is negative for some values, so the integral from the left side of the mean will cancel the integral from the right side to some extent. Thus, odd moments no longer just measure how far things are distributed from the mean. They also provide a measure of the asymmetry of the pdf-- a pdf perfectly symmetric about the mean will have null odd moments. This gives them more information than even moments in some sense, but the drawback is that they're harder to interpret taken alone-- a third moment close to zero could mean that you have a perfectly symmetric, widely distributed pdf, or a narrowly distributed, highly asymmetric pdf. As before, higher order moments will accentuate features that lie far from the mean, so a tail that decays faster on one side than on the other will make a large impact in higher order even moments.

Why standardized moments?

Most of the previous discussion took place using raw moments, but a lot of times the standardized moment is of more interest, ie

$$\tilde{\mu_n} = \mu_n/\sigma^n$$

where

$$\sigma = \sqrt{E[(X-E[X])^2]}$$

is the standard deviation.

There are two main reasons for this:

  1. It combines the knowledge of two different moments into one. This is especially useful for odd moments, as it can help distinguish between the two degenerate examples given in the previous section: the asymmetric, narrow distribution would have a high skewness while the nearly symmetric, wide distribution would have a small one, even though they both have a small third moment.
  2. It is scale invariant. It is straightforward to show that $\sigma_{\lambda X} = \lambda \sigma_X$ and $\mu_n(\lambda X)=\lambda^n \mu_n(X)$ If $\lambda$ is a real number scaling $X$. Together, this means $\tilde{\mu}_n(\lambda X)=\tilde{\mu}_n(X)$. This is an important property because in real life, we often are taking statistics of objects with units, and transformations between units are described by scaling them. You can describe the heights of people in a sample using feet or centimeters, but either way they're describing the same thing and so we would like our statistical measures to not change.

TL;DR:

Ultimately, it's impossible to give a completely general yet useful interpretation for what any one moment means. There will always be vastly different distributions that have the same moment, because fundamentally you're trying to describe something with an infinite number of degrees of freedom (the pdf) with a single number. However, there are three rough guidelines that help give a bit of intuition:

  • Even moments tell you something about how far things are distributed from the mean.
  • Odd moments tell you something about how asymmetric the distribution is about the mean (skewed left = negative, right = positive, and zero = symmetric).
  • Higher order moments put more of an emphasis on how the distribution behaves further from the mean, so tails and anomalous pdf bumps far away matter more than the bulk of the distribution that sits close to the mean.
| improve this answer | |
$\endgroup$
  • $\begingroup$ This is a really good answer, even if (as you say) a full intuitive understanding may be hopeless. "It's not pictured, but hopefully it's clear that higher moments will weight things further from the mean even more heavily." -> if you add a picture of the zeroth moment (=1 and is just the integral of the original function) and comparing 2nd/4th/6th that would be fantastic. Because then I can just point students to this answer when they ask this :) Thank you very much. $\endgroup$ – kcrisman Oct 16 '19 at 2:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.