20
$\begingroup$

I am a teaching assistant for a sophomore engineering laboratory. We use standard deviation a lot during the semester. It is an incredibly useful concept that can be used in a lot of engineering applications. We also teach students how to calculate variance, which is simply the square of standard deviation. I have never used variance for anything and I struggle to imagine how it could be applied. Is there any way that variance can actually be applied? Is there a reason to spend time on it at all?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ It's common in statistical materials (books, articles) to present the parameters of a distribution in terms of mean and variance. $\endgroup$
    – Daniel R. Collins
    Jun 7, 2018 at 21:46
  • 1
    $\begingroup$ As an engineer or businessman, SD is the much more common and intuitive metric (or normalised SD). I believe variance is useful for regressions and time series models and the linear algebra and such involved in them. Not an expert but my take. Honest, I think just showing them the thing is enough (since you may come across it in random literature). Nothing wrong with that level of coverage. Don't dwell on it for long amounts of time. Don't kill it either. $\endgroup$
    – guest
    Jun 8, 2018 at 11:31
  • 1
    $\begingroup$ I agree that this question has things backwards, insofar as we really should think of variance as the "primary" concept, and standard deviation as the square root of variance. However, it is worth noting that the notation we use expresses the relationship in precisely the opposite way: we have a symbol ($\sigma$ or $s$) for standard deviation, and denote the variance as $s^2$ or $\sigma^2$. We do not have a stand-alone symbol (say, $V$) for variance and denote the standard deviation as $\sqrt{V}$. $\endgroup$
    – mweiss
    Jun 12, 2018 at 18:37

8 Answers 8

Reset to default
52
$\begingroup$

Actually, your definitions are backwards: the standard deviation is the square root of the variance. In other words, one defines variance first --- it has a simpler formula, and it has simpler mathematical properties. Then one defines the standard deviation as the square root of the variance.

As for why to study it, there are many mathematical formulas which are simpler when expressed in terms of variance rather than in terms of standard deviation. One key example is:

The variance of a sum of independent random variables is the sum of the variances.

You can, of course, translate this into standard deviations, but its more complicated when you do that:

The standard deviation of a sum of independent random variables is the square root of the sum of the squares of the standard deviations.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ this is one of the fundamental flaws of statistics that make it one of the most hated subjects - both variance and standard deviation are so unintuitive and badly named that even after years of exposure it is easy to mix them up, and still not understand their role other than a plug in parameter. $\endgroup$
    – Dmytro
    Jun 9, 2018 at 10:12
  • 2
    $\begingroup$ Note however that the notational conventions used for denoting standard deviation and variance definitely express the idea that the latter is the square of the former. $\endgroup$
    – mweiss
    Jun 12, 2018 at 18:39
19
$\begingroup$

An analogy: speed is to standard deviation, as kinetic energy is to variance.

Energies can be added usefully; speeds can only be added in very limited circumstances.

Similarly, variances can be added across data points, or across alleged sources of the total variance; standard deviations cannot be directly added.

When analysts say things like, "This factor accounts for 20% of the total effect", they are usually performing a calculation involving either a ratio of variances, or the differences between means.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ "variances can be added across data points." A variance is a property of a data set, not a point. Under what circumstances would I add two variances? What would the resulting number tell me? $\endgroup$
    – BobTheAverage
    Jun 8, 2018 at 15:08
  • 1
    $\begingroup$ @BobTheAverage -- The total variance of the data set is the sum of the individual data points' variances. Of course, each individual datum's variance can only be calculated if the mean of the whole data set is known. $\endgroup$
    – Jasper
    Jun 8, 2018 at 16:06
  • $\begingroup$ @Jasper Can you clarify the energy/speed comparison? If I'm on a train holding a ball, it has a certain speed and energy. If I throw the ball in a way that it'd have a speed of 10m/s when standing still, it'd have speed v_train + 10m/s, but it's energy will behave in a more complicated way. (Note: there's two "Jasper"s here :). $\endgroup$
    – Jasper
    Jun 9, 2018 at 12:03
  • $\begingroup$ Kinetic energy is equal to a weighting factor (half of mass) times the square of speed. Variance is equal to a weighting factor times the square of standard deviation. There are many different ways that you can throw the ball from a 5 m/s train to have a ground-referenced speed of 10 m/s (or a kinetic energy of 50 J/kg). All of them involve adding 37.5 J/kg of kinetic energy to the ball. $\endgroup$
    – Jasper
    Jun 9, 2018 at 17:59
  • $\begingroup$ @BobTheAverage I think Jasper may be using "data point" in the sense of a physical measurement which has some associated uncertainty. The idea there is that, hypothetically, if you could do the exact same measurement repeatedly (or in a bunch of identical parallel universes), you would be sampling a probability distribution, and the variance of the data point is defined to be the variance of that distribution. $\endgroup$
    – David Z
    Jun 10, 2018 at 5:20
8
$\begingroup$

  1. The variance is calculated directly, while the SD is calculated in terms of the variance.

  2. The variance is additive for independent variables.

  3. The effect of sample size is a lot easier to explain using both variance and SD: taking n samples multiplies the variance by n, and then when we divide by n to get the mean, we divide the SD by n. It is this difference that results in larger sample sizes having less variation in the sample mean.

  4. It's a lot easier to talk about the accuracy of linear regression in terms of squared error: you have the total squared error (variance of the data points), the squared error of the residual (variance of residuals), and the explained variance (total SSE minus residual SSE). You can then take the ratio between explained variance and total SSE to get $r^2$.

Improve this answer
$\endgroup$
6
$\begingroup$

In statistics, everything is focused about variance as the second central moment of a distribution. For example, you may be using the "sample standard deviation" formula which differs from the standard deviation of a complete distribution because of a correction that makes the square of the sample standard deviation an unbiased estimator of the variance of the distribution. Without (re-)squaring that formula, it is an unbiased estimator of nothing useful.

In fact, that bias can be expressed as VX=EVX + VEX, namely the actual variance of a distribution is the expected variance of a sampling plus the variance of the sample means over the samplings. For equally weighted samples this leads to the well-known ${n\over n-1}$ correction factor.

The principal advantage of the standard deviation is that it is in the same unit/scale as the sampled variable. Which is nice for engineering purposes, namely looking up and relating material constants rather than doing actual statistical calculations.

Improve this answer
$\endgroup$
3
$\begingroup$

You are talking of "use" and "application".

But when it comes to "finding" the standard deviation then the route in probability theory is always:

  • 1) find the variance.
  • 2) find the standard deviation by taking the square root.

In short: in practice there is no way to find the standard deviation without first finding the variance.

Exceptions on this might exist, but are very very exceptional.

This justifies the focus on variance in probability theory.

Improve this answer
$\endgroup$
3
$\begingroup$

I'll work through a toy problem with both SD and Variance, and examine which is simpler.

Suppose you roll 100 fair six sided dice. Is it surprising if they sum up to 360?

Well, E(d6) = 3.5, E(100 d6) is 350. Var(d6)=(7*5)/(6^2)=35/36, Var(100 d6)=3500/36. SD(100 d6)=~sqrt(Var(100 d6))=9.72

(E(100 d6)-350)/SD(100 d6) =~ 1.03, or 1.03 standard deviations. Events 1.01 standard deviations away from the mean are not surprising.

Without variance, we are stuck with SD(d6)=sqrt(7*5/6^2)=~0.972. SD(100 d6)=sqrt( SD(d6)^2 )=sqrt( 100 * 0.972 ) = 9.72.

The rest follows the same. But instead of the nice, linear Var(100 d6) is 100 Var(d6), we had to do sqrt( 100 SD(d6)^2 ).

That step -- taking a value, squaring it, scaling it, then taking its square root -- is a bunch of steps that can introduce error into a problem.

Things get even worse when we introduce covariance. First, Variance is the Covariance of a random variable with itself, a nice property. Second, once Covariance is measured or calculated, you get Var(A+B)=Var(A)+Var(B)+2Cov(A,B) (this generalizes).

Then look at: SD(A+B) = sqrt( SD(A)^2 + SD(B)^2 + Cov(A,B) )

The same kinds of statement with standard deviations is painful. You're squaring and taking the square root of everything all over the place, and usually the way you'd calculate the things you square would have been taking the square root in the previous step!

Improve this answer
$\endgroup$
2
  • $\begingroup$ @pjs laugh, yes, that was backwards $\endgroup$
    – Yakk
    Jun 10, 2018 at 18:37
  • $\begingroup$ Var(A+B)=Var(A)+Var(B)+2Cov(A,B). You forgot factor 2. $\endgroup$
    – drhab
    Jun 12, 2018 at 6:37
2
$\begingroup$

Consider: Analysis of Variance (ANOVA), a basic technique included as a chapter in most elementary statistics textbooks.

Improve this answer
$\endgroup$
0
$\begingroup$

The intuitive concept of variability/dispersion around a central tendency, if any, is the (mean) absolute deviation, the distance. Historically, it did not become the primary concept because the absolute value is a difficult operator to manipulate.

Hence, the roundabout way: to eliminate the direction of the deviation, use the square and then sum the squares and average (variance). Then, take the square root of the variance and obtain the standard deviation, in order to go back to units of measurement that are compatible with the real-world phenomenon under study.

So the variance is the parent of the standard deviation, the latter exists because the former exists. "Teaching the variance" is required for a better comprehension as to what is the standard deviation, but apart from that, it is required if one will face calculating theoretically/empirically the standard deviation -because inherently, one will then have to unavoidably calculate the variance first.

Moreover, a lot of times it is the variance that is given as datum for the dispersion, with the (silent) understanding that when one wants to obtain actual measures comparable to reality they will take the square root of it.

Also the estimators that estimate directly the variance have usually better statistical properties than estimators that attempt to estimate directly the standard deviation (this is because of the non-linearity arising from the square root). Estimating the variance of an i.i.d. sample,

$$\hat \sigma^2 = \frac 1{n-1} \sum_{i=1}^n (x_i-\bar x)^2$$

is an unbiased estimator. Estimating the standard deviation as simply as that is not

$$\hat \sigma = \sqrt{\frac 1{n-1} \sum_{i=1}^n (x_i-\bar x)^2}$$

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.