# Teaching the difference between standard deviation and interquartile range

Are there any good examples for high school studends where:

• Interquartile range is "better" to describe "spread" in an (empirical) statistical distribution of data
• standard deviation is a "better" measure and why it is used instead of interquartile range in many applications
• No particular examples to give you, however a comment about SD: There are two good reasons to use standard deviation. One of them is that it matches the vision of stats as geometry: the distance between a point $(x_1, \dots, x_n)$ and the one where they're all the mean $(\bar{x}, \dots, \bar{x})$ is close to the standard deviation. The other is that there is a well-known distribution describing the possible standard deviations in a sample of size $n$, so probability calculations work better. Commented Jul 6, 2014 at 5:44
• It is also cool that the mean is the closest such point on the line spanned by the vector of ones. This is important for a geometric understanding of linear regression. Commented Jun 5 at 1:23

A simple example for the IQR is to consider the following two data sets:

A = {1,1,1,1,1,1,1} and B = {1,1,1,1,1,1,100000000}.

IRQ for both is 0, but SD is very different. You can argue about which is really better, but this example very nicely illustrates that the IQR tells you where the middle 50% of the data is located while the SD tells you about the spread of the data. It also shows that the IQR is very resistant to outliers (and to some degree skew) while the SD is not.

Although maybe not directly relevant to the beginning student, the standard deviation plays a central role in statistics for two reasons: it is a key factor in the central limit theorem (which explains to students why increasing the sample size of a data set should give us a "better" estimate for the mean) and also because it gives us the variance (which is fundamental for such methods as ANOVA, linear regression, etc). However, one use of the standard deviation that is very important for beginning students to understand is that for the normal distribution, we actually think of the SD as a measuring stick that helps us tell how likely an even is to occur (i.e. the all important "68–95–99.7 rule"). An example I give to my students wrt SD is to ask them to consider 2 hypothetical job offers: one is in city A, located in rural area where the salary ~ N(30K,5K) and the other is in city B, a large city where salary ~N(60K,10K). Suppose that salary offer from city A is 45K and the salary offer from city B is 60K, which should you choose? Here simply looking at how far the salary offers from the means in units of SD, tells us that city A is the better offer.

• If it's not a normal distribution you have also a weaker rule by chebycheff, i.e. that 75 percent of all points are in the 2sigma interval around the mean value Commented Jul 6, 2014 at 18:43
• The main point that I wanted to emphasize however which is often overlooked is the SD is like a "measuring stick". The point being that when we work with a normal distribution, what is of principal interest in many calculations a beginning statistics student performs is how far a value is from the mean in units of SD. This is a key point that is easy for a student to miss. Commented Jul 6, 2014 at 19:36
• I'd advise students to take the big-city offer instead, since their real income will be higher there, even with a higher cost of living. They might have higher status in their locality with the \$45 K job in the small town, but most people will have a higher quality of life with the \$60 K job in the big city.
– user173
Commented Jul 7, 2014 at 0:03

Heights and weights are roughly normal, so standard deviation is more standard for them. In those cases it's easy to translate from IQR to standard deviation by a factor of 1.35, so it's better to use the more standard number. By contrast:

Economic data is rarely normal, so interquartile range is often more useful in that area. This is why interquartile ranges are more commonly quoted for both incomes and real estate values. Consider:

What is the interquartile range of New York City incomes? We can calculate it reasonably well from a reasonable sample.

What is the standard deviation of New York City incomes? We'd need to know details about the highest-earning people like:

• Do we count David Koch as a resident, even if he spends less than half the year in the city?
• Did Michael Bloomberg claim the NYC mayoral mansion as his primary residence for tax purposes, and how did that affect the deducibility of mortgage interest on his other properties?

The standard deviation of income depends so much on these details that it won't be as useful as the interquartile range.

What makes one better or worse may depend on many things.

Ultimately (though this is not especially enlightening) - if you want to look at the kind of thing IQR measures, IQR is good at that, while if you want to look at the kind of thing standard deviation measures, standard deviation is good at that. The different ways they respond to data helps determine where they're more useful (in particular, sd is much more impacted by large outliers, so tends to be more valuable in situations where the data doesn't have big tails)

I want to give a slightly different take that may or may not be of some help.

Both IQR and standard deviation can be thought of as measures of a kind of "typical distance between data points".

For example, the IQR is effectively the distance between the median of the top half of the data and the median of the bottom half of the data, and in that sense is a kind of 'typical distance'.

The variance (up to a Bessel correction factor in the sample case) is half the average squared distance between pairs of points; the standard deviation is thereby a root-mean-square distance between pairs of points, divided by $\sqrt{2}$ (or times $\sqrt{\frac{n}{2(n-1)}}$ in samples when applying a Bessel correction in the sample case.

The different way those 'typical distance' measures are affected by observations in different parts of the data is useful; it may be worth exploring an empirical influence function (without necessarily naming it, the concept is intuitive enough).

These lines of discussion may not be suitable for every level of student, but it's a take on two common measures of spread that may be easier to motivate that some others, and has at least one advantage: it reduces the size of that 1.35 "asymptotic conversion factor" at the normal to the perhaps less surprising 0.954 (or 1.048, depending on which way you go) - that's a good deal closer to 1, and it's smaller for the measure less affected by very large observations.

To me, the differences are much like mean and median. If you have a process where all you care about is the sum of the property across a population, for example total revenue of your sales force then average is what you want.

If you are more interested in number of items within the population, median may be more meaningful. For instance, I compare my performance on a race to the median, was I better or worse than that. Note, it's also a lot quicker, simpler to calculate, especially without a transcribed set of the data.

IQR or some sort of ranges are also important, where you have some cutoff, like only top x percent get to fly jets. Or the bottom ten percent get fired. In particular, with non normal data. But of course lots of data may be non normal.

This may be a physicist answer, but because of the central limit theorem, if a process is fundamentally a counting process, and the distribution seems to be roughly Gaussian (Normal) because the underlying distribution is roughly Poisson or Binomial prior to taking the number of samples to a large limit, then the standard deviation is especially useful. Several years ago when I was more actively doing physics, interquartile range was less common to see, and a similar measure, full-width-at-half-maximum, was more common for distributions like Lorentzians, that have poorly defined standard deviations due to very long tails.

The conceptual problems seem to be the same as with median versus average.

# An activity

This one I have used in teacher education, where the students are of a comparable mathematical aptitude. I ask the students (in small groups of roughly three students) to create an artificial dataset where mode, median and average are in a given order: mode < median < average, median < average < mode, and so on. Different variations. This can be done on paper, in Excel, with Python, etc., as you will. As an extra complication, you can ask one of them to be much bigger (or smaller) than the others, and leave it to the students to discuss what kind of measure of being much bigger makes sense.

The goal of this activity is to create an understanding of how changes in dataset affect the measures of central tendency. As a teacher you need to make this explicit after the activity and emphasize what has been learnt, as usual when teaching any subject. Same with what it means for something to be much bigger than something else, in this case.

## SD and interquartile range

This part is speculative. I have not done this.

This assumes that the knowledge about the measures of central tendency have been activated, for example by the previous activity. Seeing the definitions of SD and interquartile range and talking a bit should lead to seeing that they are related to average and median, respectively. You could use the same activity, but it might be more interesting to have a different one; maybe this is the right way to go if the students had problems with the previous activity.

In any case, after the concepts are in control, you might challenge them to think of, and if time permits, find, examples of datasets where one of the measures is a lot larger than the other. You should leave the meaning of larger as something the students have to make sense of and that is part of the final, whole class discussion.