# Teaching new stats students confidence intervals, hypothesis testing, and other general techniques for inference

I have instructed for community colleges and adjuncted for major universities for just over fifteen years now.

In all my time doing this job, I've found that the worst manifestation of so-called "mathphobia" inevitably appears in my Statistics courses when we move to the material dealing with inferential methodology.

I have used many different source materials (different textbooks, websites, Youtube videos, example problem writeups posted in addition to lecture, etc.) but it invariably follows the same pattern each semester: one or two of the more vocal students will eventually interrupt me mid-lecture and will claim that they do not understand anything that's going on. For reasons that are probably psychological and similar to "giving up," they almost always claim they haven't understood anything in the class at all, including very elementary concepts like measures of center and introductory empirical probability (e.g. from small tables that give counts for just two binomial variables). This will invariably snowball into the whole class claiming the same lack of understanding, even including those who are earning the highest grades in the section. Among all the math classes I've taught, this part of this course presents the toughest psychological barriers I have to help students overcome, and, ironically, since spreadsheet formulas usually crunch the numbers for us on almost all the problems we encounter, this part also represents the least computation-focused part of any of my courses.

After facing this scenario again yesterday, I have reached two possible contributors to this phenomenon:

1. The processes of confidence intervals and hypothesis testing is difficult and counterintuitive for students learning it for the first time. Even though I've attempted to use analogies (e.g. the "innocent until proven guilty" analogy for hypothesis testing), students have a hard time understanding the concept of using a sample to draw inferences about parameters, and do not understand why error is needed, even if I show them that they already implicitly understand that even the most representative sample still leads to error.

2. Students are looking at the increasingly involved equations needed to compute standard error, z or t-scores for levels, and for test statistics, and math panic sets in even if I tell them that they will far more often lean on technology for these computations and instead should focus more on actual interpretation.

These issues inevitably pop up to form mass confusion and delayed lecturing schedules, even if I convey multiple times that patience and disciplined practice is the true teacher in learning inferential statistics.

A great example of this phenomenon is the example I used yesterday comparing two population proportions: one student just would not stop arguing with me that .51 and .5 (results for two sample proportions I compared in an example) were different numbers and therefore had to mean that the two proportions were different; the student simply would not accept my explanation that the margin of error for the difference implied that a true parameter difference could in fact be zero with the .01 difference appearing out of acceptable error due to the fact that we are sampling. The idea wouldn't sink in even after I slowly reviewed the idea of inference and error with her to her agreement; when we revisited the problem, she just slammed her fist on the table, declared a .01 difference meant two different numbers again in anger, and (as is usually the case) demanded the age-old question of "where she would ever have to use this stuff."

I apologize for venting a bit, but I guess my central point is to ask if any other math educator finds inferential statistics to be one of the top points of difficulty with students, especially math-phobic ones, and what you do to help students get past the frustration and the tendency to discount the presence of error and the fact that we are estimating (confidence interval) or testing a population claim (hypothesis testing) by use of a representative but much smaller sample. Either I have been consistently teaching the subject wrong for years, or else this should be an ordeal many other teachers face when teaching these concepts to students for their first time.

I've been teaching introductory statistics for the same amount of time at a large urban community college. I have never had this response from a class in toto. Last semester I did have one student say that privately. Every semester I definitely have sharp students who do in fact "get it".

I don't know exactly what may cause this, but here's a few tidbits from my practice:

• I use Weiss Introductory Statistics as a textbook, and follow it very closely. I think it's clear and readable. Linking closely to a textbook, I think, communicates that I'm not just personally crazy and making this stuff up any given day. The exercises are almost entirely from real-world data.
• I actually do not rely on, or usually mention, any advanced technology in the body of the course; everything is in fact hand-computed (which is against current trends). I think this is worthwhile for a first course, to really dig into the specifics, develop some intuition, and avoid yet another layer of novelty (I do mention & give some examples on the last day).
• Every day I'm machine-gunning conceptual question at the class (e.g., "Can a mean be negative? Can a standard deviation? Why? How can you check such calculations?") and will discuss/return to anything that some of the class trips over.
• In particular, from the first day to the last, I'm constantly pressing students to recognize the difference between population and sample statements. ("Is the first sentence regarding population or sample? Is the second sentence? Is this table? Is this number? What's the probability that the sample mean is equal to the population mean? Does the population need to be normally distributed for our procedure to be valid?"). This is key.
• Early on I present a paper from the JAMA full of C.I.'s and P-values, to emphasize that this is everyday scientific practice.
• Compared to other instructors, I eliminate a large swath of unnecessary probability content, so as to have plenty of time in the schedule to focus on the inferences (because: that really is the new, hard, important part). Total lecture time spent on discrete probability in my course is 90 minutes.
• In the latter part of the course, I run a concrete example of C.I.'s with playing cards (5 groups, each gets 4 cards, form a C.I. for the mean at 80% confidence; how many C.I.'s should contain the population mean? Check). I do think this generates some real intuition. ("Is the sample mean always the same as the population mean? Could we have gotten A-A-A-A? Is it likely?")
• To the consternation of an adjunct working with me, I, too, do not emphasize probability, and reserve a large amount of time to inferential statistics. I am not heavy on computation, but I do have the students compute by hand at least once or twice for the harder calculations; they always calculate needed sample sizes for a given ME and other less complicated formula. One thing to emphasize: my question is geared toward helping students "cope." In three weeks, almost all of these students will be doing this process as second nature. They don't believe this at all when I tell them this now! – Thomas Rasberry Mar 2 '17 at 17:31
• One other thing for context - both at my university now and at my previous comm college job, the stats courses were almost all nurses; nontraditional-student women comprise about three-quarters of my students, especially as Iowa moves from RN to BSN. This adds an extra layer of challenge to Stats, since most have not seen ANY math in sometimes decades and are placed here through bad advising, and since most students are older women (my claim is not sexist here - in fact, my claim is the direct opposite, since our society is built to discourage women in math!). – Thomas Rasberry Mar 2 '17 at 17:34
• @ThomasRasberry I think Daniel's 5th point is a key to interpreting your student's complaints. If the material is to be understood, procedures must be connected to something deeper, and I don't mean real life examples but mathematical meaning . Many of your successful students are probably good at learning the procedure, but given their complaints it's possible that while they can reproduce it to yield correct answers, they do not generally understand what they are doing. – Andrew Mar 2 '17 at 18:02
• @ThomasRasberry: My students are also predominantly nursing-related (incl. physical therapy assistants, mental health, etc.; hence the JAMA paper choice) and mostly women. We do have at least an extant entry-test and placement procedure (at this time, anyway). Maybe that's the key difference? If students don't have that basic skill, many would nowadays argue they need some kind of support service (e.g., co-requisite prep/lab class) to help them along. – Daniel R. Collins Mar 2 '17 at 18:41
• Daniel, yes, I think it is incumbent on us to improve those support services; we need to convince our administration that it is indeed worth the extra money to supply our help center with people intimately familiar with statistics, especially since the students would get a fresh view (I'm the first to admit some of the problem is in my own areas in need of pedagogical improvement!). Andrew - I won't drag politics too deep in the discussion (I'm actually not on either "side" in the US) but I do think e.g Big Bang Theory and other cultural themes passively discourage women in math. – Thomas Rasberry Mar 2 '17 at 19:57

I have been tutoring stats for a couple of years, and I find that there are a couple of things that students generally find difficult in stats (as compared to other courses):

The symbols: Students seem to have trouble grasping the concept that the calculations are basically the same (save for standard deviation) for both parameters and statistics, and get stressed out by the sheer number of different symbols. I talk myself hoarse every semester stressing the importance of keeping a table of symbols and what they mean. Not being fluent with the Stats symbols is like trying to think critically about a French novel without knowing any French. I also think that, while well-intentioned, efforts to "simplify" the symbols (like those in several Social Science Stats classes) are misguided. All we end up with are two different tables of symbols, which mean the same things but make both the student and the tutor's job much harder.

The tedium: While I understand that handling large data sets is kind of required for the inference part of Stats, I wonder if something could be done about it. Dealing with large data sets results in what I think is a lose-lose situation. On the one hand, large data sets encourage calculator use, which can be a double edged sword, because you can use the reliance on tech to have more time to tackle the concepts, but it also leaves students feeling like the formulas are some sort of magic box, without mathematical meaning. More than once I've had students that by their own admission have no idea what the standard deviation is about, until I take them step by step through the formula, saying things like: "this $x-\bar{x}$ bit comes from us looking for the distance between our data points and the mean." On the other hand, if we try to have them do the calculations by hand, large data sets lead to automatic behavior, and huge amounts of frustration when a sign is not inputted correctly or a decimal point is missing somewhere and the student has to start the calculation all over again.

The "word problems": Now, this is a problem for the entirety of math education, but it is specially prevalent with Stats. Students have a mighty difficult time figuring out what the interpretation questions are asking of them. It doesn't help that the wording of the problems (specially on platforms like MyMathlab) seems to be designed to trip them up. Again, this is a much bigger problem involving both mathematics and reading comprehension, but I'm sure you are familiar with it in Stats.

The difference between samples, populations, and sampling distributions: This is specially evident once the Central Limit Theorem is covered and the familiar z-score formula suddenly exchanges the standard deviation for the standard error of the mean. In my honest opinion this disconnect prevents students from seeing how the test statistic formulas are related, and this results in them compartmentalizing them separately, instead of realizing that, for instance:

$$Z=\frac{x-\mu}{\sigma}$$

Is really asking you to do the same sort of thing that:

$$t=\frac{(\bar{x}_1-\bar{x}_2)-(\mu_1-\mu_2)}{\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}}$$

The confidence intervals: More to your question, I find that they don't give my students as much trouble as I expected. Whenever we discuss them with my students, we play the following game:

• I tell them that I'll give them a 5 bucks to guess my age. They invariably give me a single number. I ask them how confident they are that this is my true age, and I also ask them if it is likely that, among all the ages I could possibly be, they picked the correct number.
• I explain to them that this is their point estimate, and proceed to ask them to give me a range, reminding them that in English we have phrases like "give or take a few."
• They (after some prodding) give me a range centered around their previous point estimate.
• I ask them again if they feel confident in their answer. Usually, after they realize that addition of the range increases their chances of winning, they up their "confidence level."
• I then ask them to give me a much bigger range, and this comes with a corresponding increase in their confidence of winning the game. I also ask if it is either fair or useful to have such a big range, if their objective is to guess my age.
• I explain to them that this is their margin of error, and that while having a large marging of error does increase confidence, it can get to the point that it is no longer useful.
• I also ask other people close to the student to guess my age, and explain that this is analogous to having a larger sample size, in that it can increase your confidence in the answer while narrowing down the margin of error.

They usually walk away with 5 dollars in hand and a better understanding of Stats.

• Ah, I forgot to add one big thing: the relationship between probability and area: Students don't seem to internalize that any z-score or t-score or chi-score has an associated with it on the distribution curve, or vice versa. The connection between things like the empirical rule and the criteria for unusual events escapes them. – Jorge Medina Mar 4 '17 at 19:25
• You can edit the question to include the forgotten part. This is preferable to keeping it as a comment only. – Tommi Nov 23 '17 at 6:37

I thought Dan had some great response for you. My advice would be to cover LESS and to cover more intuitively. Think of another topic: fluid mechanics. I have worked as an engineer and operator a LOT in many industries involving piping. The basic intuitions of "fatter pipe will flow more" and "hard to flow around corners" are actually more important and powerful than Bernoulli's equation with PDEs. I mean I know it sounds trivially simple. But you would not believe how many times I was able to figure something out on the job by having "physical intuition" rather than a derived knowledge or an equation. (And let's be real, everyone uses nomographs anyways!)

Just the way you worded your question was "hard" to me. And I LOVE statistics (at least intuitive statistics. Also, I got the sense that you were trying to build up justifications by derivation rather than intuition.

I don't mean to say all the problems will go away. Let's be real. Your students are not rocket scientists or McKinsey consultants (and even some of these struggle at times!) And it is not the only think important to them. They need to be able to count the pills correctly, get the right ones, enter the chart correctly. And maybe be a little nice to the patients at times. And a bunch of other stuff. But still. (I have dated a lot of nurses, including one who came crying up to my room after a long time patient died, one who misprescribed, etc.) But if you can get across to them some semblance of realization that stats can be tricky (not to trust every paper), that bigger sample sizes are better, and a couple other things that will help in the end.

If they just remember stats class as a formula blizzard, that will be bad. And I remember my "probes and shafts" course as a formula blizzard. And it was relatively gentle!

Look at this video. Not to use it (it is not exactly what you are teaching). It's mfg engineers and execs that it is targeted at. (Like the audience for The Goal by Goldratt). But look at it for pedagogical motivation. This is George Box! But look at how easy he makes things. And how he concentrates on things that are "easy". Isn't that a lot more fun to watch than a formula blizzard? Won't more people get things out of it?

P.s. If you got any cute caring nursing students on the market. Well... But those qualifiers are probably redundant!

... they almost always claim they haven't understood anything in the class at all... including those who are earning the highest grades in the section.

This implies a problem with your assessments: False positives. It is, apparently, possible to get high grades in your class while understanding very little content.

The world of medical testing has internalized the idea of both positive and negative results being either true or false. But the world of math education - nearly all of education, really - has not, so assessments are not designed or interpreted with this in mind. Many students can get high grades (positive test result) while mastering nothing (negative reality). Concepts are hard to teach and assess, while procedures are easy. So most assessments focus on procedures and memorizing algorithms. "If that number is 0.04, then reject, and it's obviously false..."

Another issue that bottlenecks post-secondary math courses is that even many high school graduates have ~zero understanding [I do mean ~zero] of rational numbers... but that's a whole other issue.

she just slammed her fist on the table... [and] demanded the age-old question of "where she would ever have to use this stuff."

Consider checking out the first two chapters of "Statistics" by David Freedman. The introductory chapters are basically cases of statistical inference from controlled vs uncontrolled research. It's mostly relatable, relevant (real data, useful questions), and qualitative, which is what students want to hear when asking "When will I ever use this stuff?"

Or, at the beginning of the semester, give your students a simple hypothetical case of social workers assigned specifically to combat poverty. Data is collected and...

Then pose the following task: "Generate ten different hypotheses for what you see in the table. For each hypothesis, define the minimal evidence it would take to prove or disprove it. Then state how you should collect that evidence and analyze it." I'd have them do this individually first, then compare with a peer or two, then share with the class. This would probably have to be spread over a week or two.

Of course, if they're in an introductory stats course, their answers to that task will likely be intuitively appealing but statistically terrible because we all have terrible statistical intuitions until we are trained otherwise.

You can spend the rest of the semester continually revisiting their answers and they should see their thinking improve.

Of course... it doesn't have to be social workers and poverty, it could be whatever context you wanted. :)