# Looking for a creative way to teach ANOVA

Background

I'm currently teaching a Statistics course (for the first time), one in which I been trying to teach with a hands-on approach. I have had my students (all traditional college age, non-STEM majors) collect their own data and form their own hypotheses before doing any calculations for $\chi^2$, Regression Lines, and applications of the Central Limit Theorem. I'm having difficulty, while preparing for the upcoming unit on ANOVA testing, finding motivation for ANOVA, with its tedious calculations and cumbersome notation/naming of terms.

Note that, while in class we often utilize Excel to help compute values for calculation-heavy situations, for the final exam, a departmental requirement is to have an ANOVA calculation question (with calculator only). While this example is relatively simple (3 groups of 5 observations each) it still results in a lot of calculation, and so I want to make sure my students understand the meaning behind the calculations as well as how to keep everything organized for when they will have to do it on the exam.

Question

Has anyone with more Statistics teaching experience sought and found an effective way to teach ANOVA testing? Preferably, something that clearly describes the conceptual aspects as well as the computational ones.

• I'm a grad student and I'm currently working on putting together an anova workshop that attempts to take a very "why" approach to the subject. As restrictive as the F test can be in actual usage, I think it's an extremely important test of comprehension w.r.t. both hypothesis testing and "how variance works." It should be done in the next week or so and I'll post an answer at that time. Includes a great "dashboard" for simulating the ANOVA test, with lots of graphs. – shadowtalker Nov 19 '14 at 11:33
• @ssdecontrol, While I have long since taught this project, I'm curious to see how your workshop went. Were you able to find any creative ways to teach ANOVA? – Andrew Sanfratello Dec 11 '14 at 8:15
• Maybe not totally innovative, but we got very positive feedback from our professor and classmates. The lecture notes proper still aren't finished (other projects got in the way...) and the dashboard needs some UI cleanup. But my project partner and I intend to actually teach the workshop in January so it will get done! Thanks for the reminder – shadowtalker Dec 11 '14 at 8:17
• I look forward to hearing the results! – Andrew Sanfratello Dec 11 '14 at 10:36
• I don't think this is good enough for a whole answer (others may think otherwise, so let me know) but I made this Geogebra applet to investigate ANOVA: tube.geogebra.org/student/m1175387 – DavidButlerUofA May 12 '15 at 18:19

I taught a class like yours when I was a graduate student over 30 years ago. I remember how I taught some of it, but not exactly how I taught ANOVA. I was happy with the class, and the students were happy, too. They brought refreshments on the last day of class, and gave me a card with kind comments on it. They were happy because most were afraid of taking a statistics class, but were comfortable in class and understood more than they thought they would. Since it was a successful class, I’ll share how I might have taught the lesson on ANOVA.

At that time, for a college class, I did mostly lecture. But I did encourage questions a lot, and if there were no questions I would sometimes say, “If you have no questions, then I’ll ask you questions,” and get someone to explain an important point. This is one way that I highlighted important points. When lecturing I might ask a question or pose a hypothetical situation for students to consider before explaining, because we process information better when it is used to answer a specific question or resolve a specific problem. If I asked a question when lecturing, we might or might not have had some whole class discussions, but less than I would have now. (The class included about 30 students.) In the spirit of collaboration, others can suggest revisions to improve this lesson and to create different versions.

I tried to get students to understand the concepts, not just to do calculations. A class in ANOVA would build on understanding of concepts taught earlier, primarily:

1. The mean as the middle of a distribution and the best estimate of a set of scores based on the “least squares” criterion. It is computed as the sum of scores divided by the number of scores.

2. The deviation score as the distance a score is above or below the mean. It is computed as score minus mean (x minus x-bar). This is a simple but key concept that shows how the mean is in the middle of interval data, since the sum of the positive deviations equals the sum of the negative deviations. It is the basis for the least squares criterion, where error is measured as the sum of squared deviations from the estimate. The error is lower when computed from the mean than from any other point. The deviation score is also the basis for computing variance.

3. Variance as a measure of dispersion, the extent that scores are spread away from the middle of the distribution. It is also a measure of error if the mean is taken as the estimate for a group. It is computed as the mean of the squared deviations (the “mean-square”): the sum of squared deviations divided by the number of scores, or by the degrees of freedom if estimating the population variance from sample data.

4. The logic of hypothesis testing including especially the concepts of the null hypothesis, the experimental hypothesis, alternative hypotheses which are eliminated by a good research design, and alpha error. I remember using an analogy of trying to disprove someone’s claim that he was psychic. Everyone could understand that process, so mapping the terms of hypothesis testing to aspects of that process made the terms easy to understand.

Before doing any computation, I want to highlight several concepts: the mean of each group as the value that represents the score for that group, the variance within a group as how good an estimate its mean is as a group score, and the variance of the group means (between-group variance) as a measure of how spread apart the groups are from the grand mean and from each other.

Then I want the students to think about, if they were researchers doing a study with three groups, each getting a different treatment, and they expected each group to be significantly different from each other, would they want the within group variances to be larger or smaller, why? Would they want the between group variance to be larger or smaller, why? I would expect they would want the within groups variances to be smaller because a tighter distribution for each group results in less overlap between group scores. The larger the within variance, the less accurate is the mean as a score representing its group, and a larger separation between means is needed to have confidence in group differences. They would want the between group variance to be larger because that happens when the means of the groups are spread farther apart. (I want the students to think about it, but this is straight forward so I wouldn’t have spent much time on discussion. I would have quickly explained or summed up the discussion.)

Next I want the students to think about the within group variances relative to the between group variance. I ask the students to visualize the within group variances staying the same, but the between group variance starting small then getting larger and larger. I ask the students to describe what they see. I expect that they would say the groups of scores move apart, being greatly overlapped at first then separating into more distinct groups. At some point the groups become significantly different. (I would spend a little more time on discussion of the relative sizes of the variances, on this and the next questions before lecturing on the calculations, because this is conceptually important and more complex than the first set of questions.)

If there is time, I would ask the class to visualize the reverse situation--the within group variance staying the same this time, meaning the means of the groups stay the same, but the within group variances starting out very large then becoming smaller and smaller. What do they see?

Before doing computations, I would ask the class, “If you did a study and the means of the three groups were farther apart than you expected, with two treatment groups doing better than the control group to a surprising extent, and one treatment group doing much better than the other, you would expect the statistical test to be significant. Thinking in terms of the variances, is there a situation where the statistical test is NOT significant despite the large differences in group means?” I would hope that, after the visualizations that we did of the variances changing relative to one another, the students would say that if the within group variances were extremely large, then seemingly large differences in group means may be relatively small, and there would be much overlap in group scores. For group scores to be significantly separated, the between group variance, and therefore the differences among group means, should be large relative to the within group variances, which measures the accuracy of the means as the groups’ scores. This would be a logical segue to the F-statistic, which I would explain to the class is what is used in ANOVA.

The questions and limited discussions get the students to understand the issues. I would have then just lectured on partitioning the variance and doing all the calculations, but relating it to the concepts and the discussion. I think the calculations make sense and are easier to remember when the issues are discussed first conceptually, without any numbers attached. One point in particular that would need emphasis is using degrees of freedom instead of n-size to compute a sample variance as an estimate of population variance. Otherwise, the means and sum of square calculations are similar to what they had already done in earlier parts of the class. The F-statistic calculation should make sense after discussing the importance of the relationship between the between-group and within-group variances. Calculations were shown in the textbook, and I don’t remember doing any handouts, but it would make sense to have a handout with all the calculations done for a small set of data, and a table for the F-statistic with the degrees of freedom and critical values of F highlighted for the particular data contained in the handout.

Then I think it is important to review effect size, n-size and the power of the test, and the difference between statistical significance and practical significance—all of which might have been covered with hypothesis testing in the introduction to inferential statistics. I don’t know that I actually did a review of these points in that class I taught (probably not), but I think it is more important for students to know these factors that affect the results of a study than it is to know how to compute the F-statistic from raw data, since they would use a computer to do the calculations if they actually analyzed data. These factors that affect the result of a study are important to know if they actually did a study using ANOVA, or even if they only need this knowledge to evaluate studies in their field.

The class always had homework, so they would have had to do computations from raw data. I had a system where they turned in their homework at the department office and I reviewed and wrote comments on each one, and returned them by the next class. Classes were three times a week.

I remember that I gave an optional extra credit assignment to copy the Results section of a journal article in their field and to discuss the article in terms of what we covered in class, to demonstrate their understanding of the course material.

If I taught the class now, I would incorporate what technology we had available in the class. I would also try to take a few minutes for small group work to get the students to think more deeply about real world issues related to statistical testing, especially for the social science graduate students in the class. Because of time constraints, they might first discuss the question in class, but then write up their responses as homework or for optional extra credit. For example, I would refer back to the last hypothetical situation regarding large group differences that were not statistically significant. “If you did a study and your group differences were large but not statistically significant, and you were confident in your theory and thought the study should be redone, what changes would you consider to improve the study? Remember what we discussed about why large group differences might not be statistically significant.” They might research it online and that’s fine. My goal is to get them to think more deeply about the issues, not necessarily come up with solutions all on their own.

I don't have more experience than you, but hopefully you might find an alternative experience helpful.

Where I live, high schools teach this with resampling/bootstrapping methods. It allows students to get a feel for what hypothesis testing really means before using the idealised distributions. When I taught this, I was surprised at how much more immediate and comprehensible it was than using the CLT and the "tedious calculations" you refer to. Even fairly complex hypothesis testing of was direct and comprehensible.

Like all Montecarlo methods, you require a huge number of samples to get an accurate estimation of the hypothesis distribution, but that is what computers are for.

Resampling methods are easy to understand, and all the calculation is done automatically by appropriate software. It may only take a few tutorial sessions for your students to get a good understanding of how hypothesis testing works in a variety of situations. After they understand hypothesis testing, the only hard thing is to motivate a shift to the less accurate traditional methods that are probably mandated by your syllabus!