4
$\begingroup$

Students in the basic statistics courses I teach often learn a little bit of probability and then learn hypothesis testing. The core concept that ties the course together is the p-value, but most students never adequately connect the p-value to the earlier topics in probability.

I would like my students to have an "aha moment" about the p-value by completing an assignment that has the following outline:

Question 1: Assume [Situation X]. Calculate the probability of [unlikely data Y] occurring.

Question 2: Complete a hypothesis test of the claim "We are in the situation [Situation X]," given the data [unlikely data Y].

In Question 1, the students would compute a probability. Then in Question 2, the students would find that the p-value is exactly the same thing as their answer to Question 1. This would illustrate what the p-value means precisely.

Here are some almost-answers to my question that do not satisfy me:

  • Situation X is "You have a fair coin." When students do the first question to compute a probability assuming a fair coin, they use the binomial distribution. But then inside a hypothesis test, students use the normal approximation to the binomial distribution, so the probabilities are not equal and the exercise does not work.

  • Situation X is "You have a normally distributed random variable with a known population mean and standard deviation." The problem here is that when you go to the second question, you have to awkwardly declare that you are absolutely sure about the distribution and the population standard deviation, and that the hypothesis test is only the test of a claim about the population mean. This seems really unrealistic to me.

Thanks for your insight!

Improve this question
$\endgroup$
3
  • $\begingroup$ I would be careful about your question: realistic situation that shows precise definition to illustrate (share insight). You are mapping three constraints in there. In terms of "precision", I see a lot of huffing and puffing about small definitional errors by Bayesians (which I agree are errors). But the key thing is to go from zero concept to some concept. I would be happy with a doctor or a nurse who has some basic feel for a p value or confidence interval even if she makes a minor error in hypothesis test definitions. The power is really in the basic insight (when coming from zero). $\endgroup$
    – guest
    Dec 17, 2018 at 4:46
  • $\begingroup$ It's probably wrong example for your kids, but one that I like for intuitive idea of CI is the EIA STEO price funnel. eia.gov/outlooks/steo/report/prices.php It's really Bayesian (since the CI is based on the "betting odds", literally odds for different discrete futures contracts ($70 oil, $80 oil, etc.). And it's non medical. And I can't get business people to understand it. And it's a CI, not a p-value. But to me it just sings. eia.gov/outlooks/steo/report/prices.php $\endgroup$
    – guest
    Dec 17, 2018 at 4:59
  • 1
    $\begingroup$ I don't know why you posted these comments; they don't seem to help advance the question in any way. Confidence Intervals are also part of statistics, but not relevant here. $\endgroup$
    – Chris Cunningham
    Dec 17, 2018 at 5:18

1 Answer 1

Reset to default
1
$\begingroup$

Without knowing about p-value, students could discuss their answers to:

Willy Wonka is putting a small number of golden tickets in his chocolate bars again. His claim is that the probability of winning is 1/10. How many chocolate bars would you have to buy, with no winning ticket inside, to be reasonably skeptical about his claim?

Then, the idea of p < 0.05, or 0.01, etc. could be introduced, and students could subsequently answer the question with these tolerances in mind.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ This is definitely a good tool. but it doesn't really precisely answer the question -- I want the students to see the exact p-value they get from following a hypothesis test by calculating it some other way. $\endgroup$
    – Chris Cunningham
    Dec 20, 2018 at 19:08
  • $\begingroup$ Thank you. Using your definitions, how about: [Situation X] = 1/10 of candy bars contain a golden ticket, [Situation Y] = I've purchased 30 candy bars without golden tickets $\endgroup$
    – Marian Minar
    Dec 22, 2018 at 2:18
  • 1
    $\begingroup$ This does not work; the standard method for doing a hypothesis test would not work in your situation (expected number of candy bars is less than 5) and it would not fit my requirements (the standard method uses a normal approximation to the binomial distribution so the p-values will not be equal). $\endgroup$
    – Chris Cunningham
    Dec 27, 2018 at 19:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.