# What is a realistic situation that illustrates precisely what a p-value is?

Students in the basic statistics courses I teach often learn a little bit of probability and then learn hypothesis testing. The core concept that ties the course together is the p-value, but most students never adequately connect the p-value to the earlier topics in probability.

I would like my students to have an "aha moment" about the p-value by completing an assignment that has the following outline:

Question 1: Assume [Situation X]. Calculate the probability of [unlikely data Y] occurring.

Question 2: Complete a hypothesis test of the claim "We are in the situation [Situation X]," given the data [unlikely data Y].

In Question 1, the students would compute a probability. Then in Question 2, the students would find that the p-value is exactly the same thing as their answer to Question 1. This would illustrate what the p-value means precisely.

Here are some almost-answers to my question that do not satisfy me:

• Situation X is "You have a fair coin." When students do the first question to compute a probability assuming a fair coin, they use the binomial distribution. But then inside a hypothesis test, students use the normal approximation to the binomial distribution, so the probabilities are not equal and the exercise does not work.

• Situation X is "You have a normally distributed random variable with a known population mean and standard deviation." The problem here is that when you go to the second question, you have to awkwardly declare that you are absolutely sure about the distribution and the population standard deviation, and that the hypothesis test is only the test of a claim about the population mean. This seems really unrealistic to me.

• I would be careful about your question: realistic situation that shows precise definition to illustrate (share insight). You are mapping three constraints in there. In terms of "precision", I see a lot of huffing and puffing about small definitional errors by Bayesians (which I agree are errors). But the key thing is to go from zero concept to some concept. I would be happy with a doctor or a nurse who has some basic feel for a p value or confidence interval even if she makes a minor error in hypothesis test definitions. The power is really in the basic insight (when coming from zero). Dec 17 '18 at 4:46
• It's probably wrong example for your kids, but one that I like for intuitive idea of CI is the EIA STEO price funnel. eia.gov/outlooks/steo/report/prices.php It's really Bayesian (since the CI is based on the "betting odds", literally odds for different discrete futures contracts ($70 oil,$80 oil, etc.). And it's non medical. And I can't get business people to understand it. And it's a CI, not a p-value. But to me it just sings. eia.gov/outlooks/steo/report/prices.php Dec 17 '18 at 4:59
• I don't know why you posted these comments; they don't seem to help advance the question in any way. Confidence Intervals are also part of statistics, but not relevant here. Dec 17 '18 at 5:18