Where can I find realistic data for college-level elementary statistics problems?

Question

I'm creating a large number of practice problems for my statistics students. These problems are for an elementary stats course where students:

• measure central tendency
• measure dispersion
• use linear regression to extrapolate and interpolate
• calculate the mean, and standard deviation for a discrete distribution
• generate binomial probability distributions
• use normal distributions to calculate probabilities
• use the central limit theorem to calculate probabilities for $\bar x$
• calculate confidence intervals
• perform simple hypothesis testing

It is tempting to make up numbers for these problems. However, I wonder if it is better practice to use numbers based on real-world measurements? I know what when most people make up lists of values that are meant to be random, or centered about a mean, the distribution of their made-up numbers is not always the same as measured data.

That said, my students are not expected to deal with sets of data with more than 35 elements, and I do not want confusing examples. I must minimize student frustration.

The ideal? Sets of values that seem reasonable, come from a real source and that give answers that build intuition over the core concepts in the course. Will real data help students form a more concrete mental image of the problem? Our book often has sourced data, yet I find far too many of the problems are highly technical to the point of obscuring the chapter's central concept. Complexity isn't a bad thing for the stronger students. I don't want to get hung up on converting between units, or bogged down trying hard to get my students to picture what "3.7 acre-feet of water per hour per person" might mean. At least not just yet.

I've found it very time-consuming to source "real" data. Are there any libraries online with samples that could work well in this situation? Do you have any recommendations for finding good data? I have no fanatical desire to have a real source for everything. I seek reasonable data and with the kind of noise and randomness that arises from real-world measurements.

Answer 1

(I came to mention DASL, but since it's already been mentioned, I'll give some other resources.)

opendata.stackexchange.com often has mentions of useful sources of data, some of which are small (and others of which might be sampled to generate smaller data sets).

It's also a good place to ask about data sets

The datasets subreddit http://www.reddit.com/r/datasets/ often has useful data sets (and again, is a good place to ask about data sets)

The US government data portal may also have some useful data sets: http://www.data.gov/

Answer 2

DASL (pronounced "dazzle" and short for Data And Story Library) is an online collection of stories with matching data sets to be used for educational purposes. They are real data from real research. Searchable by statistics concept and by theme of the story.

OzDASL is similar, but most of the data has an Australian or New Zealandish source. Personally I find OzDASL a bit easier to navigate.

The statistical package R comes with quite a few datasets built in, and so do several packages within it (for example, the MASS package). If you have R or RStudio installed you can type data() to see a list of the built in data sets and data(package = .packages(all.available = TRUE)) to see a list of data sets inside all the available packages.

Answer 3

This is not a good answer, but I'll mention anyway that Mathematica now connects to vast "curated" data sources. It would take some effort to master these sources, but they could provide endless streams of real data. Here is one example I just ran (NB: Up-to-date in that no Pluto!).

---

         

---

Answer 4

An alternate suggestion: EIA.gov has WONDERFUL statistics and easily downloaded. The site has a gazillion pages so you may want to navigate around. But almost all of them that have a table or graph also have a "download data" button (usually in upper right). It usually comes over in a nice formatted excel spreadsheet. See here for the site overall: https://www.eia.gov/

A lot of what they have is time series and that won't be so useful to you. But you can just take a snapshot in time.

1. For instance, if you look at 2017 oil production (annual M bpd), you get a nice Pareto pattern (80% production from 20% states). Here is the link for 2017 oil production: https://www.eia.gov/dnav/pet/pet_crd_crpdn_adc_mbblpd_a.htm (download data is in upper left). Yes the units is thousands of barrels per day...but at least it is not acre feet! And the states should be recognizable! (well actually there are a few subtleties like the GOM offshore and PAC offshore "states" and then the states that have zero production, not in database.) You may want to clean it up a little to eliminate the PADDs (or leave them in if you find that interesting). This is close enough to a real "business problem" (production of an economic good) as to be interesting. I think it just barely meets the 35 element limit (if not, combine the two Alaska series and/or the two offshore series).

2. Here is natural gas by state: https://www.eia.gov/dnav/ng/ng_prod_sum_a_EPG0_VGM_mmcf_a.htm (Probably clean it up a little by combining some of the state offshore/onshore).

3. There is a lot of others stuff on the site (pricing, oil by API gravity, overall energy, other forms of energy: coal, renewables, per capita consumption, etc.) Again a lot of it is time series, but you can pick a year.

Answer 5

... since it's not yet listed: Gapminder is not only a beautiful tool for data visualization, it also allows to download the data.

Answer 6

This is one way that textbooks could (potentially) justify their high price. If your textbook does not supply this for you, then perhaps you should switch.

Answer 7

I think the thing that makes it hard is the restriction to less than 35 elements. There are many sources of vast and intricate real world data but not so many with so few elements.

A very simple one (to do means, SD, histograms, etc.) is number of regular season wins by NFL team. There are only 32 teams.

Some simple exercises are mean for the league (forced to be 8, if you call a tie half a win..I would) You can not tell them this and just see how many note it, after they calculate it. Also, then SD (this is not forced).

You can do some comparisons (AFC versus NFC mean). Is the difference statistically relevant? (p value). How about division versus division? (Soon you get into a Bonferroni situation.)

You could also do something a little more complicated by taking 2016 results and 2017 results and see how much 2016 performance predicts 2017 results.

Here are a couple sites with tables of the data:

http://www.espn.com/nfl/standings

https://www.nfl.com/standings/division/2017/REG (this one has a little menu where you can grab 2016, etc. also)

You can cut and paste the data into excel. Have to clean up the repeated headers a little and maybe cut some columns if you want. But it should be pretty easy. Maybe add a column for division and conference. Really playing with something like this on excel (including using the "filters") is an easy exercise and good thing for people to know how to do in the work world. We haven't gotten to the point where admin assistants are proficient in playing around with filtering data in excel (but of course students are new and some will need to learn...so good).

I think the size of the data is small enough so that it is very intuitive but complex enough to be interesting (32 rows for the 32 teams and then somewhere between 4 and 12 columns depending on how complicated you want to get).

If it bothers you that it is sports, you could turn it into some sort of artificial business example...but then you're not real any more. I would just use it as is.