3
$\begingroup$

Can you please explain or provide resources that explain this issue (esp. to a lay audience): How to understand the significance of a co-relation number that studies (esp. meta-analyses) produce. The difficulty is from moving from the number to the practical implication of the number (what does it mean for the reader?)

The contrast is between an implications for a lay individual vs group/academic research purposes.

Please note that while comments about the following example are appreciated, the main purpose of the post is to be able to deal with correlations in general not the specifics of the example

Example:, the correlation between general mental ability (GMA) and job performance is .5. See the difficulty of interpreting that number here

https://www.facebook.com/groups/473795076132698/permalink/1072863099559223/

When a lay person sees the number .5, how can they better understand it? Is that high, low, moderate etc?? For me as an individual, is it worth considerable time and effort to raise my GMA? If I'm an employer, do I eliminate a candidate because of a low GMA ? Do I specify from the beginning that my applicant pool will be from a certain GMA etc..? How much important is GMA (.5) in comparison to a correlation of .4 or .2? If a correlation is .1, should I ignore it?

=== Answers/resources up to now: 1- Against Individual IQ worriers (Scott Alexander) https://slatestarcodex.com/2017/09/27/against-individual-iq-worries/

2- A video explaining the concept "Co-relation does not equal causation" https://www.youtube.com/watch?v=gxSUqr3ouYA I explain more about what I mean by "resources" and "demonstrations" here https://www.researchgate.net/post/List_of_Concept-Demonstrating_Resources

$\endgroup$
  • 1
    $\begingroup$ It might be simply that this is an area of mathematics education research that I know nothing about, but this seems to be a sociological question more than a mathematics education question. Correlation is discussed here, but it sounds like you want to know how this is applied in some way (which you have not very precisely defined), which to me seems more relevant to the specific area of application than to the concept itself. But perhaps someone more knowledgeable than I am in applied statistics (which is very little) can help. $\endgroup$ – Dave L Renfro Jan 23 at 23:23
  • 2
    $\begingroup$ I don't think correlation is actually the important piece of information, on its own. A correlation of 1 would say that smarter people always do better at their job. But that doesn't have anything at all to say about how much better they do. If the total variation is, say, worth 10 seconds a day, it's not going to be worth considering. But if the range is worth 5 days per month, that might make it worth the effort of considering. The correlation does not give that information. $\endgroup$ – Jessica B Jan 24 at 6:35
  • $\begingroup$ On a different note, would you mind reducing the text size in your question back to normal? The bold is enough emphasis. $\endgroup$ – Jessica B Jan 24 at 6:49
  • $\begingroup$ @JessicaB done! $\endgroup$ – Hashem ElAssad Jan 24 at 19:50
2
$\begingroup$

Correlation tells you how much one quantity can be relied upon as a measure of the other quantity. The usefulness of that is when you want to measure one but can't, and the other is easier to measure.

For example, you can't easily measure how much people like your webpage. But you can measure what proportion click through to a second page, or send a link to a friend...

Probably the key thing a lay person needs to know is that there is no simple interpretation of whether the value is high/low/important. In particular, they should learn not to take media reports at face value. Those who can should consider reading what they can of the original paper, as some of the time the interpretation/caveats are available and intelligible. Otherwise, if the idea seems like it might have an impact on their lives, they should file the information under 'find out more before making a decision'.


More detail:

A high correlation says that if property A increases, it is very likely that property B increases too. If the correlation is negative, it means A increases is linked to B decreasing. But the correlation doesn't say anything about the size of the increase/decrease. For that you need to look at other information from the data.

In the context of the question: a correlation of 1 would say that smarter people always do better at their job. But that doesn't have anything at all to say about how much better they do. If the total variation is, say, worth 10 seconds a day, it's not going to be worth considering. But if the range is worth 5 days per month, that might make it worth the effort of considering. The correlation does not give that information.

Deciding whether a particular correlation is high/low also requires additional information.

If there are two data points, there will (almost) always be a correlation of 1 or -1, because the second reading was either higher or lower than the first (if your measure is discrete, it could also be the same, giving a correlation of 0). Similarly, with three data points, there are usually only two really-different options: the graph keeps going in the same direction, or it changes between up and down.

In these cases, what appears to be a very strong correlation is more likely just the result of insufficient data. As the number of data points increases, you are less likely to get a high correlation coefficient by pure chance, and so getting a high number becomes more significant. The majority of researchers will check whether the number is 'high enough' by checking a table, rather than any direct understanding/calculation of their own, and any respectable researcher will report the result of this look-up.

In practice, if your news source is reputable, it is reasonably safe to assume that if the result is being reported, the correlation coefficient has been found to be high enough. But, as mentioned above, that does not tell you how you should respond. There are several factors that still need to be considered:

  • the quality of the study: poorly-done research may not give true results. In extreme cases, there have been researchers who made up their data to get funding/meet job requirements.

  • the size of the effect: (as described above). One example here is 'learning styles'. Research shows that people do respond differently to different methods of teaching. However, it also says that this effect is swamped by other factors (such as the teaching style that works best for the content being taught).

  • the applicability of the study to you: each study occurs in a particular context. How much the result applies to you personally depends on how well that context matches your circumstances. For example, a study of women's health should only be applied to men with caution.

  • other data: a single study, however large, is not guaranteed to give the truth. Other results might give conflicting information. You may or may not have access to these. Drug trials are like this - the drug company has to show that the drug does it's job and doesn't cause (too much) harm. However, they are not required to tell you how many times they tried to get evidence and failed.

  • correlation does not imply causation: eating ice-cream is correlated with heat-stroke. This does not mean that eating less ice-cream will protect you from heat stroke (indeed, I would guess the reverse is true); it is simply that the two occur in the same conditions. Knowing there is a correlation does not tell you how you should act.

  • cost/benefit ratio: being alive increases the chance of getting cancer. Deciding to stop being alive to avoid getting cancer would be ... ill-advised.

$\endgroup$
  • $\begingroup$ +1 FYI, for me at least I got more out of your first comment (which I really liked) than your answer. Maybe you could incorporate your first comment into your answer? $\endgroup$ – Dave L Renfro Jan 24 at 9:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.