Correlation Is Not Causation

Why the slogan is not enough

“Correlation is not causation” is true and, said on its own, close to useless. It stops an argument without improving it. Everyone knows the phrase, few can say what the alternative explanation is in a specific case, and the discussion ends in a shrug.

The useful version is a list. If two things move together and one does not cause the other, there are only a handful of reasons why. Naming which one applies turns a slogan into an analysis.

The four alternatives

1. Reverse causation

The causal arrow points the other way. People who use a fitness app are fitter than people who do not. It is at least as plausible that being interested in fitness causes app installation as that app installation causes fitness.

Test: does the proposed effect plausibly precede the proposed cause in time, or could it?

2. A common cause

A third factor drives both. Ice cream sales and drowning deaths rise together. Neither causes the other. Warm weather increases both.

This is the most common case in social and economic data, and it is often invisible because the third factor is not in the dataset. A study using only the variables it collected cannot rule out a confounder it never measured.

Test: what else changed at the same time, and could it affect both quantities independently?

3. Selection

The association exists in the observed group because of how the group was formed, not because of any relationship in the population. If a clinic only admits patients with two of three symptoms, those symptoms will appear correlated among its patients even if they are independent in general.

Selection effects are especially common in anything measured through a platform, a volunteer sample, or a self-reported survey, because the people who appear in the data chose to.

Test: how did the observed cases come to be observed, and could that process create the pattern?

4. Chance

With enough comparisons, some pairs of unrelated series will track each other closely. This is not a flaw in anyone’s reasoning; it is what randomness produces. It becomes a problem when only the matching pair is reported.

Test: how many relationships were examined before this one was presented, and would you know if others had been tried?

The five questions

Apply these to any causal claim you encounter.

1. What exactly is claimed to cause what? State it as “an increase in A causes an increase in B”, with units and a population. Vague causal claims cannot be tested.
2. How was the association measured? Observational data, an experiment, or an anecdote? A randomized experiment addresses confounding by design. Observational data does not, though good analysis can reduce it.
3. Which of the four alternatives is hardest to rule out here? Name it specifically. If you cannot name one, you have probably not thought about the mechanism.
4. What mechanism is proposed? A causal claim without a mechanism is weaker than one with a plausible pathway, because the mechanism generates further testable predictions.
5. What is the effect size, and does it matter? A real but tiny effect is often reported in the same language as a large one. “Associated with an increased risk” says nothing about magnitude.

Worked example

Claim: “Students who eat breakfast score higher on tests, so schools should serve breakfast.”

Question one: an increase in breakfast consumption causes an increase in test scores, among school-age students.

Question two: most of the evidence is observational, comparing students who do and do not eat breakfast.

Question three: a common cause is the hardest to rule out. Household stability, income, sleep, and adult supervision plausibly affect both whether a child eats breakfast and how they perform on tests. Selection is also relevant if the comparison uses students whose families opted into a program.

Question four: a mechanism is proposed and is physiologically plausible, which is a point in the claim’s favour. Blood glucose availability affects sustained attention.

Question five: effect sizes in this literature vary substantially by population, and the strongest effects tend to appear among students who were previously food insecure, which is exactly what the confounding account would also predict.

The conclusion is not that school breakfast programs are unjustified. It is that the strongest argument for them may be nutritional and equity based rather than a direct claim about test scores, and that presenting the test score claim as established overstates it.

Notice how much more useful this is than saying “correlation is not causation” and stopping.

When observational evidence is still strong

Ruling out an experiment is not the same as ruling out knowledge. Observational evidence becomes persuasive when several things hold together: the association is large, it appears consistently across different populations and methods, it follows a dose-response pattern, the proposed cause precedes the effect, a mechanism is understood, and proposed confounders have been measured and adjusted for without eliminating the effect.

Any one of these alone is weak. The set of them together is how many well-established causal claims outside laboratory settings came to be accepted.

Practical exercise

Find a headline containing the words “linked to”, “associated with”, or “may increase”. Write down:

1. The causal claim the headline invites you to make.
2. Whether the underlying study was observational or experimental.
3. Which of the four alternatives is hardest to rule out.
4. What the effect size actually was.

Item four is frequently absent from coverage. Its absence is itself a finding.

Summary checklist

• I stated the causal claim precisely, with a population.
• I identified whether the evidence is observational or experimental.
• I named which of reverse causation, common cause, selection, or chance is hardest to rule out.
• I checked whether a mechanism was proposed.
• I found the effect size, or noted that it was not reported.

No external references were required for this page. The reasoning patterns described here are standard in research methods teaching, and the examples are constructed for illustration.

Next step

Read: What Else Could Explain This?

Topics covered: Critical Thinking