Ice Cream Purchases and Murder Rates - Correlation Does Not Imply Causation
By Melissa Read, Ph.D. on Monday, February 5th, 2007One of the first recorded correlations was the relationship between height and salary. Taller people earned higher salaries than shorter people. That wasn’t good news for me =). But there are actually lots of correlations like this. Some work to my advantage, others don’t. Some make perfect sense and others seem impossible to explain.
Consider this. There’s a strong, positive correlation between ice cream purchases and murder rates. When ice cream purchases increase, murder rates increase. When ice cream purchases decrease, murder rates decrease. If your job was to prevent murder, what would you do with this information?
Some people say that they’d increase the presence of law enforcement in ice cream shops. This is based on the notion that murderers like to buy ice cream – perhaps to make themselves feel better about killing people. If this notion were true, murderers would definitely be a lot easier to catch. Just sit in an ice cream shop and wait. Eventually, they’ll be there. Others say that they’d prevent known criminals from purchasing ice cream. This is based on the notion that ice cream sets off some kind of murderous trigger in people who have criminal tendencies. By making sure that criminals don’t get access to ice cream, we could make society safer. We could also prevent people from becoming criminals in the first place. Restricted access to ice cream would be just another negative consequence of becoming a criminal. Still others say that they’d put a ban on ice cream sales all together. This is based on the notion that anyone can become a murderer after eating ice cream – that people who eat ice cream are like time bombs waiting to detonate. Kids everywhere would be devastated by the ban but it would be totally worth it in the end. After all, human life is much more important than dessert.
Hmmm. These all seem like decent plans until you do a little research to find out more about the relationship between ice cream and murder rates. Truth is, there’s a third variable that influences both. It’s the weather. When the weather is hot, people are more inclined to buy ice cream. When the weather is hot, people are also more prone to violent acts.
Research agencies like to talk about relationships between variables. Clients get excited by these findings and both like to make plans based on interpretations. But relationships between variables can be misleading. Variables can be correlated without causing variations in one another. In science, we say…
Correlation does not imply causation.
It’s good to identify that relationships between variables exist. But it’s better to determine why they exist. In this industry, we must be careful when interpreting reports of correlation and making recommendations based on findings like these. If not, we may end up making recommendations that aren’t useful — or that even backfire when we’re trying to reach our goals. If not, we may end up arming cold ice cream shops with law enforcement personnel when they are desperately needed in the hot streets.
Interesting. So all the research on banner ads suggests that the more “interactive” they are the better they perform. If they have video or forms within them, they perform better. Is it possible however that the reason that those banners perform better is not because they have video/forms included in them, but because the companies that can afford to spend the energy, time and budget to have those banners created on average have more experienced, more talented people working on them (either internally or through an agency) and therefore the banners are generally more optimized, better targeted and have better creative?
I read somewhere recently where someone was saying they were going to start putting their TV commercials into their banner ads because they heard that banners with videos get higher conversion rates. Maybe they should read your post
Thus the importance of controlling variables and performing good A/B tests.
Here’s something that I bet happens all the time: A/B email tests, such as testing for the more effective of 2 different email subject lines.
Are the two versions of the email sent to the same group of recipients to see which of the two was opened more frequently?
Are the two versions sent on the same day?
Are they both sent at the same time of day?
I would guess those aren’t usually controlled, but it’s definitely worth flushing out.
Good points TS! Maintaining control over variables like time of day is essential. In controlled experiments, we need to hold all variables constant – except for the ones that we are actively manipulating. Otherwise, it’s difficult to determine whether our results are caused by our experimental manipulations or by deviations in the variables that we have not controlled.
With applied internet experimentation, it can be hard, if not impossible to hold all variables constant. However, there are methods for addressing these issues. When you’re testing multiple versions of an email subject line for example, it is essential to test them simultaneously – randomly presenting each version to independent users. Simultaneous and random presentation can cancel out the effects of variability from factors that we cannot hold constant, like time of day. This gives us a clear picture of the success or failure of our experimental manipulations – those that we hope to assess.
I’d love to see a controlled test of murders… seems to really appeal.
Is there a graph or some numbers that one can use to prove this point. It’s obvious that the correlation is purely co-incidental but would still be good to graph out.
Yeah it does make sense, ice cream has sugar which makes people hyper and wanna kill. DUH
Great analogy. I remember the ice cream/murder analogy in a statistics class in college, and I use it frequently. Good illustration with website designs.