Does the scientific paper make the right conclusions?

What if you were told that people who ate ice cream were more likely to be attacked by a shark? Would you believe that statistic? This a common example used to show that just because two things are correlated, doesn’t mean that one thing caused the other. So, what do sharks and ice-cream have in common? Hot weather conditions or summer. Hot weather explains the relationship between ice cream sales and shark attacks.

Here are three things you can look for when evaluating an article’s conclusions:

1. Check if a correlation is attributed as causation (or vice versa)
2. Check for missing data
3. Check for confounders

It takes lots of scientific research and evidence to say that one thing causes another. Before scientists could definitively say that smoking causes cancer, there were decades of research that studied smokers and their health. Likewise, any scientific paper that claims that something causes a specific disease or condition will need rigorous research that spans years—or even decades. Anything less should be a red flag for you.

For example, a study found that high soda consumption in teens was linked to teen violence (Solnick and Hemenway 2012). Some media outlets interpreted this to mean that soda consumption caused teen violence.

But, that’s not what the study concluded! The researchers had found that teens who drank lots of soda were more likely to carry a gun or a knife. That is not the same as saying that drinking soda makes teens violent. There probably are other factors such as socioeconomic status that could be playing a role in the relationship between soda consumption and aggression.

Most scientific articles will discuss correlations (e.g., associations). These studies will examine the relationship between two factors and analyze the results. The results will be framed in a language such as male children are more likely than female children to have asthma (Fuseini 2017). Terms like more likely, less likely, increased risk of, and decreased risk of will be used to explain correlational findings.

Similar to the ice cream example, sometimes there’s a missing puzzle piece that helps explain a relationship.

For example, heart disease is one of the leading causes of death in women (Cho 2020). One reason is that a heart attack may present itself differently in females than in males (Barouch 2021). Women are more likely to experience atypical symptoms (such as a persistent headache) than the more commonly recognized left-sided pain that men experience. A study on heart disease and patient symptomology that doesn’t include atypical symptoms would be missing crucial information that could save a life.

Sex and age also play a role in the development of chronic diseases. For example, age is a risk factor for type 2 diabetes (Okwechime 2015). Imagine you were interested in a new screening test’s ability to detect prediabetes. A study you came across found that the new test was more effective than the standard diagnostic screening test currently being used. But, the results section showed that the group given the new screening test was older than the group who was given the standard test. Could it be that since older people were more likely to get diabetes, the test found more positive cases? Can you be sure that it was the test (and not the age distribution) that lead to the results?

Age is often a potential confounding variable. Confounding variables are factors that have a relationship with both the independent variable and the dependent variable. For example, say that you were reading a study assessing the effects of physical inactivity on the development of diabetes. Well, age is a predictor of diabetes. Age is also associated with physical inactivity. The older you are, the harder it is to stay physically active. Since age could distort the findings, any research studying this link would have to adjust for age to see the true relationship between physical inactivity and diabetes.

The next time you're analyzing a scientific study, take a minute to look at the results and ask yourself: could there be another variable affecting this relationship? Have the authors taken potential confounders into consideration? This information is usually found in the methods section, but a look at the tables in the results section can also be helpful.

Interested in learning more about confounders and missing data? Check out our Epidemiology Essentials Course to learn how to adjust statistically for confounders and account for missing data.