Is the study population a good representative sample?

Imagine that you’re a high school gym teacher recording your students’ time for running a mile. Would you expect their records to match those of Olympic runners? Of course not! Olympic runners are much faster than the average adult—let alone teenagers. Comparing their track records would be unfair to your students.

If you’re a clinician who is basing treatment decisions on scientific papers, you would also want to make sure that the study findings can be generalized to your patients.

A lot of thought goes into how study participants are selected. Ensuring that the sample population is comparable to your patient population requires more than simply checking that the demographics match. A good study will use measures to obtain a random sample and will share their reasoning within the paper. Let’s look at how study populations are chosen, and what you should consider when you look at a paper’s study participants.

In an ideal world, if you wanted to study the effects of stress on neurosurgeons, you would recruit all of the neurosurgeons in the world. Obviously, this is not remotely possible. Even if you were to try to recruit all of the neurosurgeons, you would still have to decide if part-time or retired doctors should be included.

Instead, scientists follow a more realistic approach by recruiting a random sample based on the target population. So, researchers may choose to randomly select 100 hospitals in the world and contact all of the neurosurgeons working in the selected hospitals. Keep in mind, this doesn’t guarantee that any results found in this study can be generalized to the neurosurgeons in your hospital.

So, the first step for assessing a paper’s population is to identify the target population and the recruitment process of the study. If the researchers deliberately selected their favorite doctors, then there is a higher chance that bias will be introduced.

Think about it like this: if researchers inherently believe that male neurosurgeons perform better than female doctors, they may choose male and female neurosurgeons for their study that conform to their belief. This can be done unintentionally or deliberately by the researchers. Either way, the study results may be unrepresentative of the truth since the participants were not randomly selected.

To minimize bias, researchers use statistical techniques to determine the sample size and randomly recruit the sample population.

A research paper should introduce the study population early on in the introduction section. For most papers, the first table includes demographic information. As well, the methods section will detail the process by which participants were recruited and selected.

Sometimes, it’s not so much who was included in the study but who was excluded that could be a problem. Let’s take a look at an example. In the USA, African Americans and Native Americans have higher asthma rates than Caucasians (American Lung Association 2020). Now, let’s say a study claims that a new drug manages asthma flare-ups twice as well as the standard of care. But, a closer look at the study reveals that only Caucasian and Hispanic patients were included. In fact, other ethnicities were excluded from the study. How generalizable could the results be to the public if a significant population was not included?

When looking through a study’s first table, pay attention to the population demographics and look for who could be missing—and why. If the research is focused on adults, then children may not be included. Keep in mind that this is not the same as research that excludes a significant, high-risk group for the disease being studied.

The take-home message is that when you come across an interesting study, take a minute to look at its participants and read how the researchers selected them. The more randomized the process is, the lower the risk of sample bias. No study can be completely unbiased, and in some cases finding a study that compares with your patient population can be very difficult. As a clinician, you will have to critically evaluate the study and decide whether its findings are relevant to your patients.

Want to delve deeper into sample populations and how they are calculated? Our Epidemiology Essentials Course teaches you everything you need to know about how sample sizes are calculated, how to better evaluate a study’s participants, and so much more!