There are lots of different sorts of sampling techniques (both random and non-random) and a myriad of books that explain the best one to use for any given methodology. ‘Snowball sampling‘ is my favourite – you start with one person and ‘sample’ them, then you ask them who you should talk to next, and so on. I like the convenience (when it is used well), and I love the name.

It seems to me that lots of people use a technique that I don’t like at all.

Let’s call it “Lazy sampling”.

“Participants in the study were 35 undergraduate students (24 women, 11 men) aged 18 to 26, recruited from a large university in [the area where the authors work]. We recruited participants in the [sociology] lab at the main campus and many received extra credit in their courses for participating in the study.”
[Information obscured so as not to embarrass the authors]

Just to be clear – this was not an educational research paper. It wasn’t talking about pedagogy or course development. It wasn’t a study about tertiary education. They were looking at a general social issue, using university students as their sample.

In selecting their sample, the researchers made the following decisions:

• They drew their sample from the University where they work, or (at best) a university nearby.
• They drew their sample from one laboratory on one campus.
• They either constrained the age of their sample so that only students would be selected, or they defined their age range after they had seen the ages of the students.
• They either constrained their sample to students or they worked within a reward structure that resulted in only students applying. No admin staff, no faculty staff, no visitors to the laboratory…
• They either didn’t care about gender or they didn’t try to recruit for even numbers.

Why? Why would anybody constrain their sample so tightly? One laboratory – what is that about ? Why would anyone exclude staff or visitors to the university? As a researcher, why wouldn’t you step outside the university and recruit from the local town or city? Wouldn’t it make your study stronger? Any expansion of the sample would have made this paper more interesting.

I don’t like this lazy sampling. I don’t like it at all. It really disappoints me when I open a promising article, only to find that the sample was this small and constrained.

Putting aside my own personal disappointment, lazy sampling is poor research.

It is lazy

Research is about discipline. It is hard work. Solutions are elusive and clear understanding does not come easily. It can take years to accrete the necessary knowledge that leads to a new development, to the point where there is some new consensus. Every good paper can help researchers come to the point where they can make a statement with confidence. Bad papers, on the other hand, just muddy the waters.

These is no rigour in lazy sampling. If you take the lazy way out, you will get lazier. If you flex your research muscles, you will get fitter. Aim to get fitter, and improve the strength of your research results. A strong sample won’t always produce a strong result, but a weak sample will always weaken your results.

Don’t take the easy way out. It is bad for your soul, and it is bad for research generally.

It isn’t sampling

This isn’t how sampling works. In most disciplines, you define your problem space, then you create a hypothesis, then you select a methodology to disprove the hypothesis, then you work out your population, and then you select from within that population. You aim for reproducability, generalisability and rigour.

In many lazy sampling studies, it seems like the whole study has been constructed to make the sampling as simple as possible. It seems to be a convenience sample taken to some sort of absurd extreme. Very, very, very convenient sampling, perhaps.

That isn’t sampling; it is crippling. A sample that’s small and unrepresentative makes it almost impossible to be able to say anything with integrity or confidence at the end of the study.

It is poor research

What is your research question? If your answer is something like “What are the attitudes of students in my university’s psychology department to X?” then I respectfully suggest that you are not trying hard enough. Don’t shape your research question to fit your sample – shape your sample to fit your research question.

If you constrain your sample to students, then what sort of diversity will you encounter? By definition, you are limiting yourself to a narrow educational range. Beyond that, how much diversity of ages, life experience, socio-economic status will you find in a sample of university students? Overall, the student population probably ranges across the full gamet of these social indicators. However, I’m willing to bet that the standard deviation is pretty narrow for any given university.

Will anyone be able to generalise from your results? Will they be useful to others? Universities are not paper factories. If you haven’t produced anything useful, don’t write a paper. If you end up with a study like this, think of it as a trial. Expand your horizons and wait until you do have something useful to say.

It is poor teaching

What example are you setting for these students? Through your example, they are learning that research should be conducted on whoever is handy and that rigour doesn’t matter. They are learning to take the easy way out. Is that really what you want to demonstrate to them?

I do understand that some of these studies start out as exercises for Honours or even undergraduate students, exercises that demonstrate the mechanics of research. If that is the case, don’t publish the paper. If you really want to show students how research works, construct a stronger teaching methodology – provide an environment where the mechanics of reserach are demonstrated by working with strong researchers. Let the students see them struggling with all the complexity that comes along with good research.

Take the pledge: No Lazy Sampling.

Strengthen your sample. Look outside your usual boundaries. You might be surprised by what you find.