A researcher conducted a survey to determine whether people in a certain large town prefer watching sports on TV to attending the sporting event. She asked 117 people at a local restaurant on a Saturday, and 7 people refused to respond. Which of the following factors makes it least likely that a reliable conclusion can be drawn about the sports-watching preferences of all people in the town?
A) sample size
B) population size
C) The number of people who refused
D) Where survey was given
If the researcher wants to draw a reliable conclusion about all people in the town, she needs to select her sample randomly from all people in the town. We don’t know anything about the restaurant: it could be a fancy place with waiters in tuxedos, or it could be the local burgers and wings joint with 50 TVs showing sports games. One might reasonably expect the patrons at those two hypothetical restaurants to have different sports viewing preferences. Even if it’s just your middle of the road diner, it’s still not a random sampling of people in the town. Heck, some people in the restaurant might not even live in the town!
A few mental shortcuts that will be useful if you see a question like this again:
- If the number of people surveyed is greater than 100, that’s probably enough for a reliable result if the rest of the study is designed well.
- Population size is not likely to ever be a problem in a question like this. The population is the huge group that the researcher wants to generalize about. I can’t really imagine a question where a population is too big, and if the population is really small, then the researcher could feasibly just survey the entire population and have a perfectly reliable conclusion.
- People refusing is a red herring. Don’t worry about how many people refuse—worry about how many respond.