Sampling Methods and Bias
How you pick a sample matters more than how big it is. Let chance do the choosing and your sample's errors are random and shrink with size; let people volunteer or grab whoever's handy and the errors lean one way, and no amount of extra data fixes that.
What the symbols mean
- Population size
How many individuals there are.
- Sample size
How many you choose.
Explanation
A sampling method is simply the rule you use to decide who ends up in your sample, and bias is what you get when that rule tilts the results one way. The formula just counts how many different samples of size you could pick from people (read it "N choose n"). In a simple random sample, every one of those possible samples is equally likely, which is what makes it fair. One counting formula, and the rest of the lesson is about choosing well.
Why random? Think about what happens when you choose people yourself. Even with good intentions, you'll lean toward people who are nearby, friendly or easy to reach, and those people may differ from everyone else. Letting chance pick removes your influence. The sample can still be unlucky, but its errors go up as often as down, and they shrink as the sample grows.
Random sampling methods:
- Simple random sample: every group of individuals is equally likely to be chosen.
- Stratified: split the population into groups (strata) and take a random sample from each.
- Cluster: split into groups, randomly pick whole groups, and measure everyone in them.
- Systematic: pick a random start, then every th individual.
Convenience sampling (whoever is easy to reach) is not random.
Sources of bias:
- Voluntary response: people with strong opinions opt in.
- Nonresponse: chosen people don't answer.
- Undercoverage: part of the population can't be selected.
- Response bias: leading wording or social pressure distorts answers.
Sampling error is the natural chance variation between random samples; it shrinks as grows. Bias doesn't. There's a picture further down that puts the two side by side.
A common mistake. It's tempting to think a huge sample must be accurate. If the method is biased, it isn't. A bigger biased sample just gives you the wrong answer with more confidence. In 1936 the Literary Digest collected 2.4 million responses and still called the presidential election badly wrong.
See it
Worked examples
A school surveys 10 randomly chosen students from each grade. Which method is this?
- 1
The grades split the school into groups, and a random sample is taken from every one of them, so each grade is guaranteed to be represented.
Stratified sampling
Practice
Type your answer and check it. Solve them all to complete the lesson.
A school picks 10 random students from each grade. Which method?
A researcher randomly selects 5 classrooms and surveys every student in them. Which method?
A store surveys every 20th customer after a random start. Which method?
A TV station asks viewers to phone in their opinion. What is the main problem?
How many different simple random samples of size 3 can be drawn from 10 people?
A city is 60% renters and 40% homeowners. A stratified sample of 500 residents keeps those proportions. How many renters does it include?
How many different stratified samples choose 2 of 5 renters and 1 of 3 homeowners?
The hardest one here. Get it, and you've got this lesson.
A volunteer poll's estimate is biased by 8 percentage points, and that never shrinks. A random sample's 95% margin of error is about . What is the smallest random sample whose margin of error is under 8 points?
Where it's used
The 1936 Literary Digest poll predicted the wrong US president from 2.4 million responses because its mailing lists over-represented wealthy voters; Gallup got it right with about 50,000.