Experimental Design and Ethics
Just watching people can show you that two things go together, but not that one causes the other. To show cause and effect you have to run an experiment: hand out the treatment by chance, so the groups end up alike in every other way.
Explanation
An experiment is simply a study where you decide who gets the treatment, instead of just watching what people choose for themselves. That one difference is the whole story: if people pick their own group, the groups can differ in all sorts of hidden ways, but if a coin flip picks, those differences even out. There are no formulas here, just a few careful ideas about how to set a study up so its conclusion can be trusted.
Vocabulary:
- Explanatory variable: the factor you change or compare (the treatment).
- Response variable: the outcome you measure.
- Lurking (confounding) variable: a third factor that affects both and can create a misleading association.
Principles of a good experiment:
- Control: compare against a control group (often given a placebo).
- Randomize: assign subjects to groups by chance, so lurking variables balance out.
- Replicate: use enough subjects that chance differences average out.
- Blind: in a double-blind study, neither subjects nor evaluators know who got which treatment.
Why not just observe? Ice cream sales and drownings rise together, but hot weather drives both. In an observational study there's always the chance of a lurking variable like that, and you can only adjust for the ones you thought to measure. Randomization handles them all at once, even the ones nobody thought of, because chance has no reason to put more of any kind of person in one group than the other.
A common mistake. It's tempting to think a big enough observational study can prove causation. Size doesn't fix confounding. A study of a million people who chose their own diets still compares people who differ in exercise, income and much more. Only random assignment breaks that link.
Ethics: subjects must give informed consent, risks must be minimized, data must be kept confidential, and studies on people are reviewed by an institutional review board.
See it
Worked examples
A study records the diets and heart health of 5,000 adults. Can it prove that a diet causes heart disease?
- 1
The researchers didn't assign anyone a diet, they only recorded what people ate, so it's an observational study.
Practice
Type your answer and check it. Solve them all to complete the lesson.
A study records the diets of 5,000 people and their heart health. What kind of study is it?
In a test of a new fertilizer on crop yield, what is the response variable?
Neither patients nor doctors know who received the real drug. This is:
Why assign subjects to groups at random?
Ice cream sales and drownings rise together. The likely lurking variable is:
Students who chose to join a tutoring program scored 8 points higher than students who didn't. Why can't we conclude that tutoring caused the gain?
20 subjects are split at random into two groups of 10. How many different splits into a treatment group and a control group are possible?
The hardest one here. Get it, and you've got this lesson.
In a small experiment, 4 of 8 subjects get a drug at random, and the 4 best outcomes all come from the drug group. If the drug did nothing, what's the chance the random assignment alone would put the 4 best in the drug group?
Where it's used
Drug approvals rest on randomized double-blind trials; tech companies run randomized A/B tests for the same reason: they're the only reliable way to show that a change causes an effect.