Contents
The base rate fallacy is a cognitive bias that causes us to ignore objective statistical information about a situation in general and instead focus on specific information about individuals.
For example, in Judgment Under Uncertainty: Heuristics and Biases, psychologists Daniel Kahneman and Amos Tversky describe a study in which participants were asked the following:
If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease, assuming you know nothing about the person’s symptoms or signs?
Due to this bias, we tend to focus on the 5% false-positive rate and the fact that an individual received a positive result: indeed, the response given by half the participants was a “95%” chance that they have the disease and the average answer around 56%, when in fact the correct answer (taking the base rate of 1/1000 or 0.1% into account) is only a 2% chance that someone receiving a positive result actually has the disease. To explain this, consider a random sample of 1000 people tested for the disease:
- 1 person actually has the disease (1/1000 prevalence) and produces 1 true positive result
- 999 people don’t have the disease, but due to the 5% false-positive rate produce 49.95 false positive results
Therefore, the odds that someone who tests positive actually has the disease is 1 in (49.95 + 1), or ~2%.
In Practice
As with many cognitive biases, this has very real implications for the average person’s life.
- Obviously, if you test positive for a rare but serious disease, the first question that comes to mind is “What’s the false positive rate?” but that tells you very little without also knowing the base rate of occurrence.
- CNN recently claimed in a story that because “vaccines are only 90% effective…translated into reality, that means for every million fully-vaccinated people who fly, some 100,000 could still become infected.” This framing suggests that even fully-vaccinated people have a 10% chance to catch the disease while flying, which would only be true if everyone flying were guaranteed to be exposed to the virus (a base rate of 100%). In fact, the base rate is nowhere near that high. (One study found it to be 1/4300 or .023%.) Even the base rate were 1%, fully vaccinated people would then have a 0.1% chance of still contracting the virus, not a 10% chance as CNN suggests. While overstating the odds of post-vaccination infection might seem like a good thing if it means fewer people run the risk of flying, it can also mean fewer people choosing to get vaccinated if they believe it won’t make a difference, and of course it does.