Contents
A term coined by philosopher Nick Bostrum for “a risk that arises from the dissemination of (true) information that may cause harm or enable some agent to cause harm.” In other words, truths that should not be known. One of my core values is learning and teaching others about the world in order to make it better. That’s complicated by the possibility of certain true things that may be harmful to teach or even know in the first place. Some intuitively obvious examples come to mind, such as that we’re better off not knowing the hour of our future deaths.
Bostrom classifies information hazards into two categories:
- Adversarial hazards: information that can be used by bad actors to cause harm, but is not inherently harmful
- Idea hazards: information that causes harm to the knower, regardless of intention
Nate Hagens gives the following taxonomy focused on adversarial hazards:
- Knowing/sharing information that is dangerous to the powerful can make you a target. This information is not inherently dangerous, only dangerous within a social context where others fear the truth and have the will and license to hurt you to keep you quiet.
- A discouraging consequence of this is that much of the work being done to prevent harm by the powerful is being done quietly. It looks as if the powerful reign with impunity and no one is trying to stop them, when that is not the case.
- The inoculation effect: information necessary to prevent harm, that when known by harm-doers, helps them change their strategy. This is mainly a risk when that information can’t be shared selectively. For example, you learn that an unknown person is planning to blow up a bomb in the town square at noon. To protect your fellow citizens you need to tell them to avoid the square at noon, but in getting the word out you risk letting the bomber know as well, so the bomber can choose a different time instead. Alternatively, if you don’t tell anyone and try to stop the bomber yourself, you risk your failure to do so directly contributing to the deaths.
- The dual-use hazard: information that can be used for good can also be used for evil. Classic examples include knowledge of nuclear fission that can produce both power plants and bombs, and knowledge of biology that can both cure pandemics and engineer new ones. This seems a fundamental consequence of the facts that knowledge is power and that we can’t stop people from using power to cause harm.
- True things said too early undermine your credibility: Hagens gives the example of peak oil, “where being early on a critical and relevant and nuanced issue pretty much looked identical to being wrong for about 15 years.” Sharing something true with people who aren’t ready to accept its consequences it can provoke a kind of epistemic autoimmune response where they become more opposed to the idea (and you) than if you’d waited for the right time to share it (which can be especially problematic with urgent information!) (This seems related to the Overton window)
- The self-fulfilling prophecy hazard: where being aware of a risk makes it more likely to happen.
See also
- In H.P. Lovecraft’s fiction (e.g. The Call of Cthulhu) the world was once ruled by “The Great Old Ones,” eldritch beings of immense power who sleep beneath the surface waiting to return. The few who learn this history, crucially one they can do nothing about, tend to go mad.
- infohazard on the (fan-fictional) SCP Foundation wiki