Extinction Bounties

Policy-based deterrence for the 21st century.

Policy-Research Disclaimer (click to close)

Extinction Bounties publishes theoretical economic and legal mechanisms intended to stimulate scholarly and public debate on catastrophic-risk governance. The site offers policy analysis and advocacy only in the sense of outlining possible legislative or contractual frameworks.

  • No Legal or Financial Advice. Nothing here should be treated as a substitute for qualified legal counsel, financial due-diligence, or regulatory guidance. Stakeholders remain responsible for ensuring their actions comply with the laws and professional standards of their own jurisdictions.
  • Exploratory & Personal Views. All scenarios, numerical examples and opinions are research hypotheses presented by the author in an academic capacity. They do not represent the views of the author’s employer, funding bodies, or any governmental authority.
  • Implementation Caveats. Any real-world adoption of these ideas would require democratic deliberation, statutory authority, and robust safeguards to prevent misuse. References to enforcement, penalties, or “bounties” are illustrative models, not instructions or invitations to engage in private policing or unlawful conduct.
  • No Warranty & Limited Liability. Content is provided “as is” without warranty of completeness or accuracy; the author disclaims liability for losses arising from reliance on this material.

By continuing beyond this notice you acknowledge that you have read, understood, and accepted these conditions.

Our 2-minute elevator pitch

The "you can't stop progress" fallacy

Relative to the number of people who earnestly want humanity to end, a great many more people are convinced that humanity ending in the near future by its own hand is simply inevitable, so they’d better get used to it. This is what we call the “You can’t stop progress” fallacy: The idea that scientific and technological development is like an ever-expanding balloon in all directions, unable to be directed or controlled except by the most low resolution methods.

This is false. Its falsehood can be seen by considering two very basic principles:

  1. Research has to be done by someone.
  2. Economics predicts that if you make an action more expensive, that someone will do less of it.

We all understand this intuitively. If the United States government’s head of state woke up on the wrong side of the bed one day and decided to cut all funding for oncology research, then oncology research would become much more expensive for the companies and individuals pursuing it, and the rate of progress would slow down. We can make this claim without ever wading into the territory of whether this would be a “good” or a “bad” idea - it is simply a matter of how the system works. If the USG’s head of state woke up in a really bad mood and decided anyone doing oncology research had to post bail or risk losing their legal right to do so, the system would slow down even more.

Just as importantly, the rest of R&D would be only minimally affected. Yes, there are certain interdisciplinary advantages that have to be ceded. But a 30 year expert on lymphoma is exceedingly unlikely to even talk to, say, a theoretical physicist, let alone develop a novel theory with them that explains something that isn’t lymphoma but which happens to have strong analogies to it. This is shaping the balloon. We already do it every day when we decide who gets and does not get NSF grants.

Now research and idea generation does have one interesting characteristic: It is almost totally non-excludable in the long term. No matter how tight of a lid our lymphoma researcher keeps on his novel treatment with a 95% success rate, someone, somewhere, will almost certainly leak what he learned to the public - even if it means they just discovered the same things independently and decided to be less secretive about it. Once that happens, the cat’s out of the bag, right? We suspect this is why people really think you can’t stop progress: If it isn’t you, it’ll just be someone else, someday.

If you have read our pitch, of course, you know that we don’t agree with this either. An economic mechanism which makes it more expensive globally to do a certain kind of scientific research avoids the non-excludability issue. You do not actually need international cooperation to enforce this, although it makes the process of enforcement easier. It is likely international cooperation on such a “lowest common denominator” policy would come in its own time. An economic approach also means that, if we fear that we are not slowing down progress on undesirable vectors enough, we can simply ratchet up the economic fines to rapidly disincentivize progress further.

Yes, you can shift the Nash equilibrium away from performing dangerous scientific research. You can do so on a civilizational scale, and it doesn’t even have to be expensive. It simply requires that you push hard enough on the powers that be, to get them to see the light of day and do it.