Extinction Bounties

Policy-based deterrence for the 21st century.

Policy-Research Disclaimer (click to close)

Extinction Bounties publishes theoretical economic and legal mechanisms intended to stimulate scholarly and public debate on catastrophic-risk governance. The site offers policy analysis and advocacy only in the sense of outlining possible legislative or contractual frameworks.

  • No Legal or Financial Advice. Nothing here should be treated as a substitute for qualified legal counsel, financial due-diligence, or regulatory guidance. Stakeholders remain responsible for ensuring their actions comply with the laws and professional standards of their own jurisdictions.
  • Exploratory & Personal Views. All scenarios, numerical examples and opinions are research hypotheses presented by the author in an academic capacity. They do not represent the views of the author’s employer, funding bodies, or any governmental authority.
  • Implementation Caveats. Any real-world adoption of these ideas would require democratic deliberation, statutory authority, and robust safeguards to prevent misuse. References to enforcement, penalties, or “bounties” are illustrative models, not instructions or invitations to engage in private policing or unlawful conduct.
  • No Warranty & Limited Liability. Content is provided “as is” without warranty of completeness or accuracy; the author disclaims liability for losses arising from reliance on this material.

By continuing beyond this notice you acknowledge that you have read, understood, and accepted these conditions.

Our 2-minute elevator pitch

[alpha] What if someone cannot pay an extinction bounty they are guilty for?

TL;DR: If someone is found guilty of breaking an extinction bounty fine, and even after exhausting all other options still cannot afford to pay it back, we at Extinction Bounties currently think ordinary jail is a fine fallback plan in lieu of the rest of the funds. Our primary interest is in deterring smart, well-capitalized agents from performing research along dangerous technological development vectors, and so we suspect bounty hunters will still find it well worth their time to go after these people. This topic is intricately related to the question of whether extinction bounty liability insurance should be required by law or not, but

In the original fine-insured bounty (FIB) criminal justice theory Dr. Robin Hanson states that he believes FIB liability insurance should itself be mandatory, and that in a system where fine-insured bounties are used for all criminal law that the only crime that lands you in jail is the crime of being uninsured. This makes a lot of sense for a totalizing system where the same legal liability insurance company might protect you against a vast swath of crimes and accusations.

However, extinction bounties as we describe them here are what economists might call a keyhole mechanism. They are very narrowly targeted to the world’s hardest problems, in the hopes of being as easy to actually draft legislation for, get passed, and get international agreement on as possible. As in many areas of design, the scope changes the implementation. Because the scope is so much smaller, and because the vast majority of people living today will never even get close to accidentally triggering one of these extinction bounties, it really does seem like overkill to mandate them at a population level.

What about on a professional level? There is precedent here, in most states anyone driving on the road is legally required to carry driver’s insurance. What if we required, for example, all software engineers to carry anti-AI risk liability insurance?

Truthfully, we don’t see any “knockdown arguments” for one or the other side here, mostly because of the characteristics of the populations we are dealing with. Making the insurance mandatory would raise premium prices at the margin, but those premia are likely to be small for the vast majority of people in the first place, and just as importantly people smart and conscientious enough to be working as employed software engineers or microbiology researchers tend to have enough money to afford such things. But these very characteristics - intelligence, conscientiousness, perhaps risk aversion sufficient to hold down a full time job in the first place - suggest such people would take the risk of accidentally getting swept up into an extinction bounty litigation seriously, and purchase insurance sufficient to meet their fears anyway.

Either way, our original question for this topic was what happens to people if they are found guilty, and after losing everything we can take and liquidate from them - their job, their house, etc. - what happens? Smart, consientious, employed people generally have a lot to lose, and they feel the pain of that loss quite seriously. We think jail as a fallback solution would be a fine deterrent for such people. Since these are well-remunerated and usually very remote-friendly people, the posibility exists that they can pay back their fines efficiently by working from jail, and they may even prefer this themselves to spending e.g. 20 years in the slammer and then being tossed back into a totally unfamiliar professional environment and forced to start from scratch.