(2023-03-30) ZviM On The FLI AI-risk Open Letter

Zvi Mowshowitz: On the FLI AI-Risk Open Letter. The Future of Life Institute (FLI) recently put out an open letter, calling on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. (2023 Giant AI Experiments: An Open Letter)

There was a great flurry of responses, across the spectrum. Many were for it. Many others were against it. Some said they signed, some said they decided not to. Some gave reasons, some did not. Some expressed concerns it would do harm, some said it would do nothing.

Eliezer Yudkowsky put out quite the letter in Time magazine. In it, he says the FLI letter discussed in this post is a step in the right direction and he is glad people are signing it, but he will not sign because he does not think it goes far enough, that a 6 month pause is woefully insufficient, and he calls for… a lot more. I will address that letter more in a future post. I’m choosing to do this one first for speed premium. As much as the world is trying to stop us from saying it these days… one thing at a time. 2023-04-05-ZvimEliezerYudkowskysLetterInTimeMagazine

Structure of this post is:

• I Wrote a Letter to the Postman: Reproduces the letter in full.
• You Know Those are Different, Right?: Conflation of x-risk vs. safety.
• The Six Month Pause: What it can and can’t do.
• Engage Safety Protocols: What would be real protocols?
• Burden of Proof: The letter’s threshold for approval seems hard to meet.
• New Regulatory Authority: The call for one.
• Overall Take: I am net happy about the letter.
• Some People in Favor: A selection.
• Some People in Opposition: Including their reasoning, and complication of the top arguments, some of which seem good, some of which seem not so good.
• Conclusion: Summary and reminder about speed premium conditions.