AGI

Artificial general intelligence (AGI) is the intelligence of a machine that could successfully perform any intellectual task that a human being can. It is a primary goal of some artificial intelligence research and a common topic in science fiction and future studies. Artificial general intelligence is also referred to as "strong AI",[1] "full AI"[2] or as the ability of a machine to perform "general intelligent action".[3] Academic sources reserve "strong AI" to refer to machines capable of experiencing consciousness. https://en.wikipedia.org/wiki/Artificial_general_intelligence

• would lead "immediately" to ASI? Singularity?

My thoughts (2025)

• I'm not convinced it will ever happen
• but I'm not even clear on the definition, esp how you'd "test" it, esp given how little we "know" about consciousness

older notes (2014)

"hostile AI" (vs alignment)

• Elon Musk is worried
  • ref Nick Bostrom Super Intelligence
  • see also ref there to Less Wrong discussion about Roko's Basilisk
• Is the Corporation a Hostile AI? Mike Travers http://omniorthogonal.blogspot.com/2013/02/hostile-ai-youre-soaking-in-it.html (and update)
  • or an alien invader?

Thread on examples of AI optimization per specification rather than intent (bad genie, paperclip maximizer)