(2024-12-28) Laine Capital, AGI, And Human Ambition

Rudolf Laine: Capital, AGI, and human ambition. I've heard many people say something like "money won't matter post-AGI". This has always struck me as odd, and as most likely completely incorrect.

(Edited to add: The main takeaway of this post is meant to be: Labour-replacing AI will shift the relative importance of human v non-human factors of production, which reduces the incentives for society to care about humans while making existing powers more effective and entrenched. Many people are reading this post in a way where either (a) "capital" means just "money" (rather than also including physical capital like factories and data centres), or (b) the main concern is human-human inequality (rather than broader societal concerns about humanity's collective position, the potential for social change, and human agency).)

labour-replacing AI means:

• The ability to buy results in the real world will dramatically go up
• Human ability to wield power in the real world will dramatically go down (at least without money); including because:
  • there will be no more incentive for states, companies, or other institutions to care about humans
  • it will be harder for humans to achieve outlier outcomes relative to their starting resources
• Radical equalising measures are unlikely

Overall, this points to a neglected downside of transformative AI: that society might become permanently static, and that current power imbalances might be amplified and then turned immutable.

There are two levels of the standard solution to the resulting unemployment problem:

• Governments will adopt something universal basic income (UBI).
• We will quickly hit superintelligence, and, assuming the superintelligence is aligned, live in a post-scarcity technological wonderland where everything is possible.