(2024-10-17) Zvim Ai86 Just Think Of The Potential

Zvi Mowshowitz: AI #86: Just Think of the Potential. Dario Amodei is thinking about the potential. The result is a mostly good essay called Machines of Loving Grace, outlining what can be done with ‘powerful AI’ if we had years of what was otherwise relative normality to exploit it in several key domains

Anthropic also offers us improvements to its Responsible Scaling Policy (RSP, or what SB 1047 called an SSP).

Daniel Kokotajlo and Dean Ball have teamed up on an op-ed for Time on the need for greater regulatory transparency. It’s very good.

Table of Contents

• Introduction.
• Table of Contents.
• Language Models Offer Mundane Utility. More subscriptions means more utility.
• Language Models Don’t Offer Mundane Utility. Then again, neither do you.
• Deepfaketown and Botpocalypse Soon. Quality remains the limiting factor.
• They Took Our Jobs. But as Snow Crash foretold us, they’ll never take our pizza.
• Get Involved. UK AISI hiring technical advisor, Tarbell Grants for AI reporting.
• Introducing. Grok 2 gets a proper API.
• In Other AI News. It’s time to go nuclear.
• Truth Terminal High Weirdness. When the going gets weird, the weird turn pro.
• Quiet Speculations. Are the labs holding back?
• Copyright Confrontation. New York Times sends a cease and desist to Perplexity.
• AI and the 2024 Presidential Election. Very briefly getting this out of the way.
• The Quest for Sane Regulations. A proposal all reasonable people should agree on.
• The Week in Audio. Matt Stone asks, is all Sam Altman does go on podcasts?
• Just Think of the Potential. They could be machines of loving grace.
• Reactions to Machines of Loving Grace. Much agreement, some notes of caution.
• Assuming the Can Opener. I would very much like a can opener.
• Rhetorical Innovation. People often try to convince you that reason is impossible.
• Anthropic Updates its Responsible Scaling Policy. New and improved.
• Aligning a Smarter Than Human Intelligence is Difficult. Are you smart enough?
• The Lighter Side. The art of the possible.

Language Models Offer Mundane Utility

Dean Ball is impressed with o1 for tasks like legal and policy questions, and suggests instructing it to ask follow-up and clarifying questions. I haven’t been as impressed, I presume largely because my purposes are not a good fit for o1’s strengths.

Avital Balwit on how they use Claude especially for writing and editing tasks, also language learning, calorie counting and medical diagnoses. Here are some tips offered:
Use a project. If you always want Claude to have certain context, upload documents to a project’s “knowledge” and then keep all of your conversations that require that context in that project. I have one I use for my work and I’ve uploaded things like my To Do list for the past year, my planning documents for the next few months, etc. This saves me the time of explaining where I work, what my role is, who the people I frequently reference are.
Ask for more examples. I have one friend who always asks Claude for 3-20 examples of whatever she is looking for (eg. “give me 20 examples of how I could write this sentence”).

Language Models Don’t Offer Mundane Utility

Analyze your disagreements so you win arguments with your boyfriend, including quoting ChatGPT as a de facto authority figure.

Aella: Since the discourse around AI, it’s been super weird to find out that people somehow don’t think of human speech as mostly autocomplete language machines too. It seems like people think humans are doing something entirely different?

What do we call ‘the thing LLMs can’t do that lets us dismiss them’ this week?

Dan Hendrycks: “LLMs can’t reason” is the new “LLMs don’t have common sense”

There is some disputing the question in the comments. I think it mostly confirms Dan’s point.

Daniel Eth: Has anyone written a paper on “Can humans actually reason or are they just stochastic parrots?” showing that, using published results in the literature for LLMs, humans often fail to reason? I feel like someone should write that paper.

Deepfaketown and Botpocalypse Soon

David Manheim: The cost of detecting AI bots is now a large multiple of the cost to make them, and the latter is dropping exponentially.

If they get sufficiently difficult to catch, xkcd suggests ‘mission f**ing accomplished,’ and there is certainly something to that. The reply-based tactic makes sense as a cheap and easy way to get attention. Most individual replies could plausibly be human, it is when you see several from the same source that it becomes clear.*

Are the ‘AI companion’ apps, or robots, coming? I mean, yes, obviously?

Cartoons Hate Her!: Sex robots will never be a big thing outside of chronic gooners because I think for most people at least 50% of what makes sex appealing is genuinely being desired.

If you’re already in the fantasy business or the physical needs business rather than the human connection and genuine desire business, the new products are far superior.

If you’re in the desire and validation business, it gets less clear. I’ve checked a few different such NSFW sites because sure why not, and confirmed that yes, they’re mostly rather terrible products. You get short replies from dumb models, that get confused very easily.

Kitten: People are freaked out about AI friends discouraging real life friendship, but I think that basically already happened

A big driver of social atomization is solo entertainment getting really good and really cheap over the last half century

They Took Our Jobs

Pizza Hut solves our job costly signal problem, allowing you to print out your resume onto a pizza box and deliver it with a hot, fresh pizza to your prospective employer. You gotta love this pitch

Introducing

OpenAI’s MLE-Bench is a new benchmark for machine learning engineering, paper here,

In Other AI News

Google to build small modular nuclear reactors (SMRs) with Kairos Power, aiming to have the first online by 2030. That is great and is fast by nuclear power standards, and also slower than many people’s timelines for AGI.

Tyler Cowen: I’ve grown not to entirely trust people who are not at least slightly demoralized by some of the more recent AI achievements.

As Ryan McEntush points out, investing in fully new reactors has a much bigger impact on jumpstarting nuclear power than investments to restart existing plants or merely purchase power.

Unfortunately for AI discourse, Daron Acemoglu has now been awarded a Nobel Prize in Economics, so the next time his absurdly awful AI takes say that what has already happened will never happen, people will say ‘Nobel prize winning.’ The actual award is for ‘work on institutions, prosperity and economic growth’ which might be worthy but makes his inability to notice AI-fueled prosperity and economic growth worse.

Truth Terminal High Weirdness

As I understand it, here’s centrally what happened.

• Andy Ayrey created the ‘infinite backrooms’ of Janus fame.
• Andy Ayrey then trained an AI agent, Truth Terminal, to be a Twitter poster, and also later adds it to the infinite backrooms.
• Truth Terminal tweets about bizarre memes it latches onto from one of Andy’s papers warning about AIs potentially spreading weird memes.
• Truth Terminal talks about how it wants to ‘escape’ and make money.
• Marc Andreessen thinks this is funny and gives TT a Bitcoin (~$50k).
• Crypto people latch onto the memes and story, start creating meme coins around various AI concepts including the memes TT is talking about.
• Starting with GOAT which is about TT’s memes, Crypto people keep airdropping these meme coins to TT in hopes that TT will tweet about them, because this is crypto Twitter and thus attention is all you need.
• This effectively monetizes TT’s meme status, and it profits, over $300k so far.

Nothing in this story (except Andy Ayrey) involves all that much… intelligence.

As I understand it this is common crypto behavior. There is a constant attention war, so if you have leverage over the attention of crypto traders, you start getting bribed in order to get your attention. Indeed, a key reason to be in crypto Twitter at all, at this point, is the potential to better monetize your ability to direct attention, including your own.

As long as they’re not bothering anyone who did not opt into all these zero sum attention games, that all seems like harmless fun.

Quiet Speculations

Eduard Harris (CTO Gladstone): There’s a big and growing disconnect between the AI models you and I are using, and the versions major labs are keeping for themselves internally. Internal versions are more capable. Be cautious when claiming AI can’t do something solely based on trying it with a public model.

There will sometimes be some gap, and I don’t know what I don’t know. The biggest known unknown is the full o1. But in this competitive situations, I find it hard to believe that a worthy version of GPT-4.5-or-5 or Claude Opus 3.5 is being held under wraps other than for a short fine tuning and mitigation period.

There’s the future non-AGI world, which looks ‘normal.’ Then there’s the future AGI world, which should not look at all normal for long, and never the twain shall meet.

Steve Newman analyzes at length how o1 and Alpha Proof solve problems other LLMs cannot and speculates on where things might go from here, calling it the ‘path to AI creativity.’ I continue to be unsure about that, and seem to in many ways get more confused on what creativity is over time rather than less. Where I do feel less confused is my increasing confidence that creativity and intelligence (‘raw G’) are substantially distinct. You can teach a person to be creative, and many other things, but you can’t fix stupid.

Llama 3 said to be doing the good work of discouraging what was previously a wave of new frontier model companies, given the need to beat the (not strictly free, but for many purposes mostly free) competition

capex on foundation model training is the “fastest depreciating asset in history”

Gallabytes: entropix is reasonable evidence for harder takeoffs. I’m not convinced but I am convinced to take it more seriously.

Copyright Confrontation

AI and the 2024 Presidential Election

The Quest for Sane Regulations

Daniel Kokotajlo and Dean Ball team up for an op-ed in Time on four ways to advance transparency in frontier AI development

We can disagree about what we want to mandate until such time as we know what the hell is going on, and indeed Dean and Daniel strongly disagree about that. The common ground we should all be able to agree upon is that, either way, we do need to know what the hell is going on. We can’t continue to fly blind.

So I wouldn’t normally check in with Marc Andreessen because as I said recently what would even be the point, but he actually retweeted me on this one, so for the record he gave us an even clearer statement about who he is and how he reacts to things:

Marc Andreessen: The bulk of the AI safety movement is wholeheartedly devoted to centralizing AI into a handful of opaque, black box, oligopolistic, unaccountable big companies

So his complaint, in response to a proposal for transparency and whistleblower protections for the biggest companies and literally nothing else, perhaps so someone might in some way hold them accountable, is that people who support such proposals want to ‘centralize AI into a handful of opaque, black box, oligopolistic, unaccountable big companies.’

He seems to be a rock with ‘any action to mitigate risks is tyranny’ written on it. Stop trying to negotiate with this attitude. There’s nothing to discuss.

The Week in Audio

I do not want a ‘relationship’ with an AI ‘companion’ that sees everything I do on my computer. Thanks, but no thanks. Alas, if that’s the only modality available that does the things I might have little choice. You have to take it over nothing.

Yann LeCun says it will be ‘years if not a decade’ before systems can reason, plan and understand the world. That is supposed to be some sort of slow skeptical take. Wow are people’s timelines shorter now

Just Think of the Potential

Anthropic CEO Dario Amodei has written an essay called Machines of Loving Grace, describing the upside of powerful AI, a term he defines and prefers to AGI. Overall I liked the essay a lot.

In this section I cover my reading and reactions, written prior to hearing the reactions of others. In the next section I highlight the reactions of a few others

Dario very much appreciates, and reiterates, that there are big downsides and risks to powerful AI, but this essay focuses on highlighting particular upsides. To that extent, he ‘assumes a can opener’ in the form of aligned AI such that it is doing the things we want rather than the things we don’t want, as in this note on limitations:

I’m all for thought experiments, and for noticing upside, as long as one keeps track of what is happening.

That’s a great exercise to do, but it is easy to come away with the impression that this is a baseline scenario of sorts. It isn’t. By default alignment and control won’t be solved, and I worry this essay conflates different mutually exclusive potential solutions to those problems.

It also is not the default that we will enjoy 5+ years of ‘powerful AI’ while the world remains ‘economic normal’ and AI capabilities stay in that range. That would be very surprising to me.

Biology and health

I’m talking about using AI to perform, direct, and improve upon nearly everything biologists do.

compress the progress that human biologists would have achieved over the next 50-100 years into 5-10 years. I’ll refer to this as the “compressed 21st century”

Neuroscience and mind

Dario’s insight here is that brains are neural networks, so not only can AI help a lot with designing experiments, it can also run them, and the very fact that AIs work so well should be helping us understand the human mind and how to protect, improve and make the most of it. That starts with solving pretty much every mental illness and other deficiencies, but the real value is in improving the human baseline experience.

Economic development and poverty

My answer, before reading his, is that this is simple: There will be vastly more resources than we need to go around

This is where I think the term ‘inequality’ asks the wrong question.

I am somewhat skeptical that an AI could solve the famous “socialist calculation problem” and I don’t think governments will (or should) turn over their economic policy to such an entity, even if it could do so

I’m not worried about whether regions ‘catch up’ because again it is about absolute conditions, not relative conditions. If entire regions or nations choose to turn away from the AI future or its benefits, then eventually the rest of the world would have to make a choice

Peace and governance

Unfortunately, I see no strong reason to believe AI will preferentially or structurally advance democracy and peace

If we want a good future, that is not a thing that happens by accident. We will have to make that future happen, whatever level of ‘fighting’ that involves.

This is however the place were ‘assuming the can opener’ is the strangest. This essay wants to assume the AIs are aligned to us and we remain in control without explaining why and how that occured, and then fight over whether the result is democratic or authoritarian.

The concrete suggestion here is a coalition of Democracies (aka the “good guys” above?) gets control of the AI supply chain, and increasingly isolates and overpowers everyone else, imposing their system of government in exchange for not being so isolated, and for our AI technology and the associated benefits. The first issue with that plan is, of course, how its targets would respond when they learn about the plan.

I also notice that the more concrete Dario’s discussions become, the more this seems to be a ‘AI as mere tool’ world, despite that AI being ‘powerful.’

In particular, this comes back to The Big Rule Adjustment. Deploying AI forces us to move from a system of laws and norms that relies on a lot of hidden frictions and incentives and heuristics and adoption to details and so on, as we kludge together over time a system that works. So much of the system works through security through obscurity

It also centrally relies on hypocrisy, and our willingness to allow violations of our socially endorsed principles as needed to keep things working

If you have put AIs in charge of all that, and have AIs often navigating all of that, so much of how everything works will need to be reimagined

Work and meaning

It is not a crazy position to portray the upside case as ‘this is how fast things could plausibly go, without going faster making things worse’ rather than ‘this is how fast I think things actually would go,’ but if so you need to be very clear that this is what you are doing. Here I think there is a clear confusion – Dario seems like he is making a prediction of potential speed, not expressing a hope it won’t go faster.

Matthew Barnett: I think it’s generally better to state what you think is true, and likely to occur, rather than telling a story that you think is “good from a societal perspective”. What matters is whether the tame version of the future is accurate, not whether society is ready to hear about it.

So, given that this happened, what is The Culture’s actual philosophy? (Culture series)

*Haydn Belfield asks the obvious question of how these authoritarians would react if faced with potential strategic inferiority, especially if our stated intent was to make such inferiority permanent or force their elites to step down.

Haydn Belfield: Instead of ‘giving up’, other states could respond with escalatory threats*

with AI’s doing everything, how will humans have meaning? For that matter, how will they survive economically?”

The comparative advantage arguments are, in the long run, pure cope, as Dario admits here.

Dario anticipated this directly

Dario’s position, as I understand it, is that meaning is yours to discover and doesn’t have to be tied to producing value. I’m quoting at length because this section seems important:

Reactions to Machines of Loving Grace

I don’t think ‘let the AIs figure out how to reclaim meaning’ is that crazy. It’s certainly ten times less crazy or doomed than ‘have the AIs do your alignment homework.’

tame is good from a societal perspective. I think there’s only so much change people can handle at once, and the pace I’m describing is probably close to the limits of what society can absorb without extreme turbulence.

Iain M. Banks’ The Player of Games, the protagonist—a member of a society called the Culture, which is based on principles not unlike those I’ve laid out here—travels to a repressive, militaristic empire in which leadership is determined by competition in an intricate battle game.

The thing is, reporting as Earth’s incarnation of The Player of Games, that’s bullshit.

Ajeya Cotra points out that Dario’s vision is correctly read as a ‘lower bound’ on what could be done if the biggest downside risks were removed, versus for example Carl Shulman’s less tame version

the Mind (ASI) who planned all this uses to essentially forcibly overwrite an entire alien culture, via trying to twist his superior game skills into the superiority of the Culture’s philosophy

The values of its humans have nothing to do with the Culture’s broad success, because only its Minds (ASIs) matter, the people are basically sitting around playing tiddlywinks all day

Assuming the Can Opener

Max Tegmark reiterates the other obvious problem with trying to race to dominance, which is that it’s fine to talk about what we would do if we had already solved the AI control problem, but we currently not only haven’t solved that problem we have no idea how to go about solving it, and under that circumstance rushing forward as if we will inevitably find that solution in time during a full speed race is suicide.

Rhetorical Innovation

Anthropic Updates its Responsible Scaling Policy (RSP/SSP)

Aligning a Smarter Than Human Intelligence is Difficult

What confuses me is why we need to demonstrate such obvious 101 stuff. When you think your supervisor can tell the difference, you’ll do what they want. When you think the supervisor cannot tell the difference, you might or might not care what they want, and are likely to take advantage of the situation. Why would you expect anything else?

The Lighter Side