(2023-02-17) Sacasas The Prompt Box Is A Minefield Ai Chatbots And Power Of Language

LM Sacasas: The Prompt Box is a Minefield: AI Chatbots and Power of Language. This is a brief, rather urgent reflection on the rapidly developing field of a AI-powered chatbots. I confess these thoughts are born out of an unusually acute sense of the risks posed by these chatbots as they have been deployed

In the mid-1960s, the computer scientist Joseph Weizenbaum created the world’s first chatbot.

Weizenbaum was absolutely clear that ELIZA was not a therapist in any meaningful sense, but, despite his clarity on this point, others, including professionals in the field, reacted to ELIZA as if it were capable of replacing human therapists

These reactions shook Weizenbaum. “What I had not realized,” he later wrote, “is that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.”

ELIZA’s far more powerful and versatile descendants like ChatGPT or the new chat function on Microsoft’s search engine Bing, which is also powered by OpenAI’s GPT.

My fears are related to the compelling, whimsical, and sometimes disturbing ways that these new AI-powered chatbots can simulate conversation

But while my fears have been heightened by these developments, they are not grounded in the power of the AI tool itself or the belief that these tools are on the path to sentience. Rather, they are grounded in two far more mundane beliefs.

• First, that human beings are fundamentally social creatures
• Second, that we live in an age of increasing loneliness and isolation

what if the problem was not that normal people became subject to delusional thinking, but that lonely people found it difficult to resist the illusion that they were being heard and attended to with a measure of care and interest?

We anthropomorphize because we do not want to be alone. Now we have powerful technologies, which appear to be finely calibrated to exploit this core human desire.

As an example of both the power of such attachments and the subsequent dangers posed by their termination, consider the recent case of Replika.

When the company suspended services following a legal challenge in Italy, Samantha Cole reported that “users of the AI companion chatbot Replika are reporting that it has stopped responding to their sexual advances, and people are in crisis.” “Moderators of the Replika subreddit,” she added, “made a post about the issue that contained suicide prevention resources, and the company that owns the app has remained silent on the subject.”

As bad as such emotional experimentation at scale may be, I am more disturbed by how AI chat tools will interact with a person who is already in a fragile psychological state.

I care far less about whether an AI is sentient than I do about the fact that in certain states an AI could, bereft of motive or intention, so easily trigger or reinforce the darkest patterns of thought in our own heads.

I would speculate that weaponized chatbots deployed at scale could prove far more adept at radicalization of users than YouTube.

Another sobering possibility arises from the observation of two trajectories that will almost certainly intersect. The first is the emergence of chatbots which are more likely to convince a user that they are interacting with another human being. The second is the longstanding drive to collect and analyze data with a view to predicting, influencing, and conditioning human behavior.

The ancient sophists, however we judge them, understood that language was our most powerful tool for communication and persuasion

It seems useful to frame AI-powered chatbots as a new class of automated sophists, whose indifference to either the true or the good, indeed, their utter lack of intentions, whether malicious or benign, coupled with their capacity to manipulate human language makes them a potential threat to human society and human well-being

From this perspective, I remain foolishly committed to the idea that our best hope lies still in the cultivation of friendship and community through the practice of hospitality. “If I had to choose one word to which hope can be tied,” the social critic Ivan Illich once explained, “it is hospitality.”