(2021-07-23) Marche The Chatbot Problem

Stephen Marche: The Chatbot Problem. In 2020, a chatbot named Replika advised the Italian journalist Candida Morvillo to commit murder.

Shortly after, another Italian journalist, Luca Sambucci, at Notizie, tried Replika, and, within minutes, found the machine encouraging him to commit suicide. Replika was created to decrease loneliness, but it can do nihilism if you push it in the wrong direction.

Ethics has never been a strong suit of Silicon Valley, to put the matter mildly, but, in the case of A.I., the ethical questions will affect the development of the technology. When Lemonade, an insurance app, announced that its A.I. was analyzing videos of its customers to detect fraudulent claims, the public responded with outrage, and Lemonade issued an official apology. Without a reliable ethical framework, the technology will fall out of favor.

Brian Christian’s recent book, “The Alignment Problem,” wrangles some of the initial attempts to reconcile artificial intelligence with human values.

Language is a thornier problem than other A.I. applications. For one thing, the stakes are higher.

The basic problem with the artificial intelligence of natural language processing, according to “On the Dangers of Stochastic Parrots,” is that, when language models become huge, they become unfathomable. The data set is simply too large to be comprehended by a human brain. And without being able to comprehend the data, you risk manifesting the prejudices and even the violence of the language that you’re training your models on

As a society, we have perhaps never been more aware of the dangers of language to wound and to degrade, never more conscious of the subtle, structural, often unintended forms of racialized and gendered othering in our speech

Let’s imagine an A.I. engineer who wants to create a chatbot that aligns with human values. Where is she supposed to go to determine a reliable metric of “human values”?

None of these safeguards will cure us of what needs curing. We are being forced to confront fundamental mysteries of humanity as technical issues.