(2023-02-26) Litt Chatgpt As Muse Not Oracle

Geoffrey Litt: ChatGPT as muse, not oracle. Recently lots of people have been trying very hard to make large language models like ChatGPT into better oracles—when we ask them questions, we want the perfect answer.

As an example, in my last post, I explored some techniques for helping LLMs answer complex questions more reliably by coordinating multiple steps with external tools. I’ve been wondering, though, if this framing is missing a different opportunity

What if we were to think of LLMs not as tools for answering questions, but as tools for asking us questions and inspiring our creativity?

One inspiration for this line of thinking was Gordon Brander‘s insightful writing on provoking creativity on-demand. He points out that even simple tools can lead to interesting results when they clash with the contents of our minds. ((2022-01-15) Brander Creativity Can Be Provoked On-Demand)

I’ve also been intrigued by Linus’s articulation of AI as a collaborator ((2021-11-16) Ai As A Creative Collaborator Thesephistcom), and tools like Fermat.ws that aim to apply AI as a creative thought partner.

So, below is the transcript of a short conversation I had with ChatGPT about this topic.

The current unique advantage of LLMs seems to be generating lots of options quickly, moreso than ensuring high quality for any given one

I wasn’t too impressed by the results of prompting it to be “interesting”. Most of its answers still felt generic and cliche; it didn’t convincingly mimic an expert

Reflections

I wouldn’t say this was anything close to a replacement for a conversation with a good human conversation partner about this topic. But that’s also not the point. While conversing with the model, I found it far easier to stay focused and keep thinking about the problem than if I had just been writing notes on my own without any inspiration or feedback.

maybe it’s more useful to think of the LLM in this case as a supercharged Oblique Strategies deck: a simple tool that draws random connections and makes it easier to keep going.

Typing my thoughts felt like a speed bottleneck; I wonder how this interaction would have felt if it was conducted over voice. On the other hand, writing, as always, clarified my thinking

chat worked fairly well here as an interaction paradigm; the conversational metaphor felt natural for spooling out an idea. But I’m curious about other paradigms as well; Fermat.ws has interesting ideas about a spatial canvas, for example.