(2023-02-07) Shipper I Built A Lenny Chatbot Using Gpt3 Heres How To Build Your Own

I built a Lenny chatbot using GPT-3. Here’s how to build your own. Lenny Rachitsky: Last month, Dan Shipper (co-founder and CEO of Every) launched a chatbot trained on the Huberman Lab podcast, and I was blown away. I pinged Dan to see what it would take to build something like this for my newsletter content. Dan walks us through the basics of AI and GPT-3, how to set up an environment to play with the API, and how to train it on your own data.

Dan Shipper: Preamble: GPT-3 vs. ChatGPT

You’d think since we’re building a chatbot we’d use ChatGPT, right? Unfortunately not. OpenAI hasn’t created a way for us to interact with the ChatGPT model directly—you can only use it through the ChatGPT web app. So it’s not suitable for our purposes

We’ll do it in two ways: using OpenAI’s Playground to start, and with a little bit of code after that. The Playground is a web app that lets you prompt GPT-3 directly and get responses back, making it a great place for us to experiment.

Below is a simple example in the GPT-3 Playground

Turning GPT-3 into a chatbot

Right now we have the bot answering questions, but how can we get it to actually chat with us?

It would be great if it sounded something like Lenny himself—warm, friendly, and smart.

That’s pretty simple to do with GPT-3 as well. We can ask it to behave in this way in our prompt:

every time we run the model, we feed it the entire transcript of what came before in the conversation, and that guides its responses

what if we want to get responses to questions that are harder to answer? For example, one of the biggest values of Lenny’s Newsletter is the amount of benchmark data he provides so that you can measure how well you’re doing against the best in the business.

Once we start really probing the bot, this kind of problem only gets bigger

answer sounds confident, but it is incorrect

There are two intertwined problems

GPT-3 tends to “hallucinate.”

GPT-3 might not have the right data.

all of the information it uses to produce its responses is frozen in 2021. Also, much of Lenny’s writing is behind a paywall

Ideally we want to feed GPT-3 the information it needs to answer questions on the fly

Stuffing context into the prompt

Let’s ask GPT-3 this question again—but let’s give it a little help. We’ll feed it the equivalent of a notecard. Written on the notecard will be the section of Lenny’s article that contains the answer

Success! Now it tells us that less than a third of founders were trying to solve their own problem. This is great—all we have to do is write all of Lenny’s posts on a little notecard

But this introduces another problem: space limitations.

The notecard analogy is apt because there’s limited space in the prompt—right now, about 4,000 tokens

We have to be very choosy about what we put in there.

Let’s talk about how to solve this.

At this point we’re going to have to move out of manual interactions with GPT-3’s Playground and start using chunks of code that work directly with the GPT-3 API.

We need to download and store Lenny’s archive in a way that makes it easily searchable for our bot. (RAG)

We need some code that will help find relevant chunks of text from the archive of Lenny’s content that we created in the previous step.

When a user asks a question, we want to use the code from the last step to get the chunks of text that are most likely to answer the question, and put them into the prompt that we send to GPT-3.

We’ll display the resulting answer to the user.

This is simple to do with a library called GPT Index, an open-source library created by Jerry Liu

The code looks like this: Screenshot

When we run this function, we’ll have created a file called index.json that contains chunks of Lenny’s articles converted into a format that makes them easily searchable. These are called embeddings—a condensed mathematical representation of each chunk of text.

embeddings cost $0.0004 for every 1,000 tokens.

To query the index we created in the last section, all we have to do is paste a question into GPT Index. It will then:

• Find the chunks of our index that are most relevant to the question
• Combine those chunks and our question into a prompt that it sends to GPT-3
• Print the output

Here’s what the code looks like

here are a few of the most relevant possibilities.

Content creators get a new format

Every newsletter, book, blog, and podcast that’s used as evergreen reference information by its audience can now be repackaged as a chatbot.

A new class of content creators will learn to create compelling chatbot experiences that combine their personality and worldview for their niche audience

Organizing your notes is over

to make sure that you remember what got said in a meeting, or what you learned from the latest book you read

Enterprise knowledge management is changing forever

Every company currently struggles with making sure all of the knowledge in their organization is written down, up to date, and searchable. In the future, chatbots could serve as a kind of automated librarian for company knowledge. (it's all still hallucinating)