(2022-12-03) Shipper Linus Lee Is Living With Ai

Dan Shipper: Linus Lee Is Living With AI. Linus is an independent researcher focused on building better interfaces for people to interact with generative AI models.

It’s important research, and his output is wildly prolific because his workflow is a loop. He researches generative AI, and uses what he discovers to build AI tools that will help him think better and get more done.

Perhaps it’s this tight coupling that makes him such a good researcher. He’s both a user and a fan of these tools—and that gives him access to problems, and ways to solve them that others might miss.

Linus introduces himself:

In particular, I’m interested in interfaces that allow for direct manipulation of text using latent space based language models. Basically, what that means is that I’m interested in exploring ways to use language models that look less like using prompts and more like familiar user interfaces like pinch-to-zoom or drag-and-drop.

The ones I want to share today break down into two big categories: tools for reading, and tools for personal knowledge management—like notes and search.

Tools for reading

To find papers to read I use Elicit: A big part of research is figuring out what to read.

The papers it finds are great. it also outputs a summary of the abstract, so I can quickly tell if the paper is going to answer the question that I have. It has filters for things like the number of participants in the study or what the intervention was in the study, which is relevant for social science papers.

ExplainPaper helps me understand dense papers

In ExplainPaper I can highlight a sentence that talks about an equation or a rule, and it will summarize and explain something that might be unclea

I built a visual summarization tool for scanning through articles quickly

*CoStructure. It creates a visual heat map on top of an article to show me the most important sentences. So I can quickly jump to what’s most important without having to read the whole article.

I create the heatmaps with what’s called extractive summarization, which tries to identify sentences in the text that are most representative of the text as a whole*

If I'm interested in one part, I can click on that sentence and see what other sentences are most related to it

Text is the most ubiquitous and the least user-friendly interface to information that we have. We have plots, we have graphs, we have tables. All of these are great. They are optimized for various different kinds of things for different uses. Text is not optimized for anything at all.

Personal knowledge management (PKM) tools

I take notes in a custom app called Notation.app

It’s my daily driver notes app right now, and it’s an outliner tool like Roam or Logseq.

I built this as a response to the idea that no one should have to double-bracket every note that they take.

My conclusion is that there are definitely corner cases where it’s useful to double-bracket, but for 90% of the cases, it’s possible to do it automatically by detecting semantic similarity.

I built two personal search engines to help me find all of my data (custom search engine)

Monocle and Revery. They search over an index of all of my data—journal entries, notes, contacts, tweets, bookmarks, and more.

Monocle is a full text search engine, which makes it useful for finding proper nouns and specific keywords.

Revery does semantic search—search based on similar meaning—which is more useful for finding out what I know about a topic. I use it as a Chrome extension: if I click a button

The output of all of the tools I use in my workflow is the research results I generate.

To grok these demos, it’s useful to have a working definition of what I mean by “latent spaces” and “latent space language models.”

The question I was interested in answering with this demo is: can you build a language model with a UI that lets you move around in the latent space? And where movement in certain directions can make meaningful changes to the text?

I think these kinds of interfaces are going to be extremely important for AI to get more general adoption

Book recommendations

Diaspora by Greg Egan and Exhalation by Ted Chiang are the two books that have influenced me most deeply

Diaspora is a science fiction novel about a far-off future where humanity spans our solar system, diverging into a myriad of genetically mutated biological forms and computer-enabled virtualized forms

It follows a protagonist’s pursuit of the basic, eternal questions

Exhalation is a collection of short stories by the writer Ted Chiang, whom you might know better as the author of the story that inspired the film Arrival.