(2022-11-04) Ink And Switch Potluck Dynamic Documents As Personal Software

Geoffrey Litt, Max Schoening, Paul Shen, Paul Sonnentag (of Ink and Switch) on Potluck: Dynamic documents as personal software. There’s a certain joy to scribbling down a note, whether in a physical notebook or a digital notebook app. We can write down information in whatever messy format we want, with no formal ceremony.

However, these days, we increasingly use purpose-built software applications to manage personal information like recipes, workouts, budgets, and schedules. (situated software, end-user programming)

this power comes at a cost: apps lose much of the freedom of a note.

We think a promising strategy for bridging the gap between docs and apps is gradual enrichment

One inspiration for gradual enrichment is spreadsheets

Potluck is a research prototype that demonstrates a workflow for gradually turning text documents into interactive software

It has three parts:

• Extensible searches: Users can define searches: custom patterns that detect data within the text of a note. Searches are defined in a compositional pattern language which allows reusing patterns that others have written.
• Live computations: Once data is extracted from the text document, users can write formulas that compute new values based on the extracted data. Formulas are written using JavaScript, in a live programming environment that resembles a spreadsheet.
• Dynamic annotations: Computed values can be displayed in the original text document as annotations. Potluck provides a few annotation types that can insert new text, cover up or restyle the original text, or even inject interactive widgets.

A key part of the Potluck interaction model is using a text document itself as an interactive interface, by showing computed values and interactive widgets in the text.

Modern commercial tools like Coda and Notion have also explored intertwining text and computation.

This idea has been explored at least as far back as 1991, in Eric Bier and Ken Pier’s work on Documents as User Interfaces at Xerox PARC. They demonstrated that if we treat specific segments of text as clickable buttons, then we can arrange them in a UI by simply moving the text to the appropriate place in the flow of the document.

Plaintext knowledge management tools like Emacs Org Mode, Obsidian, Logseq, and TaskTXT allow users to write plaintext files in a specific syntax that supports dyamic functionality, such as extracting todos and surfacing relationships between entities.

Another inspiration is Bret Victor’s reactive documents, which integrate a spreadsheet-like model into text.

In their 1998 paper Collaborative, Programmable Intelligent Agents, the Apple researchers Bonnie Nardi, James Miller, and David Wright describe data detectors... In order for a data detector to feel good to use in an interactive setting, it’s critical that the system can provide immediate feedback as a user types and help them develop consistent expectations for how the system will interpret their text.

One inspiration was Fantastical, which parses a natural language description into a structured calendar event

Another example of a fast predictable parser is Soulver, a “notepad calculator” app that instantly performs math computations over a notepad with flexible text input

People should have the ability to encode their own knowledge and personal micro-syntax into their tools. However, defining abstract patterns over plain text can be difficult even for skilled programmers

One category of approaches is to develop better languages. For example, LAPIS, by Rob Miller, developed a set of tools for specifying and extracting patterns from text by composing together existing patterns, including a friendly syntax that resembles natural language. Another route is programming by example (PBE): letting users provide concrete examples to specify a more general pattern.

On one sheet of scrap paper, someone can write a fuzzy sketch of a meal plan for the week, a grocery list, and some notes about other ongoing todos

In the digital world, we find a close analogue to this kind of flexibility in text and multimedia documents, where we can write whatever we want. Documents have a couple key advantages relative to applications.

Let’s examine some of the kinds of rigidity that applications impose on users. To ground our analysis, we’ll use some concrete examples from Paprika, a popular recipe management app that provides useful features like scaling ingredients and setting timers.

It’s worth noting there are still some important limitations and open questions that we haven’t yet resolved. While we care about enabling non-programmers, and have made some design decisions with them in mind, our current prototype does expect the user to have basic knowledge of JavaScript, and our test users have mostly been skilled programmers.

The idea of gradually enhancing a document into a software application is not new. It’s related to document-based productivity tools like Coda, Notion, Roam, and Logseq, as well as research systems including Documents as User Interfaces, Webstrates, and Smalltalk.

Let’s see an example of how it feels to use a simple Potluck document. Below is a simple recipe for making coffee, which includes an interactive slider that can scale up the number of servings

We’ve found that many different kinds of software can be built in Potluck. We’ve used it to build tools for tracking household chores, managing a cash register, organizing a meeting agenda, tracking workouts, splitting a bill, and planning a trip.