(2024-05-01) Sloan At Home In Highdimensional Space

Robin Sloan: At home in high-dimensional space. I’ve previously expressed enthusiasm about Val Town. Now, I’ve actually used it for something, and I can report that my enthusiasm has only grown. Val Town offers a lightweight web editor for TypeScript functions that can run in a variety of ways.

For me, the killer app is the built-in email handler

I think a particularly powerful option here is to pass the body over to a language model, along with a prompt explaining what kind of information you’d like to extract. In this way, an email handler val

I am an incorrigible sender of emails-to-self; it’s how I log and manage most of my notes (note-taking), including many of the items that you’ll find in this newsletter

Meta proclaims that its Llama 3 language model was “pretrained on 15T tokens from publicly available sources”. (LLM)

I have to confess, it remains strange to me that the AI folks worried, for so many years, about the composition of their training corpora — not only its legal status but also its structure, its origins and omissions … and then, starting sometime around 2021 or 2022, they simply: didn’t.

A recent edition of Jack Clark’s indispensable AI newsletter discussed the production, to this end, of synthetic data

The digital essay Models All the Way Down by Christo Buschek and Jer Thorp takes a beautiful swing at these complexities

It contains less about how humans see the world than it does about how search engines see the world. It is a dataset that is powerfully shaped by commercial logics.

What’s the value of these ever-growing AI models, really? I know several people working in this domain who believe the goal is one (1) thing of overriding consequence, which we might call virtual Wolfgang Pauli, or maybe on-demand Albert Einstein: an AI model that can actually produce pathbreaking scientific theory. I think this vision is weirdly more plausible than AI as “general labor replacement”. I suppose you could counter by saying, if they can engineer a virtual Pauli, they can FOR SURE engineer a virtual employee; one task is strictly “easier” than the other. But I don’t know if that’s true. Was Albert Einstein a good employee?

Reading histories of physics in the early 20th century, it’s thrilling to learn about the intellectual and social ferment, the rich network of “whys