(2023-11-29) Willison Llamafile Is The New Best Way To Run A Llm On Your Own Computer

Simon Willison: llamafile is the new best way to run a LLM on your own computer. On my M2 Mac it runs at around 55 tokens a second, which is really fast

*Mozilla’s innovation group and Justine Tunney just released llamafile, and I think it’s now the single best way to get started running Large Language Models (think your own local copy of ChatGPT) on your own computer.

A llamafile is a single multi-GB file that contains both the model weights for an LLM and the code needed to run that model—in some cases a full local server with a web UI for interacting with it.*