(2018-03-02) Sloan Voyages In Sentence Space

Robin Sloan Voyages in sentence space

Imagine a sentence. “I went looking for adventure.” Imagine another one. “I never returned.” Now imagine a sentence gradient between them—not a story, but a smooth interpolation of meaning.

Is it useful? Probably not! But I do know it’s interesting, and the larger artifact—the continuous sentence space—feels very much like something worth exploring.

you often find yourself embedding that data into numeric space. (Machine Learning)

How do we learn to map from "I went looking for adventure" to (-0.0036, -0.063, 0.014, …) and back?

One tool we can use is called a variational autoencoder

I tried to implement the paper myself. I failed.

Lucky me: not even a year later, another paper appeared

offered a few interesting additions to the technique and, even better: its code!!

With the code in hand, I was well on my way

But there was a persistent problem: it ran too slow

could I commission you to take a look at this sentence space project?

My project called SentenceSpace, now public on GitHub, serves up an API

• you can select a point anywhere in that space and get a (more or less) coherent sentence back.

After I’d gotten this up and running, I felt something similar to what I remembered from an earlier machine learning project: a sense of, well, I did it… now what? As before, that feeling is an important waystation

Maybe you’d rather establish a space all your own, built on sentences of your choosing. You could implement new operations; maybe you want to add sentences together, or subtract them. These spaces are dense with meaning and difficult to wrap your head around, and to me, that’s a very attractive combination.