Vector Podcast: Simon Eskildsen, Turbopuffer

#vectordatabase #rust #objectstorage #searchengine

Vector database and search engine behind well known tools: Cursor, Notion, Linear, Superhuman, Readwise

Vector Podcast is back — Season 4 starts now!

I’ve had a pleasure to interview @sirupsen, the creator of Turbopuffer — vector database and a search engine. Prior to co-founding his startup, Simon has spent a decade at Shopify scaling the stateful engines, like Elasticsearch, Redis, MySQL at an immense scale (up to 1M reads / sec).

If you want to create a generational DB company, I think you need 2 things: a) you need a new workload — the new workload here is that we have almost every company on Earth, sitting on their treasure trove of data and they want to connect that to LLMs (large scale analytics on unstructured data) b) new storage architecture — if you don’t have a new storage architecture, that is fundamentally a better trade-off for the particular workload, then there is no reason why tacking on a secondary index to your relational database, to your OLAP, to your existing search engine [would not work] — they would eat it.

Simon Eskildsen on Vector Podcast with Dmitry Kan

Turbopuffer uses Object Storage (S3, GSC, Blob Storage) to store client data and vectors (encrypted using customer keys), and RAM and NVMe SSDs for cache of data your application actually uses. This is where the analogy to puffer fish finds its place: the databases inflates and deflates the cache, depending on the usage. To lower the latency towards Object Storage, Turbopuffer minimizes the number of roundtrips to 3–4 (400ms), and uses techniques like Range Fetch (available in AWS, GCP, Azure) to accelerate the cold read.

Turbopuffer architecture: https://turbopuffer.com/docs/architecture

For vector search, SPFresh algorithm is used, which is based on centroid index. This reminded me of the work I did as part of team Sisu implementing KANNDI algorithm for billion-scale ANN benchmarking competition. It is interesting, that centroid-based algorithms are natural for Object Storage as they minimize round-trips and allow for collocated writes. Looks like my intuition was right, and I’m glad someone has productized this!