Vector Podcast episode: educational bit
Two weeks ago I had a pleasure to co-host a lightning session together with Trey Grainger on a novel idea in vector search called "Wormhole vectors".
The approach contrasts itself to a hybrid search approach.
In hybrid search you would convert the same input query into different representations (keywords -> embeddings), run independent queries, and then combine the results.
Wormhole vectors opens a new way to transcend vector spaces of different nature:
- Query in the current vector space
- Find a relevant document set
- Derive a "wormhole" vector to a corresponding region of another vector space
- Traverse to the other vector space with that query
- Repeat as desired across multiple traversals
More specifically, if you come from a sparse space, taking a set of returned documents you can pool their embeddings into one embedding and use it as a "wormhole" into the dense vector space.
If as input you are dealing with a set of embeddings from a vector search, you can traverse the Semantic Knowledge Graph (SKG) to derive a sparse lexical query best representing these documents.
Recording on YouTube:
What you'll learn:
- What are "Wormhole Vectors"?
Learn how wormhole vectors work & how to use them to traverse between disparate vector spaces for better hybrid search.
- Building a behavioral vector space from click stream data
Learn to generate behavioral embeddings to be integrated with dense/semantic and sparse/lexical vector queries.
- Traverse lexical, semantic, & behavioral vectors spaces
Jump back and forth between multiple dense and sparse vector spaces in the same query
- Advanced hybrid search techniques (beyond fusion algorithms)
Hybrid search is more than mixing lexical + semantic search. See advanced techniques and where wormhole vectors fit in.
Find this episode on many popular platforms:
Spotify: https://open.spotify.com/episode/3fvfbAGQREqCJeUciyRb3r
Apple Podcasts: https://podcasts.apple.com/fi/podcast/trey-grainger-wormhole-vectors/id1587568733?i=1000735675696
Trey Grainger - Wormhole Vecto…–Vector Podcast – Apple Podcasts
Podcast Episode · Vector Podcast · 07/11/2025 · 1h 19m
podcasts.apple.com
RSS: https://rss.com/podcasts/vector-podcast/2314900/
Trey Grainger - Wormhole Vectors | Podcast Episode on RSS.com
This lightning session introduces a new idea in vector search - Wormhole vectors!It has deep roots in physics and allows for transcending spaces of any nature: sparse, vector and behaviour (but could theoretically be any N-dimensional space).Blog post on Medium: https://dmitry-kan.medium.com/novel-idea-in-vector-search-wormhole-vectors-6093910593b8Session page on maven: https://maven.com/p/8c7de9/beyond-hybrid-search-with-wormhole-vectors?utm_campaign=NzI2NzIx&utm_medium=ll_share_link&utm_source=instructorTo try the managed OpenSearch (multi-cloud, automatic backups, disaster recovery, vector search and more), go here: https://console.aiven.io/signup?utm_source=youtube&utm_medium&&utm_content=vectorpodcastGet credits to use Aiven's products (PG, Kafka, Valkey, OpenSearch, ClickHouse): https://aiven.io/startupsTimecodes:00:00 Intro by Dmitry01:48 Trey's presentation03:05 Walk to the AI-Powered Search course by Trey and Doug07:07 Intro to vector spaces and embeddings19:03 Disjoint vector spaces and the need of hybrid search23:11 Different modes of search24:49 Wormhole vectors47:49 Q&AWhat you'll learn:- What are "Wormhole Vectors"?Learn how wormhole vectors work & how to use them to traverse between disparate vector spaces for better hybrid search.- Building a behavioral vector space from click stream dataLearn to generate behavioral embeddings to be integrated with dense/semantic and sparse/lexical vector queries.- Traverse lexical, semantic, & behavioral vectors spacesJump back and forth between multiple dense and sparse vector spaces in the same query- Advanced hybrid search techniques (beyond fusion algorithms)Hybrid search is more than mixing lexical + semantic search. See advanced techniques and where wormhole vectors fit in.
rss.com
Top comments (1)
This episode is 🔥