One switch in my embedding animation makes the plot look substantially different while preserving every top-five cosine neighbour.
I did not start with that as a teaching point. I had been playing with embeddings while learning about AI, first by using them to modulate sound and then by embedding code and looking for similar code. PCA and UMAP projections seemed pretty cool. At some point the next thought just popped into my head: “Huh, it'd be cool to try and animate embeddings rather than just flatten them.”
I also thought, “Oh, this is just not going to work.”
It worked, which surprised me.
The changing picture
The working Embedding Tours instrument has a deterministic eight-dimensional synthetic dataset and 180 short phrases embedded into 384 dimensions with MiniLM. You can move through raw coordinate pairs, pairs of principal components, or a seeded grand tour through projection planes that are not aligned to the raw axes.
Those are not three names for the same thing. The raw tour cycles through coordinate pairs. PCA puts the highest-variance directions first, although that does not make them the true semantic axes. The grand tour moves through more general planes.
Every frame is still a two-dimensional projection. Motion gives me more partial views; it does not recover all the information lost when hundreds of dimensions are put on a flat screen. It is also just really satisfying to watch the structure move.
The raw-coordinate mode adds a more pointed comparison. Its basis switch applies the same orthogonal transformation to the whole dataset. Individual coordinates change and the raw plot can look very different. Dot products, lengths, distances and cosines do not change under that shared rotation.
The raw projection changes after the basis switch; cosine-neighbour geometry remains fixed to floating-point precision.
In this run, the measured similarity drift is 5.55e-16 and all top-five neighbours are retained. That is visible evidence for the narrow claim I wanted the switch to make. It does not prove that every transformation preserves an embedding, and it does not mean individual axes can never be useful. It shows that the cosine-neighbour structure does not uniquely privilege the raw coordinates I happened to receive from the model.
That limitation matters when interpreting embedding coordinates. A very different-looking coordinate tour can represent the same neighbour geometry.
I had walked into an existing idea
My first version just moved from dimensions 1 and 2 towards dimensions 2 and 3, then kept going. I later learned that this belongs to an established family of visualisations called tours.
Daniel Asimov described the grand tour in 1985. The tourr project gives the useful distinction: cycling through axis-aligned views is a little tour, while a grand tour moves through general projection planes. Distill used a grand tour for neural-network activations, and the recent dtour project has a much more complete browser interface for steering through high-dimensional data.
I had not invented a new visualisation technique. I had kind of independently arrived at a known idea, or at least the entrance to one. That was probably the coolest part.
The instrument stayed brief: two datasets, three projection modes, the basis comparison, selection and playback controls. I do not expect a researcher to discover a new technique in it. Maybe it helps someone like me who is still learning and finds embeddings confusing. Maybe it does not. Maybe it is just a cool experiment.
This article was adapted with AI assistance from an original article on billiem.uk. The original article was reviewed before publication.
Top comments (0)