Plot FiftyOne visualisations with Seaborn

Why

I wanted more customisation options to draw figures for my thesis paper. It took me a while to figure this out, so I thought I'd share.

How

I'm assuming you already have a FiftyOne dataset with computed embeddings and visualization. If not, you'll need to create a dataset and compute the embeddings and visualization before proceeding.

I already have everything saved, so I load my dataset and the compute_visualization results before plotting:

import fiftyone as fo

# load dataset
dataset = fo.load_dataset("dataset_name")

# load computed visualisation
results = dataset.load_brain_results("vis_name")

I have a sample field called "vehicle_type" that I want to use as the hue in my seaborn plot. To obtain this information for each sample, I wrote a simple function:

def get_vehicle_type(sample_id):
    return dataset[sample_id]["vehicle_type"]

Next, I convert results.points into a pandas DataFrame and fetch the "vehicle_type" information from the FiftyOne dataset.

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

# turn results.points into dataframe
df_viz = pd.DataFrame(results.points, columns=["x", "y"])
# get sample ids for each sample
df_viz["sample_id"] = results.sample_ids

# use sample id to get the sample field info I need
df_viz["vehicle_type"] = df_viz["sample_id"].apply(get_vehicle_type)

Finally, I plot the results using seaborn:

sns.scatterplot(data=df_viz, x='x', y='y', 
hue='vehicle_type', palette='mako_r', 
alpha=.9, s=1, edgecolor='none')
plt.title('Image Uniqueness')
plt.axis('off')
plt.show()

Seaborn allows for greater control over the appearance of the plot. Since I don't need the plot to be interactive, this is the perfect solution for creating uniform plots for my paper.

Final result

Extra: compute embeddings and visualisation

import fiftyone.zoo as foz

# compute embeddings
model = foz.load_zoo_model("mobilenet-v2-imagenet-torch")
embeddings = dataset.compute_embeddings(model)

# pickle embeddings for later use, this the computation takes a while
with open('embeddings.pkl', 'wb') as file:
    pickle.dump(embeddings, file)

# Compute visualization
results = fob.compute_visualization(
    dataset, embeddings=embeddings, seed=42, brain_key="vis_name"
)