Vector Search Analysis - Google BigQuery vs Azure AI Search

Introduction

As more people embrace generative AI solutions, search technology has become a major focus area. Traditional search methods relied on text matching and some fuzzy logic, but generative AI has introduced vector search capabilities. This new approach significantly enhances traditional methods by adding contextual search, greatly improving the natural language search experience for users.

In this blog, we'll share our live project experience using Google BigQuery's vector search capabilities and Azure AI's search capabilities. We'll discuss our findings on how both tech stacks perform for image-related semantic search and deduplication use cases.

Analysis Details

Test Data

The test data used for this analysis included a set of images and semantically similar images. This data was prepared as part of a human rubric. The idea was to use this reference test data of similar images to evaluate how well the two tech stacks perform in terms of similar search criteria.

Criteria

The criteria used for this analysis included popular search algorithms like Euclidean distance and Cosine similarity. While other algorithms like dot product and HNSW are also available in the vector search domain, we focused on the first two as they are the most widely used for this type of use case.

• Euclidean distance (https://en.wikipedia.org/wiki/Euclidean_distance)
• Cosine similarity (https://en.wikipedia.org/wiki/Cosine_similarity)
• Dot Product (https://en.wikipedia.org/wiki/Dot_product)
• HNSW (https://en.wikipedia.org/wiki/Hierarchical_navigable_small_world)

Results

We conducted the experiment on over 50 similar images, varying in terms of image quality, composition, theme, number of items, and lifestyle aspects.

Conclusion

We found that for image search criteria, BigQuery-based search was closer to human search comparisons compared to Azure search. While the difference wasn't significant, BigQuery search almost always had a better or closer resemblance to human search results. It's also important to note that both platforms continue to evolve, and these results are subject to change.