CLIP-Powered Multi-Modal Search with Redis Vector Index and Graph

This is a submission for the Redis AI Challenge: Real-Time AI Innovators.

What I Built

I built a real-time, multi‑modal semantic search system that combines vector similarity search with graph‑based context expansion. Text and images are embedded locally using CLIP (openai/clip-vit-base-patch16, 512‑dim). Redis 8 powers the vector index (cosine distance) and fast KNN lookups, and also serves as a cache for query results. On top of the nearest neighbors, I construct and traverse a semantic graph using NetworkX in Python to discover related items beyond the initial top‑K—enabling richer, more explainable retrieval.

Key features:

• Local CLIP embeddings for text and images
• Redis 8 vector index with cosine similarity + KNN search
• Result caching in Redis for low latency and reduced recompute
• NetworkX semantic graph to link and rank related items across searches
• Duplicate prevention via SHA‑256 content hashing
• Endpoints for submit (ingest) and search.

Demo

• Screenshots:

How to try:

• Submit text or an image.
• The system embeds the input, stores/updates it, and runs a KNN search.
• Results are cached in Redis; a NetworkX graph expands context to suggest related items.

How I Used Redis 8

I used Redis 8 as the real-time data layer for indexing, search, and caching:

• Vector Index (RediSearch): I created a VECTOR HNSW index on a hash field named embedding (DIM=512, DISTANCE_METRIC=COSINE). All embeddings are stored as raw float32 bytes. KNN queries use FT.SEARCH with PARAMS to pass the query vector and SORTBY vector_score.

• Semantic Caching: I cache search responses keyed by a normalized query hash (e.g., sha256 of lowercased text). Hot queries return instantly from Redis without re‑embedding or re‑searching, with a short TTL to ensure freshness.

• Metadata and Documents: Each item is stored as a Redis hash with fields like embedding (bytes), type (text/image), content or URL, and submit_count. This allows quick retrieval for UI assembly.

• Deduplication: Before storing, I normalize and hash the input content. If the key exists, I update usage stats instead of duplicating vectors or nodes.

• Performance: Redis delivers sub‑millisecond lookups on cached results and fast vector KNN even at scale thanks to HNSW. Caching also reduces load on the local embedding server and keeps latency predictable.

solo Project by: @leoantony72

leoantony72 / multi_model_vectorSearch

Multi Model vector search with Redis

Real-Time Multi-Modal Semantic Search System

This project is a real-time, multi-modal semantic search system that combines vector similarity search with graph-based context expansion.
It supports both text and image search using locally computed CLIP embeddings.

---

📜 What I Built

• Local CLIP embeddings for text and images (openai/clip-vit-base-patch16, 512-dim)
• Redis 8 vector index with cosine similarity and KNN search
• Result caching in Redis for low latency and reduced recompute
• NetworkX semantic graph to link and rank related items beyond the initial top-K → enables richer, more explainable retrieval
• Duplicate prevention using SHA-256 content hashing
• Endpoints for
  • submit (ingest)
  • search (retrieve)

---

🛠 Architecture Overview

1. Embed: Text and images embedded locally via CLIP
2. Index: Vectors stored in Redis 8 (cosine similarity)
3. Search: Fast KNN lookups in Redis
4. Expand: Related items discovered via semantic graph traversal in NetworkX
5. Cache: Query results cached in Redis
6. Serve…

View on GitHub

Suggestions

As the development of AI increases, getting relations faster is a major step, While I was sad knowing that redis stopped support for redis graph. I encourage the redis Team to bring back support for Graph.