Embeddings: Converting Meaning into Vectors

Embeddings: Converting Meaning into Vectors

Embeddings are the bridge between human language and machine learning. They transform text into numerical vectors that capture semantic meaning—enabling semantic search, similarity matching, and AI systems that understand meaning.

What Are Embeddings?

An embedding is a numerical representation of meaning. Similar meanings cluster together in vector space.

Text: "I love programming in Java"
↓ (via embedding model)
Vector: [0.234, -0.891, 0.123, 0.456, -0.789, ...]
        (768 dimensions typical)

Text: "Java is a great programming language"
↓
Vector: [0.241, -0.887, 0.118, 0.462, -0.785, ...]
        (Very similar! Cosine similarity ≈ 0.99)

How Embeddings Work

The Magic: Semantic Similarity

Vectors of similar meaning cluster together in vector space:

"cat" •
       \ (similar meanings close)
        • "dog"

        ← "tree"
          (far from animals)

Numerically using cosine similarity:

double similarity(double[] a, double[] b) {
    double dotProduct = 0;
    double magnitudeA = 0, magnitudeB = 0;

    for (int i = 0; i < a.length; i++) {
        dotProduct += a[i] * b[i];
        magnitudeA += a[i] * a[i];
        magnitudeB += b[i] * b[i];
    }

    return dotProduct / (Math.sqrt(magnitudeA) * Math.sqrt(magnitudeB));
    // Result: 0.99 (very similar) to 0.12 (different)
}

Popular Embedding Models

OpenAI Embeddings - Premium quality

@Service
public class OpenAIEmbedder {
    private final EmbeddingClient embeddingClient;

    public double[] embedText(String text) {
        return embeddingClient.embed(text);
    }
}

Sentence Transformers - Open source, free

// all-MiniLM-L6-v2: 384-dim, fast, good quality
EmbeddingClient client = new HuggingFaceEmbedding(
    "sentence-transformers/all-MiniLM-L6-v2"
);

Cohere Embeddings - Flexible, cost-effective
E5 Models - Excellent quality, open source

Choosing an Embedding Model

Model	Dimensions	Speed	Quality	Cost	Best For
all-MiniLM-L6-v2	384	⚡⚡⚡⚡⚡	⭐⭐⭐⭐	Free	Most systems
text-embedding-3-small	1536	⚡⚡⚡	⭐⭐⭐⭐⭐	$0.02/M	Premium
E5-base	768	⚡⚡⚡⚡	⭐⭐⭐⭐⭐	Free	Balanced

Using Embeddings with Spring AI

@Service
public class EmbeddingPipeline {
    private final EmbeddingClient embeddingClient;
    private final VectorStore vectorStore;

    // Step 1: Embed documents at indexing
    public void indexDocuments(List<String> documents) {
        List<Document> docs = documents.stream()
            .map(text -> new Document(text))
            .collect(Collectors.toList());

        vectorStore.add(docs); // Embedding happens automatically
    }

    // Step 2: Embed query at search time
    public List<Document> semanticSearch(String query) {
        return vectorStore.similaritySearch(query, 10);
    }
}

Batch Embedding (Critical for Production)

public class BatchEmbedder {
    public void embedDataset(List<String> texts, int batchSize) {
        for (int i = 0; i < texts.size(); i += batchSize) {
            List<String> batch = texts.subList(i, 
                Math.min(i + batchSize, texts.size()));

            // Batch API calls for efficiency
            double[][] embeddings = embeddingClient.embedBatch(batch);
            storeBatch(batch, embeddings);
        }
    }
}

Embedding Quality Factors

1. Model Architecture - Transformer-based is standard
2. Training Data - Domain-specific models better than general
3. Input Length - Most models limit to 512-4096 tokens
4. Normalization - L2 normalization helps with similarity

Cost Optimization

public class CostOptimizer {
    private Map<String, double[]> cache = new ConcurrentHashMap<>();

    // Cache embeddings to avoid recomputation
    public double[] getEmbedding(String text) {
        return cache.computeIfAbsent(text, key -> 
            embeddingClient.embed(text)
        );
    }

    // Use smaller model for filtering, expensive for ranking
    public List<Document> tieredSearch(String query) {
        double[] embedding = smallModel.embed(query); // Fast
        List<Document> candidates = vectorStore.search(embedding, 1000);
        return rerank(candidates, query); // Expensive reranking
    }
}

Best Practices

1. Match model to use case - Don't over-engineer
2. Batch operations - Never embed one-by-one
3. Cache aggressively - Recomputing wastes money
4. Monitor quality - Track retrieval accuracy
5. Version your models - Track which model generated which embeddings
6. Normalize consistently - Use same normalization always

Multimodal Embeddings

Text and image embeddings in same space:

public class MultimodalEmbedding {
    public void indexContent(String text, BufferedImage image) {
        double[] textEmbed = textEmbedder.embed(text);
        double[] imgEmbed = imageEmbedder.embed(image);

        // Now find similar images for text queries
        vectorStore.store(text, textEmbed);
        vectorStore.store(image, imgEmbed);
    }
}

Conclusion

Embeddings convert meaning into computation. They're the foundation of modern semantic AI—enabling search, retrieval, and intelligent applications.

Start with proven open-source models, batch aggressively, cache heavily, and monitor quality. Scale specialty models only when general models reach their limits.