Toast 1: A New Embedding Model That Rivals OpenAI at a Fraction of the Cost

Toast 1: A New Embedding Model That Rivals OpenAI at a Fraction of the Cost

Mixedbread AI announced Toast 1, a new embedding model that claims to match or exceed OpenAI's text-embedding-3-large on standard benchmarks while being significantly cheaper to run. The announcement reached 173 points on Hacker News with 58 comments.

What Are Embedding Models?

Embedding models convert text into dense numerical vectors — arrays of floating-point numbers that capture semantic meaning. These vectors power:

• Semantic search: Finding documents that mean the same thing, even if they use different words
• RAG (Retrieval-Augmented Generation): Giving LLMs access to external knowledge by retrieving relevant documents
• Clustering: Grouping similar documents together
• Classification: Categorizing text based on semantic similarity to known examples
• Recommendation systems: Finding similar items based on description

If you're building any AI application that needs to search through documents, you're probably using an embedding model. The quality of that model directly determines how good your search results are.

What Makes Toast 1 Different

Toast 1's key innovations include:

1. Training data quality: Mixedbread focused on data quality over quantity, using careful curation and filtering rather than just dumping the entire web into training
2. Architecture improvements: Modifications to the transformer architecture that improve semantic understanding
3. Multilingual support: Strong performance across 50+ languages without needing separate models
4. Variable dimensionality: Support for different output dimensions (128, 256, 512, 768, 1024, 1536) so you can trade off quality vs. storage cost
5. Matryoshka embeddings: The ability to truncate the embedding vector to a smaller size while maintaining most of the semantic information

Benchmark Results

On the MTEB (Massive Text Embedding Benchmark), Toast 1 reportedly scores:

• Overall: 68.4 (vs. OpenAI text-embedding-3-large at 67.9)
• Retrieval: 58.2 (vs. OpenAI at 57.3)
• STS (Semantic Textual Similarity): 84.1 (vs. OpenAI at 83.2)
• Classification: 75.8 (vs. OpenAI at 75.1)
• Reranking: 62.3 (vs. OpenAI at 61.8)

These are marginal improvements, but the point isn't to crush OpenAI on benchmarks — it's to match them at a lower cost.

The Cost Equation

Embedding costs add up quickly at scale. If you're embedding millions of documents for a search system:

• OpenAI text-embedding-3-large: $0.13 per 1M tokens
• Toast 1: Mixedbread's pricing hasn't been finalized, but they've indicated it will be significantly cheaper
• Self-hosted: If you self-host Toast 1 (it's open-weight), the cost is just your compute

For a system processing 100M tokens per month, that's the difference between $13,000/month and potentially $1,000-3,000/month with self-hosting.

Why This Matters for RAG

Retrieval-Augmented Generation has become the standard pattern for building AI applications with custom knowledge. The quality of your RAG system depends heavily on the quality of your embeddings:

1. Recall: Can you find all relevant documents? Better embeddings = higher recall
2. Precision: Are the documents you find actually relevant? Better embeddings = higher precision
3. Latency: Faster embedding generation = faster search. Self-hosted models eliminate network round-trips
4. Cost: Lower embedding costs = more documents you can afford to index

Toast 1's combination of quality and cost makes it particularly attractive for RAG systems.

Self-Hosting Considerations

If you want to self-host Toast 1:

• Model size: Similar to other large embedding models (~1-2GB)
• GPU requirement: Can run on a single consumer GPU (8GB+ VRAM) for inference
• CPU inference: Possible but slower — suitable for batch processing, not real-time
• Integration: Compatible with standard embedding APIs (OpenAI-compatible, LangChain, etc.)

The Competitive Landscape

Toast 1 enters a crowded embedding model market:

• OpenAI text-embedding-3-large: The default choice for many developers
• Cohere embed-v3: Strong multilingual performance
• Voyage AI: Specialized for code and domain-specific embeddings
• Nomic AI: Open-source embedding models
• BGE (BAAI General Embedding): Open-source, competitive on benchmarks
• Jina embeddings: Good for long-context embeddings

What sets Toast 1 apart is the combination of open weights, multilingual performance, and Matryoshka dimensionality support. The ability to use a single model at different dimensionalities is particularly useful — you can use 1536 dimensions for your main index and 256 dimensions for a fast approximate index.

The Bottom Line

Embedding models are the foundation of most AI applications, and the market is moving toward open, cost-effective alternatives to OpenAI. Toast 1 represents another step in that direction. If you're building RAG systems, semantic search, or any application that uses embeddings, it's worth benchmarking against your current model.

---

Based on Mixedbread's announcement (173 points on Hacker News).