WHERE item LIKE '%peripheral%' returns nothing for an order that says "Mechanical keyboard". The words don't match. The meaning does.
That gap is the whole reason semantic search exists, and Day 49 of OrderHub closes it with Spring AI: every order becomes a vector, the query becomes a vector, and the nearest vectors come back ranked. The part I care about most is that it runs — and is tested — with no API key, no network, no GPU.
What an embedding actually is
An embedding model is a function: text in, a fixed-length array of floats out. The magic isn't the numbers, it's the arrangement — texts with similar meaning come out pointing in similar directions. So "find things like this" stops being string matching and becomes geometry. You compare directions with cosine similarity: 1.0 for identical direction, 0.0 for unrelated.
Two consequences follow immediately, and both bite in production:
- The query and the documents must be embedded by the same model. Vectors from two different models live in different spaces. Compare them and you get confident nonsense — high scores that mean nothing.
- Changing models invalidates your whole index. You have to re-embed everything.
Add the library, not the provider starter
The obvious first move is spring-ai-starter-model-openai. Don't — not first.
A model starter ships auto-configuration that builds a remote client from an API key at startup. Now the app needs a credential to boot: CI needs one, every contributor needs one, and your tests either need one or have to mock the entire feature away — at which point they prove nothing about the search.
Add the small library artifact instead:
<dependencyManagement><dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bom</artifactId>
<version>1.0.0</version>
<type>pom</type><scope>import</scope>
</dependency>
</dependencies></dependencyManagement>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-vector-store</artifactId>
</dependency>
That transitively brings spring-ai-model and spring-ai-commons, which is where the interfaces that matter live: Document, EmbeddingModel, VectorStore, SearchRequest. Every line of application code you write against those is byte-for-byte the code you'd write with a paid provider — so adding one later is a dependency and a property, not a rewrite.
Implement EmbeddingModel yourself
The interface is tiny:
public interface EmbeddingModel extends Model<EmbeddingRequest, EmbeddingResponse> {
EmbeddingResponse call(EmbeddingRequest request);
float[] embed(Document document);
default float[] embed(String text) { ... }
default int dimensions() { ... }
}
Nothing stops you implementing it with classical information retrieval. The default model in OrderHub is the hashing trick:
@Override public float[] embed(String text) {
float[] v = new float[dimensions];
Map<String,Integer> tf = new LinkedHashMap<>();
for (String w : words(text)) tf.merge(w, 1, Integer::sum);
for (var e : tf.entrySet()) {
String word = e.getKey(), term = stem(word);
double weight = 1.0 + Math.log(e.getValue()); // sublinear term frequency
addFeature(v, term, weight); // the word itself
for (String c : conceptsFor(word, term))
addFeature(v, c, weight * 0.55); // keyboard -> peripheral, hardware
for (String g : trigrams(term))
addFeature(v, "gram:" + g, weight * 0.22); // typo tolerance
}
return normalize(v); // ||v|| = 1
}
private void addFeature(float[] v, String feature, double weight) {
int bucket = Math.floorMod(fnv1a(feature), dimensions);
int sign = (fnv1a("sign " + feature) & 1) == 0 ? 1 : -1;
v[bucket] += (float)(sign * weight);
}
Three details worth stealing:
- Signed hashing. Without the +1/-1 sign, every accidental collision pushes the same bucket the same way and slowly turns every vector into the same noise. With it, collisions cancel out on average.
-
FNV-1a, not
String.hashCode()semantics you're unsure of. FNV-1a is specified: the same string produces the same numbers on every JVM, forever. Never build an index on a hash whose value can change between runs. -
Concepts go in the same feature namespace as words. A concept added under a
"concept:"prefix would never meet a literal word. Put them in the same buckets and the two directions meet: the order containing "keyboard" also carries a little "peripheral", and the query asking about "typing" also carries a little "keyboard".
Be honest about the limits. A trained model learns thousands of word relationships from billions of sentences; a hand-written lexicon is a dozen of them. It's scaffolding for learning and CI — not a substitute.
Wire three swappable beans
@Configuration
@ConditionalOnProperty(prefix = "orderhub.ai.semantic-search",
name = "enabled", havingValue = "true") // default OFF
public class SemanticSearchConfig {
@Bean
@ConditionalOnProperty(prefix = "orderhub.ai.semantic-search", name = "embedding",
havingValue = "local", matchIfMissing = true)
public EmbeddingModel localEmbeddingModel(SemanticSearchProperties props) {
return new LocalEmbeddingModel(props.dimensions());
}
@Bean
public VectorStore orderVectorStore(EmbeddingModel embeddingModel) {
return SimpleVectorStore.builder(embeddingModel).build();
}
}
The store is built from the model, not handed one per call. That's deliberate — it's the mechanism that guarantees documents and queries are embedded by the same model.
SimpleVectorStore is a Map from id to content+vector plus a brute-force cosine scan. Genuinely right for a few thousand rows; genuinely wrong at a million, where you want pgvector / Qdrant / Redis with an HNSW index. That swap is one @Bean.
One subtlety on the embedding: local|provider property: it's an explicit property rather than @ConditionalOnMissingBean. User configuration is registered before auto-configuration, so a missing-bean condition here would always win and a real provider's model would silently never be used.
Indexing decides whether this is good
This is the step people underestimate. A vector store searches what you put in it, and embedding models are trained on prose — so a sentence embeds far better than a pipe-delimited row.
static String describe(Order o) {
return "Order %s placed by customer %s for %d x %s. Status: %s (%s). Placed on %s."
.formatted(o.getId(), o.getCustomer(), o.getQuantity(), o.getItem(),
o.getStatus().name(), statusPhrase(o.getStatus().name()),
DATE.format(o.getCreatedAt()));
}
private static String statusPhrase(String status) {
return switch (status) {
case "PLACED" -> "a new order, waiting to be confirmed";
case "CONFIRMED" -> "confirmed and accepted, awaiting shipment";
case "SHIPPED" -> "shipped and on its way to the customer, complete";
case "CANCELLED" -> "cancelled and refunded, no longer active";
default -> "unknown state";
};
}
Expanding the status enum into a phrase is what lets "which orders are still waiting" reach a PLACED row. If a fact isn't in this string, no query will ever find it.
Set the document id to the order id and indexing becomes idempotent — re-adding replaces instead of duplicating. Metadata rides along and is never embedded; it exists so a hit can be mapped back to a real result.
The search itself is four lines
SearchRequest request = SearchRequest.builder()
.query(query) // the store embeds THIS with the same model
.topK(topK)
.similarityThreshold(minScore)
.build();
List<Document> hits = vectorStore.similaritySearch(request);
similarityThreshold is the parameter people skip and then regret. A vector store always has a nearest neighbour, so with no floor an unrelated query still returns topK rows — ranked, scored, and completely wrong. With a floor, a query about something nobody ordered correctly returns nothing.
Clamp topK rather than reject it, and echo both effective values back in the response. Semantic search has no single right answer, so "why did I get nothing?" should be a visible number rather than a mystery.
The test that makes it worth having
Because the default model is deterministic and offline, the end-to-end test mocks nothing about the search — real model, real store, real indexer, real HTTP, real cosine scores:
@Test void matchesWordsThatAreNotInTheOrder() {
String indexedText = OrderSearchIndexer.describe(repository.findById("ai-kb-1").orElseThrow());
assertThat(indexedText.toLowerCase())
.doesNotContain("peripheral").doesNotContain("hardware"); // prove the premise
SemanticSearchResponse response = search.search("computer hardware peripheral", 5, 0.1);
assertThat(response.matches()).extracting(SemanticMatch::orderId)
.contains("ai-kb-1"); // found anyway
}
The first half is what makes the second half mean something. Without it, you've only proved that search returns rows.
22 new tests across four suites — embedding maths, the feature gate in both directions, the HTTP slice, and the full-context end-to-end. Reactor 180 → 202, mvn clean install → BUILD SUCCESS.
What this is not
This is retrieval — the first half of RAG. RAG adds a second half: stuff the top-k documents into a prompt and have a chat model write a grounded answer (ChatClient + QuestionAnswerAdvisor over this same VectorStore). That needs a real model and a real budget.
Retrieval alone is often the more valuable half anyway: cheap, deterministic, can't hallucinate, and it returns rows a user can click.
And keep your keyword search. Semantic search is worse than exact match at the thing exact match is perfect at — paste an order id and an embedding will happily return five vaguely similar rows with plausible scores. Production systems eventually run both and fuse the ranked lists (reciprocal rank fusion). You don't need that on day one. You do need to resist deleting the search that already works.
The migration path
orderhub:
ai:
semantic-search:
enabled: false # no beans, no route
embedding: local # local | provider
min-score: 0.35
Going live on a real model is a dependency, one property, and a reindex — because vectors produced by one model are meaningless to another. No application code changes. That's the entire payoff of coding against EmbeddingModel instead of a vendor SDK.
Day 50 is the capstone: full end-to-end demo and the architecture write-up. 🏁
Top comments (0)