Playing With Search: Stemming

This article is part of a series. Please see the complete series above.

Let's start with this "text": {"path": "name", "query": "searches"} playground.

Playground "unit test" technique

Before we get into the textual analysis stemming topic, I want to share a different way of displaying search results in a playground. When playing with textual analysis, only a single document with a string field is needed. Rather than returning an empty array or an array containing only that one document from $search, I'd rather see literally 0 (false) or 1 (true) indicating whether the document matched the query or not.

The $searchMeta stage performs the query the same as $search, but instead of returning an array of matching documents, it returns a single document of search result metadata. In this case, only the total count is included. No actual collection documents are returned from $searchMeta.

[{
  "$searchMeta": {
    "text": {
      "path": "name",
      "query": "searches"
    },
    "count": {
      "type": "total"
    }
  }
}]

Stemming example

Will the query above match this document (given the default MongoDB Atlas Search configuration)?

  {
    "_id": 1,
    "name": "The Atlas Search Playground"
  }

No, it doesn't (count of 0):

[
  {
    "count": {
      "total": 0
    }
  }
]

The default mapping for string fields uses the lucene.standard analyzer. The standard analyzer is language-agnostic, but Unicode "word" aware, and tokenizes the name field into these lowercased terms: "the", "atlas", "search", "playground". The query of "searches" does not exactly match the indexed term "search", so the document does not match the query.

It would be better if this query would match that document, given the words "search" and "searches" are the same root word in English. Setting the analyzer to lucene.english brings in English-specific handling including stemming words to their "word stem". In this case, both "search" and "searches" (the content and the query) are analyzed to "search":

{
  "analyzer": "lucene.english",
  "mappings": {
    "dynamic": true
  }
}

And the document matches (count of 1):

[
  {
    "count": {
      "total": 1
    }
  }
]

Here's the successful lucene.english playground.

Head to the next article in this series.

Learn more about search fundamentals.