Text and advanced search queries with mongodb-object-filter-parser

Andrés Gerardo López Almazo — Fri, 08 Jan 2021 15:43:11 +0000

I want to start this post with one phrase: I wasn't aware how tedious can be define a filter for MongoDB queries.

I´m a newbie in the world of real programming, recently I finished my studies at Computers Systems Engineering, and since few months I'm working in an internship project that requires web services development using GraphQL to connect a MongoDB database using JavaScript (all technologies I've never used). Initially, it was expected to contain only two queries per service; a service for query all docs in the collection and one to query by an ID or name.

The first one was pretty easy, there no be complications; the second in a similar way wasn't difficult, but when my assessor asked to me if there be possible to make queries using any other field of the collection, that's when my journey began.

First I consider the possibility of use another input parameters, but that just can make more complicated the queries and to define logic. That wasn't a really god idea.

So after a little discussion with myself and read some questions on StackOverFlow in order to discover the MongoDB operator $text, I consider to use only one input parameter, this was called "search". This parameter can accept a String that contains all words that will be queried and also can use Logic Operators (AND, OR, NOT).

Now I only had to convert that string in a valid-syntax for $text operator, that wasn't hard, but there was another problem: the operator has limitations like no accept substrings in many cases.

In order to obtain a solution to that problem I decide to develop a NPM package to make possible to others developers that can be have trouble with the same situation adding the search function that I had and the correct solution to make textSearch with fullMatch or substrings and, why not?, an advanced search function to make it available throug NPM (besides that the other solution I found was ElasticSearch).

After a little of others troubles, mongodb-object-filter-parser is in a estable version

This module can be used in projects which use connections to MongoDB through an ODM, all functions can build a mongodb bson object in order to use in .find() functions with only send an correct-syntax String.

Search and textSearch

This functions are similar in construction and logic, the difference is in the final object and the way to query, search needs an text index, in textSearch the index is optional by the way, is required a string array that contains all fileds to be queried. The syntax used in this functions is:

(NOT) Arg1 AND|OR|NOT Arg2…

To obtain the filter you have to use:

import { searchFilter, textSearchFilter } from 'mongodb-filter-object-parser';
const testString = 'NOT blue AND tall OR short';
const simpleFilter = searchFilter(testString);
const textFilter = textSearchFilter(testString,["object.color", "object.height"]);
//use trough mongodb in a ODM as mongoose
const simpleQuery = Collection.find(simpleFilter);
const textQuery = Collection.find(textFilter);

As output we can obtain the following filters

search:

{
   "$text": {
      "$search": " -blue \"tall\""
   }
}

textSearch:

{
   "$and": [
     {
       "$or": [
         { "object.color": /tall\b/i },
         { "object.height": /tall\b/i }
       ]
      },
    {
      "$or": [
         { "object.color": /blue\b/i },
         { "object.height": /blue\b/i }
       ]
     }
   ]
}

advancedSearch

This type of filters are most exclusive or usable for more experienced users or people with high knowledge of the collection. At the same way that the previous functions, need a specific syntax in order to use them.

(NOT) (Value[Field]) OR|AND|NOT Value1|Value2[Field]...

This queries are resolved with a simple version of binary search tree that only can grow to the left at this moment. As an example I can show:

((Green[Color] OR Blue[Color]) OR Red[Color]) AND Medium[Size]

This string can be seen like:

Finally this can be parsed with the following code:

import { advancedSearchFilter } from 'mongodb-filter-object-parser';
const testString = '((Green[Color] or Blue[Color]) or Red[Color]) and Medium[Size]';
const advancedFilter = advancedSearchFilter(testString);
//use trough mongodb in a ODM as mongoose
const query = Collection.find(advancedFilter );

Filter obtained

{
  "$and": [
    {
      "Size": "Medium"
    },
    {
      "$or": [
        {
          "Color": "Red"
        },
        {
          "$or": [
            {
              "Color": "Green"
            },
            {
              "Color": "Blue"
            }
          ]
        }
      ]
    }
  ]
}

Out of scope

At this moment following cases are not possible to apply in advancedSearch

Queries with range are not available support currently
Binary Search Tree cannot grow to both sides only to the left

This is my first module that I upload to NPM so I am very excited about it and hope someone can be helpful. I would also like to know your opinions about him, I know that I still have a lot to improve but I am passionate about this world. Thanks for reading

DEV Community: Andrés Gerardo López Almazo

Text and advanced search queries with mongodb-object-filter-parser

Search and textSearch

advancedSearch

Out of scope