This was a nice read. I am going to implement this for my MEAN project. A Couple of questions.
I have a Log collection, which has 4 ObjectId type fields. i.e. client, service, executive, manager - of course apart from its own fields. And when on frontend, the Log is displayed, I am planning to populate all these 4 (i.e. Client Name, Category, Sub Category, Rating), (Service Name, Service Freq), ExecName and ManagerName.
The Log display on frontend has Search and filter options on various fields. So when a user searches or applies filters the backend Mongoose query will run again.
What would be the performance impact if the number of entries in Log collection is in the range to 5000 - 50,000?
For the above type of case would there be any other option for fetching the related data OR this is the best option.
MERN Stack Developer & Certified Neo4j Professional with over 5 years of experience. Love to learn and build new things. When not coding, you can find me exploring new tech, games & books.
Though populate is not bad. It is optimised. But there is a another way. If you know about aggregation framework in mongodb then there is a $lookup option that you can add in pipeline. I am still learning about aggregation framework. I have read that $lookup is faster than populate. Try to find more about it
Just know one thing. That populate makes a second call to database. So in your case 4 populates = 4 calls.
Or, you can do a live test :p...run query with populate and check its performance then run query with $lookup pipeline and compare its performance. (mongodb has ways to check query performance)
I hope it helps you. All the best for your project :)
MERN Stack Developer & Certified Neo4j Professional with over 5 years of experience. Love to learn and build new things. When not coding, you can find me exploring new tech, games & books.
MERN Stack Developer & Certified Neo4j Professional with over 5 years of experience. Love to learn and build new things. When not coding, you can find me exploring new tech, games & books.
Hi Paras,
This was a nice read. I am going to implement this for my MEAN project. A Couple of questions.
I have a Log collection, which has 4 ObjectId type fields. i.e. client, service, executive, manager - of course apart from its own fields. And when on frontend, the Log is displayed, I am planning to populate all these 4 (i.e. Client Name, Category, Sub Category, Rating), (Service Name, Service Freq), ExecName and ManagerName.
The Log display on frontend has Search and filter options on various fields. So when a user searches or applies filters the backend Mongoose query will run again.
What would be the performance impact if the number of entries in Log collection is in the range to 5000 - 50,000?
For the above type of case would there be any other option for fetching the related data OR this is the best option.
Though populate is not bad. It is optimised. But there is a another way. If you know about aggregation framework in mongodb then there is a
$lookupoption that you can add in pipeline. I am still learning about aggregation framework. I have read that $lookup is faster than populate. Try to find more about itJust know one thing. That populate makes a second call to database. So in your case 4 populates = 4 calls.
Or, you can do a live test :p...run query with populate and check its performance then run query with $lookup pipeline and compare its performance. (mongodb has ways to check query performance)
I hope it helps you. All the best for your project :)
$lookup cannot be sharded and so it limits your scaling, super annoying constraint because I loved this stage until I spotted that.
Okayy. thank you !! I didn't know that :)
I still have to learn more about sharding. Never tried it.
Thanks Paras for your quick response. It surely helps.
You are welcome sir !!