Top 3 Reasons to upgrade to MongoDB 3.4

Wei Shan — Tue, 02 May 2017 16:29:40 +0000

MongoDB 3.4 was released on Nov 29, 2016 withÂ tons of new features.Â In this post, I will be listing the top 3 reasons to upgrade to MongoDB 3.4.

Faster Balancing

In the previous releases, it takes ages(4s per chunk, last I seen in 3.2.10) for the balancer to move or pre-split chunks. Imagine if you have 100,000 chunks to pre-split, it's going to take days. There are 2 reason for this:

In a sharded cluster, there can only be 1 chunk migration at any point of time. It's a single threaded operation
This.

In MongoDB 3.4,Â for a sharded cluster with n shards, MongoDB can perform at most n/2 (rounded down) simultaneous chunk migrations. So, if you have a 4 shard MongoDB cluster, it will migrate 2 chunks at a time! If you have lesser than 4 shards, the behaviour will stay the same unfortunately. :(

Linearizable Read Concern

In general, MongoDB has various readConcern option. These options allow the developers to choose different readConcern suitable for their application requirement. Do useÂ readConcern:local for unimportant queries that doesn't careÂ if it reads data that could be rolled back. Using readConcern:majority preventsÂ readingÂ data that can be rolled back. However, it does not prevent you from reading stale data in unique network partitions edge cases.

In MongoDB 3.4, usingÂ readConcern:linearizable allows you to prevent stale reads too. This featureÂ is not free and has the highest performance overhead.Â readConcern:local only require 1 operation to return your results.Â readConcern:majority requires a majority acknowledgement from a majority of the nodes in your replica set which depends on your network latency.Â readConcern:linearizable requires an additionalÂ no-op majority write to the replica set to ensure the data being read is durable and not stale.

In essence, what readConcern:linearizable does is the following:

Client sends query to primary node
Node gets the results without erroors
Node sends a noop majority write to the other nodes in the cluster to confirm that it is still the primary node.
Results gets returned to the client

Network Compression

I'm not sure why this wasn't publicise widely as a new feature. I like it a lot because, now all the traffic between mongos/mongod are compressed so we save on bandwidth. This really saves you lots of money if you are running it on AWS or if you are constrained by the network bandwidth.

Summary

In general, I do agree with infoworld that MongoDB 3.4 is a no-brainer upgrade! We are currently waiting on it to be available on our O/SÂ of choice so we can start rolling it out!

Hope this helps! :)

Regards,
Wei Shan

Compound Index Vs Multiple Single Field Index in MongoDB 3.x

Wei Shan — Sat, 08 Apr 2017 12:52:25 +0000

Compound Index Vs Multiple Single Field Index in MongoDB 3.x

MongoDB supports both compound index and single field index. It's a common knowledge that index decreaseÂ your INSERTS/UPDATESÂ performance but increases your SELECTÂ performance. However, if you needÂ to index multiple fields, is it better to use a compound index or just create multiple single field indexes? This post tries to investigate which is the better option in terms of performance.

Insert Performance

Test Setup:

Single MongoDB 3.2 instance running on my Mac
POC DriverÂ uses single thread toÂ perform inserts only
Test run for 5 mins
Empty collection after every run
Default readConcern
Default writeConcern

Sample Document:

> db.testCol.findOne()
{
 "_id" : {
 "w" : 2,
 "i" : 0
 },
 "fld0" : NumberLong(530752),
 "fld1" : ISODate("2015-06-02T08:14:18.687Z"),
 "fld2" : "tempor invidunt ut labore et",
 "fld3" : "eirmod tempor invidunt ut",
 "fld4" : "Lorem ipsum dolor sit amet,",
 "fld5" : ISODate("2016-10-12T01:58:24.580Z"),
 "fld6" : NumberLong(1712708),
 "fld7" : "sit amet. Lorem ipsum dolor",
 "fld8" : "sit amet, consetetur",
 "fld9" : NumberLong(652363),
 "bin" : BinData(0,"")
}

Phase 1 (No indexes except on _id field)

After 295 seconds, 13340672 new records inserted - collection has 13340672 in total
46498 inserts per second since last report 99.97 % in under 50 milliseconds
0 keyqueries per second since last report 100.00 % in under 50 milliseconds
0 updates per second since last report 100.00 % in under 50 milliseconds
0 rangequeries per second since last report 100.00 % in under 50 milliseconds

~45,222 inserts/second

Phase 2 (Compound index onÂ fld1Â to fld9)

After 295 seconds, 7739904 new records inserted - collection has 7739904 in total
20830 inserts per second since last report 99.72 % in under 50 milliseconds
0 keyqueries per second since last report 100.00 % in under 50 milliseconds
0 updates per second since last report 100.00 % in under 50 milliseconds
0 rangequeries per second since last report 100.00 % in under 50 milliseconds

~26,236 inserts/second

Phase 3 (Single field indexes on eachÂ fld2, fld6 and fld8)

After 295 seconds, 2998272 new records inserted - collection has 2998272 in total
7012 inserts per second since last report 63.61 % in under 50 milliseconds
0 keyqueries per second since last report 100.00 % in under 50 milliseconds
0 updates per second since last report 100.00 % in under 50 milliseconds
0 rangequeries per second since last report 100.00 % in under 50 milliseconds

~10,163 inserts/second

StorageÂ Utilisation

On the same collectionÂ withÂ 2,817,024 documents:

Compound Index

db.testCol.stats().indexSizes
{
 "_id_" : 75079680,
 "fld0_1_fld1_1_fld2_1_fld3_1_fld4_1_fld5_1_fld6_1_fl7_1_fld8_1_fld9_1" : 2617061376
}

~2.44GB

Single-Field Index

> db.testCol.stats().indexSizes
{
 "_id_" : 75079680,
 "fld0_1" : 65343488,
 "fld1_1" : 79781888,
 "fld2_1" : 35471360,
 "fld3_1" : 35475456,
 "fld4_1" : 35475456,
 "fld5_1" : 79781888,
 "fld6_1" : 65339392,
 "fld7_1" : 35475456,
 "fld8_1" : 35475456,
 "fld9_1" : 65347584,
}

~0.53GB

Index Selectivity Usage

For single-field index, there's no limitation at all. MongDB optimiser will pick and choose whichever index it deems fit. For compound index, it's slightly different.

Query like these will not use the compound index:

db.testCol.find({fld9:1,fld8:1,fld7:1})
db.testCol.find({fld9:1,fld3:1,fld6:1})
db.testCol.find({fld9:1,fld8:1,fld7:1})

Query like these will use the compound index:

db.testCol.find({fld0:1,fld1:1})
db.testCol.find({fld5:1,fld8:1,fld0:1}) (not very efficient..)
db.testCol.find({fld0:1,fld3:1,fld5:1})

The general idea is that MongoDB will use it the compound index if it matchesÂ the index prefix. It it doesn't match it correctly, it might still use the index if it is still more efficient that performing a full collection scan. The only way to find out is educate and test the queries yourself!

Summary

A very short summary in key points below:

No index has the best insert performance (duh!!)
Compound index is faster thanÂ Single-field index in insert performance
Compound index uses more storage than Single-field index

Depending on the situation, choose your poison carefully :)

References:

https://github.com/johnlpage/POCDriver
https://docs.mongodb.com/v3.2/core/index-compound/
https://docs.mongodb.com/v3.2/core/index-single/
https://docs.mongodb.com/manual/indexes/
https://www.mongodb.com/collateral/mongodb-performance-best-practices

P.S: The above test does not attempt to quantify the absolute performance numbers but rather provide a good performance degradation ratio for indexes

Regards,
Wei Shan

DEV Community: Wei Shan

Top 3 Reasons to upgrade to MongoDB 3.4

Faster Balancing

Linearizable Read Concern

Network Compression

Summary

Compound Index Vs Multiple Single Field Index in MongoDB 3.x

Compound Index Vs Multiple Single Field Index in MongoDB 3.x

Insert Performance

Test Setup:

StorageÂ Utilisation

Index Selectivity Usage

Summary