MongoDB 8.2's encrypted substring search costs 20 times more per insert than equality

I inserted 200 documents into a MongoDB collection with one encrypted, substring-searchable field and it took 4.6 seconds. The same 200 documents into a collection with only encrypted equality search took 232 milliseconds. Plaintext took 7 milliseconds.

MongoDB 8.2 added public preview support for prefix, suffix and substring queries against Queryable Encryption fields, through three new aggregation operators: $encStrStartsWith, $encStrEndsWith and $encStrContains. Until now, Queryable Encryption could only do equality and range lookups on an encrypted field — nothing that resembles a LIKE '%foo%'. I built a small encrypted collection, indexed an email field for substring search, and measured what that actually costs against the same field indexed for plain equality, and against no encryption at all.

Setup

I ran MongoDB 8.2.12 (docker run mongo:8.2, Community Edition, confirmed via db.runCommand({buildInfo:1}).modules returning an empty array — no Enterprise module) as a single-node replica set, which Queryable Encryption requires. Automatic encryption needs a query-analysis component; I used the crypt_shared shared library, downloaded directly from MongoDB's public download server with no login or Enterprise license involved. The client was PyMongo 4.18.1 with pymongo[encryption] and a local (non-production) KMS key. This is a public preview feature — MongoDB's own docs say not to use it in production, and preview functionality will be incompatible with the eventual GA version.

One thing worth correcting up front: I'd read that automatic Queryable Encryption needs MongoDB Enterprise. It doesn't. The crypt_shared library is a free download, and the whole setup above ran against stock Community Edition without complaint.

What it costs

I built three versions of the same collection, differing only in how the email field is indexed, inserted the same 200 generated documents into each (three trials, fastest reported), and compared to an unencrypted collection:

Configuration	Insert cost (ms/doc, fastest of 3)	Avg document size
Plaintext, no encryption	0.03	133 bytes
Queryable Encryption, equality only	1.16	489 bytes
Queryable Encryption, substringPreview	22.9	24,129 bytes

The equality number already reflects the tax you already know about with Queryable Encryption: roughly 35x the insert cost and 3.7x the document size of plaintext, because every write also has to touch the hidden ESC and ECOC metadata collections that back the encrypted index. Substring search pays that same tax and then a much larger one on top: 20x the insert cost of equality-only encryption, and storage that dwarfs both.

For a 500-document run I measured the full picture, including the metadata collections:

Configuration	Main collection	ESC	ECOC	Total for 500 docs	Per doc
Plaintext	66.5 KB	—	—	66.5 KB	133 B
Equality QE	244.5 KB	23.5 KB	51.5 KB	319.5 KB	639 B
substringPreview QE	12.06 MB	4.07 MB	9.17 MB	25.3 MB	50.6 KB

That's 380 times the storage of plaintext, and 79 times the storage of the same field encrypted for equality only. The reason is visible in the ECOC collection: it held 86,500 entries for 500 documents, or about 173 encrypted tokens per email address. strMaxLength was set to 40 and the query length range to 3–10 characters, and MongoDB has to generate an indexable token for every substring in that length range so any of them can later be matched without decrypting the field. A 25-character email address has roughly that many valid substrings between 3 and 10 characters long, so the token count checks out.

What it refuses

The field-length and query-length bounds are enforced, not advisory. Inserting a 47-character email into a field configured with strMaxLength: 40:

EncryptionError: StrEncode: String passed in was longer than the maximum
length for substring indexing -- String len: 47, max len: 40

Querying with a 2-character substring against a field configured with strMinQueryLength: 3:

EncryptionError: StrQuery: string value was shorter than the minimum query
length for this field after folding -- folded codepoint len: 2, min query len: 3

Both fail on the client, before anything reaches the server. So does a plain $regex against an encrypted field, run through the same encrypted client — it doesn't silently return nothing, it's rejected outright: Invalid match expression operator on encrypted field 'email'.

The sharper limitation is in how many query types one field can carry. I first tried configuring email with all three preview types — substring, prefix and suffix — on the theory that one indexed field should support all three lookup styles. MongoDB refused:

OperationFailure: The number of query types for an encrypted field cannot
exceed two

Dropping to two types, substring plus prefix, still failed:

OperationFailure: Multiple query types may only include the suffixPreview
and prefixPreview query types

So substringPreview cannot be combined with anything, not even equality, on the same field. The only legal pair is prefix and suffix together. I set that up — email configured with both prefixPreview and suffixPreview — and it worked: $encStrStartsWith and $encStrEndsWith both ran correctly against it, returning 61 and 113 matches respectively, sane numbers for the test data. That combined field cost 4.3 ms/doc to insert — cheaper than substring alone, but still 130x the 0.03 ms/doc plaintext baseline — and averaged 2,526 bytes per document, about a tenth of what substring alone needed. If your schema needs both prefix/suffix search and substring search on the same logical value, today that means two separate encrypted fields carrying the same plaintext, because one field can't carry both.

Query latency

Against 500 documents, $encStrContains for the string "acme" (112 matches) took a fastest time of 25.3 ms across five runs. The equivalent plaintext $regex query against the same data took 1.0 ms. Both returned exactly 112 documents, so correctness wasn't in question, only cost — roughly 25x slower on a dataset barely large enough to notice.

What I got wrong on the way

My first concurrency test used a Python thread pool: ten threads, each running the same $encStrContains query concurrently through a shared PyMongo client. Single-threaded, the query took 28.7 ms. At ten concurrent threads, the average jumped to 485.7 ms — a 17x slowdown that looked like severe server-side lock contention on the encrypted index.

It wasn't. Query analysis for these operators happens client-side, inside crypt_shared, and that work is CPU-bound. Python threads share one interpreter lock, so ten threads doing CPU-bound cryptographic work serialize against each other regardless of what the server is doing. I reran the same test using separate OS processes instead of threads, each with its own client and no GIL to contend over. The baseline (including per-process client setup) was 34–43 ms, and at ten concurrent processes the average was 87.2 ms — about 2x, not 17x.

The lesson isn't about MongoDB, it's about benchmarking anything that does client-side crypto from Python: a thread-pool concurrency test measures your interpreter's lock contention as much as it measures the database.

Watching it in production

db.serverStatus().fle reports live counts of which query types are in use across encrypted fields on the deployment:

indexTypeStats: {
  unindexed: 2,
  equality: 1,
  substringPreview: 1,
  suffixPreview: 0,
  prefixPreview: 0,
  ...
}

That's the one place I found to check, at a glance, whether anyone has put a substringPreview field into a cluster — useful given the preview warning against doing that in production. The same section also reports compactStats and cleanupStats for the ESC and ECOC collections, which is where you'd watch the housekeeping that keeps those metadata collections from growing forever as documents are updated or deleted.

Run it yourself

This needs Docker and Python 3.11+.

docker run -d --name mongoqe --network host mongo:8.2 --replSet rs0 --port 27117 --bind_ip_all
sleep 4
docker exec mongoqe mongosh --port 27117 --quiet \
  --eval 'rs.initiate({_id:"rs0", members:[{_id:0, host:"localhost:27117"}]})'

curl -sL -o crypt_shared.tgz \
  "https://downloads.mongodb.com/linux/mongo_crypt_shared_v1-linux-x86_64-enterprise-ubuntu2204-8.2.12.tgz"
mkdir crypt_shared && tar xzf crypt_shared.tgz -C crypt_shared

python3 -m venv venv && source venv/bin/activate
pip install "pymongo[encryption]"

Then, in Python:

import os
from pymongo import MongoClient
from pymongo.encryption import ClientEncryption, AutoEncryptionOpts

uri = "mongodb://localhost:27117/?replicaSet=rs0&directConnection=true"
key = os.urandom(96)
kms_providers = {"local": {"key": key}}

client = MongoClient(uri)
ce = ClientEncryption(kms_providers, "encryption.__keyVault", client, client.codec_options)

fields = {"fields": [{
    "path": "email", "bsonType": "string",
    "queries": [{"queryType": "substringPreview", "contention": 0,
                 "strMaxLength": 40, "strMinQueryLength": 3, "strMaxQueryLength": 10,
                 "caseSensitive": False, "diacriticSensitive": False}],
}]}
coll, _ = ce.create_encrypted_collection(client["qedemo"], "users", fields, "local")

auto_opts = AutoEncryptionOpts(kms_providers, "encryption.__keyVault",
                                crypt_shared_lib_path="crypt_shared/lib/mongo_crypt_v1.so")
ecoll = MongoClient(uri, auto_encryption_opts=auto_opts)["qedemo"]["users"]
ecoll.insert_one({"email": "[e-mail]"})

print(list(ecoll.aggregate([
    {"$match": {"$expr": {"$encStrContains": {"input": "$email", "substring": "acme"}}}},
    {"$project": {"__safeContent__": 0}},
])))

That last $project matters: every document in an encrypted collection carries a __safeContent__ array of encrypted search tags, and without excluding it you'll get a wall of binary blobs back with your result. I verified every command above against a fresh container before publishing.

If you're evaluating Queryable Encryption for a field that needs pattern matching, measure the insert and storage cost on your own document size and access pattern before committing to it — the multiplier scales with how many substrings your strMaxLength/query-length settings force it to index, so a longer field or a wider query-length range will cost more than what I measured here, not less. And if you need more than one style of pattern lookup on the same value, budget for a second encrypted field, because MongoDB won't let one field carry substring search alongside anything else.