Getting Started With Queryable Encryption With the MongoDB EF Core Provider

This tutorial was written by Luce Carter

With the release of MongoDB 7.0 in August 2023, MongoDB introduced a new feature called Queryable Encryption (QE), the first of its kind. With Queryable Encryption, your data is encrypted, even at rest, with the database server unable to read it either but still able to execute queries against it. You can configure what fields to encrypt so you can decide to encrypt as much or as little of your document’s fields as you need.

The great news is, not only is this available for all paid tiers and in our C# driver, but it’s now also available in the MongoDB Entity Framework Core (EF) provider.

In this tutorial, we are going to take an existing Blazor application that uses EF and the EF Core Provider, and configure it to use QE to securely encrypt certain fields. Let’s get started!

Prerequisites

In order to follow along with this tutorial, you will need a few things in place already:

• A MongoDB M10 tier or above—this is because QE is an enterprise feature and only available on our paid tiers
• Your MongoDB Atlas connection string to your cluster .NET 9 or later
• The GitHub repo forked and cloned
  • Add your connection string to the placeholder in appsettings.Development.json
  • For example, "MongoDBConnectionString": "mongodb+srv://Luce:[e-mail]/?retryWrites=true&w=majority&appName=enterprisehealthcaredotnet-efcorewithqe"

Note: If you want to see the full code, there is a branch on the GitHub repo called with-efcore-and-qe which is a working sample application. You will just need to follow the next section on adding the Automatic Encryption Library to run it locally and ensuring you add your connection string.

Adding the Automatic Encryption Shared Library

The first thing to do is add the Automatic Encryption Shared Library to the project. You can find this in our download center. Be sure to select crypt_shared from the package dropdown box.

Download the correct version for your platform and then unzip it to the root of the project. You should end up with a folder named something like mongo_crypt_shared_v1-macos-arm64-enterprise-8.2.1. The final folder name will change depending on your platform and version downloaded but the contents should be the same.

Ensure that the folder name matches the value in appsettings.Development.json.

Add NuGet package

Now we have the code set up and ready to go, it is time to add the NuGet package that contains the methods for using QE with the MongoDB EF Core Provider.

dotnet add package MongoDB.Driver.Encryption

Create helper class

The first thing we are going to do is create a helper class inside the Services folder called QueryableEncryptionHelpers.cs. This will provide code for creating and handling the encryption keys that are used to encrypt your data.

Create the new file and then paste the following code that we will discuss afterwards:

using System.Security.Cryptography;
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.Encryption;
namespace EnterpriseHealthcareDotNet.Services;

public class QueryableEncryptionHelpers
{
  private readonly IConfigurationRoot _appSettings;
  private readonly string _cryptSharedLibPath;
  public QueryableEncryptionHelpers(IConfigurationRoot appSettings)
  {
    _appSettings = appSettings;

    // Make sure your CryptSharedLibPath folder name is added to app settings, otherwise add here
    var relativeLibPath = _appSettings["CryptSharedLibPath"] ??                          "mongo_crypt_shared_v1-macos-arm64-enterprise-8.2.0/lib/mongo_crypt_v1.dylib";
    var projectRoot = Directory.GetParent(AppContext.BaseDirectory)!.Parent!.Parent!.Parent!.FullName;
    _cryptSharedLibPath = Path.GetFullPath(Path.Combine(projectRoot, relativeLibPath));
  }

  public Dictionary<string, IReadOnlyDictionary<string, object>> 
GetKmsProviderCredentials(string kmsProviderName,
        bool generateNewLocalKey)
  {
    if(kmsProviderName == "local")
    {
      if (generateNewLocalKey)
      {
        File.Delete("customer-master-key.txt");

        // start-generate-local-key
        using var randomNumberGenerator = RandomNumberGenerator.Create();
        try
        { 
          var bytes = new byte[96];
          randomNumberGenerator.GetBytes(bytes);
          var localCustomerMasterKeyBase64 = Convert.ToBase64String(bytes);
          File.WriteAllText("customer-master-key.txt", localCustomerMasterKeyBase64);
        }
        catch (Exception e)
        {
          throw new Exception("Unable to write Customer Master Key file due to the following error: " + e.Message);
        }
    // end-generate-local-key
    }

    // start-get-local-key
    // WARNING: Do not use a local key file in a production application
   var kmsProviderCredentials = new Dictionary<string, IReadOnlyDictionary<string, object>>();
   try
   {
     var localCustomerMasterKeyBase64 = File.ReadAllText("customer-master-key.txt");
     var localCustomerMasterKeyBytes = Convert.FromBase64String(localCustomerMasterKeyBase64);

     if (localCustomerMasterKeyBytes.Length != 96)
     {
       throw new Exception("Expected the customer master key file to be 96 bytes.");
     }

     var localOptions = new Dictionary<string, object>
     {
       { "key", localCustomerMasterKeyBytes }
     };

     kmsProviderCredentials.Add("local", localOptions);
     }
      // end-get-local-key
     catch (Exception e)
     {
       throw new Exception("Unable to read the Customer Master Key due to the following error: " + e.Message);
     }
 return kmsProviderCredentials;

 }

 throw new Exception("Unrecognized value for KMS provider name \"" + kmsProviderName + "\"  encountered while retrieving KMS credentials.");
 }

public AutoEncryptionOptions GetAutoEncryptionOptions(CollectionNamespace keyVaultNamespace,
        IReadOnlyDictionary<string, IReadOnlyDictionary<string, object>> kmsProviderCredentials)
{
  var extraOptions = new Dictionary<string, object>
  {
    { "cryptSharedLibRequired", true },
    { "cryptSharedLibPath",  _cryptSharedLibPath }
  };

var autoEncryptionOptions = new AutoEncryptionOptions(
keyVaultNamespace,
kmsProviderCredentials,
extraOptions: extraOptions);
// end-auto-encryption-options

  return autoEncryptionOptions;
}

}

Note: If you have not updated appsettings.Development.json with the folder name of the Automatic Encryption Library, be sure to do that or update the name in the relativeLibPath variable declaration in the constructor.

This code in the constructor builds up the path to the platform-specific file for the automatic encryption to seamlessly handle platform path differences.

Then, it handles creating an encryption key if it doesn’t already exist, or a new one has been requested, and saves it to a local .txt file. Of course, make sure you never share this file or commit it to GitHub, for security reasons! Once the key is available, it is returned to the code that called it.

There is a second method, GetAutoEncryptionOptions, which helps build up the auto encryption options that will be required later.

Note: Although outside the scope of this tutorial, this class is also where you could add code to handle other key provider options such as Azure, GCP, or AWS. You can see an example on the with-queryable-encryption branch that doesn’t use EF Core.

Update DbContext

This is building off of an existing Blazor application with EF Core configured so HealthcareDbContext already exists in the Services folder, but we need to update it to configure our encrypted fields.

First, we are going to add a record type at the bottom of the file, outside the class definition so we can reference it in the constructor of the class:

public record EncryptionKeys(Guid SsnKeyId, Guid DobKeyId);.

Now that this is defined, replace the class definition with the following, which uses a primary constructor:

public class HealthcareDbContext(DbContextOptions<HealthcareDbContext> options, EncryptionKeys keys) : DbContext(options)

This will cause some later code to error as it is missing the new parameter value, but we don’t use that now, so delete the Create method that is erroring.

After entity.ToCollection(“Patients”);, add a new line to let EF Core know that the Id field in our Patient model will be an ObjectID in MongoDB which is the default data type for the unique _iq field:

  entity.Property(p => p.Id)
                .HasBsonRepresentation(BsonType.ObjectId);

This is where the fun begins as it is time to configure the two fields we are going to encrypt: DateOfBirth and SSN.

First up is the DateOfBirth property which is a root field on the document:

entity.Property(p => p.DateOfBirth)
                .IsEncryptedForRange(new DateTime(1900, 1, 1),
                    new DateTime(2100, 1, 1),
                    keys.DobKeyId);

There are two types of queries currently supported, equality and range queries. We want to be able to query between a range of dates, so here, we use .IsEncryptedForRange. This takes three parameters: a possible starting range, a possible ending range (you can of course tweak this to your liking), and then the GUID that will be used as the data encryption key. This is why we added the new record for EncryptionKeys in the file and then passed it in the primary constructor.

Finally, we want to update the property for SSN to add that it is encrypted for equality and passed the SsnKeyId from the keys object:

pr.Property(r => r.SSN).IsEncryptedForEquality(keys.SsnKeyId).HasElementName("sSN");

Just like that, we have properties configured for encryption!

Update Program.cs

Now that we have the configuration handled, it is time to link it all up in Program.cs.

Before we do anything else, let’s add any additionally required using statements at the top of the class that aren’t already there:

using EnterpriseHealthcareDotNet.Models;
using MongoDB.Driver;
using MongoDB.Driver.Encryption;
using MongoDB.EntityFrameworkCore;

Now, delete the code between the call to AddRazorComponents and before the line to register PatientService as a scoped service in Dependency Injection (DI). This code was there to originally configure the DbContext and MongoDB when the application doesn’t use QE, but this will be done differently later. Add the variables that will be used to set up the Queryable Encryption Helpers class and MongoDB after PatientService is registered:

var configuration = builder.Configuration;
var qeHelpers = new QueryableEncryptionHelpers(configuration);

string uri = configuration["MongoDBConnectionString"]!;
string keyVaultDb = configuration["KeyVaultDatabase"] ?? "encryption";
string keyVaultColl = configuration["KeyVaultCollection"] ?? "__keyVault";
string kmsProviderName = configuration["KmsProvider"] ?? "local";
string encryptedDb = configuration["EncryptedDatabase"] ?? "MongoDBMedical";
string cryptSharedLibPath = configuration["CryptSharedLibPath"] ?? throw new ArgumentNullException("CryptSharedLibPath", "Path to the Automatic Encryption Shared Library must be provided in appsettings.Development.json");

After that, add the following code to set up the keyvault namespace, which is the collection in the database where the keys will be stored, and fetch the value using the method in QueryableEncryptionHelpers we set up earlier:

var keyVaultNamespace = CollectionNamespace.FromFullName($"{keyVaultDb}.{keyVaultColl}");


// Generate/reuse KMS provider credentials
var kmsProviders = qeHelpers.GetKmsProviderCredentials(
    kmsProviderName,
    generateNewLocalKey: !File.Exists("customer-master-key.txt"));

Next, add the following which will configure the MongoDB client specifically with settings for QE:

// Configure MongoDB client settings for QE
MongoClientSettings.Extensions.AddAutoEncryption();
var clientSettings = MongoClientSettings.FromConnectionString(uri);
clientSettings.AutoEncryptionOptions = qeHelpers.GetAutoEncryptionOptions(
    keyVaultNamespace,
    kmsProviders);

using var clientEncryption = new ClientEncryption(
    new ClientEncryptionOptions(new MongoClient(clientSettings), keyVaultNamespace, kmsProviders));


var equalityKey = clientEncryption.CreateDataKey("local", new DataKeyOptions());
var rangeKey = clientEncryption.CreateDataKey("local", new DataKeyOptions());

Finally, let’s add everything we have configured to the DI container:

builder.Services.AddSingleton(qeHelpers);
builder.Services.AddSingleton(new EncryptionKeys(equalityKey, rangeKey));
builder.Services.AddScoped<Patient>();

// Register DbContext with QE-enabled client
builder.Services.AddDbContext<HealthcareDbContext>(options =>
{
    options.UseMongoDB(new MongoOptionsExtension()
        .WithClientSettings(clientSettings)
        .WithDatabaseName(encryptedDb)
        .WithKeyVaultNamespace(keyVaultNamespace)
        .WithCryptProvider(CryptProvider.AutoEncryptSharedLibrary, cryptSharedLibPath)
        .WithKmsProviders(kmsProviders));
});

Test it out

It hasn’t even taken that much change addition or code changes, yet here we are, ready to test it out! This is testament to how easy it is to configure QE in EF Core.

Go ahead and run the application. Make sure your connection string and correct CryptSharedLib folder name are set in appsettings.json or appsettings.Development.json (recommend running in debug and using the development version).

Play around adding a new patient and then use a tool like MongoDB Compass to view the collection, and you should see your new patient with the date of birth and sSN encrypted.

Summary

Wow, how amazing! We have added Queryable Encryption to an existing EF Core application using the latest features of the EF Core Provider for MongoDB to safely encrypt private information—in this case, the dates of birth and social security numbers of patients—not just in transit but also in the database itself, while still being able to query it.

Go ahead and give it a go! Plus, if you want to level up your MongoDB skills and get a Credly-backed badge you can show off on LinkedIn, why not try the MongoDB Overview Skills Bage