Problem

In my local development environment, using Compass, connected to a CSFLE-enabled MongoDB Enterprise client, I can see all the fields that are supposed to be encrypted by my JSON schema as clear text.

Background

• I am using MongoDB Enterprise for development purposes and have Atlas Cloud for my production deployment.
• I'm planning to use AWS KMS for my production key and a I've attempted both a local key and AWS IAM user credentials to implement automatic Client-Side Field-Level Encryption locally.
• I've checked that mongocryptd is in fact running.
• During startup, my server connects to multiple instances of MongoDB that correspond to different micro-services I maintain. This server is where I've set up my encryption credentials.
• My microservices receive the DEK and create schema maps based on their specific requirements.
• I don't see any errors in the MongoDB logs access via mongosh
• I've dropped my database and started fresh
• I'm using mongoose

MongoDB Version

MongoDB Enterprise 7.0.2

Node 18 Package Versions

"mongodb": "^6.2.0",
"mongodb-client-encryption": "^6.0.0",
"mongoose": "^8.0.0",

How I'm Initializing CSFLE

1. At the top level of my server, I'm calling an initialize method in my Encryption class that takes in options and returns an object that contains common configuration parameters, including the DEK, that my micro-services can use. This method creates an encryption client to create the encryption database, key vault collection and produce the DEK if it does not already exist, then it closes that connection.
2. I pass the returned config params from the previous step into the microservice init methods, which create the database models (that do not have any server-side encryption schemas defined), and create regular MongoDB client connections to each service using the autoEncryption config param.

My Code

NOTE: Much stuff has been removed to help clarify the code.

CSFLE Helper Class

export default class Encryption implements IEncryption {
  // ... There are several private and public variables not shown here

  // private constructor to enforce calling `initialize` method below, which calls this constructor internally
  private constructor(opts?: EncryptionConfigConstructorOpts) {
    this.tenantId = opts?.tenantId;
    this.keyVaultDbName = opts?.keyVaultDbName;
    this.keyVaultCollectionName = opts?.keyVaultCollectionName;
    this.DEKAlias = opts?.DEKAlias;

    // Detect a local development environment
    if (process.env?.ENVIRONMENT === LOCAL_DEV_ENV) {
      const keyBase64 = process.env?.LOCAL_MASTER_KEY;
      const key = Buffer.from(keyBase64, 'base64');

      // For testing, I'm manually switching between a local key and remote KMS
      // I'll leave out the production-detection code
      if (_debug) {
        this.provider = KMS_PROVIDER;
        this.kmsProviders = {
          aws: {
            accessKeyId: process.env.AWS_ACCESS_KEY_ID,
            secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
          },
        };
        this.masterKey = {
          key: process.env.KMS_MASTER_ARN,
          region: opts?.masterRegion,
        };
      } else {
        this.kmsProviders = {
           local: {
             key,
           },
        };
      }
    }

    const keyVaultNamespace = `${this.keyVaultDbName}.${this.keyVaultCollectionName}`;

    const encryptionOptions: ClientEncryptionOptions = {
      keyVaultNamespace,
      kmsProviders: this.kmsProviders,
    };

    this.encryptionOptions = encryptionOptions;
  }

  public static async initialize(
    url: string,
    opts?: EncryptionConfigConstructorOpts
  ): Promise<Encryption> {
    // Set internal attributes
    const encryption = new Encryption(opts);

    // Create key vault collection (this is idempotent, afaik)
    const client = new MongoClient(url);
    const keyVaultDB = client.db(encryption.keyVaultDbName);
    const keyVaultColl = keyVaultDB.collection(encryption.keyVaultCollectionName);
    await keyVaultColl.createIndex(
      { keyAltNames: 1 },
      {
        unique: true,
        partialFilterExpression: { keyAltNames: { $exists: true } },
      }
    );

    let dek: UUID | undefined = undefined;

    // This checks for an existing DEK, then creates/assigns or just assigns when necessary
    try {
      // Initialize client encryption
      const clientEncryption = new ClientEncryption(client, encryption.encryptionOptions!);
      const keyOptions = {
        masterKey: encryption.masterKey,
        keyAltNames: [encryption.DEKAlias],
      };
      dek = await clientEncryption.createDataKey(encryption.provider, keyOptions);
    } catch (err: any) {
      // Duplicate key error is expected if the key already exists, so we fetch the key if that happens
      if (String(err?.code) !== '11000') {
        throw err;
      } else {
        // Check if a DEK with the keyAltName in the env var DEK_ALIAS already exists
        const existingKey = await client
          .db(encryption.keyVaultDbName)
          .collection(encryption.keyVaultCollectionName)
          .findOne({ keyAltNames: encryption.DEKAlias });

        if (existingKey?._id) {
          dek = UUID.createFromHexString(existingKey._id.toHexString());
        } else {
          throw new Error('DEK could not be found or created');
        }
      }
    } finally {
      await client.close();
    }

    encryption.dek = dek;
    encryption.isReady = !!encryption.dek;
    return encryption;
  }

  // Defined as an arrow function to preserve the `this` context, since it is called as a callback elsewhere
  // This gets called after the `initialize` method from within each micro-service
  public getSchemaMap = (
    jsonSchema: Record<string, unknown>,
    encryptionMetadata?: Record<string, unknown>
  ): Record<string, unknown> => {
    if (!this?.isReady) {
      throw new Error('Encryption class cannot get schema map until it is initialized');
    }

    const schemaMapWithEncryption = {
      encryptMetadata: {
        keyId: [this.dek],
        algorithm: process.env.ALG_DETERMINISTIC,
        ...encryptionMetadata,
      },
      ...jsonSchema,
    };
    return schemaMapWithEncryption;
  };
}

Startup code

// ... Start up code
    const encryption = await Encryption.initialize(process.env.DB_CONN_STRING);
    const opts = {
      autoEncryption: {
        ...encryption.encryptionOptions
      },
    };

    await Service1Models.initialize(process.env.DB_CONN_STRING, opts, encryption.getSchemaMap);
    await Service2Models.initialize(process.env.DB_CONN_STRING, opts, encryption.getSchemaMap);

// ... More start up code and API route config

Typical Service's initialize method

// ... Init code and then assigning the schema-generated model (which does not contain any encryption syntax)
Service1Model.service1DataModel = model<IService1Document>('Service1Doc', Service1Schema, 'Service1Docs');

// Finally, connecting to the DB with a schema map generated for this service, specifically
mongoose.connect(url, {
        ...opts,
        autoEncryption: opts?.autoEncryption
          ? {
              ...opts?.autoEncryption,
              schemaMap: getSchemaMap(importedSchemaJson),
            }
          : undefined,
      } as ConnectOptions);

My encryption JSON schema

{
  "MyCollection1": {
    "properties": {
      "myDataString": {
        "encrypt": {
          "bsonType": "string"
        }
      },
      "myDataArray": {
        "encrypt": {
          "bsonType": "array"
        }
      },
      "myDataObject": {
        "bsonType": "object",
        "properties": {
          "myNestedProperty1": {
            "encrypt": {
              "bsonType": "string"
            }
          },
          "myNestedProperty2": {
            "bsonType": "string"
          }
        }
      }
    }
  }
}

If the fields are not encrypted, it's 99% related to schema map definition. In your case, you specify incorrect database.collection path where you provide only collection name without database.