How would I get distinct values between two fields when querying MongoDB (mongoose)

I am currently creating an inbox within a messaging app. To do this I would like to extract all the unique user ObjectIDs I have stored in my message schema to see all the unique conversations a user is having. The issue is that the ObjectIDs are stored in two separate fields: "to" and "from". Is there any way to get the distinct values from both fields? (i.e. if userA appears in both to and from, it is only extracted once)

Currently I have tried using both $setUnion and $addToSet with no success.

My message schema is as follows

const MessageSchema = new Schema(
    {
        message: { type: String, required: true, maxLength: 128 },
        image: { type: String },
        from: { type: Schema.Types.ObjectId, required: true, ref: "User" },
        to: [
            {
                type: Schema.Types.ObjectId,
                required: true,
                refPath: "docModel",
            },
        ],
        docModel: { type: String, required: true, enum: ["Group", "User"] },
    },
    { timestamps: true },
);

Here are some sample documents:

[
    {
        "_id": "671175112a0bcfdb66f833fc",
        "message": "hi",
        "from": "6706c5576692b95bb39dabae",
        "to": [
            "6708368712a817cd73810b5e"
        ],
        "docModel": "User",
        "createdAt": "2024-10-17T20:35:29.164Z",
        "updatedAt": "2024-10-17T20:35:29.164Z",
        "__v": 0
    },
    {
        "_id": "6711755e2a0bcfdb66f83406",
        "message": "bye",
        "from": "671175392a0bcfdb66f83401",
        "to": [
            "6706c5576692b95bb39dabae"
        ],
        "docModel": "User",
        "createdAt": "2024-10-17T20:36:46.259Z",
        "updatedAt": "2024-10-17T20:36:46.259Z",
        "__v": 0
    },
    {
        "_id": "671181f7c2db995b7ef3f976",
        "message": "1",
        "from": "6706c5576692b95bb39dabae",
        "to": [
            "671175392a0bcfdb66f83401"
        ],
        "docModel": "User",
        "createdAt": "2024-10-17T21:30:31.032Z",
        "updatedAt": "2024-10-17T21:30:31.032Z",
        "__v": 0
    }
]

In the above sample, the result I am looking to get is an array of:

[6706c5576692b95bb39dabae, 671175392a0bcfdb66f83401, 6708368712a817cd73810b5e]

Whereby it is the unique values that appear in either from or to (and if they appear in both from or to they are only pushed to the array once)

And my code for querying MongoDB is as follows. I try to query all the documents where the users id appears in either "to" or "from" which works. I then try to get the unique values of the two fields combined which doesn't work.

I plan to drop the user id of the requesting user once I have extracted all the unique ids to arrive at my final list of unique conversations.

    const uniqueUsers = await Message.aggregate([
        {
            $match: {
                $or: [
                    {
                        from: new mongoose.Types.ObjectId(`${req.user.id}`),
                    },
                    {
                        to: {
                            $in: [
                                new mongoose.Types.ObjectId(`${req.user.id}`),
                            ],
                        },
                    },
                ],
            },
        },
        {
            $group: {
                _id: null,
                unique: { $setUnion: ["$from", "$to"] },
            },
        },
    ]);

In your group stage, you will need to $addToSet all ids in the from and to fields respectively to 2 arrays. After that, $setUnion to combine the 2 arrays together.

db.collection.aggregate([
  {
    "$match": {
      $expr: {
        $let: {
          vars: {
            userIdInput: "6706c5576692b95bb39dabae"
          },
          "in": {
            "$or": [
              {
                $eq: [
                  "$$userIdInput",
                  "$from"
                ]
              },
              {
                $in: [
                  "$$userIdInput",
                  "$to"
                ]
              }
            ]
          }
        }
      }
    }
  },
  {
    "$unwind": "$to"
  },
  {
    "$group": {
      "_id": null,
      "froms": {
        "$addToSet": "$from"
      },
      "tos": {
        "$addToSet": "$to"
      }
    }
  },
  {
    "$project": {
      allIds: {
        "$setUnion": [
          "$froms",
          "$tos"
        ]
      }
    }
  }
])

Mongo Playground