mongodbaggregatemongodb4.0

Why aggregate two collections with $out never ends ? MongoDB


OK, so, I have two collections. One of them has 6M documents, the other one has 25M documents. I want them in a new collection:

Collection 1 example:

Movie

 {"movieId" : "1", "name" : "Titanic"},
 {"movieId" : "2", "name" : "King Kong"}

Collection 2 example:

Character

{"characterId": "1", "movieId": "1", "characterName": "Name 1"},
{"characterId": "2", "movieId": "1", "characterName": "Name 2"},
{"characterId": "3", "movieId": "1", "characterName": "Name 3"}

So I want a new collection like:

{
  "movieId" : "1", 
  "name" : "Titanic", 
  "characters":[ *collection 2 here* ]
},
{
  "movieId" : "2", 
  "name" : "King Kong", 
  "characters":[]
}

I tried:

db.Movie.aggregate([{ $lookup: { from: "Character",localField: "movieId", foreignField: "movieId", as: "characters" }},{ $out : "movie_characters" }])

But it never ends :( (by never I mean like 10 hours later it was still thinking) If I execute it without the $out it show results in 10 minutes maybe.

Am I doing something wrong with $out?

Thanks for any advice.


Solution

  • The major thing you leaving here is an index.

    $lookup when matching the foreign field, MongoDB uses the index.

    So, create an index in the Character collection using:

    db.Character.createIndex({ "movieId": 1 })
    

    And then apply the lookup aggregation

    db.Movie.aggregate([
      {
        $lookup: {
          from: "Character",
          localField: "movieId", 
          foreignField: "movieId", 
          as: "characters" 
        }
      },
      { 
        $out : "movie_characters" 
      }
    ])
    

    Hope this will help :)