Mongo aggregate queries use $out output and perform replace operations instead of append operations

  mongodb, question

A Problem Encountered by mongo Aggregation Query

At present, some aggregate queries are performed on a table in mongo database. considering the large amount of data, the search results from the database still need to be inserted into the database. therefore, it is decided to insert the query results into a temporary set and use mongo’s $out operation. According to business requirements, three such aggregate queries need to be executed. The test found that the output of $out to the new set is to execute the replacement operation instead of the append operation. The following is the official website explanation:

  • The $out operation creates a new collection in the current database if one does not already exist.

  • If the collection specified by the $out operation already exists, then upon completion of the aggregation, the $out stage atomically replaces the existing collection with the new results collection. The $out operation does not change any indexes that existed on the previous collection.

Now I still have such a demand. How should I put the results of multiple aggregations into a table?

Starting from 2.6, the return value of aggregation is a cursor. Since it is a cursor, you are not required to cache the entire result set in memory at the client before operating. The direct idea is to traverse the cursor and insert it into the target set one by one. For example, in the shell:

db.coll.aggregate([...]).forEach(function(doc) {
 db.target_coll.save(doc);
 });