pymongo-3.xflask-pymongo

pymongo aggregate collection keys


Using this answer here https://stackoverflow.com/a/43570730/7626649

I'd like to do this with flask-pymongo.

Specifically I'd like to perform the below query in flask-pymongo and/or pymongo (It seems like if it can be done in one then it could be done in the other..I hope):

db.things.aggregate([
  {"$project":{"arrayofkeyvalue":{"$objectToArray":"$$ROOT"}}},
  {"$unwind":"$arrayofkeyvalue"},
  {"$group":{"_id":null,"allkeys":{"$addToSet":"$arrayofkeyvalue.k"}}}
])

Here's what I've done so far:

@app.route('/reports/<site>/<system>/<data_type>', methods=['GET'])
def get_keys_for_collection(site, system, data_type):
    pipeline = [
        {"$project":{"arrayofkeyvalue":{"$objectToArray":"$$ROOT"}}},   
        {"$unwind":"$arrayofkeyvalue"},
        {"$group":{"_id": None,"allkeys":{"$addToSet":"$arrayofkeyvalue.k"}}}   
    ]
    db, options = get_db_handle(site)
    collection = "%s_%s" % (system, data_type)
    pp.pprint(db.command('aggregate', collection, pipeline=pipeline, explain=True))
    return jsonify({'results': list(db.collection.aggregate(pipeline))})

I have verified that 'db' is an active Flask-Pymongo connection to the database that contains the collection being queried. Here is the output of db.command() with explain=True:

{   'ok': 1.0,
'stages': [   {   '$cursor': {   'query': {},
                                 'queryPlanner': {   'indexFilterSet': False,
                                                     'namespace': 'MYDB.collection',
                                                     'parsedQuery': {},
                                                     'plannerVersion': 1,
                                                     'rejectedPlans': [],
                                                     'winningPlan': {   'direction': 'forward',
                                                                        'stage': 'COLLSCAN'}}}},
              {   '$project': {   '_id': True,
                                  'arrayofkeyvalue': {   '$objectToArray': [   '$$ROOT']}}},
              {'$unwind': {'path': '$arrayofkeyvalue'}},
              {   '$group': {   '_id': {'$const': None},
                                'allkeys': {   '$addToSet': '$arrayofkeyvalue.k'}}}]}

I omitted the actual database and collection name, and replaced them with MYDB.collection above. The db.command() proves that db.connection is a valid connection and the problem doesn't lie with syntax on that part of the query.

I get an empty list as the result with no errors. The URL was modified to not reveal certain information. See db.command() results for proof that the db connection and collection exist and are being connected to properly.

curl http://0.0.0.0:5000/reports/site/system/data_type
{
   "results": []
}

What I'm using:

Python 3.6.5
MongoDB shell version v3.6.5
PyMongo v3.4.0
Flask-Pymongo v0.5.2

Solution

  • There is a syntax issue with the aggregate command:

    db.collection.aggregate(pipeline)
    

    should be

    db[collection].aggregate(pipeline)