mongodbappendatomicmongoengine

Mongoengine: Atomic [create or modification]


I have the following MongoDB document-class defined with mongoengine:

class Archive( mongoengine.Document ):
    user_id         = mongoengine.StringField()
    transaction_ids = mongoengine.ListField( default=list )

I want this class to operate similarly to a Python object that could be defined as: archive = collections.defaultdict(list), where it is trivial to say archive[a_user_id].append(a_transaction_id)

The tricky part is that I would like this to operate atomically with respect to the MongoDB. I have been around and around with the mongoengine documentation and can't put together what I need, nor find examples.

In my naive mind, I am thinking of pseudocode like this:

define atomically_archive_transaction( a_user_id, a_transaction_id ):
    Archive.atomically_create_or_modify(
        query = { user_id=a_user_id },
        create_if_does_not_exist = { user_id=a_user_id, transaction_ids=[a_transaction_id,] },
        modify_if_does_exist = { push__transaction_id=a_transaction_id },
        write_to_disk_afterwards = 'all in the same atomic transaction',
        )

Is there any hope?

As a perhaps more tractable approach, it would be ok to try to always create a new document for a_user_id and have the creation fail if the document already exists. (Not clear how to do that.) Then I could simply follow with an atomic push, which would avoid possible race conditions, but I'm then left with the challenge of how to make sure the save occurs in the same atomic operation as the push. (The documentation here is not as clear as it could be on this point; it only implies that the save-to-disk occurs in its example: BlogPost.objects(id=post.id).update_one(push__tags='nosql') )

Thank you.


Solution

  • This seems to work. Criticisms welcome! :-)

    class Archive( mongoengine.Document ):
        user_id         = mongoengine.StringField()
        transaction_ids = mongoengine.ListField( default=list )
    
        @classmethod
        def AddTransaction( cls, a_user_id, a_transaction_id ):
            # Atomically create new object if none exists. If does exist, nothing happens.
            query_set = cls.objects( user_id=a_user_id )
            query_set.update_one( set__user_id=a_user_id,  upsert=True )
            # Next do an atomic push.
            query_set.update_one( push__transaction_ids=a_transaction_id )
    

    With a simple test:

    Archive.AddTransaction( 'user1', 'transaction1A' )
    Archive.AddTransaction( 'user1', 'transaction1B' )
    Archive.AddTransaction( 'user2', 'transaction2A' )
    Archive.AddTransaction( 'user2', 'transaction2B' )
    Archive.AddTransaction( 'user1', 'transaction1C' )
    for instance in Archive.objects():
        print( f"{instance.user_id} = {instance.transaction_ids}" )
    

    The result is:

    user1 = ['transaction1A', 'transaction1B', 'transaction1C']
    user2 = ['transaction2A', 'transaction2B']
    

    The first update_one statement is drawn from: How do I upsert a document in MongoDB using mongoengine for Python?

    The point that nothing happens with the second update_one statetment if the document already exists is from: Mongodb how to insert ONLY if does not exists (no update if exist)?

    The update_one documentation is at: https://docs.mongoengine.org/apireference.html