python-3.xpython-asynciorace-conditiondjango-channelsredis-cache

How to handle race condition on Django-Channels consumer?


I'm implementing a service with django-channel, I have done some explanations about my issues but you can just scroll down to bottom part where I ask my questions and ignore them.

In this service I'm using a async caching system to raise service's performance. Writing to this cache raises race condition issues. Here's two main functions of this cache

async def get_room_data_dict(self):
      data = await self.cache_client.get(self.key)
      return data

async def set_room_data_dict(self, room_data_dict):
      is_success = await self.cache_client.set(self.key, room_data_dict)
      return is_success

Now here's the problem with this method.

### coroutine 1 ###
room_data_dict = await cache_manager.get_room_data_dict()
# In the line below a context switch occurs and coroutine 2 continues to do some tasks
new_room_data_dict = await do_somthing(room_data_dict)
# coroutine 1 continue but room_data_dict is the old one and coroutine 1 going to save it so what coroutine 2 did is actually not applied
await cache_manager.set_room_data_dict(new_room_data_dict)

### coroutine 2 ###
# this coroutine continues without a context switch
room_data_dict = await cache_manager.get_room_data_dict()
new_room_data_dict = await do_somthing(room_data_dict)
await cache_manager.set_room_data_dict(new_room_data_dict)
# gets back to coroutine 1 and continues its code

Now if you've looked closely and have some OS education you will see that the change coroutine 2 does on room_data_dict is actually not applied.

Here's the thing I would've done to prevent this problem, I would change the functions like below

async def get_room_data_dict(self):
      await self.room_data_dict_semaphore.acquire()
      data = await self.cache_client.get(self.key)
      return data

async def set_room_data_dict(self, room_data_dict):
      is_success = await self.cache_client.set(self.key, room_data_dict)
      self.room_data_dict_semaphore.release()
      return is_success

This method would solve my problem if and only if the semaphore in the code is shared in group channel.

So Here's the things I'm asking and if you can answer any of this I can solve my problem:

  1. How can I share an instance of an object between two coroutines (in my problem between two group channels ) ?
  2. In python when you do an_instance.str() you get something that shows what is instance's memory address ... Can I get that specific instance with that address ?
  3. Any other solutions to my problem other than mine (using semaphores) would be very appreciated.

Solution

  • How about acquire/release the lock in the outer scope (coroutine 1).

    Anyway various locking systems allow you to identify the lock using a string key, so that it can be acquired/released from different scopes. You can even use a distributed lock like python-redis-lock.