djangomemcachedpython-memcached

django memcache cache miss always


I'm running a memcached service on my windows system and I've configured my dev settings to have the following cache settings:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '127.0.0.1:11211',
        'TIMEOUT': 3600,
        'OPTIONS': {
            'MAX_ENTRIES': 100
        }
    }
}

And I'm setting and getting the content in/from cache using the following code:

from django.core.cache import cache
def get_element(fsid):
    element = cache.get(str(fsid))  # get element from memcache if present
    if element is None:
        info("cache miss for fsid-"+str(fsid))
        fs = FlowStatusModel.objects.get(pk=fsid)
        pickle_path = fs.pickle
        gcs_pickle_path = fs.gcs_pickle
        try:
            info("reading from local disk")
            with open(pickle_path, 'rb') as handle:
                element = pickle.load(handle)
        except:
            info("local disk failed. copying file from cloud storage bucket to local disk")
            create_and_copy(gcs_pickle_path, pickle_path)
            with open(gcs_pickle_path, 'rb') as handle:
                element = pickle.load(handle)
        info("adding fsid-"+str(fsid) + " to cache with 1hour timeout")
        cache.set(str(fsid), element, 3600)
    return element

I see in the log that there is always a cache miss. I could not figure out if django was able to set the element in the cache. If I try from python manage.py shell with simple set and get, I'm able to get the data back. I tried to run the command memcached.exe -vv from the command prompgt to see if its receiving the requests but in neither scenarios(from dev server/manage.py shell) I see any set or get information printed on the console. I appreciate any help in figuring out the issue.


Solution

  • Probably because of this:

        'OPTIONS': {
            'MAX_ENTRIES': 100
        }
    

    That's a very small number. You are probably having a great deal of thrashing. Each time you add something to the cache it's very likely to be replacing something else. That's probably what's happening to your fsid key.

    It could also be possible that you have run foul of the maximum item size of memcached. Fix that by changing the memcached configuration file (memcached.conf) and adding the following

    MAXITEMSIZE=12m
    

    However it should be noted that if you are storing such large objects, memcached may not be ideal for you.