[SOLVED] django-pipeline wipes out my entries in Django Database Cache

django-pipeline wipes out my entries in Django Database Cache

I'm working on a Django application which uses django-pipeline for dealing with browsers' file caching issues (and also for other benefits).

STATIC_URL = '/static/'

STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'

STATICFILES_DIRS = (
    os.path.join(PROJECT_ROOT, 'bower'),
)

STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'pipeline.finders.PipelineFinder',
    'pipeline.finders.CachedFileFinder',
)

PIPELINE = {}
PIPELINE['DISABLE_WRAPPER'] = True
PIPELINE['JS_COMPRESSOR'] = 'pipeline.compressors.NoopCompressor'
PIPELINE['CSS_COMPRESSOR'] = 'pipeline.compressors.yuglify.YuglifyCompressor'

PIPELINE['COMPILERS'] = (
    'pipeline.compilers.sass.SASSCompiler',
    'pipeline.compilers.es6.ES6Compiler',
)

PIPELINE['JAVASCRIPT'] = {
...
}

PIPELINE['STYLESHEETS'] = {
...
}

PIPELINE['SASS_BINARY'] = 'C:\\Ruby22-x64\\bin\\sass.bat'
PIPELINE['BABEL_BINARY'] = 'c:\\Users\\Foobar\\node_modules\\.bin\\babel.cmd'

So far so good. Lately we decided to use Django's Database Cache (https://docs.djangoproject.com/en/1.9/topics/cache/#database-caching) for caching some long running statistical calculation results.

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
        'LOCATION': 'django_dbcache',
    }
}

I executed createcachetable and the table was created. I'm placing entries into this table with no expiration date, since I have my own validity check, and can decide myself if the data is up-to-date, or needs to be recalculated.

To my surprise however, when I issue a collectstatic for pipeline, it wipes the content of that table and fills it with it's own staticfiles:{md5code} key-values. (In production I saw situation when it didn't wipe out everything). But this renders my caching scheme non functional. I cannot seem to find any settings in the pipeline documentation how to stop pipeline doing this. pipeline's cache entry values in the cache are pretty short, merely contain full path to generated files. These entries' expiration are a couple of hours. I won't mind them being there, just don't wipe my stuff.

Additional note: I'm on Windows platform (see the pipeline settings above), but the same thing happens on a Linux production server.

Addition to the marked answer: knowing that anyone can mess with the default cache + staticfiles can rudely wipe it out, it's even better to separate ours and everyone else:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
        'LOCATION': 'default-cache',
    },
    'staticfiles': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
        'LOCATION': 'static-files',
    },
    'my_dbcache': {
        'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
        'LOCATION': 'django_dbcache',
    }
}

Solution

Defining a separate cache for static files will fix the issue. Django by default looks for "staticfiles" cache first. example:

CACHES = {
'default': {
    'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
    'LOCATION': 'django_dbcache',
},
'staticfiles': {
    'BACKEND': "django.core.cache.backends.locmem.LocMemCache",
    'LOCATION': 'static-files',
}