djangotastypie

Memory Error while doing `request.POST.copy()'


I have a Django application that allows users to upload files. I do that using tastypie rest, it works perfectly with normal size files. but an error occurs when uploading a 500MB file.

the error occurs at:

multipart_data = request.POST.copy()

The error is:

Traceback (most recent call last):
  File "<env_path>\lib\site-packages\tastypie\resources.py", line 227, in wrapper
    response = callback(request, *args, **kwargs)
  File "<env_path>\lib\site-packages\tastypie\resources.py", line 467, in dispatch_list
    return self.dispatch('list', request, **kwargs)
  File "<env_path>\lib\site-packages\tastypie\resources.py", line 499, in dispatch
    response = method(request, **kwargs)
  File "<env_path>\lib\site-packages\tastypie\resources.py", line 1405, in post_list
    deserialized = self.deserialize(request, request.body, format=request.META.get('CONTENT_TYPE', 'application/json'))
  File "<project_path>\apps\data_manager\rest.py", line 70, in deserialize
    multipart_data = request.POST.copy()
  File "<env_path>\lib\site-packages\django\core\handlers\wsgi.py", line 110, in _get_post
    self._load_post_and_files()
  File "<env_path>\lib\site-packages\django\http\request.py", line 315, in _load_post_and_files
    self._post, self._files = self.parse_file_upload(self.META, data)
  File "<env_path>\lib\site-packages\django\http\request.py", line 275, in parse_file_upload
    return parser.parse()
  File "<env_path>\lib\site-packages\django\http\multipartparser.py", line 254, in parse
    chunk = handler.receive_data_chunk(chunk, counters[i])
  File "<env_path>\lib\site-packages\django\core\files\uploadhandler.py", line 174, in receive_data_chunk
    self.file.write(raw_data)
MemoryError

Solution

  • As per the documentation the MemoryFileUploadHandler is not for large files.

    so the below solution is working, in the settings.py:

    FILE_UPLOAD_HANDLERS = [        
        'django.core.files.uploadhandler.TemporaryFileUploadHandler',
    ]