djangoamazon-s3celerydjango-celerypython-django-storages

Storing files to S3 not working inside Celery worker


I am facing issues with storing files in S3 from django's celery task. I am doing following things

code snippate

        try:
            excel_file = generate_xlsx_file(sheet_dfs)
        except Exception as e:
            logger.exception("Error while generating excel file")
            return

        # Save the report
        report = Report.objects.create()

        file_name = generate_file_name()

        # Save the file
        report.file.save(file_name, ContentFile(excel_file))

at the last line, celery worker just stops and don't process any other tasks as well. Also this code works fine if I run it in django shell


Solution

  • So the issue was eventlet, I was running celery workers with eventlet and removing them solved the issue. I found the workaround for that in this answer: Celery with Eventlets fails when uploading to s3 (boto3)