I’ve been reading a few guides on setting up background processes, and most of them recommend using Celery with Redis. I'm running a Django app with Nginx and Gunicorn on Windows, but I came across a guide that mentioned Celery version 4.0 could have sync issues on Windows. Given this, would using Celery and Redis still be the best option for background tasks on Windows?
The background process would be used for csv data upload to database like PgAdmin.
You're right to be cautious about using Celery with Redis on Windows. Historically, Celery on Windows has had some compatibility issues, especially with its default prefork concurrency model, which doesn't play well with Windows due to how multiprocessing works on that platform. This is especially relevant if you're using Celery 4.x, which relies heavily on multiprocessing and may cause synchronization problems or poor performance on Windows.
If you need something simpler than Celery for small tasks like CSV uploads: multiprocessing or Django Q would be good choices, especially if you're looking for something that doesn’t require as much setup or infrastructure.