pythoncelerycelerydcelery-broker

Celery worker creates "control" folder when using filesystem as a broker


I have a project using Celery with Redis backend to manage tasks. For local development I am trying to set up Celery with filesystem as broker instead of Redis. However, when I run Celery worker, it creates me control folder in the root directory with the following contents:

/control
├── celery.pidbox.exchange
├── Q1.exchange
├── Q2.exchange
├── ...

I have trouble finding any resources on what it is and for what it is used exactly. My goal is to possibly move this folder to another location (ex. .celery/ folder), so that it does not sit in root directory.

Here is my Celery configuration:

class CeleryLocalConfig:

    include = ["app.tasks"]
    broker_url = "filesystem://"
    result_backend = "file://.celery/broker/results"
    broker_transport_options = {
        "data_folder_in": ".celery/broker/out",
        "data_folder_out": ".celery/broker/out",
        "queue_order_strategy": "sorted"
    }

celery_app = Celery(__name__)
celery_app.config_from_object(CeleryLocalConfig)

So far I have tried:


Solution

  • Setting a control_folder field did the trick for me:

    CELERY_BROKER_TRANSPORT_OPTIONS = {
        "data_folder_in": os.path.join(BASE_DIR, ".celery"),
        "data_folder_out": os.path.join(BASE_DIR, ".celery"),
        "control_folder": os.path.join(BASE_DIR, ".celery"),
    }
    

    So for you, I suppose, it would be:

    broker_transport_options = {
        "data_folder_in": ".celery/broker/out",
        "data_folder_out": ".celery/broker/out",
        "queue_order_strategy": "sorted",
        "control_folder": ".celery/broker/out",
    }