I'm trying to implement the celery using flask and when i run this command celery -A src.celery_worker.celery_app worker --loglevel=debug
to start the celery worker, it throwing error:
Couldn't import 'src.celery_worker.celery_app': urls must start with a leading slash
Here's the full traceback:
Traceback (most recent call last):
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 58, in convert
return find_app(value)
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/celery/app/utils.py", line 383, in find_app
sym = symbol_by_name(app, imp=imp)
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/kombu/utils/imports.py", line 61, in symbol_by_name
reraise(ValueError,
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/kombu/exceptions.py", line 34, in reraise
raise value.with_traceback(tb)
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/kombu/utils/imports.py", line 59, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/celery/utils/imports.py", line 109, in import_from_cwd
return imp(module, package=package)
File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 848, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/sonnc/project/mac/lifescience-operator-management-bff/src/__init__.py", line 42, in <module>
api = Api(
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask_restx/api.py", line 197, in __init__
self.init_app(app)
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask_restx/api.py", line 236, in init_app
self._init_app(app)
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask_restx/api.py", line 247, in _init_app
self._register_doc(self.blueprint or app)
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask_restx/api.py", line 320, in _register_doc
app_or_blueprint.add_url_rule(self._doc, "doc", self.render_doc)
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask/scaffold.py", line 56, in wrapper_func
return f(self, *args, **kwargs)
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask/app.py", line 1083, in add_url_rule
rule = self.url_rule_class(rule, methods=methods, **options)
File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/werkzeug/routing.py", line 698, in __init__
raise ValueError("urls must start with a leading slash")
ValueError: Couldn't import 'src.celery_worker.celery_app': urls must start with a leading slash
Below is the file src/celery_worker/__init__.py
:
import os
from flask import Flask
from celery import Celery, Task
def celery_init_app(app: Flask) -> Celery:
class FlaskTask(Task):
def __call__(self, *args: object, **kwargs: object) -> object:
with app.app_context():
return self.run(*args, **kwargs)
celery_app = Celery(app.name, task_cls=FlaskTask)
celery_app.config_from_object(app.config["CELERY"])
celery_app.set_default()
app.extensions["celery"] = celery_app
return celery_app
def create_app() -> Flask:
app = Flask(__name__)
# url for local redis
# redis_url = "redis://localhost:6379"
# url for docker redis
# redis_url = "redis://redis:6379/0"
app.config.from_mapping(
CELERY=dict(
broker_url="redis://localhost:6379",
result_backend="redis://localhost:6379",
task_ignore_result=True,
# import the tasks
# imports=("src.common.send_email",),
),
)
app.config.from_prefixed_env()
celery_init_app(app)
return app
flask_app = create_app()
celery_app = flask_app.extensions["celery"]
Look like the error is related to the flask_restx
module, but I can't trace how it causing this issue.
Please help, thanks.
EDIT 1:
Here's the proof that every route
and path
in my source code are start with a slash:
Thanks you all for your answers and suggestions, I've tried to debug the next day but somehow it suddenly worked on the local environment, the issue is gone, I don't really understand why, but the issue still remains on the docker environment.
So I've tried to debug further and found that the reason it's because the setting in docker-compose.yml
didn't set the env_file
file.
So I added the env_file
to the setting of celery_worker
in docker-compose.yml
and it worked, here's how I fixed it:
celery_worker:
build:
context: .
dockerfile: src/celery_worker/Dockerfile
image: flask_celery_worker
command:
["/start-celeryworker", "sh", "-c"]
# ['celery', '-A', 'src.celery_worker.celery_app', 'worker', '-l', 'debug']
# celery -A src.celery_worker.celery_app worker --loglevel=debug
# volumes:
# - .:/lifescience-operator-management-bff-worker
env_file:
- .env
ports:
- 6900:6900
networks:
- my-network
# environment:
# - FLASK_APP=app
depends_on:
- redis
Well, I know it sounds like doesn't related to the issue that I've posted, but believe me, that's all I did to fix this issue.
Hope this answer help some one.