I am trying to implement multiprocessing or threading in my flask __init__.py to implement a sub-process that will handle some back-end work for me. However, I can't seem to pass the application context to the sub-process, and existing libraries are focused on requests - I need this process to start running in parallel when flask is run, no later.
Here is my __init__.py code (excluding setup for irrelevant pages, I only use one implementation at a time ofc):
import os
from multiprocessing import Process
from threading import Thread
from flask_executor import Executor
from flask import Flask
from . import backend
def create_app():
app = Flask(__name__, instance_relative_config=True)
# Multiprocessing Implementation:
p = Process(target=backend.start)
p.start()
# Threading Implementation:
thread = Thread(target=backend.start)
thread.daemon = True
thread.start()
# Flask Executor Implementation:
executor = Executor(app)
executor.submit(backend.start)
return app
And here is my backend.py, called by the subprocess:
from datetime import datetime
from flask import g, request, session
from flaskr.db import log
def start(app=None):
print("\nBackend Started\n")
log("INFO","Backend Started")
while True:
pass
The backend code calls a logging function which works when called from a request inside my normal flask process.
My multiprocessing and threading implementation do not work, as I cannot pass the application context to the sub-process. Process(target=backend.start, args=app)
or Thread(target=backend.start, args=app)
gives me an error, TypeError: 'Flask' object is not iterable
. I cannot add @with_appcontext flags to the start function, as it is not a request.
My Flask Executor passes the application context to the sub-process, but it cannot succeed either, as it is not called from a request:
Traceback (most recent call last):
File "c:\...\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\...\python\python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\...\Python\Python39\Scripts\flask.exe\__main__.py", line 7, in <module>
File "c:\...\python\python39\lib\site-packages\flask\cli.py", line 988, in main
cli.main()
File "c:\...\python\python39\lib\site-packages\flask\cli.py", line 579, in main
return super().main(*args, **kwargs)
File "c:\...\python\python39\lib\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
File "c:\...\python\python39\lib\site-packages\click\core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\...\python\python39\lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\...\python\python39\lib\site-packages\click\core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "c:\...\python\python39\lib\site-packages\click\decorators.py", line 84, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "c:\...\python\python39\lib\site-packages\click\core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "c:\...\python\python39\lib\site-packages\flask\cli.py", line 850, in run_command
app = DispatchingApp(info.load_app, use_eager_loading=eager_loading)
File "c:\...\python\python39\lib\site-packages\flask\cli.py", line 299, in __init__
self._load_unlocked()
File "c:\...\python\python39\lib\site-packages\flask\cli.py", line 333, in _load_unlocked
self._app = rv = self.loader()
File "c:\...\python\python39\lib\site-packages\flask\cli.py", line 389, in load_app
app = locate_app(import_name, name)
File "c:\...\python\python39\lib\site-packages\flask\cli.py", line 251, in locate_app
return find_best_app(module)
File "c:\...\python\python39\lib\site-packages\flask\cli.py", line 77, in find_best_app
app = app_factory()
File "C:\...\flaskr\__init__.py", line 51, in create_app
executor.submit(backend.start)
File "c:\...\python\python39\lib\site-packages\flask_executor\executor.py", line 162, in submit
fn = self._prepare_fn(fn)
File "c:\...\python\python39\lib\site-packages\flask_executor\executor.py", line 122, in _prepare_fn
fn = copy_current_request_context(fn)
File "c:\...\python\python39\lib\site-packages\flask\ctx.py", line 172, in copy_current_request_context
raise RuntimeError(
RuntimeError: This decorator can only be used when a request context is active, such as within a view function.
How can I implement this properly?
I found a solution to this issue. I added this code to my backend.py:
from flask import Blueprint
from threading import Thread
bp = Blueprint('backend', __name__)
def backend(app):
thread = Thread(target=start, args=(app,))
thread.daemon = True
thread.start()
I then added backend.backend(app)
to my create_app() function in __init__.py, right before the end of my function. This will call my backend() function from backend.py and pass the application context, and this function starts my sub-process.