Hypercorn runs with duplicated process

I am not sure whether this is really hypercorn issue, but could not imagine what else can be. I have searched the net but have not found any topic close to this, so please bear with me.

I am running a server with hypercorn on Ubuntu 20.04, with python3.8.10.

The problem is that it is runs with a duplicated process in background.

root     2278497  0.8  0.1  41872 33568 pts/7    S    10:03   0:00 /usr/bin/python3 /usr/local/bin/hypercorn -c config.toml main:app --reload
root     2278499  0.0  0.0  17304 11332 pts/7    S    10:03   0:00 /usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(4)
root     2278500  0.7  0.1  41648 34148 pts/7    S    10:03   0:00 /usr/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=5, pipe_handle=7) --multiprocessing-fork

The main process is 2278497, but there are duplicated processes 2278499 and 2278450. I do not know why these are started.

This causes unwanted effects by executing twice the same tasks.

How can I avoid that?

EDIT:

A minimal example:

# test_main.py
from fastapi import FastAPI

app = FastAPI()

@app.get("/")
async def root():
    return {"message": "Hello World"}

print("main module loaded.")

I then type:

sudo hypercorn test_main:app

and the stdout is:

main module loaded.
main module loaded.
[2022-11-02 15:08:45 +0100] [2364437] [INFO] Running on http://127.0.0.1:8000 (CTRL + C to quit)

Solution

I get the impression you're using Hypercorn wrong and it's not designed for you to run your own code in the same process.

That said, what you're seeing in your MWE is a master process and a worker process. You can distinguish between these by checking whether the current process is a "daemon" process (i.e. Unix-style background task) or not via:

import multiprocessing

print(multiprocessing.current_process().daemon)

this will output False for the master process, and True for all worker processes. E.g. this would increase to 5 Trues when executed as hypercorn -w5 test_main:app.

I think I'd suggest not using this hack in production and using another system (e.g. systemd or supervisord) to make sure that any background tasks are kept running. This would give you more control over them. You could still have the code in the same file, just behind the normal if __name__ == '__main__': guard.

Update with more complete example:

from fastapi import FastAPI
from multiprocessing import current_process

app = FastAPI()

# see https://asgi.readthedocs.io/en/latest/specs/lifespan.html
@app.on_event('startup')
async def on_startup():
    print("asgi lifecycle startup event")

@app.on_event('shutdown')
async def on_shutdown():
    print("asgi lifecycle shutdown event")

@app.get("/")
async def root():
    return {"message": "Hello World"}

def main():
    print("running as main module")

# see https://docs.python.org/3/library/__main__.html
if __name__ == "__main__":
    import sys
    sys.exit(main())

# warning, these will also execute if this module imported
if not current_process().daemon:
    print("main module loaded into master process")
else:
    print("main module loaded into worker process")

can be run as:

$ hypercorn -w2 test_main:app
main module loaded into master process
main module loaded into worker process
asgi lifecycle startup event
[2022-11-04 11:39:31 +0000] [24243] [INFO] Running on http://127.0.0.1:8000 (CTRL + C to quit)
main module loaded into worker process
asgi lifecycle startup event
[2022-11-04 11:39:31 +0000] [24244] [INFO] Running on http://127.0.0.1:8000 (CTRL + C to quit)
^C
asgi lifecycle shutdown event
asgi lifecycle shutdown event
$ python -m test_hyper
running as main module

not the the first line mentions "master process". This is Hypercorn's supervisor process which is responsible for looking after worker processes (e.g. clean shutdown / restarting). I also show that this code can recognise that it's being run as a main module, and could do different things there. This is because Hypercorn is importing this module into each process (i.e. whether it's a master or worker).