pythonlinuxdockermounted-volumes

Python, Starlette & Docker error walking file system: OSError [Errno 40] Too many levels of symbolic links:


I'm getting this error, even though I'm not running an os.walk or any other kind of file read/write operation with python.

When I'm running

find -L ./ -mindepth 15

I find the files involved in the loop. Locally is the /usr/bin/X11 and at server I'm getting the following:

error walking file system: OSError [Errno 40] Too many levels of symbolic links: '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr'

and then from WatchGodReload:

WARNING: WatchGodReload detected file change in '['/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/3', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/5', '/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/1', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/stdout', '/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/1', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/stderr', '/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/3', '/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/5', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/2', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/5', '/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/stderr', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/4', '/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/fd/6/dev/stdout', '/dev/fd/1']'. Reloading...

Either I run the container with the volume I usually bind:

UPLOADS_PATH=/var/opt/tmp
LOCAL_UPLOADS_PATH=/var/containers/TEST_UPLOADS

docker build --build-arg POSTGRES_USER --build-arg POSTGRES_PASSWORD --build-arg POSTGRES_SERVER --build-arg POSTGRES_DB --build-arg POSTGRES_PORT --build-arg UPLOADS_PATH --build-arg WHERE_AM_I --build-arg SESSION_SECRET --build-arg DO_YOU_WANT_USERS -t test .

docker run -d --name test_container -v ${LOCAL_UPLOADS_PATH}:${UPLOADS_PATH} -p 7000:7000 test

or without the binding, I still get the same error logs & the app is constantly restarting after every request.

P.S. I've also tried the docker system prune -a, and although there were deprecated stuff, nothing changed.

Much appreciate any suggestions on what to further explore.


Solution

  • The /dev/fd is special pseudo-device filesystem that is showing the VFS handles or simply said the open files of the PID that is examining the device. Trough a coincidence of the build and the running processes in the container, your image has a link to the /dev/fd the same /dev/fd that it's trying to read.

    However, this it's not unusual to end in loops if you follow symbolink links on the pseudo-devices, so when using find -L as a super user you should exclude the /dev, /sys and /proc when searching and following links with -xdev

    use

    find -L ./ -xdev -mindepth 15