python-3.xjupyter-notebooksubprocessipythonwin32serviceutil

Jupyter notebook kernel error when added python flask app as a service using pywin32 on windows


I have created flask app that can run jupyter notebook server.

When we click on app from the UI, a new subprocess is created using subprocess.Popen() to construct a Jupyter notebook server. I created the windows service for the flask webapp using win32serviceutil. There are three scenarios.

Scenario 1: Jupyter notebooks work properly when a flask app run using python.exe .
Scenario 2: When the Flask app service is executed in debug mode , it works fine.

command :- 
python service.py debug

Scenario 3 :- When flask app is executed as windows service jupyter notebook kernel dies.

Error :- 
Traceback (most recent call last):
  File "d:\v307\lib\site-packages\tornado\web.py", line 1704, in _execute
    result = await result
  File "C:\Program Files\Python39\lib\asyncio\tasks.py", line 328, in __wakeup
    future.result()
  File "d:\v307\lib\site-packages\tornado\gen.py", line 769, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "d:\v307\lib\site-packages\notebook\services\sessions\handlers.py", line 74, in post
    model = yield maybe_future(
  File "d:\v307\lib\site-packages\tornado\gen.py", line 762, in run
    value = future.result()
  File "d:\v307\lib\site-packages\tornado\gen.py", line 769, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "d:\v307\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 98, in create_session
    kernel_id = yield self.start_kernel_for_session(session_id, path, name, type, kernel_name)
  File "d:\v307\lib\site-packages\tornado\gen.py", line 762, in run
    value = future.result()
  File "d:\v307\lib\site-packages\tornado\gen.py", line 769, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "d:\v307\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 110, in start_kernel_for_session
    kernel_id = yield maybe_future(
  File "d:\v307\lib\site-packages\tornado\gen.py", line 762, in run
    value = future.result()
  File "C:\Program Files\Python39\lib\asyncio\futures.py", line 201, in result
    raise self._exception
  File "C:\Program Files\Python39\lib\asyncio\tasks.py", line 256, in __step
    result = coro.send(None)
  File "d:\v307\lib\site-packages\notebook\services\kernels\kernelmanager.py", line 176, in start_kernel
    kernel_id = await maybe_future(self.pinned_superclass.start_kernel(self, **kwargs))
  File "d:\v307\lib\site-packages\jupyter_client\utils.py", line 30, in wrapped
    raise e
  File "d:\v307\lib\site-packages\jupyter_client\utils.py", line 27, in wrapped
    return loop.run_until_complete(future)
  File "d:\v307\lib\site-packages\nest_asyncio.py", line 89, in run_until_complete
    return f.result()
  File "C:\Program Files\Python39\lib\asyncio\futures.py", line 201, in result
    raise self._exception
  File "C:\Program Files\Python39\lib\asyncio\tasks.py", line 256, in __step
    result = coro.send(None)
  File "d:\v307\lib\site-packages\jupyter_client\multikernelmanager.py", line 212, in _async_start_kernel
    starter = ensure_async(km.start_kernel(**kwargs))
  File "d:\v307\lib\site-packages\jupyter_client\utils.py", line 30, in wrapped
    raise e
  File "d:\v307\lib\site-packages\jupyter_client\utils.py", line 27, in wrapped
    return loop.run_until_complete(future)
  File "d:\v307\lib\site-packages\nest_asyncio.py", line 89, in run_until_complete
    return f.result()
  File "C:\Program Files\Python39\lib\asyncio\futures.py", line 201, in result
    raise self._exception
  File "C:\Program Files\Python39\lib\asyncio\tasks.py", line 256, in __step
    result = coro.send(None)
  File "d:\v307\lib\site-packages\jupyter_client\manager.py", line 83, in wrapper
    raise e
  File "d:\v307\lib\site-packages\jupyter_client\manager.py", line 75, in wrapper
    out = await method(self, *args, **kwargs)
  File "d:\v307\lib\site-packages\jupyter_client\manager.py", line 383, in _async_start_kernel
    kernel_cmd, kw = await ensure_async(self.pre_start_kernel(**kw))
  File "d:\v307\lib\site-packages\jupyter_client\utils.py", line 30, in wrapped
    raise e
  File "d:\v307\lib\site-packages\jupyter_client\utils.py", line 27, in wrapped
    return loop.run_until_complete(future)
  File "d:\v307\lib\site-packages\nest_asyncio.py", line 89, in run_until_complete
    return f.result()
  File "C:\Program Files\Python39\lib\asyncio\futures.py", line 201, in result
    raise self._exception
  File "C:\Program Files\Python39\lib\asyncio\tasks.py", line 256, in __step
    result = coro.send(None)
  File "d:\v307\lib\site-packages\jupyter_client\manager.py", line 349, in _async_pre_start_kernel
    kw = await self.provisioner.pre_launch(**kw)
  File "d:\v307\lib\site-packages\jupyter_client\provisioning\local_provisioner.py", line 190, in pre_launch
    km.write_connection_file()
  File "d:\v307\lib\site-packages\jupyter_client\connect.py", line 503, in write_connection_file
    self.connection_file, cfg = write_connection_file(
  File "d:\v307\lib\site-packages\jupyter_client\connect.py", line 164, in write_connection_file
    with secure_write(fname) as f:
  File "C:\Program Files\Python39\lib\contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "d:\v307\lib\site-packages\jupyter_core\paths.py", line 893, in secure_write
    win32_restrict_file_to_user(fname)
  File "d:\v307\lib\site-packages\jupyter_core\paths.py", line 442, in win32_restrict_file_to_user
    user, _domain, _type = win32security.LookupAccountName(
pywintypes.error: (1789, 'LookupAccountName', 'The trust relationship between this workstation and the primary domain failed.')

I have used waitress as a web server and win32serviceutil for creating service of flask app
Python version 3.9

  1. I have tried using NSSM and Winsw for adding flask app as a service but faced same error.
  2. Added creationflags=CREATE_NEW_CONSOLE argument while creating subprocess using Popen but no progress.

Expection
Way to solve issue when we run flask webapp as a service.


Solution

  • In windows OS all services run on session 0 ( 👉 Session 0 ) Session 0 does not have direct access to GPU That's why the Jupyter kernel was showing errors. Windows services have the option to log on as a different user and allow user to have direct desktop access(👉 More about service logon , 👉More about desktop interactive services). This option did not solve my issue.

    The solution was to keep Flask Server running in the terminal.