kuberneteskeycloakkubernetes-helmjupyterhub

JupyterHub starting notebook progress doesn't work with 403 error (stopping server also produce 403)


We have JupyterHub installed in k8s cluster with authentication through KeyCloak. JupyterHub is installed by jupyterhub Helm chart version 2.0.0 from https://jupyterhub.github.io/helm-chart repository.

When a user is authenticated and run a server the page with message "Your server is starting up. You will be redirected automatically when it's ready for you." doesn't show any progress and hub logs the following error: Action is not authorized with current scopes; requires any of [read:servers]"

Full log is:

[I 2023-01-11 16:41:08.977 JupyterHub app:2775] Running JupyterHub version 3.0.0
[I 2023-01-11 16:41:08.978 JupyterHub app:2805] Using Authenticator: oauthenticator.generic.GenericOAuthenticator-15.1.0
[I 2023-01-11 16:41:08.978 JupyterHub app:2805] Using Spawner: kubespawner.spawner.KubeSpawner-4.2.0
[I 2023-01-11 16:41:08.978 JupyterHub app:2805] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-3.0.0
[I 2023-01-11 16:41:09.049 JupyterHub roles:183] Role attribute user.scopes has been changed
[I 2023-01-11 16:41:09.061 JupyterHub app:1934] Not using allowed_users. Any authenticated user will be allowed.
...
...
...
[W 2023-01-11 16:41:44.763 JupyterHub base:89] Blocking Cross Origin API request.  Referer: https://styx-dev.themodelvault.com/hub/spawn-pending/stasdavydov, Host: styx-dev.themodelvault.com, Host URL: http://styx-dev.themodelvault.com/hub/
[W 2023-01-11 16:41:44.763 JupyterHub scopes:804] Not authorizing access to /hub/api/users/stasdavydov/server/progress. Requires any of [read:servers], not derived from scopes []
[W 2023-01-11 16:41:44.763 JupyterHub web:1796] 403 GET /hub/api/users/stasdavydov/server/progress (::ffff:10.0.138.202): Action is not authorized with current scopes; requires any of [read:servers]
[W 2023-01-11 16:41:44.766 JupyterHub log:186] 403 GET /hub/api/users/stasdavydov/server/progress (@::ffff:10.0.138.202) 4.57ms

The same problem happen when stop the server: API request failed (403): Action is not authorized with current scopes; requires any of [delete:servers].

UPD: I found similar problem explained but didn't see a solution: https://discourse.jupyter.org/t/cross-origin-issue-upgrading-from-1-5/15428

Appreciate for any suggestions about how to fix.


Solution

  • After all, the problem is resolved by downgrading JupyterHub chart version to 1.2.0.