[SOLVED] Guidance required - am I missing app ingress annotation or is my oauth2-proxy config wrong?

Guidance required - am I missing app ingress annotation or is my oauth2-proxy config wrong?

I have an old version of oauth2-proxy (v5.1.0) deployed into our AWS EKS v1.21 cluster and this works great for Azure AD authentication.

Now, I am testing upgrading to new(er) v7.1.3 as the old oauth2-proxy uses deprecated APIs that the next version of k8s supports.

With the new version, I get authorization but the application I am testing authentication with seems to take the value of 'user' in the reply from Azure, this value is not what the app that is being accessed is expecting.

This is from the oauth2-proxy log showing the value of 'user';

[2023/02/03 10:07:16] [stored_session.go:122] Refreshing 9m55.925357944s old session cookie for Session{email:me@mydomain.com user:fF3CtUwz6hwiCWjpR3kgHCYs0Uj3Syk7sTAUs1pOJSU PreferredUsername:me@mydomain.com token:true id_token:true created:2023-02-03 09:57:20.074642056 +0000 UTC expires:2023-02-03 11:17:21.07415738 +0000 UTC refresh_token:true} (refresh after 1m0s)

My app then only sees the value for 'user' rather than 'email' or 'PreferredUsername'

This is the oauth2-proxy config for the old version;

extraArgs:
  provider: azure
  cookie-name: _oauth2
  cookie-refresh: 59m
  cookie-domain: .mydomain.com
  whitelist-domain: .mydomain.com

  set-authorization-header: "true"
  set-xauthrequest: "true"
  pass-access-token: "true"
  pass-authorization-header: "true"
  pass-basic-auth: "true"
  pass-host-header: "true"
  pass-user-headers: "true"

  session-store-type: redis
  redis-connection-url: oauth2-redis-redis-ha
  redis-use-sentinel: "true"
  redis-sentinel-master-name: mymaster
  redis-sentinel-connection-urls: oauth2-redis-redis-ha:26379

The application ingress config looks like;

    ingress:
      enabled: true
      apiVersion: "networking.k8s.io/v1"
      hostName: myapp.mydomain.com
      path: /
      kubernetes.io/ssl-redirect: true
      labels: {}
      annotations:
        kubernetes.io/ingress.class: ingress-public
        nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
        nginx.ingress.kubernetes.io/auth-url: "http://oauth2-proxy.oauth2.svc.cluster.local/oauth2/auth"
        nginx.ingress.kubernetes.io/auth-signin: "https://login.nonprod.gmc.js-devops.co.uk/oauth2/start?rd=https://$best_http_host$escaped_request_uri"
        nginx.ingress.kubernetes.io/auth-response-headers: X-Auth-Request-Email, X-Auth-Request-User, X-Auth-Request-Access-Token

All good there!

Now, with the new version I changed the oauth2-proxy config like;

    extraArgs:
      provider: oidc
      oidc-issuer-url: "https://login.microsoftonline.com/<azure-tenant-id>/v2.0"
      azure-tenant: "<azure-tenant-id>"
      login-url: https://login.microsoftonline.com/<azure-tenant-id>/oauth2/v2.0/authorize
      redeem-url: https://login.microsoftonline.com/<azure-tenant-id>/oauth2/v2.0/token
      oidc-jwks-url: https://login.microsoftonline.com/common/discovery/keys
      profile-url: https://graph.microsoft.com/v1.0/me
      scope: 'email profile openid offline_access'
      email-domain: "*"
      skip-jwt-bearer-tokens: "true"

      cookie-name: _oauth2_test
      cookie-refresh: 1m
      cookie-domain: .mydomain.com
      whitelist-domain: .mydomain.com

      set-authorization-header: "true"
      set-xauthrequest: "true"
      pass-access-token: "true"
      pass-authorization-header: "true"
      pass-basic-auth: "true"
      pass-host-header: "true"
      pass-user-headers: "true"

      session-store-type: redis
      redis-connection-url: redis://test-redis-redis-ha
      redis-use-sentinel: "true"
      redis-sentinel-master-name: mymaster
      redis-sentinel-connection-urls: redis://test-redis-redis-ha:26379

I tried with 'provider: azure', but that did not work! I switched to 'oidc' after searching around.

My ingress looks like;

    ingress:
      enabled: true
      apiVersion: "networking.k8s.io/v1"
      labels: {}
      annotations:
        kubernetes.io/ingress.class: ingress-public
        nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
        nginx.ingress.kubernetes.io/ssl-redirect: "true"
        nginx.ingress.kubernetes.io/auth-url: "http://oauth2-proxy.oauth2-staging.svc.cluster.local/oauth2/auth"
        nginx.ingress.kubernetes.io/auth-signin: "https://oauth2.nonprod.gmc.js-devops.co.uk/oauth2/start?rd=https://$best_http_host$escaped_request_uri"
        nginx.ingress.kubernetes.io/auth-response-headers: "X-Auth-Request-Email,X-Auth-Request-User,X-Auth-Request-Access-Token,X-Auth-Request-Preferred-Username"

Now this combination authenticates, but 'myapp' expects an email to be passed to it during the authentication process/oauth2-proxy redirect in order to provide the correct level of access. What is getting passed is a user id like 'fF3CtUwz6hwiCWjpR3kgHCYs0Uj3Syk7sTAUs1pOJSU' instead of an email like me@mydomain.com, thus I get logged in, but do not have the expected access.

I have tried a few different combinations with the app ingress annotations and oauth2-proxy config but nothing seems to change in with regard to getting the email passed to the app. I also tried this annotation for the app ingress, but no luck;

    nginx.ingress.kubernetes.io/configuration-snippet: |
          auth_request_set $name_upstream_1 $upstream_cookie_name_1;
          access_by_lua_block {
              if ngx.var.name_upstream_1 ~= "" then
              ngx.header["Set-Cookie"] = "name_1=" .. ngx.var.name_upstream_1 .. ngx.var.auth_cookie:match("(; .*)")
              end
          }

Am I missing an annotation or is my config off? Thanks!

Solution

OK, I have finally sorted this. It turns out that I was focussing on the wrong thing. The issue turned out to be the application I was testing whereby the incorrect header was being used, 'X-Auth-Request-User' instead of 'X-Auth-Request-Email'. As soon as I swapped this, the correct permissions were assigned to the authenticated user