azure-active-directoryairflowazure-aksairflow-2.xairflow-webserver

Airflow ERROR - Error authorizing OAuth access token: Invalid JSON Web Key Set. [The request to sign in was denied.]


I am trying to integrate Azure EntraID authentication with Airflow [2.9.3]. The authentication is completed but the authorization is giving error like "The request to sign in was denied.". I am using Helm chart [1.15.0] for deployment.

Pod Error Logs

Airflow access error

I have created below security groups in Azure.

airflow_nonprod_admin
airflow_nonprod_op
airflow_nonprod_viewer

In Azure Enterprise Application the SAML configuration under Single SIgn On as below..

Identifier (Entity ID) :: https://airflow.xyz.com/
Reply URL (Assertion Consumer Service URL) :: https://airflow.xyz.com/oauth-authorized/azure
Sign on URL :: https://airflow.xyz.com/login/
Relay State (Optional) :: https://airflow.xyz.com/home
Logout Url (Optional) :: https://airflow.xyz.com/logout

The airflow webserverconfig code snippet as below

            from __future__ import annotations

            import os

            from airflow.www.fab_security.manager import AUTH_OAUTH
            # from airflow.www.security import AirflowSecurityManager
            from airflow.auth.managers.fab.security_manager.override import FabAirflowSecurityManagerOverride
            from airflow.utils.log.logging_mixin import LoggingMixin

            basedir = os.path.abspath(os.path.dirname(__file__))

            # Flask-WTF flag for CSRF
            WTF_CSRF_ENABLED = True
            WTF_CSRF_TIME_LIMIT = None
            AAD_TENANT_ID = <tenant id>
            AAD_CLIENT_ID = <APP Registration client id>
            AAD_CLIENT_SECRET = <App Registration client secret>

            AUTH_TYPE = AUTH_OAUTH

            OAUTH_PROVIDERS = [{
                'name':'azure',
                'token_key':'access_token',
                'icon':'fa-windows',
                'remote_app': {
                    'api_base_url': f"https://login.microsoftonline.com/{AAD_TENANT_ID}",
                    'client_kwargs': {
                        "scope": "User.read name preferred_username email profile upn",
                        "resource": f"{AAD_CLIENT_ID}",
                        # Optionally enforce signature JWT verification
                        "verify_signature": False
                    },            
                    'request_token_url': None,
                    'request_token_params': {
                        'scope': 'openid email profile'
                    },
                    'access_token_url': f"https://login.microsoftonline.com/{AAD_TENANT_ID}/oauth2/v2.0/token",
                    "access_token_params": {
                        'scope': 'openid email profile'
                    },
                    'authorize_url': f"https://login.microsoftonline.com/{AAD_TENANT_ID}/oauth2/v2.0/authorize",
                    "authorize_params": {
                        'scope': 'openid email profile'
                    },
                    'client_id': f"{AAD_CLIENT_ID}",
                    'client_secret': f"{AAD_CLIENT_SECRET}",
                    'jwks_uri': 'https://login.microsoftonline.com/common/discovery/v2.0/keys',
                    'redirect_uri': 'https://airflow.xyz.com/oauth-authorized/azure'            
                }
            }]

            AUTH_USER_REGISTRATION_ROLE = "Public"
            AUTH_USER_REGISTRATION = True
            AUTH_ROLES_SYNC_AT_LOGIN = True
            # First you MUST create a role like"Admin with value Admin" in the App Registration "App Roles" section in the Azure Portal under Microsoft Entra ID.
            # Then groups MUST be linked from the Microsoft Entra ID "Enterprise Application" section in the Azure Portal under the "Users and Groups" section.
            # Each groups or users MUST be assigned a role e.g.: Admin, Op, Viewer in the "Users and Groups"
            AUTH_ROLES_MAPPING = {
                "airflow_nonprod_admin": ["Admin"],
                "airflow_nonprod_op": ["Op"],
                "airflow_nonprod_viewer": ["Viewer"],
            }

            class AzureCustomSecurity(FabAirflowSecurityManagerOverride, LoggingMixin):
                def get_oauth_user_info(self, provider, response=None):
                    self.log.debug(f"Parsing JWT token for provider : {provider}")

                    try:   # the try and except are optional - strictly you only need the me= line.
                        me = super().get_oauth_user_info(provider, response)
                    except Exception as e:
                        import traceback
                        traceback.print_exc()
                        self.log.debug(e)

                    self.log.debug(f"Parse JWT token : {me}")
                    return {
                        "name": me["userprincipalname"],
                        "email": me["mail"],
                        "first_name": me["givenname"],
                        "last_name": me["surname"],
                        "id": me["userprincipalname"],
                        "username": me["givenname"],
                        "role_keys": me["groups"]
                    }

            # the first of these two appears to work with older Airflow versions, the latter newer.
            FAB_SECURITY_MANAGER_CLASS = 'webserver_config.AzureCustomSecurity'
            SECURITY_MANAGER_CLASS = AzureCustomSecurity

Solution

  • After adding User.Read in all parameters like below it works.

    OAUTH_PROVIDERS = [{
      'name': 'azure',
      'token_key': 'access_token',
      'icon': 'fa-windows',
      'remote_app': {
        'api_base_url': f "https://login.microsoftonline.com/{AAD_TENANT_ID}/",
        'client_kwargs': {
          "scope": "User.read name preferred_username email profile upn",
          "resource": f "{AAD_CLIENT_ID}",
          #Optionally enforce signature JWT verification "verify_signature": False
        },
        'request_token_url': None,
        'request_token_params': {
          'scope': 'User.read openid email profile'
        },
        'access_token_url': f "https://login.microsoftonline.com/{AAD_TENANT_ID}/oauth2/v2.0/token",
        "access_token_params": {
          'scope': 'User.read openid email profile',
        },
        'authorize_url': f "https://login.microsoftonline.com/{AAD_TENANT_ID}/oauth2/v2.0/authorize",
        "authorize_params": {
          'scope': 'User.read openid email profile',
        },
        'client_id': f "{AAD_CLIENT_ID}",
        'client_secret': f "{AAD_CLIENT_SECRET}",
        'jwks_uri': 'https://login.microsoftonline.com/common/discovery/v2.0/keys',
        'redirect_url': 'https://airflow.xyz.com/oauth-authorized/azure',
      }
    }]