Service account request to IAP-protected app results in 'Invalid GCIP ID token: JWT signature is invalid'

I am trying to programmatically access an IAP-protected App Engine Standard app via Python from outside of the GCP environment. I have tried various methods, including the method shown in the docs here: https://cloud.google.com/iap/docs/authentication-howto#iap-make-request-python. Here is my code:

from google.auth.transport.requests import Request
from google.oauth2 import id_token
import requests

def make_iap_request(url, client_id, method='GET', **kwargs):
    """Makes a request to an application protected by Identity-Aware Proxy.

    Args:
      url: The Identity-Aware Proxy-protected URL to fetch.
      client_id: The client ID used by Identity-Aware Proxy.
      method: The request method to use
              ('GET', 'OPTIONS', 'HEAD', 'POST', 'PUT', 'PATCH', 'DELETE')
      **kwargs: Any of the parameters defined for the request function:
                https://github.com/requests/requests/blob/master/requests/api.py
                If no timeout is provided, it is set to 90 by default.

    Returns:
      The page body, or raises an exception if the page couldn't be retrieved.
    """
    # Set the default timeout, if missing
    if 'timeout' not in kwargs:
        kwargs['timeout'] = 90

    # Obtain an OpenID Connect (OIDC) token from metadata server or using service
    # account.
    open_id_connect_token = id_token.fetch_id_token(Request(), client_id)
    print(f'{open_id_connect_token=}')

    # Fetch the Identity-Aware Proxy-protected URL, including an
    # Authorization header containing "Bearer " followed by a
    # Google-issued OpenID Connect token for the service account.
    resp = requests.request(
        method, url,
        headers={'Authorization': 'Bearer {}'.format(
            open_id_connect_token)}, **kwargs)
    print(f'{resp=}')
    if resp.status_code == 403:
        raise Exception('Service account does not have permission to '
                        'access the IAP-protected application.')
    elif resp.status_code != 200:
        raise Exception(
            'Bad response from application: {!r} / {!r} / {!r}'.format(
                resp.status_code, resp.headers, resp.text))
    else:
        return resp.text

if __name__ == '__main__':
    res = make_iap_request(
        'https://MYAPP.ue.r.appspot.com/',
        'Client ID from IAP>App Engine app>Edit OAuth Client>Client ID'
        )
    print(res)

When I run it locally, I have the GOOGLE_APPLICATION_CREDENTIALS environment variable set to a local JSON credential file containing the keys for the service account I want to use. I have also tried running this in Cloud Functions so it would presumably use the metadata service to pick up the App Engine default service account (I think?).

In both cases, I am able to generate a token that appears valid. Using jwt.io, I see that it contains the expected data and the signature is valid. However, when I make a request to the app using the token, I always get this exception:

Bad response from application: 401 / {'X-Goog-IAP-Generated-Response': 'true', 'Date': 'Tue, 09 Feb 2021 19:25:43 GMT', 'Content-Type': 'text/html', 'Server': 'Google Frontend', 'Content-Length': '47', 'Alt-Svc': 'h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"'} / 'Invalid GCIP ID token: JWT signature is invalid'

What could I be doing wrong?

The solution to this problem is to exchange the Google Identity Token for an Identity Platform Identity Token.

The reason for the error Invalid GCIP ID token: JWT signature is invalid is caused by using a Google Identity Token which is signed by a Google RSA private key and not by a Google Identity Platform RSA private key. I overlooked GCIP in the error message, which would have told me the solution once we validated that the token was not corrupted in use.

In the question, this line of code fetches the Google Identity Token:

open_id_connect_token = id_token.fetch_id_token(Request(), client_id)

The above line of code requires that Google Cloud Application Default Credentials are setup. Example: set GOOGLE_APPLICATION_CREDENTIALS=c:\config\service-account.json

The next step is to exchange this token for an Identity Platform token:

def exchange_google_id_token_for_gcip_id_token(google_open_id_connect_token):
    SIGN_IN_WITH_IDP_API = 'https://identitytoolkit.googleapis.com/v1/accounts:signInWithIdp'
    API_KEY = '';

    url = SIGN_IN_WITH_IDP_API + '?key=' + API_KEY;

    data={
        'requestUri': 'http://localhost',
        'returnSecureToken': True,
        'postBody':'id_token=' + google_open_id_connect_token + '&providerId=google.com'}

    try:
        resp = requests.post(url, data)

        res = resp.json()

        if 'error' in res:
            print("Error: {}".format(res['error']['message']))
            exit(1)

        # print(res)

        return res['idToken']
    except Exception as ex:
        print("Exception: {}".format(ex))
        exit(1)

The API Key can be found in the Google Cloud Console -> Identity Platform. Top right "Application Setup Details". This will show the apiKey and authDomain.

More information can be found at this link:

Exchanging a Google token for an Identity Platform token