403 The caller does not have permission when calling Apps Script API from a Cloud Function

I'm building a Workspace addon that you launch from Google Drive and that will be executed as a background task.

To do so my apps script addon calls a Google Cloud Function that calls the Apps Script back using the Apps Script API. I do see because the process can take quite some time.

In my addon, I have declared some scopes:

 "oauthScopes": [
    "https://www.googleapis.com/auth/userinfo.email",
    "https://www.googleapis.com/auth/script.external_request",
    "https://www.googleapis.com/auth/drive.addons.metadata.readonly",
    "https://www.googleapis.com/auth/drive.file",
    "https://www.googleapis.com/auth/script.scriptapp"
  ],

1. The user opens the addon, it has to grant access to those scopes
2. He selects a Google Drive file on which the process will be executed. He pushes a button.
3. My Apps script calls a Google Cloud Function and sends the Oauth token of the user
4. The Cloud Function calls my Apps Script function, with the Apps Script API, using the Oauth token as authentication bearer
5. When someone, not me, tries the addon (someone who has no permission on the GCP project nor the Apps Script project), it fails with a 403 error.

Both the Apps Script and the Cloud Function are on the same GCP project and the Apps Script API is enabled.

Calling the GCF from Apps Script

const userToken = ScriptApp.getOAuthToken();
const resp = UrlFetchApp.fetch(BACKGROUND_GCF_URL + '?userToken=' + userToken + '&fileId=' + fileId + '&execUid=' + execUid, {
    headers: {
      Authorization: 'Bearer ' + bearer // it is here an identity token that I've generated
    },
    muteHttpExceptions: true
})

Calling the Apps Script API from the GCF

const body = {
    function: 'receiveGCFcall',
    parameters: [
        execUid, fileId, userToken
    ],
    devMode: true
 }

 const config = {
     headers: {
         Authorization: 'Bearer ' + userToken
     }
 }
 
 axios
  .post(URL, body, config)
  .then(resp => {
    console.log(`statusCode: ${resp.status}`);
    console.log(`data: ${JSON.stringify(resp.data)}`)
    if(resp.data.result === 'NOT OK') {
      console.error(`[${execUid}] Something went wrong, the slides have not been updated`)
    }
  }).catch(...) // it goes here with a 403 error

A drawing if it helps

• The Apps Script project is deployed as a Workspace addon (and currently being reviewed by the Marketplace team). The project is not shared to anyone but me.
• I use a service account (and its identity token) to call the GCF from the Apps Script
• I try to use the connected user Oauth token when calling back the Apps Script from the GCF
• I have a "executionAPI access: anyone" in the manifest, but I'm not sure if it's useful

edit after few other tries, I figured out that if I try the process with an account who is editor of the Apps Script project, all works fine. If the account is only reader on the Apps Script, or nothing, it won't work. So basically the bearer that I put in the header, from GCF, to Apps Script, seems not to be enough to authorize an account...

My friend @st3ph found the problem: I have to remove the devMode: true when calling the Apps Script API cause this dev mode is only available for users who have write access to the Apps Script project.