I use Google Functions in my project.
For the function I have a system test based on this concept in the docs
The assertion in the test uses the following check to determine if the function has been running and pass the test
const logs = childProcess
.execSync(`gcloud functions logs read ${functionName} --start-time=${startTime}`)
.toString();
It works perfect on my own system when testing locally. However when I run my tests through Gitlab CI pipeline I get the following error everytime,
Pub sub system tests
should make a new log when a device make an unlock pub sub:
Error: Command failed: gcloud functions logs read --account=<Service Account name manually removed from log> --start-time=2022-09-19T11:26:03.762Z
ERROR: (gcloud.functions.logs.read) PERMISSION_DENIED: Permission 'logging.views.access' denied on resource (or it may not exist).
- '@type': type.googleapis.com/google.rpc.ErrorInfo
domain: iam.googleapis.com
metadata:
permission: logging.views.access
reason: IAM_PERMISSION_DENIED
at checkExecSyncError (node:child_process:871:11)
at Object.execSync (node:child_process:943:15)
at /builds/<My ACcount infortmation and my repo name>/log-functions/test/system/pubsub.system.test.ts:127:18
In the gitlab-ci.yml file I choose a service account with this command before the test are run:
gcloud auth activate-service-account --key-file=keyfile.json
I have verified that the correct account has been used in the log from the gitlab pipeline.
Furthermore, in pure desperation, have tried and give the account all the log related permissions I could find in Google Cloud Console. Among other I have given the service account the following roles:
But the same error keeps appearing.
At this point I feel like I have tried everything. I have no more ideas. So I am hoping that somebody can give me a pointer in the right direction.
Any help is appreciated.
Through several methods some suggested by @DazWilkin I ended up solving my issue.
Leaving an answer here with the general directions in case anybody in a similar situation finds it.
I think the most confusing thing about these problems are that running the tests on my own machine uses my own gcloud environment which must have some different configuration. Because even though I activated the same service account it behaved differently. I repeatedly experienced that the tests where working in my local machine but failing in the Gitlab CI pipeline. With the same commands.
Therefore I would suggest starting to test the commands in an interactive shell in a docker machine with the same setup than you use for the pipeline. That way you encounter exactly the same issues with permissions in you shell session as in the pipeline.
I simply used this command docker run -it node:lastest /bin/bash to start a shell session in a docker machine where I could try and run the same commands as the pipeline in exactly the same environment.
Furthermore what was extremely helpful was the suggestion from @DazWilkin to express the project (and most other things explicitly). As soon as you specify everything explicitly you might get a much more precise description of the errors. Doing it in the interactive docker shell meant that I could iterate quite quickly compared to running everything in the pipeline and came up with a solution.
In the end I ended up specifying a lot of things in the command. Even the region of the function. The command I ended up using is here:
const logs = childProcess
.execSync(`gcloud functions logs read ${functionName} --start-time=${startTime} --project=${testProjectID} --region=${process.env.GCF_REGION}`)
.toString();
Regarding permissions I ended up creating a custom role in Google Cloud which only contains the permission logging.views.accessto my service account, as that was more than enough.