I've given the service account for the functions the necessary permissions ('Secret Manager Secret Accessor') and when deployed, the firebase functions are able to access the secrets without any problems.
However, when using firebase serve or firebase emulators:start --only functions in local development, I'm getting the following error
Unhandled error Error: 7 PERMISSION_DENIED: Permission 'secretmanager.versions.access' denied for resource
I've found in the documentation that setting export GOOGLE_APPLICATION_CREDENTIALS=pathtoserviceaccount.json is needed to be entered in the terminal, though this did also not work for me.
I would be thankful for all pointers. Cheers.
I've found the answer myself:
When the functions are emulated locally, they do not get run by the App Engine default service account per default, this needs to be enabled as well.
So I had to follow this tutorial https://firebase.google.com/docs/functions/local-shell
The App Engine default service account needs a key which can be created in the Service Accounts settings in the Google Cloud, and then
I had to enter
export GOOGLE_APPLICATION_CREDENTIALS="path/to/key.json"
in the terminal. By running then firebase emulators:start they also got permission to access the Secret Manager.
So while I was on the right track, I was exporting the wrong Service Account key, and not the one that was allowed to run access the Secret Manager.