I'm trying to create a github action that runs a python script that has access to google drive. I use OIDC to auth to google drive in the github action, but I'm banging my head against a wall trying to figure out how to connect to google in the python file. I'm using pydrive2.
Clearly I'm going about the python authentication to google piece incorrectly, but how? My expectation is that the file created by github OIDC would properly create credentials file that could be directly used to authenticate.
My github action looks like this:
name: 'Test Auth'
on: [workflow_dispatch, push]
jobs:
ExploreActions:
permissions:
contents: 'read'
id-token: 'write'
runs-on: ubuntu-latest
steps:
# actions/checkout MUST come before auth
- uses: 'actions/checkout@v3'
with:
ref: ${{ github.ref }}
- name: 'Authenticate to Google Cloud'
uses: 'google-github-actions/auth@v1'
with:
workload_identity_provider: 'projects/###############/locations/global/workloadIdentityPools/my-pool/providers/my-provider'
service_account: '<user>@<service>.iam.gserviceaccount.com'
create_credentials_file: true
- name: Install Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install Dependencies
run: pip install pydrive2
shell: bash
- name: Test python
run: python experimental/test_action.py
shell: bash
My python file is simple:
from pydrive2.auth import GoogleAuth
from pydrive2.drive import GoogleDrive
import os
creds = os.environ.get("GOOGLE_APPLICATION_CREDENTIALS")
print(creds)
gauth = GoogleAuth()
gauth.LoadCredentialsFile(creds)
gauth.LocalWebserverAuth()
drive = GoogleDrive(gauth)
When the action runs, I get the following output (editted for clarity):
2023-06-13T19:52:03.2759862Z Requested labels: ubuntu-latest
2023-06-13T19:52:14.2353436Z ##[group]Run google-github-actions/auth@v1
2023-06-13T19:52:14.2353828Z with:
2023-06-13T19:52:14.2354261Z workload_identity_provider: projects/########/locations/global/workloadIdentityPools/my-pool/providers/my-provider
2023-06-13T19:52:14.2354906Z service_account: xxxxxxxxx
2023-06-13T19:52:14.2355322Z create_credentials_file: true
2023-06-13T19:52:14.2355641Z export_environment_variables: true
2023-06-13T19:52:14.2355973Z cleanup_credentials: true
2023-06-13T19:52:14.2356231Z access_token_lifetime: 3600s
2023-06-13T19:52:14.2356660Z access_token_scopes: https://www.googleapis.com/auth/cloud-platform
2023-06-13T19:52:14.2357018Z retries: 3
2023-06-13T19:52:14.2357239Z backoff: 250
2023-06-13T19:52:14.2357534Z id_token_include_email: false
2023-06-13T19:52:14.2357818Z ##[endgroup]
2023-06-13T19:52:14.4899340Z Created credentials file at "/home/runner/work/small_projects/small_projects/gha-creds-e6ba77a2da78ef5b.json"
2023-06-13T19:52:14.5123821Z ##[group]Run actions/setup-python@v4
2023-06-13T19:52:23.2009265Z pythonLocation: /opt/hostedtoolcache/Python/3.11.4/x64
2023-06-13T19:52:23.2009636Z PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.4/x64/lib/pkgconfig
2023-06-13T19:52:23.2010012Z Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.4/x64
2023-06-13T19:52:23.2010370Z Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.4/x64
2023-06-13T19:52:23.2010724Z Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.4/x64
2023-06-13T19:52:23.2011067Z LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.4/x64/lib
2023-06-13T19:52:23.2011362Z ##[endgroup]
2023-06-13T19:52:23.6651848Z /home/runner/work/small_projects/small_projects/gha-creds-e6ba77a2da78ef5b.json
2023-06-13T19:52:23.6654092Z Traceback (most recent call last):
2023-06-13T19:52:23.6665200Z File "/home/runner/work/small_projects/small_projects/experimental/test_action.py", line 9, in <module>
2023-06-13T19:52:23.6668147Z gauth.LoadCredentialsFile(creds)
2023-06-13T19:52:23.6670573Z File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/pydrive2/auth.py", line 418, in LoadCredentialsFile
2023-06-13T19:52:23.6673662Z self.credentials = self._default_storage.get()
2023-06-13T19:52:23.6675655Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13T19:52:23.6677537Z File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/oauth2client/client.py", line 407, in get
2023-06-13T19:52:23.6680768Z return self.locked_get()
2023-06-13T19:52:23.6688395Z ^^^^^^^^^^^^^^^^^
2023-06-13T19:52:23.6690735Z File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/oauth2client/file.py", line 54, in locked_get
2023-06-13T19:52:23.6693956Z credentials = client.Credentials.new_from_json(content)
2023-06-13T19:52:23.6696512Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13T19:52:23.6698658Z File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/oauth2client/client.py", line 302, in new_from_json
2023-06-13T19:52:23.6702354Z module_name = data['_module']
2023-06-13T19:52:23.6704914Z ~~~~^^^^^^^^^^^
2023-06-13T19:52:23.6706966Z KeyError: '_module'
2023-06-13T19:52:23.7152157Z ##[error]Process completed with exit code 1.
########################### Edit
With the suggestion from @Saxtheowl, I changed the python file to auth using the google auth apis and got it to work. A note that I had to actually share a file in google drive with my service user in order to get any files to show up:
import os
import google.auth
import googleapiclient.discovery
creds, project = google.auth.default(scopes=['https://www.googleapis.com/auth/drive.readonly'])
service = googleapiclient.discovery.build('drive', 'v3', credentials=creds)
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
print(items)
I think the credentials file created by GitHub doesn't match the format that PyDrive2 expects.
You could try Google's google-auth library
from google.oauth2.service_account import Credentials
from googleapiclient.discovery import build
creds = Credentials.from_service_account_file(
os.environ.get("GOOGLE_APPLICATION_CREDENTIALS"),
# Add the scope you need to access
scopes=["https://www.googleapis.com/auth/drive"]
)
service = build('drive', 'v3', credentials=creds)
Now you should have a authenticated Drive service you can use to interact with Google Drive.