I have been trying to use Google Cloud Functions (python 3.7) to basically clone a Google Source Repository. I'm using GitPython library and the service account for that Cloud Functions has Source Repository Reader access on the repository that I want to clone.
Initially, I tried passing gcloud.sh credential.helper to git config, but it seems that Cloud SDK is not installed on Cloud Functions environment (at least in Python 3.7 environment). Here is a gist of my code:
from git import Repo
import tempfile
def git_clone():
local_repo = tempfile.gettempdir()+'/localrepo'
repo = Repo.init(local_repo)
origin = repo.create_remote('origin', 'https://source.developers.google.com/p/PROJECT/r/REPO')
repo.config_writer().set_value("user", "name", "gsr-to-gcs-backup-function").release()
repo.config_writer().set_value("user", "email", "gsr-to-gcs-backup-function@PROJECT.iam.gserviceaccount.com").release()
repo.config_writer().set_value("credential \"https://source.developers.google.com\"", "helper", "gcloud.sh").release()
assert origin.exists()
assert origin == repo.remotes.origin == repo.remotes['origin']
origin.fetch()
The function will throw the below error if run on Cloud Functions since by default if there is no credential helper, the https method will ask for Username and Password.
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128) cmdline: git fetch -v origin stderr: 'fatal: could not read Username for 'https://source.developers.google.com': Input/output error'
I could only find the below answer to pass token along with the git clone command but it doesn't answer how to pass the token:
Clone Google Source Repository without gcloud
If I initiate that command from cloud shell, it will be just hung:
gcloud auth git-helper store --account=YOUR_ACCOUNT --ignore-unknown $@
Here's something similar that I plan to achieve using the above (not a correct code):
Repo.clone_from("https://source.developers.google.com/p/PROJECT/r/REPO",tempfile.gettempdir(), multi_options=['--config credential.helper={}'.format(MYTOKEN)], branch='master')
I don't want to put an SSH key instead as a method to clone as I want to deploy it to production later on and rotating keys would be cumbersome.
The Username that is asked is nothing but the service account address and credentials can be a temporarily generated OAuth 2.0 access token. So, what I ended up doing was:
from git import Repo
import tempfile
import urllib.parse
import google.auth
import google.auth.transport.requests
def get_token():
creds, project = google.auth.default()
# creds.valid is False, and creds.token is None
# # Need to refresh credentials to populate those
auth_req = google.auth.transport.requests.Request()
creds.refresh(auth_req)
# Now you can use creds.token
return creds.token
def url_parse():
query = 'gsr-to-gcs-backup-function@PROJECT.iam.gserviceaccount.com'
return urllib.parse.quote(query)
def git_clone():
encoded_url = url_parse()
credentials = get_token()
local_repo = tempfile.gettempdir()+'/localrepo'
Repo.clone_from(f"https://{encoded_url}:{credentials}@source.developers.google.com/p/PROJECT/r/REPO",local_repo, branch='master')
def main(*args):
git_clone()
NOTE: This is only a part of my code. What I want is to copy this to a GCS bucket. But that is out of purview of this question.