gitgoogle-cloud-functionsgitpythongoogle-cloud-source-reposgoogle-source-repositories

Is there a way to pass access tokens when calling git clone over HTTPS for Google Source Repository


I have been trying to use Google Cloud Functions (python 3.7) to basically clone a Google Source Repository. I'm using GitPython library and the service account for that Cloud Functions has Source Repository Reader access on the repository that I want to clone.

Initially, I tried passing gcloud.sh credential.helper to git config, but it seems that Cloud SDK is not installed on Cloud Functions environment (at least in Python 3.7 environment). Here is a gist of my code:

from git import Repo
import tempfile

def git_clone():
    
    local_repo = tempfile.gettempdir()+'/localrepo'
    repo = Repo.init(local_repo)
    origin = repo.create_remote('origin', 'https://source.developers.google.com/p/PROJECT/r/REPO')

    repo.config_writer().set_value("user", "name", "gsr-to-gcs-backup-function").release()
    repo.config_writer().set_value("user", "email", "gsr-to-gcs-backup-function@PROJECT.iam.gserviceaccount.com").release()
    repo.config_writer().set_value("credential \"https://source.developers.google.com\"", "helper", "gcloud.sh").release()

    assert origin.exists()
    assert origin == repo.remotes.origin == repo.remotes['origin']
    
    origin.fetch()

The function will throw the below error if run on Cloud Functions since by default if there is no credential helper, the https method will ask for Username and Password.

git.exc.GitCommandError: Cmd('git') failed due to: exit code(128) cmdline: git fetch -v origin stderr: 'fatal: could not read Username for 'https://source.developers.google.com': Input/output error'

I could only find the below answer to pass token along with the git clone command but it doesn't answer how to pass the token:

Clone Google Source Repository without gcloud

If I initiate that command from cloud shell, it will be just hung:

gcloud auth git-helper store --account=YOUR_ACCOUNT --ignore-unknown $@

Here's something similar that I plan to achieve using the above (not a correct code):

Repo.clone_from("https://source.developers.google.com/p/PROJECT/r/REPO",tempfile.gettempdir(), multi_options=['--config credential.helper={}'.format(MYTOKEN)], branch='master')

I don't want to put an SSH key instead as a method to clone as I want to deploy it to production later on and rotating keys would be cumbersome.


Solution

  • The Username that is asked is nothing but the service account address and credentials can be a temporarily generated OAuth 2.0 access token. So, what I ended up doing was:

    from git import Repo
    import tempfile
    import urllib.parse
    import google.auth
    import google.auth.transport.requests
    
    def get_token():
        creds, project = google.auth.default()
        # creds.valid is False, and creds.token is None
        # # Need to refresh credentials to populate those
        auth_req = google.auth.transport.requests.Request()
        creds.refresh(auth_req)
        # Now you can use creds.token
        return creds.token
    
    def url_parse():
        query = 'gsr-to-gcs-backup-function@PROJECT.iam.gserviceaccount.com'
        return urllib.parse.quote(query)
    
    def git_clone():
        encoded_url = url_parse()
        credentials = get_token()
        
        local_repo = tempfile.gettempdir()+'/localrepo' 
        Repo.clone_from(f"https://{encoded_url}:{credentials}@source.developers.google.com/p/PROJECT/r/REPO",local_repo, branch='master')
    
    def main(*args):
        git_clone()
    

    NOTE: This is only a part of my code. What I want is to copy this to a GCS bucket. But that is out of purview of this question.