I'm attempting to automatically pull the latest version of a GitHub repo into my Databricks workspace every time a new push is made to the repo. Everything works fine until the Databricks CLI requests the host URL after which it fails with "Error: Process completed with exit code 1." I'm assuming it's an issue with my token and host credentials stored as secrets not properly loading into the environment. According to Databricks, "CLI 0.8.0 and above supports the following environment variables: DATABRICKS_HOST, DATABRICKS_USERNAME, DATABRICKS_PASSWORD, DATABRICKS_TOKEN". I've added both DATABRICKS_HOST and DATABRICKS_TOKEN as repository secrets, so I'm not sure what I'm doing wrong.
on:
push:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: setup python
uses: actions/setup-python@v2
with:
python-version: 3.8 #install the python version needed
- name: execute py
env:
DATABRICKS_HOST: $(DATABRICKS_HOST)
DATABRICKS_TOKEN: $(DATABRICKS_TOKEN)
run: |
python -m pip install --upgrade databricks-cli
databricks configure --token
databricks repos update --repo-id REPOID-ENTERED --branch "Development"
The error:
Successfully built databricks-cli
Installing collected packages: tabulate, certifi, urllib3, six, pyjwt, oauthlib, idna, click, charset-normalizer, requests, databricks-cli
Successfully installed certifi-2021.10.8 charset-normalizer-2.0.12 click-8.1.3 databricks-cli-0.16.6 idna-3.3 oauthlib-3.2.0 pyjwt-2.4.0 requests-2.27.1 six-1.16.0 tabulate-0.8.9 urllib3-1.26.9
WARNING: You are using pip version 22.0.4; however, version 22.1 is available.
You should consider upgrading via the '/opt/hostedtoolcache/Python/3.8.12/x64/bin/python -m pip install --upgrade pip' command.
Aborted!
Databricks Host (should begin with https://):
Error: Process completed with exit code 1.
Just remove databricks configure --token from your command - it's not required. Databricks CLI will use environment variables in this case. See working pipeline for Azure DevOps here.