I am trying to make unit tests for my azure functions, written in Python. I have a python file that does some setup (making the cosmos db databases and containers) and I do have a github actions yaml file to pull a docker container and then run the scripts.
The error: For some reason, I do get an error when running the Python script: azure.core.exceptions.ServiceRequestError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1006)
I have already tried to install the CA certificate, provided by the docker container. I think this worked correctly but the error still persists.
The yaml file:
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Start Cosmos DB Emulator
run: docker run --detach --publish 8081:8081 --publish 1234:1234 mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
- name: pause
run : sleep 120
- name : emulator certificate
run : |
retry_count=0
max_retry_count=10
until sudo curl --insecure --silent --fail --show-error "https://localhost:8081/_explorer/emulator.pem" --output "/usr/local/share/ca-certificates/cosmos-db-emulator.crt"; do
if [ $retry_count -eq $max_retry_count ]; then
echo "Failed to download certificate after $retry_count attempts."
exit 1
fi
echo "Failed to download certificate. Retrying in 5 seconds..."
sleep 5
retry_count=$((retry_count+1))
done
sudo update-ca-certificates
sudo ls /etc/ssl/certs | grep emulator
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Cache dependencies
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Set up Azure Functions Core Tools
run: |
wget -q https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install azure-functions-core-tools-4
- name: Log in with Azure
uses: azure/login@v1
with:
creds: '${{ secrets.AZURE_CREDENTIALS }}'
- name: Start Azurite
run: |
docker run -d -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite
- name: Wait for Azurite to start
run: sleep 5
- name: Get Emulator Connection String
id: get-connection-string
run: |
AZURE_STORAGE_CONNECTION_STRING="AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VR2Vo3Fl+QUFOzQYzRPgAzF1jAd+pQ==;"
echo "AZURE_STORAGE_CONNECTION_STRING=${AZURE_STORAGE_CONNECTION_STRING}" >> $GITHUB_ENV
- name: Setup test environment in Python
run : python Tests/setup.py
- name: Run tests
run: |
python -m unittest discover Tests
The Python script
urllib3.disable_warnings()
print(DEFAULT_CA_BUNDLE_PATH)
connection_string : str = os.getenv("COSMOS_DB_CONNECTION_STRING")
database_client_string : str = os.getenv("COSMOS_DB_CLIENT")
container_client_string : str = os.getenv("COSMOS_DB_CONTAINER_MEASUREMENTS")
cosmos_client : CosmosClient = CosmosClient.from_connection_string(
conn_str=connection_string
)
cosmos_client.create_database(
id=database_client_string,
offer_throughput=400
)
database_client : DatabaseProxy = cosmos_client.get_database_client(database_client_string)
database_client.create_container(
id=container_client_string,
partition_key=PartitionKey(path="/path")
)
Output of the certificate installation step
Updating certificates in /etc/ssl/certs...
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
/etc/ssl/certs/adoptium/cacerts successfully populated.
Updating Mono key store
Mono Certificate Store Sync - version 6.12.0.200
Populate Mono certificate store from a concatenated list of certificates.
Copyright 2002, 2003 Motus Technologies. Copyright 2004-2008 Novell. BSD licensed.
Importing into legacy system store:
I already trust 146, your new list has 147
Certificate added: CN=localhost
1 new root certificates were added to your trust store.
Import process completed.
Importing into BTLS system store:
I already trust 146, your new list has 147
Certificate added: CN=localhost
1 new root certificates were added to your trust store.
Import process completed.
Done
done.
cosmos-db-emulator.pem
My thoughts I think that the issue arrises at the part where I create the database in Python script. Once I comment those lines, the error will not show. But I do need it :)
Question Why might my solution not have worked, and what can I do to solve the issue?
After bits of puzzling around for a few days I got it to work:
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Start Cosmos DB Emulator
run: docker run --detach --publish 8081:8081 --publish 1234:1234 mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview --protocol https
- name: pause
run : sleep 120
- name: Set environment variables
run: |
echo "EMULATOR_HOST=localhost" >> $GITHUB_ENV
echo "EMULATOR_PORT=8081" >> $GITHUB_ENV
echo "EMULATOR_CERT_PATH=/tmp/cosmos_emulator.cert" >> $GITHUB_ENV
- name: Fetch Emulator Certificate
run: |
openssl s_client -connect ${EMULATOR_HOST}:${EMULATOR_PORT} </dev/null | \
sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > $EMULATOR_CERT_PATH
- name: Install Certificate as a Trusted CA
run: |
sudo cp $EMULATOR_CERT_PATH /usr/local/share/ca-certificates/emulator_cert.crt
sudo update-ca-certificates
- name: Verify CA Installation
run: |
openssl s_client -connect ${EMULATOR_HOST}:${EMULATOR_PORT} -CAfile /etc/ssl/certs/ca-certificates.crt
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11' # Adjust to your required Python version
- name: Cache dependencies
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Set up Azure Functions Core Tools
run: |
wget -q https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install azure-functions-core-tools-4
- name: Log in with Azure
uses: azure/login@v1
with:
creds: '${{ secrets.AZURE_CREDENTIALS }}'
- name: Start Azurite
run: |
docker run -d -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite
- name: Wait for Azurite to start
run: sleep 5
- name: Setup test environment in Python
run : python Tests/setup.py
- name: Run tests
run: |
python -m unittest discover Tests