I have a Python web app (Dash) with the following requirements.txt:
dash==2.17.0
numpy==1.26.4
pandas==1.5.3
dash-mantine-components==0.14.3
python-dotenv==1.0.1
dash-iconify==0.1.2
dash-bootstrap-components==1.6.0
pyspark==3.5.1
pytz==2024.1
package @ git+https://${GITHUB_TOKEN}:x-oauth-basic@github.com/package
...
and within this package, this is the pipfile:
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
pandas = "==1.5.3"
pyspark = "==3.5.1"
... other dependencies
[dev-packages]
[requires]
python_version = "3.9"
And on Azure App Service, this is the configuration: Python stack with Python version 3.12.
But when I run my pipeline on Azure DevOps to build and deploy my web app, it is taking at least 20 minutes for each stage. Primarily with this step taking so long (at least around 10-12 mins):
Building wheel for pandas (pyproject.toml): started
Building wheel for pandas (pyproject.toml): still running...
Building wheel for pandas (pyproject.toml): still running...
Building wheel for pandas (pyproject.toml): still running...
Building wheel for pandas (pyproject.toml): still running...
Building wheel for pandas (pyproject.toml): still running...
Building wheel for pandas (pyproject.toml): still running...
Building wheel for pandas (pyproject.toml): still running...
Building wheel for pandas (pyproject.toml): still running...
Building wheel for pandas (pyproject.toml): still running...
Building wheel for pandas (pyproject.toml): still running...
Lastly, this is the code I used to install python dependencies in my build&deploy.yml file:
- script: |
python -m venv antenv
source antenv/bin/activate
python -m pip install --upgrade pip
pip install setup
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
displayName: 'Install Python dependencies'
Furthermore, the build&deploy uses python version 3.12, versionSpec: '3.12'
Is there a way to optimize this Python app deployment on Azure with Azure DevOps pipeline? Am I doing something wrong in terms of versioning and compatibility?
I can reproduce the same issue when using pandas==1.5.3
and Python 3.12.
pandas==1.5.3
does not wheels built for your version, so your system builds them for itself each time. It will take long time to complete. You can check the available download files from here.
To solve this issue, you can upgrade the pandas version to 2.2.2
.
For example:
dash==2.17.0
numpy==1.26.4
pandas==2.2.2
dash-mantine-components==0.14.3
python-dotenv==1.0.1
dash-iconify==0.1.2
dash-bootstrap-components==1.6.0
pyspark==3.5.1
pytz==2024.1
Then the package install step will be optimized.
Result:
For more detailed info, you can refer to this release note: pandas 2.2.2
Update:
Since you are required to use pandas <2.2.0, you can use the version Pandas version 2.1.4.
For example:
pandas==2.1.4
For more detailed info, you can refer to the download files in pandas 2.1.4
Result: