I am trying to upgrade my cloud composer instance from composer-2.13.9-airflow-2.9.3 to composer-2.13.9-airflow-2.10.5. I am getting many warnings in the dependency resolver like so:
WARNING: google-cloud-aiplatform 0.7.1 does not provide the extra 'evaluation'
WARNING: google-cloud-aiplatform 0.7.0 does not provide the extra 'evaluation'
WARNING: google-cloud-aiplatform 0.6.0 does not provide the extra 'evaluation'
WARNING: google-cloud-aiplatform 0.5.1 does not provide the extra 'evaluation'
WARNING: google-cloud-aiplatform 0.5.0 does not provide the extra 'evaluation'
WARNING: google-cloud-aiplatform 0.4.0 does not provide the extra 'evaluation'
WARNING: google-cloud-aiplatform 0.3.1 does not provide the extra 'evaluation'
I need to understand if this is a common issue with the google-cloud-aiplatform and if so, how to solve it / what packages most conflict with this library.
I have solved this issue. I was not able to directly solve the evaluation dependency warning loop, however, replicating the installation on a local virtual environment allowed me to know what exact packages were causing the dependency solver to loop.
I created a local python environment via the python -m venv .venv python package.. Make sure you use the same version of python as the one in your composer instance (3.11.8 in my case), you can check the python version used in the specific composer instance in the official version documentation.
Here you will also see the list of packages used for you specific composer instance, in my case:
absl-py==2.3.1
agate==1.9.1
aiodebug==2.3.0
aiofiles==24.1.0
aiohappyeyeballs==2.6.1
aiohttp==3.12.15
aiohttp-cors==0.8.1
aiomysql==0.2.0
aiosignal==1.4.0
alembic==1.16.4
amqp==5.3.1
annotated-types==0.7.0
anyio==4.10.0
apache-airflow==2.10.5+composer
apache-airflow-providers-apache-beam==6.1.3
apache-airflow-providers-apache-kafka==1.10.2
apache-airflow-providers-celery==3.10.0
apache-airflow-providers-cncf-kubernetes==10.7.0
apache-airflow-providers-common-compat==1.7.3
apache-airflow-providers-common-io==1.6.2
apache-airflow-providers-common-sql==1.27.5
apache-airflow-providers-dbt-cloud==4.4.2
apache-airflow-providers-fab==1.5.3
apache-airflow-providers-ftp==3.13.2
apache-airflow-providers-google==17.1.0
apache-airflow-providers-hashicorp==4.3.2
apache-airflow-providers-http==5.3.3
apache-airflow-providers-imap==3.9.2
apache-airflow-providers-mysql==6.3.3
apache-airflow-providers-openlineage==2.6.1
apache-airflow-providers-postgres==6.2.3
apache-airflow-providers-redis==4.2.0
apache-airflow-providers-sendgrid==4.1.3
apache-airflow-providers-smtp==2.2.0
apache-airflow-providers-sqlite==4.1.2
apache-airflow-providers-ssh==4.1.3
apache-airflow-providers-standard==1.5.0
apache-beam==2.66.0
apispec==6.8.2
appdirs==1.4.4
argcomplete==3.6.2
asgiref==3.9.1
astunparse==1.6.3
asyncpg==0.30.0
attrs==25.3.0
babel==2.17.0
backoff==2.2.1
backports.tarfile==1.2.0
bcrypt==4.3.0
billiard==4.2.1
blinker==1.9.0
build==1.3.0
CacheControl==0.14.3
cachelib==0.13.0
cachetools==5.5.2
cattrs==25.1.1
celery==5.5.3
certifi==2025.8.3
cffi==1.17.1
chardet==5.2.0
charset-normalizer==3.4.3
click==8.2.2
click-didyoumean==0.3.1
click-plugins==1.1.1.2
click-repl==0.3.0
clickclick==20.10.2
colorama==0.4.6
colorful==0.5.7
colorlog==6.9.0
ConfigUpdater==3.2
confluent-kafka==2.11.0
connexion==2.14.2
crcmod==1.7
cron-descriptor==1.4.5
croniter==6.0.0
cryptography==45.0.6
daff==1.4.2
db-dtypes==1.4.3
dbt-adapters==1.16.3
dbt-bigquery==1.10.1
dbt-common==1.27.1
dbt-core==1.10.7
dbt-extractor==0.6.0
dbt-protos==1.0.348
dbt-semantic-interfaces==0.9.0
decorator==5.2.1
deepdiff==7.0.1
Deprecated==1.2.18
diff_cover==9.6.0
dill==0.3.1.1
distlib==0.3.9
distro==1.9.0
dnspython==2.7.0
docopt==0.6.2
docstring_parser==0.17.0
durationpy==0.10
ecdsa==0.19.1
email_validator==2.2.0
fastavro==1.12.0
fasteners==0.20
fastjsonschema==2.21.1
filelock==3.17.0
firebase-admin==6.9.0
Flask==2.2.5
Flask-AppBuilder==4.5.3
Flask-Babel==2.0.0
Flask-Bcrypt==1.0.1
Flask-Caching==2.3.1
Flask-JWT-Extended==4.7.1
Flask-Limiter==3.12
Flask-Login==0.6.3
Flask-Session==0.5.0
Flask-SQLAlchemy==2.5.1
Flask-WTF==1.2.2
flatbuffers==25.2.10
flower==2.0.1
frozenlist==1.7.0
fsspec==2025.7.0
gast==0.6.0
gcloud-aio-auth==5.4.2
gcloud-aio-bigquery==7.1.0
gcloud-aio-storage==9.5.0
gcsfs==2025.7.0
google-ads==28.0.0
google-analytics-admin==0.24.1
google-api-core==2.25.1
google-api-python-client==2.178.0
google-apitools==0.5.32
google-auth==2.40.3
google-auth-httplib2==0.2.0
google-auth-oauthlib==1.2.2
google-cloud-access-context-manager==0.2.2
google-cloud-aiplatform==1.108.0
google-cloud-alloydb==0.4.8
google-cloud-appengine-logging==1.6.2
google-cloud-asset==3.30.1
google-cloud-audit-log==0.3.2
google-cloud-automl==2.16.4
google-cloud-batch==0.17.36
google-cloud-bigquery==3.35.1
google-cloud-bigquery-datatransfer==3.19.2
google-cloud-bigquery-storage==2.32.0
google-cloud-bigtable==2.32.0
google-cloud-build==3.31.3
google-cloud-common==1.5.2
google-cloud-compute==1.34.0
google-cloud-container==2.59.0
google-cloud-core==2.4.3
google-cloud-datacatalog==3.27.1
google-cloud-datacatalog-lineage==0.3.11
google-cloud-datacatalog-lineage-producer-client==0.2.2
google-cloud-dataflow-client==0.9.0
google-cloud-dataform==0.6.2
google-cloud-dataplex==2.11.0
google-cloud-dataproc==5.21.0
google-cloud-dataproc-metastore==1.19.0
google-cloud-datastore==2.21.0
google-cloud-dlp==3.31.0
google-cloud-documentai==3.5.0
google-cloud-filestore==1.13.2
google-cloud-firestore==2.21.0
google-cloud-kms==3.5.1
google-cloud-language==2.17.2
google-cloud-logging==3.12.1
google-cloud-managedkafka==0.1.12
google-cloud-memcache==1.12.2
google-cloud-monitoring==2.27.2
google-cloud-orchestration-airflow==1.17.5
google-cloud-org-policy==1.14.0
google-cloud-os-config==1.20.2
google-cloud-os-login==2.17.2
google-cloud-pubsub==2.31.1
google-cloud-pubsublite==0.6.1
google-cloud-redis==2.18.1
google-cloud-resource-manager==1.14.2
google-cloud-run==0.10.19
google-cloud-secret-manager==2.24.0
google-cloud-spanner==3.56.0
google-cloud-speech==2.33.0
google-cloud-storage==2.19.0
google-cloud-storage-transfer==1.17.0
google-cloud-tasks==2.19.3
google-cloud-texttospeech==2.27.0
google-cloud-translate==3.21.1
google-cloud-videointelligence==2.16.2
google-cloud-vision==3.10.2
google-cloud-workflows==1.18.2
google-crc32c==1.7.1
google-genai==1.29.0
google-pasta==0.2.0
google-re2==1.1.20250805
google-resumable-media==2.7.2
googleapis-common-protos==1.70.0
greenlet==3.2.4
grpc-google-iam-v1==0.14.2
grpc-interceptor==0.15.4
grpcio==1.65.5
grpcio-gcp==0.2.2
grpcio-status==1.65.5
gunicorn==23.0.0
h11==0.16.0
h2==4.2.0
h5py==3.14.0
hdfs==2.7.3
hf-xet==1.1.7
hpack==4.1.0
httpcore==1.0.9
httplib2==0.22.0
httpx==0.28.1
huggingface-hub==0.34.4
humanize==4.12.3
hvac==2.3.0
hyperframe==6.1.0
idna==3.10
immutabledict==4.2.1
importlib_metadata==8.7.0
inflection==0.5.1
iniconfig==2.1.0
isodate==0.6.1
itsdangerous==2.2.0
jaraco.classes==3.4.0
jaraco.context==6.0.1
jaraco.functools==4.2.1
jeepney==0.9.0
Jinja2==3.1.6
jiter==0.10.0
jmespath==1.0.1
joblib==1.5.1
jsonpickle==3.4.2
jsonschema==4.25.0
jsonschema-specifications==2025.4.1
jupyter_core==5.8.1
keras==3.11.1
keyring==25.6.0
keyrings.google-artifactregistry-auth==1.1.2
kombu==5.5.4
kubernetes==32.0.1
kubernetes_asyncio==32.3.2
lazy-object-proxy==1.11.0
leather==0.4.0
libclang==18.1.1
limits==5.5.0
linkify-it-py==2.0.3
litellm==1.75.5.post1
lockfile==0.12.2
looker-sdk==25.10.0
Mako==1.3.10
Markdown==3.8.2
markdown-it-py==3.0.0
MarkupSafe==3.0.2
marshmallow==3.26.1
marshmallow-oneofschema==3.2.0
marshmallow-sqlalchemy==0.28.2
mashumaro==3.14
mdit-py-plugins==0.5.0
mdurl==0.1.2
methodtools==0.4.7
ml_dtypes==0.5.3
more-itertools==10.7.0
msgpack==1.1.1
multidict==6.6.3
mysql-connector-python==9.4.0
mysqlclient==2.2.7
namex==0.1.0
nbformat==5.10.4
networkx==3.5
numpy==1.26.4
oauth2client==4.1.3
oauthlib==3.3.1
objsize==0.7.1
openai==1.99.6
opencensus==0.11.4
opencensus-context==0.1.3
openlineage-integration-common==1.36.0
openlineage-python==1.36.0
openlineage_sql==1.36.0
opentelemetry-api==1.36.0
opentelemetry-exporter-otlp==1.36.0
opentelemetry-exporter-otlp-proto-common==1.36.0
opentelemetry-exporter-otlp-proto-grpc==1.36.0
opentelemetry-exporter-otlp-proto-http==1.36.0
opentelemetry-exporter-prometheus==1.12.0rc1
opentelemetry-proto==1.36.0
opentelemetry-sdk==1.36.0
opentelemetry-semantic-conventions==0.57b0
opt_einsum==3.4.0
optree==0.17.0
ordered-set==4.1.0
orjson==3.11.1
overrides==6.5.0
packaging==25.0
pandas==2.1.4
pandas-gbq==0.29.2
paramiko==3.5.1
parsedatetime==2.6
pathspec==0.12.1
pem==23.1.0
pendulum==3.1.0
pip==23.2.1
pipdeptree==2.23.1
platformdirs==4.3.6
pluggy==1.6.0
prison==0.2.1
prometheus_client==0.22.1
prompt_toolkit==3.0.51
propcache==0.3.2
proto-plus==1.26.1
protobuf==5.29.5
psutil==7.0.0
psycopg2-binary==2.9.10
py-spy==0.4.1
pyarrow==18.1.0
pyarrow-hotfix==0.7
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycparser==2.22
pydantic==2.11.7
pydantic_core==2.33.2
pydata-google-auth==1.9.1
pydot==1.4.2
Pygments==2.19.2
PyJWT==2.10.1
pymongo==4.14.0
PyMySQL==1.1.1
PyNaCl==1.5.0
pyOpenSSL==25.1.0
pyparsing==3.2.3
pyproject_hooks==1.2.0
pytest==8.4.1
python-daemon==3.1.2
python-dateutil==2.9.0.post0
python-dotenv==1.1.1
python-http-client==3.3.7
python-nvd3==0.16.0
python-slugify==8.0.4
pytimeparse==1.1.8
pytz==2025.2
PyYAML==6.0.2
ray==2.48.0
redis==5.2.1
referencing==0.36.2
regex==2025.7.34
requests==2.32.4
requests-oauthlib==2.0.0
requests-toolbelt==1.0.0
rfc3339-validator==0.1.4
rich==13.9.4
rich-argparse==1.7.1
rpds-py==0.27.0
rsa==4.9.1
ruamel.yaml==0.18.14
ruamel.yaml.clib==0.2.12
scikit-learn==1.7.1
scipy==1.16.1
SecretStorage==3.3.3
sendgrid==6.12.4
setproctitle==1.3.6
setuptools==80.9.0
shapely==2.1.1
six==1.17.0
smart_open==7.3.0.post1
sniffio==1.3.1
snowplow-tracker==1.1.0
sortedcontainers==2.4.0
SQLAlchemy==1.4.54
sqlalchemy-bigquery==1.15.0
SQLAlchemy-JSONField==1.0.2
sqlalchemy-spanner==1.14.0
SQLAlchemy-Utils==0.41.2
sqlfluff==2.3.3
sqllineage==1.4.8
sqlparse==0.5.3
sshtunnel==0.4.0
statsd==4.0.1
tabulate==0.9.0
tblib==3.1.0
tenacity==9.1.2
tensorboard==2.19.0
tensorboard-data-server==0.7.2
tensorflow==2.19.0
tensorflow-io-gcs-filesystem==0.37.1
termcolor==3.1.0
text-unidecode==1.3
threadpoolctl==3.6.0
tiktoken==0.11.0
tokenizers==0.21.4
tornado==6.5.2
tqdm==4.67.1
traitlets==5.14.3
types-protobuf==6.30.2.20250809
typing_extensions==4.14.1
typing-inspection==0.4.1
tzdata==2025.2
uc-micro-py==1.0.3
universal_pathlib==0.2.6
uritemplate==4.2.0
urllib3==2.5.0
vine==5.1.0
virtualenv==20.28.1
wcwidth==0.2.13
websocket-client==1.8.0
websockets==15.0.1
Werkzeug==2.2.3
wheel==0.46.1
wirerope==1.0.0
wrapt==1.17.2
WTForms==3.1.2
yarl==1.20.1
zipp==3.23.0
zstandard==0.23.0
Make sure you follow this stack overflow post to find the correct package versions that might not work locally. Comment out any small useless packages that give you an python version error. In my case I had to comment out the mysql client package.
Then, run pip install of these requirements, plus you local requirements and check the logs. In my case, it was as simple as solving:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
mypackage 1.74.0 requires google-cloud-bigquery~=3.16.0, but you have google-cloud-bigquery 3.35.1 which is incompatible.
mypackage 1.74.0 requires python-dateutil~=2.8.2, but you have python-dateutil 2.9.0.post0 which is incompatible.
mypackage 1.74.0 requires requests~=2.31.0, but you have requests 2.32.4 which is incompatible.
analytics-python 1.4.post1 requires backoff==1.10.0, but you have backoff 2.2.1 which is incompatible.
This warning was not shown in the composer dependency resolver! Once I solved these recommendations, I was able to upgrade safely my composer instance to the latest version (composer-2.14.0-airflow-2.10.5).