Newly installed Airflow providers don't show in Web UI
I'm using MWAA and when I add new providers via requirements.txt they do not show in the "Airflow - Admin > Providers" UI, while the default providers installed on the MWAA instance show just fine.
An example:
- I add
apache-airflow-providers-databricks==2.0.2to therequirements.txt - upload it to the MWAA S3 bucket
- wait for the instance to reboot
- successfully run a test DAG to ensure the provider installed correctly.
Result: After all of that, this is the "Admin > Providers" tab:
I just want to be able to easily see all providers installed. Any advice is appreciated.
Solution
So it looks like this is actually an internal VPC issue. I didn't think this would be relevant at the time, but when I run a public (vs. private) instance I do not have the same issue.
- How airflow loads/updates DagBag from dags home folder on google cloud platform?
- First time login to Apache Airflow asks for username and password, what is the username and password?
- Airflow variable masking the value "airflow"
- Airflow: set a default value in code when Variable doesn't exist without an exception
- How to get the JobID for the airflow dag runs?
- Airflow Composer - unable to access config file stored in gcp storage, when using PythonOperator
- XCom limit of 1024 in airflow
- Trigger a task iff direct upstream failed
- airflow users create command not working with 3.0 version
- Is there a way to limit Apache Airflow catchup interval
- DeprecationWarning 'HTTP_422_UNPROCESSABLE_ENTITY' is deprecated. Use 'HTTP_422_UNPROCESSABLE_CONTENT' instead
- {{ yesterday_ds }} deprecation warning, but what should I be using instead?
- (Django) ORM in airflow - is it possible?
- Export all Airflow variables
- DAG not visible in Web-UI
- Airflow 2 / 3 - ImportError: cannot import name 'BashOperator' from 'airflow.operators'
- AWS MWAA (Managed Apache Airflow) where to put the python code used in the dags?
- How to change order and number of tasks to be executed in airflow
- Using the output of one Python task and using as the input to another Python Task on Airflow
- Not able to connect to Azure SQL database from Airflow DAG
- S3KeySensor to monitor files across multiple S3 buckets
- Airflow 3.0.0 : Rest API : Not authenticated
- The datetime in airflow Task instance keeps on updating itself
- Accessing Docker Host's Docker Socket from a Containerized Airflow Setup (Permission Denied Issue)
- How to ignore "unused expression" warning from Pylance when using bit-shift operator to compose relationships in airflow
- Apache Airflow: Executor reports task instance finished (failed) although the task says its queued
- Passing parameters to Airflow's jobs through UI
- Airflow: Trigger DAG via UI with Parameters/Config
- Airflow: how to delete a DAG?
- How to override the argument of a particular task in airflow DAG