postgresqlairflowairflow-schedulerparallel-execution

Airflow parallelism failuer while changing the DB to postgres

I have installed airflow locally and I am changing the executor to run parallel tasks For that, I changed

1- the Database to Postgres 13.3

2- in the config file sql_alchemy_conn = postgresql+psycopg2://postgres:postgres@localhost/postgres

3- executor = LocalExecutor

I have checked the DB and no errors using airflow db check --> INFO - Connection successful. airflow db init --> Initialization done

Errors that I receive and I don't use SQLite at all

1- {dag_processing.py:515} WARNING - Because we cannot use more than 1 thread (parsing_processes = 2 ) when using SQLite. So we set parallelism to 1.

2- I receive this error from airflow web-interface

The scheduler does not appear to be running.
The DAGs list may not update, and new tasks will not be scheduled.

So shall i do any other change ?

Solution

  • Did you actually restart your Airflow webserver/scheduler after you changed the config?