mysqlairflowgoogle-cloud-composer

How to connect Airflow composer with mysql database


i want to connect my composer with my mySQL database. I have create a environment in GCP composer. In Airflow ui i have add a new connection like this admin -> connection -> add new connection etc. There i add my connection id name, theConnection Type= mysql, host, schema, Login, password, port and in extras i have add the ssl credentials in JSON format like this.

{
 "ssl": {
    "ssl_ca": "/home/airflow/gcs/data/my-creds/server-ca.pem",
    "ssl_cert": "/home/airflow/gcs/data/my-creds/client-cert.pem",
    "ssl_key": "/home/airflow/gcs/data/my-creds/client-key.pem"
 }
}

I have add the ssl credentials into my composer bucket into data folder. I want to clarify that i am obliged to access my mySQL database with ssl credentials. After that i trigger my dag that includes this code:

retrieve_ids_task = SQLExecuteQueryOperator(
    task_id="retrieve_ids",
    retry_on_failure=False,
    conn_id=GCP_SQL_CONNECTION,
    sql="""                   
        SELECT ....
 )

The problem that i face is this ERROR - Exception in ...: (2003, "Can't connect to MySQL server on 'hostname:port' (110)"). I have check the paths and seems not to having a problem there. Can anyone help me with my problem or suggest other way ?


Solution

  • Problem solved, i have an error with my network configurations in Composer