Calling a Google cloud function within Airflow DAG

I have a google cloud function that is working, I am trying to call it from an Airflow DAG.

what I have tried so far is to use the SimpleHttpOperator:

MY_TASK_NAME = SimpleHttpOperator(
        task_id= "MY_TASK_NAME",
        method='POST',
        http_conn_id='http_default',
        endpoint='https://us-central1-myprojectname.cloudfunctions.net/MyFunctionName',
        data=({"schema": schema, "table": table}),
        headers={"Content-Type": "application/json"},
        xcom_push=False
    )

but digging into the logs, it says it cannot find the resource:

{base_task_runner.py:98} INFO - Subtask:

The requested URL /https://us-central1-myprojectname.cloudfunctions.net/MyFunctionName was not found on this server. That’s all we know.

also I noticed that it actually posts to https://www.google.com/ + the url I gave:

Sending 'POST' to url: https://www.google.com/https://us-central1-myprojectname.cloudfunctions.net/MyFunctionName

what is the proper way to call the function ? Thanks

This is because you are using the http_conn_id='http_default'.

The http_default connection looks as follows:

If you check the Hosts field, it says http://www.google.com/.

Either create a new Connection with HTTP Connection type or modify the http_default connection and change the host to https://us-central1-myprojectname.cloudfunctions.net/

Then update the endpoint field in your task to:

MY_TASK_NAME = SimpleHttpOperator(
        task_id= "MY_TASK_NAME",
        method='POST',
        http_conn_id='http_default',
        endpoint='MyFunctionName',
        data=({"schema": schema, "table": table}),
        headers={"Content-Type": "application/json"},
        xcom_push=False
    )

Edit: Added / at the end of URLs