Google SMTP returns required authentication but it's done as an app password

I am setting up SMTP service on Airflow 2.1.2. I get this error which is quite obvious:

smtplib.SMTPSenderRefused: (530, b'5.7.0 Authentication Required. Learn more at\n5.7.0  https://support.google.com/mail/?p=WantAuthError w5sm3433302wra.87 - gsmtp'

But authentication is done, i am using an email that has 2 steps authentication, so I created an app password and I recognized the activity in google account security tab..

My setup is:

AIRFLOW__EMAIL__EMAIL_BACKEND   airflow.utils.email.send_email_smtp
AIRFLOW__SMTP__SMTP_HOST    smtp.gmail.com
AIRFLOW__SMTP__SMTP_MAIL_FROM   my email
AIRFLOW__SMTP__SMTP_PASSWORD    my app pwd of 16 chars
AIRFLOW__SMTP__SMTP_PORT    587
AIRFLOW__SMTP__SMTP_SSL False
AIRFLOW__SMTP__SMTP_STARTTLS    True
AIRFLOW__SMTP__SMTP_USER    my email

I checked and there is no extra space or something like that neither in pwd or email.

Why is it returning required authentication when authentication is really done?

Solution

Not sure why it is not working. But do you see:

Fetching SMTP credentials from configuration variables will be deprecated in a future warning? You should if your environment variables will be used.

If you do not see it, then you likely have smtp_default connection in the Connection database.

In Airflow 2 the authentication for SMTP has been brought to the same level as other connections and rather than passing user/password via environment variable, they should be configured as login/password in the 'smtp_default' connection.

The variables should still work as fallback, but if you already have smtp_default connection created and has some login/password (even empty) configured, those will be used instead of the ENV vars.

In your case likely the solution will be to set login/password in the connection.