oauth-2.0prometheusexchange-serverprometheus-alertmanager

Prometheus Alert Manager Integration with OAuth2 Credentials Not Working


Snippet of my alertmanager.yml file:

global:
  smtp_smarthost: "smtp.office365.com:587"
  smtp_from: "noreply.myteam@myorg.com"
  http_config:
   oauth2:
     client_id: "76214c65-6283-4368-abbe-987********"
     client_secret: "Vp_8Q~********~97~*********~b~D"
     token_url: "https://login.microsoftonline.com/5d471751-917b-********/oauth2/v2.0/token"
     scopes: ["https://outlook.office365.com/.default"]
     tls_config:
       insecure_skip_verify: true
     endpoint_params:
        grant_type: "client_credentials"

But as I load the alert manager and wait for the emails being sent I get the below error:

time=2025-05-08T15:16:47.536Z level=ERROR source=dispatch.go:360 msg="Notify for alerts failed" component=dispatcher num_alerts=1 err="email-alert/email[0]: notify retry canceled after 7 attempts: send MAIL command: 530 5.7.57 Client not authenticated to send mail. [*****.INDP287.PROD.OUTLOOK.COM 2025-05-08T15:16:42.409Z.****]"

What is wrong with my configuration in prometheus? I have duly followed the links here

UPDATE

I tried the below steps but it didn't work since the SMTP server confuses it to be a BASIC Authentication.

err="email-alert/email[0]: notify retry canceled after 4 attempts: email.loginAuth auth: 535 5.7.139 Authentication unsuccessful, basic authentication is disabled. [MA0***0074.INDPRD01.PROD.OUTLOOK.COM 2025-05-09T07:00:20.974Z ********]"

It seems that despite providing the access_token the SMTP server is confusing it as a basic authentication. (?)

global:
  smtp_smarthost: 'smtp.office365.com:587'
  smtp_from: 'your-email@domain.com'
  smtp_auth_username: 'your-email@domain.com'
  smtp_auth_password: 'YOUR_ACCESS_TOKEN'
  smtp_require_tls: true 

Solution

  • Prometheus that does not support OAuth2 based authentication for SMTP email configuration.
    I got an answer here in this link https://groups.google.com/g/prometheus-users/c/dB0kAnHNJ8I. Thanks a ton to @ Bryan Boreham's profile photo Bryan Boreham!