My application has an existing module I use for sending emails that accesses the SMTP server and authorizes using a user (email address) and password. Now I am trying to use Gmail to do the same using my Gmail account, which, for the sake of argument, we say is booboo@gmail.com (it's actually something different).

First, I created a Gmail application. On the consent screen, which was a bit confusing, I started to add scopes that were either "sensitive" or "restricted". If I wanted to make the application "production" I was told that it had to go through a verification process and I had to produce certain documentation. This was not for me as I, the owner of this account, am only trying to connect to it for the sake of sending emails programmatically. I them created credentials for a desktop application and downloaded it to file credentials.json.

Next I acquired an access token with the following code:

from google_auth_oauthlib.flow import InstalledAppFlow

SCOPES = ['https://mail.google.com/']

def get_initial_credentials(*, token_path, credentials_path):
        flow = InstalledAppFlow.from_client_secrets_file(credentials_path, SCOPES)
        creds = flow.run_local_server(port=0)

        with open(token_path, 'w') as f:
            f.write(creds.to_json())

if __name__ == '__main__':
    get_initial_credentials(token_path='token.json', credentials_path='credentials.json')

A browser window opens up saying that this is not a verified application and I am given a chance to go "back to safety" but I click on the Advanced link and eventually get my token.

I then try to send an email with the following code:

import smtplib
from email.mime.text import MIMEText

import base64
import json

from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow

SCOPES = ['https://www.googleapis.com/auth/gmail.send']

def get_credentials(token_path):

    with open(token_path) as f:
        creds = Credentials.from_authorized_user_info(json.load(f), SCOPES)

    if not creds.valid:
        creds.refresh(Request())
        with open(token_path, 'w') as f:
            f.write(creds.to_json())

    return creds


def generate_OAuth2_string(access_token):
    auth_string = f'user=booboo\1auth=Bearer {access_token}\1\1'
    return base64.b64encode(auth_string.encode('utf-8')).decode('ascii')

message = MIMEText('I need lots of help!', "plain")
message["From"] = 'booboo@gmail.com'
message["To"] = 'booboo@gmail.com'
message["Subject"] = 'Help needed with Gmail'

creds = get_credentials('token.json')
xoauth_string = generate_OAuth2_string(creds.token)

with smtplib.SMTP('smtp.gmail.com', 587) as conn:
    conn.starttls()
    conn.docmd('AUTH', 'XOAUTH2 ' + xoauth_string)
    conn.sendmail('booboo', ['booboo@gmail.com'], message.as_string())

This works but note that I used a different scope https://www.googleapis.com/auth/gmail.send instead of the https://mail.google.com/ I used to obtain the initial access token.

I then edited the application to add the scope https://www.googleapis.com/auth/gmail.send. That required me to put the application in testing mode. I did not understand the section to add "test users", that is I had no idea what I could have/should have entered here. I then generated new credentials and a new token as above. Then when I go to send my email, I see (debugging turned on):

...
reply: b'535-5.7.8 Username and Password not accepted. Learn more at\r\n'
reply: b'535 5.7.8  https://support.google.com/mail/?p=BadCredentials l19-20020ac84a93000000b0041b016faf7esm2950068qtq.58 - gsmtp\r\n'
reply: retcode (535); Msg: b'5.7.8 Username and Password not accepted. Learn more at\n5.7.8  https://support.google.com/mail/?p=BadCredentials l19-20020ac84a93000000b0041b016faf7esm2950068qtq.58 - gsmtp'
...

But I never sent up a password, but rather the XOAUTH2 authorization string. I don't know whether this occurred because I hadn't added test users. For what it's worth, I do not believe that this new token had expired yet and therefore it was not refreshed.

I didn't try it, but had I made the application "production", would it have worked? Again, I don't want to have to go through a whole verification process with Gmail. Unfortunately, I don't have a specific question other than I would like to define an application with the more restricted scope and use that, but it seems impossible without going through this verification. Any suggestions?

Okay first off as this is going to be a single user app. You the developer will be the only one using it, and your just sending emails programticlly lets clear a few things up to begin with.

1. You do not need to verify this app. Yes you will need to just by pass that not a verified application screen as you have done. No worries.
2. Setting the application in Production by clicking the send to production button. Will enable you to request refresh tokens that will not expire. You Want this. Again ignore the screen that says you will need to verify your app you don't need to.
3. Test users, as long as you only login with the user you created the project with you dont need test users. ignore it.
4. Its just you using the app use https://mail.google.com/ scope
5. Don't worry about adding scopes to the google cloud project its just for verification. what matters is what is in your code.

Okay all clear on that.

You now have two options.

1. XOauth2 which is what you are doing now.
2. Apps password. Just create an apps password on your google account and use that in place of your actual google password and your existing code will work. How I send emails with Python. In 2.2 minutes flat!

Xoauth2

If you want to use XOauth2 then you can use the Google api python client library to help you grab the authorization token you need

#   To install the Google client library for Python, run the following command:
#   pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib


from __future__ import print_function

import base64
import os.path
import smtplib
from email.mime.text import MIMEText

import google.auth.exceptions
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.errors import HttpError

# If modifying these scopes, delete the file token.json.
SCOPES = ['https://mail.google.com/']

# usr token storage

USER_TOKENS = 'token.json'

# application credentials

CREDENTIALS = 'C:\YouTube\dev\credentials.json'


def getToken() -> str:
    """ Gets a valid Google access token with the mail scope permissions. """

    creds = None

    # The file token.json stores the user's access and refresh tokens, and is
    # created automatically when the authorization flow completes for the first
    # time.
    if os.path.exists(USER_TOKENS):
        try:
            creds = Credentials.from_authorized_user_file(USER_TOKENS, SCOPES)
            creds.refresh(Request())
        except google.auth.exceptions.RefreshError as error:
            # if refresh token fails, reset creds to none.
            creds = None
            print(f'An error occurred: {error}')
    # If there are no (valid) credentials available, let the user log in.
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                CREDENTIALS, SCOPES)
            creds = flow.run_local_server(port=0)
        # Save the credentials for the next run
        with open(USER_TOKENS, 'w') as token:
            token.write(creds.to_json())

    try:

        return creds.token
    except HttpError as error:
        # TODO(developer) - Handle errors from authorization request.
        print(f'An error occurred: {error}')


def generate_oauth2_string(username, access_token, as_base64=False) -> str:

    # creating the authorization string needed by the auth server.
    #auth_string = 'user=%s\1auth=Bearer %s\1\1' % (username, access_token)

    auth_string = 'user=' + username + '\1auth=Bearer ' + access_token + '\1\1'
    if as_base64:
        auth_string = base64.b64encode(auth_string.encode('ascii')).decode('ascii')
    return auth_string


def send_email(host, port, subject, msg, sender, recipients):
    access_token = getToken()
    auth_string = generate_oauth2_string(sender, access_token, as_base64=True)

    msg = MIMEText(msg)
    msg['Subject'] = subject
    msg['From'] = sender
    msg['To'] = ', '.join(recipients)

    server = smtplib.SMTP(host, port)
    server.starttls()
    server.docmd('AUTH', 'XOAUTH2 ' + auth_string)
    server.sendmail(sender, recipients, msg.as_string())
    server.quit()


def main():
    host = "smtp.gmail.com"
    port = 587

    user = "test@gmail.com"

    subject = "Test email Oauth2"
    msg = "Hello world"
    sender = user
    recipients = [user]
    send_email(host, port, subject, msg, sender, recipients)


if __name__ == '__main__':
    main()