pythonformspython-requestsinvisible-recaptchagrecaptcha

How to use Python Requests to submit a form with invisible reCAPTCHA?


I want to send anonymous emails using Python. I use an online anonymous email sending tool called emkei.cz. I want to use this same tool programmatically.

How to fill out that form on the website (emkei.cz) and submit it to send an anonymous email using python-requests?

I do not want to use anything like selenium or mechanize because they are slow (even when I run selenium headlessly) and are not required for a basic HTML form which I can simulate through requests.

What I have tried

I filled out the form and checked the requests made when submitting the form in the network tab of Microsoft Edge developer tools. I tried to simulate those requests using requests library in Python.

The mail was sent successfully. I noted down the headers and payload (data) as well. I wrote a simple Python script with the same form values, trying to send a mail. However, it didn't work.

To debug, I checked r.text. It was returning the same input form I just filled instead of the success message saying 'Email sent successfully'.

Here is the code I am using:

import requests


def send_email(to, subject, body, debug):
    headers = {
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "Accept-Encoding": "gzip, deflate, br",
        "Accept-Language": "en-US,en;q=0.9",
        "Cache-Control": "max-age=0",
        "Connection": "keep-alive",
        "Content-Length": "3072",
        "Content-Type": "multipart/form-data; boundary=----WebKitFormBoundaryIGzbcUtI3oNRwVLD",
        "Cookie": "__gads=ID=a33e3b44296022c7-22066d337bd100ce:T=1648910614:RT=1648910614:S=ALNI_MZLGzNvZhCPKcpiV2aS8Nkg4um4SQ",
        "Host": "emkei.cz",
        "Origin": "null",
        "sec-ch-ua": '" Not A;Brand";v="99", "Chromium";v="99", "Microsoft Edge";v="99"',
        "sec-ch-ua-mobile": "?0",
        "sec-ch-ua-platform": '"Windows"',
        "Sec-Fetch-Dest": "document",
        "Sec-Fetch-Mode": "navigate",
        "Sec-Fetch-Site": "same-origin",
        "Sec-Fetch-User": "?1",
        "Upgrade-Insecure-Requests": "1",
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.74 Safari/537.36 Edg/99.0.1150.55"
    }

    payload = {
        "fromname": "LifeAsAnRPG Team",
        "from": "team@lifeasanrpg.com",
        "rcpt": to,
        "subject": subject,
        "attachment": "(binary)",
        "reply": "",
        "errors": "",
        "cc": "",
        "bcc": "",
        "importance": "normal",
        "xmailer": "0",
        "customxm": "",
        "confirmd": "",
        "confirmr": "",
        "addh": "",
        "smtp": "",
        "smtpp": "",
        "current": "on",
        "charset": "utf-8",
        "mycharset": "",
        "encrypt": "no",
        "ctype": "plain",
        "rte": "0",
        "text": body,
        "g-recaptcha-response": "03AGdBq24yOL4Cas-N8rzxpVSHZnJR0Ec7V_8tylGd_6IpLZotF1hqQZo2Ukyt9qw3CWAqDV7onb2TeJ25cTx9fWPf9icUaK8QCE3HGoxFMO9wYvXB5RNDSkQGbpuU_7mRZl_RDs3RVx6Savi0-PENoz1fvfUBmcKhPbPDXnRWyfayDjS1DrTU0hTivr2Xkp4W3KxBpPBg0lp7W_hgujMxqa5fjXz46Do9ZUq3G2DCRciuwBLYXS3v9nSEW1wqhFtdWfRbby50iougT0DGAWzN5vbs6o0X7YzTit6uyNO2zF0-ZECTH6YNpTMgdlC4t4QquS0-BhXPBOdDCICccYafyGoQgioaPcQt--NfaPFSYvLnVhCjFJ2y2Kl7sFFviGn-lgnvK65NpSKlNjYrSHB29LsLcF1zghmwjPZtWJ7q7rljAhz7rH9Iyxs",
        "ok": "Send"
    }

    request = requests.request(
        method="POST",
        url="https://emkei.cz/",
        headers=headers,
        data=payload
    )

    if debug:
        print(request.text)
        print(request.status_code)

    if request.status_code != 200:
        return -1

    return 0


send_email(
    to="test-test@mailinator.com",
    subject="Test subject.",
    body="""
        Test line one.
        Test line two.
        """,
    debug=True
)

My guess is that it has something to do with the captcha and g-recaptcha-response payload. I was not asked for any captcha when filling the form though.

Please try visiting the website (Emkei's Anonymous Mailer, linked above as well) and tell me how to send an email programmatically through it.


Solution

  • There is invisible reCAPTCHA, so you need to render the page to get g-recaptcha-response.
    https://developers.google.com/recaptcha/docs/versions#recaptcha_v2_invisible_recaptcha_badge

    You can use requests-html, which will automatically download Chromium on the first render.
    https://pypi.org/project/requests-html/

    1. Render the page and set the captcha in send_email function, before the POST request:

      from requests_html import HTMLSession
      session = HTMLSession()
      response = session.get("https://emkei.cz/")
      # response.html.render()
      for _ in range(10):
          if response.html.search('name="g-recaptcha-response" value="{}"') is None:
              response.html.render()
      payload['g-recaptcha-response'] = response.html.search('name="g-recaptcha-response" value="{}"')[0]
      
    2. Comment out the "Content-Type": "multipart/form-data; boundary=... header:

      # "Content-Type": "multipart/form-data; boundary=...
      

      You shouldn't specify your own boundary, since the multipart data is built by requests. https://github.com/psf/requests/issues/1997

    3. Add the following failure check(s) alongside request.status_code != 200:

      # if "The invisible reCAPTCHA test wasn't successful. Please, try again." in request.text:
      #     return -1
      if "E-mail sent successfully" not in request.text:
          return -1
      

    On Windows, you may encounter issues with requests-html trying to download Chromium.
    https://github.com/psf/requests-html/issues/325

    Invisible reCAPTCHA may still block requests

    Initially, a single response.html.render() worked perfectly.

    After running xx times within 1 hour, I needed for _ in range(10) and I occasionally get TimeoutError during render():

    pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 8000 ms exceeded.

    After running xxx times within 1 hour, https://emkei.cz/ mostly returns "The invisible reCAPTCHA test wasn't successful. Please, try again.".