amazon-web-servicesweb-scrapingbotsamazon-wafweb-traffic

AWS Bot Control Labels user_triggered and developer_platform are unclear


The AWS-provided definitions of the "Bot Control labels"

awswaf:managed:aws:bot-control:bot:   user_triggered        :verified

and

awswaf:managed:aws:bot-control:bot:   developer_platform    :verified

are hard to comprehend for me. The definitions are pasted below and can be found via the link above.

The questions are:

  1. As per the definition, user_triggered is "similar to a verified bot" yet behaves like a human. My first thought was that this could be a developer testing a verified bot/scraper via a manually controlled browser. In this case, the developer would sit at the computer or inside of the corporate network, so outgoing requests might look like they originate from the bot that usually runs on that network. Is that the correct way of thinking about this label?

  2. The name of awswaf:managed:aws:bot-control:bot:user_triggered:verified suggests that this is a verified bot. But it is neither a bot, nor is it verified. Can somebody explain why this traffic is treated as unverified?

  3. Again, what makes awswaf:managed:aws:bot-control:bot:developer_platform:verified "similar to a verified bot," but still not equal?

  4. What is meant by "developer platforms [...] Google Apps Script"? Is this a bot that a developer would write on a Google-hosted service (such as a Google Sheets script) and that is ran directly on Google's infrastructure, thus looking like verified traffic (because it comes from Google), yet being potentially triggered by any Google Workspace or Gmail user? And the same may hold for any other platform that lets users run their scripts on fully serviced infrastructure?

Appendix: Definitions From AWS (Link)

awswaf:managed:aws:bot-control:bot: user_triggered :verified – Used to indicate a bot that is similar to a verified bot, but that might be directly invoked by end users. This category of bot is treated by the Bot Control rules like an unverified bot.

awswaf:managed:aws:bot-control:bot: developer_platform :verified – Used to indicate a bot that is similar to a verified bot, but that is used by developer platforms for scripting, for example Google Apps Script. This category of bot is treated by the Bot Control rules like an unverified bot.


Solution

  • bot:verified label is for bots that AWS trusts and sees as legitimate. As mentioned here AWS uses IP address to identify if the bot is verified or not.

    Bot Control uses the IP address from the web request origin to help determine whether a bot is verified.

    However, that's not enough to establish a legitimacy of a bot. so I went digging and found this video. According to this AWS maintains an internal database of verified bots and uses signature based methods to identify and maintain that data.

    Based on above, any bot with a label other than bot:verified is actually an unverified bot.

    So user_triggered:verified means that AWS thinks that this bot/script is run by a user locally and not by any botnet on the network, and developer_platform:verified means this bot/script was run via some developer platform like Google Apps Script as mentioned in the docs. Both of these are unverified bots since they are not part of verified bots database and don't match the verified bot signatures.