We can tell bots to crawl or not to crawl our website in robot.txt. On the other hand, we can control the crawling speed in Google Webmasters (how much Google bot crawls the website). I wonder if it is possible to limit the crawler activities by robots.txt
I mean accepting bots to crawl pages but limit their presence by time or pages or size!
Not that I have found. Robots.txt is a place to put directories or files you would like bots to include or exclude. If there is a way it is not standard yet. Remember that whoever creates the bots chooses whether or not to respect robots.txt, not all bots respect this file.
Currently if there were settings to reduce crawl speed, time on site, etc. it would be on a bot by bot basis and not standardized into robots.txt values.
More info: http://www.robotstxt.org/robotstxt.html