I'm aware that a web application should render it's pages as fast as possible, with few database requests only in milliseconds. What are the guidelines about this response time (like Microsoft guidelines for UI or something like that)?
What is the absolute maximum time a webpage should respond under?
Are there any "limits" or general guidelines for this?
When should I put jobs into task queues (like Python celery for example)?
My concrete problem is that I have to parse a bunch of text files, which users submits. The average time these can be parsed are 2-3 seconds (response times are 3-4sec with database inserts) but if the file is very big, it takes 8sec to parse (10sec to respond).
Is it okay to let the user without feedback for this time? If not, what is the best way for handling these kinds of situations?
Is it even okay to put these in the request-response cycle ?
Is there any difference if I provide a REST API vs a website form ? Are these "allowed" slower to respond?
I think this is really hard to answer. Different guidelines exist.
When I was at university during interface / interaction design courses I learned that no user should be left with response times over 50 ms. If that is exceeded, something like a loading icon etc should be displayed.
Also users are educated enough to expect right loading times form websites... So the user will accept 2 seconds loading time for a ticket booking page but not accept more than 300 ms from a search engine.
The limits I hear about during this days are 0.1 sec, 1 sec and 10 sec.
0.1 feels instantly to the user on websites 1 sec is slow but no interruption 10 sec is the maximum for the user to endure before loosing attention (for example light a smoke, check facebook feed in the meantime, etc...)
There is a nice article along with a lot of useful comments which I lately read which I would like to point you to:
http://www.nngroup.com/articles/response-times-3-important-limits/
I think it answers your questions well.
Please understand that this is all purely subjective but I think this is a very subjective topic...