pythonluascrapyweb-crawlerscrapy-splash

Splash: How to achieve infinite loop in script?

I want to crawl a site which posts real-time data via AJAX and pass that data to an alerting system.

As I understand, I can only execute requests against splash, and these have to be finished after a certain time or the request will fail. Furthermore, I don't know how to return the data without exiting the request/script.

I would be grateful for a hint.

Solution

  • You can always define a function and put the call to the function in a while loop.

    def myFun():
    //Request language.
    return YourVar

while True:
    myFun()