javascriptweb-scrapingpuppeteerapify

How to increase the default timeout of a function in Apify

I'm currently using the web-scraper actor of Apify
I'm trying to scrape a page which is quite long to load. I already faced this kind of situation and using the code underneath always worked.

async function pageFunction(context) {...
await context.waitFor(...);
...}

however this time the page is really slow to load and it reach the function [await context.waitFor(...)] timeout of 20 seconds. Is there a way to increase this duration above the default one ?

Solution

  • There is the second parameter options, where you can set a timeout.

    async function pageFunction(context) {
  await context.waitFor('selector', { timeoutMillis: 60000 });
}

    I have to search this parameter in GitHub repo of web-scraper. I will create an issue for document this in the readme.