I've been working on a project that has a lot of tests that are very flakey. Often, a selector isn't found, so until now, our kludgey solution was to add manual delays here and there.
Previously, we had code like this:
const selector = 'my-selector';
const ele = await page.$(selector);
await ele.click();
I recently tried adding a waitForSelector before the $ call like so:
const selector = 'my-selector';
await page.waitForSelector(selector);
const ele = await page.$(selector);
await ele.click();
It seems to be more reliable now. Is it basically essential to always use page.waitForSelector() before accessing an elementpage.$()? Or is there a better way of doing this?
No, it's not always necessary to use waitForSelector. If the element is already on the page, then you can select it with non-wait selections like page.$, page.$eval, etc. In cases where you want an immediate true/false existence check for a condition, page.$ is better than a waitForSelector. But if you're not sure whether an element will be there or not, best to wait for it.
waitForSelector is basically a common-case convenience wrapper on the more general waitForFunction which either registers a requestAnimationFrame loop or MutationObserver to wait for the callback predicate to be true. The waitForSelector docs claim "If at the moment of calling the method the selector already exists, the method will return immediately" but I don't see that it runs a preliminary page.$-type query before attaching a requestAnimationFrame from the present code.
On the other hand, page.$ and page.$eval and company simply run a document.querySelector for you with minimal overhead. Nonetheless, I wouldn't worry too much about performance; if you were to replace all page.$ calls with waitForSelector calls, it'd probably not be super noticable.
Note that waitForSelector returns the first matching element, so the code can be written as
const selector = 'my-selector';
const ele = await page.waitForSelector(selector);
await ele.click();
Whatever you do, avoid sleeping. waitForTimeout is being removed from the Puppeteer API, which is a good thing.
If you get tired of typing out page.waitForSelector, you can always write a little wrapper for it:
const [page] = await browser.pages();
const $ = (...args) => page.waitForSelector(...args);
const foo = await $(".foo");
const bar = await $(".bar");
const baz = await $(".baz");
Since the original post, Puppeteer now supports locators, which auto wait.
Disclosure: I'm the author of the linked blog post.