From Page interactions | Puppeteer:
Locators is the recommended way to select an element and interact with it. Locators encapsulate the information on how to select an element and they allow Puppeteer to automatically wait for the element to be present in the DOM and to be in the right state for the action. You always instantiate a locator using the page.locator() or frame.locator() function. If the locator API doesn't offer a functionality you need, you can still use lower level APIs such as page.waitForSelector() or ElementHandle.
This table sums up my understanding on the difference between locator()
and $()
:
Page method |
Returned class | Wait for element to present in the DOM? | Need await ? |
Input methods |
---|---|---|---|---|
locator() |
Locator |
Yes (retry until success or timeout) | No | Only have click , hover , scroll , |
$() |
ElementHandle |
No | Yes | Various |
I have some questions:
locator()
waits for element to present in the DOM, then isn't that it has to have await
? Similarly, if $()
doesn't wait, then isn't that it doesn't need await
?$()
, a method that doesn't wait for elements to present in DOM, have more device input methods than the one waits for them?locator()
is the combination of waitForSelector()
and $()
?Related question: What makes an element unclickable when using locator()
, but clickable when using $()
?
Before diving into your specific questions, I suggest checking out this similar thread on locator fundamentals: Understanding Playwright locator Promises. Playwright originally inherited Puppeteer-style methods like page.$()
, then adopted locators and deprecated or discouraged the legacy page.$()
-style methods. Puppeteer later borrowed locators from Playwright. The libraries have a pretty different approach to locators, but the fundamental idea is the same.
Terminology: since page.$()
, page.waitForSelector()
and other functions return ElementHandles, I'll call this the "element handle API".
The main difference between the locator and the element handle APIs is that locators are lazy, and don't do anything until an action, such as .click()
or .fill()
, is called on them. Calling page.locator()
declares a selection strategy that can potentially have actions called on it later. There's no await
because there's no action yet, and no asynchronous CDP (Chrome Devtools Protocol) browser automation network call.
On the other hand, element handle-based calls like $()
run an immediate query and return an element handle which represents a link to a real DOM element that can be manipulated further with methods like elementHandle.click()
. Within the element handle API, waitForSelector()
is comparable to part of what a locator does. Auto-waiting by default is the first advantage of locators over the element handle API.
When you call await page.locator("p").click()
, the .click()
part is a chained action. Consider:
const elementHandle = await page.waitForSelector("p");
await elementHandle.click();
versus
const locator = page.locator("p");
await locator.click();
The difference might seem trivial in the above example, but the locator only makes one CDP call to the browser rather than two, and queries the element at the exact moment of the action, avoiding the chance of the page changing between CDP calls.
Conversely, the waitForSelector()
version makes two calls, opening up a race condition due to the separate query and click actions. If the element is invalidated or changes unexpectedly, a difficult-to-debug failure can occur.
The locator version can be declared once and reused easily, with a lot of "space" (in terms of code and time) between declaration and usage. This facilitates best-practice testing approaches like the POM and makes large tests easier to maintain.
Locators are easily composable, so you can chain further filters at will and still run one race-condition-free query per action.
Note that because Playwright is geared more towards testing than Puppeteer, the advantages of locators are much more apparent in Playwright. At the time of writing, Playwright's locator implementation is significantly more advanced than Puppeteer's, and Playwright is much more opinionated about encouraging users to use locators exclusively.
As for your specific questions:
If
locator()
waits for element to present in the DOM, then isn't that it has to haveawait
? Similarly, if$()
doesn't wait, then isn't that it doesn't needawait
?
Auto-waiting or not is irrelevant to whether you need to use await
. await
is used whenever there's a network call, a system call, or inter-process communication that needs to send a command to another process or the operating system and wait for a response. Node is single-threaded, so for it to operate efficiently, it uses asynchronous code constructs like promises to allow it to perform other tasks while the network call is in flight.
Since the browser you're automating with Puppeteer is a separate process from Node and exposes a socket for network communication, any CDP call is asynchronous, regardless of whether the call involves waiting for a condition or not. So the only calls that don't need await
are ones that don't involve CDP calls, like page.locator()
. Don't mix up the words "wait" and "await"--totally different here! See Why are almost all Puppeteer calls asynchronous? for details.
Why does
$()
, a method that doesn't wait for elements to present in DOM, have more input methods than the one waits for them?
If you mean "more input options/parameters". This is because page.$()
is a "select and act in one" method that includes a selector (the string parameter) and immediately takes a CDP action, going to the browser and querying the element, then returning an element handle back to Node.
Actions typically have timeouts and other configuration parameters that are decoupled from the locator()
call, which is strictly a selection configuration. In the locator API, timeouts and other configuration options are specified through chaining and on action calls, enhancing reusability.
If you mean what you say (and it sounds like you did, based on the comments), the Puppeteer locator API is simply less mature than the element handle API, so there are fewer methods available at the present time. But if Puppeteer follows suit with Playwright, its locator API could eventually become complete enough to let the maintainers deprecate the element handle API, as Playwright did.
Is it that under the hood,
locator()
is the combination ofwaitForSelector()
and$()
?
Hopefully answered above. locator()
just declares a selection strategy, but doesn't do anything beyond that. So, no, locator()
isn't similar to either method, although once a locator has an action called on it, it does auto-wait like waitForSelector()
. You could think of locators as akin to the "p"
string parameter that you pass to page.$()
, except augmented with chainable/composable methods.
While I agree that the element handle API is lower level than the locator API, it's not technically possible to implement the locator API out of element handle calls like waitForSelector()
yourself. The reason is the time gap between querying for an element and taking action on it--locators select and act synchronously (as far as what goes on in the browser context) in a single CDP call, avoiding stale handles. But this is a fairly narrow window for failure, so realistically you could approximate locators with element handles (I've attempted to do just this for a scraping library I authored).
By the way, there's a third API style offered by Puppeteer and Playwright worth noting. I'll call it the "evaluate API" and it consists of page.evaluate()
, page.$eval()
, page.$$eval()
, page.waitForFunction()
, and so forth. Like the element handle API, there's no auto-waiting, and query and action are rolled into one operation.
The evaluate API is even lower level than handles, although it's not a subset and can't do many of the things handles can, like issuing trusted events.
But unlike the element handle API, each query and action in the evaluate API is synchronous and atomic, and the syntax for working within the browser synchronously is generally less cumbersome than dealing with element handles. Evaluate calls give you a great deal of precise control over the page.
In Puppeteer, my web scraping and PDF generation scripts almost exclusively use the evaluate API, except for filling input fields, where element handles or locators become necessary. This may change as Puppeteer's locator API matures, but even if it achieved parity with Playwright, locators tend to be more geared towards testing. Reusability, composability, trusted events and DRYness matter less in scraping scripts than they do in testing.
A final thought on locators is that method chaining is syntactically ergonomic in async
/await
environments, avoiding a lot of the const foo = await ...
intermediate variable assignments and nasty await (await page.$()).click()
-type syntax in non-chained async environments. Puppeteer probably takes chained locators too far, moving configuration like timeouts to chained calls rather than configuration objects, which is cumbersome in other respects. But I'm getting a bit off topic here so I'll set that discussion aside for now.