I read online that when using the Selenium package in R, its possible to actually view the browser being opened and the actions you are performing.
For example, here is a simple script (took a long time, only works with chromote Connecting to Selenium):
library(chromote)
library(rvest)
myfun <- function(n = 5) {
b <- ChromoteSession$new()
b$Page$navigate("https://www.reddit.com/search/?q=toronto")
Sys.sleep(8)
html <- b$Runtime$evaluate("document.documentElement.outerHTML")$result$value
b$close()
urls <- read_html(html) %>%
html_elements('a[href*="/comments/"]') %>%
html_attr('href') %>%
head(n)
ifelse(grepl("^/", urls), paste0("https://www.reddit.com", urls), urls)
}
res <- myfun(5)
#######################
library(selenider)
myfun <- function(n = 5) {
session <- selenider_session("chromote")
open_url("https://www.reddit.com/search/?q=toronto")
Sys.sleep(8)
elements <- ss('a[href*="/comments/"]')
urls <- sapply(elements, function(elem) {
elem_attr(elem, "href")
})
urls <- head(urls, n)
close_session()
ifelse(grepl("^/", urls), paste0("https://www.reddit.com", urls), urls)
}
res <- myfun(5)
While this code ran, is it possible to see the Chrome browser open and see each action being executed?
Solutions based on answer provided below:
library(selenider)
myfun <- function(n = 5) {
session <- selenider_session("chromote")
open_url("https://www.reddit.com/search/?q=toronto")
Sys.sleep(8)
elements <- ss('a[href*="/comments/"]')
urls <- sapply(elements, function(elem) {
elem_attr(elem, "href")
})
urls <- head(urls, n)
close_session()
ifelse(grepl("^/", urls), paste0("https://www.reddit.com", urls), urls)
}
res <- myfun(5)
library(chromote)
library(rvest)
myfun <- function(n = 5) {
b <- ChromoteSession$new()
b$view()
b$Page$navigate("https://www.reddit.com/search/?q=toronto")
Sys.sleep(8)
html <- b$Runtime$evaluate("document.documentElement.outerHTML")$result$value
b$close()
urls <- read_html(html) %>%
html_elements('a[href*="/comments/"]') %>%
html_attr('href') %>%
head(n)
ifelse(grepl("^/", urls), paste0("https://www.reddit.com", urls), urls)
}
res <- myfun(5)
Like it says in the vignette, you can add options = chromote_options(headless = FALSE)
in the selenider_session
call to disable headless mode which will make the browser window visible.