Im trying to deal with testing tools likewise nightmare, phantom etc. And seems to be stuck with some basic DOM manipulation. Im using jquery here for later use of $().parent() methods. Ive already tryed all posiible selector that come in collection with no use. It only returns some pieces of data. Where it actually fully exists on a page.
....
nightmare
.goto(link)
.inject('js', 'jquery-2.2.4.min.js')
.wait()
.evaluate( () => {
$('.sport--head:not(.folding--open)').click()
})
.wait(4000)
.evaluate( () => {
let events = [];
$('.view-wrapper .events--list > li').each( (i, elem) => {
let event = {
'name' : $(elem).text()
}
events.push(event);
});
let data = JSON.stringify(events, null, '\t');
return data;
})
.end()
....
It returns empty fields where they are actually not:
[{ "name": "" }, { "name": "" }, { "name": "" }, { "name": "contents" } ]
Why could this happen? Any ideas?
You do not know the pages loads completely within 4 seconds.
Either there are some Ajax data and while you are scraping it appears only at that moment. Or, the selector is actually wrong.
Instead of a specific number, use .wait to wait for a target selector,
.wait(4000)
.wait('.view-wrapper .events--list > li')
Also make sure the code actually returns the data by putting it on console.log of your actual browser.