I've been working on some automation process with Nightmare and cron. After Nightmare runs, it gets the value of the result and appends it to a file in my local computer (no network issues detected).
var Nightmare = require('nightmare');
var fs = require('fs');
var filePath = `${process.env['HOME']}/Documents/ResultCron.txt`;
Nightmare()
.goto('http://mywebsite.com/form')
.type('input[id="email"]', 'XXX@gmail.com')
.click('#submit-button')
.wait('.formSubmitted')
.evaluate(() => document.querySelector('.formSubmitted').value)
.end()
.then(result => {
console.log(`The form submitted is:\n${result}`)
fs.appendFileSync(filePath, `\n${result}: ${new Date()}`);
})
.catch(error => {
console.error(error)
fs.appendFileSync(filePath, `\nERROR: ${error}: ${new Date()}`);
})
When I run this with node /home/user/Documents/nightmareForm.js
it's working perfectly and writting in the file. However, when executed from the cron it doesn't append anything to the file.
I've added the config to the crontab as following:
crontab -e
# It will be executed everyday at 13:00
0 13 * * * node /home/user/Documents/nightmareForm.js
And if I do grep nightmareForm.js /var/log/syslog -C 10
I can see it has been executed:
Apr 24 13:00:00 PC CRON[1223]: (root) CMD (node /home/user/Documents/nightmareForm.js)
Continuing the investigation, the file has this rights (Ubuntu 18.10): -rwxr-xr-x
so it should be executable and the created file has the rights: -rw-rw-rw-
Update: After the ideas of @ponury-kostek I verified that the script can write in the file, but it never reaches the then() or catch() block
Update2: I tried to catch the logs from the crontab but it doesn't display any error: 0 13 * * * /usr/local/bin/node /home/user/Documents/nightmareForm.js >> /home/user/Documents/nightmareForm.log 2>&1
Any ideas why from crontab the script is not able to reach the then() or catch() blocks? Thank you
I had the same Problem.
what did the job for me was using xvfb-run to start the node process.
xvfb-run node myscript.js
Actually i made shellscript (go.sh) that only contained 3 lines:
#!/bin/bash
cd /home/me/nodestuff/spkfetch
xvfb-run node fetch_umsaetze.js
And added this to the crontab:
* * * * * cd /home/norbert/nodestuff/spkfetch && sh go.sh >> /home/norbert/nodestuff/spkfetch/log/log.txt 2>&1
I never tried to put the xvfb-run call directly into the crontab, but i think its possibly an issue of the script not being allowed to start an X process(window) or something like that. Maybe its worth to try to put it directly into the cron command.
Here is the page that led me to this solution:
https://www.nickhart.co.uk/2019/08/09/scraping-with-nightmarejs-getting-started/#run_on_server