node.jspython-3.xscrapypm2forever

How do you run scrapyrt with forever.js or pm2?


I moved a project to a live server and now I need something like pm2 or forever.js to run scrapyrt.

However, neither command seemed to work...

forever.js

I ran this command which failed (and I was in an active virtual environment):

# forever start scrapyrt -p 5003
warn:    --minUptime not set. Defaulting to: 1000ms
warn:    --spinSleepTime not set. Your script will exit if it does not stay up for at least 1000ms
info:    Forever processing file: scrapyrt
error:   Cannot start forever
error:   script /root/my-scrapy-project/scrapyrt does not exist.

pm2

# pm2 start scrapyrt -p 5003
# pm2 list
┌─────┬─────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id  │ name        │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
├─────┼─────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0   │ scrapyrt    │ default     │ N/A     │ fork    │ 0        │ 0      │ 15   │ errored   │ 0%       │ 0b       │ root     │ disabled │
└─────┴─────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘


How do you set scrapyrt to permanently be turned on and automatically restarted like with pm2 or forever?


Solution

  • I found that if the server IP address is 65.124.80.15 and I want to run scrapyrt on port 5003 off it, I can use the following commands to get pm2 running scrapyrt...

    cd <into_scrapy_projectfile_containing_scrapy.cfg_file>
    pm2 start "scrapyrt -p 5003 -i 65.124.80.15"