Node's spawn() silently failing when called from a forever script scheduled on boot

Question

This is kind of a doozy. This issue is most likely server related and so my first recourse was AskUbuntu over here.

I'm trying to have crontab or rc.local or init.d to start a forever script on boot. It attaches a server to a port I can ping with some information and have it run a headless browser for me.

That said, it seems that I'm unable to get a response from Node.js's spawn():

var CASPER_PATH = '/home/ubuntu/dev/casperjs/bin/casperjs'; // actual binary location, not a symlink
var SCRIPTS_PATH = '/home/custom_user/endpoints/server.js';

var fileName = req.body.source + '_' + req.body.type + '.coffee'; // looks like: mysource_my_scrape_type.coffee
var scrapeId = 'test_scrape';
var user = 'user123';
var pass = 'pass123';
if (fs.existsSync(SCRIPTS_PATH + fileName)) {
  // If file is in place, spawn casperjs
  var sP = spawn(CASPER_PATH, 
    [SCRIPTS_PATH + fileName, '--ssl-protocol=any', '--user='+user, '--scrapeId='+scrapeId, '--pass='+pass], 
    { detached: true }, 
    function (err, stdout, stderr) {});
  sP.stdout.on('data', function(data) { console.log('stdout', data.toString('utf8')); });
  sP.stderr.on('data', function(data) { console.log('stderr', data.toString('utf8')); });
  sP.stdout.on('close', function(code) { console.log('close', code); });
  res.send({ scheduled: true, key: scrapeId });
} else {
  res.send({ scheduled: false, error: 'Incorrect source, type or the script is missing.' });
}

Before I added the PHANTOMJS_EXECUTABLE env to crontab or rc.local (doesnt seem to matter no matter the user level), stdout was useful:

stdout Fatal: [Errno 2] No such file or directory; did you install phantomjs?

close false

Now that the environment var is there, there is no output at all after spawn().

Mind you, Casper starts up just fine if a user (of any privilege level) runs node/forever from bash.

How can I see why spawn() is failing?

Answer 1

This actually looks like a combo-bug between forever, spawn and casperjs (maybe phantomjs). I was able to reproduce your problem, here is the full code of my test application.

You didn't show the full code, so my guess is that you have an express application and there is a special URL to run the casperjs script.

I build a simple app like this and it behaved this way:

• Just start app with node script.js (script.js is the express app which runs the casperjs script in server.js) - it works OK, renders response and writes output from the child process event handlers to console
• Start app as root with init.d script - doesn't work, once the child is spawned, no event handlers are triggered
• Start app as root with init.d script, replace casperjs with echo - the same, doesn't work (see, here we have this problem with just forever running as root, spawn and echo)
• Start app as a regular user (not root) with init.d, replace casperjs with 'echo' - it works, event handlers are triggered, here I was almost sure the issue is solved, but ... :(
• Start app as a regular user (not root) with init.d, put back casperjs - it doesn't work again, event handlers are not triggered

The practical solution to this it to use pm2, I did this:

# install pm2
sudo npm install -g pm2
# generate init.d scripts for pm2
# this command will fail, but hint about the correct format with sudo
pm2 startup ubuntu
# do this in the folder with your application
pm2 start script.js
# remember your application
pm2 save
# also useful
# sudo service stop/start/restart pm2
# pm2 stop/start/restart script

Now pm2 will start automatically with the system and it will launch your application. Everything works, child process event handlers are triggered.