Node.js stream data from a terminal command to the client

Question

There were already a few questions here about node.js executing commands and outputting the data, but I still can't get this working. What I want is that using node.js I want to execute a python script that runs for a long time and produces some intermediate outputs. I want to stream these outputs to the client as soon as they are produced. I have tried the following, but what I get is that I get the whole output only once the command has finished. How can I make it pass on the data into the socket in real time? Thanks.

function run_cmd(cmd, args) {
  var spawn = require('child_process').spawn,
  child = spawn(cmd, args);
  return child;
}

io.sockets.on('connection', function (socket) {
  var foo = new run_cmd('python', ['test.py']);
  foo.stdout.setEncoding('utf-8');
  foo.stdout.on('data', function(data) {
     console.log('sending data');
     io.sockets.emit('terminal', {output: data});;
  });
);

Answer 1

all your node.js code is okay.your code sends data only once because your code gets data only once.

The point is puts or print commands are not enough to trigger foo.stdout.on try adding '$stdout.flush' at the point you want to send chunk in ruby code. You need to order explicitly to output data.

here is my test code.

js

var spawn = require('child_process').spawn;
var cmd  = spawn('ruby', ['testRuby.rb']);
var counter = 0;
cmd.stdout.on('data', function(data) {
    counter ++;
  console.log('stdout: ' + data);
});

cmd.stderr.on('data', function(data) {
  console.log('stderr: ' + data);
});

cmd.on('exit', function(code) {
  console.log('exit code: ' + code);
  console.log(counter);
});

testRuby.rb

def execute_each_sec(sleep_sec)
  yield
  sleep sleep_sec
end

5.times do
  execute_each_sec(1) do ||
    puts "CurrentTime:#{Time.now}"
    $stdout.flush
  end
end

as you can see I'm calling $stdout.flush to output data explicitly in testRuby.rb. if I remove that,node.js won't get anything until execution of testRuby.rb's finished.

edited lol my bad. I used ruby instead of python. in the case of python, use sys.stdout.flush() like svkk says

Edit: In python you can also use -u flag to force it to flush after each print.