How to message child process in Firefox add-on like Chrome native messaging

Question

I am trying to emulate Chrome's native messaging feature using Firefox's add-on SDK. Specifically, I'm using the child_process module along with the emit method to communicate with a python child process.

I am able to successfully send messages to the child process, but I am having trouble getting messages sent back to the add-on. Chrome's native messaging feature uses stdin/stdout. The first 4 bytes of every message in both directions represents the size in bytes of the following message so the receiver knows how much to read. Here's what I have so far:

Add-on to Child Process

var utf8 = new TextEncoder("utf-8").encode(message);
var latin = new TextDecoder("latin1").decode(utf8);

emit(childProcess.stdin, "data", new TextDecoder("latin1").decode(new Uint32Array([utf8.length])));
emit(childProcess.stdin, "data", latin);
emit(childProcess.stdin, "end");

Child Process (Python) from Add-on

text_length_bytes = sys.stdin.read(4)
text_length = struct.unpack('i', text_length_bytes)[0]
text = sys.stdin.read(text_length).decode('utf-8')

Child Process to Add-on

sys.stdout.write(struct.pack('I', len(message)))
sys.stdout.write(message)
sys.stdout.flush()

Add-on from Child Process

This is where I'm struggling. I have it working when the length is less than 255. For instance, if the length is 55, this works:

childProcess.stdout.on('data', (data) => { // data is '7' (55 UTF-8 encoded)
    var utf8Encoded = new TextEncoder("utf-8).encode(data);
    console.log(utf8Encoded[0]); // 55
}

But, like I said, it does not work for all numbers. I'm sure I have to do something with TypedArrays, but I'm struggling to put everything together.

Answer 1

The problem here, is that Firefox is trying to read stdout as UTF-8 stream by default. Since UTF-8 doesn't use the full first byte, you get corrupted characters for example for 255. The solution is to tell Firefox to read in binary encoding, which means you'll have to manually parse the actual message content later on.

var childProcess = spawn("mybin", [ '-a' ], { encoding: null });

Your listener would then work like

var decoder = new TextDecoder("utf-8");
var readIncoming = (data) => {
    // read the first four bytes, which indicate the size of the following message
    var size = (new Uint32Array(data.subarray(0, 4).buffer))[0];
    //TODO: handle size > data.byteLength - 4
    // read the message
    var message = decoder.decode(data.subarray(4, size));
    //TODO: do stuff with message
    // Read the next message if there are more bytes.
    if(data.byteLength > 4 + size)
        readIncoming(data.subarray(4 + size));
};
childProcess.stdout.on('data', (data) => {
    // convert the data string to a byte array
    // The bytes got converted by char code, see https://dxr.mozilla.org/mozilla-central/source/addon-sdk/source/lib/sdk/system/child_process/subprocess.js#357
    var bytes = Uint8Array.from(data, (c) => c.charCodeAt(0));
    readIncoming(bytes);
});