nodejs/express and binary data in POST

Question

I'm trying to send binary data to an express app. It works fine, as long as my values are smaller than 0x80. If a single value is 0x80 or greater, it messes up the whole buffer.

Express Handler:

binary = require('binary');

exports.api = function(req, res){
    var body = req.body.name;
    var buf = new Buffer(body,'binary');

    console.log('body',req.body);
    console.log('req body len', body.length);
    console.log('buf len', buf.length);

    var g = binary.parse(buf)
        .word16bu('a') // unsigned 16-bit big-endian value
        .word16bu('b').vars

    console.log('g.a', g.a);
    console.log('g.b', g.b);

  res.send("respond with a resource");
};

Python client (Content-Type: application/x-www-form-urlencoded):

import requests
from struct import pack
# send two unsigned shorts (16-bits each).
requests.post('http://localhost:3000/api', data={'name':pack('!HH',1,2)})

Express output when data = 1,2. This is what I'd expect.

body { name: '\u0000\u0001\u0000\u0002' }
req body len 4
buf len 4
g.a 1
g.b 2
POST /api 200 1ms - 23b

Express output when data = 1,0xFF. It's interesting to note that 9520 is actually 0x25 0x30 in hex, which corresponds to "%0" in ASCII. Yes, it appears to be parsing the '%00%01...' string. I wish I knew how to prevent this!!!

body { name: '%00%01%00%FF' }
req body len 12
buf len 12
g.a 9520
g.b 12325
POST /api 200 2ms - 23b

Answer 1

It turned out to be an encoding issue. Everything seems to default to 'utf8', but in this case it needs to be set to 'binary'.

For example:

var buf = new Buffer(body.toString('binary'),'binary');

Equally as important, I needed to set nodejs multiparty parser's encoding to 'binary'. See: https://github.com/superjoe30/node-multiparty/issues/37