Extracting Binary Data (a jpeg image) in the Request Body Sent to a Node.js Server

Question

Problem To Solve

Hello. I have been given an iPhone app that, as part of its functionality, sends a request to the server with a binary jpeg image in the request body. I have to find a way to read that binary data, send it to a client via socket.io, and log a copy of the image to the disk on the server.

First Try:

I have tried a few approaches with no luck. First I tried this:

var http = require('http');
var fs = require('fs');

http.createServer(function (req, res) {
console.log("Request In");
    var body = '';
    req.on('data', function (data) {
        body += data;
    });
    req.on('end', function () {
        fs.writeFile("./1.jpeg", body, function(err) {
        if(err) {
            console.log(err);
           } else {
            console.log("The file was saved!");
     }
});
    });
res.writeHead(200, {'Content-Type':'text/plain'});
res.end('Image Saved!');
}).listen(3000);
console.log('Server running at http://127.0.0.1:3000/');

This didn't work. I could not get an image that would open in Windows Photo Viewer. The Windows Photo Viewer gave me the error Windows Photo Viewer can't open this picture because either Photo Viewer doesn't support this file format, or you don't have the latest updates to Photo Viewer I'm sure I have the latest version. I next wondered if the request was getting messed up between the phone and the server.

Checking Request Fidelity

I installed Fiddler 2. I setup a reverse proxy, detailed here. I clicked on the request and then went to the hex view. I save the bytes after the header information by right-clicking selecting save bytes. I named the file test.jpg and it opened up in Windows Photo Viewer just fine.

I chose which bytes to save based on the different colors on the "header bytes" and the "body bytes"

Here is an example of the image I ended up getting from saving those bytes

Second Try:

Now that I know that the image is in the request and I know I can get to using fiddler, I searched stackoverflow and google to find how to capture the request body. I tried a bunch of solutions but none gave me an image that I could open.

//Express.js
var express = require('express');
var app = express();
var fs = require('fs');

app.configure(function(){
    app.use(express.bodyParser());
    app.use(function(req, res, next) {
        req.rawBody = '';
        // req.setEncoding('base64');
        req.setEncoding(null);
        req.on('data', function(chunk) { 
            req.rawBody += chunk;
        });
        req.on('end', function() {
            next();
        });
    });
});

app.listen(3000);

app.get('*', function(req, res){
    res.writeHead(200);
    res.end('Hello World');
    console.log(req.rawBody);
});

app.put('*',  function(req, res){
    console.log("putted");
    res.writeHead(200);
    res.end('Hello World');
    fs.writeFile("./1.jpg", req.rawBody, function(err) {
        if(err) {
            console.log(err);
        } else {
            console.log("The file was saved!");
        }
    });
    console.log(req.headers);
    console.log(req.params);
    console.log(req.body);
});
console.log("running");

As I understand, the express bodyParser looks for json key-value pairs and because the request body is just a bunch of binary data, its return an empty array/object (not sure which). I wrote the custom middleware that adds a rawBody variable to the request object (req) at the suggestion of this answer on stackoverflow. A few articles said to use req.setEncoding('something') to let the Node.js know how that data is encoded. I tried null base64 hex and binary.

I have uploaded the saved request from Fiddler 2, if you would like to take a look. Fiddler 2 Request. Note: I wasn't sure what file extension to use so I didn't use any file extension.

I am currently stuck. I have tried all the solutions I could find on google and stackoverflow. Please ask for clarification if you need it. I tried to be as detailed as possible (sorry for the length). Thank you for taking the time to read this far!

Side Note/Extra

I don't know much about Node.js buffers work. I would like to be able to accept a "stream" of these images if possible. I don't know how a buffer might play into an answer but if it does please try to explain it, assuming I have little knowledge about buffers.

I also mentioned wanting to send the image to a browser over socket.io. If you would like to include this second set in your answer it would be greatly appreciated.

If you would like, please include information about how your solution would perform. (Heavy on CPU or mem or light on one, etc)

Answer 1

This should do it:

var http = require('http');
var fs = require('fs');

var app = http.createServer(function (req, res) {
  req.pipe(fs.createWriteStream(__dirname + req.url));

  res.writeHead(201);
  res.end();
});

app.listen(3000);

Verified with curl:

curl -H 'Content-Type: application/octet-stream' --data-binary @file.jpg localhost:3000/foo.jpg

If this doesn't work, I'd test it with curl to make sure it's not the client which is doing something weird.

When it comes to sending it over socket.io, I wouldn't do that, but send the URL instead. If that's not an option I guess you would have to base64 encode the data.