Break HTTP file uploading from server side by PHP or Apache

Question

When uploading big file (>100M) to server, PHP always accept entire data POST from browser first. We cannot inject into the process of uploading.

For example, check the value of "token" before entire data send to server is IMPOSSIBLE in my PHP code:

<form enctype="multipart/form-data" action="upload.php?token=XXXXXX" method="POST">
    <input type="hidden" name="MAX_FILE_SIZE" value="3000000" />
    Send this file: <input name="userfile" type="file" />
    <input type="submit" value="Send File" />
</form>

So I've try to use mod_rewrite like this:

RewriteEngine On
RewriteMap mymap prg:/tmp/map.php
RewriteCond %{QUERY_STRING} ^token=(.*)$ [NC]
RewriteRule ^/upload/fake.php$ ${mymap:%1} [L]

map.php

#!/usr/bin/php
<?php
define("REAL_TARGET", "/upload/real.php\n");
define("FORBIDDEN", "/upload/forbidden.html\n");

$handle = fopen ("php://stdin","r");
while($token = trim(fgets($handle))) {
file_put_contents("/tmp/map.log", $token."\n", FILE_APPEND);
    if (check_token($token)) {
        echo REAL_TARGET;
    } else {
        echo FORBIDDEN;
    }
}

function check_token ($token) {//do your own security check
    return substr($token,0,4) === 'alix';
}

But ... It fails again. mod_rewrite looks working too late in this situation. Data still transfer entirely.

Then I tried Node.js, like this (code snip):

var stream = new multipart.Stream(req);
stream.addListener('part', function(part) {
    sys.print(req.uri.params.token+"\n");
    if (req.uri.params.token != "xxxx") {//check token
      res.sendHeader(200, {'Content-Type': 'text/plain'});
      res.sendBody('Incorrect token!');
      res.finish();
      sys.puts("\n=> Block");
      return false;
    }

Result is ... fail again.

So please help me to find the correct path to resolve this issue or tell me there is no way.

Related questions:

Can PHP (with Apache or Nginx) check HTTP header before POST request finished?

Can some tell me how to make this script check for the password before it starts the upload process instead of after the file is uploaded?

Answer 1

First of all, you can try this code yourself using the GitHub repo I created for this. Just clone the repository and run node header.

(Spoiler, if you're reading this and are under time pressure to get something to work and not in the mood to learn ( :( ), there is a simpler solution at the end)

The general idea

This is a great question. What you are asking for is very possible and no clientside is needed, just a deeper understanding of how the HTTP protocol works while showing how node.js rocks :)

This can be made easy if we go one level deeper to the underlying TCP protocol and process the HTTP requests ourselves for this specific case. Node.js lets you do this easily using the built in net module.

The HTTP Protocol

First, let's look at how HTTP requests work.

An HTTP request consists of a headers section in the general format of key:value pairs seperated by CRLF (\r\n). We know that the header section ended when we reach a double CRLF (that is \r\n\r\n).

A typical HTTP GET request might look something like this:

GET /resource HTTP/1.1  
Cache-Control: no-cache  
User-Agent: Mozilla/5.0 

Hello=World&stuff=other

The top part before the 'empty line' is the headers section and the bottom part is the body of the request. Your request will look a bit differently in the body section since it is encoded with multipart/form-data but the header will remain similarLet's explore how this applies to us.

TCP in nodejs

We can listen to the raw request in TCP and read the packets we get until we read that double crlf we talked about. Then we will check the short header section which we already have for whatever validation we need. After we do that, we can either end the request if validation did not pass (For example by simply ending the TCP connection), or pass it through. This allows us to not receive or read the request body, but just the headers which are much smaller.

One easy way to embed this into an already existing application is to proxy requests from it to the actual HTTP server for the specific use case.

Implementation details

This solution is as bare bones as it gets. It is just a suggestion.

Here is the work flow:

1. We require the net module in node.js which allows us to create tcp servers in node.js

2. Create a TCP server using the net module which will listen to data: var tcpServer = net.createServer(function (socket) {... . Don't forget to tell it to listen to the correct port

• Inside that callback, listen to data events socket.on("data",function(data){ , which will trigger whenever a packet arrives.
• read the data of the passed buffer from the 'data' event, and store that in a variable
• check for double CRLF, this ensures that the request HEADER section has ended according to the HTTP protocol
• Assuming that the validation is a header (token in your words) check it after parsing just the headers , (that is, we got the double CRLF). This also works when checking for the content-length header.
• If you notice that the headers don't check out, call socket.end() which will close the connection.

Here are some things we'll use

A method for reading the headers:

function readHeaders(headers) {
    var parsedHeaders = {};
    var previous = "";    
    headers.forEach(function (val) {
        // check if the next line is actually continuing a header from previous line
        if (isContinuation(val)) {
            if (previous !== "") {
                parsedHeaders[previous] += decodeURIComponent(val.trimLeft());
                return;
            } else {
                throw new Exception("continuation, but no previous header");
            }
        }

        // parse a header that looks like : "name: SP value".
        var index = val.indexOf(":");

        if (index === -1) {
            throw new Exception("bad header structure: ");
        }

        var head = val.substr(0, index).toLowerCase();
        var value = val.substr(index + 1).trimLeft();

        previous = head;
        if (value !== "") {
            parsedHeaders[head] = decodeURIComponent(value);
        } else {
            parsedHeaders[head] = null;
        }
    });
    return parsedHeaders;
};

A method for checking double CRLF in a buffer you get on a data event, and return its location if it exists in an object:

function checkForCRLF(data) {
    if (!Buffer.isBuffer(data)) {
        data = new Buffer(data,"utf-8");
    }
    for (var i = 0; i < data.length - 1; i++) {
        if (data[i] === 13) { //\r
            if (data[i + 1] === 10) { //\n
                if (i + 3 < data.length && data[i + 2] === 13 && data[i + 3] === 10) {
                    return { loc: i, after: i + 4 };
                }
            }
        } else if (data[i] === 10) { //\n

            if (data[i + 1] === 10) { //\n
                return { loc: i, after: i + 2 };
            }
        }
    }    
    return { loc: -1, after: -1337 };
};

And this small utility method:

function isContinuation(str) {
    return str.charAt(0) === " " || str.charAt(0) === "\t";
}

Implementation

var net = require("net"); // To use the node net module for TCP server. Node has equivalent modules for secure communication if you'd like to use HTTPS

//Create the server
var server = net.createServer(function(socket){ // Create a TCP server
    var req = []; //buffers so far, to save the data in case the headers don't arrive in a single packet
    socket.on("data",function(data){
        req.push(data); // add the new buffer
        var check = checkForCRLF(data);
        if(check.loc !== -1){ // This means we got to the end of the headers!
            var dataUpToHeaders= req.map(function(x){
                return x.toString();//get buffer strings
            }).join("");
            //get data up to /r/n
            dataUpToHeaders = dataUpToHeaders.substring(0,check.after);
            //split by line
            var headerList = dataUpToHeaders.trim().split("\r\n");
            headerList.shift() ;// remove the request line itself, eg GET / HTTP1.1
            console.log("Got headers!");
            //Read the headers
            var headerObject = readHeaders(headerList);
            //Get the header with your token
            console.log(headerObject["your-header-name"]);

            // Now perform all checks you need for it
            /*
            if(!yourHeaderValueValid){
                socket.end();
            }else{
                         //continue reading request body, and pass control to whatever logic you want!
            }
            */


        }
    });
}).listen(8080); // listen to port 8080 for the sake of the example

If you have any questions feel free to ask :)

Ok, I lied, there is a simpler way!

But what's the fun in that? If you skipped here initially, you wouldn't learn how HTTP works :)

Node.js has a built in http module. Since requests are chunked by nature in node.js, especially long requests, you can implement the same thing without the more advanced understanding of the protocol.

This time, let's use the http module to create an http server

server = http.createServer( function(req, res) { //create an HTTP server
    // The parameters are request/response objects
    // check if method is post, and the headers contain your value.
    // The connection was established but the body wasn't sent yet,
    // More information on how this works is in the above solution
    var specialRequest = (req.method == "POST") && req.headers["YourHeader"] === "YourTokenValue";
    if(specialRequest ){ // detect requests for special treatment
      // same as TCP direct solution add chunks
      req.on('data',function(chunkOfBody){
              //handle a chunk of the message body
      });
    }else{
        res.end(); // abort the underlying TCP connection, since the request and response use the same TCP connection this will work
        //req.destroy() // destroy the request in a non-clean matter, probably not what you want.
    }
}).listen(8080);

This is based on the fact the request handle in a nodejs http module actually hooks on after the headers were sent (but nothing else was performed) by default. (this in the server module , this in the parser module)

User igorw suggested a somewhat cleaner solution using the 100 Continue header assuming browsers you're targeting supports it. 100 Continue is a status code designed to do exactly what you're attempting to:

The purpose of the 100 (Continue) status (see section 10.1.1) is to allow a client that is sending a request message with a request body to determine if the origin server is willing to accept the request (based on the request headers) before the client sends the request body. In some cases, it might either be inappropriate or highly inefficient for the client to send the body if the server will reject the message without looking at the body.

Here it is :

var http = require('http');
 
function handle(req, rep) {
    req.pipe(process.stdout); // pipe the request to the output stream for further handling
    req.on('end', function () {
        rep.end();
        console.log('');
    });
}
 
var server = new http.Server();
 
server.on('checkContinue', function (req, rep) {
    if (!req.headers['x-foo']) {
        console.log('did not have foo');
        rep.writeHead(400);
        rep.end();
        return;
    }
 
    rep.writeContinue();
    handle(req, rep);
});
 
server.listen(8080);

You can see sample input/output here. This would require your request to fire with the appropriate Expect: header.