POST request to PHP7 with chunked encoding does not properly return result

Question

I'm sending a POST request from a client (tested with curl and custom nodejs script) and don't get the response properly back. The whole thing works fine with PHP 5.6.

Environment

The whole thing is reduced as much as possible:

• everything running inside Vagrant VM Ubuntu 14.04 LTS
• nginx 1.9.7 from http://nginx.org/packages/ubuntu/
• PHP7 FPM compiled from official sources with --disable-all --enable-fpm

The minimal nginx site config I'm using:

server {
  listen 80;
  server_name localhost;
  location / {
    fastcgi_param REQUEST_METHOD $request_method;
    fastcgi_pass  unix:/var/run/php/php7.0-fpm-api.sock;
    fastcgi_param SCRIPT_FILENAME /vagrant/index.php;
  }
}

Example PHP script from /vagrant/index.php:

<?php
echo str_repeat('.', 512);
flush(); // not necessary, only due testing

curl call I'm using: curl -XPOST http://localhost/ -H "Transfer-Encoding: chunked" -d ''

NodeJS script I'm using:

'use strict';

var http = require('http');
var url = require('url');

var uri = url.parse(process.env.URL);
var options = {
  method: 'POST', protocol: uri.protocol, hostname: uri.hostname,
  port: uri.port, path: uri.path,
};
var data = '';

var httpRequest = http.request(options, function(res) {
    res.on('data', function(chunk) {
      console.log('received data', chunk.length);
      data += chunk;
    });
    res.on('end', function() { console.log('final size', data.length); });
  })
  .on('error', function(err) { console.log(err); });

httpRequest.write('');
httpRequest.end();

Sending my test requests to PHP 5.6

$ curl http://localhost/
..........[cut off]

$ curl -XPOST http://localhost/ -H "Transfer-Encoding: chunked" -d ''
..........[cut off]

$ URL=http://localhost/ node php7test.js
received data 512
final size 512

Sending my test requests to PHP 7.0

$ curl http://localhost/
..........[cut off]

$ URL=http://localhost/ node php7test.js
final size 0

$ curl -XPOST http://localhost/ -H "Transfer-Encoding: chunked" -d ''
curl: (18) transfer closed with outstanding read data remaining

Why am I mucking around with chunked encoding?

There's no business reason to do so, however I was using a very similar NodeJS code which defaults to chunked encoding which suddenly stopped working when switching to PHP7.

I've found the following to work from the nodejs side: explicitly setting a Content-Length header removes the implicit Transfer-Encoding: chunked header sent by NodeJS and thus works with both PHP versions.

However I'd like to understand why PHP7 behaves differently here and whether I'm in error or what's really going on here.

Update 1:

• I compared the sapi/fpm/ sources between 5.6 and 7.0 and there's almost no difference I could spot except changes due PHP internal changes
• The built-in server (php -S) is not affected, all tests

Update 2:

I bisected the PHP sources and was able to pinpoint when the behavior changed:

• last commit I was able to compile which works: https://github.com/php/php-src/commit/16265a59ac6bd433bfb636e4e44da1ad57cdcda9
• first commit I was able to compile again which didn't work: https://github.com/php/php-src/commit/86de98cabada88f4667839794c176ea37648498b

In between, output from git bisect, commits I couldn't compile:

$ git bisect skip
There are only 'skip'ped commits left to test.
The first bad commit could be any of:
ba5ecf355fe792a5a2a8e6582d5e081d02b16fbf
e383cb4493031a7cd952cfcaed3297e583149c07
fef18f4bea1980a59a9283c2197bd090aaf500cb
18cf4e0a8a574034f60f4d123407c173e57e54ec
We cannot bisect more!

Having a feeling this could be a bug, I wrote this to internals, maybe they have some insights: https://marc.info/?l=php-internals&m=145090900217798&w=2

Answer 1

From your post I'm guessing that you're using PHP7.0.0. If my guess is (bool) TRUE then I'll suggest you migrate to PHP7.0.1.

PHP7.0.0 has approximately 27 bugs that were squashed in PHP7.0.1; among other fixed items.
Source: PHP.net changelog.

I also looked at your PHP code above (with my google glasses) but it's ridiculously simple. I doubt that anything can be wrong with it. Although I'm guessing it has to do with the way PHP7 handles flush() and outputs.

Furthermore:

Noting your updates (specifically Update 1, and Update 2); I must really commend you there! THAT'S VERY IMPRESSIVE. Be sure to check out this fixed bug #61751. If I was right about your PHP version, then this fixed bug might have solved your issue; you'll just need to migrate to PHP7.0.1.

---

NOTE: I know that I should check out the internals of the fixed bug but ...