curl issue: same request, different responses

Question

Here is a problem.

I need to send a request to the API, and get the string from a response. In this particular case I want to get a string "order already cancelled"

First of all, I checked it in the Postman. It works fine there. After that, I wrote a php code, using GuzzleHttp package to make a request. But there I face the problem for the first time.

The response had a header "Content-Length: 4" and seemed like the body was empty. That was a reason why i got NULL when i tried to call $response->getBody();

Then i ran GuzzleHttp in the debug mode to see the raw http request/response and compare with the Postman.

In general, there were no differences between them, except some unimportant headers like "User-Agent", "Accept-Encoding" and "Postman-Token".

I decided that the problem was with the GuzzleHttp and i rewrote the code with clean built-in curl functions.

$basic_auth_base64 = base64_encode("{$this->_http_auth_login}:{$this->_http_auth_pass}");

$curl = curl_init();

curl_setopt_array($curl, array(
  CURLOPT_URL => "{$this->_base_url}{$uri}?" . http_build_query($params),
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => "",
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => "GET",
  CURLOPT_HTTPHEADER => array(
    "Authorization: Basic $basic_auth_base64",
    "Cache-Control: no-cache"
  ),
));

$response = curl_exec($curl);
$err = curl_error($curl);        
curl_close($curl);

Then I got the same problem! A bad empty response from the API with a headers:

Content-Type: application/json; charset=UTF-8
Content-Length: 4

But I need to get a response like:

Content-Type: text/html
Content-Length: 23

I even checked this request from a bash with curl and it worked fine!

curl -X GET \
  'http://website-server.com/api/order/delete/?OrderId=6693' \
  -H 'Accept: */*' \
  -H 'Accept-Encoding: deflate, gzip' \
  -H 'Authorization: Basic base64LoginAndPass' \
  -H 'Cache-Control: no-cache' \
  -H 'User-Agent:' -v

I got a good response from the api, so the problem is only with my php code and environment.

Here is the logs from curl. From the bash curl and from the php curl:

bash curl:

*   Trying 82.236.10.100...
* TCP_NODELAY set
* Connected to website-server.com (82.236.10.100) port 80 (#0)
> GET /api/order/delete/?OrderId=6693 HTTP/1.1
> Host: website-server.com
> Accept: */*
> Accept-Encoding: deflate, gzip
> Authorization: Basic base64LoginPassword
> Cache-Control: no-cache
> 
< HTTP/1.1 200 OK
< Server: nginx-reuseport/1.13.4
< Date: Fri, 13 Apr 2018 17:04:27 GMT
< Content-Type: text/html
< Content-Length: 23
< Connection: keep-alive
< Keep-Alive: timeout=30
< X-Powered-By: PHP/5.6.30
< 
* Connection #0 to host website-server.com left intact
order already cancelled    

php curl:

*   Trying 82.236.10.100...
* TCP_NODELAY set
* Connected to website-server.com (82.236.10.100) port 80 (#0)
> GET /api/order/delete/?OrderID=6693 HTTP/1.1
Host: website-server.com
Accept: */*
Accept-Encoding: deflate, gzip
Authorization: Basic base64LoginPassword
Cache-Control: no-cache

< HTTP/1.1 200 OK
< Server: nginx-reuseport/1.13.4
< Date: Fri, 13 Apr 2018 16:44:59 GMT
< Content-Type: application/json; charset=UTF-8
< Content-Length: 4
< Connection: keep-alive
< Keep-Alive: timeout=30
< X-Powered-By: PHP/5.6.30
< 
* Curl_http_done: called premature == 0
* Connection #0 to host website-server.com left intact

As you can see, here is a totally identical requests! But different responses! The main difference is the header "Content-Type: application/json; charset=UTF-8" and the response's body.

What should i do?

Answer 1

I don't know a possible reason, but I 'll try to recommend how to debug.

You are seeing that identical requests produce different results. Now let's make a reasonable assumption: We can be quite sure that the server will return exactly the same result if the requests are really and completely identical.

So far, you are only believing that the queries from PHP and CURL are identical. I am not sure how exactly CURL and PHP CURL write their logs (i.e. I am not sure how and if they can be configured to log really all headers).

Notably, it would be interesting to see what headers CURL and PHP CURL are sending with their request. I would log the network traffic using tcpdump or Wireshark. I am convinced that the requests will turn out to not be identical.