How can I approximate proxy connection time for a cURL request using CURLINFO_*_TIME values?

Question

Below is a small dataset from which I'm trying to answer two questions:

1. How much time did the proxy take to connect to the API server?
2. How much time did the API request take to return?

The basic code looks like this:

$c             = curl_init();           // assume all options set correctly
$time          = microtime(true);
$response      = curl_exec($c);
$curl_info     = curl_getinfo($c);      // Returns each `*_TIME` field
$response_time = microtime(true)-$time; // Returns total PHP execution time

From the above I build this:

id  response_time   NAMELOOKUP_TIME CONNECT_TIME    APPCONNECT_TIME PRETRANSFER_TIME    STARTTRANSFER_TIME  REDIRECT_TIME   TOTAL_TIME
1   0.250691        0.000191        0.025070        NULL            0.181040            0.250239            0.000000        0.250306
2   0.958577        0.000129        0.022764        NULL            0.136846            0.664099            0.000000        0.957881
3   0.578614        0.000053        0.021111        NULL            0.127998            0.440123            0.000000        0.577812

How much time was spent on proxy vs. api request for each of the above?

---

The cURL Documentation is helpful but I'm not sure how to answer my questions above with the relevant section from the docs:

TOTAL_TIME           Total time of previous transfer.
NAMELOOKUP_TIME      Time from start until name resolving completed.
CONNECT_TIME         Time from start until remote host or proxy completed.
APPCONNECT_TIME      Time from start until SSL/SSH handshake completed.
PRETRANSFER_TIME     Time from start until just before the transfer begins.
STARTTRANSFER_TIME   Time from start until just when the first byte is received.
REDIRECT_TIME        Time taken for all redirect steps before the final transfer.

The included chart is helpful to see how these times stack up:

|
|--NAMELOOKUP
|--|--CONNECT
|--|--|--APPCONNECT
|--|--|--|--PRETRANSFER
|--|--|--|--|--STARTTRANSFER
|--|--|--|--|--|--TOTAL
|--|--|--|--|--|--REDIRECT

But I'm still not sure which to attribute proxy connection time. Here's the same chart with my comments:

|
|--NAMELOOKUP                 // DNS, clearly not proxy. Also insignificant values.
|--|--CONNECT                 // Does this count toward Proxy Time?
|--|--|--APPCONNECT           // Not set (likely due to non-https transaction)
|--|--|--|--PRETRANSFER       // Does this count toward Proxy Time?
|--|--|--|--|--STARTTRANSFER  // Stop proxy time? So Proxy Time = STARTTRANSFER?
|--|--|--|--|--|--TOTAL       // Would TOTAL-STARTRANSFER = API Request Time?
|--|--|--|--|--|--REDIRECT    // Always 0 (???)

Here is a chart of how HTTP Proxies work. Where do the above CURLINFO_*_TIME items fit into this chart?

_{(source: thousandeyes.com)}

Answer 1

I don't think there is any way to accurately calculate what you're looking for.

cURL connects to the proxy, sends the request and awaits the response. Everything the proxy does time-wise (it's own DNS resolution, connecting to the host, sending (proxying) the request, waiting for the reply, reading the reply, and proxying it back is a black box to cURL.

There is no way of knowing how long any of those steps took individually solely from the HTTP/SOCKS proxy.

The only thing you can accurately know is the sum of all of those actions (CURLINFO_TOTAL_TIME - CURLINFO_STARTTRANSFER_TIME). I suppose it would even be possible that there could be a slight delay from the time the proxy finishes the request to when cURL gets the first byte back (caching perhaps?).

One other possibility I'm not quite sure of, (could depend on the proxy configuration and the response headers sent by the API), is when the proxy actually sends the data back. It might need to download the entire HTTP response, or it might start sending headers and data as it reads them. This could result in some potentially significant differences in your calculations.

Ultimately, I think the bottom line is because you have no connection details from the proxy, there is no way of knowing or accurately approximating how long the proxy connection to the API took, and how long it took to get a response. Otherwise, you are correct that TOTAL - STARTTRANSFER is your best approximation for response time if you are using a proxy.

Without knowing exactly what you are trying to do or why, maybe your best bet would be to lease some VPS or cloud instances at various geographic locations to run some PHP instances using cURL to directly connect to the API so you do have access to all of the metrics you are looking for.