Strategies in reducing network delay from 500 milliseconds to 60-100 milliseconds

Question

I am building an autocomplete functionality and realized the amount of time taken between the client and server is too high (in the range of 450-700ms)

My first stop was to check if this is result of server delay.

But as you can see these Nginx logs are almost always 0.001 milliseconds (request time is the last column). It’s hardly a cause of concern.

So it became very evident that I am losing time between the server and the client. My benchmarks are Google Instant's response times. Which almost often is in the range of 30-40 milliseconds. Magnitudes lower.

Although it’s easy to say that Google's has massive infrastructural capabilities to deliver at this speed, I wanted to push myself to learn if this is possible for someone who is not that level. If not 60 milliseconds, I want to shave off 100-150 milliseconds.

Here are some of the strategies I’ve managed to learn.

1. Enable httpd slowstart and initcwnd
2. Ensure SPDY if you are on https
3. Ensure results are http compressed
4. Etc.

What are the other things I can do here?

e.g

• Does have a persistent connection help?
• Should I reduce the response size dramatically?

Edit: Here are the ping and traceroute numbers. The site is served via cloudflare from a Fremont Linode machine.

    mymachine-Mac:c name$ ping site.com
    PING site.com (160.158.244.92): 56 data bytes
    64 bytes from 160.158.244.92: icmp_seq=0 ttl=58 time=95.557 ms
    64 bytes from 160.158.244.92: icmp_seq=1 ttl=58 time=103.569 ms
    64 bytes from 160.158.244.92: icmp_seq=2 ttl=58 time=95.679 ms
    ^C  
    --- site.com ping statistics --- 
    3 packets transmitted, 3 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 95.557/98.268/103.569/3.748 ms
    mymachine-Mac:c name$ traceroute site.com
    traceroute: Warning: site.com has multiple addresses; using 160.158.244.92
    traceroute to site.com (160.158.244.92), 64 hops max, 52 byte packets
     1  192.168.1.1 (192.168.1.1)  2.393 ms  1.159 ms  1.042 ms
     2  172.16.70.1 (172.16.70.1)  22.796 ms  64.531 ms  26.093 ms
     3  abts-kk-static-ilp-241.11.181.122.airtel.in (122.181.11.241)  28.483 ms  21.450 ms  25.255 ms
     4  aes-static-005.99.22.125.airtel.in (125.22.99.5)  30.558 ms  30.448 ms  40.344 ms
     5  182.79.245.62 (182.79.245.62)  75.568 ms  101.446 ms  68.659 ms
     6  13335.sgw.equinix.com (202.79.197.132)  84.201 ms  65.092 ms  56.111 ms
     7  160.158.244.92 (160.158.244.92)  66.352 ms  69.912 ms  81.458 ms
    mymachine-Mac:c name$  site.com (160.158.244.92): 56 data bytes

Answer 1

I may well be wrong, but personally I smell a rat. Your times aren't justified by your setup; I believe that your requests ought to run much faster.

If at all possible, generate a short query using curl and intercept it with tcpdump on both the client and the server.

It could be a bandwidth/concurrency problem on the hosting. Check out its diagnostic panel, or try estimating the traffic.

You can try and save a response query into a static file, then requesting that file (taking care as not to trigger the local browser cache...), to see whether the problem might be in processing the data (either server or client side).

Does this slowness affect every request, or only the autocomplete ones? If the latter, and no matter what nginx says, it might be some inefficiency/delay in recovering or formatting the autocompletion data for output.

Also, you can try and serve a static response bypassing nginx altogether, in case this is an issue with nginx (and for that matter: have you checked out nginx' error log?).