How can clients using my web service bypass their ISP's transparent proxy cache to ensure their requests reach my server?

Question

I've written a RESTful web service which is consumed only by devices, never browsers. The devices access the internet via the owner's existing household router and communicate with the web service by sending HTTP requests through the router as often as every 30 seconds. These requests are mostly "polling" requests to see if the web service has any new information for the device.

I want to prevent any ISP transparent proxies from intercepting the request and returning a cached response. I've read that one way to do this is to append a random query string onto the end of the URL of the request to fool the proxy into thinking it's a unique request. For example:

http://webservicedomain.com/poll/?randomNumber=384389

I have the ability to do this, but is this the best way? Kinda seems like a hack.

Answer 1

You should use HTTP's Cache-Control header to achieve this.

In the response you should send:

Cache-Control: private, must-revalidate, max-age=0

• private - Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache.
• max-age=0 - Indicates that the client is willing to accept a response whose age is no greater than 0 seconds. I.e. responses are immediately stale.
• must-revalidate - When present in a response received by a cache, that cache MUST NOT use the entry after it becomes stale to respond to a subsequent request without first revalidating it with the origin server.

You shoud also send a Pragma header for legacy HTTP/1.0 intermediary servers:

Pragma: no-cache

Related reading:

• The caching chapter of the HTTP spec;
• Mark Nottingham's HTTP caching tutorial.

Answer 2

You could try using encrypted connection. I think cache proxies are not supposed to store responses from encrypted communication.

One solution might be to configure HTTPS on your server, another one might be to configure client to use one of SSL proxies to send requests to your HTTP server.

Answer 3

With the appropriate Cache-control and other headers of course, use POST and voila ! That could solve the problem you have stated.

You might want to see this -- discussion on caching

Then the extra parameter pass could also be avoided.