RSS: refresh rate?

Question

I'm writing a little application for my own use which will consume a publicly published RSS feed.

As far as I can tell, there's no subscribe/post mechanism in the protocol; I need to have my application HTTP-GET the RSS feed periodically.

If that's the case, I'd like to grab it every ten minutes or so, but I'm worried about being seen as an abuser. I'd certainly be concerned if I saw someone poking my server every ten minutes for weeks on end.

Is this a valid concern? Is there any general advice on what a "reasonable" refresh rate is? Do I even have my facts straight?

Answer 1

Since RSS is built on the HTTP protocol, in general, most sites should respect the If-Modified-Since HTTP header. This is fairly lightweight and most servers should be able to return this information quickly.

So for the client-side, you'll need to keep track of the last time you've sent the request and pass it to the server. If the server returns a 304 code, then you'll know that nothing has changed. But even more importantly, the server doesn't need to return the feed info, saving bytes of traffic. If the server returns a 200, then you'll need to process the results and save the response date.

Ultimately, the answer to this question depends on what type of information is at the other end of the RSS feed. If it is a blog, then probably once every 4-8 hours is sufficient. But if RSS feed is a feed of stock quote (not likely, just an example), then every 10 minutes is not sufficient.