Rails: Scheduled task to warm up the cache?

Question

I am using the following to cache a slow loading page using memcached:

caches_action :complex_report, :expires_in => 1.day

The controller action is protected by Devise authentication.

The page currently gets cached the first time a user requests it. Subsequent request that day are then pulled from the cache.

The problem with this is that the initial request takes 20-30 seconds to load. Is it possible to populate the cache in advance by way of a scheduled task?

Any suggestions much appreciated.

Answer 1

Here is an expansion on the previous cron based solution which uses curl's ability to store cookies so that you can auth in one step and then use the cookie again as an authenticated user in the next step. So if you put these lines in a script called "prepare_cache.sh"

rm /tmp/cookiejar
curl --request POST -d "login=<username>" -d "password=<password>" -c /tmp/cookiejar http://yourwebpages.url/login
curl --request GET -b -c /tmp/cookiejar http://yourwebpages.url/page_to_cache
rm /tmp/cookiejar

replacing the login and password parameters with ones which match the variables used in your login form and obviously the urls to call. I'm removing the cookiejar before to make sure there isn't a file there already and removing it at the end to make sure there isn't a cookie floating about with access levels it shouldn't have.

Then you can call this script with the cron job:

*/15 * * * * /home/myname/prepare_cache.sh > /dev/null 2>&1

And hopefully that should work. Seemed to work for me when I tried it.

Answer 2

If it's the process of running the report and collecting results that is time-consuming, you could cache those results (in place of, or along-side action caching) using Rails.cache.write and Rails.cache.read.

Then, because you needn't worry about authentication or making requests to the server, the act of running the query and caching the results from a cron job would be considerably simpler.

Answer 3

Probably the most basic solution would be to set up a simple cron entry to load up the page you'll want to have a 'hot' cache. This can be as easy adding the following to the crontab of a user on your server using crontab -e to open an editor:

*/15 * * * * wget -q http://yourwebpages.url/ > /dev/null 2>&1

What this will do is use wget to fetch the data at the provided url every 15 minutes of every hour, day, month and year, ignore the results and not send *nix mail in case something goes wrong.