Disadvantages to rack-cache vs. Varnish in Heroku cedar stack?

Question

The previous 2 Heroku application stacks came with a Varnish layer which automatically reverse-proxy-cached content based on http headers.

The new Heroku cedar stack doesn't have this Varnish layer. Heroku suggests using rack-cache and memcache instead.

Does this have disadvantages compared to the previous stacks with the varnish layer? With rack-cache, aren't there fewer servers serving the caching layer, and in a less optimized way?

• http://devcenter.heroku.com/articles/http-routing
• http://devcenter.heroku.com/articles/http-caching
• http://devcenter.heroku.com/articles/cedar

Answer 1

There is absolutely no question that removing the varnish layer is a huge downgrade in cache performance -- both latency and throughput -- from Heroku's bamboo to cedar stack. For one, your application requests are competing with, and may be queued behind, cache hits for dyno time.

The disadvantages, to name a few, are: interpreted ruby (vs. compiled C) application level (vs. proxy level) memcached based (vs. in process memory based) blocking I/O based (vs. nonblocking I/O based). Any suggestion that the two caching strategies could compete at scale is ridiculous. You won't see much difference on a small scale. But if you have hundreds or even thousands of cache hits per second, varnish will not substantially degrade, while the cedar stack would require many dynos just to serve static assets in a performant fashion. (I'm seeing ~ 5-10ms dyno processing time for cache hits on cedar).

Cedar was built the way it was built not to improve performance, but to introduce new levels of flexibility over your application. While it allows you to do non blocking I/O operations like long polling, and fine grained control over static asset caching parameters, it's clear that heroku would like you to bring your own caching/bandwidth if your goal is internet scale.

Answer 2

I don't know how the rack-cache performance compares to Varnish in terms of raw requests. The best bet would be to create a simple app benchmark it and switch stacks.

It's worth bearing in mind that because the heroku.com stack was Nginx->Varnish->App as long as you've set the correct HTTP headers you're App layer will be doing much less work. As most of the delivery will be handled by Varnish, as well as Varnish being pretty fast, this frees up your Dyno for actual app-handling-requests.

With the herokuapp.com stack hitting your app earlier, it's down to you and your app to handling caching efficiently, this might mean you choose to use rack-cache to cache full page output or you might choose to use memcached to deal with database requests or both.

In the end it depends on what your app is doing, if it's serving the same static content to a lot of users then you'd benefit from Varnish, but if you've got an application where users login and interact with content then you won't see might benefit from Varnish so caching partial content or raw database queries might be more efficient. If you install the New Relic addon you'll be able to take a peek under the hood and see what's slowing your app down.

Answer 3

There are also 3rd party options for hosted Varnish. I wrote a quick post about setting it up with Fastly/Varnish.

Fastly is a hosted Varnish solution that will sit in front of your Heroku app and serve cached responses.

Update link: https://medium.com/@harlow/scale-rails-with-varnish-http-caching-layer-6c963ad144c6

I've seen really great response times. If you can get a good cache hit-rate with Varnish you should be able to throttle back a good percentage of your dyno's.