How to properly set up Varnish for Symfony2 sites?

Question

I have a website (with ESI) that uses Symfony2 reverse proxy for caching. Average response is around 100ms. I tried to install Varnish on server to try it out. I followed guide from Symfony cookbook step by step, deleted everything in cache folder, but http_cache folder was still created when I tried it out. So I figured I could try to comment out $kernel = new AppCache($kernel); from app.php. That worked pretty well. http_cache wasn't created anymore and by varnishstat, Varnish seemed to be working:

12951         0.00         0.08 cache_hitpass - Cache hits for pass  1153         0.00         0.01 cache_miss - Cache misses 

That was out of around 14000 requests, so I thought everything would be alright. But after echoping I found out responses raised to ~2 seconds.

Apache runs on port 9000 and Varnish on 8080. So I echoping using echoping -n 10 -h http://servername/ X.X.X.X:8080.

I have no idea what could be wrong. Are there any additional settings needed to use Varnish with Symfony2? Or am I simply doing something wrong?

---

Per requests, here's my default.vcl with modifications I've done so far.

I found 2 issues with Varnish's default config:

• it doesn't cache requests with cookies (and everyone in my app has session assigned)
• it ignores Cache-Control: no-cache header

So I added conditions for these cases to my config and it performs fairly well now (~175 req/s up from ~160 with S2 reverse proxy - but honestly, I expected bit more). I just have no idea how to check if it's all set ok, so any inputs are welcome.

Most of pages have cache varied by cookie, with s-maxage 1200. Common ESI includes aren't varied by cookie, with s-maxage quite low (articles, article lists). User profile pages aren't cached at all (no-cache) and I'm not really sure if ESI includes on these are even being cached by Varnish. Only ESI that's varied by cookies is header with user specific information (that's on 100% of pages).

Everything in this post is Varnish 3.X specific (I'm personally using 3.0.2).

Also, after few weeks of digging into this, I have really no idea what I'm doing anymore, so if you find something odd in configs, just let me know.

Answer 1

I'm surprised this hasn't had a really full answer in 10 months. This could be a really useful page.

You pointed out yourself that:

• Varnish doesn't cache requests with cookies
• Varnish ignores Cache-Control: no-cache header

The first thing is, does everyone in your app need a session? If not, don't start the session, or at least delay starting it until it's really necessary (i.e. they log in or whatever).

If you can still cache pages when users are logged in, you need to be really careful you don't serve a user a page which was meant for someone else. But if you're going to do it, edit vcl_recv() to strip the session cookie for the pages that you want to cache.

You can easily get Varnish to process the no-cache directive in vcl_fetch() and in fact you've already done that.

Another problem which I found is that Symfony by default sets max-age to 0, which means that they won't ever get cached by the default logic in vcl_fetch

I also noticed that you had the port set in Varnish to:

backend default {     .host = "127.0.0.1";     .port = "80"; } 

You yourself said that Apache is running on port 9000, so this doesn't seem to match. You would normally set Varnish to listen on the default port (80) and set Varnish to look up the backend on port 9000 or whatever.

Answer 2

If that's your entire configuration, the vcl_recv is configured twice.

In the pages you want to cache, can you send the caching headers? This would make the most sense, since images probably already have your apache caching headers and the app logic decide the pages that can be actually cached, but you can force this in varnish also.

You could use a vcl_recv like this:

# Called after a document has been successfully retrieved from the backend. sub vcl_fetch {      # set minimum timeouts to auto-discard stored objects     # set beresp.prefetch = -30s;     set beresp.grace = 120s;      if (beresp.ttl < 48h) {       set beresp.ttl = 48h;}      if (!beresp.cacheable)          {pass;}      if (beresp.http.Set-Cookie)          {pass;}      # if (beresp.http.Cache-Control ~ "(private|no-cache|no-store)")      # {pass;}      if (req.http.Authorization && !beresp.http.Cache-Control ~ "public")          {pass;}  } 

This one caches, in varnish, only the requests that are set cacheable. Also, be aware that your configuration doesn't cache requests with cookies.