How does varnish deal with dynamic content?

Question

I am studying up on caching and I am looking into varnish for caching. I am wondering though how does varnish deal with dynamically generated content?

All over the place people are saying you shouldn't really cache content that might change a lot but on the other hand when I look at the response headers for stackoverflow I see pages being served up via varnish.

Content here changes by the second so how does this even work? Excuse me if it's a bit of a simple question, I will research some more while this question is up.

Answer 1

You need to define dynamic :

• if the content depends on the user (through Cookies for example), it should not be cached as you'll have lots of different contents and your HIT/MISS ration will not be high since every user has a different content.
• if the content changes in time, you can always cache the content a little, for example a few seconds.
• if the content changes in time, a better option is to separate the "static content" from the dynamic one. You may cache the page template and do ajax calls to refresh the content. You may also use esi, it's an old technology but it lets you specify differents "zones" in your pages, each having its cache duration.
• you can benefit from IMS requests. Telling the backend to send the response body only if it has changed since the last request can save you lots of processing time. I think varnish does this from version 4

As for the stackoverflow architecture, you may learn a lot reading Nick Craver's blog post about it : http://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/