varnish caching of POST requests

Question

What I'm doing is mildly insane, but since GET requests have very strict size limit, solr uses POST requests to /solr/select URL to do what is "semantically" a GET.

I'm trying to put varnish in front of solr to do some caching. I put this in vcl_recv function:

 if (!(req.request == "GET" || req.request == "HEAD" ||
     (req.request == "POST" && req.url == "/solr/select"))) {
     /* We only deal with GET and HEAD by default */
     /* Modified to support POST to /solr/select */
     return (pass);
 }

and varnish now tries to handle that except it automatically converts a POST to a GET.

I'm aware all of that is fairly ridiculous and far from any best practices, but in any case, is there an easy way to use varnish this way?

Answer 1

I got it working after reading this tutorial from.

What the tutorial doesn't say is that there is a bug in one of the required VMODS when using with Varnish 4.1, this bug has the effect that the first POST request is passed to the backend with a truncated body.

I solved this by using Varnish 5 and works like a charm.

If you want to give it a try I have a Dockerfile for this:

Dockerfile:

FROM alpine:3.7

LABEL maintainer lloiacono@*******.com

RUN apk update \
    && apk add --no-cache varnish \
    && apk add git \
    && git clone https://github.com/varnish/varnish-modules.git \
    && apk add automake && apk add varnish-dev \
    && apk add autoconf && apk add libtool \
    && apk add py-docutils && apk add make \
    && cd varnish-modules/ \
    && ./bootstrap && ./configure && make && make install

COPY start.sh /usr/local/bin/docker-app-start

RUN chmod +x /usr/local/bin/docker-app-start

CMD ["docker-app-start"]

start.sh

#!/bin/sh
set -xe

varnishd -a :80 -f /etc/varnish/default.vcl -s malloc,256m
varnishlog

Answer 2

You could try changing the req.POST into a GET, and transform the POST data to GET parameters (you probably would have to use inline-C) and do a lookup / fetch.

This GET request limit from the HTTP spec is not necessarily implemented by either Varnish or your back-end server. As you don't depend on intermediate caches and User-Agents outside your control to handle long urls, you could give it a try.