IIS as a reverse proxy - compression of rewritten response from backend server

Question

I am implementing a reverse proxy for routing requests to a backend server.

Functionally everything works correctly, however I am concerned that all responses from the backend server are transferred to the client (web browser) without compression.

The setup is as follows:

• Backend server, not accessible for public, on an internal domain. Hosts a web application on https://internal.app
• Front web server with IIS 7.5, hosting the main public website and acting as a proxy for the backend server. The main site is at https://site.com.

I want to route all requests to https://site.com/app/WHATEVER to https://internal.app/WHATEVER in a way that is transparent to clients.

My current setup is based on URL Rewrite 2.0 and Application Request Routing IIS extensions. The general approach is based on guidelines from the following articles:

• Setting up a Reverse Proxy using IIS, URL Rewrite and ARR
• Reverse Proxy with URL Rewrite v2 and Application Request Routing

The relevant section of web.config of the site.com app:

<system.webServer>
    <rewrite>
        <rules>
            <rule name="Route the requests for backend app" stopProcessing="true">
                <match url="^app/(.*)" />
                <conditions>
                    <add input="{CACHE_URL}" pattern="^(https?)://" />
                </conditions>
                <action type="Rewrite" url="{C:1}://internal.app/{R:1}" />
                <serverVariables>
                    <set name="HTTP_ACCEPT_ENCODING" value="" />
                </serverVariables>
            </rule>
        </rules>
        <outboundRules>
            <rule name="RewriteBackendAbsoluteUrlsInResponse" preCondition="ResponseIsHtml1">
                <match filterByTags="A, Area, Base, Form, Frame, Head, IFrame, Img, Input, Link, Script" pattern="^http(s)?://internal.app(\:80)?/(.*)" />
                <action type="Rewrite" value="/app/{R:3}" />
            </rule>
            <rule name="RewriteBackendAbsoluteUrlsInRedirects" preCondition="ResponseIsHtml1">
                <match serverVariable="RESPONSE_LOCATION" pattern="^http(s)?://internal.app(\:80)?/(.*)" />
                <action type="Rewrite" value="/app/{R:3}" />
            </rule>
            <rule name="RewriteBackendRelativeUrlsInResponse" preCondition="ResponseIsHtml1">
                <match filterByTags="A, Area, Base, Form, Frame, Head, IFrame, Img, Input, Link, Script" pattern="^/(.*)" negate="false" />
                <conditions>
                    <add input="{URL}" pattern="^/app/.*" />
                </conditions>
                <action type="Rewrite" value="/app/{R:1}" />
            </rule>
            <rule name="RewriteBackendRelativeUrlsInRedirects" preCondition="ResponseIsHtml1">
                <match serverVariable="RESPONSE_LOCATION" pattern="^/(.*)" negate="false" />
                <conditions>
                    <add input="{URL}" pattern="^/app/.*" />
                </conditions>
                <action type="Rewrite" value="/app/{R:1}" />
            </rule>
            <preConditions>
                <preCondition name="ResponseIsHtml1">
                    <add input="{RESPONSE_CONTENT_TYPE}" pattern="^text/html" />
                </preCondition>
            </preConditions>
        </outboundRules>
    </rewrite>
    <urlCompression dynamicCompressionBeforeCache="false" />
</system.webServer>

The problem is that as soon as I stop clearing the HTTP_ACCEPT_ENCODING server variable, each request which matches the above rule ends with the following error: HTTP Error 500.52 - URL Rewrite Module Error. Outbound rewrite rules cannot be applied when the content of the HTTP response is encoded ("gzip").

I am aware of this thread and I have followed those instructions. I have set dynamicCompressionBeforeCache="false" as can be seen above, I have added the necessary registry entry and I have assured that the modules are in correct order in IIS.

However, this only seems to work only if the rewriting happens within one web app. If I remove the above rules and add a simple one (and respective outbound rules) to rewrite e.g. /x/WHATEVER to just /WHATEVER, all works perfectly without a need to clear HTTP_ACCEPT_ENCODING - the rule works and compression is enabled for the rewritten requests.

But as soon as I re-add my rule which rewrites the response to a different web app, and I don't clear the HTTP_ACCEPT_ENCODING header, the same error appears again.

From what I understand, if the rewriting involves another web app, there are more constraint on what can be done. E.g. URL rewriter must receive an uncompressed response from the backend server in order to be able to rewrite it using the outbound rules. I guess clearing HTTP_ACCEPT_ENCODING in this scenario is a must because of this.

However, I would expect that since the compression module is listed on top of the modules list, the final rewritten response should be compressed no matter where it originated from. It seems IIS makes some shortcuts and returns the response to the client bypassing the compression module. Or the HTTP_ACCEPT_ENCODING header is removed soon enough to completely disable compression (not only in server-to-server communication).

So finally, my question is: is there a way to compress those responses?

Answer 1

I have figured it out myself.

What needs to be done to get it working:

• Accept-Encoding header must be removed before routing the request to the backend server, so that the response can be rewritten using outbound rules
• the header must be restored by an additional accompanying outbound rule, so that it is present when the compression module kicks in before the response is sent to the client

I have decided to do it like this:

• add a new server variable to the rewrite rule to hold he original header sent by the client:

  <set name="HTTP_X_ORIGINAL_ACCEPT_ENCODING" value="{HTTP_ACCEPT_ENCODING}" />
  

  (I put it before the line which clears the HTTP_ACCEPT_ENCODING variable)

• add a new outbound rule:

  <rule name="RestoreAcceptEncoding" preCondition="NeedsRestoringAcceptEncoding">
    <match serverVariable="HTTP_ACCEPT_ENCODING" pattern="^(.*)" />
    <action type="Rewrite" value="{HTTP_X_ORIGINAL_ACCEPT_ENCODING}" />
  </rule>
  

  and an accompanying precondition:

  <preCondition name="NeedsRestoringAcceptEncoding">
    <add input="{HTTP_X_ORIGINAL_ACCEPT_ENCODING}" pattern=".+" />
  </preCondition>
  

Works like a charm so far.

Answer 2

To address the original poster's issue while still preserving gzip compressed responses, one simply needs to do the following:

1. update the registry on your public facing web server as such:

   a. For 64bit web sites, run the following in a command console with admin rights: reg add HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\InetStp\Rewrite /v LogRewrittenUrlEnabled /t REG_DWORD /d 0

   b. For 32bit web sites, run the following in a command console with admin rights: reg add HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432node\Microsoft\Inetstp\Rewrite /v LogRewrittenUrlEnabled /t REG_DWORD /d 0

2. reset IIS

3. disable static compression

   a. to accomplish this, I inserted the following my web config: <urlCompression doStaticCompression="false" doDynamicCompression="true" dynamicCompressionBeforeCache="false" />

4. in the server node in IIS manager, double click on the Modules icon, then on the right, click "view ordered List" and verify that your static/dynamic compression modules are towards the top and the URL rewrite module is toward the bottom

Please Note

I see a lot of resolutions to this issue floating around the net where the HTTP_CONTENT_TYPE request header is being cleared out as part of the URL rewrite rule (including answers on this page). One should be aware that while this does resolve the original issue of the 500.52 error, gzip compression on the response is REMOVED. This may be the desired outcome, but if gzip compression required, the above solution will do the trick

Answer 3

PS: The below solution only works if you have control over your app server.

It's basically letting the web server do the compression, and let the app server do the heavy duty of what the app is supposed to do (without compression).

If you disable the compression on the app server, the response you get from app server is uncompressed. On the web server, you should enable the compression, so web server will honor the HTTP header "Accept-Encoding: gzip,deflate" while responding to client (browser).

This configuration will offload the CPU on the app server but will increase the network traffic between your web server and app server. If you are on the internal network, it doesn't have much performance impact.

Answer 4

Just adding a "serverVariables" tag in the rewrite rule did the trick for me:

<rewrite>
        <rules>
            <rule name="ReverseProxyInboundRule1" stopProcessing="true">
                <match url="(.*)" />
                <action type="Rewrite" url="http://otherurl{R:1}" />

                <serverVariables>
                    <set name="HTTP_ACCEPT_ENCODING" value="" />
                </serverVariables>
            </rule>

       </rules>
</rewrite>