Generating PDF, error with IE and HTTPS

Question

I am streaming a PDF to the browser in ASP.NET 2.0. This works in all browsers over HTTP and all browsers except IE over HTTPS. As far as I know, this used to work (over the past 5 years or so) in all versions of IE, but our clients have only recently started to report issues. I suspect the Do not save encrypted pages to disk security option used to be disabled by default and at some point became enabled by default (Internet Options -> Advanced -> Security). Turning this option off helps, as a work-around, but is not viable as a long term solution.

The error message I am receiving is:

Internet Explorer cannot download OutputReport.aspx from www.sitename.com.

Internet Explorer was not able to open this Internet site. The requested site is either unavailable or cannot be found. Please try again later.

The tool used to create the PDF is ActiveReports from DataDynamics. Once the PDF is created, here is the code to send it down:

Response.ClearContent()
Response.ClearHeaders()
Response.AddHeader("cache-control", "max-age=1")
Response.ContentType = "application/pdf"
Response.AddHeader("content-disposition", "attachment; filename=statement.pdf")
Response.AddHeader("content-length", mem_stream.Length.ToString)
Response.BinaryWrite(mem_stream.ToArray())
Response.Flush()
Response.End()  

Note: If I don't explicitly specify cache-control then .NET sends no-cache on my behalf, so I have tried setting cache-control to: private or public or maxage=#, but none of those seem to work.

Here is the twist: when I run Fiddler to inspect the response headers, everything works fine. The headers that I receive are:

HTTP/1.1 200 OK
Cache-Control: max-age=1
Date: Wed, 29 Jul 2009 17:57:58 GMT
Content-Type: application/pdf
Server: Microsoft-IIS/6.0
MicrosoftOfficeWebServer: 5.0_Pub
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
content-disposition: attachment; filename=statement.pdf
Content-Encoding: gzip
Vary: Accept-Encoding
Transfer-Encoding: chunked

As soon as I turn Fiddler off and try again, it fails again. One other thing that I noticed is that when Fiddler is running I get a There is a problem with this website's security certificate warning message, and I have to click Continue to this website (not recommended) to get through. When Fiddler is off, I do not encounter this security warning and it fails right away.

I am curious what is happening between Fiddler and the browser so that it works when Fiddler is running but breaks when it's not, but more importantly, does anyone have any ideas how I could change my code so streaming PDFs to IE will work without making changes to the client machine?

Update: The Fiddler issues are resolved, thank you very much EricLaw, so now it behaves consistently (broken, with or without Fiddler running).

Based on Google searching, there seem to be plenty of reports of this same issue all over the web, each with it's own specific combination of response headers that seem to fix the problem for their individual cases. I've tried many of these suggestions, including adding an ETag, LastModified date, removing the Vary header (using Fiddler) and dozens of combinations of the Cache-Control and/or Pragma headers. I tried "Content-Transfer-Encoding: binary" as well as "application/force-download" for the ContentType. Nothing has helped so far. There are a few Microsoft KB articles, all of which indicate that Cache-Control: no-cache is the culprit. Any other ideas?

Update: By the way, for completeness, this same issue occurs with Excel and Word outputs as well.

Update: No progress has been made. I emailed the .SAZ file from Fiddler to EricLaw and he was able to reproduce the problem when debugging IE, but there are no solutions yet. Bounty is going to expire...

Answer 1

Your Cache-Control header is incorrect. It should be Cache-Control: max-age=1 with the dash in the middle. Try fixing that first to see if it makes a difference.

Typically, I would say that the most likely culprit is your Vary header, as such headers often cause problems with caching in IE: http://blogs.msdn.com/ieinternals/archive/2009/06/17/9769915.aspx. You might want to try adding a ETAG to the response headers.

Fiddler should have no impact on cacheability (unless you've written rules), and it sounds like you're saying that it does, which suggests that perhaps there's a timing problem of some sort.

>Do not save encrypted pages to disk security option used to be disabled by default

This option is still disabled by default (in IE6, 7, and 8), although IT Administrators can turn it on via Group Policy, and some major companies do so.

Incidentally, the reason you see the certificate error while running Fiddler is that you haven't elected to trust the Fiddler root certificate; see http://www.fiddler2.com/fiddler/help/httpsdecryption.asp for more on this topic.

Answer 2

After two weeks on a wild goose chase, I have not been able to find any combination of code changes that will allow this method of streaming PDF, Excel or Word documents when the 'Do not save encrypted pages to disk' option is turned on.

Microsoft has said this behavior is by design in a number of KB articles and private emails. It appears that when the 'Do not save encrypted pages to disk' option is turned on that IE is behaving correctly and doing what it is told to do. This post is the best resource I have found so far that explains why this setting would be enabled and the Pros and Cons of enabling it:

"The 'Do not save encrypted pages to disk' comes into play when dealing with SSL (HTTPS) connections. Just like a web server can send done information about how to cache a file one can basically set Internet Explorer up to not save files to the cache during an SSL (HTTPS) connection regardless if the web server advises you can.

What is the upside for turning this feature on, security is the number one reason why the feature is turned on. Pages are not stored in the Temporary Internet Files cache.

What is the downside? Slow performance, since nothing is saved to the cache even that 1 byte gif image used a dozen times on the page must be fetched from the webserver each time. To make matters worse some user actions may fail such as downloaded files will be deleted and an error presented or opening PDF documents will fail to name a few scenarios."

The best solution we can find at this point is to communicate to our clients and users that alternatives exist to using this setting:

"Use 'Empty Temporary Internet Files folder when browser is closed'. Each time the browser closes all files will be purged from the cache assuming there is not a lock on a file from another instance of the browser or some external application.

A lot of consideration needs to be given before utilizing 'Do not save encrypted pages to disk'. Sounds like a great security feature and it is but the results of using this feature may cause your help desk calls to go up for download failures or slow performance."