Copy file from remote server to client browser via my server without writing the entire file to disk or memory

Question

This is the scenario.

There is a file on a remote file server (say I have a file hosted on DropBox)

I want to offer that file as a download on my web application (c# asp.net 4.0)

I want to hide the location 100% of the original file (I want it to appear to come from my sever).

I do not want to write this file to memory or to disk on my server.

I had assumed that I would want to use a stream to stream copy. example

Stream inputStream = response.GetResponseStream();
inputStream.CopyTo(Response.OutputStream, 4096);

inputStream.Flush();

Response.Flush();
Response.End();

This however copies the entire stream into memory before writing it out to the client browser. Any ideas would be awesome.

I need my server to basically just act as a proxy and shield the original file location

Thanks for any help.

Answer 1

While I don't see reasons why CopyTo will read whole stream in memory (as there is no intermediate stream anywhere), you can write the same Copy manually to be sure it behaves the way you want.

Consider using Async versions of read/write and if you can use async/awit from C# 4.0 to make code readable.

Old way: Asynchronous Stream Processing, new ways Asynchronous File I/O

Answer 2

The following code is what I ended up with (the buffer size will change in production) This gets a file from a url, and starts streaming it chunk by chunk to a client through my server. The part that took me a bit to figure out was flushing the Response after each chunk was written.

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(digitalAsset.FullFilePath);
            HttpWebResponse response = (HttpWebResponse)request.GetResponse();
            if (response.StatusCode == HttpStatusCode.OK)
            {
                Response.ClearHeaders();
                Response.ClearContent();
                Response.Clear();
                Response.ContentType = "application/octet-stream";
                Response.AddHeader("Content-Disposition", "attachment;filename=" +     digitalAsset.FileName);

                Stream inputStream = response.GetResponseStream();

                byte[] buffer = new byte[512];
                int read;
                while ((read = inputStream.Read(buffer, 0, buffer.Length)) > 0)
                {
                    Response.OutputStream.Write(buffer, 0, read);
                    Response.Flush();
                }
                Response.End();
            }

This can stream any size file through my server without waiting for my server to fist store it into memory or onto disk. The real advantage is that it uses very little memory as only the buffered chunk is stored. To the client the downloads begins instantly. (this is probably obvious to most but was very cool for this novice to see) so we are simultaneously downloading the file from one location and uploading it to another using the server as a sort of proxy.