502 error when redirecting stream from another site

Question

In my WEB Api 2 controller I want to request file from one site and return this file from my controller. Here is the code

public HttpResponseMessage GetLecture()
{            
    HttpWebRequest request = WebRequest.CreateHttp("http://openmedia.yale.edu/cgi-bin/open_yale/media_downloader.cgi?file=/courses/spring11/phil181/mp3/phil181_01_011111.mp3");
    request.Referer = @"http://oyc.yale.edu/courses/";

    var receivedResponse = request.GetResponse();

    HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);                    
    response.Content = new StreamContent(receivedResponse.GetResponseStream());
    response.Content.Headers.ContentType = MediaTypeHeaderValue.Parse(receivedResponse.ContentType);
    response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
    response.Content.Headers.ContentDisposition.FileName = "phil181_01_011111.mp3";
    response.Content.Headers.ContentLength = receivedResponse.ContentLength;

    return response;
}

Locally it works fine and I can download the file but when I deploy it to Azure I'm getting 502 Error. Web server received an invalid response while acting as a gateway or proxy server.

Logging shows that it fails after returning response so no exceptions during method execution.

It's ~50MB file. For smaller files code works fine.

How can I make this code works on Azure for 50 MB files?

Answer 1

First off, I will recommend configuring remote IIS administration for your Azure Web App since that will allow you direct access to IIS logs and associated settings. I have a strong feeling that the problem has to do with IIS/ASP.NET limits. You already pointed out that the method completes execution within seconds so this is likely not a timeout problem.

You also mentioned that the code works for smaller file sizes but fails when the size is increased. This points towards IIS bufferingLimit for your website being set to a small value (or perhaps the default value of ~4 MB). Here is a link describing how you can increase this limit to the desired value. Hope this works for you. If it doesn't, I would recommend digging into those IIS logs for your website.