Get webpage page content and HTTP status code in C#

Question

In a C# Windows Forms application I can get the contents of a webpage using:

string content = webClient.DownloadString(url);

And I can get the HTTP header using:

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
string response = ((HttpWebResponse)request.GetResponse()).StatusCode.ToString();

Is there a way to get both the contents and the HTTP status code (if it fails) in one trip to the server instead of twice?

Thanks.

Answer 1

You can read the data from the Stream inside the HttpWebResponse object:

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
    HttpStatusCode statusCode = ((HttpWebResponse)response).StatusCode;
    string contents = reader.ReadToEnd();
}

In this way you will have to detect the encoding by hand, or using a library to detect encoding. You can read the encoding as a string from the HttpWebResponse object as well, when one exists, it is inside the ContentType property. If the page is Html, then you will have to parse it for a possible encoding change in the top of the document or inside the head.

Read handling the encoding from ContentType header

var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
string content;
HttpStatusCode statusCode;
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
{
    var contentType = response.ContentType;
    Encoding encoding = null;
    if (contentType != null)
    {
        var match = Regex.Match(contentType, @"(?<=charset\=).*");
        if (match.Success)
            encoding = Encoding.GetEncoding(match.ToString());
    }

    encoding = encoding ?? Encoding.UTF8;

    statusCode = ((HttpWebResponse)response).StatusCode;
    using (var reader = new StreamReader(stream, encoding))
        content = reader.ReadToEnd();
}