Add request headers with WebClient C#

Question

I have the following code with which I download a web-page into a byte array and then print it with Response.Write:

WebClient client = new WebClient();

byte[] data = client.DownloadData(requestUri);

  /***********        Init response headers    ********/
  WebHeaderCollection responseHeaders = client.ResponseHeaders;
  for (int i = 0; i < responseHeaders.Count; i++)
       {
            Response.Headers.Add(responseHeaders.GetKey(i), responseHeaders[i]);
       }
  /***************************************************/

Besides of the response headers, I need to add request headers as well. I try to do it with the following code:

  /***********        Init request headers    ********/
  NameValueCollection requestHeaders = Request.Headers;
  foreach (string key in requestHeaders)
  {
      client.Headers.Add(key, requestHeaders[key]);
  }
  /***************************************************/

However it does not work and I get the following exception:

This header must be modified using the appropriate property.Parameter name: name

Could anybody help me with this? What's the correct way of adding request headers with WebClient?

Thank you.

Answer 1

The headers collection "protects" some of the possible headers as described on the msdn page here: http://msdn.microsoft.com/en-us/library/system.net.webclient.headers.aspx

That page seems to give all the answer you need but to quote the important part:

Some common headers are considered restricted and are protected by the system and cannot be set or changed in a WebHeaderCollection object. Any attempt to set one of these restricted headers in the WebHeaderCollection object associated with a WebClient object will throw an exception later when attempting to send the WebClient request.

Restricted headers protected by the system include, but are not limited to the following:

Date

Host

In addition, some other headers are also restricted when using a WebClient object. These restricted headers include, but are not limited to the following:

Accept

Connection

Content-Length

Expect (when the value is set to "100-continue"

If-Modified-Since

Range

Transfer-Encoding

The HttpWebRequest class has properties for setting some of the above headers. If it is important for an application to set these headers, then the HttpWebRequest class should be used instead of the WebRequest class.

I suspect the reason for this is that many of the headers such as Date and host must be set differently on a different request. You should not be copying them. Indeed I would personally probably suggest that you should not be copying any of them. Put in your own user agent - If the page you are getting relies on a certain value then I'd think you want to make sure you always send a valid value rather than relying on the original user to give you that information.

Essentially work out what you need to do rather than finding something that works and doing that without fully understanding what you are doing.

Answer 2

Looks like you're trying to set some header which is must be set using one of the WebClient properties (CachePolicy, ContentLength or ContentType)

Moreover, it's not very good to blindly copy all the headers, you need to get just those you really need.