How can I set the user-agent in the http header

Question

I'm trying to set the user-agent in my http header in our override function GetWebRequest in C#

protected override System.Net.WebRequest GetWebRequest(Uri uri)
{
    System.Net.HttpWebRequest request = base.GetWebRequest(uri) as System.Net.HttpWebRequest;                
    request.Headers.Add("User-Agent", ([email protected]"));
    request.ProtocolVersion = System.Net.HttpVersion.Version10;
    request.CachePolicy = new RequestCachePolicy(RequestCacheLevel.NoCacheNoStore);
    request.KeepAlive = false;
    return request;
} 

My request is being returned with an error and I was told they are not seeing the user-agent. I've tried other ways of setting the header user-agent but nothing is working so far. Some other ways I've tried are

request.Headers["User-Agent"] = "[email protected]");
request.UserAgent = "[email protected]";
request.Headers.Set("User-Agent", "[email protected]");

The site I am trying to send my request is telling me it needs a way to identify us in case of issues and "please make sure your request includes the "user-agent" http header." They've sent me an example but it is using CURL

curl_setopt($ch, CURLOPT_USERAGENT,'[email protected])

Answer 1

User-agent represent your browser like Firefox or Chrome

Use the UserAgent property on HttpWebRequest by casting it to a HttpWebRequest

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.UserAgent = "my user agent";

or

request = new HttpClient();
request.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0");

Answer 2

If you write your own application requesting http content it is best practice to set any form of identification in the user agent. It can be email, website or product name if that is widely known.

According to HTTP specification the product tokens in the user agent:

(...) SHOULD be short and to the point. They MUST NOT be used for advertising or other non-essential information. Although any token character MAY appear in a product-version, this token SHOULD only be used for a version identifier (i.e., successive versions of the same product SHOULD only differ in the product-version portion of the product value).

In its easiest form it can be one of the following:

request.UserAgent = "MyAppName/1.0.0";
request.UserAgent = "MyAppName/1.0.0 ([email protected])";
request.UserAgent = "MyAppName/1.0.0 (+http://www.example.com)";

URLs in comment must be prefixed with + (documented in this answer by Brendon). Also worth noticing that any mail address should go to company or developer responsible for the product, not to the user using it.

If you need compatibility with modern browsers (if you expect the result to be HTML, and not an API with xml/json/etc.), you can also add the "default" Mozilla version along with your operating system info:

request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) MyAppName/1.0.0 ([email protected])"

It is best to send correct system info, and not just copy a found user agent string with incorrect operating system info.