Menu
An odd one, I'm trying to read the <Head> section of a lot of different websites out there, and one particular type of server, Apache, sometimes gives the code 403 forbidden. Not all apache servers do this, so it may be a config setting or a particular version of the server.
When I then check the url with a web browser (Firefox, for example) the page loads fine. The code sorta looks like this:
var client = new WebClient();
var stream = client.OpenRead(new Uri("http://en.wikipedia.org/wiki/Barack_Obama"));
Normally, a 403 is a access permission failed sort of thing, but these are normally unsecure pages. I'm thinking that Apache is filtering on something in the request headers since I'm not bothering to create any.
Maybe someone who knows more about Apache can give me some ideas of what's missing in the headers. I'd like to keep the headers as small as possible to minimize bandwidth.
Thanks
655
Choose Tools > Internet Options. Click the Advanced tab. Under Browsing, deselect Show Friendly HTTP Error Messages. Save the changes.
Try setting the UserAgent header:
string _UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)";
client.Headers.Add(HttpRequestHeader.UserAgent, _UserAgent);
I had a similar problem and below setting solved it
Client.Headers["Accept"] = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*";
Client.Headers["User-Agent"] ="Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MDDC)";
answered Sep 30 '22 16:09
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With