XmlReader.Create() The remote server returned an error: (403) Forbidden

Question

_rssSource = @"http://feeds.bbci.co.uk/news/world/rss.xml";
XmlReader reader = XmlReader.Create(_rssSource);

I have the above code which is throwing the following error at GetResponse():

System.Net.WebException was unhandled by user code
  Message=The remote server returned an error: (403) Forbidden.
  Source=System
  StackTrace:
       at System.Net.HttpWebRequest.GetResponse()
       at System.Xml.XmlDownloadManager.GetNonFileStream(Uri uri, ICredentials credentials, IWebProxy proxy, RequestCachePolicy cachePolicy)
   at System.Xml.XmlDownloadManager.GetStream(Uri uri, ICredentials credentials, IWebProxy proxy, RequestCachePolicy cachePolicy)
   at System.Xml.XmlUrlResolver.GetEntity(Uri absoluteUri, String role, Type ofObjectToReturn)
   at System.Xml.XmlReaderSettings.CreateReader(String inputUri, XmlParserContext inputContext)
   at System.Xml.XmlReader.Create(String inputUri, XmlReaderSettings settings, XmlParserContext inputContext)
   at System.Xml.XmlReader.Create(String inputUri)

I have tried to trace what is being requested via fiddler but no requests are showing apart from the request for the page that contains the code. The link work fine through ie/visual studio.

I'm not sure if its something to do with the proxy server/IIS/internet settings.

Any idea what is going on?

Answer 1

Try to access the same feed with a browser running on the same machine as your code, to eliminate easy-to-detect issues like required credentials or network limitations. If your browser can read the feed just fine, then you may need to tweak the request properties.

The default .NET user-agent supplied by XmlReader.Create() might be explicitly forbidden by the RSS service. An easy way to test for this is to explicitly set your user agent string within a WebClient object, and then call the OpenRead() method on your URL:

WebClient webClient = new WebClient();
webClient.Headers.Add("user-agent", "MyRSSReader/1.0");

XmlReader reader = XmlReader.Create(webClient.OpenRead(_rssSource));

If your invented user agent string doesn't work, you might try spoofing the string of a well known Feed Reader or browser.