webclient The remote server returned an error: (500) Internal Server Error

Question

I'm trying to download a webpage using webclient and get the 500 internal error

public class AsyncWebClient
    {
        public string GetContent(string url)
        {
            return GetWebContent(url).Result.ToString();
        }
        private Task<string> GetWebContent(string url)
        {
            var wc = new WebClient();
            TaskCompletionSource<string> tcs = new TaskCompletionSource<string>();

            wc.DownloadStringCompleted += (obj, args) =>
            {
                if (args.Cancelled == true)
                {
                    tcs.TrySetCanceled();
                    return;
                }

                if (!String.IsNullOrEmpty(args.Result))
                    tcs.TrySetResult(args.Result);
            };

            wc.DownloadStringAsync(new Uri(url));
            return tcs.Task;
        }
    }

and call:

var wc =new AsyncWebClient();
var html = wc.GetContent("http://truyen.vnsharing.net/");    >> always get the above error

if i use other site, then it works just fine. don't know what's special in this site.

Please help !!

Answer 1

The server is most likely expecting a correct User-Agent.

Update your code to the following:

var wc = new WebClient();
wc.Headers.Add ("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");
TaskCompletionSource<string> tcs = new TaskCompletionSource<string>();