Menu
I'm scraping a site with HttpWebRequest, but the site is returning an error. The page works fine when I hit it from my browser. I'd like to compare them to see what may be causing the error. I know how to intercept the request from my browser to inspect the headers, but how do I view the data sent by HttpWebRequest?
588
To view the request or response HTTP headers in Google Chrome, take the following steps : In Chrome, visit a URL, right click , select Inspect to open the developer tools. Select Network tab. Reload the page, select any HTTP request on the left panel, and the HTTP headers will be displayed on the right panel.
In a nutshell, WebRequest—in its HTTP-specific implementation, HttpWebRequest—represents the original way to consume HTTP requests in . NET Framework. WebClient provides a simple but limited wrapper around HttpWebRequest. And HttpClient is the new and improved way of doing HTTP requests and posts, having arrived with .
In order to compare what you do in code, and what the browser does, I am sure a HTTP debugging tool such as Fiddler, would be the easiest solution.
Fiddler acts as a proxy between client and server, and displays all information sent over the HTTP protocol.
It is possible that you will need to configure your .NET app to use the proxy that Fiddler provides. This blog post provides details on the subject.
140
http://www.fiddler2.com/fiddler2/ is a great tool for such things.
33
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
