wget/curl in C#

Question

I'm writing a scraper in C# and I'd like to download some data to files and submit some forms. I've been using wget and curl so far for that. How would I do that in C# (on Linux)? (I mean a library for that, not calling shell commands via system() or whatnot).

Answer 1

You can use System.Net.WebClient, which is the simplest interface for downloading resources in .NET. If you need more control on the requests look at HttpWebRequest.

For WebClient, just instantiate an instance, and call one of the Download methods which fits your needs:

var cli = new WebClient();
string data = cli.DownloadString("http://www.stackoverflow.com");

Answer 2

WebRequest is one of the .NET classes for retrieving web content.

A good library to parse the HTML is the HTML Agility Pack that can also directly download the page.