Submit form using HttpClient in C#

Question

I'm getting the website form by htmlagilitypack, setting the form variable and trying to submit the form. Everything looks that is working fine, but the response from the form submit is null.

static void Main(string[] args)
    {
        string urlAddress = "mywebsite";

        HtmlWeb web = new HtmlWeb();
        HtmlDocument doc = web.Load(urlAddress);

        // Post link
        var post = doc.GetElementbyId("post").GetAttributeValue("href", "");

        doc = web.Load(post);

        // get the form
        var form = doc.DocumentNode.SelectSingleNode("//form[@class='picker']");

        // get the form URI
        string actionValue = form.Attributes["action"]?.Value;
        System.Uri uri = new System.Uri(actionValue);

        // Populate the form variable
        var formVariables = new List<KeyValuePair<string, string>>();
        formVariables.Add(new KeyValuePair<string, string>("id", "ho"));
        var formContent = new FormUrlEncodedContent(formVariables);

        // submit the form
        HttpClient client = new HttpClient();
        var response = client.PostAsync(uri, formContent);

    }

Does anyone know why my variable response is null?

Thanks

Answer 1

HttpClient.PostAsync return a Task<HttpResponseMessage> so normally it would need to be awaited. As you are using it in main method you would have to get the result from the task

var response = client.PostAsync(uri, formContent).GetAwaiter().GetResult();

Or the simpler

var response = client.PostAsync(uri, formContent).Result;

In both cases response would be an instance of HttpResponseMessage. You can inspect that instance for HTTP status and the content of the response.

If using .net core you can even use an async Main method like

static async Task Main(string[] args) {

    //..code removed for brevity

    var response = await client.PostAsync(uri, formContent);
    var content = await response.Content.ReadAsStringAsync();
    //...
}

Answer 2

You are missing .Result in your PostAsync. Simply, change the line to:

var response = client.PostAsync(uri, formContent).Result;

.Result however, runs the tasks synchronously. It returns AggregateException if there is an exception making it slightly difficult to debug.

If you are using Visual Studio 2017 Update 15.3 or above and C# 7.1 however, there is support for aysnc main now.

You can modify your code as follows:

static async Task Main()
{
    string urlAddress = "mywebsite";

        HtmlWeb web = new HtmlWeb();
        HtmlDocument doc = web.Load(urlAddress);

        // Post link
        var post = doc.GetElementbyId("post").GetAttributeValue("href", "");

        doc = web.Load(post);

        // get the form
        var form = doc.DocumentNode.SelectSingleNode("//form[@class='picker']");

        // get the form URI
        string actionValue = form.Attributes["action"]?.Value;
        System.Uri uri = new System.Uri(actionValue);

        // Populate the form variable
        var formVariables = new List<KeyValuePair<string, string>>();
        formVariables.Add(new KeyValuePair<string, string>("id", "ho"));
        var formContent = new FormUrlEncodedContent(formVariables);

        // submit the form
        HttpClient client = new HttpClient();
        var response = await client.PostAsync(uri, formContent);
}

This will help you to await the async task. In case you get any exception, you would receive the actual exception rather than AggregateException.

It is important to note that C# 7.1 currently is currently not enabled by default. To enable the 7.1 features, you would need to change the language version setting for your project.

Refer this link for more details.