POST request with IdHTTP

Question

Hi I'm trying to fill in a form using the component IdHTTP POST method, the code that I have is this:

var
  par2: TIdMultiPartFormDataStream;
  rta: string;

begin

    par2 := TIdMultiPartFormDataStream.Create;
    par2.AddFormField('ipaddress', ip.text);
    par2.AddFormField('submit', 'Submit');

    rta := idhttp1.Post
      ('http://www.melissadata.com/lookups/iplocation.asp?ipaddress=', par2);

    memo.Lines.Add(rta);

end;

And the code of the form is this:

<form method="post" action="iplocation.asp">
<table class="Tableresultborderblue" align="right" border="0" cellpadding="0" cellspacing="0" width="300">
<tbody><tr><td align="center"><span style="font-size:12px;">Your IP Address: 181.92.20.173</span></td></tr>
<tr><td align="center" height="35px"><strong>Enter an IP address</strong></td></tr>
<tr><td align="center"><input id="ipaddress" name="ipaddress" value="" class="inputoff" type="text"></td></tr>
<tr><td height="10"></td></tr>
<tr><td align="center" height="45px"><input title="Click to process Address" class="btn" value="Submit" type="submit"></td></tr>
<tr><td height="10"></td></tr>
</tbody></table>   
</form>

The problem is that I do not return the appropriate response form, which brings me back is the empty form as if he had it all wrong

What am I doing wrong?

Answer 1

You are submitting the webform data using a TIdMultipartFormDataStream, which posts the data using the multipart/form-data format. However, the HTML <form> tag does not have an enctype=multipart/form-data attribute, so the server is not expecting that format. It is expecting the data to be submitted using the default application/x-www-webform-urlencoded format instead. That is accomplished by posting the data using a TStrings object instead.

The following code works fine for me, it receives an HTML response that includes the results of the IP address lookup (which you would have to parse after receiving):

var
  PostData: TStringList;
  rta: string;
begin
  PostData := TStringList.Create;
  try
    PostData.Add('ipaddress='+ip.Text);
    rta := IdHTTP1.Post('http://www.melissadata.com/lookups/iplocation.asp', PostData);
  finally
    PostData.Free;
  end;

  Memo1.Lines.Text := rta;
end;

That being said, to be politically correct there are two other factors to consider. If you use a packet sniffer like Wireshark to see what a normal web browser submits, you will notice that:

1. there is a Referer header on the webform submission, to let the server know where the submission originated from. You are omitting that header. Sometimes web servers validate the Referer to make sure a request is actually coming from their own site and not somewhere else, so you should provide a Referer when appropriate.

2. the server sends a cookie with the initial HTML, which then gets sent back to the server with the webform submission. Sometimes web servers require those cookies to ensure the client visited the original site before submitting the webform data. So you should download the initial HTML to let TIdHTTP obtain any necessary cookies so it can send them back to the server.

The following code also works fine for me, receives the same HTML response that includes the results of the IP address lookup:

var
  PostData: TStringList;
  rta: string;
begin
  // Get webform HTML and any cookies that go with it
  IdHTTP1.Get('http://www.melissadata.com/lookups/iplocation.asp');

  // now post the webform data back to the server
  PostData := TStringList.Create;
  try
    PostData.Add('ipaddress=23.241.61.8');
    IdHTTP1.Request.Referer := 'http://www.melissadata.com/lookups/iplocation.asp';
    rta := IdHTTP1.Post('http://www.melissadata.com/lookups/iplocation.asp', PostData);
  finally
    PostData.Free;
  end;

  Memo1.Lines.Text := rta;
end;

Lastly, you are using an HTML-based lookup service, which is not a very good decision. HTML is meant for presenting data for human consumption, not well suited for machine parsing. There are plenty of other IP lookup services available that provide more efficient REST-based APIs for providing results in machine-parsable formats, like XML or JSON. You should seriously consider switching to one of those services instead.

Answer 2

I checked the code again and it really seems that you should not send it as multipart form data. The site does not accept that. Use this

var
  PostData: TStringList;
  res: string;
begin
  PostData:=TStringList.Create;
  try
    PostData.Add('ipaddress='+ip.text);

    res:=IdHTTP1.Post('http://www.melissadata.com/lookups/iplocation.asp', PostData);

    Memo1.Lines.Text:=res;
  finally
    PostData.Free;
  end;
end;