Delphi indy10 http server and ExtJS form submit

Question

I have a problem I don't know how to solve.

I have an Indy10 HTTP server. I have used both Indy9 and Indy10 HTTP servers in many applications and never had any problems. But now I am using Indy10 HTTP server with ExtJS javascript RAI framework.

The problem is when I submit data that contains non-ansi characters. For instance when I submit letter "č" which is a letter in 1250 codepage (slovenian, croatian...) I get the following in Indy under "unparsed params" -> "%C4%8D". This is correct hexadecimal representation of the "č" letter in utf-8 encoding. All my pages are utf-8 and I never had any problems submiting form data to Indy. I debugged the code and saw that I actually get a sequence of bytes like this: [37, 67, 52, 37, 56, 68]. This is the byte representation of the string "%C4%8D". But of course Indy cannot encode this correctly to UTF-16. So as an example. The actual form field:

FirstName=črt

comes out like this when submited:

FirstName=%C4%8Drt

I don't know how to solve this. I looked at ExtJS forums, but there is nothing on this topic. Anybody know anything about this kind of problem?

EDIT:

If I encode params ad JSON they arrive correctly. I also tried to URL decode the params, but the result is not correct. Maybe I missed something. I will look at this again. And yes it seems that ExtJS URL encodes the params

EDIT2:

Ok, I have discovered more. I compared the actual content of the post data. It is like this:

Delphi 2006 (Indy10): FirstName=%C4%8D
Delphi 2010 (Indy10): FirstName=%C4%8D

In both case the unparsed params are identical. I have ParseParams turned on and in BDS2006 they are correctly parsed, but under 2010 they are not. This is Indy10 bulked with delphi. Is there a bug in this version or am I doing something wrong?

EDIT3:

I downloaded the latest nightly build od Indy10. Still the same issue.

EDIT4:

I am forced to accept my own answer.

Answer 1

To answer on this topic.

This is definitely not working as it should under unicode. Indy uses unicode strings internally. The problem is when parameters are decoded to TStringList. The problem is the line:

Params.Add(TIdURI.URLDecode(s));

found in the "TIdHTTPRequestInfo.DecodeAndSetParams". It does not decode params correctly, probably because it is working over unicode strings.

The workaround I found is to use "HTTPDecode" from "HTTPApp.pas".

Params := TStringList.Create;
try
  Params.StrictDelimiter := True;
  Params.Delimiter := '&';

  // parse the parameters and store them into temporary string list
  Params.DelimitedText := UTF8ToString(HTTPDecode(UTF8String(Request.UnparsedParams)));
  // do something with params... 
finally
  Params.Free;
end;

But I cannot believe that such a common task is not working correctly. Can someone confirm this is really a bug or am I just doing something wrong?

Answer 2

It appears the string is URL encoded, so you use the following code to decode:

uses
  idURI;

value := TIdURI.URLDecode( value );

edit

It appears there is a case where the decoder does not properly decode the double bytes as a single character. Looking at the source, it does appear that it would decode properly if the character is coded like %UC48D but in my testing this still does not decode properly. What is interesting is that the TidURI.ParamsEncode function generates the proper encoding, but this encoding is not reversible using the proper routines in the latest version of Indy 10.

Answer 3

I´m using Delphi 7 and migrate to Indy 10. I found likely problem with portuguese characters and solve this changing the source below:

procedure TIdHTTPRequestInfo.DecodeAndSetParams(const AValue: String);
  ...
  //Params.Add(TIdURI.URLDecode(s)); //-- UTF8 supose
  Params.Add(TIdURI.URLDecode(s,TIdTextEncoding.Default)); //-- ASCII worked
  ...

end;