Handling a Unicode String in Delphi Versions <= 2007

Question

Background: This question relates to versions of Delphi below 2009 (ie without Unicode support built in). I have a specification that requires me to transmit a Unicode encoded string over a TCP connection but I do not have Delphi 2009.

Question Is there a single function or very small library (I don't need too much bulk) that I can use to encode a single string into UTF-8 immediately prior sending over the wire? As a second part of my question: if there are UTF-8 encoded strings being sent back as a response, I guess I would then need another function to get it back into a Delphi string format. I understand the limitations of such Unicode support in this way.

Answer 1

Delphi versions prior to Delphi 2009 do have Unicode support built in. The WideString type has been available since Delphi 4, I think, maybe earlier. WideString isn't as nice as the new UnicodeString type, but it still holds 16-bit Unicode characters, and you can type-cast it to PWideChar to send strings to Unicode API functions. The Windows unit declares most of the "wide" versions of the API functions, and there's nothing to stop you from declaring other functions yourself if you find some missing.

What prior versions don't have is Unicode support in the VCL. For that, you can use the Tnt Unicode controls. They used to be free. Looks like there are a few places where the latest free version is still available: (1), (2).

The JCL has a couple of units for working with Unicode. The JclWideStrings unit has mostly light-weight utility functions. The JclUnicode unit is more complete, but it also includes a sizable resource for determining character properties of all Unicode characters.

With the JCL you have a few choices for classes to hold lists of WideString values. I think Delphi 7 even comes with a class for that.

Don't think that just because you don't have Delphi 2009 you can't write a Unicode program.

If you have a WideString value, and you want to encode it as UTF-8, then call the Utf8Encode function. It will return an AnsiString value, or possibly Utf8String, if your Delphi version declares that type. It's not the same as Delphi 2009's Utf8String type, though. Delphi 2009's will automatically convert to UnicodeString or AnsiString(x) and vice versa in assignment statements. Prior versions just have a single AnsiString type, so you need to keep track for yourself which variables hold UTF-8 data and which hold Ansi data. (Hungarian notation on your variable and parameter names can help you keep track.) And of course, there's also a Utf8Decode function for converting UTF-8 data back to WideString.

For handling other character encodings, you want to check out Open XML, a free XML library for Delphi. As part of its XML handling, it has support for converting between 70 different encodings.