SetWindowSubclass changes ANSI windows to UNICODE

Question

Is SetWindowSubClass() supposed to change an ANSI window into a UNICODE widow? I didn't find anything in the documentation, or on the web, about this behavior.

I created a test application (full source) just to illustrate how SetWindowSubclass (I believe) changes the type of the affected window from ANSI to UNICODE, as it shouldn't! IsWindowUnicode() confirms the change.

 program TwoWaySubclassing;

 {$apptype gui}
 {$R Generic.res}                          

 {
 { I created this test application just to illustrate how SetWindowSubclass()
 { changes -- I believe -- the type of the affected window from ANSI to UNICODE,
 { as it shouldn't! IsWindowUnicode() confirms that.
 {
 { The Delphi 7 (all ANSI) application has 2 edit controls:
 {   1. The smaller, which is subclassed in 2 switchable ways (called Modes).
 {   2. The bigger, like a memo, not subclassed. Just for dumping info.
 {   3. A button for switching between modes, on-the-fly.
 {
 { The default subclassing Mode uses SetWindowLong (the classic way).
 { When pressing the button, the edit control is subclassed via SetWindowSubclass.
 { Pressing it again brings the edit control back to the default SetWindowLong mode.
 {
 { The main window (and all child controls) are created using the ANSI version
 { of the API procedure, so the message handler should receive, in "lParam",
 { a pointer to an ANSI text (along with the wm_SetText message), always!
 {
 { The problem is that's not happening when the edit control is subclassed using
 { the SetWindowSubclass mode! SetWindowSubclass() simply changes the window
 { from ANSI to UNICODE and starts sending a PWideChar(lParam) rather than the
 { expected PAnsiChar(lParam).
 {
 { Once back to the default SetWindowLong mode, the window becomes ANSI again!
 { Just run the application and try switching between modes. Look carefully at the
 { detailed info shown in the bigger edit control.
 {
 { Screenshots:
 {   1. http://imgh.us/mode1.png
 {   2. http://imgh.us/mode2.png
 {
 { Environment:
 {   Windows 7 32-bit
 {   Delphi 7 (all-ANSI)
 {
 { Regards,
 {   Paulo França Lacerda
 }

 uses
   Windows,
   Messages,
   SysUtils;

 type
   UINT_PTR  = Cardinal;
   DWORD_PTR = Cardinal;

   TSubClassProc = function (hWnd:HWND; uMsg:UINT; wParam:WPARAM; lParam:LPARAM; uIdSubclass:UINT_PTR; dwRefData:DWORD_PTR) :LRESULT; stdcall;

   TSubMode = (
     subSetWindowLong,
     subSetWindowSubclass);

 const
   LtBool    :Array[Boolean]  of String = ('False', 'True');
   LtSubMode :Array[TSubMode] of String = ('SetWindowLong', 'SetWindowSubclass');

   strTextUsingPAnsiChar = 'ANSI Text in PAnsiChar(lParam)';
   strTextUsingPWideChar = 'UNICODE Text in PWideChar(lParam)';

 const
   cctrl = Windows.comctl32;

 function SetWindowSubclass    (hWnd:Windows.HWND; pfnSubclass:TSubClassProc; uIdSubclass:UINT_PTR; dwRefData:DWORD_PTR) :BOOL; stdcall; external cctrl name 'SetWindowSubclass';
 function RemoveWindowSubclass (hWnd:Windows.HWND; pfnSubclass:TSubClassProc; uIdSubclass:UINT_PTR) :BOOL;                      stdcall; external cctrl name 'RemoveWindowSubclass';
 function DefSubclassProc      (hWnd:HWND; uMsg:UINT; wParam:WPARAM; lParam:LPARAM) :LRESULT;                                   stdcall; external cctrl name 'DefSubclassProc';

 var
   wc  :TWndClass;
   Msg :TMsg;

   hButton :HWnd;
   hEdit   :HWnd;
   hEdit2  :HWnd;
   hFont   :HWnd;
   hFont2  :HWnd;

   hMainHandle :HWnd;
   swl_OldProc :Pointer;  // Default Procedure for Subclassing #1 (via SetWindowLong)
   SubMode   :TSubMode;


 procedure Release_Resources;
 begin
   DestroyWindow (hButton);  hButton := 0;
   DestroyWindow (hEdit);    hEdit   := 0;
   DestroyWindow (hEdit2);   hEdit2  := 0;
   DeleteObject  (hFont);    hFont   := 0;
   DeleteObject  (hFont2);   hFont2  := 0;
 end;

 procedure MsgBox (S:String);
 begin
   MessageBox (hMainHandle, PChar(S), 'Information', mb_Ok or mb_IconInformation);
 end;

 procedure Reveal_Text (lParam:LPARAM);
 const
   lf  = #13#10;
   lf2 = lf+lf;
 var
   S :String;
   AnsiTxt :String;
   UnicTxt :String;
   Remarks :Array[1..3] of String;
 begin
   if   IsWindowUnicode(hEdit)
   then Remarks[1] := '    (Man! SetWindowSubclass changed it to Unicode!!)'
   else Remarks[1] := '    (great! as designed)';

   AnsiTxt := PAnsiChar(lParam);

   if  (Length(AnsiTxt) = 1)
   then Remarks[2] := '    (text is obviously truncated)'
   else Remarks[2] := '    (text is healthy and is ANSI, as it should)';

   UnicTxt := PWideChar(lParam);

   if  (Pos('?',UnicTxt) > 0)
   then Remarks[3] := '    (text is obviously garbaged)'
   else Remarks[3] := '    (text is healthy, but I want it to be ANSI)';

   S :=
          'Subclassed using: '
     +lf +'    '+LtSubMode[SubMode]+'()'
     +lf2+ 'IsUnicodeWindow(hEdit)? '
     +lf +'    '+LtBool[IsWindowUnicode(hEdit)]
     +lf +       Remarks[1]
     +lf2+'PAnsiChar(lParam):'
     +lf +'    "'+PAnsiChar(lParam)+'"'
     +lf +       Remarks[2]
     +lf2+ 'PWideChar(lParam):'
     +lf +'    "'+PWideChar(lParam)+'"'
     +lf +       Remarks[3];

   SetWindowText (hEdit2, PChar(S));
 end;

 function swl_EditWndProc (hWnd:HWnd; uMsg:UInt; wParam:WParam; lParam:LParam) :LResult; stdcall;
 begin
   Result := CallWindowProc (swl_OldProc, hWnd, uMsg, wParam, lParam);
   if (uMsg = wm_SetText) then Reveal_Text(lParam);
 end;

 function sws_EditWndProc (hWnd:HWND; uMsg:UINT; wParam:WPARAM; lParam:LPARAM; uIdSubclass:UINT_PTR; dwRefData:DWORD_PTR) :LRESULT; stdcall;
 begin
   Result := DefSubclassProc (hWnd, uMsg, wParam, lParam);
   if (uMsg = wm_SetText) then Reveal_Text(lParam);
 end;

 procedure do_SetWindowSubclass;
 begin
   if   not SetWindowSubclass (hEdit, @sws_EditWndProc, 1, dword_ptr($1234{whatever}))
   then RaiseLastOSError;

   SubMode := subSetWindowSubclass;
 end;

 procedure undo_SetWindowSubclass;
 begin
   if   not RemoveWindowSubclass (hEdit, @sws_EditWndProc, 1)
   then RaiseLastOSError;

   SubMode := subSetWindowLong;  // restored
 end;

 function AppWindowProc (hWnd:HWnd; uMsg:UInt; wParam:WParam; lParam:LParam) :LResult; stdcall;
 begin
   case uMsg of
     wm_Command:
     begin
       if (lParam = hButton) then
       case SubMode of
         subSetWindowLong:
         begin
           do_SetWindowSubclass;  // now using SetWindowSubclass()
           SetWindowText (hEdit,   PChar(strTextUsingPWideChar));
           SetWindowText (hButton, PChar('Switch back to SetWindowLong mode'));
         end;

         subSetWindowSubclass:
         begin
           undo_SetWindowSubclass;  // back to SetWindowLong()
           SetWindowText (hEdit,   PChar(strTextUsingPAnsiChar));
           SetWindowText (hButton, PChar('Switch to SetWindowSubclass mode'));
         end;
       end;
     end;

     wm_Destroy:
     begin
       Release_Resources;
       PostQuitMessage (0);
       Exit;
     end;
   end;

   Result := DefWindowProc (hWnd, uMsg, wParam, lParam);
 end;

 var
   W,H :Integer;

 begin
   wc.hInstance     := hInstance;
   wc.lpszClassName := 'ANSI_Wnd';
   wc.Style         := cs_ParentDC;
   wc.hIcon         := LoadIcon(hInstance,'MAINICON');
   wc.lpfnWndProc   := @AppWindowProc;
   wc.hbrBackground := GetStockObject(white_brush);
   wc.hCursor       := LoadCursor(0,IDC_ARROW);

   RegisterClass(wc);  // ANSI (using Delphi 7, so all Windows API is mapped to ANSI).

   W := 500;
   H := 480;

   hMainHandle := CreateWindow (  // ANSI (using Delphi 7, so all Windows API is mapped to ANSI).
     wc.lpszClassName,'2-Way Subclassing App',
     ws_OverlappedWindow or ws_Caption or ws_MinimizeBox or ws_SysMenu or ws_Visible,
     ((GetSystemMetrics(SM_CXSCREEN)-W) div 2),  //   vertically centered in screen
     ((GetSystemMetrics(SM_CYSCREEN)-H) div 2),  // horizontally centered in screen
     W,H,0,0,hInstance,nil);

   // create the fonts
   hFont := CreateFont (-14,0,0,0,0,0,0,0, default_charset, out_default_precis, clip_default_precis, default_quality, variable_pitch or ff_swiss, 'Tahoma');
   hFont2:= CreateFont (-14,0,0,0,0,0,0,0, default_charset, out_default_precis, clip_default_precis, default_quality, variable_pitch or ff_swiss, 'Courier New');

   // create the edits
   hEdit :=CreateWindowEx (WS_EX_CLIENTEDGE,'EDIT','some text', WS_VISIBLE or WS_CHILD or ES_LEFT or ES_AUTOHSCROLL,                10,35,W-40, 23,hMainHandle,0,hInstance,nil);
   hEdit2:=CreateWindowEx (WS_EX_CLIENTEDGE,'EDIT','details',   WS_VISIBLE or WS_CHILD or ES_LEFT or ES_AUTOHSCROLL or ES_MULTILINE,10,72,W-40,300,hMainHandle,0,hInstance,nil);
   SendMessage(hEdit, WM_SETFONT,hFont, 0);
   SendMessage(hEdit2,WM_SETFONT,hFont2,0);

   // create the button
   hButton:=CreateWindow ('Button','Switch to SetWindowSubclass mode', WS_VISIBLE or WS_CHILD or BS_PUSHBUTTON or BS_TEXT, 90,H-95,290,32,hMainHandle,0,hInstance,nil);
   SendMessage(hButton,WM_SETFONT,hFont,0);

   // subclass the Edit using the default method.
   swl_OldProc := Pointer(GetWindowLong(hEdit,GWL_WNDPROC));
   SetWindowLong (hEdit,GWL_WNDPROC,Longint(@swl_EditWndProc));

   SubMode := subSetWindowLong;
   SetWindowText (hEdit, PChar(strTextUsingPAnsiChar));

   // message loop
   while GetMessage(Msg,0,0,0) do
   begin
     TranslateMessage(Msg);
     DispatchMessage(Msg);
   end;
 end.

The application has 2 edit controls:

1. The smaller one, which is subclassed in 2 switchable ways (here called Modes).
2. The bigger one, like a memo, not subclassed. Just for dumping info.

There is also a button for switching between the modes.

The default subclassing mode uses SetWindowLong() (the classic way):

In Delphi 2007 and earlier, the main window (and all child controls) are created using the ANSI version of the Win32 API procedures, so the message handler (of the subclassed control) should receive ANSI text (along with the WM_SETTEXT message), always!

The problem is that's not happening when the edit control is subclassed using SetWindowSubclass()! SetWindowSubClass() changes the window from ANSI to UNICODE and it starts receiving Unicode text rather than the expected ANSI text.

Pressing the button subclasses the edit control via SetWindowSubclass():

Pressing the button again subclasses the edit control via SetWindowLong().

Once back to the SetWindowLong() mode, the edit control automatically receives ANSI text again:

Just run the application and try switching between modes. Look carefully at the detailed info shown in the bigger edit control.

Just to be clear: I think this is a Microsoft bug. However, in case it's a "feature", could someone lead me to the respective documentation? I could not find it anywhere.

Answer 1

According to MSDN:

Subclassing Controls Using ComCtl32.dll version 6

Note ComCtl32.dll version 6 is Unicode only. The common controls supported by ComCtl32.dll version 6 should not be subclassed (or superclassed) with ANSI window procedures.

...

Note All strings passed to the procedure are Unicode strings even if Unicode is not specified as a preprocessor definition.

So it seems this is as designed.

comctl32.dll in my c:\windows\syswow64 folder is version 6.1.