Unable to click custom control in design time

Question

I'm building a custom control (inherited from TCustomControl) in Delphi XE2 (and have had this issue in other controls of mine) and in design time, I'm unable to click them. I know it has to do with the mouse capture, and catching the mouse events and handling them differently in design time than run time, but I do not know how to properly accommodate for this. In other words, of the many work arounds I can think of, I can not decide which one is the proper (or most efficient) way.

I'm sure there must be some very simple standard to this, most likely utilizing the ControlStyle or CreateParams but do not know what.

In this particular control (and I have not seen a pattern in this issue), I am capturing messages including WM_NCHITTEST and WM_LBUTTONDOWN. In design-time, the control is 100% active as if it were run time, and when clicking, it's instead performing the run time code.

I have a feeling it's in my hit test message handler, so here's that code (some things renamed):

procedure TMyCustomControl.WMNCHitTest(var Message: TWMNCHitTest);
var
  P: TPoint;
  Poly: TPoints;
  X: Integer;
  I: TMyCollectionItem;
  Ch: Bool; //Need to improve invalidation
begin
  Ch:= False;
  P:= ScreenToClient(Point(Message.Pos.X, Message.Pos.Y));
  for X := 0 to Items.Count - 1 do begin
    I:= Items[X];
    Poly:= I.Points;
    FMouseIndex:= -1;
    FMouseState:= bmNone;
    if PointInPolygon(P, Poly) then begin //checks if point is within polygon
      FMouseIndex:= X;
      FMouseState:= bmHover;
      Ch:= True;
      Break;
    end;
  end;
  if Ch then Invalidate;
end;

And also my control's constructor (stripped):

constructor TMyCustomControl.Create(AOwner: TComponent);
begin
  inherited;
  ControlStyle:= ControlStyle - [csDesignInteractive];
end;

Answer 1

But of course you are right. You are not returning anything in the WM_NCHITTEST handler. Your Mmessage.Result is '0' (HTNOWHERE) when your handler is called and you are not assigning anything else to it.

Either call inherited at some point, or implement your logic and return (set the Message.Result) HTCLIENT for the points you consider to be the interior of your control.

It that's already the desired behavior at runtime, you can include a design-time check (but I guess you should be doing all that calculation for a reason):

if csDesigning in ComponentState then
  Msg.Result := HTCLIENT;

Answer 2

The official way to support mouse interactions at design-time is to respond with a non-zero result to the CM_DESIGNHITTEST message. The component will then receive normal mouse messages.