Qt5 QWidget::create() with Win32 HWND embedding not longer working after port from Qt4

Question

the following code tries to embed a native Win32 HWND of a custom created OpenGL windows into a QWidget using the create method:

viewer_widget::viewer_widget(
    QWidget* parent,
    const viewer::viewer_attributes&     view_attrib,
    const wm::context::attribute_desc&   ctx_attrib,
    const wm::surface::format_desc&      win_fmt)
  : QWidget(parent)
{
    setMouseTracking(true);
    setFocusPolicy(Qt::StrongFocus);

    setAttribute(Qt::WA_NativeWindow, true);
    setAttribute(Qt::WA_PaintOnScreen, true); // disables qt double buffering (seems X11 only since qt4.5, ...)
    setAttribute(Qt::WA_NoSystemBackground, true);
    setAutoFillBackground(false);

    _viewer = make_shared<viewer>(math::vec2ui(100, 100), parent->winId(), view_attrib, ctx_attrib, win_fmt);

    // ok set the native window as this widgets window...and hold thumbs
    QWidget::create(_viewer->window()->window_handle(), true, true);
}

The viewer creates a native Win32 (or X11) window wit the parent of the QWidget as a parent. It also creates and initializes an OpenGL context. This was/is done for more control over the context creation and live time (I know that Qt5 is much improved in that regard). The QWidget::create() method now takes the HWND of the native window and embeds it into the current QWidget, so that event handling is completely done through Qt.

This works perfectly on Qt4 (latest used is Qt 4.8.6 x64 on Windows 7/8.1 x64 on Visual Studio 2013).

Now when porting to Qt5 the same code should still work according to the Qt5 documentation. It required minor changes to account for the change in the WId type. The QWidget::winId() methods still return the native HWND handles of the widgets, which I verified using spyxx.exe (Visual Studio Tools).

However, the code does not work anymore (using Qt 5.4.0 x64 on Windows 7/8.1 x64 on Visual Studio 2013). The native window is just not embedded. In Qt4 when inspecting the created QWidget its native handle (winId) after the create call was identical to the native HWND, which meant the embedding worked. Now using Qt5 the QWidget contains its own HWND which I could again confirm using spyxx.exe. Now there is the parent widget/window and two child widgets/windows where there should only be one child (the native one). I looked at the source code of the create() method for both Qt versions and I do not understand why it is not working anymore.

Ok, after the first night trying to figure this out I tried several other methods I could find on forums or the documentation:

• QWindow::fromWinId(HWND) and QWidget::createWindowContainer(QWindow): This one seems to be the way Qt-Devs want me to do it:

_viewer = make_shared<viewer>(math::vec2ui(100, 100), par_hndl, view_attrib, ctx_attrib, win_fmt);
QWindow* native_wnd  = QWindow::fromWinId((WId)_viewer->window()->window_handle());
QWidget* native_wdgt = QWidget::createWindowContainer(native_wnd);

QHBoxLayout* lo        = new QHBoxLayout(this);
lo->setContentsMargins(0, 0, 0, 0);
lo->addWidget(native_wdgt);

This at least at least behaves almost the way I expect. I see my window and in the newly created widget the native window is embedded. BUT: I have found no way to get any mouse events/input from that newly created widget. I added an eventFilter without luck. I tried a QStackedLayout with a transparent top-level widget to catch the input, but this does not work as the widget created through createWindowContainer() is always on top of the windows stack. I tries creating a QWindow-derived class to intercept the nativeEvent() invocation, without luck...

I am at the end of my ideas to get this working as with Qt4. Is there anything I can to differently to get the old behavior back? The keyboard input I can track through the parent widget using a Qt::StrongFocus policy but the mouse input in complete swallows by the native window. I cannot move the the Qt5 OpenGL window code and need to use our custom OpenGL context tools as we are doing things with the contexts that Qt5 still does not fully support.

I could not yet try Qt5 on Linux, but the Qt4 code as seen above did work there.

Answer 1

I have largely solved communication from Qt to the window by subclassing QWidget, using create() as you originally did, and reimplementing the QWidget event functions to make changes to the native window directly. The ones I've found so far (the big ones) are the focus events, resizeEvent, moveEvent, and the contents rect and enable changeEvents. Font, palette, tooltip, etc. changeEvents are probably a lower priority.

The above won't solve the reverse issue, that messages from the native window never arrive in Qt's event dispatcher. You will need to post messages from your WndProc to the widget's HWND (i.e., the return from calling winId()). This also nearly steps around trying to translate virtual key codes to Qt::Keys and back.

Some notes about why this doesn't work gracefully, as it did in Qt4:

• QWindow only works on X11 with native child widgets right now. Try testing on Linux as you mentioned.
• QWidget::create() and QWindow have been problems since 5.1 at the least. The problems you're having in both implementations seem to fit.
• Here is a consolidated bug report about QWindow intended usage and issues

Other notes:

• QWidget::effectiveWinId() is preferred even though you are creating an immediate child window. This, unfortunately, breaks in 5.4.1.
• Update to 5.4.1. There are binary compatibility issues with 5.4.0 and all other versions of Qt5.