PyQt5: mouseClick and source-code in QWebEngineView

Question

I have a working script that uses PyQt-5.5.1, which I now want to port to a new PyQt version (5.7). Adapting most of the things was fine, but I faced two major problems: (1) to perform a (simulated) mouseclick, (2) to access (let's say: print) the html source-code of a webpage which is currently displayed in the QWebView or QWebEngineView, respectively.

For example, I could do the following using QWebView in PyQt-5.5.1:

QTest.mouseClick(self.wvTest, Qt.LeftButton, QPoint(x, y))

and

frame = self.wvTest.page().mainFrame()
print(frame.toHtml().encode('utf-8'))

I am aware of the docs as well as this page about porting to QWebEngineView but unable to convert C++ notation to a working Python code.

How can I adapt this to QWebEngineView in PyQt-5.7? Below is a fully working snippet for PyQt-5.5.1, which fails for the new PyQt-version:

• for Button1: no mouse click reaction at all.
• for Button2: AttributeError: 'QWebEnginePage' object has no attribute 'mainFrame', and when I delete the mainframe(): TypeError: toHtml(self, Callable[..., None]): not enough arguments.

import sys
from PyQt5.QtWidgets import QWidget, QPushButton, QApplication
from PyQt5.QtCore import QRect, Qt, QUrl, QPoint, QEvent
from PyQt5.QtTest import QTest
from PyQt5.Qt import PYQT_VERSION_STR

if PYQT_VERSION_STR=='5.5.1': from PyQt5 import QtWebKitWidgets
else: from PyQt5 import QtWebEngineWidgets

class Example(QWidget):
    def __init__(self):
        super().__init__()
        self.initUI()

    def initUI(self):
        self.button1 = QPushButton('Button1', self)
        self.button1.clicked.connect(self.buttonOne)
        self.button1.setGeometry(QRect(10, 10, 90, 20))

        self.button2 = QPushButton('Button2', self)
        self.button2.clicked.connect(self.buttonTwo)
        self.button2.setGeometry(QRect(110, 10, 90, 20))

        if PYQT_VERSION_STR=='5.5.1': self.wvTest = QtWebKitWidgets.QWebView(self)
        else: self.wvTest = QtWebEngineWidgets.QWebEngineView(self)
        self.wvTest.setGeometry(QRect(10, 40, 430, 550))
        self.wvTest.setUrl(QUrl('http://www.startpage.com'))
        self.wvTest.setObjectName('wvTest')

        self.setGeometry(300, 300, 450, 600)
        self.setWindowTitle('WebView minimalistic')
        self.show()


    def buttonOne(self):
        qp = QPoint(38, 314)
        QTest.mouseClick(self.wvTest, Qt.LeftButton, pos=qp)  # or: QTest.mouseMove(self.wvTest, pos=self.qp)
        print('Button1 pressed.')


    def buttonTwo(self):
        frame = self.wvTest.page().mainFrame()
        print(frame.toHtml().encode('utf-8'))


if __name__ == '__main__':
    app = QApplication(sys.argv)
    ex = Example()
    sys.exit(app.exec_())

Answer 1

The QWebEngineView class is not a drop-in replacement for QWebView. As the porting guide makes clear, many of the APIs have fundamentally changed, and some major features are completely missing. This will probably make it impossible to write a compatibility layer unless your browser implementation is very, very simple. It will most likely be easier to write a separate test-suite (unless you don't mind writing a huge amount of conditional code).

To start with, you will need to implement a work-around for QTBUG-43602. The current design of QWebEngineView means that an internal QOpenGLWidget handles mouse-events, so your code will need to get a new reference to that whenever the page is loaded:

class Example(QWidget):
    ...

    def initUI(self):
        ...    
        self._glwidget = None

        if PYQT_VERSION_STR=='5.5.1':
            self.wvTest = QtWebKitWidgets.QWebView(self)
        else:
            self.wvTest = QtWebEngineWidgets.QWebEngineView(self)
            self.wvTest.installEventFilter(self)
        ...

    def eventFilter(self, source, event):
        if (event.type() == QEvent.ChildAdded and
            source is self.wvTest and
            event.child().isWidgetType()):
            self._glwidget = event.child()
            self._glwidget.installEventFilter(self)
        elif (event.type() == QEvent.MouseButtonPress and
              source is self._glwidget):
            print('web-view mouse-press:', event.pos())
        return super().eventFilter(source, event)

    def buttonOne(self):
        qp = QPoint(38, 314)
        widget = self._glwidget or self.wvTest
        QTest.mouseClick(widget, Qt.LeftButton, pos=qp)

For accessing the html of the page, you will need some conditional code, because the web-engine API works asynchronously, and requires a callback. Also there are no built-in APIs for handling frames in web-engine (you need to use javascript for that), so everything needs to go through the web-page:

    def buttonTwo(self):
        if PYQT_VERSION_STR=='5.5.1':
            print(self.wvTest.page().toHtml())
        else:
            self.wvTest.page().toHtml(self.processHtml)

    def processHtml(self, html):
        print(html)