PyQT4 - painting on an image for region selection

Question

I would like to write an image labeling tool using PyQT4:

• load a number of images from a specified folder; for each image:
  • the user selects objects (e.g., car) from the images, by painting the region for that object with the mouse
  • when the selection is done, the object mask is shown overlaid on the original image
  • when selection of all objects completed, the program saves each object mask (background: 0, foreground: 255) as a separate png image
• the user should be able to zoom in/out of the image

I already wrote a similar program (without zoom in/out) in c++ with wxWidgets. I am quite new to PyQT4 & trying to learn how things work. The most difficult part seems painting and getting the object masks correctly even when the user zooms in/out.

Which PyQT classes would be ideal for this problem? How can I get the object masks correctly (maybe as numpy array) and save them?

Thanks a lot.

---

Following your recommendation I wrote a piece of code, to display an image and draw on the image with the mouse (still in the experimentation and learning stage).

I store the image in QGraphicsPixmapItem, add it to the scene. Then, I paint the image by overriding its paint method. Finally, I override the mouse events to get the mouse position and draw a circle there. But when I move the mouse, the old circle gets deleted and a new one is painted. That is, the circle is not painted on the image itself. I think, I should use something like the following, so that the painting is permanent on the image:

painter = QPainter()
painter.begin(pixmap)
# here do the drawing
painter.end() 

But, the problem is, the paint function already takes a painter as an argument; re-creating a new one within the paint function does not work (obviously)..

Here is the code:

from PyQt4.QtCore import *
from PyQt4.QtGui import *

class ImageDrawPanel(QGraphicsPixmapItem):
    def __init__(self, pixmap=None, parent=None, scene=None):
        super(ImageDrawPanel, self).__init__()
        self.x, self.y = -1, -1        
        self.radius = 10

        self.pen = QPen(Qt.SolidLine)
        self.pen.setColor(Qt.black)
        self.pen.setWidth(2)

        self.brush = QBrush(Qt.yellow)


    def paint(self, painter, option, widget=None):               
        painter.drawPixmap(0, 0, self.pixmap())                
        painter.setPen(self.pen)
        painter.setBrush(self.brush)        
        if self.x >= 0 and self.y >= 0:
            painter.drawEllipse(self.x-self.radius, self.y-self.radius, 2*self.radius, 2*self.radius)
            self.x, self.y = -1, -1

    def mousePressEvent (self, event):
        print 'mouse pressed'
        self.x=event.pos().x()
        self.y=event.pos().y()            
        self.update()

    def mouseMoveEvent (self, event):
        print 'mouse moving'
        self.x=event.pos().x()
        self.y=event.pos().y()            
        self.update()        

class MainWindow(QMainWindow):
    def __init__(self):
        super(MainWindow, self).__init__()

        self.scene = QGraphicsScene()
        self.scene.setSceneRect(0, 0, 800, 600)

        pixmap=self.openImage()        
        self.imagePanel = ImageDrawPanel(scene = self.scene)
        self.imagePanel.setPixmap(pixmap)
        self.scene.addItem(self.imagePanel)

        self.view = QGraphicsView(self.scene)

        layout = QHBoxLayout()        
        layout.addWidget(self.view)

        self.widget = QWidget()
        self.widget.setLayout(layout)

        self.setCentralWidget(self.widget)
        self.setWindowTitle("Image Draw")

    def openImage(self):
        fname = QFileDialog.getOpenFileName(self, "Open image", ".", "Image Files (*.bmp *.jpg *.png *.xpm)")
        if fname.isEmpty(): return None
        return QPixmap(fname)        

import sys
if __name__ == "__main__":    
    app = QApplication(sys.argv)
    mainWindow = MainWindow()
    mainWindow.show()
    sys.exit(app.exec_())

What should I do now, to permanently draw on the image? I can store all the points and re-draw them in paint, but this does not seem efficient. Should I do the drawing in QGraphicsScene, rather than in the QGraphicsPixmapItem itself?

The second problem is, after drawing on the image, how can I get the selected region mask? Something like, creating a new image with an alpha channel, and then extracting the pixel values? Or, paint on an empty image in parallel? Then, I should also keep track of zoom in/out..

Answer 1

You have a number of different options that I'll order from higher level to lower level:

1. Use QGraphicsScene, QGraphicsView, and QGraphicsItems. This sets probably forms the main option and best option for graphics intensive operations. By setting a QGLWidget as the viewport you'll even have hardware acceleration on many systems.
2. Use QScrollArea to support zooming in on your image, which can be a simple QLabel. By changing the viewed region you'll have effectual zoom. QLabel could be used to draw the image but you'll have to manually keep track of selected regions and do any selection overlays.
3. Use a single QWidget and do custom painting. By calling update() after various events you'll be able to draw any necessary changes. Expect to do virtually everything manually.

I'd recommend approach number 1. You can use a QGraphicsPixmapItem to hold your image. You can then create a graphics item that represents your selection and use its bounding rectangle to find the intersecting areas. QGraphicsView can handle all the zooming for you.