C++: OpenCV: fast pixel iteration

Question

I'm trying to get BGR values from a streaming webcam image. I'm getting a memory access violation because I'm not using the pointer correctly in the nested for loop but I don't know what the syntax should be. I can't find documentation that is specific enough to the seemingly basic task I'm trying to do.

In addition to solving he memory access violation, I want to also be able to edit each pixel on the fly without having to do a deep copy but don't know what he syntax should be for that also.

This is the code I have so far:

int main(int argc, char** argv)
{

    int c;
    Mat img;
    VideoCapture capture(0);
    namedWindow("mainWin", CV_WINDOW_AUTOSIZE);
    bool readOk = true;

    while (capture.isOpened()) {

        readOk = capture.read(img);

        // make sure we grabbed the frame successfully 
        if (!readOk) {
            std::cout << "No frame" << std::endl;
            break;
        }

        int nChannels = img.channels();
        int nRows = img.rows;
        int nCols = img.cols * nChannels;

        if (img.isContinuous())
        {
            nCols *= nRows;
            nRows = 1;
        }

        int i, j;
        uchar r, g, b;
        for (i = 0; i < nRows; ++i)
        {
            for (j = 0; j < nCols; ++j)
            {
                r = img.ptr<uchar>(i)[nChannels*j + 2];
                g = img.ptr<uchar>(i)[nChannels*j + 1];
                b = img.ptr<uchar>(i)[nChannels*j + 0];
            }
        }

        if (!img.empty()) imshow("mainWin", img);
        c = waitKey(10);
        if (c == 27)
            break;
    }
}

Answer 1

Your scanning loop is not correct. You should be only getting a pointer to the row once per row. Since pixels are 3 byte quantities, it is easiest to treat them as a Vec3b.

You should have something like

    uchar r, g, b;
    for (int i = 0; i < img.rows; ++i)
    {
        cv::Vec3b* pixel = img.ptr<cv::Vec3b>(i); // point to first pixel in row
        for (int j = 0; j < img.cols; ++j)
        {
            r = pixel[j][2];
            g = pixel[j][1];
            b = pixel[j][0];
        }
    }

OR

    uchar r, g, b;
    for (int i = 0; i < img.rows; ++i)
    {
        uchar* pixel = img.ptr<uchar>(i);  // point to first color in row
        for (int j = 0; j < img.cols; ++j)
        {
            b = *pixel++;
            g = *pixel++;
            r = *pixel++;
        }
    }

---

NOTE

It is fairly common to see Mat::at() used to access pixels sequentially like:

    // DON'T DO THIS!
    uchar r, g, b;
    for (int i = 0; i < img.rows; ++i)
    {
        for (int j = 0; j < img.cols; ++j)
        {
            cv::Vec3b pixel = img.at<cv::Vec3b>(i, j); 
            r = pixel[2];
            g = pixel[1];
            b = pixel[0];
        }
    }

However such uses are inappropriate. For every pixel access, at() needs to calculate an index by multiplying the row number and row length - and over a whole image that calculation can result in processing times considerably slower than with the code above (where ptr() does an equivalent calculation once per row. Furthermore, in debug mode at() has an assertion that makes it much slower again.

If you are sure there is no padding between rows, it is possible to go faster by eliminating the call to ptr(). In this case the pixel pointer in the second loop above will after the end of each line be pointing at the start of the next line. But that wont work if your Mat is for example some region of interest of some other Mat.

On the other hand, if you were accessing pixels in a random fashion, rather than scanning sequentially like above, at() is then very appropriate.