OpenCV+python: HoughLines accumulator access since 3.4.2

Question

In OpenCV 3.4.2 the option to return the number of votes (accumulator value) for each line returned by HoughLines() was added. In python this seems to be supported as well as read in the python docstring of my OpenCV installation:

"Each line is represented by a 2 or 3 element vector (ρ, θ) or (ρ, θ, votes) ."

It is also included in the docs (with some broken formatting). However I can find no way to return the 3 element option (ρ, θ, votes) in python. Here is code demonstrating the problem:

import numpy as np
import cv2
print('OpenCV should be at least 3.4.2 to test: ', cv2.__version__)
image = np.eye(10, dtype='uint8')
lines = cv2.HoughLines(image, 1, np.pi/180, 5)
print('(number of lines, 1, output vector dimension): ', lines.shape)
print(lines)

outputs

OpenCV should be at least 3.4.2 to test:  3.4.2
(number of lines, 1, output vector dimension):  (3, 1, 2)
[[[ 0.         2.3212879]]

 [[ 1.         2.2340214]]

 [[-1.         2.4609141]]]

The desired behavior is an extra column with the amount of votes each line received. With the vote values more advanced options than the standard thresholding can be applied, as such it has been often requested and asked about on SE (here, here, here and here) with sometimes the equivalent for HoughCircles(). However both the questions and answers (such as modifying source and recompiling) are from before it was added officially, and therefore do not apply to the current situation.

Answer 1

As of vanilla OpenCV 3.4.3, you can't use this functionality from Python.

How it Works in C++

First of all in the implementation of HoughLines, we can see code that selects the type of the output array lines:

int type = CV_32FC2;
if (lines.fixedType())
{
    type = lines.type();
    CV_CheckType(type, type == CV_32FC2 || type == CV_32FC3, "Wrong type of output lines");
}

We can then see this parameter used in implementation of HoughLinesStandard when populating lines:

if (type == CV_32FC2)
{
    _lines.at<Vec2f>(i) = Vec2f(line.rho, line.angle);
}
else
{
    CV_DbgAssert(type == CV_32FC3);
    _lines.at<Vec3f>(i) = Vec3f(line.rho, line.angle, (float)accum[idx]);
}

Similar code can be seen in HoughLinesSDiv.

Based on this, we need to pass in an _OutputArray that is fixed type, and stores 32bit floats in 3 channels. How to make a fixed type (but not fixed size, since the algorithm needs to be able to resize it) _OutputArray? Let's look at the implementation again:

• A generic cv::Mat is not fixed type, neither is cv::UMat
• One option is std::vector<cv::Vec3f>
• Another option is cv::Mat3f (that's a cv::Matx<_Tp, m, n>)

Sample Code:

#include <opencv2/opencv.hpp>

int main()
{
    cv::Mat image(cv::Mat::eye(10, 10, CV_8UC1) * 255);

    cv::Mat2f lines2;
    cv::HoughLines(image, lines2, 1, CV_PI / 180, 4); // runs the actual detection
    std::cout << lines2 << "\n";

    cv::Mat3f lines3;;
    cv::HoughLines(image, lines3, 1, CV_PI / 180, 4); // runs the actual detection
    std::cout << lines3 << "\n";

    return 0;
}

Console Output:

[0, 2.3212879;
 1, 2.2340214;
 -1, 2.4609141]
[0, 2.3212879, 10;
 1, 2.2340214, 6;
 -1, 2.4609141, 6]

---

How the Python Wrapper Works

Let's look at the autogenerated code wrapping the HoughLines function:

static PyObject* pyopencv_cv_HoughLines(PyObject* , PyObject* args, PyObject* kw)
{
    using namespace cv;

    {
    PyObject* pyobj_image = NULL;
    Mat image;
    PyObject* pyobj_lines = NULL;
    Mat lines;
    double rho=0;
    double theta=0;
    int threshold=0;
    double srn=0;
    double stn=0;
    double min_theta=0;
    double max_theta=CV_PI;

    const char* keywords[] = { "image", "rho", "theta", "threshold", "lines", "srn", "stn", "min_theta", "max_theta", NULL };
    if( PyArg_ParseTupleAndKeywords(args, kw, "Oddi|Odddd:HoughLines", (char**)keywords, &pyobj_image, &rho, &theta, &threshold, &pyobj_lines, &srn, &stn, &min_theta, &max_theta) &&
        pyopencv_to(pyobj_image, image, ArgInfo("image", 0)) &&
        pyopencv_to(pyobj_lines, lines, ArgInfo("lines", 1)) )
    {
        ERRWRAP2(cv::HoughLines(image, lines, rho, theta, threshold, srn, stn, min_theta, max_theta));
        return pyopencv_from(lines);
    }
    }
    PyErr_Clear();

    // Similar snippet handling UMat...

    return NULL;
}

To summarize this, it tries to convert the object passed in the lines parameter to a cv::Mat, and then it calls cv::HoughLines with the cv::Mat as the output parameter. (If this fails, then it tries the same thing with cv::UMat) Unfortunately, this means that there is no way to give cv::HoughLines a fixed type lines, so as of 3.4.3 this functionality is inaccessible from Python.

---

Solutions

The only solutions, as far as I can see, involve modifying the OpenCV source code, and rebuilding.

Quick Hack

This is trivial, edit the implementation of cv::HoughLines and change the default type to be CV_32FC3:

int type = CV_32FC3;

However this means that you will always get the votes (which also means that the OpenCL optimization, if present, won't get used).

Better Patch

Add an optional boolean parameter return_votes with default value false. Modify the code such that when return_votes is true, the type is forced to CV_32FC3.

Header:

CV_EXPORTS_W void HoughLines( InputArray image, OutputArray lines,
                              double rho, double theta, int threshold,
                              double srn = 0, double stn = 0,
                              double min_theta = 0, double max_theta = CV_PI,
                              bool return_votes = false );

Implementation:

void HoughLines( InputArray _image, OutputArray lines,
                 double rho, double theta, int threshold,
                 double srn, double stn, double min_theta, double max_theta,
                 bool return_votes )
{
    CV_INSTRUMENT_REGION()

    int type = CV_32FC2;
    if (return_votes)
    {
         type = CV_32FC3;
    }
    else if (lines.fixedType())
    {
        type = lines.type();
        CV_CheckType(type, type == CV_32FC2 || type == CV_32FC3, "Wrong type of output lines");
    }
    // the rest...