OpenCV - approxPolyDP for edge maps (not contours)

Question

I have successfully applied the method cv::approxPolyDP on contours (cv::findContours), in order to represent a contour with a simpler polygon and implicitly do some denoising.

I would like to do the same thing on an edge map acquired from an RGBD camera (which is in general very noisy), but with not much success up to now and I cannot find relative examples online. The reason I need this, is that by means of an edge map one can also use the edges between fingers, edges created by finger occlusion or edges created in the palm.

Is this method applicable to general edge maps, other than contours?

Could someone pinpoint me to an example?

Some images attached:

Successful example for contours:

Problematic case for edge maps:

Most probably I draw things in the wrong way, but drawing just the pixels returned by the method shows that probably large areas are not represented in the end result (and this doesn't change much according to the epsilon-parameter).

I attach also a depth image, similar to the ones I use in the experimental pipeline descibed above. This depth image was not acquired by a depth camera, but was synthetically generated by reading the depth buffer of the gpu, using OpenGL.

Just for reference, this is also the edge map of the depth image acquired straight from the depth camera (using the raw image, no smoothing etc applied)

(hand as viewd from a depth camera, palm facing upwards, fingers "closing" towards the palm)

Answer 1

Your issue with approxPolyDP is due to the formatting of the input into approxPolyDP.

Explanation

approxPolyDP expects its input to be a vector of Points. These points define a polygonal curve that will be processed by approxPolyDP. The curve could be open or closed, which can be controlled by a flag.

The ordering of the points in the list is important. Just as one traces out a polygon by hand, each subsequent point in the vector must be the next vertex of the polygon, clockwise or counter-clockwise.

If the list of points is stored in raster order (sorted by Y and then X), then the point[k] and point[k+1] do not necessarily belong to the same curve. This is the cause of the problem.

This issue is explained with illustrations in OpenCV - How to extract edges form result of Canny Function? . Quote from Mikhail: "Canny doesn't connect pixels into chains or segments."

---

Illustration of "raster order" that is generated by Canny.

---

Illustration of "contour order" that is expected by approxPolyDP

---

What is needed

What you need is a list of "chains of edge pixels". Each chain must contain edge pixels that are adjacent to each other, just like someone tracing out an object's outline by a pencil, without the tip of the pencil leaving the paper.

This is not what is returned from edge detection methods, such as Canny. Further processing is needed to convert an edge map into chains of adjacent (continuous) edge pixels.

Suggested solutions

(1) Use binary threshold instead of edge detection as the input to findContours

This would be applicable if there exists a threshold value that separates the hand from the background, and that this value works for the whole hand (not just part of the hand).

(2) Scan the edge map, and build the list of adjacent pixels by examining the neighbors of each edge pixel.

This is similar to the connected-components algorithm, except that instead of finding a blob (where you only need to know each pixel's membership), you try to find chains of pixels such that you can tell the previous and next edge pixels along the chain.

(3) Use an alternative edge detection algorithm, such as Edge Drawing.

Details can be found at http://ceng.anadolu.edu.tr/cv/EdgeDrawing/

Unfortunately, this is not provided out-of-the-box from OpenCV, so you may have to find an implementation elsewhere.

---

Sample code for option #1.

#include <stdint.h>
#include <iostream>
#include <vector>
#include <opencv2/opencv.hpp>

using namespace std;
using namespace cv;

int main()
{
    Mat matInput = imread("~/Data/mA9EE.png", false);

    // ---- Preprocessing of depth map. (Optional.) ----

    GaussianBlur(matInput, matInput, cv::Size(9, 9), 4.0);

    // ---- Here, we use cv::threshold instead of cv::Canny as explained above ----

    Mat matEdge;

    //Canny(matInput, matEdge, 0.1, 1.0);

    threshold(matInput, matEdge, 192.0, 255.0, THRESH_BINARY_INV);

    // ---- Use findContours to find chains of consecutive edge pixels ----

    vector<vector<Point> > contours;
    findContours(matEdge, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);

    // ---- Code below is only used for visualizing the result. ----

    Mat matContour(matEdge.size(), CV_8UC1);

    for (size_t k = 0; k < contours.size(); ++k)
    {
        const vector<Point>& contour = contours[k];
        for (size_t k2 = 0; k2 < contour.size(); ++k2)
        {
            const Point& p = contour[k2];
            matContour.at<uint8_t>(p) = 255;
        }
    }

    imwrite("~/Data/output.png", matContour);
    cout << "Done!" << endl;
    return 0;
}