Computer vision to calculate the digit (finger) ratio

Question

If someone scans their right hand pressed against the glass of a scanner, the result would look like this:

(without the orange and white annotations). How could we determine someone's 2D:4D ratio from an image of their hand?

Answer 1

You've already tagged this opencv which is great - I'd highly recommend taking a look at openFrameworks and the openCV addon, as the basic examples there will give you some great starting blocks for this.

The general approach to this I would take is to first distill the image to light and dark areas, detect the edges of the hand and fingers, and then simplify your data until you have lines representing the edges and tips of the fingers. Finally, take the lower inseam between 2nd and 3rd finger, stopping at the tip of the 2nd, and the inseam of the 3rd and 4th, stopping at the tip of the 4th, which should give you your 2D:4D ratio.

First, you'll need to process your images to get to black and white images openCV can easily handle. You may have to play with various thresholds to get both the outline of the hand and the inseams of the fingers to be detected. (You may even need two passes to detect both the outline and inseams)

While there are many approaches to feature detection, OpenCV will generally return arrays of "blobs" detected. With the right thresholds, I believe you would be able to reliably and simply find contiguous horizontal blobs (or nearly contiguous, allowing for some distance between nearby blobs) for the inseams of each finger.

A simple algorithm for detecting the inseams would be to walk through the detected blobs starting from the top left and proceeding left-to-right through the image, as if reading a page. Assemble an array of detected horizontal lines from the blobs in your image, and play with various image processing thresholds, minimum accepted line length, and distance allowances between detected blobs which you still consider part of the same line until you're satisfied you're detecting the finger edges well.

Once you have detected the horizontal lines, you can process the blobs again, looking for the vertical lines that represent the tips of the fingers (stopping when you hit the previously detected horizontal lines)

Finally, find the lines which represent the correct inseams, measure them until they intersect with the appropriate fingertips, and you should have your ratio!

Answer 2

Interesting question. I'd go about it this way:

First, binarize the image by Otsu's thresholding. Then find the skeleton of the image using a Medial-Axis Transform (MAT). This would mean doing a distance transform on the image, then using adaptive thresholding to get the local maxima in the distance transform. This gives a rough and ready skeleton of your image. Sample code from here.

The obtained hand-skeleton may be slightly disconnected, in which case use the OpenCV morphology "CLOSE" (not "open") function can connect it into a single skeleton. Then checking convexity defects of the resulting hand should give an estimate.