Line Detection on a Musical Sheet Image in Android

Question

I am working on Android Project about Optical Music Recognition. I take a photo of a sheet music via mobile camera and I try to recognize the notes and symbols. I am very familiar with methodology after preprocessing but these kind of work is usually done with a help of scanner. (instead I have to use a mobile camera)

Intuitively, I used hough transform first to detect staves(staff lines), then calculate the angle between them and horizontal axis and rotate the image as much as the angle. This could work theoratically but hough finds line segments and the angle of these segments are not consistent. So it did not work.(maybe because of the effect of perspective)

After that, I tried many edge detection and line finding algorithms but none of them worked stable. (including frequency domain rotation and line segment detectors)

Now I am thinking of a new approach similar as the document scanners on Android. I try to convert my image into a form like scanned document. I want to align my image with horizontal axis and get rid of the effect because of tilting and perspective.

I wonder if anybody has any idea about how to achieve this "document scanning". Or if you have any idea about aligning the document or stable line detection it would be more than helpful for my future work. Thank you.

Answer 1

You could simply measure the height of the top or bottom stave at the left and the right of the sheet, and calculate the angle from that. The only difficulty may be finding a good point to measure the height automatically (i.e. find which column to scan vertically for the stave). One way to make it robust would be to check for 6 or however many there are consecutive black pixels separated by white zones of near constant width. When you find that, you are certain there was no stray note or symbol throwing off the measurement.

Answer 2

You should first remove the camera distortions:

http://en.wikipedia.org/wiki/Distortion_(optics)#Software_correction

Next, following openOMR's way of implementation, you can use the following piece of code to perform FFT in order to find the angle the whole image is skewed at:

http://www.koders.com/java/fid38ED5C64ADB6C4F5F01FB05FB08744AF159A8244.aspx?s=FFT#L23