recognition of boggle/scrabble letters from an image [closed]

Question

I am interested in recognizing letters on a Boggle board, probably using openCV. The letters are all the same font but could be rotated, so using a standard text recognition library is a bit of a problem. Additionally the M and W have underscores to differentiate them, and the Q is actually a Qu. I am fairly confident I can isolate the seperate letters in the image, I am just wondering how to do the recognition part.

Answer 1

It depends on how fast you need to be. If you can isolate the square of the letter and rotate it so that the sides of the square containing the letter are horizontal and vertical then I would suggest you:

• convert the images to black/white (with the letter the one colour and the rest of the die the other
• make a dataset of reference images of all letters in all four possible orientations (i.e. upright and rotated 90, 180 and 270 degrees)
• use a template matching function such as cvMatchTemplate to find the best matching image from your dataset for each new image.

This will take a bit of time, so optimisations are possible, but I think it will get you a reasonable result. If getting them in a proper orientation is difficult you could also generate rotated versions of your new input on the fly and match those to your reference dataset.

If the letters have different scale then I can think of two options:

• If orientation is not an issue (i.e. your boggle block detection can also put the block in the proper orientation) then you can use the boundingbox of the area that has the letter colour as rough indicator of the scale of the incoming picture, and scale that to be the same size as the boundingbox on your reference images (this might be different for each reference image)
• If orientation is an issue then just add scaling as a parameter of your search space. So you search all rotations (0-360 degrees) and all reasonable sizes (you should probably be able to guess a reasonable range from the images you have).

Answer 2

You can use a simple OCR like Tesseract. It is simple to use and is quite fast. You'll have to do the 4 rotations though (as mentioned in @jilles de wit's answer).

Answer 3

I made an iOS-app that does just this, based on OpenCV. It's called SnapSolve. I wrote a blog about how the detection works. Basically, I overlay all 26x4 possible letters + rotations on each shape, and see which letter overlaps most. A little tweak to this is to smooth the overlay image, to get rid of artefacts where letters almost overlap but not quite.