Font Recognition From free Hand drawing

Question

I have been working on an application that involves font recognition based on a users free hand drawing characters in Android Canvas.

In this application the user is asked to enter some predefined characters in a predefined order (A,a,B,c). Based on this, is there any way to show the very similar font which matches the user's hand writing.

I have researched on this topic found some papers & articles but most of them are recognizing font from a captured image. In that case they are having a lot of problems by segmenting paragraphs, individual letters and so on. But in my scenario I know what letter the user is drawing.

I have some knowledge in OpenCV and Machine Learning. Need help on how to proceed with this problem.

Answer 1

It is not exactly clear to me what you want to accomplish with your application but I assume that you are trying to output a font from a database of fonts that matches a users handwriting the most.

In Machine Learning this would be a classification problem. The number of classes will by equal to the number of different fonts in your database.

You could solve this with the help of a Convolutional neural network which are widely used for image and video recognition related tasks. If you've never implemented a CNN before I would suggest that you look up this resources to learn about Torch which is a easy-to-start-with toolkit to implement CNN's. (Of course there are more Frameworks such as: Tensor Flow, Caffe, Lasagne, ...)

• Torch Homepage
• Deep learning with Torch: 60 minutes blitz
• Torch Cheatsheet

The main obstacle you will face is that Neural Networks need thousands of images (>100.000) to properly train them and to achieve satisfying results. Furthermore you do not only need the images but also a correct label for each image. Will say, you would need a training image such as a handwritten character and the corresponding font it matches the most out of your database as its label.

I would suggest that you read about so called transfer learning which can give you an initial boost as you do not need to set up a CNN model completely by yourself. In addition people have pre-trained such a model for a related task so that you safe extra time as you would not need to train it for many hours on a GPU. (see CUDA)

A great resource to start with is the paper: How transferable are features in deep neural networks?, which could be helpful for the stated reasons.

To get tons of training and testing data you can look up the following open datasets that provide all types of characters that can be helpful for your task:

• Artificial Characters Data Set
• UJI Pen Characters Data Set
• The Chars74K dataset
• Hand written - Datasets
• A New Benchmark Dataset for Handwritten Character Recognition

For access to a lot of fonts and maybe even the possibility to create further datasets on your own you can have a look at Google Fonts.