Reading Character from Image

Question

I am working on an application which requires matching of numbers from a scanned image file to database entry and update the database with the match result.

Say I have image- employee1.jpg. This image will have two two handwritten entries - Employee number and the amount to be paid to the employee. I have to read the employee number from the image and query the database for the that number, update the employee with the amount to be paid as got from the image. Both the employee number and amount to be paid are written inside two boxes at a specified place on the image.

Is there any way to automate this. Basically I want a solution in .net using c#. I know this can be done using artificial neural networks.

Any ideas would be much appreciated.

Answer 1

You can use Microsoft Office Document Imaging Library (MODI), which is contained in the Office 2003/2007.

Links:

• OCR with Microsoft® Office - Code Project - example of using MODI
• Microsoft Office Document Imaging - Wikipedia - contains a simple example in VB.NET

Answer 2

Pattern recognition is a basic example when neural networks are studied. I don't know if is any library/framework to work with AI in C#. If you find one, first you have to do is to train the network (supervised learning) and for this you need to prepare a big sample set of images; more examples -> result more accurate. In the other hand you can use OpenCV (C/C++, Python and Java) that is a library specialized in computer vision and has a module to implement AI methods.

Have a nice day! Oscar.

Answer 3

I think this is very hard to automate. The problem is just because you need some kind of very good OCR software. And even if you got this, what if it reads something wrong, cause of the frouzy handwriting of someone? If the ID is wrong the paid is booked to the wrong employee and if the amount is wrong he got the wrong salary!

Both are things you won't really happen. Just to show you how hard a good ocr to find is, just take a look out there on how a captcha works. The principle is nothing more than an image of an hard to read text.

So my opinion would be, that you can't really automate this process. At least you can write a program to assist a human by entering the values manually (also take a look at Amazon Mechanical Turk):

• Show on the right the picture with the handwritten values, or if they are always on the same position or specially marked (with a box around them, etc.), try to find these places automatically and show them to the user.
• On the left offer two textboxes, where the user can enter the values.

To get this to a fast and fluent process, you have to take great care about how the user can enter easily the values by just using the keyboard:

• When showing a new picture, set the focus to the id textbox
• If the user id is always a specific length, switch to the next box if all numbers are entered
  • (If you allow this, a backspace in the empty next box should focus back to the previous one)
• Otherwise allow a change to the next textbox by hitting tab or return
• Normally these textboxes are arranged above each other (not side by side), thus you should support switching between them using the up down arrow keys.
• After finishing the entry in the last textbox automatically show the next image.
  • Also in this case of a fresh new entry (nothing already entered) allow a easy switch back to the old entry by using the backspace or left arrow key)

By using such a process a single person can enter many entries into your database and the costs are much cheaper than finding wrong entries in your database afterwards.

Just a last suggestion:
Cause this is a boring process for a human which can easily leads to errors, maybe let two people enter these values and only if both are entering the same than take this value as approved. This should lead to a correct rate somewhere above 99%. If you need absolutely 100% think about letting 4-5 people checking one entry and only if all of them enter the same values take it as approved. To get also a comparsion about how good your ocr software would be, just let it also run over your images and compare this results to the human entered values to get an idea, when you can really rely on your ocr only.