How to create a specialized machine vision OCR solution?

Question

We need to read some text from photos of sales receipts taken by iPad camera. Here is a sample similar to what we need to read from:

There are a few constraints to this problem:

1. We need to read the total amount which always appears after a text marker (such as Grand Total in this example).
2. The font is always the same.
3. The app must work offline without network connectivity.

This is what we have tried so far:

• Google Mobile Vision text extraction worked like magic. But text extraction is available in Android only. And we need to build the solution in iOS.
• Google and Microsoft have cloud based machine vision solutions which also work very accurately. But our app needs to work offline.
• Use tesseract OCR. It performed very poorly. No doubt because we have a photo instead of scanned black & white image.

We are now thinking of creating a custom solution using convolutional NN. The question I have is how can we build a model that takes advantage of these two constraints to create a simpler and yet very accurate solution?

1. The total amount always appears after a text marker. We can safely ignore the rest of the text.
2. The text is always in English and in the same font.

This is the general pipeline we have come up with so far.

1. Straighten the image and scale it to a standard size.
2. Doing conv net to locate the text marker (Grad Total) should be fairly easy. We can completely skip the top half of the image.

We are not sure what else to do at this point. Any tips, advice and help will be great.

PS. I realize this is a question about design methodology and not a specific programming question. I apologize if this violates SO guidelines.

Answer 1

I propose for you to consider deeplearning4j.org solution. You can train their network on powerful machine and then save state of network and use it at android. Here they explained how to use their network at android app with help of java.