OCR engines designed for screen-reading

Question

Are there any OCR engines designed for identifying text in screen-captured images rather than scanned text? I have a project where I need to retrieve and identify text in an application, and none of the OCR engines I've tried so far have faired well with screenshots.

Ideally the engine should work well with color and with background noise, although I can make some allowances if nothing like that is available.

It will need to be .NET compatible; either written in .NET or having a .NET-callable API.

Answer 1

I've found Tesseract OCR to be pretty solid for an open source project. I've found that it can even read and decode simple captchas, like Megaupload's. I'd think with a little tweaking this could work pretty well.

The only pain is that it only accepts uncompressed TIFF images, which can be annoying.

EDIT: Philip Daubmeier already found a .NET integration, but below is code to convert a Bitmap to uncompressed TIFF.

private void ConvertBitmapToTIF(Bitmap convert)
{
    ImageCodecInfo codecInfo = GetEncoderInfo("image/tiff");
    System.Drawing.Imaging.Encoder encodeCom = System.Drawing.Imaging.Encoder.Compression;
    System.Drawing.Imaging.Encoder encodeBPP = System.Drawing.Imaging.Encoder.ColorDepth;

    EncoderParameters parms = new EncoderParameters(2);
    EncoderParameter param0 = new EncoderParameter(encodeCom, (long)EncoderValue.CompressionNone);
    EncoderParameter param1 = new EncoderParameter(encodeBPP, 8L);
    parms.Param[0] = param0;
    parms.Param[1] = param1;

    convert.Save("output.tif", codecInfo, parms);
}

This saves to a file, but the Bitmap.Save method can write to a stream also.

Answer 2

Usually OCR technolgy is tuned to work with scanned text, which is at at least 200 dpi, however 300 dpi is recommended for reliable OCR quality. Thus you need to put some efforts into tweaking settings and everything to make it work on screen text, which is typically considered to be something near to 96 dpi.

ABBYY has screen shot OCR software: http://www.abbyy.com/screenshot_reader/ which proves that its technology is able to work in this conditions well. I use it, it just works. Thus you may want to contact ABBYY for OCR SDK: http://www.abbyy.com/ocr_sdk/ (can be used from .NET)

It is not cheap, but it works. Disclaimer: I work for ABBYY

Answer 3

You're essentially looking for the CAPTCHA circumvention tools various researchers have tried, some with success.

Another approach would be to use smoothing algorithms to interpolate 96 DPI captures and convert them to 300 DPI (eg, photoshop it), then use standard OCR tools.