Is it possible to know what text is displaying nearby the mouse?

Question

I am just planning to implement some features like "Look up in Dictionary" when the mouse hovers over some text/words on the screen on the Mac OS X platform.

How can I get the displaying text/words nearby the mouse on the screen, even if they're not in my own application.

What I can do is:

1. Using WorkSpace to know what applications are running.
2. Use accessibility API to know the top UIElement on the screen.
3. Use accessibility API to know the selected string of a UIElement if the application use something like NSTextView (I get get it by check the "selected text" attribute of the UIElement).

What I can't do is:

1. Some applications like Safari, using WebKit framework, the only things I can get from Accessibility API are "value attribute", which is the current content of the HTML, and some attributes named like "AXTextMarker". These attributes can't not be found in Google or any document.
2. Some applications even do not support Accessibility API, what I can get is just a screen shot of it.

Is there any way to call system API to recognize the text in the image? In Snow Leopard, there is a handwriting recognizer feature for input in Chinese or Japanese, so should it be some OCR feature? I can't find any open API for it.

Answer 1

What you can probably do is this:
Use a meta-key + a key chord to activate the dictionary once text is selected. I'd actually find it incredibly obnoxious if whenever I was near text for a bit, a dictionary popped up. Particularly because I read a tremendous number of white papers, where almost any dictionary imaginable would spend most of its time choking and dying.