Screen scraping an application window and interacting with the mouse and keyboard

Question

The other day I found myself addicted to a Flash game and frustrated by the thing at the same time. In a moment of frustration with the game I thought I would make a 'bot' to beat it for me. Well, I really wouldn't, but it made me realize: I don't know how to interact with another application in a way to do this. Which brings me to the question, how would one take screenshots of another running application and interact with it with the keyboard and mouse. Ideally the solution would be in a managed language like c#.

When doing the background reading the net was drowning with articles on scraping HTML. There were not many articles on actually screen scraping an application.

Diverse answers are appreciated as I’m really looking at surveying what’s out there.

UPDATE

I'm looking for a way to interface with another application rather than script/macro another application.

UPDATE

Could something like Xming be used to redirect the interface? http://www.straightrunning.com/XmingNotes/

Perhaps a Terminal Services client? http://www.codeproject.com/KB/cs/RemoteDesktop_CSharpNET.aspx

Answer 1

Check out Sikuli, it is basically what you are looking for. It is written in Java however.

http://groups.csail.mit.edu/uid/sikuli/

Answer 2

I ended up making the bot which did all this and documented it in a post

http://www.charlesrcook.com/archive/2010/09/05/creating-a-bejeweled-blitz-bot-in-c.aspx