How can double-clicking a file in explorer be hooked?

Question

When a user double clicks a file to start it, I need to carry out a custom action if the file meets certain criteria.

In WindowsXP, detours could be used to hook ShellExecuteEx in explorer.exe, and the file name was accessible. I need to do something similar in Windows 7 and 8.

How can I execute some code when files are double clicked, and have the name of the file available?

I need a method that will continue to work if Windows Updates patch explorer.

Answer 1

A brief check using procmon shows that every time a file is activated by explorer (by either double clicking it or by pressing enter). Explorer searches for HKCR\*\ShellEx\DataHandler.

DataHandler is one of many types of windows shell extensions.

So while the documentation states that it's for drag&drop actions it seems that it always calls it when opening a file.

I've followed the How to Create Data Handlers tutorial on how to register one, just with one minor change: I've added the DataHandler under HKCR\*\ShellEx\DataHandler with a custom GUID and a corresponding CLSID entry and linked it to a dll that does nothing. And now it was loaded every time I've clicked or "entered" a file. I didn't go as far as implementing all the required interfaces but according to the documentation:

The Shell initializes the handler through its IPersistFile interface. It uses this interface to request the handler's class identifier (CLSID) and provides it with the file's name.

So theoretically it should be accessible to a dll that implements the required interface, and you could just access the file name on each access execute the custom action and do nothing else.

I don't currently have the time to write such a dll myself( maybe later) but it looked like a nice direction that you can explore.