jsctypes - problems using SHChangeNotifyRegister for MEDIA/DRIVE events

Question

I'm trying to use js-ctypes in Firefox to receive USB media/drive notifications, but I'm having a few issues and I can't tell if it's because I'm very inexperienced at Win32 API or awful at js-ctypes (or both!)

I've started by adapting an example I found on Alexandre Poirot's blog:

• Blog Entry
• Full JS Source

That example uses js-ctypes to create a "message-only" window, and then interacts with the shell service for the purpose of communicating with the Windows notification tray.

It seems simple enough, so after some research on the merits of RegisterDeviceNotification vs SHChangeNotifyRegister, I'm trying to adapt that (working!) example to register for device updates via SHChangeNotifyRegister.

The code resides in a bootstrapped (restartless) Firefox extension (code below).

The implementation of the WindowProc works well, as in the original example. My JavaScript callback logs the Window messages that come in (just numerically for this example).

Problems:

Firstly, it seems that calling DestroyWindow crashes Firefox (almost always) on shutdown() of the extension. Is there some Windows message I should handle on the "message-only" window to gracefully handle DestryWindow ?

Secondly, although it looks from the console output (below) that I'm getting meaningful values out of the calls to SHGetSpecialFolderLocation and SHChangeNotifyRegister (the return values aren't errors and the PIDLISTITEM pointer is some real address) I'm not getting Device/Drive messages in the JavaScript callback.

Also, I tried to reproduce the PIDLISTITEM structures to no avail (couldn't get js-ctypes to recognise them in calls to SHChangeNotifyRegister) and after studying some other non C++ examples, it seems that most folks are just using long* instead -- I hope that's the source of my misunderstanding!

I've verified via similar C++ sample project from Microsoft that the messages themselves are received when the SHChangeNotifyRegistration succeeds and I generate USB media events (ny inserting & removing USB flash media).

Minimal code to reproduce the issues follows:

---

install.rdf:

<?xml version="1.0"?>
<RDF xmlns="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:em="http://www.mozilla.org/2004/em-rdf#">
  <Description about="urn:mozilla:install-manifest">
  <em:id>[email protected]</em:id>
  <em:type>2</em:type>
  <em:name>TEST WNDPROC</em:name>
  <em:version>1.0</em:version>
  <em:bootstrap>true</em:bootstrap>
  <em:unpack>true</em:unpack>
  <em:description>Testing wndProc via JS-CTYPES on WIN32.</em:description>
  <em:creator>David</em:creator>

  <!-- Firefox Desktop -->
  <em:targetApplication>
    <Description>
    <em:id>{ec8030f7-c20a-464f-9b0e-13a3a9e97384}</em:id>
    <em:minVersion>4.0.*</em:minVersion>
    <em:maxVersion>29.0.*</em:maxVersion>
    </Description>
  </em:targetApplication>
  </Description>
</RDF>

bootstrap.js:

const Cc = Components.classes;
const Ci = Components.interfaces;
const Cu = Components.utils;

Components.utils.import("resource://gre/modules/ctypes.jsm");
let consoleService = Cc["@mozilla.org/consoleservice;1"]
                       .getService(Ci.nsIConsoleService);  
function LOG(msg) { 
    consoleService.logStringMessage("TEST-WNDPROC: "+msg); 
} 

var WindowProcType, DefWindowProc, RegisterClass, CreateWindowEx, 
    DestroyWindow, SHGetSpecialFolderLocation, WNDCLASS, wndclass, 
    messageWin, libs = {};

var windowProcJSCallback = function(hWnd, uMsg, wParam, lParam) {
  LOG("windowProc: "+JSON.stringify([uMsg, wParam, lParam]));
  //
  // TODO: decode uMsg, wParam, lParam to interpret 
  //       the incoming ShChangeNotifyEntry messages!
  //
  return DefWindowProc(hWnd, uMsg, wParam, lParam);
};

function startup(data, reason) {
  try {
    LOG("loading USER32.DLL ...");
    libs.user32 = ctypes.open("user32.dll");

    LOG("loading SHELL32.DLL ...");
    libs.shell32 = ctypes.open("shell32.dll");

    LOG("registering callback ctype WindowProc ...");
    WindowProc = ctypes.FunctionType(
        ctypes.stdcall_abi, ctypes.int, 
        [ctypes.voidptr_t, ctypes.int32_t, 
         ctypes.int32_t, ctypes.int32_t]).ptr;

    LOG("registering API CreateWindowEx ...");
    CreateWindowEx = libs.user32.declare("CreateWindowExA", 
        ctypes.winapi_abi, ctypes.voidptr_t, ctypes.long, 
        ctypes.char.ptr, ctypes.char.ptr, ctypes.int,
        ctypes.int, ctypes.int, ctypes.int, ctypes.int, 
        ctypes.voidptr_t, ctypes.voidptr_t, ctypes.voidptr_t, 
        ctypes.voidptr_t);

    LOG("registering API DestroyWindow ...");
    DestroyWindow = libs.user32.declare("DestroyWindow", 
        ctypes.winapi_abi, ctypes.bool, ctypes.voidptr_t);

    /*

    // previously using....

    LOG("registering ctype SHITEMID ...");
    var ShItemId = ctypes.StructType("ShItemId", [
      { cb: ctypes.unsigned_short },
      { abID: ctypes.uint8_t.array(1) }
    ]);

    LOG("registering ctype ITEMIDLIST ...");
    var ItemIDList = ctypes.StructType("ItemIDList", [
      { mkid: ShItemId }
    ]);

    */

    LOG("registering ctype SHChangeNotifyEntry ...");
    var SHChangeNotifyEntry = ctypes.StructType(
        "SHChangeNotifyEntry", [
            { pidl: ctypes.long.ptr   }, /* ItemIDList.ptr ??? */
            { fRecursive: ctypes.bool }
        ]);

    LOG("registering API SHChangeNotifyRegister ...");
    SHChangeNotifyRegister = libs.shell32.declare(
      "SHChangeNotifyRegister", ctypes.winapi_abi, 
      ctypes.unsigned_long, 
      ctypes.voidptr_t, ctypes.int, ctypes.long, 
      ctypes.unsigned_int,  ctypes.int, 
      SHChangeNotifyEntry.array() /* SHChangeNotifyEntry.ptr ??? */
    );

    LOG("registering ctype WNDCLASS ...");
    WNDCLASS = ctypes.StructType("WNDCLASS", [
      { style          : ctypes.uint32_t  },
      { lpfnWndProc    : WindowProc       }, 
      { cbClsExtra     : ctypes.int32_t   },
      { cbWndExtra     : ctypes.int32_t   },
      { hInstance      : ctypes.voidptr_t },
      { hIcon          : ctypes.voidptr_t },
      { hCursor        : ctypes.voidptr_t },
      { hbrBackground  : ctypes.voidptr_t },
      { lpszMenuName   : ctypes.char.ptr  },
      { lpszClassName  : ctypes.char.ptr  }
    ]);

    LOG("registering API SHGetSpecialFolderLocation ...");
    SHGetSpecialFolderLocation = libs.shell32.declare(
      "SHGetSpecialFolderLocation", ctypes.winapi_abi, 
      ctypes.long, ctypes.voidptr_t, ctypes.int, 
      ctypes.long.ptr        /* ItemIDList.ptr ??? */
    );

    LOG("registering API RegisterClass ...");
    RegisterClass = libs.user32.declare("RegisterClassA", 
        ctypes.winapi_abi, ctypes.voidptr_t, WNDCLASS.ptr);

    LOG("registering API DefWindowProc ...");
    DefWindowProc = libs.user32.declare("DefWindowProcA", 
        ctypes.winapi_abi, ctypes.int, ctypes.voidptr_t, 
        ctypes.int32_t, ctypes.int32_t, ctypes.int32_t);

    LOG("instatiating WNDCLASS (using windowProcJSCallback) ...");
    var cName = "class-testingmessageonlywindow";
    wndclass = WNDCLASS();
    wndclass.lpszClassName = ctypes.char.array()(cName);
    wndclass.lpfnWndProc = WindowProc(windowProcJSCallback);

    LOG("calling API: RegisterClass ...");
    RegisterClass(wndclass.address());

    LOG("calling API: CreateWindowEx ...");
    var HWND_MESSAGE = -3; // message-only window
    messageWin = CreateWindowEx(
      0, wndclass.lpszClassName,
      ctypes.char.array()("my-testing-window"),
      0, 0, 0, 0, 0, 
      ctypes.voidptr_t(HWND_MESSAGE), 
      null, null, null
    );

    LOG("instantiating pidl ...");
    var pidl = ctypes.long();
    LOG("Prior to call, pidl = "+pidl);

    LOG("calling API: SHGetSpecialFolderLocation ...");
    var CSIDL_DESKTOP = 0;
    var hr = SHGetSpecialFolderLocation(
        messageWin, 
        CSIDL_DESKTOP, 
        pidl.address()
    );
    LOG("got back: "+hr);
    LOG("After the call, pidl = "+pidl);

    LOG("instantiating pschcne ...");
    var SHCNE = SHChangeNotifyEntry.array(1);
    var shcne = SHCNE();
    shcne[0].pidl = pidl.address();
    shcne[0].fRecursive = false;

    var WM_SHNOTIFY           = 1025;    // 0x401
    var SHCNE_DISKEVENTS      = 145439;  // 0x2381F
    var SHCNE_DRIVEADD        = 256;     // 256
    var SHCNE_DRIVEREMOVED    = 128;     // 128
    var SHCNE_MEDIAINSERTED   = 32;      // 32
    var SHCNE_MEDIAREMOVED    = 64;      // 64
    var SHCNRF_ShellLevel     = 2;       // 0x0002
    var SHCNRF_InterruptLevel = 1;       // 0x0001
    var SHCNRF_NewDelivery    = 32768;   // 0x8000

    var nSources = SHCNRF_ShellLevel | 
                   SHCNRF_InterruptLevel | 
                   SHCNRF_NewDelivery; 
    var lEvents  = SHCNE_DISKEVENTS | SHCNE_DRIVEADD | 
                   SHCNE_DRIVEREMOVED | SHCNE_MEDIAINSERTED | 
                   SHCNE_MEDIAREMOVED;
    var uMsg     = WM_SHNOTIFY;

    LOG("DEBUG: nSources="+nSources);
    LOG("DEBUG: lEvents="+lEvents);
    LOG("DEBUG: uMsg="+uMsg);

    LOG("calling API: SHChangeNotifyRegister ...");
    var reg_id = SHChangeNotifyRegister(
        messageWin, nSources, lEvents, uMsg, 1, shcne
    );
    if (reg_id > 0) {
      LOG("SUCCESS: Registered with ShellService for "+
          "DRIVE/MEDIA notifications! reg-id: "+reg_id);
    } else {
      LOG("ERROR: Couldn't register for DRIVE/MEDIA "+
          "notifications from ShellService!");
    }       

    LOG("done!");
  } catch (e) {
    LOG("ERROR: "+e);
  }
}

function shutdown(data, reason) {
  if (reason == APP_SHUTDOWN) return;
  try {

    //LOG("destroying hidden window... ");
    //DestroyWindow(messageWin);  // crash!!!

    LOG("unloading USER32.DLL ...");
    libs.user32.close();

    LOG("unloading SHELL32.DLL ...");
    libs.shell32.close();

    LOG("done!");
  } catch (e) {
    LOG("ERROR: "+e);
  }
}

---

Console output:

17:08:25.518 TEST-WNDPROC: loading USER32.DLL ...
17:08:25.518 TEST-WNDPROC: loading SHELL32.DLL ...
17:08:25.518 TEST-WNDPROC: registering callback ctype WindowProc ...
17:08:25.518 TEST-WNDPROC: registering API CreateWindowEx ...
17:08:25.518 TEST-WNDPROC: registering API DestroyWindow ...
17:08:25.518 TEST-WNDPROC: registering ctype SHChangeNotifyEntry ...
17:08:25.518 TEST-WNDPROC: registering API SHChangeNotifyRegister ...
17:08:25.518 TEST-WNDPROC: registering ctype WNDCLASS ...
17:08:25.518 TEST-WNDPROC: registering API SHGetSpecialFolderLocation ...
17:08:25.518 TEST-WNDPROC: registering API RegisterClass ...
17:08:25.518 TEST-WNDPROC: registering API DefWindowProc ...
17:08:25.519 TEST-WNDPROC: instatiating WNDCLASS (using windowProcJSCallback) ...
17:08:25.519 TEST-WNDPROC: calling API: RegisterClass ...
17:08:25.519 TEST-WNDPROC: calling API: CreateWindowEx ...
17:08:25.519 TEST-WNDPROC: windowProc: [36,0,2973696]
17:08:25.519 TEST-WNDPROC: windowProc: [129,0,2973652]
17:08:25.519 TEST-WNDPROC: windowProc: [131,0,2973728]
17:08:25.519 TEST-WNDPROC: windowProc: [1,0,2973608]
17:08:25.519 TEST-WNDPROC: instantiating pidl ...
17:08:25.519 TEST-WNDPROC: Prior to call, pidl = ctypes.long(ctypes.Int64("0"))
17:08:25.519 TEST-WNDPROC: calling API: SHGetSpecialFolderLocation ...
17:08:25.519 TEST-WNDPROC: got back: 0
17:08:25.519 TEST-WNDPROC: After the call, pidl = ctypes.long(ctypes.Int64("224974424"))
17:08:25.519 TEST-WNDPROC: instantiating pschcne ...
17:08:25.519 TEST-WNDPROC: DEBUG: [nSources=32771][lEvents=145919][uMsg=1025]
17:08:25.519 TEST-WNDPROC: calling API: SHChangeNotifyRegister ...
17:08:25.520 TEST-WNDPROC: SUCCESS: Registered with ShellService for DRIVE/MEDIA
                           notifications! reg-id: 15
17:08:25.520 TEST-WNDPROC: done!
----- &< -------
17:09:31.391 TEST-WNDPROC: unloading USER32.DLL ...
17:09:31.391 TEST-WNDPROC: unloading SHELL32.DLL ...
17:09:31.391 TEST-WNDPROC: done!

Answer 1

For anyone else looking to do this, I'm posting a nasty hack of a work-around. (I won't accept this answer, in the hope that eventually someone will post how to do it properly).

---

After extensive reading, the only other recommended way I could of enumerating and/or determining the status of USB volumes was using WMI. The following WQL query did the trick:

select Caption, Size from win32_LogicalDisk where DriveType = 2

To use WQL from C++, you have to use COM. Using that from js-ctypes is not insignificant engineering task. You need to arrange for the DLL to be loaded and used from a ChromeWorker and sometimes I found that I specifically needed to make sure that JavaScript callback functions were being called from the correct Firefox thread, and that COM is not initialised in a multithreaded apartment.

Caption is the drive letter. It would seem that once a USB drive is ejected Size reports as zero.

It was then reasonably simple to call this in a polling loop inside the ChromeWorker thread, model the changes to the mounted volumes, and raise synthetic USB-Mounted/Ejected/Removed events in my DOM windows.

---

Unfortunately, there was one huge problem with this. If you insert a USB flash drive it generally takes between 2 and 30 seconds (depending on size) to be mounted by Windows. During that time (particularly after the 1st second or so), if you run the above WQL query, it will BLOCK THE USB VOLUME FROM BEING MOUNTED BY THE OPERATING SYSTEM (?!?) Effectively causing a denial of service.

However much incredulity this caused me, after enquiring I was assured that if I used asynchronous (rather than synchronous or semisynchronous) WQL queries, that the denial of service would not occur.

SELECT * FROM __instanceoperationevent WITHIN 2 
WHERE TargetInstance ISA 'Win32_LogicalDisk' and TargetInstance.DriveType = 2

If the __instanceoperationevent ISA __InstanceCreationEvent then the volume was added. If the __instanceoperationevent ISA __InstanceDeletionEvent then the volume was removed.

It would seem that when __instanceoperationevent ISA __InstanceModificationEvent then the volume was ejected, but it's not clear to me what other kind of operation could cause this. As the volume is still connected at this point, it's probably safe to deterministically query its Size using the first synchronous query (above) to check.

asynchronous WQL queries seem to callable in two different ways, as either temporary or permanent WMI event consumers. The difference isn't huge, but permanent filters+consumers seem to be recommended and don't seem to run afoul of WQL query "quotas".

Either way, there's no sane way to handle the resulting WMI events using JavaScript callbacks passed in via js-ctypes. :-( That left looking for a way to consume the events and then communicate them back to Firefox.

I ended up using a Strawberry perl script based on DBD::WMI per @Corion's answer to a perlmonks question to poll asynchronously for events every 2 seconds and then used IO::Socket::INET report the results to Firefox by sending them over a TCP socket. (You could do this in any language at all - I happen to be comfortable with Perl).

I then implemented nsIServerSocket from within my addon, waiting for \n terminated lines to parse the collected input and do the same modelling and synthetic events as described above.