"Swing-Shell" java.lang.InternalError: Could not initialize COM: HRESULT=0x80010106

Question

I have a Java 9 app that I'm trying to package for the Windows Store. The strange thing is that it works as expected when I run the exe-launcher directly, but I get the following strange error when I run the launcher via the APPX package:

Exception in thread "Swing-Shell" java.lang.InternalError: Could not 
initialize COM: HRESULT=0x80010106 
at java.desktop/sun.awt.shell.Win32ShellFolderManager2.initializeCom(Native Method) 
at java.desktop/sun.awt.shell.Win32ShellFolderManager2$ComInvoker$1.run(Unknown Source at java.base/java.lang.Thread.run(Unknown Source)

HRESULT=0x80010106 means RPC_E_CHANGED_MODE which I guess means that COM is somehow already initialized in MTA mode. But why is this only an issue in the Windows Bridge sandbox? Does the Windows Bridge somehow pre-initialize COM somehow for some reason?

I'm not sure if this is a Java 9 issue, or a Desktop Bridge issue, or both. Does anybody have any ideas on how to identify the cause of the issue or workaround?

I have made a minimal Sample Project to reproduce the issue

The application works when executed directly, but not when executed via the APPX launcher. Why?

Answer 1

Analysis

The JDK parts, that rely on COM being initialized (D3DPipeline, Sound and windows shellfolder access) all follow the same pattern:

• the run the code in a separate thread
• for this thread CoInitialize is called
• at the end of the user level code CoUnitialize is called

This is in line with the documentation the MSDN holds for COM and the analysis is correct, the error indicates, that the COM subsystem is already initialized to be MTA threaded.

So I modified the java launcher (jvm.dll) and inserted debugging statements into some of the native methods in os_windows.cpp. I focused on the threading methods. What I found was this:

• create_main_thread, create_os_thread, pd_start_thread all run with COM not yet initialized
• the native thread initializer (thread_native_entry) is already running with COM initialized

I looked more into in _beginthreadex and indeed I finally found a lead on stackoverflow. That pointed me to the sourcecode of threadex.c, which is part of the Visual Studio 2013 Express Installation.

There you find, that _beginthreadex does not directly start the supplied thread function, but runs a library initializer (_threadstartex) first. Part of this initializer reads:

_ptd->_initapartment = __crtIsPackagedApp();
if (_ptd->_initapartment)
{
    _ptd->_initapartment = _initMTAoncurrentthread();
}

_callthreadstartex();

_crtIsPackagedApp detects via a kernel function if the application is run as a "PackagedApp" (i.e. AppX package) and if so, then the RoInitialize function is called, which from my understanding acts like the big brother of CoInitialize.

Long story short: If your application is build with Visual Studio 2013 and run as a packaged application, you get a broken environment.

It was confirmed, that the working versions of the Oracle JDK were build with VS2010 SP1 and the broken version was build with VS2013SP4.

Workaround

With the above information google finally found a reference, that supports the analysis:

https://blogs.msdn.microsoft.com/vcblog/2016/07/07/using-visual-c-runtime-in-centennial-project/

Quote from that article:

Note that the existing VC++ 12.0 libraries created during the Windows 8 timeframe have runtime checks to determine whether the app is running under the app container or not. When running desktop apps as a packaged app, these checks might limit the functionality of the desktop app or cause it to behave like a UWA (Universal Windows Application) (limited file system access or create thread initializing MTA etc.). We have fixed this behavior in the VC++ libraries contained in these framework packages and thus removing the modern app limitations from your desktop applications.

So I see to options to fix applications, that shall be distributed as AppX packages:

• either force the users to have the updates VC++ 12.0 binaries installed (by introducing the dependency cited in the blog post) or
• replace the msvcr120.dll that is bundled with Java 9 (and I assume also Java 10) with the fixed version from the update packages

I would go with the second version and I tested this. Starting with a clean up-to-date Windows 10 System, I installed JDK 9.0.4, I cloned the supplied testcase, modified it use the locally installed JRE (not the JDK!) and build the appx package. Running this, I reproduced the problem. I then replaced the msvcr120.dll's in the JRE folder of my systems installation with the one contained in the APPX container from:

https://www.microsoft.com/en-us/download/details.aspx?id=53176

Hint: *.appx are just ZIP files with additional signatures, so you can just rename them and extract the contents.

I rebuild the testcase and it is working as it should (no COM initialization errors anymore).