My dll code works from an exe file, but fails to load from Java loadLibrary

Question

I have created a C++ module to build into a shared library file and then call it from Java using JNI.

I have 2 environments, Windows and Unix and I have a C++ executable program and a Java program that I just recompile for each environment.

• When I compile my tester.exe program in Unix and run it using methods from my library (.so) it works fine.
• When I compile my Java program in Unix and load my library (.so) with Java's loadLibrary, it works fine.

• When I compile my tester.exe program in Windows and run it using methods from my library (.dll) it works fine. Just like the unix version.

• When I compile my Java program in Windows and load my library (.dll) with Java's loadLibrary it fails. It says Attempt to access invalid address.

I cannot figure out why it will not work with the Java loadLibrary when running in Windows, but it works everywhere else using the same code. If I delay load a dependent DLL that my library uses, then my library loads in Java but is not functional. I know there is specific code that causes the issue with Java loading my library, but I can't figure out why my C++ exe has no problem with the same methods and libraries.

---

My dll has 1 exposed method which calls 4 methods from some existing libraries. If I comment those 4 methods out, then my dll loads in Java fine. I know it's something to do with these methods from a library my dll links to. Is there something different with how Java sees the dependent libraries? I've tried loading the dependent libraries first, but one of the dll files I load causes a recursion error and the stack overflows.

Anyone know a way around a DLL that causes a stack overflow from a recursion error? I need the methods in it, but I cannot load it with java loadLibrary.

---

Here is more detail about the files involved and the actual error message. I added a DllMain to my inital dll file just to see what loads and when. If I compile that same program (my_plain_dll_to_call_JNI_DLL) as an exe file, everything works fine. If I compile it and load it from my java program this happens.

• myJavaProgram, simply calls System.loadLibrary() to load a basic .dll file that calls a method in my other dll that contains JNI code.
• my_plain_dll_to_call_JNI_DLL is a dll I created by linking it to my dll library file just to test the dependency. It just calls a method from the other dll which is calling the native code I need.
• my_JNI_DLL.ll is a dll file linked with existing C++ programming libraries that I need to access from JNI. It contains direct calls to methods in existing source code libraries.

I wrote the filename displaying the text to the left of each line to show what layer the execution is in.

c:\java myJavaProgram
myJavaProgram: Java Static Method Entry.

myJavaProgram: Java Calling System.loadLibrary(my_plain_dll_to_call_JNI_DLL)

my_JNI_DLL.dll: Entering DllMain

my_JNI_DLL.dll: DLL_PROCESS_ATTACH

my_plain_dll_to_call_JNI_DLL: DLL_PROCESS_ATTACH
my_plain_dll_to_call_JNI_DLL: DLL_THREAD_ATTACH
my_plain_dll_to_call_JNI_DLL: DLL_THREAD_DETACH
my_plain_dll_to_call_JNI_DLL: DLL_PROCESS_DETACH

myJavaProgram: my_plain_dll_to_call_JNI_DLL Loaded!

myJavaProgram: Java Static Method Exit.

myJavaProgram: Entering Main().

my_plain_dll_to_call_JNI_DLL: In call_my_JNI_DLL_method

my_JNI_DLL.dll: In my_JNI_DLL_method

my_JNI_DLL.dll: Entering my_JNI_DLL_CheckEnvironmentVariables()

my_JNI_DLL.dll: Exiting my_JNI_DLL_CheckEnvironmentVariables

my_JNI_DLL.dll: Calling StartExistingNativeCode.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (0xc0fb007e), pid=7500, tid=7552
#
# JRE version: 6.0_21-b06
# Java VM: Java HotSpot(TM) Client VM (17.0-b16 mixed mode, sharing windows-x86 )
# Problematic frame:
# C  [KERNELBASE.dll+0x9673]
#
# An error report file with more information is saved as:
# C:\hs_err_pid7500.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

my_plain_dll_to_call_JNI_DLL: DLL_PROCESS_DETACH

my_JNI_DLL.dll: Entering DllMain

my_JNI_DLL.dll DLL_PROCESS_DETACH

---

Update I've narrowed down the issue to a memory management library that is linked in from another dll that my program uses. The dll that it uses is sh33w32.dll, it's called SmartHeap and is by a company named Microquil I think. I have version 3.3, and when Java LoadLibrary tries to load that dll, it fails. I'm not sure what I could do to have java handle loading that library. It must have something to do with the area of memory that Java can access, versus what windows allows an exe to access. The exe has no problem with the SmartHeap library, but Java will not allow me to use it. Any ideas or experience dealing with this? I've tried to remove the linked library by recompiling the other libraries, but then the normal calls in the code fail that normally work.

---

Additional Information Found The function that is in the dll that fails to load in java is called MemRegisterTask. It's from a product called SmartHeap by Microquill. Here is the documentation I found about this function. I think this memory allocation is what causes java to fail to load it.

MemRegisterTask initializes the SmartHeap Library. On most platforms, you don’t need to call MemRegisterTask because SmartHeap will initialize itself when you make the first call.

SmartHeap maintains a registration reference count for each task or process. Each time you call MemRegisterTask, this reference count is incremented. If your last call to SmartHeap occurs before your application is ready to terminate, you can call MemUnregisterTask to terminate SmartHeap. MemUnregisterTask decrements the registration reference count by one — when the count is zero, SmartHeap will free any SmartHeap-allocated memory and debugging state associated with the current task or process.

Answer 1

Anything useful in the hs_err ... log file. Usually there's a stack backtrace etc. pointing something out.

Also tried to run java.exe (with parameters running a test that loads the stuff) inside the debugger?

From trace above one can see loading seems to work fine (the trace output suggest that dllentrypoint/dllmain has been augmented w/ trace output by you).

Sequence in loading is the following:

1. load dependent dlls
2. load dll itself
3. call the dllentrypoint/dllmain w/ process attach

So this is already beyond loading the DLL.

Have you checked whether you are using debug/release runtimes from Windows? Debug may clash with release - Java is release, your sample exe likely was same as your dll build.