JNI "symbol lookup error" in shared library on Linux

Question

What do you do when the Java VM has a “symbol lookup error” when executing a JNI function? The symbol lookup error is not in the primary shared object library that support the JNI interface, nor is it directly within a library linked to the primary object library, but in a library that’s linked to a library that’s linked to your JNI shared object? (what an incredibly awkward sentence ☺) Particularly, what do you do when you don’t control the code for the library that contains the offending symbol?

I have a problem using JNI to access the SDK for a scientific camera (Andor NEO CMOS). I’m using the Netbeans C/C++ plugin on RHEL 6 to create a shared library (AndorC.so) that basically creates JNI wrappers around the methods provided by the cameras SDK. The camera SDK provides a set of methods to access the camera, which are packaged in a shared object library (libatcore.so). The libatcore.so library uses a series of additional libraries, libatdevregcam.so (for the real camera), libatdevsimcam.so (for the simulated camera), libatcl_bitflow.so (low level video board driver), etc. .

I have tested the camera SDK extensively and I can access the camera with no difficulty from C/C++. I have linked to the shared library (AndorC) that implements the JNI functions from a test C program (using a separate header file) and everything runs properly (i.e. I can read an image and the program completes normally).

My Java code can execute the “InitializeLibrary” and “FinalizeLibrary” functions from the SDK through the JNI interface so there is no problem finding the primary libatcore.so library or my AndorC.so library. There doesn’t appear to be any problem with the basic setup of the JNI. However, when I try to execute the “Open” function for the camera (i.e. the function that actually connects to the real and/ or simulated camera) I get an undefined symbol error in one of the libraries that the libatcore.so libraries uses at runtime (libdevsimcam.so).

/usr/local/java/jdk1.7.0_03/jre/bin/java: symbol lookup error: /usr/local/lib/libatdevsimcam.so: undefined symbol: _ZN20TAndorLibDebugOutput6OutputEPKciS1_xx24TAndorDebugFunctionLevel16TAndorDebugLevelS1_z
Java Result: 127

Essentially, the Java VM doesn’t have a problem with the libatcore.so (i.e. the main library file) and it doesn’t have a problem locating the associated runtime library (/usr/local/lib/libatdevsimcam.so) but it runs into an undefined symbol in that library when it tries to open the camera (note I’m actually opening the real camera NOT the simulated camera).

When I checked the dependencies on each of the libraries (libAndorC.so, libatcore.so, libatdevsimcam.so) I don’t find any undefined symbols. Obviously, the undefined symbol in the libatdevsimcam.so library isn’t a problem when the program is run directly from C/C++ but it’s causing a problem when the Java VM tries to load libatdevsimcam.so.

 libAndorC.so
linux-vdso.so.1 =>  (0x00007fff507ff000)
libatcore.so.3 => /usr/local/lib/libatcore.so.3 (0x00007fe23a278000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007fe239f4a000)
libm.so.6 => /lib64/libm.so.6 (0x00007fe239cc6000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fe239ab0000)
libc.so.6 => /lib64/libc.so.6 (0x00007fe239730000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe239513000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fe23930f000)
librt.so.1 => /lib64/librt.so.1 (0x00007fe239106000)
/lib64/ld-linux-x86-64.so.2 (0x0000003f07000000)
libatcore.so
linux-vdso.so.1 =>  (0x00007fffd53ff000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7e6b645000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f7e6b440000)
librt.so.1 => /lib64/librt.so.1 (0x00007f7e6b238000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f7e6af31000)
libm.so.6 => /lib64/libm.so.6 (0x00007f7e6acac000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f7e6aa96000)
libc.so.6 => /lib64/libc.so.6 (0x00007f7e6a717000)
/lib64/ld-linux-x86-64.so.2 (0x0000003f07000000)
ldd libatdevsimcam.so
linux-vdso.so.1 =>  (0x00007fff247ef000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5219dc4000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f5219bbf000)
librt.so.1 => /lib64/librt.so.1 (0x00007f52199b7000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f52196b0000)
libm.so.6 => /lib64/libm.so.6 (0x00007f521942b000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f5219215000)
libc.so.6 => /lib64/libc.so.6 (0x00007f5218e96000)
/lib64/ld-linux-x86-64.so.2 (0x0000003f07000000)

When I specifically check the symbol that the Java VM is having a problem with it’s clear that the symbol is undefined.

nm libatdevsimcam.so | grep _ZN20TAndor
             U _ZN20TAndorLibDebugOutput6OutputEPKciS1_xx24TAndorDebugFunctionLevel16TAndorDebugLevelS1_z

I thought perhaps the symbol wasn’t defined in the release vs. debug version of the libatdevsimcam.so library so I tried building a release version of the libAndorC.so but I got the same problem.

I have two basic questions

1. Could this be a problem with C++ name mangling? I’ve tried compiling my camera test program with gcc instead of g++ and I run into a large number of compilation errors. I’ve read that C++ name mangling can cause problems.

2. Is this a problem in the libatdevsimcam.so library provided by the manufacturer? I have no control over the code supplied by Andor to support the camera (all I can do is complain).

I’m basically stuck at this point because all of the SDK functions require a reference to the camera handle returned by the “Open” method. If I can’t open the camera I can’t proceed with developing the JNI interface to the camera that’s I need.

I’ve research this question extensively and I haven’t found any answers that directly address the issue. This isn’t the standard “unsatisfied link” error that’s so common in JNI posts and the basic JNI functionality seems to work (i.e. you can initialize and finalize the library or call any function that doesn’t directly access the camera) . This seems to be a situation where the Java VM runs into a problem with the native code that simply running the code directly doesn’t produce. How do you deal with this kind of issue? Is this a problem I need to take to the manufacturer or is there a configuration / compilation / linking method of dealing with this?

Some additional details:

• Netbeans 6.9.1 development environment
• JDK 1.7_0_03 64 bit server
• Compiling sources as 64bit (i.e. there should be no 32 vs 64 bit problems)

Answer 1

You can get rid of it by using LD_PRELOAD (but i don't think this is a proper solution)

The Story

I faced the same problem, and my scenario is...

*javaHelloWorldApp.java --> JNI_Hello_world.c --> this native c function calls to snmp library*

java: symbol lookup error: /home/source/bin/libmytest.so: undefined symbol: init_snmp

I used LD_PRELOAD as shown here and solves the problem for now. export LD_PRELOAD=/usr/local/lib/libnetsnmp.so.30

Open Question

ldd gives, (note, there is no reference to snmp library)

ldd ../libmytest/bin/libmytest.so linux-gate.so.1 => (0xb7777000) libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb75a3000) /lib/ld-linux.so.2 (0xb7778000)

with export LD_PRELOAD=/usr/local/lib/libnetsnmp.so.30 I see the references to snmp,

snmp$ldd ../libredsnmp/bin/libredsnmp.so

linux-gate.so.1 =>  (0xb770c000)
    /usr/local/lib/libnetsnmp.so.30 (0xb763a000)
    libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb746c000)
    librt.so.1 => /lib/i386-linux-gnu/librt.so.1 (0xb7462000)
    libcrypto.so.1.0.0 => /lib/i386-linux-gnu/libcrypto.so.1.0.0 (0xb72b7000)
    /lib/ld-linux.so.2 (0xb770d000)
    libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xb729c000)
    libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xb7297000)
    libz.so.1 => /lib/i386-linux-gnu/libz.so.1 (0xb7281000)

my compilation line,

gcc -fPIC -shared -o ./bin/libmytest.so -I/usr/lib/jvm/java-6-openjdk-i386/include/ -I/usr/lib/jvm/java-6-openjdk-i386/include/linux -static -lc JNI_Hello_world.c

Other options i tried, -L/usr/local/lib -lnetsnmp, doesn't have any effect so i removed from compilation.

EDIT-1

got it worked with these two commands and no other environment variable, First, compile the source

gcc -c -fPIC ./mytest.c -o mytest.o -I$(JAVA_INC_PATH) -I$(JAVA_INC_PATH)linux

Now, use -rpath linker option when building the shared library.

gcc -shared -o ./bin/libmytest.so  mytest.o -Wl,-rpath,/usr/local/lib -L/usr/local/lib -lnetsnmp

For Detailed Reference

For a detailed description refer this book (or just this google book search result, @ ch.41.10)

Answer 2

I had similar problem, and later I found it was the problem of linking order:

you need to put -L/usr/local/lib -lnetsnmp after the -o libmytest.so for it to work.