Bundle native dependencies in runnable .jar with Maven

Question

I have a project managed in Maven that has some native dependencies (LWJGL).

Everything works fine in development, but now I want to set up Maven so that it will build a runnable .jar file that I can redistribute. In particular, I want it to be very easy for users to run the app without having to mess around with library paths or unpacking native libraries etc.

Currently I am able to build a .jar file that includes all the dependencies, but if I run it then (unsurprisingly) I get an unsatisfied link error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: no lwjgl in java.libr
ary.path
        at java.lang.ClassLoader.loadLibrary(Unknown Source)
        at java.lang.Runtime.loadLibrary0(Unknown Source)
        at java.lang.System.loadLibrary(Unknown Source)
        at org.lwjgl.Sys$1.run(Sys.java:73)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.lwjgl.Sys.doLoadLibrary(Sys.java:66)
        at org.lwjgl.Sys.loadLibrary(Sys.java:95)
        at org.lwjgl.Sys.<clinit>(Sys.java:112)
        at org.lwjgl.opengl.Display.<clinit>(Display.java:132)
        at glaze.TestApp.start(TestApp.java:10)
        at glaze.TestApp.main(TestApp.java:31)

Obviously I can make it work by manually installing the native libraries and running the jar with java -Djava.library.path=/path/to/libs but that isn't something I can expect my users to do.

Here's the pom.xml in case it is relevant: https://github.com/mikera/glaze/blob/master/pom.xml

It is possible to set up Maven so that it will create a runnable .jar that includes the native dependencies and will run successfully when double-clicked?

Answer 1

This is some code I used to use to load dll or so libraries that are bundled in the jar.

The libraries must be added as resources. We used maven and put them in this hierarchy:

src/main/resources/lib/win-x86/<dlls for 32-bit windows>
src/main/resources/lib/linux-x86/<so for 32-bit linux>
src/main/resources/lib/linux-x86_64/<so for 64-bit linux>
src/main/resources/lib/linux-ia64/<so for 64-bit linux on itanium>

The shared libraries will be unpacked to the tmp-directory for the platform and also have a temporary name when unpacked. This is to let several processes load the dll/so without sharing the actual extracted dll/so since the unpacking could overwrite existing ones if having the same name (with very strange behavior on some platforms when the file was replaced).

The file is also set to have deleteOnExit set but that does not work on windows AFAIK.

NativeLoader.java

public class NativeLoader {

    public static final Logger LOG = Logger.getLogger(NativeLoader.class);

    public NativeLoader() {
    }

    public void loadLibrary(String library) {
        try {
            System.load(saveLibrary(library));
        } catch (IOException e) {
            LOG.warn("Could not find library " + library +
                    " as resource, trying fallback lookup through System.loadLibrary");
            System.loadLibrary(library);
        }
    }


    private String getOSSpecificLibraryName(String library, boolean includePath) {
        String osArch = System.getProperty("os.arch");
        String osName = System.getProperty("os.name").toLowerCase();
        String name;
        String path;

        if (osName.startsWith("win")) {
            if (osArch.equalsIgnoreCase("x86")) {
                name = library + ".dll";
                path = "win-x86/";
            } else {
                throw new UnsupportedOperationException("Platform " + osName + ":" + osArch + " not supported");
            }
        } else if (osName.startsWith("linux")) {
            if (osArch.equalsIgnoreCase("amd64")) {
                name = "lib" + library + ".so";
                path = "linux-x86_64/";
            } else if (osArch.equalsIgnoreCase("ia64")) {
                name = "lib" + library + ".so";
                path = "linux-ia64/";
            } else if (osArch.equalsIgnoreCase("i386")) {
                name = "lib" + library + ".so";
                path = "linux-x86/";
            } else {
                throw new UnsupportedOperationException("Platform " + osName + ":" + osArch + " not supported");
            }
        } else {
            throw new UnsupportedOperationException("Platform " + osName + ":" + osArch + " not supported");
        }

        return includePath ? path + name : name;
    }

    private String saveLibrary(String library) throws IOException {
        InputStream in = null;
        OutputStream out = null;

        try {
            String libraryName = getOSSpecificLibraryName(library, true);
            in = this.getClass().getClassLoader().getResourceAsStream("lib/" + libraryName);
            String tmpDirName = System.getProperty("java.io.tmpdir");
            File tmpDir = new File(tmpDirName);
            if (!tmpDir.exists()) {
                tmpDir.mkdir();
            }
            File file = File.createTempFile(library + "-", ".tmp", tmpDir);
            // Clean up the file when exiting
            file.deleteOnExit();
            out = new FileOutputStream(file);

            int cnt;
            byte buf[] = new byte[16 * 1024];
            // copy until done.
            while ((cnt = in.read(buf)) >= 1) {
                out.write(buf, 0, cnt);
            }
            LOG.info("Saved libfile: " + file.getAbsoluteFile());
            return file.getAbsolutePath();
        } finally {
            if (in != null) {
                try {
                    in.close();
                } catch (IOException ignore) {
                }
            }
            if (out != null) {
                try {
                    out.close();
                } catch (IOException ignore) {
                }
            }
        }
    }
}

The libraries are loaded by creating an instance of the NativeLoader and then by calling loadLibrary("thelibrary") without the os-specific prefixes and extensions.

This worked well for us but you will have to add the shared libraries manually to the different resource directories and then build the jar.

I realize that some code in this class may be strange or obsolete but bare in mind that this is code I wrote some years ago and it has been working really well.

Answer 2

Did you try to use maven-assembly-plugin here is an example :

<build>
   <plugins>
      <plugin>
         <artifactId>maven-assembly-plugin</artifactId>
         <configuration>
         <archive>
             <manifest>
                <mainClass>your.main.Class</mainClass>
             </manifest>
         </archive>
         <descriptorRefs>
             <descriptorRef>jar-with-dependencies</descriptorRef>
         </descriptorRefs>
         </configuration>
       </plugin>
   </plugins>
</build>

And for your native dependencies you may want to use Bundle-NativeCode in your manifest file. See http://wiki.osgi.org/wiki/Bundle-NativeCode.

You also may want to have a look at the maven-bundle-plugin : http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html to generate it with Maven.