How to combine multiple Java modules into one JAR?

Question

The goal:

1. to combine multiple JARs and the modules in them into a single JAR
2. allow consumers to include a single package dependency in their maven/gradle/... files
3. make all of the modules available to the consumers

I tried shade and assembly plugin for days now, and keep failing. To keep this short, I'll omit all the wrong approaches; I am looking for one that works.

Before JPMS, it was simply shading all the contents into one JAR. Can't do that with modules, since there is only one top-level module-info.class for the entire JAR.

If there was a transformer for shade combining module-info.class files like it does for e.g. Services, that would be the obvious solution. But to my knowledge, it doesn't exist.

How can this be done - multiple JPMS modules in the same JAR combined - working with JDK11+?

Background:

• Business requirement 1 was to package the maven modules to a single (signed) JAR. This has proven to be useful, and is to be continued.
• Business requirement 2 to use Java Modules (JPMS) came up new.

The question is about finding a technical solution that fulfills both business requirements. It makes sense to probe whether JPMS can be omitted/ignored, but that's not what this question is about.

Answer 1

If I understand correctly, you have multiple modular libraries that are relatively coupled together. And you want to provide a way to make it easier for consumers to declare dependencies on your libraries. In particular, you want to provide a way for consumers to declare a single dependency that gives access to all your libraries.

You've considered shading your modules into a single artifact and publishing it. As you've figured out, this doesn't work well with JPMS because a standard JAR can only contain a single module. But I'd argue this is not a good idea even without JPMS modules being involved. In my opinion, a library should never be shaded. It makes it more difficult, if not impossible, for consumers to control their dependencies (e.g., exclusion, using a specific version, etc.) when they're all included in a single artifact. Maybe it's not so bad if only libraries from the same multi-project build are shaded, but I think the decision to shade a JAR should be left up to the application developers, not the library developers.

This leaves you with a few options (plus any I'm not thinking of).

1. Just Don't Try

You can simply not try to publish a monolith artifact. Just publish each individual artifact like normal.

This may make depending on your libraries more verbose, but it's still a viable solution. Keep in mind that editing a build file should not be an overly common occurrence outside of updating versions. Consumers of your libraries only need to declare the dependencies once and they're done. This isn't something that will plague them during every second of development. The same goes for module-info.java files.

You can publish a BOM (Bill of Materials) to ease version management. Many multi-module projects do this (e.g,. Spring, JUnit, etc.).

2. Create an Aggregate Module

Note: I find this to be the ideal solution of those presented in this answer.

You can create an aggregate module. In this case, "module" means both a Maven module and a JPMS module. The Maven module would declare dependencies on your other modules:

<parent>
  <groupId>com.example</groupId>
  <artifactId>mylib</artifactId>
  <version>${revision}</version>
</parent>

<groupId>com.example</groupId>
<artifactId>mylib-aggregate</artifactId>

<dependencies>
  <dependency>
    <groupId>com.example</groupId>
    <artifactId>mylib-core</artifactId>
    <version>${project.version}</version>
  </dependency>
  <dependency>
    <groupId>com.example</groupId>
    <artifactId>mylib-util</artifactId>
    <version>${project.version}</version>
  </dependency>
</dependencies>

^{- For Gradle projects, these would be api dependencies (not implementation).}

This module would have only one source file, a single module-info.java file:

module com.example.mylib.aggregate {
  requires transitive com.example.mylib.core;
  requires transitive com.example.mylib.util;
}

And that's it. You'd publish this aggregate module alongside all the individual modules.

Consumer

A consumer would just need to declare a dependency on the aggregate and all the transitive dependencies will be pulled in:

<groupId>org.company</groupid>
<artifactId>company-app</artifactId>
<version>0.1.0-SNAPSHOT</version>

<dependencies>
  <dependency>
    <groupId>com.example</groupId>
    <artifactId>mylib-aggregate</artifactId>
    <version>1.0.0</version>
  <dependency>
</dependencies>

Additionally, the consumer would now only need to declare a single requires directive instead of one for each module:

module org.company.app {
  requires com.example.mylib.aggregate;
}

Assuming the consuming project is modular in the first place.

Real Examples

There are examples of such aggregate modules in "the real world". Two of those are:

1. The java.se module included in the Java standard library.

2. The org.junit.jupiter module from the JUnit Jupiter API.

3. Try to Merge Module Descriptors

Note: I do not think this is a good or trivial approach.

If you really want to create a "shaded module", then you'll have to find a way merge each dependency's module-info descriptor into a single one. You can easily read existing compiled descriptors via the ModuleDescriptor class. Writing a new one is a little more involved though. You can either:

1. Write the source code for the new aggregate module, then compile it after merging everything. This may require using the --patch-module option:

   --patch-module <aggregate-module>=<path-to-shaded-classes>
   

2. Write the class file directly using a byte-code manipulation library. There's a few such libraries out there. If you're running the build on Java 24+ then you could even make use of the new Class-File API (specifically one of the ClassFile::buildModule[To] methods) added to the standard library.

If you want to do this via the Maven Shade Plugin, then unfortunately this does not seem possible with a ResourceTransformer. A ResourceTransformer is never passed class files, and so will never be passed any module-info.class. You might be able to do this with a custom Shader implementation. Also, keep in mind that the Maven Shade Plugin can relocate classes (see Relocator). You'd have to take that into account when merging the module-info descriptor.

Regardless, there are some things to note:

• Need to add a way to either specify the monolith module's name or generate it from other information.

• What should the version of the monolith module be, if any? Is the version specified explicitly or is it computed based on the included modules?

• How should non-modular dependencies be handled?

• What happens if some but not all included modules are open? If one included module is open, should the monolith module be open? Or should packages from the open module be opens in the monolith module?

• Exclude any requires directives for modules that are being included in the monolith.

• What happens if there's some combination of requires, requires static, requires transitive, and requires static transitive for a single module across the modules included in the monolith? Which modifiers take precedence? Keep in mind that static requires typically are not resolved automatically at link-time (i.e., jlink) or run-time.

• What happens if two or more included modules require different versions of the same module? Which version takes precedence?

• Exclude any targets in qualified exports or opens directives for modules that are being included in the monolith. Exclude the entire qualified exports or opens directive if all its targets are excluded.

• Exclude all opens directives if the monolith module ends up being open.

• Don't forget to merge the "packages" attributes of the module-info descriptors.

• May want to maintain the encounter order of module-info descriptors and their requires, exports, opens, uses, and provides directives. Could help with creating reproducible builds.

And there could be other corner cases I'm not thinking of.

4. Ignore JPMS

You can ignore JPMS by just excluding all module-info.class files when creating the shaded artifact. This does unfortunately mean the shaded artifact will not be modular, but that won't necessarily be a problem for consumers which aren't using JPMS themselves. Though again, I think publishing a shaded library is not a good idea in the first place.

5. Manually Create Module Descriptor

You could always manually create the module-info descriptor for your shaded JAR file. Though the maintenance may become tedious as you'll have to manually ensure it's still valid every time a dependency is updated.

Answer 2

In addition to the amazing answer by Slaw, I want to add another approach: using two artifacts (both can be signed).

As Slaw mentioned, having a fat JAR in a Maven repository causes many issues. This is especially the case for applications getting the same transitive dependency from two different sources as such applications will suffer from split packages (which don't work on the modulepath) and dependency conflict (you might have classes from different versions of the same dependency on your classpath/modulepath).

To solve that, you could produce two artifacts: One fat JAR that includes all dependencies and one thin JAR that doesn't include any dependencies which is published in the Maven repository (if you really want to, you can also publish the fat JAR to the Maven repository using a classifier). The fat JAR would then not use modules or use a merged module descriptor (options 3 and 4 in Slaw's answer) while the thin JAR is modularized normally.
You can then sign both JARs as well as sign all dependency JARs and also make the dependency JARs available in your Maven repository.

If someone wants to consume your artifact with Maven or Gradle (and possibly also use other dependencies), they can use the modularized thin JAR. If their build requires all dependencies to be signed, they can use your Maven repository for these dependencies as well and they only get signed artifacts (provided they don't depend on any artifacts that aren't signed) If they want to consume your JAR without using any build tool (which is the only reason one would ever want to use a far JAR as a dependency), they can just import your fat JAR in their IDE of choice.