Maven: Using Java 8 libraries in applications instrumented with retrolambda-maven-plugin and DEX-ed with android-maven-plugin

Question

I have written a small set of minilibraries for my internal use. This is set is built using Maven. The libraries are targetted for "regular" Java, GWT and Android. Some of them are written in Java 8 because I hadn't any intention to run them on GWT or Android, thus the other libraries are written in old Java 6 to support these two. I have a plan of full migration of my libs to Java 8 (in terms of language features), and I succeeded to run Java 8 rewritten libraries on not yet released GWT 2.8.0. However, I cannot make the Java 8 rewritten libs to compile for Android applications. The issue is that Retrolambda (the retrolambda-maven-plugin plugin) seems to be able to process the current Maven module classes only, and fully ignores the dependency classes. Thus, the android-maven-plugin breaks the target application build with:

[INFO] UNEXPECTED TOP-LEVEL EXCEPTION:
[INFO] com.android.dx.cf.iface.ParseException: bad class file magic (cafebabe) or version (0034.0000)
[INFO]  at com.android.dx.cf.direct.DirectClassFile.parse0(DirectClassFile.java:472)
[INFO]  at com.android.dx.cf.direct.DirectClassFile.parse(DirectClassFile.java:406)
[INFO]  at com.android.dx.cf.direct.DirectClassFile.parseToInterfacesIfNecessary(DirectClassFile.java:388)
[INFO]  at com.android.dx.cf.direct.DirectClassFile.getMagic(DirectClassFile.java:251)
[INFO]  at com.android.dx.command.dexer.Main.processClass(Main.java:665)
[INFO]  at com.android.dx.command.dexer.Main.processFileBytes(Main.java:634)
[INFO]  at com.android.dx.command.dexer.Main.access$600(Main.java:78)
[INFO]  at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:572)
[INFO]  at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:284)
[INFO]  at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:166)
[INFO]  at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)
[INFO]  at com.android.dx.command.dexer.Main.processOne(Main.java:596)
[INFO]  at com.android.dx.command.dexer.Main.processAllFiles(Main.java:498)
[INFO]  at com.android.dx.command.dexer.Main.runMonoDex(Main.java:264)
[INFO]  at com.android.dx.command.dexer.Main.run(Main.java:230)
[INFO]  at com.android.dx.command.dexer.Main.main(Main.java:199)
[INFO]  at com.android.dx.command.Main.main(Main.java:103)
[INFO] ...while parsing foo/bar/FooBar.class

The retrolambda-maven-plugin is configured as follows:

<plugin>
    <groupId>net.orfjackal.retrolambda</groupId>
    <artifactId>retrolambda-maven-plugin</artifactId>
    <version>2.0.2</version>
    <executions>
        <execution>
            <phase>compile</phase>
            <goals>
                <goal>process-main</goal>
                <goal>process-test</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <target>1.6</target>
        <defaultMethods>true</defaultMethods>
    </configuration>
</plugin>

Is it possible to configure the Retrolambda plugin to process the library classes as well so all dependencies? Or maybe I just could use another bytecode processing tool?

---

UPDATE #1

I think I'm wrong about the Retrolambda fail. Digging into it a little more, I've figured out that android-maven-plugin can be blamed for it because it picks the not instrumented JAR files from the Maven repository directly, and not from the target directory. Enabling verbose logging discovered this pseudo-code command invoked by the android-maven-plugin:

$JAVA_HOME/jre/bin/java
-Xmx1024M
-jar "$ANDROID_HOME/sdk/build-tools/android-4.4/lib/dx.jar"
--dex
--output=$BUILD_DIRECTORY/classes.dex
$BUILD_DIRECTORY/classes
$M2_REPO/foo1/bar1/0.1-SNAPSHOT/bar1-0.1-SNAPSHOT.jar
$M2_REPO/foo2/bar2/0.1-SNAPSHOT/bar2-0.1-SNAPSHOT.jar
$M2_REPO/foo3/bar3/0.1-JAVA-8-SNAPSHOT/bar3-0.1-JAVA-8-SNAPSHOT.jar

My idea is executing the maven-dependency-plugin plugin to obtain those three artifacts into the $BUILD_DIRECTORY/classes directory to let the retrolambda-maven-plugin instrument the dependencies. Let's say:

STEP 1: Copy the dependencies to the target directory

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <executions>
        <execution>
            <phase>process-classes</phase>
            <goals>
                <goal>unpack-dependencies</goal>
            </goals>
            <configuration>
                <includeScope>runtime</includeScope>
                <outputDirectory>${project.build.directory}/classes</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

STEP 2: Instrument the copied dependencies using Retrolambda

Invoke retrolambda-maven-plugin

STEP 3: Compile the DEX file

Invoke android-maven-plugin excluding the dependencies that were copied to the target directory, because they are all supposed to be located in the target directory

But this fails too, because I can't find a way to exclude artifacts from being DEXed with the android-maven-plugin.

How do I suppress the artifacts from being fetched from the repository where they are stored in non-instrumented state?

My plugins configuration:

• org.apache.maven.plugins:maven-dependency-plugin:2.10
• net.orfjackal.retrolambda:retrolambda-maven-plugin:2.0.2
• com.jayway.maven.plugins.android.generation2:android-maven-plugin:3.9.0-rc.3

---

UPDATE #2

The Simpligility team has released Android Maven Plugin 4.4.1 featuring the scenario described below. See more at the plugin changelog.

<plugin>
    <groupId>com.simpligility.maven.plugins</groupId>
    <artifactId>android-maven-plugin</artifactId>
    <version>4.4.1</version>
</plugin>

Example scenario can be found at http://simpligility.github.io/android-maven-plugin/instrumentation.html

Answer 1

Four months later, after having some time for investigation, I have finally made it work. First of all, the crucial thing about the solution is patching android-maven-plugin. I have forked the original plugin at GitHub and added a few configuration filter options that allow to include or exclude artifacts by group IDs, artifact IDs, and their respective versions. This is very important to configure retrolambda-maven-plugin. The overall workflow is basically the same as I noted in the question. Here it is:

• maven-compiler-plugin Enable Java 8 language features support. Optional, as the main objective is processing Java 8 dependencies.
• maven-dependency-plugin Unpack ALL Java 8 dependencies to the current project build target directory for the further processing.
• retrolambda-maven-plugin Process all the obtained class files using the Retrolambda plugin.
• android-maven-plugin Compile the DEX and APK files with the fork of the original android-maven-plugin.

Installing the fork:

#!/bin/bash

# The last upstream merge revision
PLUGIN_COMMIT=a79e45bc0721bfea97ec139311fe31d959851476

# Clone the fork
git clone https://github.com/lyubomyr-shaydariv/android-maven-plugin.git

# Ensure proper revision
cd android-maven-plugin
git checkout $PLUGIN_COMMIT

# Build the forked plugin, no tests
mvn clean package -Dmaven.test.skip=true

# Clone plugin JAR
cd target
cp android-maven-plugin-4.3.1-SNAPSHOT.jar android-maven-plugin-4.3.1-SNAPSHOT-$PLUGIN_COMMIT.jar

# Clone and modify pom.xml
cp ../pom.xml pom-$PLUGIN_COMMIT.xml
sed -i "s/<version>4.3.1-SNAPSHOT<\\/version>/<version>4.3.1-SNAPSHOT-$PLUGIN_COMMIT<\\/version>/g" pom-$PLUGIN_COMMIT.xml

# Update the plugin descriptor
unzip android-maven-plugin-4.3.1-SNAPSHOT-$PLUGIN_COMMIT.jar META-INF/maven/plugin.xml
sed -i "s/<version>4.3.1-SNAPSHOT<\\/version>/<version>4.3.1-SNAPSHOT-$PLUGIN_COMMIT<\\/version>/g" META-INF/maven/plugin.xml
zip android-maven-plugin-4.3.1-SNAPSHOT-$PLUGIN_COMMIT.jar META-INF/maven/plugin.xml

# Install the plugin
mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file -DpomFile=pom-$PLUGIN_COMMIT.xml -Dfile=android-maven-plugin-4.3.1-SNAPSHOT-$PLUGIN_COMMIT.jar

Done. Next, register the fork in your pom.xml and configure the build plugins:

<!-- enable Java 8 for the current module -->

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.2</version>
    <configuration>
        <source>1.8</source>
        <target>1.8</target>
    </configuration>
</plugin>

<!-- unpack Java 8 dependency classes to the current module build directory -->

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <version>2.10</version>
    <executions>
            <execution>
                <phase>process-classes</phase>
                <goals>
                    <goal>unpack-dependencies</goal>
                </goals>
                <configuration>
                    <includeScope>runtime</includeScope>
                    <includeGroupIds>foo-group,bar-group,baz-group</includeGroupIds>
                    <outputDirectory>${project.build.directory}/classes</outputDirectory>
                </configuration>
        </execution>
    </executions>
</plugin>

<!-- Convert Java 8 to Java 6 -->

<plugin>
    <groupId>net.orfjackal.retrolambda</groupId>
    <artifactId>retrolambda-maven-plugin</artifactId>
    <version>2.0.6</version>
    <executions>
        <execution>
            <phase>process-classes</phase>
            <goals>
                <goal>process-main</goal>
                <goal>process-test</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <defaultMethods>true</defaultMethods>
        <target>1.6</target>
    </configuration>
</plugin>

<!-- DEXify and build the APK excluding the Java 8 dependencies as they are already processed -->

<plugin>
    <groupId>com.simpligility.maven.plugins</groupId>
    <artifactId>android-maven-plugin</artifactId>
    <version>4.3.1-SNAPSHOT-a79e45bc0721bfea97ec139311fe31d959851476</version>
    <executions>
        <execution>
            <phase>package</phase>
        </execution>
    </executions>
    <configuration>
        <androidManifestFile>${project.basedir}/src/main/android/AndroidManifest.xml</androidManifestFile>
        <assetsDirectory>${project.basedir}/src/main/android/assets</assetsDirectory>
        <resourceDirectory>${project.basedir}/src/main/android/res</resourceDirectory>
        <sdk>
            <platform>19</platform>
        </sdk>
        <undeployBeforeDeploy>true</undeployBeforeDeploy>
        <proguard>
            <skip>true</skip>
            <config>${project.basedir}/proguard.conf</config>
        </proguard>
        <excludes>
            <exclude>foo-group</exclude>
            <exclude>bar-group</exclude>
            <exclude>baz-group</exclude>
        </excludes>
    </configuration>
    <extensions>true</extensions>
    <dependencies>
        <dependency>
            <groupId>net.sf.proguard</groupId>
            <artifactId>proguard-base</artifactId>
            <version>5.2.1</version>
            <scope>runtime</scope>
        </dependency>
    </dependencies>
</plugin>

It should work.

---

Update

Recently I patched the fork for the Proguard Maven mojo.

Update 2

The android-maven-plugin repository owner merged my commits, so the artifact filtering is scheduled for the next version.