Can't build multiple Android dex files with Ant from external jars

Question

I'm developing an Android app that requires multiple libraries (for Facebook, Google Maps v2 and Quickblox among others), resulting in a method amount overflow that goes over the 64K limit:

Unable to execute dex: method ID not in [0, 0xffff]: 65536
Conversion to Dalvik format failed: Unable to execute dex: method ID not in [0, 0xffff]: 65536

As I can't do without any of those libraries, I looked for a solution for the method limit bug. I found a popular blog entry from Android Developers, where a source code division is recommended. (The blog entry I'm talking about can be found here: http://android-developers.blogspot.com.es/2011/07/custom-class-loading-in-dalvik.html). I've been trying this solution with no success.

The problem I have now is that the biggest amount of code is not in my app itself, but in the required libraries, so I have to spread those libraries among the different dex files I must load in my app. My knowledge of Ant is very limited, and what I'd like to know is what I should write in my build.xml file to make dex copy each library where I want:

            <!-- Primary dex to include my source code and some libraries. -->
            <copy todir="${out.classes.absolute.dir}.1" >
                <fileset dir="${out.classes.absolute.dir}" >

                ...

                </fileset>
            </copy>


            <!-- Secondary dex to include some other libraries. -->
            <copy todir="${out.classes.absolute.dir}.2" >
                <fileset dir="${out.classes.absolute.dir}" >

                ...  

                </fileset>
            </copy>

Any help would be truly appreciated. Thanks in advance, kind regards!

Answer 1

The best answer I have heard to this problem so far is to use ProGuard in optimized mode (proguard-android-optimize.txt) with the -dontobfuscate flag to remove unused classes and methods from the final APK (without obfuscating them). Then you can use the ProGuard mapping.txt to help you strip the unused classes out of the library JARs that you use (I'm not aware of a good tool to do this). Unfortunately I don't think there is a feature in ProGuard to do this automatically.

ProGuard only runs when doing an Export in Eclipse, not when doing a Run As -> Android Application. This means it won't help avoid the limit on debug builds, unless you use a custom build process. The creator of ProGuard suggests using its commercial sibling DexGuard, which will run in Eclipse on both debug and release builds.

Using ProGuard is strongly encouraged for release builds as it will reduce your code size, improve performance (e.g. by inlining code), and will obfuscate your source too. Make sure you do enough testing on the final APK as it will be dramatically different to the debug one. Unfortunately, ProGuard on its own wasn't enough to solve my problem, so I removed one of my JAR dependencies as a workaround.

I checked the number of methods in my dependent JARs using this command:

dx --dex --output=temp.dex library.jar
cat temp.dex | head -c 92 | tail -c 4 | hexdump -e '1/4 "%d\n"'

Examples numbers of methods for our biggest libraries:

11222 guava-11.0.1.jar    
10452 aws-android-sdk-1.5.0-core.jar    
5761 org.restlet.jar    
5129 protobuf-java-2.4.1.jar    
2499 aws-android-sdk-1.5.0-s3.jar    
2024 ormlite-core-4.41.jar    
1145 gson-2.2.2.jar    
1716 google-http-client-1.11.0-beta.jar    

FYI, you can check the number of methods in your APK using dexdump (which you can find in your SDK build-tools folder, e.g. "Android Studio.app/sdk/build-tools/21.0.2/dexdump"):

dexdump -f MyApp.apk | grep method_ids_size
method_ids_size     : 64295

In our case we were only using Guava for doing YouTube searches, so it was easy to remove that dependency and give us more breathing room.

Update: I found the easiest way to strip out unused code from large JARs was to use dex2jar on my Proguard build, then use JD-GUI to compare that final JAR with the specific input JAR I wanted to trim. Then I remove top level packages that I know get removed by Proguard. This allowed me to remove >8000 methods from aws-android-sdk-1.5.0-core.jar.