DuplicateFileException error while building Android APK

Question

I am using Android Studio 2.0 on Windows 7. When building my Android project, I get the following error:

:app:transformResourcesWithMergeJavaResForDebug FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:transformResourcesWithMergeJavaResForDebug'.
> com.android.build.api.transform.TransformException: com.android.builder.packaging.DuplicateFileException: Duplicate files copied in APK com/google/appengine/repackaged/org/apache/commons/codec/language/bm/sep_approx_spanish.txt
File1: C:\Users\USER\.gradle\caches\modules-2\files-2.1\com.google.appengine\appengine-api-1.0-sdk\1.9.28\e92c18272b555027d9ec666e7a89162f10638314\appengine-api-1.0-sdk-1.9.28.jar
File2: C:\Users\USER\.gradle\caches\modules-2\files-2.1\com.google.appengine\appengine-endpoints\1.9.28\bf2e8a74bd28e388b3487fc78a0c7adfa592fd5d\appengine-endpoints-1.9.28.jar`

I have been digging around for hours and can't see anything that causes this. I've deleted .gradle in my home directory, but it just rebuilds it all the same. Does anyone know what causes this?

Full gradle output:

Executing tasks: [:app:clean, :app:generateDebugSources, :app:mockableAndroidJar, :app:prepareDebugUnitTestDependencies, :app:generateDebugAndroidTestSources, :app:assembleDebug]

Configuration on demand is an incubating feature.
WARNING: Dependency org.apache.httpcomponents:httpclient:4.0.1 is ignored for debug as it may be conflicting with the internal version provided by Android.
     In case of problem, please repackage it with jarjar to change the class packages
WARNING: Dependency org.apache.httpcomponents:httpclient:4.0.1 is ignored for release as it may be conflicting with the internal version provided by Android.
     In case of problem, please repackage it with jarjar to change the class packages
Incremental java compilation is an incubating feature.
:app:clean
:app:preBuild UP-TO-DATE
:app:preDebugBuild UP-TO-DATE
:app:checkDebugManifest
:app:preReleaseBuild UP-TO-DATE
:backend:appengineDownloadSdk
:backend:compileJava
:backend:processResources UP-TO-DATE
:backend:classes
:backend:appengineEndpointsGetClientLibs UP-TO-DATE
:backend:appengineEndpointsExpandClientLibs UP-TO-DATE
:backend:compileEndpointsSrcJava
:backend:processEndpointsSrcResources UP-TO-DATE
:backend:endpointsSrcClasses
:backend:_appengineEndpointsAndroidArtifact UP-TO-DATE
:backend:jar UP-TO-DATE
:app:prepareComAndroidSupportAnimatedVectorDrawable2330Library
:app:prepareComAndroidSupportAppcompatV72330Library
:app:prepareComAndroidSupportSupportV42330Library
:app:prepareComAndroidSupportSupportVectorDrawable2330Library
:app:prepareComGoogleAndroidGmsPlayServicesAuth840Library
:app:prepareComGoogleAndroidGmsPlayServicesBase840Library
:app:prepareComGoogleAndroidGmsPlayServicesBasement840Library
:app:prepareComGoogleAndroidGmsPlayServicesMeasurement840Library
:app:prepareDebugDependencies
:app:compileDebugAidl
:app:compileDebugRenderscript
:app:generateDebugBuildConfig
:app:generateDebugAssets UP-TO-DATE
:app:mergeDebugAssets
:app:generateDebugResValues UP-TO-DATE
:app:processDebugGoogleServices
:app:generateDebugResources
:app:mergeDebugResources
:app:processDebugManifest
:app:processDebugResources
:app:generateDebugSources
:app:mockableAndroidJar UP-TO-DATE
:app:preDebugUnitTestBuild UP-TO-DATE
:app:prepareDebugUnitTestDependencies
:app:preDebugAndroidTestBuild UP-TO-DATE
:app:prepareDebugAndroidTestDependencies
:app:compileDebugAndroidTestAidl
:app:processDebugAndroidTestManifest
:app:compileDebugAndroidTestRenderscript
:app:generateDebugAndroidTestBuildConfig
:app:generateDebugAndroidTestAssets UP-TO-DATE
:app:mergeDebugAndroidTestAssets
:app:generateDebugAndroidTestResValues UP-TO-DATE
:app:generateDebugAndroidTestResources
:app:mergeDebugAndroidTestResources
:app:processDebugAndroidTestResources
:app:generateDebugAndroidTestSources
:app:compileDebugJavaWithJavac
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:app:compileDebugNdk UP-TO-DATE
:app:compileDebugSources
:app:buildInfoDebugLoader
:app:transformClassesWithExtractJarsForDebug
:app:transformClassesWithInstantRunVerifierForDebug
:app:transformClassesWithJavaResourcesVerifierForDebug UP-TO-DATE
:app:mergeDebugJniLibFolders
:app:transformNative_libsWithMergeJniLibsForDebug
:app:processDebugJavaRes UP-TO-DATE
:app:transformResourcesWithMergeJavaResForDebug FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:transformResourcesWithMergeJavaResForDebug'.
> com.android.build.api.transform.TransformException:     com.android.builder.packaging.DuplicateFileException: Duplicate files copied in APK com/google/appengine/repackaged/org/apache/commons/codec/language/bm/sep_approx_spanish.txt
File1: C:\Users\USER\.gradle\caches\modules-2\files-2.1\com.google.appengine\appengine-api-1.0-sdk\1.9.28\e92c18272b555027d9ec666e7a89162f10638314\appengine-api-1.0-sdk-1.9.28.jar
File2: C:\Users\USER\.gradle\caches\modules-2\files-2.1\com.google.appengine\appengine-endpoints\1.9.28\bf2e8a74bd28e388b3487fc78a0c7adfa592fd5d\appengine-endpoints-1.9.28.jar


* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 15.364 secs

From what I can tell, it's a conflict between two dependencies:

dependencies {
  appengineSdk 'com.google.appengine:appengine-java-sdk:1.9.28'
  compile 'com.google.appengine:appengine-endpoints:1.9.28'
  compile 'com.google.appengine:appengine-endpoints-deps:1.9.28'
  compile 'javax.servlet:servlet-api:2.5'
  compile 'com.google.http-client:google-http-client-jackson2:1.20.0'
  compile 'com.google.http-client:google-http-client-gson:1.19.0'
}

I think it's these two dependencies:

appengineSdk 'com.google.appengine:appengine-java-sdk:1.9.28'
compile 'com.google.appengine:appengine-endpoints:1.9.28'
compile 'com.google.appengine:appengine-endpoints-deps:1.9.28'

I'm not able to remove any of these, as it causes compile errors on the backend module. It looks like those two dependencies are almost identical, however. Should they both be in there? Any advice is appreciated!

Answer 1

To complete @dsh's answer:

In your dependency tree there are 2 jar files containing the same file named sep_approx_spanish.txt , this is not allowed.

To resolve your problem you have to track down which of your gradle dependencies has the two jars named appengine-api-1.0-sdk-1.9.28.jar and appengine-endpoints-1.9.28.jar; to do this you can open android studio's terminal console and type in:

//if ou are on windows
gradlew dependencies

//if you are on unix based os
./gradlew dependencies

You will see gradle doing a lot of things, but at a certain point it will print the entire dependency tree. Once you find which are the 2 dependencies are causing the problem, add this piece of code at the end of one of the dependency declarations

{
    exclude "sep_approx_spanish.txt"
}

So for example if the two dependencies are these

compile 'com.google.appengine:appengine-endpoints:1.9.28'
compile 'com.google.appengine:appengine-endpoints-deps:1.9.28

if must become

compile 'com.google.appengine:appengine-endpoints:1.9.28'{
        exclude "sep_approx_spanish.txt"
    }
 compile 'com.google.appengine:appengine-endpoints-deps:1.9.28

or

compile 'com.google.appengine:appengine-endpoints:1.9.28'
compile 'com.google.appengine:appengine-endpoints-deps:1.9.28'{
        exclude "sep_approx_spanish.txt"
    }

in this way you tell to gradle to not take into consideration that file for one of the two libraries, avoiding the file's duplication inside the apk.

Answer 2

The error tells you that building your APK would result in two files named com/google/appengine/repackaged/org/apache/commons/codec/language/bm/sep_approx_spanish.txt. This is a failure because the APK can only contain one file at any given path. This is probably caused by two different jar files in your classpath that contain the file. In my project, this only occurred with files that were not actually used at runtime so I excluded them from the APK entirely.