Gradle Android Build System NDK issues

Question

I'll be the first to admit that I'm not terribly expert with Gradle and the new Android build system but, unfortunately, I had to move over to it (from ant) because of issue 21479 (https://code.google.com/p/android/issues/detail?id=21479) and the comment "This will not be fixed. We are focusing on finishing the Gradle-based build system which will replace Ant." Unfortunately, I couldn't get the stuff to build again after I added the Millenium Media advertising library. This was in addition to the Android OpenCV library, Chilkat's encryption library and the support v4 library but the MMedia library was the one that broke it all.

So, I thought, good a reason as any to migrate to the new Gradle based build system. Unfortunately, despite leaving a broken ant build system, the new system doesn't appear to be finished yet; particularly the ndk support.

The application I'm working on uses OpenCV for some of the image processing and a couple of the operations run way too slowly unless they're compiled to native (LOTS of memory moves and compares which are too slow across the JNI boundary). So, rather than trying to pass the data back and forward between the VM and native code, I leave all this stuff on the native side and just have one call from the VM to get the results.

The first problem was getting the ndk stuff to compile. I couldn't get the settings in the ndk closure to work so I had to resort to using the ndk-build command and execute this as a task:

task ndkBuild(type: Exec) {
    String MainDirectory = System.getProperty("user.dir") + '/app/src/main'
    println "Main app directory for NDK build " + MainDirectory
    if (Os.isFamily(Os.FAMILY_WINDOWS)) {
       commandLine 'gradle-ndk-build.cmd', MainDirectory, '-j'
    }
    else {
       commandLine 'gradle-ndk-build', MainDirectory, '-j'
    }
}

tasks.withType(JavaCompile) {
    compileTask -> compileTask.dependsOn ndkBuild
}

This works perfectly; it compiles the ndk code and generates the .so library without error. Unfortunately, it won't put the resulting .so file in the final package. It puts all the other native libraries in okay, but not this one - no idea why though.

I've found lots of alleged fixes for this problem such as:

tasks.withType(com.android.build.gradle.tasks.PackageApplication) { pkgTask ->
    pkgTask.jniFolders = new HashSet<File>()
    pkgTask.jniFolders.add(new File(buildDir, 'native-libs'))
}

But adding this just results in an apk file with NO native libraries in at all. I've seen other people have the same problem (such as https://groups.google.com/forum/#!msg/adt-dev/QbDHM41QT2E/J4jHCC_RuIEJ) but I've tried all the proposed solutions and none of them work for me.

As I don't change the native code very often, I've just done a hack to copy the generated native library (libndklib.so) from app/src/main/libs to /app/src/main/jni after it's been compiled; then it ends up in the apk package. Obviously, this is a bit nasty as, if anyone takes this code over, they'll be wondering why their changes to the native code never appear in the app.

So, my questions are: Is there something I can run within the gradle script that will execute after I've run the ndk command (gradle-ndk-build) that will copy the generated file(s) from app/src/main/libs/armeabi/libndklib.so to /app/src/main/jni/armeabi/libndklib.so (for each of the architectures - armeabi, armeabi-v7, x86, mips) so it ends up in the apk package?

OR

Is there any way to make the Gradle ndk closure handle the following ndk make files properly: Application.mk

APP_STL := gnustl_static
APP_CPPFLAGS := -frtti -fexceptions
APP_ABI := all
APP_PLATFORM := android-8

Android.mk

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
# OpenCV
OPENCV_CAMERA_MODULES:=on
OPENCV_INSTALL_MODULES:=on
OPENCV_LIB_TYPE:=SHARED
include /home/myname/tools/OpenCV-2.4.8-android-sdk/sdk/native/jni/OpenCV.mk
LOCAL_MODULE    := ndklib
LOCAL_SRC_FILES := ndklib.cpp motion.cpp
LOCAL_LDLIBS +=  -lm -llog
include $(BUILD_SHARED_LIBRARY)
# Add prebuilt chilkat library
include $(CLEAR_VARS)
LOCAL_MODULE := lib-chilkat
LOCAL_SRC_FILES := $(TARGET_ARCH_ABI)/libchilkatemail.so
include $(PREBUILT_SHARED_LIBRARY)

I looked at the Gradle source for the plug-in but I couldn't see many of these directives supported.

OR

Can I just add some sort of hack that runs at the end of the Gradle script that just forces the appropriate copy of libndklib.so (for the correct architecture) into the generated apk? I can live with the latter until the ndk stuff is finished for the Android plug-in for the gradle build.

=======================================

Edit - After ph0b's answer This is the final build.gradle file with the proposed mod in it. Creates the .apk's perfectly build.gradle (in app directory)

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:0.9.+'
    }
}

apply plugin: 'android'

import org.apache.tools.ant.taskdefs.condition.Os

android {
    compileSdkVersion 19
    buildToolsVersion "19.0.3"

    signingConfigs {
        debug {
            storeFile file("dbgkeystore")
            storePassword "nopass"
            keyAlias "mainkeyname"
            keyPassword "nopass"
        }

        release {
            storeFile file("keystore")
            storePassword "xxxxxxxx"
            keyAlias "mainkeyname"
            keyPassword "yyyyyyyy"
        }
    }

    // Autoincrement the version properties file
    // ******************************************
    def versionPropsFile = file('version.properties')
    def code = 1

    def majorversion = 1
    def minorversion = 1

    defaultConfig {
        versionCode code
        versionName "${majorversion}.${minorversion}.${code}"
        minSdkVersion 10
        targetSdkVersion 19
    }

    buildTypes {
        release {
            runProguard true
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
            signingConfig signingConfigs.release
        }

        debug {
           runProguard false
           proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
           packageNameSuffix ".debug"
           versionNameSuffix "-debug"
           signingConfig signingConfigs.debug
        }
    }

    sourceSets {
        main {
            jni.srcDirs = []
            // jniLibs.srcDir 'src/main/jni' // - Doesn't work, leaves out the .so files generated by ndk-build
            jniLibs.srcDir 'src/main/libs'
        }
    }

    flavorDimensions "version", "abi"

    productFlavors {
        pro {
            flavorDimension "version"
            packageName "org.somedomainname.myAppPro1"
        }

        lite {
            flavorDimension "version"
            packageName "org.somedomainname.myAppLite1"
        }

        arm {
            flavorDimension "abi"

            ndk {
                abiFilter "armeabi"
            }

            if (versionPropsFile.canRead()){
                def Properties versionProps = new Properties()

                versionProps.load(new FileInputStream(versionPropsFile))

                code = versionProps['VERSION_CODE'].toInteger() + 1

                versionProps['VERSION_CODE']=code.toString()
                versionProps.store(versionPropsFile.newWriter(), null)

                versionCode code
                versionName "${majorversion}.${minorversion}.${code}"
            }
            else {
                throw new GradleException("Could not read version.properties!")
            }

        }

        armv7 {
            flavorDimension "abi"

            ndk {
                abiFilter "armeabi-v7a"
            }

            if (versionPropsFile.canRead()){
                def Properties versionProps = new Properties()

                versionProps.load(new FileInputStream(versionPropsFile))

                code = versionProps['VERSION_CODE'].toInteger() + 1

                versionProps['VERSION_CODE']=code.toString()
                versionProps.store(versionPropsFile.newWriter(), null)

                versionCode code
                versionName "${majorversion}.${minorversion}.${code}"
            }
            else {
                throw new GradleException("Could not read version.properties!")
            }
        }

        x86 {
            flavorDimension "abi"

            ndk {
                abiFilter "x86"
            }

            if (versionPropsFile.canRead()){
                def Properties versionProps = new Properties()

                versionProps.load(new FileInputStream(versionPropsFile))

                code = versionProps['VERSION_CODE'].toInteger() + 1

                versionProps['VERSION_CODE']=code.toString()
                versionProps.store(versionPropsFile.newWriter(), null)

                versionCode code
                versionName "${majorversion}.${minorversion}.${code}"
            }
            else {
                throw new GradleException("Could not read version.properties!")
            }
        }
    }

    lintOptions {
        checkReleaseBuilds false
        // Or, if you prefer, you can continue to check for errors in release builds,
        // but continue the build even when errors are found:
        abortOnError false
    }

repositories {
    mavenCentral()
    flatDir {
        dirs '/home/myname/maindrive/work/dynamic/android/UtilLib/aarlib'
    }
}

dependencies {
    compile 'com.android.support:appcompat-v7:+'
    compile fileTree(dir: 'libs', include: ['*.jar'])

    // Note: org.somedomainname.UtilLib on the depency below is ignored when usng flatdir
    compile 'org.somedomainname.UtilLib:library:1.0.0@aar'
}

task ndkBuild(type: Exec) {
    String MainDirectory = System.getProperty("user.dir") + '/app/src/main'
    println '************************************************************************'
    println "Main app directory for NDK build " + MainDirectory
    println '************************************************************************'

    if (Os.isFamily(Os.FAMILY_WINDOWS)) {
       commandLine 'gradle-ndk-build.cmd', MainDirectory, '-j'
    }
    else {
       commandLine 'gradle-ndk-build', MainDirectory, '-j'
    }
}

tasks.withType(JavaCompile) {
    compileTask -> compileTask.dependsOn ndkBuild
}


android.applicationVariants.all { variant ->
    variant.assemble.doLast {
           rename_and_moveout_apk(variant)
    }
}

// allprojects {
//     tasks.withType(Compile) {
//         options.compilerArgs << "-Xlint:deprecation"
//     }
// }

def rename_and_moveout_apk(targetVariant) {
    // replace output apk name to <product>-<version>-<buildtype>-<githash>.apk
    def versionSuffix = targetVariant.buildType.versionNameSuffix ? targetVariant.buildType.versionNameSuffix : ""
    def versionName = targetVariant.mergedFlavor.versionName + versionSuffix;

    if (targetVariant.zipAlign) {
        def apkFinal = targetVariant.outputFile;
        def apkFinalNewName = "myApp-" + apkFinal.name.replace(targetVariant.buildType.name, versionName);
        copy {
            from "$apkFinal"
            into "$rootProject.projectDir/apk_release"
            rename ("$apkFinal.name", "$apkFinalNewName")
            println "*************** Renaming zipalign apk file from: ${apkFinal.name} to ${apkFinalNewName}"
        }
    }

}

gradle-ndk-build (modified version of ndk-build used for debugging the parameters)

#!/bin/bash
export NDK_PROJECT_PATH=$1
export NDK_PROJECT_DIRECTORY=$1
bash -c "ndk-build"

Directory Structure

------ apk_release
------ app
-- -- ---- src
--     ------ lite
--     --  -- ---- java
--     --      -- ---- org
--     --          -- ---- somedomainname
--     --              -- ---- myApp
--     ------ main
--     --  ------ assets
--     --  ------ java
--     --  --  -- ---- org
--     --  --      ------ chilkatsoft
--     --  --      -- ---- somedomainname
--     --  --          -- ---- myApp
--     --  ------ jni
--     --  --  ------ armeabi
--     --  --  ------ armeabi-v7a
--     --  --  ------ mips
--     --  --  -- ---- x86
--     --  ------ libs
--     --  --  ------ armeabi
--     --  --  ------ armeabi-v7a
--     --  --  ------ mips
--     --  --  -- ---- x86
--     --  ------ obj
--     --  --  -- ---- local
--     --  --      ------ armeabi
--     --  --      --  -- ---- objs
--     --  --      --      -- ---- ndklib
--     --  --      ------ armeabi-v7a
--     --  --      --  -- ---- objs
--     --  --      --      -- ---- ndklib
--     --  --      ------ mips
--     --  --      --  -- ---- objs
--     --  --      --      -- ---- ndklib
--     --  --      -- ---- x86
--     --  --          -- ---- objs
--     --  --              -- ---- ndklib
--     --  -- ---- res
--     --      ------ drawable
--     --      ------ drawable-hdpi
--     --      ------ drawable-ldpi
--     --      ------ drawable-mdpi
--     --      ------ drawable-xhdpi
--     --      ------ drawable-xxhdpi
--     --      ------ layout
--     --      ------ raw
--     --      ------ values
--     --      -- ---- xml
--     -- ---- pro
--         -- ---- java
--             -- ---- somedomainname
--                 -- ---- myApp

Answer 1

gradle will automatically look for .so files inside jniLibs/ABI/.

You can change this behavior to make it use your regular libs directory by setting it inside your build.gradle file:

android {
    sourceSets.main {
        jniLibs.srcDir 'src/main/libs'
    }
}