How come maven dependencies in POM file can't be used?

Question

Background

I'm tasked to provide a way for developers to add a single dependency into build.gradle file on Android project, to use some (obfuscated) SDK I've prepared (AAR file - Android library).

The SDK itself uses various dependencies, so it's important that they will be available in the final code, meaning the whoever uses the SDK won't see crashes due to class-not-found.

The SDK is stored on Github as a private repository, and available for developers via Jitpack, which scans POM file and some other files on the repository.

Using the AAR file alone, you have to set the same (or newer) dependencies as on the SDK.

However, if you use POM file while publishing, the consumer (whoever uses the SDK) shouldn't need to write each dependency that's on the SDK, as it gets added from the POM file.

The problem

For some reason, this behavior doesn't exist for me. Meaning that it's as if the SDK doesn't use any depdendency, so if I try to use any of the depdendencies I can't reach the classes, and if I try to use the SDK when it uses some depdendency, it causes an exception of ClassNotFoundException.

At first I thought it's because of some Proguard rules, but then I noticed that it happens on debug-variant too (of the app that uses the SDK).

So for example, as the library uses this in the dependencies:

implementation 'androidx.security:security-crypto:1.1.0-alpha03'

And if I don't use the same, I can't reach any class of this dependency, and if the SDK tries to reach any class of it, the app will crash.

So instead of a single dependency of using the SDK and that's it, I have to add all of the dependencies that the SDK already uses.

What I've tried

To generate the AAR file, I use "assembleRelease" gradle task, and to generate the POM file, I use the "generatePomFileForReleasePublication" (or "publishReleasePublicationToMavenLocal") gradle task.

The library has quite some dependencies. This is the build.gradle file of it:

plugins {
    id 'com.android.library'
    id 'kotlin-android'
    id 'kotlin-kapt'
    id 'kotlin-parcelize'
    id 'io.michaelrocks.paranoid'
    id 'maven-publish'
    id 'com.github.dcendents.android-maven'
}

android {
    compileSdkVersion 30
    defaultConfig {
        minSdkVersion 21
        targetSdkVersion 30
        versionCode 1
        versionName "1.0"
    }

    buildTypes {
        release {
            minifyEnabled true
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
            consumerProguardFiles 'consumer-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    kotlinOptions {
        jvmTarget = '1.8'
    }
    configurations {
        all {
            exclude module: 'httpclient'
        }
    }
    packagingOptions {
        //for google login token
        exclude 'META-INF/DEPENDENCIES'
    }
}

dependencies {
    api "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
    api 'androidx.core:core-ktx:1.6.0-alpha01'
    api 'androidx.collection:collection-ktx:1.2.0-alpha01'
    api 'androidx.security:security-crypto:1.1.0-alpha03'
    def room_version = "2.2.6"
    api "androidx.room:room-runtime:$room_version"
    kapt "androidx.room:room-compiler:$room_version"
    api "androidx.room:room-ktx:$room_version"
    api 'com.squareup.retrofit2:retrofit:2.9.0'
    api 'com.squareup.retrofit2:converter-gson:2.9.0'
    api 'com.squareup.okhttp3:okhttp:5.0.0-alpha.2'
    api 'com.squareup.okhttp3:logging-interceptor:5.0.0-alpha.2'
    api('com.google.android.gms:play-services-auth:19.0.0')
    api('com.google.auth:google-auth-library-oauth2-http:0.25.0')
    api('com.google.apis:google-api-services-people:v1-rev99-1.22.0') {
        exclude group: 'com.google.guava'
    }
    api 'io.michaelrocks:libphonenumber-android:8.12.20'
}

afterEvaluate {
    publishing {
        publications {
            release(MavenPublication) {
                from components.release
            }
        }
    }
}

Running the task of "generatePomFileForReleasePublication" I got the POM file "pom-default.xml" :

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <!-- This module was also published with a richer model, Gradle metadata,  -->
  <!-- which should be used instead. Do not delete the following line which  -->
  <!-- is to indicate to Gradle or any Gradle module metadata file consumer  -->
  <!-- that they should prefer consuming it instead. -->
  <!-- do_not_remove: published-with-gradle-metadata -->
  <modelVersion>4.0.0</modelVersion>
  <groupId>MySdk</groupId>
  <artifactId>library</artifactId>
  <version>unspecified</version>
  <packaging>aar</packaging>
  <dependencies>
    <dependency>
      <groupId>org.jetbrains.kotlin</groupId>
      <artifactId>kotlin-stdlib</artifactId>
      <version>1.4.32</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>androidx.core</groupId>
      <artifactId>core-ktx</artifactId>
      <version>1.6.0-alpha01</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>androidx.collection</groupId>
      <artifactId>collection-ktx</artifactId>
      <version>1.2.0-alpha01</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>androidx.security</groupId>
      <artifactId>security-crypto</artifactId>
      <version>1.1.0-alpha03</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>androidx.room</groupId>
      <artifactId>room-runtime</artifactId>
      <version>2.2.6</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>androidx.room</groupId>
      <artifactId>room-ktx</artifactId>
      <version>2.2.6</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.squareup.retrofit2</groupId>
      <artifactId>retrofit</artifactId>
      <version>2.9.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.squareup.retrofit2</groupId>
      <artifactId>converter-gson</artifactId>
      <version>2.9.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.squareup.okhttp3</groupId>
      <artifactId>okhttp</artifactId>
      <version>5.0.0-alpha.2</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.squareup.okhttp3</groupId>
      <artifactId>logging-interceptor</artifactId>
      <version>5.0.0-alpha.2</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.google.android.gms</groupId>
      <artifactId>play-services-auth</artifactId>
      <version>19.0.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.google.auth</groupId>
      <artifactId>google-auth-library-oauth2-http</artifactId>
      <version>0.25.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.google.apis</groupId>
      <artifactId>google-api-services-people</artifactId>
      <version>v1-rev99-1.22.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
        <exclusion>
          <artifactId>*</artifactId>
          <groupId>com.google.guava</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>io.michaelrocks</groupId>
      <artifactId>libphonenumber-android</artifactId>
      <version>8.12.20</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.jetbrains.kotlin</groupId>
      <artifactId>kotlin-parcelize-runtime</artifactId>
      <version>1.4.32</version>
      <scope>runtime</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>io.michaelrocks</groupId>
      <artifactId>paranoid-core</artifactId>
      <version>0.3.3</version>
      <scope>runtime</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
  </dependencies>
</project>

According to various websites, this seems valid and good, but sadly as I wrote, when I use it, it's as if none of these dependencies exist, and I need to declare them all on the consumer side.

In Jitpack's file ("jitpack.yml" ) , I have something like this:

install: 
  - FILE="-Dfile=library-release.aar" 
  - mvn install:install-file $FILE -DgroupId=my-sdk -DartifactId=MySdk -Dversion=1.0 -Dpackaging=aar -DgeneratePom=true

What I tried:

1. I tried using either "implementation" and "api". Didn't help.
2. I tried to remove the scope from all of the dependency tags. Didn't help.
3. I tried adding transitive = true on consumer side, but this didn't do anything, probably because it's as such by default anyway.
4. Removing the POM file showed no warning on Jitpack, showing that it might not even check it out.
5. I've noticed the POM file mentions "This module was also published with a richer model, Gradle metadata, which should be used instead", so I tried finding this file, but couldn't. Running publishToMavenLocal task, I got another file ("module.json"), and even putting it instead of just pom-default.xml file alone (and also as a complete replacement), it didn't help.
6. I thought that there is some inconsistency between the file of "jitpack.yml" and the gradle file, so I've set on both of them the same values for "groupId","artifactId", and "version".

The questions

1. How come the dependencies are ignored on consumer side? How can I fix this? It's probably something very simple that I've missed...
2. Which files exactly does Jitpack need to have? Which are mandatory? Which are optional?
3. Would it also be possible to avoid the part of packagingOptions {exclude 'META-INF/DEPENDENCIES' } on the consumer's side ?
4. Bonus: I've noticed that JavaDocs (or Kdocs in my case, as I wrote everythign in Kotlin) are also completely ignored. How can I make these reachable and readable too?

Answer 1

OK I've found the way to do this. The answer came from Jitpack support itself. First, the quote from them:

This is caused by the install command and ' -DgeneratePom=true' parameter. It will generate a new pom file but it doesn't know anything about dependencies.

If you already have a pom file with all the dependencies then you could add it to the GitHub repository. Instead of ' -DgeneratePom=true' you could use '-DpomFile='

We recommend trying that command locally before running on JitPack.

The other option is to put all the source code of the library on JitPack and build from source.

And indeed, if I use this (and have "api" in the library's dependencies) I got most of my questions answered:

1. It fixed the dependencies issue (and yes, you don't have to customize the -DgroupId=my-sdk -DartifactId=MySdk -Dversion=1.0 ) :

install: 
  - FILE="-Dfile=library-release.aar" 
  - mvn install:install-file $FILE -Dpackaging=aar -DpomFile=pom-default.xml

2. The files that are currently there seem to be just those: jitpack.yml, library-release.aar, pom-default.xml .

3. Apparently it's not needed now. Probably fixed by the same thing that fixed the dependencies issue.

That being said, now I have these questions about it:

1. What's with the recommendation inside "pom-default.xml" to use a different file ("This module was also published with a richer model, Gradle metadata, which should be used instead.") ? Where is it? Is it perhaps the "module.json" file? I tried to use it instead, but then Jitpack got an error with it.

2. Where is the documentation of Jitpack of this use case? They have something about private repositories, but not about AAR files. In fact there is nothing I could see about the "jitpack.yml" file.

3. What's the purpose of customizing the -DgroupId=my-sdk -DartifactId=MySdk -Dversion=1.0 part? Either in the "jitpack.yml" file or in the gradle file of the library? I don't think they do anything at all... It seems the Github repository itself is the one that determines to the consumer how to use it.

4. How can I also add JavaDocs and Kdocs (of Kotlin) to be used here?

Even though this is the main answer for my questions, I will grant the bounty for whoever answers me well for the rest :)