Maven shade plugin warning: we have a duplicate - how to fix?

Question

This is my project POM (link to the paste, so you can right click > save as pom.xml)

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">     <modelVersion>4.0.0</modelVersion>      <groupId>com.zybnet</groupId>     <artifactId>excel-reporter</artifactId>     <version>0.0.1-SNAPSHOT</version>     <packaging>jar</packaging>      <name>mvn1</name>     <url>http://maven.apache.org</url>      <properties>         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>     </properties>      <dependencies>         <dependency>             <groupId>junit</groupId>             <artifactId>junit</artifactId>             <version>3.8.1</version>             <scope>test</scope>         </dependency>         <dependency>             <groupId>log4j</groupId>             <artifactId>log4j</artifactId>             <version>1.2.17</version>         </dependency>         <dependency>             <groupId>org.apache.poi</groupId>             <artifactId>poi</artifactId>             <version>3.8</version>         </dependency>         <dependency>             <groupId>net.sf.jasperreports</groupId>             <artifactId>jasperreports</artifactId>             <version>4.6.0</version>         </dependency>         <dependency>             <groupId>org.codehaus.groovy</groupId>             <artifactId>groovy-all</artifactId>             <version>2.0.0</version>         </dependency>     </dependencies>      <build>         <resources>             <resource>                 <directory>${project.build.sourceDirectory}</directory>             </resource>         </resources>         <plugins>             <plugin>                 <artifactId>maven-jar-plugin</artifactId>                 <configuration>                     <finalName>${artifactId}-${version}-tmp</finalName>                 </configuration>             </plugin>             <plugin>                 <groupId>org.apache.maven.plugins</groupId>                 <artifactId>maven-shade-plugin</artifactId>                 <version>1.7.1</version>                 <executions>                     <execution>                         <phase>package</phase>                         <goals>                             <goal>shade</goal>                         </goals>                         <configuration>                             <filters>                                 <filter>                                     <artifact>*:*</artifact>                                     <excludes>                                         <exclude>META-INF/*.SF</exclude>                                         <exclude>META-INF/*.DSA</exclude>                                         <exclude>META-INF/*.RSA</exclude>                                     </excludes>                                 </filter>                             </filters>                             <transformers>                                 <transformer                                     implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">                                     <mainClass>com.zybnet.Main</mainClass>                                 </transformer>                             </transformers>                         </configuration>                     </execution>                 </executions>             </plugin>         </plugins>     </build>  </project> 

I followed the advice of configuring the default jar plugin as advertised in the FAQ, but still when I run mvn package about 20K warnings are issued. Running mvn clean does not help either.

According to this answer, I could manually exclude some dependencies. However, I don't know if it's the right way, and the dependency tree is rather complex so it's difficult to argue where to start.

I know that these issues are not harmful, but I'm used to treat warnings as something that must be fixed. Moreover, I'm a beginner with Maven, so I'd like to understand what's wrong with my understanding, and how to troubleshoot problems.

(Using maven assembly plugin is not an option here)

Answer 1

Sometimes it happens that the same class definition is found in two or more JAR files (usually these are dependency JARs). In such a case, there is nothing the developer can do except trying to figure out what to manually exclude from the dependencies or from the final artifact (maybe with the help of mvn dependency:tree -Ddetail=true). I opened an issue and submitted a patch, to help the developer with a slightly prettier output like

[WARNING] xml-apis-1.3.02.jar, xmlbeans-2.3.0.jar define 4 overlappping classes: [WARNING]   - org.w3c.dom.TypeInfo [WARNING]   - org.w3c.dom.DOMConfiguration [WARNING]   - org.w3c.dom.DOMStringList [WARNING]   - org.w3c.dom.UserDataHandler [WARNING] maven-shade-plugin has detected that some .class files [WARNING] are present in two or more JARs. When this happens, only [WARNING] one single version of the class is copied in the uberjar. [WARNING] Usually this is not harmful and you can skip these [WARNING] warnings, otherwise try to manually exclude artifacts [WARNING] based on mvn dependency:tree -Ddetail=true and the above [WARNING] output [WARNING] See http://docs.codehaus.org/display/MAVENUSER/Shade+Plugin 

Using this output and that from mvn dependency:tree I added sections like

<dependency>     <groupId>net.sf.jasperreports</groupId>     <artifactId>jasperreports</artifactId>     <version>4.7.0</version>     <exclusions>         <exclusion>             <groupId>org.apache.poi</groupId>             <artifactId>poi-ooxml</artifactId>         </exclusion>         <exclusion>             <groupId>bouncycastle</groupId>             <artifactId>bcmail-jdk14</artifactId>         </exclusion>         <exclusion>             <groupId>bouncycastle</groupId>             <artifactId>bcprov-jdk14</artifactId>         </exclusion>         <exclusion>             <groupId>org.bouncycastle</groupId>             <artifactId>bctsp-jdk14</artifactId>         </exclusion>     </exclusions> </dependency> 

and managed to reduce the number of warnings from several thousands to a couple dozens. Still, this is not perfect. Still, they continue to copypaste classes that lead to name clashes (I can't understand why). Still, this solution is specific to this particular project and can't be easily ported on anything else.