Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS Lambda Jar unable to zip after adding selenium dependencies in pom

This is a weird error. After adding the selenium dependencies to the pom of my maven project and upload it to a lambda, it says it is unable to unzip the file. However after removing the dependencies, the lambda is able to unzip the file just fine (however it comes up with a class not found afterwards). I have tried removing the dependencies one by one but each one triggers the error.

Any ideas on how to solve this?

Class not found error

org/openqa/selenium/WebDriver: java.lang.NoClassDefFoundError
java.lang.NoClassDefFoundError: org/openqa/selenium/WebDriver

lambda cannot zip error

Calling the invoke API action failed with this message: Lambda was not able to unzip the file

The dependencies causing the issue

    <dependency>
        <groupId>org.seleniumhq.webdriver</groupId>
        <artifactId>webdriver-common</artifactId>
        <version>0.9.7376</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-chrome-driver</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>3.141.59</version>
    </dependency>

updated dependancies (for Vishal)

    <dependency>
        <groupId>org.seleniumhq.webdriver</groupId>
        <artifactId>webdriver-common</artifactId>
        <version>0.9.7376</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-chrome-driver</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-api</artifactId>
        <version>2.0rc2</version>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-remote-driver</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-support</artifactId>
        <version>3.141.59</version>
    </dependency>

Configuration

 <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.6.0</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
          <encoding>UTF-8</encoding>
          <forceJavacCompilerUse>true</forceJavacCompilerUse>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
like image 989
Meerfall the dewott Avatar asked Oct 18 '19 08:10

Meerfall the dewott


3 Answers

The shade plugin combines all dependencies with the developed code and plops them in one Uber JAR. The downside is that it can overwrite resource files, and doesn't play well with signed jars (in my experience at least).

I would recommend moving away from the shade plugin if at all possible.

That said, if you have to use it - you're issue may be with the combining the jar resources. There are many transformers that you can use to resolve this, and you'll need to investigate which one is really needed. I would start with something like this

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>2.4.3</version>
    <configuration>
        <shadedArtifactAttached>true</shadedArtifactAttached>
        <shadedClassifierName>${executable.classifier}</shadedClassifierName>
        <filters>
            <filter>
                <artifact>*:*</artifact>
                <excludes>
                    <exclude>META-INF/*.SF</exclude>
                    <exclude>META-INF/*.DSA</exclude>
                    <exclude>META-INF/*.RSA</exclude>
                </excludes>
            </filter>
        </filters>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>shade</goal>
            </goals>
            <configuration>
                <transformers>
                    <transformer
                        implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
                    <transformer
                        implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                        <mainClass>fully.qualified.ClassName</mainClass>
                    </transformer>
                </transformers>
            </configuration>
        </execution>
    </executions>
</plugin>

You can find more tranformers on the Apache plugin here

The alternative that I would suggest is Spring Boot, which uses the Jar-in-Jar structure with a custom ClassLoader to load classes from the internal jar(s).

This is the easier method due to not having to re-write files as the Shade plugin approach and it handles the dependencies a little better.

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <version>1.3.6.RELEASE</version>
    <configuration>
        <classifier>${executable.classifier}</classifier>
        <layout>ZIP</layout>
        <mainClass>fully.qualified.ClassName</mainClass>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>repackage</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Seriously, look at the simpler configuration!

NOTE: Most of this came from my own notes - version numbers may be a little old...

like image 112
Jason Warner Avatar answered Oct 16 '22 02:10

Jason Warner


Try to tell your dependencies to output zip, maybe jar is messing up with things

Add this to maven-assembly-plugin configuration:

     <formats>
        <format>zip</format>
     </formats>

For example:

<plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-assembly-plugin</artifactId>
     ...
     <configuration>
         ...
         <formats>
            <format>zip</format>
         </formats>
     </configuration>
</plugin>

Same is suggested here

like image 45
Carlos Robles Avatar answered Oct 16 '22 02:10

Carlos Robles


I figured it out. The java selenium seemed to cause the major issue. Downgrading to 3.10 fixed the issue although I have no idea why.

like image 35
Meerfall the dewott Avatar answered Oct 16 '22 01:10

Meerfall the dewott