Excluding "provided" scope dependencies during "test" execution phase?

Question

I'm seeing a jar with provided scope in my classpath during test execution. Given the definition of the provided scope, I would not expect that jar to appear on the classpath during any execution phase.

I've confirmed that the scope is correct using the M2E "Dependency Hierarchy" view. I've already tried using the classpathDependencyScopeExclude option in maven-surefire-plugin to keep the dependency from showing up, and it hasn't worked:

<build>
  <pluginManagement>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.14.1</version>
        <configuration>
          <!--
            We always want to exclude provided deps. I'm not sure why this
            isn't the default.
          -->
          <classpathDependencyScopeExclude>provided</classpathDependencyScopeExclude>
        </configuration>
      </plugin>
    </plugins>
  </pluginManagement>
</build>

Here is the dependency definition in question:

Parent POM:

<dependency>
  <groupId>javax.ws.rs</groupId>
  <artifactId>jsr311-api</artifactId>
  <version>1.1.1</version>
</dependency>

Module POM:

<dependency>
  <groupId>javax.ws.rs</groupId>
  <artifactId>jsr311-api</artifactId>
  <scope>provided</scope>
</dependency>

Is it normal that the jar is there? What can I do to keep it from showing up?

EDIT: I've run mvn dependency:tree on the parent POM. Here's the output for the project in question:

[INFO] ------------------------------------------------------------------------
[INFO] Building sibyl.transport.impl.http.server 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ sibyl.transport.impl.http.server ---
[INFO] com.sigpwned.analytics:sibyl.transport.impl.http.server:jar:0.0.1-SNAPSHOT
[INFO] +- com.sigpwned.analytics:sibyl.transport:jar:0.0.1-SNAPSHOT:compile
[INFO] +- com.sigpwned.analytics:commons:jar:0.0.1-SNAPSHOT:compile
[INFO] +- javax.ws.rs:jsr311-api:jar:1.1.1:provided
[INFO] +- com.sun.jersey:jersey-server:jar:1.17.1:test (scope not updated to compile)
[INFO] |  +- asm:asm:jar:3.1:test
[INFO] |  \- com.sun.jersey:jersey-core:jar:1.17.1:test
[INFO] +- com.sun.jersey:jersey-servlet:jar:1.17.1:compile
[INFO] +- org.eclipse.jetty:jetty-servlet:jar:9.0.3.v20130506:test
[INFO] |  \- org.eclipse.jetty:jetty-security:jar:9.0.3.v20130506:test
[INFO] |     \- org.eclipse.jetty:jetty-server:jar:9.0.3.v20130506:test
[INFO] |        +- org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:test
[INFO] |        +- org.eclipse.jetty:jetty-http:jar:9.0.3.v20130506:test
[INFO] |        |  \- org.eclipse.jetty:jetty-util:jar:9.0.3.v20130506:test
[INFO] |        \- org.eclipse.jetty:jetty-io:jar:9.0.3.v20130506:test
[INFO] +- com.sigpwned.analytics:sibyl.transport.impl.http.client:jar:0.0.1-SNAPSHOT:test
[INFO] |  \- org.apache.httpcomponents:httpclient:jar:4.2.5:test
[INFO] |     +- org.apache.httpcomponents:httpcore:jar:4.2.4:test
[INFO] |     +- commons-logging:commons-logging:jar:1.1.1:test
[INFO] |     \- commons-codec:commons-codec:jar:1.6:test
[INFO] \- junit:junit:jar:4.11:test
[INFO]    \- org.hamcrest:hamcrest-core:jar:1.3:test

Again, the jar with provided scope that I expect not to show up on the classpath but does is the JAX-RS spec API, shown above as javax.ws.rs:jsr311-api:jar:1.1.1:provided. I believe this confirms that this dependency only appears in the provided scope.

EDIT: If I run the test, here is the command line that gets run (from ps):

sigpwned       11771   0.0  1.2  4019788 100240   ??  S    10:58PM   0:05.92 /Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:59974 -Dfile.encoding=UTF-8 -classpath /Users/aboothe/Documents/workspaces/w2oanalytics/sibyl.transport.impl.http.server/target/test-classes:/Users/aboothe/Documents/workspaces/w2oanalytics/sibyl.transport.impl.http.server/target/classes:/Users/aboothe/Documents/workspaces/w2oanalytics/sibyl/sibyl.transport/target/classes:/Users/aboothe/Documents/workspaces/w2oanalytics/commons/target/classes:/Users/aboothe/.m2/repository/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar:/Users/aboothe/.m2/repository/com/sun/jersey/jersey-server/1.17.1/jersey-server-1.17.1.jar:/Users/aboothe/.m2/repository/asm/asm/3.1/asm-3.1.jar:/Users/aboothe/.m2/repository/com/sun/jersey/jersey-core/1.17.1/jersey-core-1.17.1.jar:/Users/aboothe/.m2/repository/com/sun/jersey/jersey-servlet/1.17.1/jersey-servlet-1.17.1.jar:/Users/aboothe/.m2/repository/org/eclipse/jetty/jetty-servlet/9.0.3.v20130506/jetty-servlet-9.0.3.v20130506.jar:/Users/aboothe/.m2/repository/org/eclipse/jetty/jetty-security/9.0.3.v20130506/jetty-security-9.0.3.v20130506.jar:/Users/aboothe/.m2/repository/org/eclipse/jetty/jetty-server/9.0.3.v20130506/jetty-server-9.0.3.v20130506.jar:/Users/aboothe/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar:/Users/aboothe/.m2/repository/org/eclipse/jetty/jetty-http/9.0.3.v20130506/jetty-http-9.0.3.v20130506.jar:/Users/aboothe/.m2/repository/org/eclipse/jetty/jetty-util/9.0.3.v20130506/jetty-util-9.0.3.v20130506.jar:/Users/aboothe/.m2/repository/org/eclipse/jetty/jetty-io/9.0.3.v20130506/jetty-io-9.0.3.v20130506.jar:/Users/aboothe/Documents/workspaces/w2oanalytics/sibyl.transport.impl.http.client/target/classes:/Users/aboothe/.m2/repository/org/apache/httpcomponents/httpclient/4.2.5/httpclient-4.2.5.jar:/Users/aboothe/.m2/repository/org/apache/httpcomponents/httpcore/4.2.4/httpcore-4.2.4.jar:/Users/aboothe/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/Users/aboothe/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6.jar:/Users/aboothe/.m2/repository/junit/junit/4.11/junit-4.11.jar:/Users/aboothe/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/aboothe/Downloads/eclipse/configuration/org.eclipse.osgi/bundles/350/1/.cp/:/Users/aboothe/Downloads/eclipse/configuration/org.eclipse.osgi/bundles/349/1/.cp/ org.eclipse.jdt.internal.junit.runner.RemoteTestRunner -version 3 -port 59973 -testLoaderClass org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader -loaderpluginname org.eclipse.jdt.junit4.runtime -classNames com.sigpwned.analytics.sibyl.transport.impl.http.server.HttpServerTransportTests

If you pull out the classpath, here's what that looks like:

/Users/sigpwned/Documents/workspaces/siganalytics/sibyl.transport.impl.http.server/target/test-classes
/Users/sigpwned/Documents/workspaces/siganalytics/sibyl.transport.impl.http.server/target/classes
/Users/sigpwned/Documents/workspaces/siganalytics/sibyl/sibyl.transport/target/classes
/Users/sigpwned/Documents/workspaces/siganalytics/commons/target/classes
/Users/sigpwned/.m2/repository/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar
/Users/sigpwned/.m2/repository/com/sun/jersey/jersey-server/1.17.1/jersey-server-1.17.1.jar
/Users/sigpwned/.m2/repository/asm/asm/3.1/asm-3.1.jar
/Users/sigpwned/.m2/repository/com/sun/jersey/jersey-core/1.17.1/jersey-core-1.17.1.jar
/Users/sigpwned/.m2/repository/com/sun/jersey/jersey-servlet/1.17.1/jersey-servlet-1.17.1.jar
/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-servlet/9.0.3.v20130506/jetty-servlet-9.0.3.v20130506.jar
/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-security/9.0.3.v20130506/jetty-security-9.0.3.v20130506.jar
/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-server/9.0.3.v20130506/jetty-server-9.0.3.v20130506.jar
/Users/sigpwned/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar
/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-http/9.0.3.v20130506/jetty-http-9.0.3.v20130506.jar
/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-util/9.0.3.v20130506/jetty-util-9.0.3.v20130506.jar
/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-io/9.0.3.v20130506/jetty-io-9.0.3.v20130506.jar
/Users/sigpwned/Documents/workspaces/siganalytics/sibyl.transport.impl.http.client/target/classes
/Users/sigpwned/.m2/repository/org/apache/httpcomponents/httpclient/4.2.5/httpclient-4.2.5.jar
/Users/sigpwned/.m2/repository/org/apache/httpcomponents/httpcore/4.2.4/httpcore-4.2.4.jar
/Users/sigpwned/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar
/Users/sigpwned/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6.jar
/Users/sigpwned/.m2/repository/junit/junit/4.11/junit-4.11.jar
/Users/sigpwned/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar
/Users/sigpwned/Downloads/eclipse/configuration/org.eclipse.osgi/bundles/350/1/.cp/
/Users/sigpwned/Downloads/eclipse/configuration/org.eclipse.osgi/bundles/349/1/.cp/

EDIT: After investigating the classpathDependencyScopeExclude documentation, I determined that you can only exclude compile, runtime, and test dependencies, not provided. This explains why that functionality doesn't work. After trying the suggestion from @hello below, I've determined that classpathDependencyExclude works when running directly with maven. Here's the classpath that I ripped out of the surefire jar's MANIFEST:

  file:/Users/sigpwned/.m2/repository/org/apache/maven/surefire/surefire-booter/2.14.1/surefire-booter-2.14.1.jar
  file:/Users/sigpwned/.m2/repository/org/apache/maven/surefire/surefire-api/2.14.1/surefire-api-2.14.1.jar
  file:/Users/sigpwned/Documents/workspaces/siganalytics/sibyl/sibyl.transport.impl.http.server/target/test-classes/
  file:/Users/sigpwned/Documents/workspaces/siganalytics/sibyl/sibyl.transport.impl.http.server/target/classes/
  file:/Users/sigpwned/.m2/repository/com/siggroup/analytics/sibyl.transport/0.0.1-SNAPSHOT/sibyl.transport-0.0.1-SNAPSHOT.jar
  file:/Users/sigpwned/.m2/repository/com/siggroup/analytics/commons/0.0.1-SNAPSHOT/commons-0.0.1-SNAPSHOT.jar
  file:/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-server/9.0.3.v20130506/jetty-server-9.0.3.v20130506.jar
  file:/Users/sigpwned/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar
  file:/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-http/9.0.3.v20130506/jetty-http-9.0.3.v20130506.jar
  file:/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-util/9.0.3.v20130506/jetty-util-9.0.3.v20130506.jar
  file:/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-io/9.0.3.v20130506/jetty-io-9.0.3.v20130506.jar
  file:/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-servlet/9.0.3.v20130506/jetty-servlet-9.0.3.v20130506.jar
  file:/Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-security/9.0.3.v20130506/jetty-security-9.0.3.v20130506.jar
  file:/Users/sigpwned/.m2/repository/com/sun/jersey/jersey-server/1.17.1/jersey-server-1.17.1.jar
  file:/Users/sigpwned/.m2/repository/asm/asm/3.1/asm-3.1.jar
  file:/Users/sigpwned/.m2/repository/com/sun/jersey/jersey-core/1.17.1/jersey-core-1.17.1.jar
  file:/Users/sigpwned/.m2/repository/com/sun/jersey/jersey-servlet/1.17.1/jersey-servlet-1.17.1.jar
  file:/Users/sigpwned/.m2/repository/com/siggroup/analytics/sibyl.transport.impl.http.client/0.0.1-SNAPSHOT/sibyl.transport.impl.http.client-0.0.1-SNAPSHOT.jar
  file:/Users/sigpwned/.m2/repository/org/apache/httpcomponents/httpclient/4.2.4/httpclient-4.2.4.jar
  file:/Users/sigpwned/.m2/repository/org/apache/httpcomponents/httpcore/4.2.4/httpcore-4.2.4.jar
  file:/Users/sigpwned/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar
  file:/Users/sigpwned/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6.jar
  file:/Users/sigpwned/.m2/repository/com/siggroup/analytics/sibyl.transport/0.0.1-SNAPSHOT/sibyl.transport-0.0.1-SNAPSHOT-tests.jar
  file:/Users/sigpwned/.m2/repository/junit/junit/4.11/junit-4.11.jar
  file:/Users/sigpwned/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar

Notice that jsr311-api.jar is not present. Success! Here's the configuration that worked:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>${maven-surefire-plugin.version}</version>
  <configuration>
    <!--
      We never want to include these API deps on an execution classpath.
      I would think this is how the "provided" scope works - just
      compile against the jar, and then I'll bring it myself at
      execution time - but provided is only excluded during the runtime
      phase, not the testing phase. So, we have to exclude all our
      API JARs by hand.
    -->
    <classpathDependencyExcludes>
      <classpathDependencyExclude>javax.ws.rs:jsr311-api</classpathDependencyExclude>
    </classpathDependencyExcludes>
    <includes>
      <include>**/*Tests.*</include>
    </includes>
  </configuration>
</plugin>

However, this solution does not seem to work on M2E. Here's the classpath I've pulled out from a ps during test execution using M2E:

  /Users/sigpwned/Documents/workspaces/siganalytics/sibyl/sibyl.transport.impl.http.server/target/test-classes
  /Users/sigpwned/Documents/workspaces/siganalytics/sibyl/sibyl.transport.impl.http.server/target/classes
  /Users/sigpwned/Documents/workspaces/siganalytics/sibyl/sibyl.transport/target/classes
  /Users/sigpwned/Documents/workspaces/siganalytics/commons/target/classes
> /Users/sigpwned/.m2/repository/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar
  /Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-server/9.0.3.v20130506/jetty-server-9.0.3.v20130506.jar
  /Users/sigpwned/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar
  /Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-http/9.0.3.v20130506/jetty-http-9.0.3.v20130506.jar
  /Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-util/9.0.3.v20130506/jetty-util-9.0.3.v20130506.jar
  /Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-io/9.0.3.v20130506/jetty-io-9.0.3.v20130506.jar
  /Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-servlet/9.0.3.v20130506/jetty-servlet-9.0.3.v20130506.jar
  /Users/sigpwned/.m2/repository/org/eclipse/jetty/jetty-security/9.0.3.v20130506/jetty-security-9.0.3.v20130506.jar
  /Users/sigpwned/.m2/repository/com/sun/jersey/jersey-server/1.17.1/jersey-server-1.17.1.jar
  /Users/sigpwned/.m2/repository/asm/asm/3.1/asm-3.1.jar
  /Users/sigpwned/.m2/repository/com/sun/jersey/jersey-core/1.17.1/jersey-core-1.17.1.jar
  /Users/sigpwned/.m2/repository/com/sun/jersey/jersey-servlet/1.17.1/jersey-servlet-1.17.1.jar
  /Users/sigpwned/Documents/workspaces/siganalytics/sibyl/sibyl.transport.impl.http.client/target/classes
  /Users/sigpwned/.m2/repository/org/apache/httpcomponents/httpclient/4.2.4/httpclient-4.2.4.jar
  /Users/sigpwned/.m2/repository/org/apache/httpcomponents/httpcore/4.2.4/httpcore-4.2.4.jar
  /Users/sigpwned/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar
  /Users/sigpwned/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6.jar
  /Users/sigpwned/Documents/workspaces/siganalytics/sibyl/sibyl.transport/target/test-classes
  /Users/sigpwned/.m2/repository/junit/junit/4.11/junit-4.11.jar
  /Users/sigpwned/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar
  /Users/sigpwned/Downloads/eclipse/configuration/org.eclipse.osgi/bundles/350/1/.cp/
  /Users/sigpwned/Downloads/eclipse/configuration/org.eclipse.osgi/bundles/349/1/.cp/

Unfortunately, jsr311-api.jar is still there. This leads me to believe that this question is answered (with that answer being "use classpathDependencyExcludes"), but there's a bug in M2E. I ran "Check for Updates" in Eclipse and found no update for M2E, so I should be running the latest stable release. I'll file a bug on M2E and update with the issue here so anyone who finds this issue can track.

EDIT: This issue was already raised with M2E, and the answer appears to be "Surefire integration is outside the scope of M2E at this time." Not encouraging. :/

Answer 1

Given the definition of the provided scope, I would not expect that jar to appear on the classpath during any execution phase.

This is not quite the behaviour of provided scope; see Dependency Scope. provided scope, in common with compile, is for classes that are necessary to compile and test your classes. If they were missing during test then you would have a NoClassDefFoundError.

The difference comes during packaging and deployment, where these artifacts are omitted from wars under the assumption that your target environment (for example, a servlet container) will provide its own copy of the same classes.