java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V

Question

My spark project run correctly (2.4.0), but when I add the next dependecy :

            <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-streaming_2.12</artifactId>
                <version>2.4.2</version>
                <scope>provided</scope>
             </dependency>

I have next stack trace :

Exception in thread "main" java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V
    at scala.xml.Null$.<init>(Null.scala:23)
    at scala.xml.Null$.<clinit>(Null.scala)
    at org.apache.spark.ui.jobs.AllJobsPage.<init>(AllJobsPage.scala:43)
    at org.apache.spark.ui.jobs.JobsTab.<init>(JobsTab.scala:45)
    at org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:61)
    at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:80)
    at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:175)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:444)
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926)
    at package.SparkUtils.initSession(SparkUtils.java:23)
    at package.MainClass.main(MainClass.java:80)

Answer 1

Scala versión 2.12 is not compatible with 2.11, so make sure that the spark core scala versión is used con al other dependencies.

From above it looks tour streaming library is using scala 2.12 and another dependency is not using 2.12