Object streaming is not a member of package org.apache.spark

Question

I'm trying to compile a simple scala program and I'm using StreamingContext , here is a snippet of my code:

import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.scheduler.SparkListener
import org.apache.spark.scheduler.SparkListenerStageCompleted
import org.apache.spark.streaming.StreamingContext._ //error:object streaming is not a member of package org.apache.spark
object FileCount {
    def main(args: Array[String]) {
    val conf = new SparkConf()
    .setAppName("File Count")
    .setMaster("local")

    val sc = new SparkContext(conf)
    val textFile = sc.textFile(args(0))
    val ssc = new StreamingContext(sc, Seconds(10)) //error : not found: type StreamingContext
    sc.stop()    
  }
}

I have these two error:

object streaming is not a member of package org.apache.spark

and

not found: type StreamingContext

any help please !!

Answer 1

You'll need to add the dependency of spark-streaming into your build manager.

Answer 2

If you are using sbt, add the following library dependencies:

libraryDependencies += "org.apache.spark" %% "spark-streaming" % "2.1.0" % "provided"

If you are using maven, add the below to pom.xml

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming_2.11</artifactId>
    <version>2.1.0</version>
    <scope>provided</scope>
</dependency>