spark-streaming tutorials

How many RDDs does DStream generate for a batch interval?

Dec 26, 2016

apache-spark spark-streaming

Spark streaming checkpoints for DStreams

Nov 20, 2022

apache-spark spark-streaming checkpointing

Error: Could not find or load main class org.test.spark.streamExample

Feb 05, 2022

spark-streaming scala-ide

Is foreachRDD executed on the Driver?

Oct 08, 2019

apache-spark spark-streaming

what is exact difference between Spark Transform in DStream and map.?

Oct 26, 2022

apache-spark spark-streaming

Use schema to convert AVRO messages with Spark to DataFrame

Sep 21, 2022

scala apache-spark apache-kafka spark-streaming avro

Spark Streaming + Kafka: SparkException: Couldn't find leader offsets for Set

Oct 06, 2022

apache-spark apache-kafka spark-streaming

How to save latest offset that Spark consumed to ZK or Kafka and can read back after restart

Sep 20, 2022

apache-spark apache-kafka spark-streaming kafka-consumer-api

Spark Streaming get warn "replicated to only 0 peer(s) instead of 1 peers"

May 19, 2021

java apache-spark streaming twitter4j spark-streaming

In Apache Spark, how to convert a slow RDD/dataset into a stream?

Sep 19, 2022

scala apache-spark apache-spark-sql spark-streaming

Spark Streaming: foreachRDD update my mongo RDD

Dec 08, 2019

mongodb apache-spark spark-streaming

Spark Streaming mapWithState seems to rebuild complete state periodically

Sep 19, 2022

scala apache-spark spark-streaming

How to fix "java.io.NotSerializableException: org.apache.kafka.clients.consumer.ConsumerRecord" in Spark Streaming Kafka Consumer?

Jan 07, 2022

apache-spark serialization apache-kafka spark-streaming

What is the correct way to start/stop spark streaming jobs in yarn?

Sep 29, 2022

hadoop apache-spark spark-streaming hadoop-yarn cloudera

spark ssc.textFileStream is not streamining any files from directory

Oct 19, 2018

filesystems apache-spark spark-streaming data-stream

How to specify which java version to use in spark-submit command?

Aug 30, 2022

java hadoop-yarn spark-streaming

SBT Test Error: java.lang.NoSuchMethodError: net.jpountz.lz4.LZ4BlockInputStream

Jun 25, 2022

scala apache-spark sbt spark-streaming scalatest

Amazon s3a returns 400 Bad Request with Spark

Nov 16, 2022

amazon-web-services amazon-s3 apache-spark hdfs spark-streaming

Spark off heap memory leak on Yarn with Kafka direct stream

Sep 06, 2020

apache-spark spark-streaming hadoop-yarn apache-spark-1.4

New posts in spark-streaming