apache-spark tutorials and guides

How to show the scheme (including type) of a parquet file from command line or spark shell?

Mar 29, 2022

scala apache-spark parquet

Starting a single Spark Slave (or Worker)

Aug 16, 2022

apache-spark

How to sum values in an iterator in a PySpark groupByKey()

Jun 01, 2022

python apache-spark iterator pyspark rdd

How to get default property values in Spark

Mar 31, 2022

scala apache-spark apache-spark-sql

How to encode categorical features in Apache Spark

Sep 07, 2022

scala apache-spark apache-spark-mllib apache-spark-1.2

Output Dstream of Apache Spark in Python

Mar 26, 2022

python apache-spark apache-kafka spark-streaming

How to submit a Scala job to Spark?

Jun 24, 2021

scala apache-spark hadoop-yarn

Yarn container is running out of memory

May 18, 2022

java hadoop apache-spark cloudera hadoop-yarn

Apache Spark: How do I convert a Spark DataFrame to a RDD with type RDD[(Type1,Type2, ...)]?

Nov 02, 2022

scala apache-spark

Error when creating a StreamingContext

Nov 02, 2022

apache-spark spark-streaming

Register UDF to SqlContext from Scala to use in PySpark

Aug 23, 2018

scala apache-spark pyspark user-defined-functions apache-zeppelin

pandas str.contains in pyspark dataframe in Pyspark

Feb 19, 2019

apache-spark pyspark

How to define Kafka (data source) dependencies for Spark Streaming?

Apr 15, 2022

apache-spark sbt spark-streaming spark-streaming-kafka

Spark 2.0 DataSets groupByKey and divide operation and type safety

Aug 17, 2019

scala apache-spark apache-spark-sql apache-spark-dataset

SPARK, DataFrame: difference of Timestamp columns over consecutive rows

Jan 17, 2019

apache-spark spark-dataframe

spark kafka producer serializable

Jun 13, 2022

scala apache-spark kafka-producer-api

SPARK: YARN kills containers for exceeding memory limits

Jul 07, 2019

apache-spark hadoop-yarn

Sort by dateTime in scala

Jan 20, 2022

scala apache-spark rdd

Spark Dataframes- Reducing By Key

Oct 09, 2021

scala apache-spark apache-spark-sql apache-spark-dataset

How to reference a dataframe when in an UDF on another dataframe?

Aug 26, 2022

apache-spark dataframe pyspark user-defined-functions broadcast

New posts in apache-spark