apache-spark-dataset tutorials

Can Spark read data directly into a nested case class?

Oct 22, 2022

scala apache-spark apache-spark-dataset

Should cache and checkpoint be used together on DataSets? If so, how does this work under the hood?

Sep 15, 2022

apache-spark apache-spark-sql apache-spark-dataset

How to drop malformed rows while reading csv with schema Spark?

Oct 19, 2022

scala csv apache-spark apache-spark-dataset

Convert scala list to DataFrame or DataSet

Nov 16, 2022

scala apache-spark apache-spark-sql apache-spark-dataset apache-spark-encoders

When to use Spark DataFrame/Dataset API and when to use plain RDD?

Oct 25, 2022

apache-spark apache-spark-sql spark-dataframe apache-spark-dataset

Spark 2.0 implicit encoder, deal with missing column when type is Option[Seq[String]] (scala)

Nov 14, 2019

scala apache-spark apache-spark-dataset

What is the difference between Spark DataSet and RDD

Oct 27, 2018

apache-spark rdd apache-spark-dataset

Spark 2 Dataset Null value exception

Sep 05, 2022

scala apache-spark apache-spark-sql apache-spark-dataset

Create Spark Dataset from a CSV file

Oct 20, 2022

apache-spark apache-spark-dataset

How to lower the case of column names of a data frame but not its values?

Aug 19, 2022

apache-spark apache-spark-sql apache-spark-dataset

How to convert the datasets of Spark Row into string?

Sep 19, 2022

java string apache-spark apache-spark-sql apache-spark-dataset

Why do columns change to nullable in Apache Spark SQL?

Oct 22, 2022

apache-spark apache-spark-sql apache-spark-dataset

How to read ".gz" compressed file using spark DF or DS?

Aug 31, 2022

apache-spark apache-spark-sql gzip apache-spark-dataset

Apache Spark 2.0: java.lang.UnsupportedOperationException: No Encoder found for java.time.LocalDate

Mar 14, 2022

scala apache-spark apache-spark-sql apache-spark-dataset apache-spark-encoders

Encode an ADT / sealed trait hierarchy into Spark DataSet column

Apr 18, 2022

scala apache-spark apache-spark-dataset apache-spark-encoders

Why is predicate pushdown not used in typed Dataset API (vs untyped DataFrame API)?

Oct 15, 2022

apache-spark dataframe apache-spark-sql apache-spark-dataset

Mapping Spark DataSet row values into new hash column

Mar 24, 2022

scala apache-spark spark-dataframe apache-spark-dataset

Partition data for efficient joining for Spark dataframe/dataset

Nov 04, 2022

apache-spark apache-spark-sql spark-dataframe partitioning apache-spark-dataset

Spark Dataset select with typedcolumn

Sep 06, 2022

scala apache-spark apache-spark-dataset

New posts in apache-spark-dataset