apache-spark-sql tutorials

Peak Execution Memory in Spark

May 18, 2022

apache-spark apache-spark-sql

Find median in spark SQL for multiple double datatype columns

Oct 15, 2022

apache-spark apache-spark-sql hive-udf

Apache spark case with multiple when clauses on different columns

Jun 02, 2022

apache-spark hadoop apache-spark-sql

How to load a csv directly into a Spark Dataset?

Oct 23, 2022

scala apache-spark apache-spark-sql

merge two dataset which are having different column names in Apache spark

Aug 23, 2022

java apache-spark apache-spark-sql spark-dataframe

Why does spark-shell fail with "The root scratch dir: /tmp/hive on HDFS should be writable."?

Jul 31, 2022

apache-spark windows-10 apache-spark-sql

Why does a query fail with "AnalysisException: Expected only partition pruning predicates"?

May 15, 2019

apache-spark apache-spark-sql

What type should it be , after using .toArray() for a Spark vector?

Sep 10, 2022

python numpy apache-spark pyspark apache-spark-sql

Self-join not working as expected with the DataFrame API

Sep 05, 2022

scala apache-spark dataframe apache-spark-sql

Apply a transformation to multiple columns pyspark dataframe

Jun 29, 2022

apache-spark pyspark apache-spark-sql pyspark-sql

Is it possible to ignore null values when using lead window function in Spark

Feb 14, 2022

scala apache-spark apache-spark-sql

Does the SparkSQL Dataframe function explode preserve order?

Feb 11, 2021

scala apache-spark apache-spark-sql

How to sort array of struct type in Spark DataFrame by particular column?

Mar 16, 2022

dataframe scala apache-spark apache-spark-sql

Partitioning of Data Frame in Pyspark using Custom Partitioner

Aug 24, 2022

pyspark apache-spark-sql

How to expire state of dropDuplicates in structured streaming to avoid OOM?

Nov 21, 2022

apache-spark duplicates apache-spark-sql out-of-memory spark-structured-streaming

Does Kryo help in SparkSQL?

Sep 16, 2022

apache-spark apache-spark-sql kryo

How to write a Dataset to Kafka topic?

Oct 25, 2022

scala apache-spark apache-kafka apache-spark-sql

how to use spark lag and lead over group by and order by

Nov 15, 2022

apache-spark apache-spark-sql apache-spark-dataset

Adding a new column in the first ordinal position in a pyspark dataframe

Mar 06, 2022

python apache-spark pyspark apache-spark-sql

Pyspark Error:- dataType <class 'pyspark.sql.types.StringType'> should be an instance of <class 'pyspark.sql.types.DataType'>

Nov 10, 2022

python apache-spark pyspark apache-spark-sql

New posts in apache-spark-sql