apache-spark-sql tutorials

Spark-Monotonically increasing id not working as expected in dataframe?

Oct 02, 2022

scala apache-spark apache-spark-sql

Limiting maximum size of dataframe partition

Apr 13, 2022

scala apache-spark apache-spark-sql

How to optimize partitioning when migrating data from JDBC source?

Apr 16, 2022

apache-spark jdbc hive apache-spark-sql partitioning

Apply MinMaxScaler on multiple columns in PySpark

Mar 18, 2022

python pyspark apache-spark-sql

Pandas Dataframe to RDD

Nov 04, 2022

pandas apache-spark dataframe pyspark apache-spark-sql

Why does using cache on streaming Datasets fail with "AnalysisException: Queries with streaming sources must be executed with writeStream.start()"?

Nov 04, 2018

scala apache-spark apache-spark-sql apache-spark-2.0 spark-structured-streaming

How to turn off scientific notation in pyspark?

Feb 03, 2020

apache-spark pyspark apache-spark-sql spark-dataframe

How to filter rows for a specific aggregate with spark sql?

Nov 02, 2022

sql apache-spark aggregate apache-spark-sql spark-dataframe

How to aggregate over rolling time window with groups in Spark

Mar 19, 2019

sql apache-spark pyspark apache-spark-sql window-functions

spark sbt error: value toDF is not a member of Seq[DataRow]

Jan 03, 2021

apache-spark apache-spark-sql

How to refresh a table and do it concurrently?

Sep 13, 2022

apache-spark apache-spark-sql spark-streaming

How to drop a column from a Databricks Delta table?

Sep 15, 2022

sql apache-spark apache-spark-sql databricks delta-lake

Spark Sql: TypeError("StructType can not accept object in type %s" % type(obj))

Feb 22, 2020

python apache-spark apache-spark-sql spark-dataframe

ValueError: Cannot convert column into bool

May 12, 2022

apache-spark pyspark apache-spark-sql pyspark-sql

Spark dataframe add new column with random data

Nov 13, 2022

python apache-spark pyspark apache-spark-sql

Filling gaps in timeseries Spark

Aug 02, 2021

scala apache-spark apache-spark-sql time-series

Using Spark UDFs with struct sequences

Oct 17, 2022

scala apache-spark apache-spark-sql

PySpark / Spark Window Function First/ Last Issue

Oct 26, 2022

sql apache-spark pyspark apache-spark-sql window-functions

How to convert a case-class-based RDD into a DataFrame?

Mar 28, 2022

scala apache-spark dataframe apache-spark-sql rdd

Creating a new Spark DataFrame with new column value based on column in first dataframe Java

Aug 23, 2022

java apache-spark dataframe apache-spark-sql

New posts in apache-spark-sql