Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spark Datasets available in Python?

Here, it is stated:

..you can create Datasets within a Scala or Python..

while here, the following is stated:

Python does not have the support for the Dataset API

Are datasets available in python?

like image 1000
user3579222 Avatar asked Mar 31 '26 04:03

user3579222


1 Answers

Perhaps the question is about Typed Spark Datasets.

If so, then the answer is no.

Mentioned spark datasets are only available in Scala and Java.

In Python implementation of Spark (or PySpark) you have to choose between DataFrames as the preferred choice and RDD.

Reference: RDD vs. DataFrame vs. Dataset

Update 2022-09-26: Clarification regarding typed spark datasets

like image 51
Alexander Volok Avatar answered Apr 02 '26 08:04

Alexander Volok



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!