Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

How to transform Dask.DataFrame to pd.DataFrame?

Tags:

python

pandas

dask

How can I transform my resulting dask.DataFrame into pandas.DataFrame (let's say I am done with heavy lifting, and just want to apply sklearn to my aggregate result)?

like image 676
Philipp_Kats Avatar asked Aug 18 '16 00:08

Philipp_Kats

People also ask

Is Dask compatible with pandas?

Pandas does most of the things pretty well but screws in quite a few. Dask does not support these two things as well. There are numerous other things for which you'll have to use pandas.

Which is better Dask or pandas?

Dask runs faster than pandas for this query, even when the most inefficient column type is used, because it parallelizes the computations. pandas only uses 1 CPU core to run the query. My computer has 4 cores and Dask uses all the cores to run the computation.

1 Answers

You can call the .compute() method to transform a dask.dataframe to a pandas dataframe:

df = df.compute()
like image 162
MRocklin Avatar answered Sep 30 '22 19:09

MRocklin
Sign in to Comment

Related questions

List of IP addresses/hostnames from local network in Python
Image.show() won't display the picture
Download image with selenium python
Why there is no do while loop in python
Data-frame Object has no Attribute
How do I set environment variables in pipenv?
Using a Django custom model method property in order_by()
Is it safe to use the python word "type" in my code?
Log-log lmplot with seaborn
Driving Excel from Python in Windows
Sort nested dictionary by value, and remainder by another value, in Python
How to group elements in python by n elements [duplicate]
Set partitions in Python
GroupBy column and filter rows with maximum value in Pyspark
Checking dict keys to ensure a required key always exists, and that the dict has no other key names beyond a defined set of names
What is the difference between raise StopIteration and a return statement in generators?
What exactly is contained within a obj.__closure__?
Python finite difference functions?
vlookup in Pandas using join
Matplotlib giving error "OverflowError: In draw_path: Exceeded cell block limit"

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!